| Name | puzzle-sim JSON |
| Version |
1.0.1
JSON |
| download |
| home_page | None |
| Summary | Implementation of PuzzleSim, a cross-refence image similarity metric designed for artifact detection in novel view synthesis methods. |
| upload_time | 2025-10-07 03:06:51 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.8 |
| license | MIT License
Copyright (c) 2025 Nicolai Hermann, Jorge Condor, Piotr Didyk
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
|
| keywords |
puzzle similarity
similarity
no reference
cross reference
image metric
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
torch
torchvision
matplotlib
pillow
torchmetrics
numpy
huggingface_hub
transformers
accelerate
termcolor
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# <img src="https://www.svgrepo.com/show/510149/puzzle-piece.svg" width="22"/> Puzzle Similarity
<p align="left">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.7.2/css/all.min.css">
<img src="https://raw.githubusercontent.com/FortAwesome/Font-Awesome/6.x/svgs/solid/cube.svg" width=13 height=13> <a target="_blank" href="https://nihermann.github.io/puzzlesim/index.html">Project Page</a>
<img src="https://raw.githubusercontent.com/FortAwesome/Font-Awesome/6.x/svgs/solid/file-pdf.svg" width=13 height=13> <a target="_blank" href="https://arxiv.org/abs/2411.17489">Paper</a>
<img src="https://raw.githubusercontent.com/FortAwesome/Font-Awesome/6.x/svgs/solid/database.svg" width=13 height=13> <a target="_blank" href="https://huggingface.co/datasets/nihermann/annotated-3DGS-artifacts">Data</a>
<img src="https://raw.githubusercontent.com/FortAwesome/Font-Awesome/6.x/svgs/solid/file-circle-plus.svg" width=13 height=13> <a target="_blank" href="https://nihermann.github.io/puzzlesim/data/Puzzle_Similarity_Supplemental.pdf">Supplemental</a>
</p>
by Nicolai Hermann, Jorge Condor, and Piotr Didyk
-----
<p align="left">
<a target="_blank" href="https://arxiv.org/abs/2411.17489"><img src=https://img.shields.io/badge/arXiv-2411.17489-b31b1b.svg></a>
<img src="https://github.com/nihermann/PuzzleSim/actions/workflows/tests.yml/badge.svg" alt="test results">
<a target="_blank" href="https://pypi.org/project/puzzle_sim/"><img src="https://img.shields.io/pypi/v/puzzle_sim" alt="PyPI version"></a>
<a target="_blank" href="https://pypi.org/project/puzzle_sim/"><img src="https://img.shields.io/pypi/pyversions/puzzle_sim" alt="PyPI version"></a>
<img src="https://img.shields.io/badge/license-MIT-green" alt="License">
</p>
This repository contains the implementation of the cross-reference metric PuzzleSim and a dedicated demo for the paper "Puzzle Similarity: A Perceptually-Guided Cross-Reference Metric for Artifact Detection in 3D Scene Reconstructions".
### News
- (04-09-2025) Dinov3 🦖 backbones (ConvNeXt & Vits) are now supported
- (31-08-2025) Major refactoring allowing to easily add custom backbones and automated cross-platform testing
- (25-06-2025) PuzzleSim was officially accepted to ICCV 2025 in Hawaii 🌸!
- (29-11-2024) Official code release
### Requirements
If you simply want to use the metric use:
```shell
pip install puzzle_sim
```
If you want to extend it please install it locally as a package. The package requires Python 3.8 or higher. If you wish to use dinov3 backbones you must have Python 3.10 or higher and `transformers>=4.56`:
```shell
pip install -e .
```
### Usage
You can use the metric in your own code as follows:
```python
from puzzle_sim import PuzzleSim
priors = ... # load priors from file with shape (N, C, H, W) in [0, 1]
test_image = ... # load test image (C, H, W) or (1, C, H, W) in [0, 1]
puzzle = PuzzleSim(reference=priors, net_type='squeeze')
similarity_map = puzzle(test_image) # (H, W) similarity map in [0, 1]
```
To use dinov3 backbones you must be logged in to HuggingFace (`hf auth login`), requested access to the models on HuggingFace and review the necessary requirements above. You can request access to the models [here](https://huggingface.co/collections/facebook/dinov3-68924841bd6b561778e31009).
In code, you have to adapt the `puzzle()` call as default arguments assume the configuration from the paper. We have not tested optimal weights for dinov3 backbones yet, so we recommend to use a simple average over all layers (which has shown similar performance to the configurations in the paper):
```python
from puzzle_sim import PuzzleSim
priors = ... # load priors from file with shape (N, C, H, W) in [0, 1]
test_image = ... # load test image (C, H, W) or (1, C, H, W) in [0, 1]
puzzle = PuzzleSim(reference=priors, net_type='convnext_tiny')
similarity_map = puzzle(test_image, layers=range(5), weights=None, reduction='mean') # (H, W) similarity map in [0, 1]
```
> If your GPU runs out of memory, try reducing the `stride` parameter in the forward call, this will reduce memory consumption. On the other hand, with small image dimensions the naive implementation might be faster although requiring much more memory (set `mem_save=False`).
### Demo
Please find the demo in `demo.ipynb` to see how to run the metric on some example sets. In order to run the demo, you need to pull the data from another repository. Do this by either cloning the repository using
```shell
git clone https://github.com/nihermann/PuzzleSim.git --recursive
```
or if you already cloned the repository without the data submodule, you can download the submodule using
```shell
git submodule update --init --recursive
```
### Add Your Own Backbones
You can extend PuzzleSim with your own backbone models. To get started, inherit from `adapters.FeatureExtractor` and implement the `compute_features` method.
There are two ways to use your backbone:
1. Directly in the constructor:
```python
PuzzleSim(..., net_type=YourBackbone())
```
2. Via the factory function: Register your backbone in `adapters.get_feature_extractor` and add the corresponding string to `adapters.net_type`, so you can refer to it by that string:
```python
PuzzleSim(..., 'your_backbone')
```
#### 💡 Contributing
If you’d like to share your backbone with the community, feel free to open a pull request. Please make sure that:
- [ ] Your backbone is publicly available (e.g., on HuggingFace or PyTorch Hub)
- [ ] you’ve registered it in the factory function `adapters.get_feature_extractor`,
- [ ] extended `adapters.net_type` (so the tests pick it up automatically),
- [ ] and all tests pass (run `pytest` in the project root).
For development, we recommend installing the package in editable mode with dev requirements:
```shell
pip install -e .[dev]
```
### Citation
If you find this work useful, please consider citing:
```bibtex
@inproceedings{hermann2025puzzlesim,
title={Puzzle Similarity: A Perceptually-Guided Cross-Reference Metric for Artifact Detection in 3D Scene Reconstructions},
author={Nicolai Hermann and Jorge Condor and Piotr Didyk},
booktitle={ICCV},
year={2025},
}
```
Raw data
{
"_id": null,
"home_page": null,
"name": "puzzle-sim",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "Nicolai Hermann <hermann.nicolai@gmail.com>",
"keywords": "puzzle similarity, similarity, no reference, cross reference, image metric",
"author": null,
"author_email": "Nicolai Hermann <hermann.nicolai@gmail.com>, Jorge Condor <jorge.condor@usi.ch>, Piotr Didyk <piotr.didyk@usi.ch>",
"download_url": "https://files.pythonhosted.org/packages/47/7d/a6e6ee20fc32dcd7d58b2a6defad3983d8a014d407005326a65272a0b039/puzzle_sim-1.0.1.tar.gz",
"platform": null,
"description": "# <img src=\"https://www.svgrepo.com/show/510149/puzzle-piece.svg\" width=\"22\"/> Puzzle Similarity\n\n<p align=\"left\">\n <link rel=\"stylesheet\" href=\"https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.7.2/css/all.min.css\">\n <img src=\"https://raw.githubusercontent.com/FortAwesome/Font-Awesome/6.x/svgs/solid/cube.svg\" width=13 height=13> <a target=\"_blank\" href=\"https://nihermann.github.io/puzzlesim/index.html\">Project Page</a>\n <img src=\"https://raw.githubusercontent.com/FortAwesome/Font-Awesome/6.x/svgs/solid/file-pdf.svg\" width=13 height=13> <a target=\"_blank\" href=\"https://arxiv.org/abs/2411.17489\">Paper</a>\n <img src=\"https://raw.githubusercontent.com/FortAwesome/Font-Awesome/6.x/svgs/solid/database.svg\" width=13 height=13> <a target=\"_blank\" href=\"https://huggingface.co/datasets/nihermann/annotated-3DGS-artifacts\">Data</a>\n <img src=\"https://raw.githubusercontent.com/FortAwesome/Font-Awesome/6.x/svgs/solid/file-circle-plus.svg\" width=13 height=13> <a target=\"_blank\" href=\"https://nihermann.github.io/puzzlesim/data/Puzzle_Similarity_Supplemental.pdf\">Supplemental</a>\n</p>\n\nby Nicolai Hermann, Jorge Condor, and Piotr Didyk \n\n-----\n\n\n<p align=\"left\">\n <a target=\"_blank\" href=\"https://arxiv.org/abs/2411.17489\"><img src=https://img.shields.io/badge/arXiv-2411.17489-b31b1b.svg></a>\n <img src=\"https://github.com/nihermann/PuzzleSim/actions/workflows/tests.yml/badge.svg\" alt=\"test results\">\n <a target=\"_blank\" href=\"https://pypi.org/project/puzzle_sim/\"><img src=\"https://img.shields.io/pypi/v/puzzle_sim\" alt=\"PyPI version\"></a>\n <a target=\"_blank\" href=\"https://pypi.org/project/puzzle_sim/\"><img src=\"https://img.shields.io/pypi/pyversions/puzzle_sim\" alt=\"PyPI version\"></a>\n <img src=\"https://img.shields.io/badge/license-MIT-green\" alt=\"License\">\n</p>\n\nThis repository contains the implementation of the cross-reference metric PuzzleSim and a dedicated demo for the paper \"Puzzle Similarity: A Perceptually-Guided Cross-Reference Metric for Artifact Detection in 3D Scene Reconstructions\".\n\n### News\n- (04-09-2025) Dinov3 \ud83e\udd96 backbones (ConvNeXt & Vits) are now supported\n- (31-08-2025) Major refactoring allowing to easily add custom backbones and automated cross-platform testing\n- (25-06-2025) PuzzleSim was officially accepted to ICCV 2025 in Hawaii \ud83c\udf38!\n- (29-11-2024) Official code release\n\n\n### Requirements\nIf you simply want to use the metric use:\n```shell\npip install puzzle_sim\n```\n\nIf you want to extend it please install it locally as a package. The package requires Python 3.8 or higher. If you wish to use dinov3 backbones you must have Python 3.10 or higher and `transformers>=4.56`:\n```shell\npip install -e .\n```\n\n\n### Usage\nYou can use the metric in your own code as follows:\n```python\nfrom puzzle_sim import PuzzleSim\n\npriors = ... # load priors from file with shape (N, C, H, W) in [0, 1]\ntest_image = ... # load test image (C, H, W) or (1, C, H, W) in [0, 1]\npuzzle = PuzzleSim(reference=priors, net_type='squeeze')\n\nsimilarity_map = puzzle(test_image) # (H, W) similarity map in [0, 1]\n```\nTo use dinov3 backbones you must be logged in to HuggingFace (`hf auth login`), requested access to the models on HuggingFace and review the necessary requirements above. You can request access to the models [here](https://huggingface.co/collections/facebook/dinov3-68924841bd6b561778e31009).\nIn code, you have to adapt the `puzzle()` call as default arguments assume the configuration from the paper. We have not tested optimal weights for dinov3 backbones yet, so we recommend to use a simple average over all layers (which has shown similar performance to the configurations in the paper):\n```python\nfrom puzzle_sim import PuzzleSim\n\npriors = ... # load priors from file with shape (N, C, H, W) in [0, 1]\ntest_image = ... # load test image (C, H, W) or (1, C, H, W) in [0, 1]\npuzzle = PuzzleSim(reference=priors, net_type='convnext_tiny')\n\nsimilarity_map = puzzle(test_image, layers=range(5), weights=None, reduction='mean') # (H, W) similarity map in [0, 1]\n```\n> If your GPU runs out of memory, try reducing the `stride` parameter in the forward call, this will reduce memory consumption. On the other hand, with small image dimensions the naive implementation might be faster although requiring much more memory (set `mem_save=False`).\n\n### Demo\nPlease find the demo in `demo.ipynb` to see how to run the metric on some example sets. In order to run the demo, you need to pull the data from another repository. Do this by either cloning the repository using\n```shell\ngit clone https://github.com/nihermann/PuzzleSim.git --recursive\n```\nor if you already cloned the repository without the data submodule, you can download the submodule using\n```shell\ngit submodule update --init --recursive\n```\n\n### Add Your Own Backbones\nYou can extend PuzzleSim with your own backbone models. To get started, inherit from `adapters.FeatureExtractor` and implement the `compute_features` method.\n\nThere are two ways to use your backbone:\n1. Directly in the constructor: \n```python\nPuzzleSim(..., net_type=YourBackbone())\n```\n2. Via the factory function: Register your backbone in `adapters.get_feature_extractor` and add the corresponding string to `adapters.net_type`, so you can refer to it by that string: \n```python\nPuzzleSim(..., 'your_backbone')\n```\n\n#### \ud83d\udca1 Contributing\nIf you\u2019d like to share your backbone with the community, feel free to open a pull request. Please make sure that:\n- [ ] Your backbone is publicly available (e.g., on HuggingFace or PyTorch Hub)\n- [ ] you\u2019ve registered it in the factory function `adapters.get_feature_extractor`,\n- [ ] extended `adapters.net_type` (so the tests pick it up automatically),\n- [ ] and all tests pass (run `pytest` in the project root).\n\nFor development, we recommend installing the package in editable mode with dev requirements:\n```shell\npip install -e .[dev]\n```\n\n### Citation\nIf you find this work useful, please consider citing:\n```bibtex\n@inproceedings{hermann2025puzzlesim,\n title={Puzzle Similarity: A Perceptually-Guided Cross-Reference Metric for Artifact Detection in 3D Scene Reconstructions},\n author={Nicolai Hermann and Jorge Condor and Piotr Didyk},\n booktitle={ICCV},\n year={2025},\n}\n```\n",
"bugtrack_url": null,
"license": "MIT License\n \n Copyright (c) 2025 Nicolai Hermann, Jorge Condor, Piotr Didyk\n \n Permission is hereby granted, free of charge, to any person obtaining a copy\n of this software and associated documentation files (the \"Software\"), to deal\n in the Software without restriction, including without limitation the rights\n to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n copies of the Software, and to permit persons to whom the Software is\n furnished to do so, subject to the following conditions:\n \n The above copyright notice and this permission notice shall be included in all\n copies or substantial portions of the Software.\n \n THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n SOFTWARE.\n ",
"summary": "Implementation of PuzzleSim, a cross-refence image similarity metric designed for artifact detection in novel view synthesis methods.",
"version": "1.0.1",
"project_urls": {
"Homepage": "https://nihermann.github.io/puzzlesim/index.html",
"Issues": "https://github.com/nihermann/PuzzleSim/issues",
"Repository": "https://github.com/nihermann/PuzzleSim.git"
},
"split_keywords": [
"puzzle similarity",
" similarity",
" no reference",
" cross reference",
" image metric"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "7579390f00988cdbc9e927928fff390f7b3185a0ab35206ccb42d1e9691b8ab6",
"md5": "9f8006c65b575de298509dcb3b391c0b",
"sha256": "726aaecc8f61e28be9a548482231e61ab81d679eb82a13584d3f291d94cde7df"
},
"downloads": -1,
"filename": "puzzle_sim-1.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9f8006c65b575de298509dcb3b391c0b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 15555,
"upload_time": "2025-10-07T03:06:49",
"upload_time_iso_8601": "2025-10-07T03:06:49.480468Z",
"url": "https://files.pythonhosted.org/packages/75/79/390f00988cdbc9e927928fff390f7b3185a0ab35206ccb42d1e9691b8ab6/puzzle_sim-1.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "477da6e6ee20fc32dcd7d58b2a6defad3983d8a014d407005326a65272a0b039",
"md5": "38ff391ad0f93343ed03b30a4bafb9f5",
"sha256": "6198c74886bb876a3eb8ce66802466387404bc64e9818cc2e0d3034294825318"
},
"downloads": -1,
"filename": "puzzle_sim-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "38ff391ad0f93343ed03b30a4bafb9f5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 777606,
"upload_time": "2025-10-07T03:06:51",
"upload_time_iso_8601": "2025-10-07T03:06:51.058114Z",
"url": "https://files.pythonhosted.org/packages/47/7d/a6e6ee20fc32dcd7d58b2a6defad3983d8a014d407005326a65272a0b039/puzzle_sim-1.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-07 03:06:51",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "nihermann",
"github_project": "PuzzleSim",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "torch",
"specs": []
},
{
"name": "torchvision",
"specs": []
},
{
"name": "matplotlib",
"specs": []
},
{
"name": "pillow",
"specs": []
},
{
"name": "torchmetrics",
"specs": []
},
{
"name": "numpy",
"specs": []
},
{
"name": "huggingface_hub",
"specs": []
},
{
"name": "transformers",
"specs": []
},
{
"name": "accelerate",
"specs": []
},
{
"name": "termcolor",
"specs": []
}
],
"lcname": "puzzle-sim"
}
