| Name | romatch-roicat JSON |
| Version |
0.1.0
JSON |
| download |
| home_page | None |
| Summary | None |
| upload_time | 2024-10-17 22:23:57 |
| maintainer | None |
| docs_url | None |
| author | Johan Edstedt |
| requires_python | None |
| license | None |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
#
<p align="center">
<h1 align="center"> <ins>RoMa</ins> đïž:<br> Robust Dense Feature Matching <br> âCVPR 2024â</h1>
<p align="center">
<a href="https://scholar.google.com/citations?user=Ul-vMR0AAAAJ">Johan Edstedt</a>
·
<a href="https://scholar.google.com/citations?user=HS2WuHkAAAAJ">Qiyu Sun</a>
·
<a href="https://scholar.google.com/citations?user=FUE3Wd0AAAAJ">Georg Bökman</a>
·
<a href="https://scholar.google.com/citations?user=6WRQpCQAAAAJ">MÄrten WadenbÀck</a>
·
<a href="https://scholar.google.com/citations?user=lkWfR08AAAAJ">Michael Felsberg</a>
</p>
<h2 align="center"><p>
<a href="https://arxiv.org/abs/2305.15404" align="center">Paper</a> |
<a href="https://parskatt.github.io/RoMa" align="center">Project Page</a>
</p></h2>
<div align="center"></div>
</p>
<br/>
<p align="center">
<img src="https://github.com/Parskatt/RoMa/assets/22053118/15d8fea7-aa6d-479f-8a93-350d950d006b" alt="example" width=80%>
<br>
<em>RoMa is the robust dense feature matcher capable of estimating pixel-dense warps and reliable certainties for almost any image pair.</em>
</p>
## Setup/Install
In your python environment (tested on Linux python 3.10), run:
```bash
pip install -e .
```
## Demo / How to Use
We provide two demos in the [demos folder](demo).
Here's the gist of it:
```python
from romatch import roma_outdoor
roma_model = roma_outdoor(device=device)
# Match
warp, certainty = roma_model.match(imA_path, imB_path, device=device)
# Sample matches for estimation
matches, certainty = roma_model.sample(warp, certainty)
# Convert to pixel coordinates (RoMa produces matches in [-1,1]x[-1,1])
kptsA, kptsB = roma_model.to_pixel_coordinates(matches, H_A, W_A, H_B, W_B)
# Find a fundamental matrix (or anything else of interest)
F, mask = cv2.findFundamentalMat(
kptsA.cpu().numpy(), kptsB.cpu().numpy(), ransacReprojThreshold=0.2, method=cv2.USAC_MAGSAC, confidence=0.999999, maxIters=10000
)
```
**New**: You can also match arbitrary keypoints with RoMa. See [match_keypoints](romatch/models/matcher.py) in RegressionMatcher.
## Settings
### Resolution
By default RoMa uses an initial resolution of (560,560) which is then upsampled to (864,864).
You can change this at construction (see roma_outdoor kwargs).
You can also change this later, by changing the roma_model.w_resized, roma_model.h_resized, and roma_model.upsample_res.
### Sampling
roma_model.sample_thresh controls the thresholding used when sampling matches for estimation. In certain cases a lower or higher threshold may improve results.
## Reproducing Results
The experiments in the paper are provided in the [experiments folder](experiments).
### Training
1. First follow the instructions provided here: https://github.com/Parskatt/DKM for downloading and preprocessing datasets.
2. Run the relevant experiment, e.g.,
```bash
torchrun --nproc_per_node=4 --nnodes=1 --rdzv_backend=c10d experiments/roma_outdoor.py
```
### Testing
```bash
python experiments/roma_outdoor.py --only_test --benchmark mega-1500
```
## License
All our code except DINOv2 is MIT license.
DINOv2 has an Apache 2 license [DINOv2](https://github.com/facebookresearch/dinov2/blob/main/LICENSE).
## Acknowledgement
Our codebase builds on the code in [DKM](https://github.com/Parskatt/DKM).
## Tiny RoMa
If you find that RoMa is too heavy, you might want to try Tiny RoMa which is built on top of XFeat.
```python
from romatch import tiny_roma_v1_outdoor
tiny_roma_model = tiny_roma_v1_outdoor(device=device)
```
Mega1500:
| | AUC@5 | AUC@10 | AUC@20 |
|----------|----------|----------|----------|
| XFeat | 46.4 | 58.9 | 69.2 |
| XFeat* | 51.9 | 67.2 | 78.9 |
| Tiny RoMa v1 | 56.4 | 69.5 | 79.5 |
| RoMa | - | - | - |
Mega-8-Scenes (See DKM):
| | AUC@5 | AUC@10 | AUC@20 |
|----------|----------|----------|----------|
| XFeat | - | - | - |
| XFeat* | 50.1 | 64.4 | 75.2 |
| Tiny RoMa v1 | 57.7 | 70.5 | 79.6 |
| RoMa | - | - | - |
IMC22 :'):
| | mAA@10 |
|----------|----------|
| XFeat | 42.1 |
| XFeat* | - |
| Tiny RoMa v1 | 42.2 |
| RoMa | - |
## BibTeX
If you find our models useful, please consider citing our paper!
```
@article{edstedt2024roma,
title={{RoMa: Robust Dense Feature Matching}},
author={Edstedt, Johan and Sun, Qiyu and Bökman, Georg and WadenbÀck, MÄrten and Felsberg, Michael},
journal={IEEE Conference on Computer Vision and Pattern Recognition},
year={2024}
}
```
Raw data
{
"_id": null,
"home_page": null,
"name": "romatch-roicat",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "Johan Edstedt",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/3e/f5/8e6aec515e10087f388f7d44da0b6097fc5b5078fbdb76e808b1f0f015fa/romatch_roicat-0.1.0.tar.gz",
"platform": null,
"description": "# \n<p align=\"center\">\n <h1 align=\"center\"> <ins>RoMa</ins> \ud83c\udfdb\ufe0f:<br> Robust Dense Feature Matching <br> \u2b50CVPR 2024\u2b50</h1>\n <p align=\"center\">\n <a href=\"https://scholar.google.com/citations?user=Ul-vMR0AAAAJ\">Johan Edstedt</a>\n \u00b7\n <a href=\"https://scholar.google.com/citations?user=HS2WuHkAAAAJ\">Qiyu Sun</a>\n \u00b7\n <a href=\"https://scholar.google.com/citations?user=FUE3Wd0AAAAJ\">Georg B\u00f6kman</a>\n \u00b7\n <a href=\"https://scholar.google.com/citations?user=6WRQpCQAAAAJ\">M\u00e5rten Wadenb\u00e4ck</a>\n \u00b7\n <a href=\"https://scholar.google.com/citations?user=lkWfR08AAAAJ\">Michael Felsberg</a>\n </p>\n <h2 align=\"center\"><p>\n <a href=\"https://arxiv.org/abs/2305.15404\" align=\"center\">Paper</a> | \n <a href=\"https://parskatt.github.io/RoMa\" align=\"center\">Project Page</a>\n </p></h2>\n <div align=\"center\"></div>\n</p>\n<br/>\n<p align=\"center\">\n <img src=\"https://github.com/Parskatt/RoMa/assets/22053118/15d8fea7-aa6d-479f-8a93-350d950d006b\" alt=\"example\" width=80%>\n <br>\n <em>RoMa is the robust dense feature matcher capable of estimating pixel-dense warps and reliable certainties for almost any image pair.</em>\n</p>\n\n## Setup/Install\nIn your python environment (tested on Linux python 3.10), run:\n```bash\npip install -e .\n```\n## Demo / How to Use\nWe provide two demos in the [demos folder](demo).\nHere's the gist of it:\n```python\nfrom romatch import roma_outdoor\nroma_model = roma_outdoor(device=device)\n# Match\nwarp, certainty = roma_model.match(imA_path, imB_path, device=device)\n# Sample matches for estimation\nmatches, certainty = roma_model.sample(warp, certainty)\n# Convert to pixel coordinates (RoMa produces matches in [-1,1]x[-1,1])\nkptsA, kptsB = roma_model.to_pixel_coordinates(matches, H_A, W_A, H_B, W_B)\n# Find a fundamental matrix (or anything else of interest)\nF, mask = cv2.findFundamentalMat(\n kptsA.cpu().numpy(), kptsB.cpu().numpy(), ransacReprojThreshold=0.2, method=cv2.USAC_MAGSAC, confidence=0.999999, maxIters=10000\n)\n```\n\n**New**: You can also match arbitrary keypoints with RoMa. See [match_keypoints](romatch/models/matcher.py) in RegressionMatcher.\n\n## Settings\n\n### Resolution\nBy default RoMa uses an initial resolution of (560,560) which is then upsampled to (864,864). \nYou can change this at construction (see roma_outdoor kwargs).\nYou can also change this later, by changing the roma_model.w_resized, roma_model.h_resized, and roma_model.upsample_res.\n\n### Sampling\nroma_model.sample_thresh controls the thresholding used when sampling matches for estimation. In certain cases a lower or higher threshold may improve results.\n\n\n## Reproducing Results\nThe experiments in the paper are provided in the [experiments folder](experiments).\n\n### Training\n1. First follow the instructions provided here: https://github.com/Parskatt/DKM for downloading and preprocessing datasets.\n2. Run the relevant experiment, e.g.,\n```bash\ntorchrun --nproc_per_node=4 --nnodes=1 --rdzv_backend=c10d experiments/roma_outdoor.py\n```\n### Testing\n```bash\npython experiments/roma_outdoor.py --only_test --benchmark mega-1500\n```\n## License\nAll our code except DINOv2 is MIT license.\nDINOv2 has an Apache 2 license [DINOv2](https://github.com/facebookresearch/dinov2/blob/main/LICENSE).\n\n## Acknowledgement\nOur codebase builds on the code in [DKM](https://github.com/Parskatt/DKM).\n\n## Tiny RoMa\nIf you find that RoMa is too heavy, you might want to try Tiny RoMa which is built on top of XFeat.\n```python\nfrom romatch import tiny_roma_v1_outdoor\ntiny_roma_model = tiny_roma_v1_outdoor(device=device)\n```\nMega1500:\n| | AUC@5 | AUC@10 | AUC@20 |\n|----------|----------|----------|----------|\n| XFeat | 46.4 | 58.9 | 69.2 |\n| XFeat* | 51.9 | 67.2 | 78.9 |\n| Tiny RoMa v1 | 56.4 | 69.5 | 79.5 |\n| RoMa | - | - | - |\n\nMega-8-Scenes (See DKM):\n| | AUC@5 | AUC@10 | AUC@20 |\n|----------|----------|----------|----------|\n| XFeat | - | - | - |\n| XFeat* | 50.1 | 64.4 | 75.2 |\n| Tiny RoMa v1 | 57.7 | 70.5 | 79.6 |\n| RoMa | - | - | - |\n\nIMC22 :'):\n| | mAA@10 |\n|----------|----------|\n| XFeat | 42.1 |\n| XFeat* | - |\n| Tiny RoMa v1 | 42.2 |\n| RoMa | - |\n\n## BibTeX\nIf you find our models useful, please consider citing our paper!\n```\n@article{edstedt2024roma,\ntitle={{RoMa: Robust Dense Feature Matching}},\nauthor={Edstedt, Johan and Sun, Qiyu and B\u00f6kman, Georg and Wadenb\u00e4ck, M\u00e5rten and Felsberg, Michael},\njournal={IEEE Conference on Computer Vision and Pattern Recognition},\nyear={2024}\n}\n```\n",
"bugtrack_url": null,
"license": null,
"summary": null,
"version": "0.1.0",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "88989985c38615b25534c93590ece73a1998552fe8f7f36b7d9a85d0d20c8fd8",
"md5": "8f2f90ec1aa9cef8aa6c685ed3bb1f6e",
"sha256": "19547d90bf0d14807a5d16ed0edbbb322d1725f999a9e0ffad6fac03952822b2"
},
"downloads": -1,
"filename": "romatch_roicat-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8f2f90ec1aa9cef8aa6c685ed3bb1f6e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 64587,
"upload_time": "2024-10-17T22:23:56",
"upload_time_iso_8601": "2024-10-17T22:23:56.158426Z",
"url": "https://files.pythonhosted.org/packages/88/98/9985c38615b25534c93590ece73a1998552fe8f7f36b7d9a85d0d20c8fd8/romatch_roicat-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3ef58e6aec515e10087f388f7d44da0b6097fc5b5078fbdb76e808b1f0f015fa",
"md5": "2725cf5f3b12b2c1b1c3e87d9997925c",
"sha256": "73da00f77cbef89846e1feaedbe766ddb0902df48d9522ea2435e97ddae52dd6"
},
"downloads": -1,
"filename": "romatch_roicat-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "2725cf5f3b12b2c1b1c3e87d9997925c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 49459,
"upload_time": "2024-10-17T22:23:57",
"upload_time_iso_8601": "2024-10-17T22:23:57.788471Z",
"url": "https://files.pythonhosted.org/packages/3e/f5/8e6aec515e10087f388f7d44da0b6097fc5b5078fbdb76e808b1f0f015fa/romatch_roicat-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-17 22:23:57",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "romatch-roicat"
}
