| Name | robust-segment-anything JSON |
| Version |
1.0.0
JSON |
| download |
| home_page | None |
| Summary | RobustSAM: Segment Anything Robustly on Degraded Images. |
| upload_time | 2024-06-18 10:35:14 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.10 |
| license | MIT License Copyright (c) 2024 robustsam Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
deep-learning
pytorch
vision
object-segmentation
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# RobustSAM: Segment Anything Robustly on Degraded Images (CVPR 2024 Highlight)
Official repository for RobustSAM: Segment Anything Robustly on Degraded Images
[Project Page](https://robustsam.github.io/) | [Paper](https://openaccess.thecvf.com/content/CVPR2024/html/Chen_RobustSAM_Segment_Anything_Robustly_on_Degraded_Images_CVPR_2024_paper.html) | [Video](https://www.youtube.com/watch?v=Awukqkbs6zM) | [Dataset](https://robustsam.github.io/)
## Updates
- June 2024: ✨ Inference code of RobustSAM was released!
- Feb 2024: ✨ RobustSAM was accepted into CVPR 2024!
## Introduction
Segment Anything Model (SAM) has emerged as a transformative approach in image segmentation, acclaimed for its robust zero-shot segmentation capabilities and flexible prompting system. Nonetheless, its performance is challenged by images with degraded quality. Addressing this limitation, we propose the Robust Segment Anything Model (RobustSAM), which enhances SAM's performance on low-quality images while preserving its promptability and zero-shot generalization.
Our method leverages the pre-trained SAM model with only marginal parameter increments and computational requirements. The additional parameters of RobustSAM can be optimized within 30 hours on eight GPUs, demonstrating its feasibility and practicality for typical research laboratories. We also introduce the Robust-Seg dataset, a collection of 688K image-mask pairs with different degradations designed to train and evaluate our model optimally. Extensive experiments across various segmentation tasks and datasets confirm RobustSAM's superior performance, especially under zero-shot conditions, underscoring its potential for extensive real-world application. Additionally, our method has been shown to effectively improve the performance of SAM-based downstream tasks such as single image dehazing and deblurring.
<img width="1096" alt="image" src='figures/architecture.jpg'>
## Setup
1) Create a conda environment and activate it.
```
conda create --name robustsam python=3.10 -y
conda activate robustsam
```
2) Clone and enter into repo directory.
```
git clone https://github.com/robustsam/RobustSAM
cd RobustSAM
```
3) Use command below to check your CUDA version.
```
nvidia-smi
```
4) Replace the CUDA version with yours in command below.
```
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu[$YOUR_CUDA_VERSION]
# For example: pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117 # cu117 = CUDA_version_11.7
```
5) Install remaining dependencies
```
pip install -r requirements.txt
```
6) Download pretrained [RobustSAM checkpoint](https://drive.google.com/file/d/197EEnWYvchupfJrK44-ki3wDhv3-UuGv/view?usp=sharing) and place into current directory.
## Demo
We have prepared some images im **demo_images** folder for demo purpose. Besides, two prompting modes are available (box prompts and point prompts).
- For box prompt:
```
python eval.py --bbox
```
- For point prompt:
```
python eval.py
```
In default, demo results will be saved to **demo_result/[$PROMPT_TYPE]**.
## Comparison of computational requirements
<img width="720" alt="image" src='figures/Computational requirements.PNG'>
## Visual Comparison
<table>
<tr>
<td>
<img src="figures/gif_output/blur_back_n_forth.gif" width="390">
</td>
<td>
<img src="figures/gif_output/haze_back_n_forth.gif" width="390">
</td>
</tr>
<tr>
<td>
<img src="figures/gif_output/lowlight_back_n_forth.gif" width="390">
</td>
<td>
<img src="figures/gif_output/rain_back_n_forth.gif" width="390">
</td>
</tr>
</table>
<img width="1096" alt="image" src='figures/qualitative_result.PNG'>
## Quantitative Comparison
### Seen dataset with synthetic degradation
<img width="720" alt="image" src='figures/seen_dataset_with_synthetic_degradation.PNG'>
### Unseen dataset with synthetic degradation
<img width="720" alt="image" src='figures/unseen_dataset_with_synthetic_degradation.PNG'>
### Unseen dataset with real degradation
<img width="600" alt="image" src='figures/unseen_dataset_with_real_degradation.PNG'>
## Reference
If you find this work useful, please consider citing us!
```python
@inproceedings{chen2024robustsam,
title={RobustSAM: Segment Anything Robustly on Degraded Images},
author={Chen, Wei-Ting and Vong, Yu-Jiet and Kuo, Sy-Yen and Ma, Sizhou and Wang, Jian},
journal={CVPR},
year={2024}
}
```
## Acknowledgements
We thank the authors of [SAM](https://github.com/facebookresearch/segment-anything) from which our repo is based off of.
Raw data
{
"_id": null,
"home_page": null,
"name": "robust-segment-anything",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "deep-learning, pytorch, vision, object-segmentation",
"author": null,
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/18/55/60b6d2f424fee8a80af41031a76c9b7a994a072a9e217b44bf322e1f35d3/robust_segment_anything-1.0.0.tar.gz",
"platform": null,
"description": "# RobustSAM: Segment Anything Robustly on Degraded Images (CVPR 2024 Highlight)\n\nOfficial repository for RobustSAM: Segment Anything Robustly on Degraded Images\n\n\n\n[Project Page](https://robustsam.github.io/) | [Paper](https://openaccess.thecvf.com/content/CVPR2024/html/Chen_RobustSAM_Segment_Anything_Robustly_on_Degraded_Images_CVPR_2024_paper.html) | [Video](https://www.youtube.com/watch?v=Awukqkbs6zM) | [Dataset](https://robustsam.github.io/)\n\n\n## Updates\n- June 2024: \u2728 Inference code of RobustSAM was released!\n- Feb 2024: \u2728 RobustSAM was accepted into CVPR 2024!\n\n\n## Introduction\nSegment Anything Model (SAM) has emerged as a transformative approach in image segmentation, acclaimed for its robust zero-shot segmentation capabilities and flexible prompting system. Nonetheless, its performance is challenged by images with degraded quality. Addressing this limitation, we propose the Robust Segment Anything Model (RobustSAM), which enhances SAM's performance on low-quality images while preserving its promptability and zero-shot generalization.\n\nOur method leverages the pre-trained SAM model with only marginal parameter increments and computational requirements. The additional parameters of RobustSAM can be optimized within 30 hours on eight GPUs, demonstrating its feasibility and practicality for typical research laboratories. We also introduce the Robust-Seg dataset, a collection of 688K image-mask pairs with different degradations designed to train and evaluate our model optimally. Extensive experiments across various segmentation tasks and datasets confirm RobustSAM's superior performance, especially under zero-shot conditions, underscoring its potential for extensive real-world application. Additionally, our method has been shown to effectively improve the performance of SAM-based downstream tasks such as single image dehazing and deblurring.\n\n<img width=\"1096\" alt=\"image\" src='figures/architecture.jpg'>\n\n## Setup\n1) Create a conda environment and activate it.\n```\nconda create --name robustsam python=3.10 -y\nconda activate robustsam\n```\n2) Clone and enter into repo directory.\n```\ngit clone https://github.com/robustsam/RobustSAM\ncd RobustSAM\n```\n3) Use command below to check your CUDA version.\n```\nnvidia-smi\n```\n4) Replace the CUDA version with yours in command below.\n```\npip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu[$YOUR_CUDA_VERSION]\n# For example: pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117 # cu117 = CUDA_version_11.7\n```\n5) Install remaining dependencies\n```\npip install -r requirements.txt\n```\n\n6) Download pretrained [RobustSAM checkpoint](https://drive.google.com/file/d/197EEnWYvchupfJrK44-ki3wDhv3-UuGv/view?usp=sharing) and place into current directory.\n\n## Demo\nWe have prepared some images im **demo_images** folder for demo purpose. Besides, two prompting modes are available (box prompts and point prompts).\n- For box prompt:\n```\npython eval.py --bbox\n```\n- For point prompt:\n```\npython eval.py\n```\nIn default, demo results will be saved to **demo_result/[$PROMPT_TYPE]**.\n\n## Comparison of computational requirements\n<img width=\"720\" alt=\"image\" src='figures/Computational requirements.PNG'>\n\n## Visual Comparison\n<table>\n <tr>\n <td>\n <img src=\"figures/gif_output/blur_back_n_forth.gif\" width=\"390\">\n </td>\n <td>\n <img src=\"figures/gif_output/haze_back_n_forth.gif\" width=\"390\">\n </td>\n </tr>\n <tr>\n <td>\n <img src=\"figures/gif_output/lowlight_back_n_forth.gif\" width=\"390\">\n </td>\n <td>\n <img src=\"figures/gif_output/rain_back_n_forth.gif\" width=\"390\">\n </td>\n </tr>\n</table>\n\n<img width=\"1096\" alt=\"image\" src='figures/qualitative_result.PNG'>\n\n## Quantitative Comparison\n### Seen dataset with synthetic degradation\n<img width=\"720\" alt=\"image\" src='figures/seen_dataset_with_synthetic_degradation.PNG'>\n\n### Unseen dataset with synthetic degradation\n<img width=\"720\" alt=\"image\" src='figures/unseen_dataset_with_synthetic_degradation.PNG'>\n\n### Unseen dataset with real degradation\n<img width=\"600\" alt=\"image\" src='figures/unseen_dataset_with_real_degradation.PNG'>\n\n## Reference\nIf you find this work useful, please consider citing us!\n```python\n@inproceedings{chen2024robustsam,\n title={RobustSAM: Segment Anything Robustly on Degraded Images},\n author={Chen, Wei-Ting and Vong, Yu-Jiet and Kuo, Sy-Yen and Ma, Sizhou and Wang, Jian},\n journal={CVPR},\n year={2024}\n}\n```\n\n\n## Acknowledgements\nWe thank the authors of [SAM](https://github.com/facebookresearch/segment-anything) from which our repo is based off of.\n\n",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2024 robustsam Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ",
"summary": "RobustSAM: Segment Anything Robustly on Degraded Images.",
"version": "1.0.0",
"project_urls": {
"repository": "https://github.com/robustsam/RobustSAM"
},
"split_keywords": [
"deep-learning",
" pytorch",
" vision",
" object-segmentation"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e33e5482f5176f64298e2baeb3fbfbdedb52c2f78136f5c177512475b1b8b811",
"md5": "eda0d62dcf7c16a496b5930363ef492f",
"sha256": "e4ad924578ee197e438a37535416d3a3c8dbab1d9017241f6ce12ea8433080ca"
},
"downloads": -1,
"filename": "robust_segment_anything-1.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "eda0d62dcf7c16a496b5930363ef492f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 39661,
"upload_time": "2024-06-18T10:35:11",
"upload_time_iso_8601": "2024-06-18T10:35:11.499066Z",
"url": "https://files.pythonhosted.org/packages/e3/3e/5482f5176f64298e2baeb3fbfbdedb52c2f78136f5c177512475b1b8b811/robust_segment_anything-1.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "185560b6d2f424fee8a80af41031a76c9b7a994a072a9e217b44bf322e1f35d3",
"md5": "72bf5ba21a463cf039154c550fe48b07",
"sha256": "a6064227bf4ff780f6a4bb815a8f508802c085fe46fd960a86ada8f2f7db9940"
},
"downloads": -1,
"filename": "robust_segment_anything-1.0.0.tar.gz",
"has_sig": false,
"md5_digest": "72bf5ba21a463cf039154c550fe48b07",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 33505,
"upload_time": "2024-06-18T10:35:14",
"upload_time_iso_8601": "2024-06-18T10:35:14.742888Z",
"url": "https://files.pythonhosted.org/packages/18/55/60b6d2f424fee8a80af41031a76c9b7a994a072a9e217b44bf322e1f35d3/robust_segment_anything-1.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-06-18 10:35:14",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "robustsam",
"github_project": "RobustSAM",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "robust-segment-anything"
}
