<div align="center">
<img src="https://github.com/lartpang/PySODMetrics/blob/main/images/logo.png?raw=true" alt="Logo" width="320" height="auto">
</br>
<h2>PySODMetrics: A simple and efficient implementation of SOD metrics</h2>
<a href="./readme_zh.md"><img src="https://img.shields.io/badge/README-%E4%B8%AD%E6%96%87-blue"></a>
<img src="https://img.shields.io/pypi/v/pysodmetrics">
<img src="https://img.shields.io/pypi/dm/pysodmetrics?label=pypi%20downloads&logo=PyPI&logoColor=white">
<img src="https://img.shields.io/github/last-commit/lartpang/PySODMetrics">
<img src="https://img.shields.io/github/last-commit/lartpang/PySODMetrics">
<img src="https://img.shields.io/github/release-date/lartpang/PySODMetrics">
</div>
## Introduction
A simple and efficient implementation of SOD metrics.
- Based on `numpy` and `scipy`
- Verification based on Fan's matlab code <https://github.com/DengPingFan/CODToolbox>
- The code structure is simple and easy to extend
- The code is lightweight and fast
Your improvements and suggestions are welcome.
### Related Projects
- [PySODEvalToolkit](https://github.com/lartpang/PySODEvalToolkit): A Python-based Evaluation Toolbox for Salient Object Detection and Camouflaged Object Detection
### Supported Metrics
| Metric | Sample-based | Whole-based | Related Class |
| ----------------------------------------- | --------------- | ----------- | ------------------------------------- |
| MAE | soft | | `MAE` |
| S-measure $S_{m}$ | soft | | `Smeasure` |
| weighted F-measure ($F^{\omega}_{\beta}$) | soft | | `WeightedFmeasure` |
| Multi-Scale IoU | bin | | `MSIoU` |
| E-measure ($E_{m}$) | max,avg,adp | | `Emeasure` |
| F-measure (old) ($F_{beta}$) | max,avg,adp | | `Fmeasure` |
| F-measure (new) ($F_{beta}$, $F_{1}$) | max,avg,adp,bin | bin | `FmeasureV2`+`FmeasureHandler` |
| BER | max,avg,adp,bin | bin | `FmeasureV2`+`BERHandler` |
| Dice | max,avg,adp,bin | bin | `FmeasureV2`+`DICEHandler` |
| FPR | max,avg,adp,bin | bin | `FmeasureV2`+`FPRHandler` |
| IoU | max,avg,adp,bin | bin | `FmeasureV2`+`IOUHandler` |
| Kappa | max,avg,adp,bin | bin | `FmeasureV2`+`KappaHandler` |
| Overall Accuracy | max,avg,adp,bin | bin | `FmeasureV2`+`OverallAccuracyHandler` |
| Precision | max,avg,adp,bin | bin | `FmeasureV2`+`PrecisionHandler` |
| Recall | max,avg,adp,bin | bin | `FmeasureV2`+`RecallHandler` |
| Sensitivity | max,avg,adp,bin | bin | `FmeasureV2`+`SensitivityHandler` |
| Specificity | max,avg,adp,bin | bin | `FmeasureV2`+`SpecificityHandler` |
| TNR | max,avg,adp,bin | bin | `FmeasureV2`+`TNRHandler` |
| TPR | max,avg,adp,bin | bin | `FmeasureV2`+`TPRHandler` |
## Usage
The core files are in the folder `py_sod_metrics`.
- **[Latest, but may be unstable]** Install from the source code: `pip install git+https://github.com/lartpang/PySODMetrics.git`
- **[More stable]** Install from PyPI: `pip install pysodmetrics`
### Examples
- [examples/test_metrics.py](./examples/test_metrics.py)
- [examples/metric_recorder.py](./examples/metric_recorder.py)
## Reference
- [Matlab Code](https://github.com/DengPingFan/CODToolbox) by DengPingFan(<https://github.com/DengPingFan>): In our comparison (the test code can be seen under the `test` folder), the result is consistent with the code.
- The matlab code needs to change `Bi_sal(sal>threshold)=1;` to `Bi_sal(sal>=threshold)=1;` in <https://github.com/DengPingFan/CODToolbox/blob/910358910c7824a4237b0ea689ac9d19d1958d11/Onekey_Evaluation_Code/OnekeyEvaluationCode/main.m#L102>. For related discussion, please see [the issue](https://github.com/DengPingFan/CODToolbox/issues/1).
- 2021-12-20 (version `1.3.0`): Due to the difference between numpy and matlab, in version `1.2.x`, there are very slight differences on some metrics between the results of the matlab code and ours. The [recent PR](https://github.com/lartpang/PySODMetrics/pull/3) alleviated this problem. However, there are still very small differences on E-measure. The results in most papers are rounded off to three or four significant figures, so, there is no obvious difference between the new version and the version `1.2.x` for them.
- <https://en.wikipedia.org/wiki/Precision_and_recall>
```text
@inproceedings{Fmeasure,
title={Frequency-tuned salient region detection},
author={Achanta, Radhakrishna and Hemami, Sheila and Estrada, Francisco and S{\"u}sstrunk, Sabine},
booktitle=CVPR,
number={CONF},
pages={1597--1604},
year={2009}
}
@inproceedings{MAE,
title={Saliency filters: Contrast based filtering for salient region detection},
author={Perazzi, Federico and Kr{\"a}henb{\"u}hl, Philipp and Pritch, Yael and Hornung, Alexander},
booktitle=CVPR,
pages={733--740},
year={2012}
}
@inproceedings{Smeasure,
title={Structure-measure: A new way to evaluate foreground maps},
author={Fan, Deng-Ping and Cheng, Ming-Ming and Liu, Yun and Li, Tao and Borji, Ali},
booktitle=ICCV,
pages={4548--4557},
year={2017}
}
@inproceedings{Emeasure,
title="Enhanced-alignment Measure for Binary Foreground Map Evaluation",
author="Deng-Ping {Fan} and Cheng {Gong} and Yang {Cao} and Bo {Ren} and Ming-Ming {Cheng} and Ali {Borji}",
booktitle=IJCAI,
pages="698--704",
year={2018}
}
@inproceedings{wFmeasure,
title={How to evaluate foreground maps?},
author={Margolin, Ran and Zelnik-Manor, Lihi and Tal, Ayellet},
booktitle=CVPR,
pages={248--255},
year={2014}
}
@inproceedings{MSIoU,
title = {Multiscale IOU: A Metric for Evaluation of Salient Object Detection with Fine Structures},
author = {Ahmadzadeh, Azim and Kempton, Dustin J. and Chen, Yang and Angryk, Rafal A.},
booktitle = ICIP,
year = {2021},
}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/lartpang/PySODMetrics",
"name": "pysodmetrics",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "salient object detection,saliency detection,metric,deep learning",
"author": "lartpang",
"author_email": "lartpang@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/21/69/3d63f26b1e35f9a81473ce21810068698f8585e9e426814b46c05fb263ca/pysodmetrics-1.4.2.tar.gz",
"platform": null,
"description": "<div align=\"center\">\n <img src=\"https://github.com/lartpang/PySODMetrics/blob/main/images/logo.png?raw=true\" alt=\"Logo\" width=\"320\" height=\"auto\">\n </br>\n <h2>PySODMetrics: A simple and efficient implementation of SOD metrics</h2>\n <a href=\"./readme_zh.md\"><img src=\"https://img.shields.io/badge/README-%E4%B8%AD%E6%96%87-blue\"></a>\n <img src=\"https://img.shields.io/pypi/v/pysodmetrics\">\n <img src=\"https://img.shields.io/pypi/dm/pysodmetrics?label=pypi%20downloads&logo=PyPI&logoColor=white\">\n <img src=\"https://img.shields.io/github/last-commit/lartpang/PySODMetrics\">\n <img src=\"https://img.shields.io/github/last-commit/lartpang/PySODMetrics\">\n <img src=\"https://img.shields.io/github/release-date/lartpang/PySODMetrics\">\n</div>\n\n## Introduction\n\nA simple and efficient implementation of SOD metrics.\n\n- Based on `numpy` and `scipy`\n- Verification based on Fan's matlab code <https://github.com/DengPingFan/CODToolbox>\n- The code structure is simple and easy to extend\n- The code is lightweight and fast\n\nYour improvements and suggestions are welcome.\n\n### Related Projects\n\n- [PySODEvalToolkit](https://github.com/lartpang/PySODEvalToolkit): A Python-based Evaluation Toolbox for Salient Object Detection and Camouflaged Object Detection\n\n### Supported Metrics\n\n| Metric | Sample-based | Whole-based | Related Class |\n| ----------------------------------------- | --------------- | ----------- | ------------------------------------- |\n| MAE | soft | | `MAE` |\n| S-measure $S_{m}$ | soft | | `Smeasure` |\n| weighted F-measure ($F^{\\omega}_{\\beta}$) | soft | | `WeightedFmeasure` |\n| Multi-Scale IoU | bin | | `MSIoU` |\n| E-measure ($E_{m}$) | max,avg,adp | | `Emeasure` |\n| F-measure (old) ($F_{beta}$) | max,avg,adp | | `Fmeasure` |\n| F-measure (new) ($F_{beta}$, $F_{1}$) | max,avg,adp,bin | bin | `FmeasureV2`+`FmeasureHandler` |\n| BER | max,avg,adp,bin | bin | `FmeasureV2`+`BERHandler` |\n| Dice | max,avg,adp,bin | bin | `FmeasureV2`+`DICEHandler` |\n| FPR | max,avg,adp,bin | bin | `FmeasureV2`+`FPRHandler` |\n| IoU | max,avg,adp,bin | bin | `FmeasureV2`+`IOUHandler` |\n| Kappa | max,avg,adp,bin | bin | `FmeasureV2`+`KappaHandler` |\n| Overall Accuracy | max,avg,adp,bin | bin | `FmeasureV2`+`OverallAccuracyHandler` |\n| Precision | max,avg,adp,bin | bin | `FmeasureV2`+`PrecisionHandler` |\n| Recall | max,avg,adp,bin | bin | `FmeasureV2`+`RecallHandler` |\n| Sensitivity | max,avg,adp,bin | bin | `FmeasureV2`+`SensitivityHandler` |\n| Specificity | max,avg,adp,bin | bin | `FmeasureV2`+`SpecificityHandler` |\n| TNR | max,avg,adp,bin | bin | `FmeasureV2`+`TNRHandler` |\n| TPR | max,avg,adp,bin | bin | `FmeasureV2`+`TPRHandler` |\n\n## Usage\n\nThe core files are in the folder `py_sod_metrics`.\n\n- **[Latest, but may be unstable]** Install from the source code: `pip install git+https://github.com/lartpang/PySODMetrics.git`\n- **[More stable]** Install from PyPI: `pip install pysodmetrics`\n\n### Examples\n\n- [examples/test_metrics.py](./examples/test_metrics.py)\n- [examples/metric_recorder.py](./examples/metric_recorder.py)\n\n## Reference\n\n- [Matlab Code](https://github.com/DengPingFan/CODToolbox) by DengPingFan(<https://github.com/DengPingFan>): In our comparison (the test code can be seen under the `test` folder), the result is consistent with the code.\n - The matlab code needs to change `Bi_sal(sal>threshold)=1;` to `Bi_sal(sal>=threshold)=1;` in <https://github.com/DengPingFan/CODToolbox/blob/910358910c7824a4237b0ea689ac9d19d1958d11/Onekey_Evaluation_Code/OnekeyEvaluationCode/main.m#L102>. For related discussion, please see [the issue](https://github.com/DengPingFan/CODToolbox/issues/1).\n - 2021-12-20 (version `1.3.0`): Due to the difference between numpy and matlab, in version `1.2.x`, there are very slight differences on some metrics between the results of the matlab code and ours. The [recent PR](https://github.com/lartpang/PySODMetrics/pull/3) alleviated this problem. However, there are still very small differences on E-measure. The results in most papers are rounded off to three or four significant figures, so, there is no obvious difference between the new version and the version `1.2.x` for them.\n- <https://en.wikipedia.org/wiki/Precision_and_recall>\n\n```text\n@inproceedings{Fmeasure,\n title={Frequency-tuned salient region detection},\n author={Achanta, Radhakrishna and Hemami, Sheila and Estrada, Francisco and S{\\\"u}sstrunk, Sabine},\n booktitle=CVPR,\n number={CONF},\n pages={1597--1604},\n year={2009}\n}\n\n@inproceedings{MAE,\n title={Saliency filters: Contrast based filtering for salient region detection},\n author={Perazzi, Federico and Kr{\\\"a}henb{\\\"u}hl, Philipp and Pritch, Yael and Hornung, Alexander},\n booktitle=CVPR,\n pages={733--740},\n year={2012}\n}\n\n@inproceedings{Smeasure,\n title={Structure-measure: A new way to evaluate foreground maps},\n author={Fan, Deng-Ping and Cheng, Ming-Ming and Liu, Yun and Li, Tao and Borji, Ali},\n booktitle=ICCV,\n pages={4548--4557},\n year={2017}\n}\n\n@inproceedings{Emeasure,\n title=\"Enhanced-alignment Measure for Binary Foreground Map Evaluation\",\n author=\"Deng-Ping {Fan} and Cheng {Gong} and Yang {Cao} and Bo {Ren} and Ming-Ming {Cheng} and Ali {Borji}\",\n booktitle=IJCAI,\n pages=\"698--704\",\n year={2018}\n}\n\n@inproceedings{wFmeasure,\n title={How to evaluate foreground maps?},\n author={Margolin, Ran and Zelnik-Manor, Lihi and Tal, Ayellet},\n booktitle=CVPR,\n pages={248--255},\n year={2014}\n}\n\n@inproceedings{MSIoU,\n title = {Multiscale IOU: A Metric for Evaluation of Salient Object Detection with Fine Structures},\n author = {Ahmadzadeh, Azim and Kempton, Dustin J. and Chen, Yang and Angryk, Rafal A.},\n booktitle = ICIP,\n year = {2021},\n}\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A simple and efficient implementation of SOD metrics.",
"version": "1.4.2",
"project_urls": {
"Homepage": "https://github.com/lartpang/PySODMetrics"
},
"split_keywords": [
"salient object detection",
"saliency detection",
"metric",
"deep learning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1ad6f947e0354970c29eff0995effa378fdd1b9dffd1eff01acdac1e030ae196",
"md5": "07d94568d4f9a23379704604376c5532",
"sha256": "ef591420845f652cd83c4fc60a89747caa860b45e9457c88c0e51d39d656bd5d"
},
"downloads": -1,
"filename": "pysodmetrics-1.4.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "07d94568d4f9a23379704604376c5532",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 16106,
"upload_time": "2024-01-16T07:44:35",
"upload_time_iso_8601": "2024-01-16T07:44:35.520710Z",
"url": "https://files.pythonhosted.org/packages/1a/d6/f947e0354970c29eff0995effa378fdd1b9dffd1eff01acdac1e030ae196/pysodmetrics-1.4.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "21693d63f26b1e35f9a81473ce21810068698f8585e9e426814b46c05fb263ca",
"md5": "bb7730428e5be0e3ba66ffe312ba884a",
"sha256": "e49f77bc4b62ffd6ba548967795907b3e5704932dc8452a4dc0116b1b45733a8"
},
"downloads": -1,
"filename": "pysodmetrics-1.4.2.tar.gz",
"has_sig": false,
"md5_digest": "bb7730428e5be0e3ba66ffe312ba884a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 16591,
"upload_time": "2024-01-16T07:44:36",
"upload_time_iso_8601": "2024-01-16T07:44:36.573474Z",
"url": "https://files.pythonhosted.org/packages/21/69/3d63f26b1e35f9a81473ce21810068698f8585e9e426814b46c05fb263ca/pysodmetrics-1.4.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-16 07:44:36",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "lartpang",
"github_project": "PySODMetrics",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "pysodmetrics"
}
JSON
