<div align="center">
[](https://github.com/sleep3r/garrus)
**In the middle of some calibrations...**
[](https://www.codefactor.io/repository/github/sleep3r/garrus)
[](https://pypi.org/project/garrus/)
[](https://github.com/sleep3r/garrus/wiki)
[](https://pepy.tech/project/garrus)
[](https://t.me/sleep3r)
[](https://github.com/sleep3r/garrus/graphs/contributors)
[](https://github.com/sleep3r/garrus/badge.svg?branch=main&event=push)
[](https://github.com/sleep3r/garrus/badge.svg?branch=main&event=push)
[](https://github.com/sleep3r/garrus/badge.svg?branch=main&event=push)
</div>
Garrus is a python framework for better confidence estimate of deep neural networks. Modern networks are overconfident estimators, that makes themselves unreliable and therefore limits the deployment of them in safety-critical applications.
Garrus provides tools for high quality confidence estimation such as confidence calibration and ordinal ranking methods, helping networks to **know correctly what they do not know**.
----
## Installation:
```bash
pip install -U garrus
```
## Documentation:
- [master v0.2.0](https://github.com/sleep3r/garrus/wiki/0.2.0@master-documentation)
- [develop v0.2.0](https://github.com/sleep3r/garrus/wiki/0.2.0@develop-documentation)
## Roadmap:
- Core:
- Calibration metrics:
- [x] ECE
- [x] NLL
- [x] Brier
- Ordinal Ranking Metrics:
- [x] AURC
- [x] E-AURC
- [x] AUPRE
- [x] FPR-n%-TPR
- Visualizations:
- [x] Reliability Diagram
- [x] Confidence Histogram
- [ ] Garrus Profiling
- Confidence Calibration:
- Scaling:
- [x] Platt
- [x] Temperature
- Binning:
- [ ] Histogram
- [ ] Isotonic Regression
- Confidence Regularization:
- Losses:
- [ ] Correctness Ranking Loss
- [ ] Focal Entropy Penalized Loss
- [ ] Language Model Beam Search
- Confidence Networks:
- [ ] ConfidNet
- [ ] GarrusNet
---
### Citation:
Please use this bibtex if you want to cite this repository in your publications:
@misc{garrus,
author = {Kalashnikov, Alexander},
title = {Deep neural networks calibration framework},
year = {2021},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/sleep3r/garrus}},
}
### References:
|Papers|
|---|
| [[1]](https://arxiv.org/pdf/1706.04599.pdf) Guo, Chuan, et al. "On calibration of modern neural networks." International Conference on Machine Learning. PMLR, 2017. APA |
| [[2]](https://arxiv.org/pdf/2007.01458.pdf) Moon, Jooyoung, et al. "Confidence-aware learning for deep neural networks." international conference on machine learning. PMLR, 2020. |
| [[3]](https://arxiv.org/pdf/1909.10155.pdf) Kumar, Ananya, Percy Liang, and Tengyu Ma. "Verified uncertainty calibration." arXiv preprint arXiv:1909.10155 (2019). |
Raw data
{
"_id": null,
"home_page": "https://github.com/sleep3r/garrus",
"name": "garrus",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.12",
"maintainer_email": null,
"keywords": "machine-learning, data-visualization, pytorch",
"author": "sleep3r",
"author_email": "sleep3r@icloud.com",
"download_url": "https://files.pythonhosted.org/packages/e7/a1/28e868a8439db242eddabcca8dee4a305e9d4f58f192c1ac6b2e54bac7b4/garrus-0.3.0.tar.gz",
"platform": null,
"description": "<div align=\"center\">\n\n[](https://github.com/sleep3r/garrus)\n\n**In the middle of some calibrations...**\n\n[](https://www.codefactor.io/repository/github/sleep3r/garrus)\n[](https://pypi.org/project/garrus/)\n[](https://github.com/sleep3r/garrus/wiki)\n[](https://pepy.tech/project/garrus)\n\n[](https://t.me/sleep3r)\n[](https://github.com/sleep3r/garrus/graphs/contributors)\n\n[](https://github.com/sleep3r/garrus/badge.svg?branch=main&event=push)\n[](https://github.com/sleep3r/garrus/badge.svg?branch=main&event=push)\n[](https://github.com/sleep3r/garrus/badge.svg?branch=main&event=push)\n\n</div>\n\nGarrus is a python framework for better confidence estimate of deep neural networks. Modern networks are overconfident estimators, that makes themselves unreliable and therefore limits the deployment of them in safety-critical applications.\n\nGarrus provides tools for high quality confidence estimation such as confidence calibration and ordinal ranking methods, helping networks to **know correctly what they do not know**. \n\n----\n\n## Installation:\n```bash\npip install -U garrus\n```\n\n## Documentation:\n - [master v0.2.0](https://github.com/sleep3r/garrus/wiki/0.2.0@master-documentation)\n - [develop v0.2.0](https://github.com/sleep3r/garrus/wiki/0.2.0@develop-documentation)\n\n## Roadmap:\n- Core:\n - Calibration metrics:\n - [x] ECE\n - [x] NLL\n - [x] Brier\n - Ordinal Ranking Metrics:\n - [x] AURC\n - [x] E-AURC\n - [x] AUPRE\n - [x] FPR-n%-TPR\n - Visualizations:\n - [x] Reliability Diagram\n - [x] Confidence Histogram\n - [ ] Garrus Profiling\n- Confidence Calibration:\n - Scaling:\n - [x] Platt\n - [x] Temperature\n - Binning: \n - [ ] Histogram\n - [ ] Isotonic Regression\n- Confidence Regularization:\n - Losses:\n - [ ] Correctness Ranking Loss\n - [ ] Focal Entropy Penalized Loss\n - [ ] Language Model Beam Search\n- Confidence Networks:\n - [ ] ConfidNet\n - [ ] GarrusNet\n\n---\n\n### Citation:\nPlease use this bibtex if you want to cite this repository in your publications:\n\n @misc{garrus,\n author = {Kalashnikov, Alexander},\n title = {Deep neural networks calibration framework},\n year = {2021},\n publisher = {GitHub},\n journal = {GitHub repository},\n howpublished = {\\url{https://github.com/sleep3r/garrus}},\n }\n \n### References:\n|Papers|\n|---|\n| [[1]](https://arxiv.org/pdf/1706.04599.pdf) Guo, Chuan, et al. \"On calibration of modern neural networks.\" International Conference on Machine Learning. PMLR, 2017. APA |\n| [[2]](https://arxiv.org/pdf/2007.01458.pdf) Moon, Jooyoung, et al. \"Confidence-aware learning for deep neural networks.\" international conference on machine learning. PMLR, 2020. |\n| [[3]](https://arxiv.org/pdf/1909.10155.pdf) Kumar, Ananya, Percy Liang, and Tengyu Ma. \"Verified uncertainty calibration.\" arXiv preprint arXiv:1909.10155 (2019). |",
"bugtrack_url": null,
"license": "MIT",
"summary": "A Python package for machine learning and data visualization",
"version": "0.3.0",
"project_urls": {
"Homepage": "https://github.com/sleep3r/garrus",
"Repository": "https://github.com/sleep3r/garrus"
},
"split_keywords": [
"machine-learning",
" data-visualization",
" pytorch"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c262637e75f3e3431d5fd0807386621bce5a6686bdc1a6d6ae83096841b9ed37",
"md5": "6030824f03a6fecb35c1f097958271a5",
"sha256": "d6a9ba09f61e9875771939c099a461ba0daa7dafd3892e33c5cbfd968c55ff1d"
},
"downloads": -1,
"filename": "garrus-0.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6030824f03a6fecb35c1f097958271a5",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.12",
"size": 16923,
"upload_time": "2024-12-25T17:00:03",
"upload_time_iso_8601": "2024-12-25T17:00:03.030839Z",
"url": "https://files.pythonhosted.org/packages/c2/62/637e75f3e3431d5fd0807386621bce5a6686bdc1a6d6ae83096841b9ed37/garrus-0.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e7a128e868a8439db242eddabcca8dee4a305e9d4f58f192c1ac6b2e54bac7b4",
"md5": "ea68665873276e759bd6eed9ea1b8ff6",
"sha256": "62d2510a0ed89e42a272aed9037ca39c070f6938085c99764cc97cf56e057c41"
},
"downloads": -1,
"filename": "garrus-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "ea68665873276e759bd6eed9ea1b8ff6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.12",
"size": 12448,
"upload_time": "2024-12-25T17:00:05",
"upload_time_iso_8601": "2024-12-25T17:00:05.452830Z",
"url": "https://files.pythonhosted.org/packages/e7/a1/28e868a8439db242eddabcca8dee4a305e9d4f58f192c1ac6b2e54bac7b4/garrus-0.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-25 17:00:05",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "sleep3r",
"github_project": "garrus",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "garrus"
}
JSON
