| Name | green-score JSON |
| Version |
0.0.5
JSON |
| download |
| home_page | None |
| Summary | None |
| upload_time | 2024-05-17 19:41:36 |
| maintainer | None |
| docs_url | None |
| author | Jean-Benoit Delbrouck |
| requires_python | >=3.8 |
| license | MIT |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# GREEN: Generative Radiology Report Evaluation and Error Notation
[[Project Page](https://stanford-aimi.github.io/green.html)][[Paper](https://arxiv.org/pdf/2405.03595)]
## Installation
```bash
pip install green-score
```
## Usage
```python
from green_score import GREEN
model = GREEN(
model_id_or_path="StanfordAIMI/GREEN-radllama2-7b",
do_sample=False, # should be always False
batch_size=16,
return_0_if_no_green_score=True,
cuda=True,
)
refs = [
"Interstitial opacities without changes.",
"Interval development of segmental heterogeneous airspace opacities throughout the lungs . No significant pneumothorax or pleural effusion . Bilateral calcified pleural plaques are scattered throughout the lungs . The heart is not significantly enlarged .",
"Lung volumes are low, causing bronchovascular crowding. The cardiomediastinal silhouette is unremarkable. No focal consolidation, pleural effusion, or pneumothorax detected. Within the limitations of chest radiography, osseous structures are unremarkable.",
"Lung volumes are low, causing bronchovascular crowding. The cardiomediastinal silhouette is unremarkable. No focal consolidation, pleural effusion, or pneumothorax detected. Within the limitations of chest radiography, osseous structures are unremarkable.",
]
hyps = [
"Interstitial opacities at bases without changes.",
"Interval development of segmental heterogeneous airspace opacities throughout the lungs . No significant pneumothorax or pleural effusion . Bilateral calcified pleural plaques are scattered throughout the lungs . The heart is not significantly enlarged .",
"Endotracheal and nasogastric tubes have been removed. Changes of median sternotomy, with continued leftward displacement of the fourth inferiomost sternal wire. There is continued moderate-to-severe enlargement of the cardiac silhouette. Pulmonary aeration is slightly improved, with residual left lower lobe atelectasis. Stable central venous congestion and interstitial pulmonary edema. Small bilateral pleural effusions are unchanged.",
"Endotracheal and nasogastric tubes have been removed. Changes of median sternotomy, with continued leftward displacement of the fourth inferiomost sternal wire. There is continued moderate-to-severe enlargement of the cardiac silhouette. Pulmonary aeration is slightly improved, with residual left lower lobe atelectasis. Stable central venous congestion and interstitial pulmonary edema. Small bilateral pleural effusions are unchanged.",
]
mean_green, greens, explanations = model(refs=refs, hyps=hyps)
print("Mean reward for the given examples is: ", mean_green)
print("Array of rewards for the given examples is: ", greens)
print(explanations[0]) # LLM output for first pair
```
## Benchmark
| Model | Eval x | sec/sample | Batch Size* |
|-------------------------------------------------------------------------------------------|--------|------------|-------------|
| [StanfordAIMI/GREEN-radllama2-7b](https://huggingface.co/StanfordAIMI/GREEN-radllama2-7b) | x | 2.3 | 16 |
* A100 (40GB)
## Reference
```bibtex
@article{ostmeier2024green,
title={GREEN: Generative Radiology Report Evaluation and Error Notation},
author={Ostmeier, Sophie and Xu, Justin and Chen, Zhihong and Varma, Maya and Blankemeier, Louis and Bluethgen, Christian and Michalson, Arne Edward and Moseley, Michael and Langlotz, Curtis and Chaudhari, Akshay S and others},
journal={arXiv preprint arXiv:2405.03595},
year={2024}
}
```
Raw data
{
"_id": null,
"home_page": null,
"name": "green-score",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": null,
"author": "Jean-Benoit Delbrouck",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/8f/91/a4bf6f20560dbf9e3ef943494be15fb25b0e949b4451e9c94dfe986a8848/green_score-0.0.5.tar.gz",
"platform": null,
"description": "# GREEN: Generative Radiology Report Evaluation and Error Notation\n[[Project Page](https://stanford-aimi.github.io/green.html)][[Paper](https://arxiv.org/pdf/2405.03595)]\n\n## Installation\n\n```bash\npip install green-score\n```\n\n## Usage\n\n```python\nfrom green_score import GREEN\n\nmodel = GREEN(\n model_id_or_path=\"StanfordAIMI/GREEN-radllama2-7b\",\n do_sample=False, # should be always False\n batch_size=16,\n return_0_if_no_green_score=True,\n cuda=True,\n)\n\nrefs = [\n \"Interstitial opacities without changes.\",\n \"Interval development of segmental heterogeneous airspace opacities throughout the lungs . No significant pneumothorax or pleural effusion . Bilateral calcified pleural plaques are scattered throughout the lungs . The heart is not significantly enlarged .\",\n \"Lung volumes are low, causing bronchovascular crowding. The cardiomediastinal silhouette is unremarkable. No focal consolidation, pleural effusion, or pneumothorax detected. Within the limitations of chest radiography, osseous structures are unremarkable.\",\n \"Lung volumes are low, causing bronchovascular crowding. The cardiomediastinal silhouette is unremarkable. No focal consolidation, pleural effusion, or pneumothorax detected. Within the limitations of chest radiography, osseous structures are unremarkable.\",\n]\nhyps = [\n \"Interstitial opacities at bases without changes.\",\n \"Interval development of segmental heterogeneous airspace opacities throughout the lungs . No significant pneumothorax or pleural effusion . Bilateral calcified pleural plaques are scattered throughout the lungs . The heart is not significantly enlarged .\",\n \"Endotracheal and nasogastric tubes have been removed. Changes of median sternotomy, with continued leftward displacement of the fourth inferiomost sternal wire. There is continued moderate-to-severe enlargement of the cardiac silhouette. Pulmonary aeration is slightly improved, with residual left lower lobe atelectasis. Stable central venous congestion and interstitial pulmonary edema. Small bilateral pleural effusions are unchanged.\",\n \"Endotracheal and nasogastric tubes have been removed. Changes of median sternotomy, with continued leftward displacement of the fourth inferiomost sternal wire. There is continued moderate-to-severe enlargement of the cardiac silhouette. Pulmonary aeration is slightly improved, with residual left lower lobe atelectasis. Stable central venous congestion and interstitial pulmonary edema. Small bilateral pleural effusions are unchanged.\",\n]\n\nmean_green, greens, explanations = model(refs=refs, hyps=hyps)\nprint(\"Mean reward for the given examples is: \", mean_green)\nprint(\"Array of rewards for the given examples is: \", greens)\nprint(explanations[0]) # LLM output for first pair\n\n```\n\n## Benchmark\n\n\n| Model | Eval x | sec/sample | Batch Size* |\n|-------------------------------------------------------------------------------------------|--------|------------|-------------|\n| [StanfordAIMI/GREEN-radllama2-7b](https://huggingface.co/StanfordAIMI/GREEN-radllama2-7b) | x | 2.3 | 16 |\n* A100 (40GB)\n\n## Reference\n\n```bibtex\n@article{ostmeier2024green,\n title={GREEN: Generative Radiology Report Evaluation and Error Notation},\n author={Ostmeier, Sophie and Xu, Justin and Chen, Zhihong and Varma, Maya and Blankemeier, Louis and Bluethgen, Christian and Michalson, Arne Edward and Moseley, Michael and Langlotz, Curtis and Chaudhari, Akshay S and others},\n journal={arXiv preprint arXiv:2405.03595},\n year={2024}\n}\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": null,
"version": "0.0.5",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8f91a4bf6f20560dbf9e3ef943494be15fb25b0e949b4451e9c94dfe986a8848",
"md5": "03418982a23af2effad499e46f9696b4",
"sha256": "1745517771f91d9c80f8dde266ee6980d1ceb1c763ea1ba208efd4f27eb8791e"
},
"downloads": -1,
"filename": "green_score-0.0.5.tar.gz",
"has_sig": false,
"md5_digest": "03418982a23af2effad499e46f9696b4",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 7837,
"upload_time": "2024-05-17T19:41:36",
"upload_time_iso_8601": "2024-05-17T19:41:36.460635Z",
"url": "https://files.pythonhosted.org/packages/8f/91/a4bf6f20560dbf9e3ef943494be15fb25b0e949b4451e9c94dfe986a8848/green_score-0.0.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-17 19:41:36",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "green-score"
}
