| Name | icu-sepsis JSON |
| Version |
2.0.1
JSON |
| download |
| home_page | None |
| Summary | Python implementation of the ICU-Sepsis environment. |
| upload_time | 2024-09-19 06:12:04 |
| maintainer | None |
| docs_url | None |
| author | Kartik Choudhary, Dhawal Gupta, Philip S. Thomas |
| requires_python | >=3.10 |
| license | MIT License Copyright (c) 2024 Kartik Choudhary, Dhawal Gupta, Philip S. Thomas Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
icu
reinforcement-learning
sepsis
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# The ICU-Sepsis Environment
The **ICU-Sepsis** environment is a reinforcement learning environment that
simulates the treatment of sepsis in an intensive care unit (ICU).
## Environment description
ICU-Sepsis is a tabular MDP with $N_S = 716$ states ($\\{0,1,\dots,715\\}$) and $N_A = 25$ actions ($\\{0,1,\dots,24\\}$).
Each episode simulates the treatment of one sepsis patient in the ICU.
An episode ends when the patient survives (state $714$) and gets a reward of `+1`, or dies (state $713$) and gets a reward
of `0`, while all the intermediate rewards are `0`. The discount factor is set as $\gamma = 1$.
## Baselines
Some baseline results are shown below as a reference.
<table>
<tr>
<th></th>
<th>Random</th>
<th>Expert</th>
<th>Optimal</th>
</tr>
<tr>
<th>Avg. return</th>
<td>0.78</td>
<td>0.78</td>
<td>0.88</td>
</tr>
<tr>
<th>Avg. episode length</th>
<td>9.45</td>
<td>9.22</td>
<td>10.99</td>
</tr>
</table>
The three baseline policies used are:
1. **Random:** Each action is taken uniformly randomly out of all the actions in any given state.
2. **Expert:** The estimated policy used by clinicians in the real world, computed using the data from the MIMIC-III dataset.
3. **Optimal:** Optimal policy computed using value iteration (requires knowledge of the transition parameters)
## Admissible actions
In the MIMIC-III dataset, not all actions are taken enough times in each state
to reliably estimate the transition probabilities, and such actions are
considered inadmissible. To deal with this issue, the transition probabilities
for inadmissible actions are set to the mean of the transition probabilities of
all admissible actions in the state. This way, the environment can be used with
all actions in all states. This is an implementation detail and does not need to
be considered for normal use. See the paper for more details.
## Installation
ICU-Sepsis can be used with Python `3.10` or later, with gymnasium `0.28.1` or
later, and gym `0.21.0` or later. The environment can be installed using
the `pip` command:
```bash
pip install icu-sepsis
```
### Uninstallation
To uninstall, use the `pip uninstall` command:
```bash
pip uninstall icu_sepsis -y
```
## Quickstart
The environment can be loaded with the Gym or Gymnasium packages and follows
the standard Gym API. The following code snippet demonstrates how to create
the environment, reset it, and take a step:
```python
import gymnasium as gym
import icu_sepsis
env = gym.make('Sepsis/ICU-Sepsis-v2')
state, info = env.reset()
print('Initial state:', state)
print('Extra info:', info)
next_state, reward, terminated, truncated, info = env.step(0)
print('\nTaking action 0:')
print('Next state:', next_state)
print('Reward:', reward)
print('Terminated:', terminated)
print('Truncated:', truncated)
```
You can also run the script `examples/quickstart.py` to verify that the
installation was successful.
### Version 2 changes
As mentioned previously, not all actions are admissible in all states, so the
transition probabilities for inadmissible actions are set to the mean of the
transition probabilities of all admissible actions in the state. This was the
only mode of operation in version 1, and all the baseline numbers are based on
this mode.
In version 2, this mode remains the default, so creating the environment with
`gym.make('Sepsis/ICU-Sepsis-v2')` without providing any additional arguments
is equivalent to the version 1 behavior.
However, in version 2, the environment creation can take an optional argument
`inadmissible_action_strategy` which can be set to the following values:
1. `'mean'` (default): The transition probabilities for inadmissible actions
are set to the mean of the transition probabilities of all admissible actions
in the state.
2. `'terminate'`: The environment terminates the episode if an inadmissible
action is taken in any state, and the patient is sent to the "death" state.
3. `'raise_exception'`: The environment raises an exception if an inadmissible
action is taken in any state.
Raw data
{
"_id": null,
"home_page": null,
"name": "icu-sepsis",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": "Kartik Choudhary <kartikchoudh@cs.umass.edu>",
"keywords": "icu, reinforcement-learning, sepsis",
"author": "Kartik Choudhary, Dhawal Gupta, Philip S. Thomas",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/62/4f/483a2a6638685640a37f8a76c95214d7e279e0bf3bdc975061e8341a2511/icu_sepsis-2.0.1.tar.gz",
"platform": null,
"description": "# The ICU-Sepsis Environment\n\nThe **ICU-Sepsis** environment is a reinforcement learning environment that\nsimulates the treatment of sepsis in an intensive care unit (ICU).\n\n## Environment description\n\nICU-Sepsis is a tabular MDP with $N_S = 716$ states ($\\\\{0,1,\\dots,715\\\\}$) and $N_A = 25$ actions ($\\\\{0,1,\\dots,24\\\\}$).\nEach episode simulates the treatment of one sepsis patient in the ICU.\n\nAn episode ends when the patient survives (state $714$) and gets a reward of `+1`, or dies (state $713$) and gets a reward\nof `0`, while all the intermediate rewards are `0`. The discount factor is set as $\\gamma = 1$.\n\n## Baselines\n\nSome baseline results are shown below as a reference.\n\n<table>\n <tr>\n <th></th>\n <th>Random</th>\n <th>Expert</th>\n <th>Optimal</th>\n </tr>\n <tr>\n <th>Avg. return</th>\n <td>0.78</td>\n <td>0.78</td>\n <td>0.88</td>\n </tr>\n <tr>\n <th>Avg. episode length</th>\n <td>9.45</td>\n <td>9.22</td>\n <td>10.99</td>\n </tr>\n</table>\n\nThe three baseline policies used are:\n1. **Random:** Each action is taken uniformly randomly out of all the actions in any given state.\n2. **Expert:** The estimated policy used by clinicians in the real world, computed using the data from the MIMIC-III dataset.\n3. **Optimal:** Optimal policy computed using value iteration (requires knowledge of the transition parameters)\n\n## Admissible actions\n\nIn the MIMIC-III dataset, not all actions are taken enough times in each state\nto reliably estimate the transition probabilities, and such actions are\nconsidered inadmissible. To deal with this issue, the transition probabilities\nfor inadmissible actions are set to the mean of the transition probabilities of\nall admissible actions in the state. This way, the environment can be used with\nall actions in all states. This is an implementation detail and does not need to\nbe considered for normal use. See the paper for more details.\n\n## Installation\n\nICU-Sepsis can be used with Python `3.10` or later, with gymnasium `0.28.1` or\nlater, and gym `0.21.0` or later. The environment can be installed using\nthe `pip` command:\n\n```bash\npip install icu-sepsis\n```\n\n### Uninstallation\n\nTo uninstall, use the `pip uninstall` command:\n\n```bash\npip uninstall icu_sepsis -y\n```\n\n## Quickstart\n\nThe environment can be loaded with the Gym or Gymnasium packages and follows\nthe standard Gym API. The following code snippet demonstrates how to create\nthe environment, reset it, and take a step:\n\n```python\nimport gymnasium as gym\nimport icu_sepsis\n\nenv = gym.make('Sepsis/ICU-Sepsis-v2')\n\nstate, info = env.reset()\nprint('Initial state:', state)\nprint('Extra info:', info)\n\nnext_state, reward, terminated, truncated, info = env.step(0)\nprint('\\nTaking action 0:')\nprint('Next state:', next_state)\nprint('Reward:', reward)\nprint('Terminated:', terminated)\nprint('Truncated:', truncated)\n```\n\nYou can also run the script `examples/quickstart.py` to verify that the\ninstallation was successful.\n\n### Version 2 changes\n\nAs mentioned previously, not all actions are admissible in all states, so the\ntransition probabilities for inadmissible actions are set to the mean of the\ntransition probabilities of all admissible actions in the state. This was the\nonly mode of operation in version 1, and all the baseline numbers are based on\nthis mode.\n\nIn version 2, this mode remains the default, so creating the environment with\n`gym.make('Sepsis/ICU-Sepsis-v2')` without providing any additional arguments\nis equivalent to the version 1 behavior.\n\nHowever, in version 2, the environment creation can take an optional argument\n`inadmissible_action_strategy` which can be set to the following values:\n\n1. `'mean'` (default): The transition probabilities for inadmissible actions\n are set to the mean of the transition probabilities of all admissible actions\n in the state.\n2. `'terminate'`: The environment terminates the episode if an inadmissible\n action is taken in any state, and the patient is sent to the \"death\" state.\n3. `'raise_exception'`: The environment raises an exception if an inadmissible\n action is taken in any state.\n",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2024 Kartik Choudhary, Dhawal Gupta, Philip S. Thomas Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.",
"summary": "Python implementation of the ICU-Sepsis environment.",
"version": "2.0.1",
"project_urls": null,
"split_keywords": [
"icu",
" reinforcement-learning",
" sepsis"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e45162fa3f95951f148bbb695d5cb4c6b56144cdcbfe8fbdc18e0d7868ada5c0",
"md5": "ead242ab37ccee9ebb53ac04111c754c",
"sha256": "7f55ff190f11b252593000814b6cd56ecf9b5de6fdca256e4966c0a0dc369466"
},
"downloads": -1,
"filename": "icu_sepsis-2.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ead242ab37ccee9ebb53ac04111c754c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 1208209,
"upload_time": "2024-09-19T06:12:01",
"upload_time_iso_8601": "2024-09-19T06:12:01.764659Z",
"url": "https://files.pythonhosted.org/packages/e4/51/62fa3f95951f148bbb695d5cb4c6b56144cdcbfe8fbdc18e0d7868ada5c0/icu_sepsis-2.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "624f483a2a6638685640a37f8a76c95214d7e279e0bf3bdc975061e8341a2511",
"md5": "253ccaa709c662e6ed31c0ceb3abca0e",
"sha256": "93cc0f70d06b23ad024856214942077edef58c62492c6641d1a47ac38cfb1f2f"
},
"downloads": -1,
"filename": "icu_sepsis-2.0.1.tar.gz",
"has_sig": false,
"md5_digest": "253ccaa709c662e6ed31c0ceb3abca0e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 1206592,
"upload_time": "2024-09-19T06:12:04",
"upload_time_iso_8601": "2024-09-19T06:12:04.013362Z",
"url": "https://files.pythonhosted.org/packages/62/4f/483a2a6638685640a37f8a76c95214d7e279e0bf3bdc975061e8341a2511/icu_sepsis-2.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-19 06:12:04",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "icu-sepsis"
}
