| Name | posggym JSON |
| Version |
0.5.1
JSON |
| download |
| home_page | |
| Summary | A collection of diverse environments and reference agents for reinforcement learning and planning research in Partially Observable Stochastic Games (POSGs). . |
| upload_time | 2023-11-22 22:33:03 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.8 |
| license | MIT License |
| keywords |
multiagent-systems
reinforcement-learning
planning
gymnasium
posg
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
[](https://opensource.org/licenses/MIT)
[](https://github.com/psf/black)
# POSGGym
POSGGym is an open source Python library providing implementations of Partially Observable Stochastic Game (POSG) environments coupled with dynamic models of each environment, all under a unified API. While there are a number of amazing open-source implementations for POSG environments, very few have support for dynamic models that can be used for planning, especially for continuous domains. The aim of this library is to fill this gap.
A key goal of POSGGym is to provide easy to use and understand open-source implementations for many of the environments commonly used in the partially observable multi-agent planning literature.
POSGGym also provides a collection of reference agents for it's various environments and a unified Agents API for using these agents. These agents can be used to evaluate algorithms, and includes both hand-coded bots and agents trained using reinforcement learning.
POSGGym is directly inspired by and adapted from the [Gymnasium](https://gymnasium.farama.org/) and [PettingZoo](https://pettingzoo.farama.org/) libraries for reinforcement learning. The key additions in POSGGym are support for environment models which can be used for planning and reference agents. POSGGym's API aims to stay as close to the Gymnasium and PettingZoo Parallel APIs as possible while incorporating models and agents into the mix.
Any POSGGym environment can be converted into a PettingZoo environment using a simple wrapper (`posggym.wrappers.petting_zoo.PettingZoo`). This allows for easy integration with the ecosystem of libraries that support PettingZoo.
## Documentation
The documentation for the project is available at [posggym.readthedocs.io/](https://posggym.readthedocs.io/).
For a guide to building the documentation locally see [docs/README.md](docs/README.md).
## Installation
POSGGym supports and tests for `Python>=3.8`. We recommend using a virtual environment to install POSGGym (e.g. [conda](https://docs.conda.io/projects/conda/en/latest/index.html), [venv](https://docs.python.org/3/library/venv.html)).
### Using pip
The latest release version of POSGGym can be installed using `pip` by running:
```bash
pip install posggym
```
This will install the base dependencies for running the main environments and download the agent models (so may take a few minutes), but but in order to minimise the number of unused dependencies installed may not include all dependencies for all environments or for rendering some environments, and will not include dependencies for running many posggym agents.
You can install all dependencies for a family of environments like `pip install posggym[grid-world]` and `pip install posggym[continuous]` or dependencies for all environments using `pip install posggym[envs-all]`.
You can install dependencies for POSGGym agents using `pip install posggym[agents]` or to install dependencies for all environments and agents use `pip install posggym[all]`.
### Installing from source
To install POSGGym from source, first clone the repository then run:
```bash
cd posggym
pip install -e .
```
This will install the base dependencies and download the agent models (so may take a few minutes). You can optionally install extras as described above. E.g. to install all dependencies for all environments and agents use:
```bash
pip install -e .[all]
```
To run tests, install the test dependencies and then run the tests:
```bash
pip install -e .[test]
pytest
```
Or alternatively you can run one of the examples from the `examples` directory:
```bash
python examples/run_random_agents.py --env_id Driving-v0 --num_episodes 10 --render_mode human
```
## Environments
POSGGym includes the following families of environments (for a full list of environments and their descriptsion see the [documentation](https://posggym.readthedocs.io/)).
- [Classic](https://posggym.readthedocs.io/en/latest/environments/classic.html) - These are classic POSG problems from the literature.
- [Grid-World](https://posggym.readthedocs.io/en/latest/environments/grid_world.html) - These environments are all based in a 2D Gridworld.
- [Continuous](https://posggym.readthedocs.io/en/latest/environments/continuous.html) - 2D environments with continuous state, actions, and observations.
## Environment API
POSGGym models each environment as a python `env` class. Creating environment instances and interacting with them is very simple, and flows almost identically to the Gymnasium user flow. Here's an example using the `PredatorPrey-v0` environment:
```python
import posggym
env = posggym.make("PredatorPrey-v0")
observations, infos = env.reset(seed=42)
for t in range(100):
env.render()
actions = {i: env.action_spaces[i].sample() for i in env.agents}
observations, rewards, terminations, truncations, all_done, infos = env.step(actions)
if all_done:
observations, infos = env.reset()
env.close()
```
## Model API
Every environment provides access to a model of the environment in the form of a python `model` class. Each model implements a generative model, which can be used for planning, along with functions for sampling initial states. Some environments also implement a full POSG model including the transition, joint observation and joint reward functions.
The following is an example of accessing and using the environment model:
```python
import posggym
env = posggym.make("PredatorPrey-v0")
model = env.model
model.seed(seed=42)
state = model.sample_initial_state()
observations = model.sample_initial_obs(state)
for t in range(100):
actions = {i: model.action_spaces[i].sample() for i in model.get_agents(state)}
state, observations, rewards, terminations, truncations, all_done, infos = model.step(state, actions)
if all_done:
state = model.sample_initial_state()
observations = model.sample_initial_obs(state)
```
The base model API is very similar to the environment API. The key difference that all methods are stateless so can be used repeatedly for planning. Indeed the `env` class for the built-in environments are mainly just a wrappers over the underlying `model` class that manage the state and add support for rendering.
Note that unlike for the `env` class, for convenience the output of the `model.step()` method is a `dataclass` instance and so it's components can be accessed as attributes. For example:
```python
timestep = model.step(state, actions)
observations = timestep.observations
infos = timestep.infos
```
Both the `env` and `model` classes support a number of other methods, please see the documentation [posggym.readthedocs.io/](https://posggym.readthedocs.io/) for more details.
## Agents API
POSGGym.Agents models each agent as a python `policy` class, which at it's simplest accepts an observation and returns the next action. Here's an example using one of the K-Level Reasoning policies in the `PursuitEvasion-v0` environment:
```python
import posggym
import posggym.agents as pga
env = posggym.make("PursuitEvasion-v0", grid="16x16")
policies = {
'0': pga.make("PursuitEvasion-v0/grid=16x16/klr_k1_seed0_i0-v0", env.model, '0'),
'1': pga.make("PursuitEvasion-v0/grid=16x16/klr_k1_seed0_i1-v0", env.model, '1')
}
obs, info = env.reset(seed=42)
for i, policy in policies.items():
policy.reset(seed=7)
for t in range(100):
actions = {i: policies[i].step(obs[i]) for i in env.agents}
obs, rewards, termination, truncated, all_done, info = env.step(actions)
if all_done:
obs, info = env.reset()
for i, policy in policies.items():
policy.reset()
env.close()
for policy in policies.values():
policy.close()
```
For a full explanation of the agent API please see the [POSGGym Agents Getting Started documentation](https://posggym.readthedocs.io/en/latest/agents/getting_started.html). A full list of implemented agents is also available in the documentation.
## Citation
You can cite POSGGym as:
```bibtex
@misc{schwartzPOSGGym2023,
title = {POSGGym},
urldate = {2023-08-08},
author = {Schwartz, Jonathon and Newbury, Rhys and Kurniawati, Hanna},
year = {2023},
}
```
Raw data
{
"_id": null,
"home_page": "",
"name": "posggym",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "multiagent-systems,reinforcement-learning,planning,gymnasium,POSG",
"author": "",
"author_email": "Jonathon Schwartz <jonathon.schwartz@anu.edu.au>",
"download_url": "https://files.pythonhosted.org/packages/ce/b3/6a5c7379f0951b1d2126dc9bc3b3abf7e2fa5df9266234713f0f87678223/posggym-0.5.1.tar.gz",
"platform": null,
"description": "[](https://opensource.org/licenses/MIT)\n[](https://github.com/psf/black)\n\n\n# POSGGym\n\nPOSGGym is an open source Python library providing implementations of Partially Observable Stochastic Game (POSG) environments coupled with dynamic models of each environment, all under a unified API. While there are a number of amazing open-source implementations for POSG environments, very few have support for dynamic models that can be used for planning, especially for continuous domains. The aim of this library is to fill this gap.\n\nA key goal of POSGGym is to provide easy to use and understand open-source implementations for many of the environments commonly used in the partially observable multi-agent planning literature.\n\nPOSGGym also provides a collection of reference agents for it's various environments and a unified Agents API for using these agents. These agents can be used to evaluate algorithms, and includes both hand-coded bots and agents trained using reinforcement learning.\n\nPOSGGym is directly inspired by and adapted from the [Gymnasium](https://gymnasium.farama.org/) and [PettingZoo](https://pettingzoo.farama.org/) libraries for reinforcement learning. The key additions in POSGGym are support for environment models which can be used for planning and reference agents. POSGGym's API aims to stay as close to the Gymnasium and PettingZoo Parallel APIs as possible while incorporating models and agents into the mix.\n\nAny POSGGym environment can be converted into a PettingZoo environment using a simple wrapper (`posggym.wrappers.petting_zoo.PettingZoo`). This allows for easy integration with the ecosystem of libraries that support PettingZoo.\n\n\n## Documentation\n\nThe documentation for the project is available at [posggym.readthedocs.io/](https://posggym.readthedocs.io/).\n\nFor a guide to building the documentation locally see [docs/README.md](docs/README.md).\n\n## Installation\n\nPOSGGym supports and tests for `Python>=3.8`. We recommend using a virtual environment to install POSGGym (e.g. [conda](https://docs.conda.io/projects/conda/en/latest/index.html), [venv](https://docs.python.org/3/library/venv.html)).\n\n### Using pip\n\nThe latest release version of POSGGym can be installed using `pip` by running:\n\n```bash\npip install posggym\n```\n\nThis will install the base dependencies for running the main environments and download the agent models (so may take a few minutes), but but in order to minimise the number of unused dependencies installed may not include all dependencies for all environments or for rendering some environments, and will not include dependencies for running many posggym agents.\n\nYou can install all dependencies for a family of environments like `pip install posggym[grid-world]` and `pip install posggym[continuous]` or dependencies for all environments using `pip install posggym[envs-all]`.\n\nYou can install dependencies for POSGGym agents using `pip install posggym[agents]` or to install dependencies for all environments and agents use `pip install posggym[all]`.\n\n### Installing from source\n\nTo install POSGGym from source, first clone the repository then run:\n\n```bash\ncd posggym\npip install -e .\n```\n\nThis will install the base dependencies and download the agent models (so may take a few minutes). You can optionally install extras as described above. E.g. to install all dependencies for all environments and agents use:\n\n```bash\npip install -e .[all]\n```\n\nTo run tests, install the test dependencies and then run the tests:\n\n```bash\npip install -e .[test]\npytest\n```\n\nOr alternatively you can run one of the examples from the `examples` directory:\n\n```bash\npython examples/run_random_agents.py --env_id Driving-v0 --num_episodes 10 --render_mode human\n```\n\n\n## Environments\n\nPOSGGym includes the following families of environments (for a full list of environments and their descriptsion see the [documentation](https://posggym.readthedocs.io/)).\n\n- [Classic](https://posggym.readthedocs.io/en/latest/environments/classic.html) - These are classic POSG problems from the literature.\n- [Grid-World](https://posggym.readthedocs.io/en/latest/environments/grid_world.html) - These environments are all based in a 2D Gridworld.\n- [Continuous](https://posggym.readthedocs.io/en/latest/environments/continuous.html) - 2D environments with continuous state, actions, and observations.\n\n\n## Environment API\n\nPOSGGym models each environment as a python `env` class. Creating environment instances and interacting with them is very simple, and flows almost identically to the Gymnasium user flow. Here's an example using the `PredatorPrey-v0` environment:\n\n```python\nimport posggym\nenv = posggym.make(\"PredatorPrey-v0\")\nobservations, infos = env.reset(seed=42)\n\nfor t in range(100):\n env.render()\n actions = {i: env.action_spaces[i].sample() for i in env.agents}\n observations, rewards, terminations, truncations, all_done, infos = env.step(actions)\n\n if all_done:\n observations, infos = env.reset()\n\nenv.close()\n```\n\n## Model API\n\nEvery environment provides access to a model of the environment in the form of a python `model` class. Each model implements a generative model, which can be used for planning, along with functions for sampling initial states. Some environments also implement a full POSG model including the transition, joint observation and joint reward functions.\n\nThe following is an example of accessing and using the environment model:\n\n\n```python\nimport posggym\nenv = posggym.make(\"PredatorPrey-v0\")\nmodel = env.model\nmodel.seed(seed=42)\n\nstate = model.sample_initial_state()\nobservations = model.sample_initial_obs(state)\n\nfor t in range(100):\n actions = {i: model.action_spaces[i].sample() for i in model.get_agents(state)}\n state, observations, rewards, terminations, truncations, all_done, infos = model.step(state, actions)\n\n if all_done:\n state = model.sample_initial_state()\n observations = model.sample_initial_obs(state)\n```\n\nThe base model API is very similar to the environment API. The key difference that all methods are stateless so can be used repeatedly for planning. Indeed the `env` class for the built-in environments are mainly just a wrappers over the underlying `model` class that manage the state and add support for rendering.\n\nNote that unlike for the `env` class, for convenience the output of the `model.step()` method is a `dataclass` instance and so it's components can be accessed as attributes. For example:\n\n```python\ntimestep = model.step(state, actions)\nobservations = timestep.observations\ninfos = timestep.infos\n```\n\nBoth the `env` and `model` classes support a number of other methods, please see the documentation [posggym.readthedocs.io/](https://posggym.readthedocs.io/) for more details.\n\n## Agents API\n\nPOSGGym.Agents models each agent as a python `policy` class, which at it's simplest accepts an observation and returns the next action. Here's an example using one of the K-Level Reasoning policies in the `PursuitEvasion-v0` environment:\n\n\n```python\nimport posggym\nimport posggym.agents as pga\nenv = posggym.make(\"PursuitEvasion-v0\", grid=\"16x16\")\n\npolicies = {\n '0': pga.make(\"PursuitEvasion-v0/grid=16x16/klr_k1_seed0_i0-v0\", env.model, '0'),\n '1': pga.make(\"PursuitEvasion-v0/grid=16x16/klr_k1_seed0_i1-v0\", env.model, '1')\n}\n\nobs, info = env.reset(seed=42)\nfor i, policy in policies.items():\n policy.reset(seed=7)\n\nfor t in range(100):\n actions = {i: policies[i].step(obs[i]) for i in env.agents}\n obs, rewards, termination, truncated, all_done, info = env.step(actions)\n\n if all_done:\n obs, info = env.reset()\n for i, policy in policies.items():\n policy.reset()\n\nenv.close()\nfor policy in policies.values():\n policy.close()\n```\n\nFor a full explanation of the agent API please see the [POSGGym Agents Getting Started documentation](https://posggym.readthedocs.io/en/latest/agents/getting_started.html). A full list of implemented agents is also available in the documentation.\n\n## Citation\n\nYou can cite POSGGym as:\n\n```bibtex\n@misc{schwartzPOSGGym2023,\n title = {POSGGym},\n urldate = {2023-08-08},\n author = {Schwartz, Jonathon and Newbury, Rhys and Kurniawati, Hanna},\n year = {2023},\n}\n```\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "A collection of diverse environments and reference agents for reinforcement learning and planning research in Partially Observable Stochastic Games (POSGs). .",
"version": "0.5.1",
"project_urls": {
"Bug Report": "https://github.com/RDLLab/posggym/issues",
"Documentation": "https://posggym.readthedocs.io/",
"Homepage": "https://github.com/RDLLab/posggym",
"Repository": "https://github.com/RDLLab/posggym/"
},
"split_keywords": [
"multiagent-systems",
"reinforcement-learning",
"planning",
"gymnasium",
"posg"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ceb36a5c7379f0951b1d2126dc9bc3b3abf7e2fa5df9266234713f0f87678223",
"md5": "11ae391be254229d64d8025713833147",
"sha256": "d66b14600787ba7387d4b3b72ec8129b6b21ce087b3be2ac00c15c86517c6102"
},
"downloads": -1,
"filename": "posggym-0.5.1.tar.gz",
"has_sig": false,
"md5_digest": "11ae391be254229d64d8025713833147",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 314922,
"upload_time": "2023-11-22T22:33:03",
"upload_time_iso_8601": "2023-11-22T22:33:03.231604Z",
"url": "https://files.pythonhosted.org/packages/ce/b3/6a5c7379f0951b1d2126dc9bc3b3abf7e2fa5df9266234713f0f87678223/posggym-0.5.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-22 22:33:03",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "RDLLab",
"github_project": "posggym",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "posggym"
}
