# POSGGym Agents
POSGGym-Agents is a collection of agent policies and policy training code for [POSGGym](https://github.com/RDLLab/posggym) environments. The goal of the library is to provide a diverse set of policies and a simple API that makes it easy to import and use policies in your own research.
## Installation
This project depends on the [PyTorch](https://pytorch.org/) and [Ray RLlib](https://docs.ray.io/en/latest/rllib/index.html) libraries. Specifically pytorch version `>= 1.11` and rllib version `>=2.3`. We recommend install `torch` before installing the POSGGym-Agents package to ensure you get the version of `torch` that works with your CPU/GPU.
You can install the latest version of POSGGym-Agents using `pip install posggym-agents`. Note that this will install the base library without all the agent policy files, which will be downloaded from the posggym-agents github repo upon request (i.e. when `posggym_agents.make` is called), the policy file is then stored locally with the posggym-agents code for reuse.
## API usage
POSGGym-Agents models each agent as a python `policy` class, which at it's simplest accepts an observation and returns the next action. Here's an example using one of the K-Level Reasoning policies in the `PursuitEvasion-v0` environment:
```python
import posggym
import posggym_agents as pga
env = posggym.make("PursuitEvasion-v0", grid="16x16")
policies = {
'0': pga.make("PursuitEvasion-v0/grid=16x16/klr_k1_seed0_i0-v0", env.model, '0'),
'1': pga.make("PursuitEvasion-v0/grid=16x16/klr_k1_seed0_i1-v0", env.model, '1')
}
obs, info = env.reset(seed=42)
for i, policy in policies.items():
policy.reset(seed=7)
for t in range(100):
actions = {i: policies[i].step(obs[i]) for i in env.agents}
obs, rewards, termination, truncated, all_done, info = env.step(actions)
if all_done:
obs, info = env.reset()
for i, policy in agents.items():
policy.reset()
env.close()
for policy in policies.values():
policy.close()
```
In the above code we initialize two of the implemented policies for the `PursuitEvasion-v0` environment by calling the `posggym_agents.make` function and passing in the full policy ID of each policy, the `posggym.Env` environmnent model and the agent ID of the agent the policy will be used for in the environment (this ensures it uses the correct environment properties such as action and observation space).
The policy ID is made up of four parts:
1. `env_id` - the ID of the environment the policy is for: `PursuitEvasion-v0`
2. `env_args_id` - a string representation of the environment arguments used in the version of the environment the policy is for: `grid=16x16`
3. `policy_name` - the name of the policy: `klr_k1_seed0_i0` and `klr_k1_seed0_i1`
4. `version` - the version of the policy: `v0`
The `env_id` and `env_args_id` may be omitted depending on the policy. If the policy is environment agnostic (e.g. the `Random-v0` policy works for any environment) then both the `env_id` and `env_args_id` can be omitted. While if the policy is environment specific, but works for all variations of the environment or the environment has only a single variation (it doesn't have any parameters) then the `env_args_id` can be omitted (e.g. `PursuitEvasion-v0/shortestpath-v0`).
## List of Policies
The project currently has policies implemented for the following POSGGym environments:
- Driving
- Level Based Foraging
- Predator Prey
- Pursuit Evasion
The full list of policies can be obtained using the following code:
```python
import posggym_agents as pga
pga.pprint_registry()
# will display all policies organized by `env_id/env_args_id/`
```
## Authors
**Jonathon Schwartz** - Jonathon.schwartz@anu.edu.au
## License
`MIT` © 2022, Jonathon Schwartz
Raw data
{
"_id": null,
"home_page": "",
"name": "posggym-agents",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "reinforcement learning,games,RL,AI,planning,posg",
"author": "",
"author_email": "Jonathon Schwartz <jonathon.schwartz@anu.edu.au>",
"download_url": "https://files.pythonhosted.org/packages/18/23/fb4f70ced80ba0232704b3a4266284dc0e71fc210dfee2bd33dbfc322ac3/posggym-agents-0.3.1.tar.gz",
"platform": null,
"description": "# POSGGym Agents\n\nPOSGGym-Agents is a collection of agent policies and policy training code for [POSGGym](https://github.com/RDLLab/posggym) environments. The goal of the library is to provide a diverse set of policies and a simple API that makes it easy to import and use policies in your own research.\n\n## Installation\n\nThis project depends on the [PyTorch](https://pytorch.org/) and [Ray RLlib](https://docs.ray.io/en/latest/rllib/index.html) libraries. Specifically pytorch version `>= 1.11` and rllib version `>=2.3`. We recommend install `torch` before installing the POSGGym-Agents package to ensure you get the version of `torch` that works with your CPU/GPU.\n\nYou can install the latest version of POSGGym-Agents using `pip install posggym-agents`. Note that this will install the base library without all the agent policy files, which will be downloaded from the posggym-agents github repo upon request (i.e. when `posggym_agents.make` is called), the policy file is then stored locally with the posggym-agents code for reuse.\n\n## API usage\n\nPOSGGym-Agents models each agent as a python `policy` class, which at it's simplest accepts an observation and returns the next action. Here's an example using one of the K-Level Reasoning policies in the `PursuitEvasion-v0` environment:\n\n\n```python\nimport posggym\nimport posggym_agents as pga\nenv = posggym.make(\"PursuitEvasion-v0\", grid=\"16x16\")\n\npolicies = {\n '0': pga.make(\"PursuitEvasion-v0/grid=16x16/klr_k1_seed0_i0-v0\", env.model, '0'),\n '1': pga.make(\"PursuitEvasion-v0/grid=16x16/klr_k1_seed0_i1-v0\", env.model, '1')\n}\n\nobs, info = env.reset(seed=42)\nfor i, policy in policies.items():\n policy.reset(seed=7)\n\nfor t in range(100):\n actions = {i: policies[i].step(obs[i]) for i in env.agents}\n obs, rewards, termination, truncated, all_done, info = env.step(actions)\n\n if all_done:\n obs, info = env.reset()\n for i, policy in agents.items():\n policy.reset()\n\nenv.close()\nfor policy in policies.values():\n policy.close()\n```\n\nIn the above code we initialize two of the implemented policies for the `PursuitEvasion-v0` environment by calling the `posggym_agents.make` function and passing in the full policy ID of each policy, the `posggym.Env` environmnent model and the agent ID of the agent the policy will be used for in the environment (this ensures it uses the correct environment properties such as action and observation space).\n\nThe policy ID is made up of four parts:\n\n1. `env_id` - the ID of the environment the policy is for: `PursuitEvasion-v0`\n2. `env_args_id` - a string representation of the environment arguments used in the version of the environment the policy is for: `grid=16x16`\n3. `policy_name` - the name of the policy: `klr_k1_seed0_i0` and `klr_k1_seed0_i1`\n4. `version` - the version of the policy: `v0`\n\nThe `env_id` and `env_args_id` may be omitted depending on the policy. If the policy is environment agnostic (e.g. the `Random-v0` policy works for any environment) then both the `env_id` and `env_args_id` can be omitted. While if the policy is environment specific, but works for all variations of the environment or the environment has only a single variation (it doesn't have any parameters) then the `env_args_id` can be omitted (e.g. `PursuitEvasion-v0/shortestpath-v0`).\n\n\n## List of Policies\n\nThe project currently has policies implemented for the following POSGGym environments:\n\n- Driving\n- Level Based Foraging\n- Predator Prey\n- Pursuit Evasion\n\nThe full list of policies can be obtained using the following code:\n\n```python\nimport posggym_agents as pga\npga.pprint_registry()\n# will display all policies organized by `env_id/env_args_id/`\n```\n\n\n## Authors\n\n**Jonathon Schwartz** - Jonathon.schwartz@anu.edu.au\n\n\n## License\n\n`MIT` \u00a9 2022, Jonathon Schwartz\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "A collection of agents and agent training code for posggym.",
"version": "0.3.1",
"split_keywords": [
"reinforcement learning",
"games",
"rl",
"ai",
"planning",
"posg"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "bc2d59f4d67704621c0ed5694f1c117a6af509b7bbe41a95bd164e1d4885e683",
"md5": "647f277c2297059afc47297bb5a4ed63",
"sha256": "bccc0f39d6e4909644fd0d91c06d632da9d9ebded521f7ace4e820114139fba8"
},
"downloads": -1,
"filename": "posggym_agents-0.3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "647f277c2297059afc47297bb5a4ed63",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 67298,
"upload_time": "2023-03-13T14:48:58",
"upload_time_iso_8601": "2023-03-13T14:48:58.752871Z",
"url": "https://files.pythonhosted.org/packages/bc/2d/59f4d67704621c0ed5694f1c117a6af509b7bbe41a95bd164e1d4885e683/posggym_agents-0.3.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "1823fb4f70ced80ba0232704b3a4266284dc0e71fc210dfee2bd33dbfc322ac3",
"md5": "1eccab67807311328c2c36cf3a233ffe",
"sha256": "2f57ab6833ff7aae1008a723b38b511e671c239414aea47dab818a260262cbcf"
},
"downloads": -1,
"filename": "posggym-agents-0.3.1.tar.gz",
"has_sig": false,
"md5_digest": "1eccab67807311328c2c36cf3a233ffe",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 50107,
"upload_time": "2023-03-13T14:49:00",
"upload_time_iso_8601": "2023-03-13T14:49:00.347324Z",
"url": "https://files.pythonhosted.org/packages/18/23/fb4f70ced80ba0232704b3a4266284dc0e71fc210dfee2bd33dbfc322ac3/posggym-agents-0.3.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-03-13 14:49:00",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "posggym-agents"
}
JSON