# Gymnasium Search Race
[](https://github.com/Quentin18/gymnasium-search-race/actions/workflows/build.yml)
[](https://badge.fury.io/py/gymnasium-search-race)
[](https://badge.fury.io/py/gymnasium-search-race)
[](https://pre-commit.com/)
[](https://github.com/psf/black)
[](https://pycqa.github.io/isort/)
Gymnasium environments for
the [Search Race CodinGame optimization puzzle](https://www.codingame.com/multiplayer/optimization/search-race)
and [Mad Pod Racing CodinGame bot programming game](https://www.codingame.com/multiplayer/bot-programming/mad-pod-racing).
https://github.com/user-attachments/assets/1862b04b-9e33-4f55-a309-ad665a1db2f1
<table>
<tbody>
<tr>
<td>Action Space</td>
<td><code>Box([-1, 0], [1, 1], float64)</code></td>
</tr>
<tr>
<td>Observation Space</td>
<td><code>Box([0, 0, 0, 0, 0, 0, 0, -1, -1, 0], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1], float64)</code></td>
</tr>
<tr>
<td>import</td>
<td><code>gymnasium.make("gymnasium_search_race:gymnasium_search_race/SearchRace-v1")</code></td>
</tr>
</tbody>
</table>
## Installation
To install `gymnasium-search-race` with pip, execute:
```bash
pip install gymnasium_search_race
```
From source:
```bash
git clone https://github.com/Quentin18/gymnasium-search-race
cd gymnasium-search-race/
pip install -e .
```
## Environment
### Action Space
The action is a `ndarray` with 2 continuous variables:
- The rotation angle between -18 and 18 degrees, normalized between -1 and 1.
- The thrust between 0 and 200, normalized between 0 and 1.
### Observation Space
The observation is a `ndarray` of 10 continuous variables:
- 1 if the next checkpoint is the last one, 0 otherwise.
- The x and y coordinates of the next checkpoint.
- The x and y coordinates of the checkpoint after next checkpoint.
- The x and y coordinates of the car.
- The horizontal speed vx and vertical speed vy of the car.
- The facing angle of the car.
The values are normalized between 0 and 1, or -1 and 1 if negative values are allowed.
### Reward
The goal is to visit all checkpoints as quickly as possible, as such the agent is penalised with a reward of `-0.1` for
each timestep.
When a checkpoint is visited, the agent is awarded with a reward of `1000/total_checkpoints`.
### Starting State
The starting state is generated by choosing a random CodinGame test case.
### Episode End
The episode ends if either of the following happens:
1. Termination: The car visit all checkpoints before the time is out.
2. Truncation: Episode length is greater than 600.
### Arguments
- `test_id`: test case id to generate the checkpoints (see
choices [here](https://github.com/Quentin18/gymnasium-search-race/tree/main/src/gymnasium_search_race/envs/maps)). The
default value is `None` which selects a test case randomly when the `reset` method is called.
```python
import gymnasium as gym
gym.make("gymnasium_search_race:gymnasium_search_race/SearchRace-v1", test_id=1)
```
### Version History
- v1: Add boolean to indicate if the next checkpoint is the last checkpoint in observation
- v0: Initial version
## Discrete environment
The `SearchRaceDiscrete` environment is similar to the `SearchRace` environment except the action space is discrete.
```python
import gymnasium as gym
gym.make("gymnasium_search_race:gymnasium_search_race/SearchRaceDiscrete-v1", test_id=1)
```
### Action Space
There are 74 discrete actions corresponding to the combinations of angles from -18 to 18 degrees and thrust 0 and 200.
### Version History
- v1: Add all angles in action space
- v0: Initial version
## Mad Pod Racing
### Runner
The `MadPodRacing` and `MadPodRacingDiscrete` environments can be used to train a runner for
the [Mad Pod Racing CodinGame bot programming game](https://www.codingame.com/multiplayer/bot-programming/mad-pod-racing).
They are similar to the `SearchRace` and `SearchRaceDiscrete` environments except the following differences:
- The maximum thrust value is 100 instead of 200.
- The maps are generated the same way Codingame generates them.
- The car position is rounded and not truncated.
```python
import gymnasium as gym
gym.make("gymnasium_search_race:gymnasium_search_race/MadPodRacing-v0")
gym.make("gymnasium_search_race:gymnasium_search_race/MadPodRacingDiscrete-v0")
```
https://github.com/user-attachments/assets/ce4b1837-4591-40dd-a203-9eec9146b94b
### Blocker
The `MadPodRacingBlocker` environment can be used to train a blocker for
the [Mad Pod Racing CodinGame bot programming game](https://www.codingame.com/multiplayer/bot-programming/mad-pod-racing).
```python
import gymnasium as gym
gym.make("gymnasium_search_race:gymnasium_search_race/MadPodRacingBlocker-v0")
```
https://github.com/user-attachments/assets/57387372-823f-44a2-9a03-23a9332752ab
## Usage
You can use [RL Baselines3 Zoo](https://github.com/DLR-RM/rl-baselines3-zoo) to train and evaluate agents:
```bash
pip install rl_zoo3
```
### Train an Agent
The hyperparameters are defined in `hyperparams/ppo.yml`.
To train a PPO agent for the Search Race game, execute:
```bash
python -m rl_zoo3.train \
--algo ppo \
--env gymnasium_search_race/SearchRace-v1 \
--tensorboard-log logs \
--eval-freq 20000 \
--eval-episodes 10 \
--gym-packages gymnasium_search_race \
--conf-file hyperparams/ppo.yml \
--progress
```
For the Mad Pod Racing game, you can add an opponent with the `opponent_path` argument:
```bash
python -m rl_zoo3.train \
--algo ppo \
--env gymnasium_search_race/MadPodRacingBlocker-v0 \
--tensorboard-log logs \
--eval-freq 20000 \
--eval-episodes 10 \
--gym-packages gymnasium_search_race \
--env-kwargs "opponent_path:'rl-trained-agents/ppo/gymnasium_search_race-MadPodRacing-v0_1/best_model.zip'" \
--conf-file hyperparams/ppo.yml \
--progress
```
### Enjoy a Trained Agent
To see a trained agent in action on random test cases, execute:
```bash
python -m rl_zoo3.enjoy \
--algo ppo \
--env gymnasium_search_race/SearchRace-v1 \
--n-timesteps 1000 \
--deterministic \
--gym-packages gymnasium_search_race \
--load-best \
--progress
```
### Run Test Cases
To run test cases with a trained agent, execute:
```bash
python -m scripts.run_test_cases \
--path rl-trained-agents/ppo/gymnasium_search_race-SearchRace-v1_1/best_model.zip \
--env gymnasium_search_race:gymnasium_search_race/SearchRace-v1 \
--record-video \
--record-metrics
```
### Record a Video of a Trained Agent
To record a video of a trained agent on Mad Pod Racing, execute:
```bash
python -m scripts.record_video \
--path rl-trained-agents/ppo/gymnasium_search_race-MadPodRacing-v0_1/best_model.zip \
--env gymnasium_search_race:gymnasium_search_race/MadPodRacing-v0
```
For Mad Pod Racing Blocker, execute:
```bash
python -m scripts.record_video \
--path rl-trained-agents/ppo/gymnasium_search_race-MadPodRacingBlocker-v0_1/best_model.zip \
--opponent-path rl-trained-agents/ppo/gymnasium_search_race-MadPodRacing-v0_1/best_model.zip \
--env gymnasium_search_race:gymnasium_search_race/MadPodRacingBlocker-v0
```
## Tests
To run tests, execute:
```bash
pytest
```
## Citing
To cite the repository in publications:
```bibtex
@misc{gymnasium-search-race,
author = {Quentin Deschamps},
title = {Gymnasium Search Race},
year = {2024},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/Quentin18/gymnasium-search-race}},
}
```
## References
- [Gymnasium](https://github.com/Farama-Foundation/Gymnasium)
- [RL Baselines3 Zoo](https://github.com/DLR-RM/rl-baselines3-zoo)
- [Stable Baselines3](https://github.com/DLR-RM/stable-baselines3)
- [CGSearchRace](https://github.com/Illedan/CGSearchRace)
- [CSB-Runner-Arena](https://github.com/Agade09/CSB-Runner-Arena)
- [Coders Strikes Back by Magus](http://files.magusgeek.com/csb/csb_en.html)
### Assets
- https://www.flaticon.com/free-icon/space-ship_751036
- https://www.flaticon.com/free-icon/space-ship_784925
## Author
[Quentin Deschamps](mailto:quentindeschamps18@gmail.com)
Raw data
{
"_id": null,
"home_page": null,
"name": "gymnasium-search-race",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "Reinforcement Learning, game, RL, AI, gymnasium, pygame",
"author": null,
"author_email": "Quentin Deschamps <quentindeschamps18@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/bc/78/6ebf8af2267754a1ff94684c55830da8b6655d5292a90a4b7db78b6f21f5/gymnasium_search_race-3.0.0.tar.gz",
"platform": null,
"description": "# Gymnasium Search Race\n\n[](https://github.com/Quentin18/gymnasium-search-race/actions/workflows/build.yml)\n[](https://badge.fury.io/py/gymnasium-search-race)\n[](https://badge.fury.io/py/gymnasium-search-race)\n[](https://pre-commit.com/)\n[](https://github.com/psf/black)\n[](https://pycqa.github.io/isort/)\n\nGymnasium environments for\nthe [Search Race CodinGame optimization puzzle](https://www.codingame.com/multiplayer/optimization/search-race)\nand [Mad Pod Racing CodinGame bot programming game](https://www.codingame.com/multiplayer/bot-programming/mad-pod-racing).\n\nhttps://github.com/user-attachments/assets/1862b04b-9e33-4f55-a309-ad665a1db2f1\n\n<table>\n <tbody>\n <tr>\n <td>Action Space</td>\n <td><code>Box([-1, 0], [1, 1], float64)</code></td>\n </tr>\n <tr>\n <td>Observation Space</td>\n <td><code>Box([0, 0, 0, 0, 0, 0, 0, -1, -1, 0], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1], float64)</code></td>\n </tr>\n <tr>\n <td>import</td>\n <td><code>gymnasium.make(\"gymnasium_search_race:gymnasium_search_race/SearchRace-v1\")</code></td>\n </tr>\n </tbody>\n</table>\n\n## Installation\n\nTo install `gymnasium-search-race` with pip, execute:\n\n```bash\npip install gymnasium_search_race\n```\n\nFrom source:\n\n```bash\ngit clone https://github.com/Quentin18/gymnasium-search-race\ncd gymnasium-search-race/\npip install -e .\n```\n\n## Environment\n\n### Action Space\n\nThe action is a `ndarray` with 2 continuous variables:\n\n- The rotation angle between -18 and 18 degrees, normalized between -1 and 1.\n- The thrust between 0 and 200, normalized between 0 and 1.\n\n### Observation Space\n\nThe observation is a `ndarray` of 10 continuous variables:\n\n- 1 if the next checkpoint is the last one, 0 otherwise.\n- The x and y coordinates of the next checkpoint.\n- The x and y coordinates of the checkpoint after next checkpoint.\n- The x and y coordinates of the car.\n- The horizontal speed vx and vertical speed vy of the car.\n- The facing angle of the car.\n\nThe values are normalized between 0 and 1, or -1 and 1 if negative values are allowed.\n\n### Reward\n\nThe goal is to visit all checkpoints as quickly as possible, as such the agent is penalised with a reward of `-0.1` for\neach timestep.\nWhen a checkpoint is visited, the agent is awarded with a reward of `1000/total_checkpoints`.\n\n### Starting State\n\nThe starting state is generated by choosing a random CodinGame test case.\n\n### Episode End\n\nThe episode ends if either of the following happens:\n\n1. Termination: The car visit all checkpoints before the time is out.\n2. Truncation: Episode length is greater than 600.\n\n### Arguments\n\n- `test_id`: test case id to generate the checkpoints (see\n choices [here](https://github.com/Quentin18/gymnasium-search-race/tree/main/src/gymnasium_search_race/envs/maps)). The\n default value is `None` which selects a test case randomly when the `reset` method is called.\n\n```python\nimport gymnasium as gym\n\ngym.make(\"gymnasium_search_race:gymnasium_search_race/SearchRace-v1\", test_id=1)\n```\n\n### Version History\n\n- v1: Add boolean to indicate if the next checkpoint is the last checkpoint in observation\n- v0: Initial version\n\n## Discrete environment\n\nThe `SearchRaceDiscrete` environment is similar to the `SearchRace` environment except the action space is discrete.\n\n```python\nimport gymnasium as gym\n\ngym.make(\"gymnasium_search_race:gymnasium_search_race/SearchRaceDiscrete-v1\", test_id=1)\n```\n\n### Action Space\n\nThere are 74 discrete actions corresponding to the combinations of angles from -18 to 18 degrees and thrust 0 and 200.\n\n### Version History\n\n- v1: Add all angles in action space\n- v0: Initial version\n\n## Mad Pod Racing\n\n### Runner\n\nThe `MadPodRacing` and `MadPodRacingDiscrete` environments can be used to train a runner for\nthe [Mad Pod Racing CodinGame bot programming game](https://www.codingame.com/multiplayer/bot-programming/mad-pod-racing).\nThey are similar to the `SearchRace` and `SearchRaceDiscrete` environments except the following differences:\n\n- The maximum thrust value is 100 instead of 200.\n- The maps are generated the same way Codingame generates them.\n- The car position is rounded and not truncated.\n\n```python\nimport gymnasium as gym\n\ngym.make(\"gymnasium_search_race:gymnasium_search_race/MadPodRacing-v0\")\ngym.make(\"gymnasium_search_race:gymnasium_search_race/MadPodRacingDiscrete-v0\")\n```\n\nhttps://github.com/user-attachments/assets/ce4b1837-4591-40dd-a203-9eec9146b94b\n\n### Blocker\n\nThe `MadPodRacingBlocker` environment can be used to train a blocker for\nthe [Mad Pod Racing CodinGame bot programming game](https://www.codingame.com/multiplayer/bot-programming/mad-pod-racing).\n\n```python\nimport gymnasium as gym\n\ngym.make(\"gymnasium_search_race:gymnasium_search_race/MadPodRacingBlocker-v0\")\n```\n\nhttps://github.com/user-attachments/assets/57387372-823f-44a2-9a03-23a9332752ab\n\n## Usage\n\nYou can use [RL Baselines3 Zoo](https://github.com/DLR-RM/rl-baselines3-zoo) to train and evaluate agents:\n\n```bash\npip install rl_zoo3\n```\n\n### Train an Agent\n\nThe hyperparameters are defined in `hyperparams/ppo.yml`.\n\nTo train a PPO agent for the Search Race game, execute:\n\n```bash\npython -m rl_zoo3.train \\\n --algo ppo \\\n --env gymnasium_search_race/SearchRace-v1 \\\n --tensorboard-log logs \\\n --eval-freq 20000 \\\n --eval-episodes 10 \\\n --gym-packages gymnasium_search_race \\\n --conf-file hyperparams/ppo.yml \\\n --progress\n```\n\nFor the Mad Pod Racing game, you can add an opponent with the `opponent_path` argument:\n\n```bash\npython -m rl_zoo3.train \\\n --algo ppo \\\n --env gymnasium_search_race/MadPodRacingBlocker-v0 \\\n --tensorboard-log logs \\\n --eval-freq 20000 \\\n --eval-episodes 10 \\\n --gym-packages gymnasium_search_race \\\n --env-kwargs \"opponent_path:'rl-trained-agents/ppo/gymnasium_search_race-MadPodRacing-v0_1/best_model.zip'\" \\\n --conf-file hyperparams/ppo.yml \\\n --progress\n```\n\n### Enjoy a Trained Agent\n\nTo see a trained agent in action on random test cases, execute:\n\n```bash\npython -m rl_zoo3.enjoy \\\n --algo ppo \\\n --env gymnasium_search_race/SearchRace-v1 \\\n --n-timesteps 1000 \\\n --deterministic \\\n --gym-packages gymnasium_search_race \\\n --load-best \\\n --progress\n```\n\n### Run Test Cases\n\nTo run test cases with a trained agent, execute:\n\n```bash\npython -m scripts.run_test_cases \\\n --path rl-trained-agents/ppo/gymnasium_search_race-SearchRace-v1_1/best_model.zip \\\n --env gymnasium_search_race:gymnasium_search_race/SearchRace-v1 \\\n --record-video \\\n --record-metrics\n```\n\n### Record a Video of a Trained Agent\n\nTo record a video of a trained agent on Mad Pod Racing, execute:\n\n```bash\npython -m scripts.record_video \\\n --path rl-trained-agents/ppo/gymnasium_search_race-MadPodRacing-v0_1/best_model.zip \\\n --env gymnasium_search_race:gymnasium_search_race/MadPodRacing-v0\n```\n\nFor Mad Pod Racing Blocker, execute:\n\n```bash\npython -m scripts.record_video \\\n --path rl-trained-agents/ppo/gymnasium_search_race-MadPodRacingBlocker-v0_1/best_model.zip \\\n --opponent-path rl-trained-agents/ppo/gymnasium_search_race-MadPodRacing-v0_1/best_model.zip \\\n --env gymnasium_search_race:gymnasium_search_race/MadPodRacingBlocker-v0\n```\n\n## Tests\n\nTo run tests, execute:\n\n```bash\npytest\n```\n\n## Citing\n\nTo cite the repository in publications:\n\n```bibtex\n@misc{gymnasium-search-race,\n author = {Quentin Deschamps},\n title = {Gymnasium Search Race},\n year = {2024},\n publisher = {GitHub},\n journal = {GitHub repository},\n howpublished = {\\url{https://github.com/Quentin18/gymnasium-search-race}},\n}\n```\n\n## References\n\n- [Gymnasium](https://github.com/Farama-Foundation/Gymnasium)\n- [RL Baselines3 Zoo](https://github.com/DLR-RM/rl-baselines3-zoo)\n- [Stable Baselines3](https://github.com/DLR-RM/stable-baselines3)\n- [CGSearchRace](https://github.com/Illedan/CGSearchRace)\n- [CSB-Runner-Arena](https://github.com/Agade09/CSB-Runner-Arena)\n- [Coders Strikes Back by Magus](http://files.magusgeek.com/csb/csb_en.html)\n\n### Assets\n\n- https://www.flaticon.com/free-icon/space-ship_751036\n- https://www.flaticon.com/free-icon/space-ship_784925\n\n## Author\n\n[Quentin Deschamps](mailto:quentindeschamps18@gmail.com)\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A reinforcement learning environment for the Search Race CG puzzle based on Gymnasium",
"version": "3.0.0",
"project_urls": {
"Repository": "https://github.com/Quentin18/gymnasium-search-race"
},
"split_keywords": [
"reinforcement learning",
" game",
" rl",
" ai",
" gymnasium",
" pygame"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a3befeb1f39597e837c8be3e92588d35c872b63d7dd6917827f5874cd1898dcf",
"md5": "67ad9c31af2ffd00134746e91bca49ca",
"sha256": "2c73fedc8bf5190288a75a289b5fb0b576e0c1aac2e89e59640935070c372daa"
},
"downloads": -1,
"filename": "gymnasium_search_race-3.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "67ad9c31af2ffd00134746e91bca49ca",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 420801,
"upload_time": "2024-11-15T23:06:56",
"upload_time_iso_8601": "2024-11-15T23:06:56.944275Z",
"url": "https://files.pythonhosted.org/packages/a3/be/feb1f39597e837c8be3e92588d35c872b63d7dd6917827f5874cd1898dcf/gymnasium_search_race-3.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "bc786ebf8af2267754a1ff94684c55830da8b6655d5292a90a4b7db78b6f21f5",
"md5": "2706ab508e2afa04ea853e4e6a262e0b",
"sha256": "11ce215a10c908119e2cc2e4634e2c3403df2c8b9a86ffe6260019ce5c4338e2"
},
"downloads": -1,
"filename": "gymnasium_search_race-3.0.0.tar.gz",
"has_sig": false,
"md5_digest": "2706ab508e2afa04ea853e4e6a262e0b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 2862533,
"upload_time": "2024-11-15T23:06:59",
"upload_time_iso_8601": "2024-11-15T23:06:59.390740Z",
"url": "https://files.pythonhosted.org/packages/bc/78/6ebf8af2267754a1ff94684c55830da8b6655d5292a90a4b7db78b6f21f5/gymnasium_search_race-3.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-15 23:06:59",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Quentin18",
"github_project": "gymnasium-search-race",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "gymnasium-search-race"
}
JSON
