gymnasium-search-race


Namegymnasium-search-race JSON
Version 3.0.1 PyPI version JSON
download
home_pageNone
SummaryA reinforcement learning environment for the Search Race CG puzzle based on Gymnasium
upload_time2024-12-01 10:13:00
maintainerNone
docs_urlNone
authorNone
requires_python>=3.10
licenseMIT
keywords reinforcement learning game rl ai gymnasium pygame
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # Gymnasium Search Race

[![Build Python Package](https://github.com/Quentin18/gymnasium-search-race/actions/workflows/build.yml/badge.svg)](https://github.com/Quentin18/gymnasium-search-race/actions/workflows/build.yml)
[![Python](https://img.shields.io/pypi/pyversions/gymnasium-search-race.svg)](https://badge.fury.io/py/gymnasium-search-race)
[![PyPI](https://badge.fury.io/py/gymnasium-search-race.svg)](https://badge.fury.io/py/gymnasium-search-race)
[![PyPI Downloads](https://static.pepy.tech/badge/gymnasium-search-race)](https://pepy.tech/projects/gymnasium-search-race)
[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://pre-commit.com/)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
[![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat&labelColor=ef8336)](https://pycqa.github.io/isort/)

Gymnasium environments for
the [Search Race CodinGame optimization puzzle](https://www.codingame.com/multiplayer/optimization/search-race)
and [Mad Pod Racing CodinGame bot programming game](https://www.codingame.com/multiplayer/bot-programming/mad-pod-racing).

https://github.com/user-attachments/assets/1862b04b-9e33-4f55-a309-ad665a1db2f1

<table>
    <tbody>
        <tr>
            <td>Action Space</td>
            <td><code>Box([-1, 0], [1, 1], float64)</code></td>
        </tr>
        <tr>
            <td>Observation Space</td>
            <td><code>Box([0, 0, 0, 0, 0, 0, 0, -1, -1, 0], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1], float64)</code></td>
        </tr>
        <tr>
            <td>import</td>
            <td><code>gymnasium.make("gymnasium_search_race:gymnasium_search_race/SearchRace-v1")</code></td>
        </tr>
    </tbody>
</table>

## Installation

To install `gymnasium-search-race` with pip, execute:

```bash
pip install gymnasium_search_race
```

From source:

```bash
git clone https://github.com/Quentin18/gymnasium-search-race
cd gymnasium-search-race/
pip install -e .
```

## Environment

### Action Space

The action is a `ndarray` with 2 continuous variables:

- The rotation angle between -18 and 18 degrees, normalized between -1 and 1.
- The thrust between 0 and 200, normalized between 0 and 1.

### Observation Space

The observation is a `ndarray` of 10 continuous variables:

- 1 if the next checkpoint is the last one, 0 otherwise.
- The x and y coordinates of the next checkpoint.
- The x and y coordinates of the checkpoint after next checkpoint.
- The x and y coordinates of the car.
- The horizontal speed vx and vertical speed vy of the car.
- The facing angle of the car.

The values are normalized between 0 and 1, or -1 and 1 if negative values are allowed.

### Reward

The goal is to visit all checkpoints as quickly as possible, as such the agent is penalised with a reward of `-0.1` for
each timestep.
When a checkpoint is visited, the agent is awarded with a reward of `1000/total_checkpoints`.

### Starting State

The starting state is generated by choosing a random CodinGame test case.

### Episode End

The episode ends if either of the following happens:

1. Termination: The car visit all checkpoints before the time is out.
2. Truncation: Episode length is greater than 600.

### Arguments

- `test_id`: test case id to generate the checkpoints (see
  choices [here](https://github.com/Quentin18/gymnasium-search-race/tree/main/src/gymnasium_search_race/envs/maps)). The
  default value is `None` which selects a test case randomly when the `reset` method is called.

```python
import gymnasium as gym

gym.make("gymnasium_search_race:gymnasium_search_race/SearchRace-v1", test_id=1)
```

### Version History

- v1: Add boolean to indicate if the next checkpoint is the last checkpoint in observation
- v0: Initial version

## Discrete environment

The `SearchRaceDiscrete` environment is similar to the `SearchRace` environment except the action space is discrete.

```python
import gymnasium as gym

gym.make("gymnasium_search_race:gymnasium_search_race/SearchRaceDiscrete-v1", test_id=1)
```

### Action Space

There are 74 discrete actions corresponding to the combinations of angles from -18 to 18 degrees and thrust 0 and 200.

### Version History

- v1: Add all angles in action space
- v0: Initial version

## Mad Pod Racing

### Runner

The `MadPodRacing` and `MadPodRacingDiscrete` environments can be used to train a runner for
the [Mad Pod Racing CodinGame bot programming game](https://www.codingame.com/multiplayer/bot-programming/mad-pod-racing).
They are similar to the `SearchRace` and `SearchRaceDiscrete` environments except the following differences:

- The maximum thrust value is 100 instead of 200.
- The maps are generated the same way Codingame generates them.
- The car position is rounded and not truncated.

```python
import gymnasium as gym

gym.make("gymnasium_search_race:gymnasium_search_race/MadPodRacing-v0")
gym.make("gymnasium_search_race:gymnasium_search_race/MadPodRacingDiscrete-v0")
```

https://github.com/user-attachments/assets/ce4b1837-4591-40dd-a203-9eec9146b94b

### Blocker

The `MadPodRacingBlocker` environment can be used to train a blocker for
the [Mad Pod Racing CodinGame bot programming game](https://www.codingame.com/multiplayer/bot-programming/mad-pod-racing).

```python
import gymnasium as gym

gym.make("gymnasium_search_race:gymnasium_search_race/MadPodRacingBlocker-v0")
```

https://github.com/user-attachments/assets/57387372-823f-44a2-9a03-23a9332752ab

## Usage

You can use [RL Baselines3 Zoo](https://github.com/DLR-RM/rl-baselines3-zoo) to train and evaluate agents:

```bash
pip install rl_zoo3
```

### Train an Agent

The hyperparameters are defined in `hyperparams/ppo.yml`.

To train a PPO agent for the Search Race game, execute:

```bash
python -m rl_zoo3.train \
  --algo ppo \
  --env gymnasium_search_race/SearchRace-v1 \
  --tensorboard-log logs \
  --eval-freq 20000 \
  --eval-episodes 10 \
  --gym-packages gymnasium_search_race \
  --conf-file hyperparams/ppo.yml \
  --progress
```

For the Mad Pod Racing game, you can add an opponent with the `opponent_path` argument:

```bash
python -m rl_zoo3.train \
  --algo ppo \
  --env gymnasium_search_race/MadPodRacingBlocker-v0 \
  --tensorboard-log logs \
  --eval-freq 20000 \
  --eval-episodes 10 \
  --gym-packages gymnasium_search_race \
  --env-kwargs "opponent_path:'rl-trained-agents/ppo/gymnasium_search_race-MadPodRacing-v0_1/best_model.zip'" \
  --conf-file hyperparams/ppo.yml \
  --progress
```

### Enjoy a Trained Agent

To see a trained agent in action on random test cases, execute:

```bash
python -m rl_zoo3.enjoy \
  --algo ppo \
  --env gymnasium_search_race/SearchRace-v1 \
  --n-timesteps 1000 \
  --deterministic \
  --gym-packages gymnasium_search_race \
  --load-best \
  --progress
```

### Run Test Cases

To run test cases with a trained agent, execute:

```bash
python -m scripts.run_test_cases \
  --path rl-trained-agents/ppo/gymnasium_search_race-SearchRace-v1_1/best_model.zip \
  --env gymnasium_search_race:gymnasium_search_race/SearchRace-v1 \
  --record-video \
  --record-metrics
```

### Record a Video of a Trained Agent

To record a video of a trained agent on Mad Pod Racing, execute:

```bash
python -m scripts.record_video \
  --path rl-trained-agents/ppo/gymnasium_search_race-MadPodRacing-v0_1/best_model.zip \
  --env gymnasium_search_race:gymnasium_search_race/MadPodRacing-v0
```

For Mad Pod Racing Blocker, execute:

```bash
python -m scripts.record_video \
  --path rl-trained-agents/ppo/gymnasium_search_race-MadPodRacingBlocker-v0_1/best_model.zip \
  --opponent-path rl-trained-agents/ppo/gymnasium_search_race-MadPodRacing-v0_1/best_model.zip \
  --env gymnasium_search_race:gymnasium_search_race/MadPodRacingBlocker-v0
```

## Tests

To run tests, execute:

```bash
pytest
```

## Citing

To cite the repository in publications:

```bibtex
@misc{gymnasium-search-race,
  author = {Quentin Deschamps},
  title = {Gymnasium Search Race},
  year = {2024},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/Quentin18/gymnasium-search-race}},
}
```

## References

- [Gymnasium](https://github.com/Farama-Foundation/Gymnasium)
- [RL Baselines3 Zoo](https://github.com/DLR-RM/rl-baselines3-zoo)
- [Stable Baselines3](https://github.com/DLR-RM/stable-baselines3)
- [CGSearchRace](https://github.com/Illedan/CGSearchRace)
- [CSB-Runner-Arena](https://github.com/Agade09/CSB-Runner-Arena)
- [Coders Strikes Back by Magus](http://files.magusgeek.com/csb/csb_en.html)

### Assets

- https://www.flaticon.com/free-icon/space-ship_751036
- https://www.flaticon.com/free-icon/space-ship_784925

## Author

[Quentin Deschamps](mailto:quentindeschamps18@gmail.com)

            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "gymnasium-search-race",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.10",
    "maintainer_email": null,
    "keywords": "Reinforcement Learning, game, RL, AI, gymnasium, pygame",
    "author": null,
    "author_email": "Quentin Deschamps <quentindeschamps18@gmail.com>",
    "download_url": "https://files.pythonhosted.org/packages/02/0f/2b4c6c63ca3fccb2ecc3b25e64e8628634eefffd7c36255bcff7546f70a1/gymnasium_search_race-3.0.1.tar.gz",
    "platform": null,
    "description": "# Gymnasium Search Race\n\n[![Build Python Package](https://github.com/Quentin18/gymnasium-search-race/actions/workflows/build.yml/badge.svg)](https://github.com/Quentin18/gymnasium-search-race/actions/workflows/build.yml)\n[![Python](https://img.shields.io/pypi/pyversions/gymnasium-search-race.svg)](https://badge.fury.io/py/gymnasium-search-race)\n[![PyPI](https://badge.fury.io/py/gymnasium-search-race.svg)](https://badge.fury.io/py/gymnasium-search-race)\n[![PyPI Downloads](https://static.pepy.tech/badge/gymnasium-search-race)](https://pepy.tech/projects/gymnasium-search-race)\n[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://pre-commit.com/)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n[![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat&labelColor=ef8336)](https://pycqa.github.io/isort/)\n\nGymnasium environments for\nthe [Search Race CodinGame optimization puzzle](https://www.codingame.com/multiplayer/optimization/search-race)\nand [Mad Pod Racing CodinGame bot programming game](https://www.codingame.com/multiplayer/bot-programming/mad-pod-racing).\n\nhttps://github.com/user-attachments/assets/1862b04b-9e33-4f55-a309-ad665a1db2f1\n\n<table>\n    <tbody>\n        <tr>\n            <td>Action Space</td>\n            <td><code>Box([-1, 0], [1, 1], float64)</code></td>\n        </tr>\n        <tr>\n            <td>Observation Space</td>\n            <td><code>Box([0, 0, 0, 0, 0, 0, 0, -1, -1, 0], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1], float64)</code></td>\n        </tr>\n        <tr>\n            <td>import</td>\n            <td><code>gymnasium.make(\"gymnasium_search_race:gymnasium_search_race/SearchRace-v1\")</code></td>\n        </tr>\n    </tbody>\n</table>\n\n## Installation\n\nTo install `gymnasium-search-race` with pip, execute:\n\n```bash\npip install gymnasium_search_race\n```\n\nFrom source:\n\n```bash\ngit clone https://github.com/Quentin18/gymnasium-search-race\ncd gymnasium-search-race/\npip install -e .\n```\n\n## Environment\n\n### Action Space\n\nThe action is a `ndarray` with 2 continuous variables:\n\n- The rotation angle between -18 and 18 degrees, normalized between -1 and 1.\n- The thrust between 0 and 200, normalized between 0 and 1.\n\n### Observation Space\n\nThe observation is a `ndarray` of 10 continuous variables:\n\n- 1 if the next checkpoint is the last one, 0 otherwise.\n- The x and y coordinates of the next checkpoint.\n- The x and y coordinates of the checkpoint after next checkpoint.\n- The x and y coordinates of the car.\n- The horizontal speed vx and vertical speed vy of the car.\n- The facing angle of the car.\n\nThe values are normalized between 0 and 1, or -1 and 1 if negative values are allowed.\n\n### Reward\n\nThe goal is to visit all checkpoints as quickly as possible, as such the agent is penalised with a reward of `-0.1` for\neach timestep.\nWhen a checkpoint is visited, the agent is awarded with a reward of `1000/total_checkpoints`.\n\n### Starting State\n\nThe starting state is generated by choosing a random CodinGame test case.\n\n### Episode End\n\nThe episode ends if either of the following happens:\n\n1. Termination: The car visit all checkpoints before the time is out.\n2. Truncation: Episode length is greater than 600.\n\n### Arguments\n\n- `test_id`: test case id to generate the checkpoints (see\n  choices [here](https://github.com/Quentin18/gymnasium-search-race/tree/main/src/gymnasium_search_race/envs/maps)). The\n  default value is `None` which selects a test case randomly when the `reset` method is called.\n\n```python\nimport gymnasium as gym\n\ngym.make(\"gymnasium_search_race:gymnasium_search_race/SearchRace-v1\", test_id=1)\n```\n\n### Version History\n\n- v1: Add boolean to indicate if the next checkpoint is the last checkpoint in observation\n- v0: Initial version\n\n## Discrete environment\n\nThe `SearchRaceDiscrete` environment is similar to the `SearchRace` environment except the action space is discrete.\n\n```python\nimport gymnasium as gym\n\ngym.make(\"gymnasium_search_race:gymnasium_search_race/SearchRaceDiscrete-v1\", test_id=1)\n```\n\n### Action Space\n\nThere are 74 discrete actions corresponding to the combinations of angles from -18 to 18 degrees and thrust 0 and 200.\n\n### Version History\n\n- v1: Add all angles in action space\n- v0: Initial version\n\n## Mad Pod Racing\n\n### Runner\n\nThe `MadPodRacing` and `MadPodRacingDiscrete` environments can be used to train a runner for\nthe [Mad Pod Racing CodinGame bot programming game](https://www.codingame.com/multiplayer/bot-programming/mad-pod-racing).\nThey are similar to the `SearchRace` and `SearchRaceDiscrete` environments except the following differences:\n\n- The maximum thrust value is 100 instead of 200.\n- The maps are generated the same way Codingame generates them.\n- The car position is rounded and not truncated.\n\n```python\nimport gymnasium as gym\n\ngym.make(\"gymnasium_search_race:gymnasium_search_race/MadPodRacing-v0\")\ngym.make(\"gymnasium_search_race:gymnasium_search_race/MadPodRacingDiscrete-v0\")\n```\n\nhttps://github.com/user-attachments/assets/ce4b1837-4591-40dd-a203-9eec9146b94b\n\n### Blocker\n\nThe `MadPodRacingBlocker` environment can be used to train a blocker for\nthe [Mad Pod Racing CodinGame bot programming game](https://www.codingame.com/multiplayer/bot-programming/mad-pod-racing).\n\n```python\nimport gymnasium as gym\n\ngym.make(\"gymnasium_search_race:gymnasium_search_race/MadPodRacingBlocker-v0\")\n```\n\nhttps://github.com/user-attachments/assets/57387372-823f-44a2-9a03-23a9332752ab\n\n## Usage\n\nYou can use [RL Baselines3 Zoo](https://github.com/DLR-RM/rl-baselines3-zoo) to train and evaluate agents:\n\n```bash\npip install rl_zoo3\n```\n\n### Train an Agent\n\nThe hyperparameters are defined in `hyperparams/ppo.yml`.\n\nTo train a PPO agent for the Search Race game, execute:\n\n```bash\npython -m rl_zoo3.train \\\n  --algo ppo \\\n  --env gymnasium_search_race/SearchRace-v1 \\\n  --tensorboard-log logs \\\n  --eval-freq 20000 \\\n  --eval-episodes 10 \\\n  --gym-packages gymnasium_search_race \\\n  --conf-file hyperparams/ppo.yml \\\n  --progress\n```\n\nFor the Mad Pod Racing game, you can add an opponent with the `opponent_path` argument:\n\n```bash\npython -m rl_zoo3.train \\\n  --algo ppo \\\n  --env gymnasium_search_race/MadPodRacingBlocker-v0 \\\n  --tensorboard-log logs \\\n  --eval-freq 20000 \\\n  --eval-episodes 10 \\\n  --gym-packages gymnasium_search_race \\\n  --env-kwargs \"opponent_path:'rl-trained-agents/ppo/gymnasium_search_race-MadPodRacing-v0_1/best_model.zip'\" \\\n  --conf-file hyperparams/ppo.yml \\\n  --progress\n```\n\n### Enjoy a Trained Agent\n\nTo see a trained agent in action on random test cases, execute:\n\n```bash\npython -m rl_zoo3.enjoy \\\n  --algo ppo \\\n  --env gymnasium_search_race/SearchRace-v1 \\\n  --n-timesteps 1000 \\\n  --deterministic \\\n  --gym-packages gymnasium_search_race \\\n  --load-best \\\n  --progress\n```\n\n### Run Test Cases\n\nTo run test cases with a trained agent, execute:\n\n```bash\npython -m scripts.run_test_cases \\\n  --path rl-trained-agents/ppo/gymnasium_search_race-SearchRace-v1_1/best_model.zip \\\n  --env gymnasium_search_race:gymnasium_search_race/SearchRace-v1 \\\n  --record-video \\\n  --record-metrics\n```\n\n### Record a Video of a Trained Agent\n\nTo record a video of a trained agent on Mad Pod Racing, execute:\n\n```bash\npython -m scripts.record_video \\\n  --path rl-trained-agents/ppo/gymnasium_search_race-MadPodRacing-v0_1/best_model.zip \\\n  --env gymnasium_search_race:gymnasium_search_race/MadPodRacing-v0\n```\n\nFor Mad Pod Racing Blocker, execute:\n\n```bash\npython -m scripts.record_video \\\n  --path rl-trained-agents/ppo/gymnasium_search_race-MadPodRacingBlocker-v0_1/best_model.zip \\\n  --opponent-path rl-trained-agents/ppo/gymnasium_search_race-MadPodRacing-v0_1/best_model.zip \\\n  --env gymnasium_search_race:gymnasium_search_race/MadPodRacingBlocker-v0\n```\n\n## Tests\n\nTo run tests, execute:\n\n```bash\npytest\n```\n\n## Citing\n\nTo cite the repository in publications:\n\n```bibtex\n@misc{gymnasium-search-race,\n  author = {Quentin Deschamps},\n  title = {Gymnasium Search Race},\n  year = {2024},\n  publisher = {GitHub},\n  journal = {GitHub repository},\n  howpublished = {\\url{https://github.com/Quentin18/gymnasium-search-race}},\n}\n```\n\n## References\n\n- [Gymnasium](https://github.com/Farama-Foundation/Gymnasium)\n- [RL Baselines3 Zoo](https://github.com/DLR-RM/rl-baselines3-zoo)\n- [Stable Baselines3](https://github.com/DLR-RM/stable-baselines3)\n- [CGSearchRace](https://github.com/Illedan/CGSearchRace)\n- [CSB-Runner-Arena](https://github.com/Agade09/CSB-Runner-Arena)\n- [Coders Strikes Back by Magus](http://files.magusgeek.com/csb/csb_en.html)\n\n### Assets\n\n- https://www.flaticon.com/free-icon/space-ship_751036\n- https://www.flaticon.com/free-icon/space-ship_784925\n\n## Author\n\n[Quentin Deschamps](mailto:quentindeschamps18@gmail.com)\n",
    "bugtrack_url": null,
    "license": "MIT",
    "summary": "A reinforcement learning environment for the Search Race CG puzzle based on Gymnasium",
    "version": "3.0.1",
    "project_urls": {
        "Repository": "https://github.com/Quentin18/gymnasium-search-race"
    },
    "split_keywords": [
        "reinforcement learning",
        " game",
        " rl",
        " ai",
        " gymnasium",
        " pygame"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "fcd0b460aba74124ad0e1badc48c309c2a392e5e5cc8da64c0f6a4cd6710ba45",
                "md5": "b59a705a3d6aa92bd696bafc81cd936f",
                "sha256": "4bdbc4ccbe9d88323966d7eb35a426d66606653efd9adc2523fe47584ed7debf"
            },
            "downloads": -1,
            "filename": "gymnasium_search_race-3.0.1-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "b59a705a3d6aa92bd696bafc81cd936f",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.10",
            "size": 421954,
            "upload_time": "2024-12-01T10:12:54",
            "upload_time_iso_8601": "2024-12-01T10:12:54.169454Z",
            "url": "https://files.pythonhosted.org/packages/fc/d0/b460aba74124ad0e1badc48c309c2a392e5e5cc8da64c0f6a4cd6710ba45/gymnasium_search_race-3.0.1-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "020f2b4c6c63ca3fccb2ecc3b25e64e8628634eefffd7c36255bcff7546f70a1",
                "md5": "1306d12d42aa14d1ca8f51f00dac375b",
                "sha256": "dc4d5ef14ad4e83cef4638c50a68bd4aa661f0282cef9eff974911ee10389ef5"
            },
            "downloads": -1,
            "filename": "gymnasium_search_race-3.0.1.tar.gz",
            "has_sig": false,
            "md5_digest": "1306d12d42aa14d1ca8f51f00dac375b",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.10",
            "size": 2887291,
            "upload_time": "2024-12-01T10:13:00",
            "upload_time_iso_8601": "2024-12-01T10:13:00.904423Z",
            "url": "https://files.pythonhosted.org/packages/02/0f/2b4c6c63ca3fccb2ecc3b25e64e8628634eefffd7c36255bcff7546f70a1/gymnasium_search_race-3.0.1.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-12-01 10:13:00",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "Quentin18",
    "github_project": "gymnasium-search-race",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "gymnasium-search-race"
}
        
Elapsed time: 0.69662s