sumo-rl


Namesumo-rl JSON
Version 1.4.5 PyPI version JSON
download
home_pageNone
SummaryRL environments and learning code for traffic signal control in SUMO.
upload_time2024-05-07 14:51:35
maintainerNone
docs_urlNone
authorNone
requires_python>=3.7
licenseMIT License
keywords reinforcement learning traffic signal control sumo rl pettingzoo gymnasium
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            <img src="docs/_static/logo.png" align="right" width="30%"/>

[![DOI](https://zenodo.org/badge/161216111.svg)](https://zenodo.org/doi/10.5281/zenodo.10869789)
[![tests](https://github.com/LucasAlegre/sumo-rl/actions/workflows/linux-test.yml/badge.svg)](https://github.com/LucasAlegre/sumo-rl/actions/workflows/linux-test.yml)
[![PyPI version](https://badge.fury.io/py/sumo-rl.svg)](https://badge.fury.io/py/sumo-rl)
[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://pre-commit.com/)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
[![License](http://img.shields.io/badge/license-MIT-brightgreen.svg?style=flat)](https://github.com/LucasAlegre/sumo-rl/blob/main/LICENSE)

# SUMO-RL

<!-- start intro -->

SUMO-RL provides a simple interface to instantiate Reinforcement Learning (RL) environments with [SUMO](https://github.com/eclipse/sumo) for Traffic Signal Control.

Goals of this repository:
- Provide a simple interface to work with Reinforcement Learning for Traffic Signal Control using SUMO
- Support Multiagent RL
- Compatibility with gymnasium.Env and popular RL libraries such as [stable-baselines3](https://github.com/DLR-RM/stable-baselines3) and [RLlib](https://docs.ray.io/en/main/rllib.html)
- Easy customisation: state and reward definitions are easily modifiable

The main class is [SumoEnvironment](https://github.com/LucasAlegre/sumo-rl/blob/main/sumo_rl/environment/env.py).
If instantiated with parameter 'single-agent=True', it behaves like a regular [Gymnasium Env](https://github.com/Farama-Foundation/Gymnasium).
For multiagent environments, use [env](https://github.com/LucasAlegre/sumo-rl/blob/main/sumo_rl/environment/env.py) or [parallel_env](https://github.com/LucasAlegre/sumo-rl/blob/main/sumo_rl/environment/env.py) to instantiate a [PettingZoo](https://github.com/PettingZoo-Team/PettingZoo) environment with AEC or Parallel API, respectively.
[TrafficSignal](https://github.com/LucasAlegre/sumo-rl/blob/main/sumo_rl/environment/traffic_signal.py) is responsible for retrieving information and actuating on traffic lights using [TraCI](https://sumo.dlr.de/wiki/TraCI) API.

For more details, check the [documentation online](https://lucasalegre.github.io/sumo-rl/).

<!-- end intro -->

## Install

<!-- start install -->

### Install SUMO latest version:

```bash
sudo add-apt-repository ppa:sumo/stable
sudo apt-get update
sudo apt-get install sumo sumo-tools sumo-doc
```
Don't forget to set SUMO_HOME variable (default sumo installation path is /usr/share/sumo)
```bash
echo 'export SUMO_HOME="/usr/share/sumo"' >> ~/.bashrc
source ~/.bashrc
```
Important: for a huge performance boost (~8x) with Libsumo, you can declare the variable:
```bash
export LIBSUMO_AS_TRACI=1
```
Notice that you will not be able to run with sumo-gui or with multiple simulations in parallel if this is active ([more details](https://sumo.dlr.de/docs/Libsumo.html)).

### Install SUMO-RL

Stable release version is available through pip
```bash
pip install sumo-rl
```

Alternatively, you can install using the latest (unreleased) version
```bash
git clone https://github.com/LucasAlegre/sumo-rl
cd sumo-rl
pip install -e .
```

<!-- end install -->

## MDP - Observations, Actions and Rewards

### Observation

<!-- start observation -->

The default observation for each traffic signal agent is a vector:
```python
    obs = [phase_one_hot, min_green, lane_1_density,...,lane_n_density, lane_1_queue,...,lane_n_queue]
```
- ```phase_one_hot``` is a one-hot encoded vector indicating the current active green phase
- ```min_green``` is a binary variable indicating whether min_green seconds have already passed in the current phase
- ```lane_i_density``` is the number of vehicles in incoming lane i dividided by the total capacity of the lane
- ```lane_i_queue```is the number of queued (speed below 0.1 m/s) vehicles in incoming lane i divided by the total capacity of the lane

You can define your own observation by implementing a class that inherits from [ObservationFunction](https://github.com/LucasAlegre/sumo-rl/blob/main/sumo_rl/environment/observations.py) and passing it to the environment constructor.

<!-- end observation -->

### Action

<!-- start action -->

The action space is discrete.
Every 'delta_time' seconds, each traffic signal agent can choose the next green phase configuration.

E.g.: In the [2-way single intersection](https://github.com/LucasAlegre/sumo-rl/blob/main/experiments/dqn_2way-single-intersection.py) there are |A| = 4 discrete actions, corresponding to the following green phase configurations:

<p align="center">
<img src="docs/_static/actions.png" align="center" width="75%"/>
</p>

Important: every time a phase change occurs, the next phase is preeceded by a yellow phase lasting ```yellow_time``` seconds.

<!-- end action -->

### Rewards

<!-- start reward -->

The default reward function is the change in cumulative vehicle delay:

<p align="center">
<img src="docs/_static/reward.png" align="center" width="25%"/>
</p>

That is, the reward is how much the total delay (sum of the waiting times of all approaching vehicles) changed in relation to the previous time-step.

You can choose a different reward function (see the ones implemented in [TrafficSignal](https://github.com/LucasAlegre/sumo-rl/blob/main/sumo_rl/environment/traffic_signal.py)) with the parameter `reward_fn` in the [SumoEnvironment](https://github.com/LucasAlegre/sumo-rl/blob/main/sumo_rl/environment/env.py) constructor.

It is also possible to implement your own reward function:

```python
def my_reward_fn(traffic_signal):
    return traffic_signal.get_average_speed()

env = SumoEnvironment(..., reward_fn=my_reward_fn)
```

<!-- end reward -->

## API's (Gymnasium and PettingZoo)

### Gymnasium Single-Agent API

<!-- start gymnasium -->

If your network only has ONE traffic light, then you can instantiate a standard Gymnasium env (see [Gymnasium API](https://gymnasium.farama.org/api/env/)):
```python
import gymnasium as gym
import sumo_rl
env = gym.make('sumo-rl-v0',
                net_file='path_to_your_network.net.xml',
                route_file='path_to_your_routefile.rou.xml',
                out_csv_name='path_to_output.csv',
                use_gui=True,
                num_seconds=100000)
obs, info = env.reset()
done = False
while not done:
    next_obs, reward, terminated, truncated, info = env.step(env.action_space.sample())
    done = terminated or truncated
```

<!-- end gymnasium -->

### PettingZoo Multi-Agent API

<!-- start pettingzoo -->

For multi-agent environments, you can use the PettingZoo API (see [Petting Zoo API](https://pettingzoo.farama.org/api/parallel/)):

```python
import sumo_rl
env = sumo_rl.parallel_env(net_file='nets/RESCO/grid4x4/grid4x4.net.xml',
                  route_file='nets/RESCO/grid4x4/grid4x4_1.rou.xml',
                  use_gui=True,
                  num_seconds=3600)
observations = env.reset()
while env.agents:
    actions = {agent: env.action_space(agent).sample() for agent in env.agents}  # this is where you would insert your policy
    observations, rewards, terminations, truncations, infos = env.step(actions)
```

<!-- end pettingzoo -->

### RESCO Benchmarks

In the folder [nets/RESCO](https://github.com/LucasAlegre/sumo-rl/tree/main/nets/RESCO) you can find the network and route files from [RESCO](https://github.com/jault/RESCO) (Reinforcement Learning Benchmarks for Traffic Signal Control), which was built on top of SUMO-RL. See their [paper](https://people.engr.tamu.edu/guni/Papers/NeurIPS-signals.pdf) for results.

<p align="center">
<img src="nets/RESCO/maps.png" align="center" width="60%"/>
</p>

### Experiments

Check [experiments](https://github.com/LucasAlegre/sumo-rl/tree/main/experiments) for examples on how to instantiate an environment and train your RL agent.

### [Q-learning](https://github.com/LucasAlegre/sumo-rl/blob/main/agents/ql_agent.py) in a one-way single intersection:
```bash
python experiments/ql_single-intersection.py
```

### [RLlib PPO](https://docs.ray.io/en/latest/_modules/ray/rllib/algorithms/ppo/ppo.html) multiagent in a 4x4 grid:
```bash
python experiments/ppo_4x4grid.py
```

### [stable-baselines3 DQN](https://github.com/DLR-RM/stable-baselines3/blob/master/stable_baselines3/dqn/dqn.py) in a 2-way single intersection:
Obs: you need to install stable-baselines3 with ```pip install "stable_baselines3[extra]>=2.0.0a9"``` for [Gymnasium compatibility](https://stable-baselines3.readthedocs.io/en/master/guide/install.html).
```bash
python experiments/dqn_2way-single-intersection.py
```

### Plotting results:
```bash
python outputs/plot.py -f outputs/4x4grid/ppo_conn0_ep2
```
<p align="center">
<img src="outputs/result.png" align="center" width="50%"/>
</p>

## Citing

<!-- start citation -->

If you use this repository in your research, please cite:
```bibtex
@misc{sumorl,
    author = {Lucas N. Alegre},
    title = {{SUMO-RL}},
    year = {2019},
    publisher = {GitHub},
    journal = {GitHub repository},
    howpublished = {\url{https://github.com/LucasAlegre/sumo-rl}},
}
```

<!-- end citation -->

<!-- start list of publications -->

List of publications that use SUMO-RL (please open a pull request to add missing entries):
- [Quantifying the impact of non-stationarity in reinforcement learning-based traffic signal control (Alegre et al., 2021)](https://peerj.com/articles/cs-575/)
- [Information-Theoretic State Space Model for Multi-View Reinforcement Learning (Hwang et al., 2023)](https://openreview.net/forum?id=jwy77xkyPt)
- [A citywide TD-learning based intelligent traffic signal control for autonomous vehicles: Performance evaluation using SUMO (Reza et al., 2023)](https://onlinelibrary.wiley.com/doi/full/10.1111/exsy.13301)
- [Handling uncertainty in self-adaptive systems: an ontology-based reinforcement learning model (Ghanadbashi et al., 2023)](https://link.springer.com/article/10.1007/s40860-022-00198-x)
- [Multiagent Reinforcement Learning for Traffic Signal Control: a k-Nearest Neighbors Based Approach (Almeida et al., 2022)](https://ceur-ws.org/Vol-3173/3.pdf)
- [From Local to Global: A Curriculum Learning Approach for Reinforcement Learning-based Traffic Signal Control (Zheng et al., 2022)](https://ieeexplore.ieee.org/abstract/document/9832372)
- [Poster: Reliable On-Ramp Merging via Multimodal Reinforcement Learning (Bagwe et al., 2022)](https://ieeexplore.ieee.org/abstract/document/9996639)
- [Using ontology to guide reinforcement learning agents in unseen situations (Ghanadbashi & Golpayegani, 2022)](https://link.springer.com/article/10.1007/s10489-021-02449-5)
- [Information upwards, recommendation downwards: reinforcement learning with hierarchy for traffic signal control (Antes et al., 2022)](https://www.sciencedirect.com/science/article/pii/S1877050922004185)
- [A Comparative Study of Algorithms for Intelligent Traffic Signal Control (Chaudhuri et al., 2022)](https://link.springer.com/chapter/10.1007/978-981-16-7996-4_19)
- [An Ontology-Based Intelligent Traffic Signal Control Model (Ghanadbashi & Golpayegani, 2021)](https://ieeexplore.ieee.org/abstract/document/9564962)
- [Reinforcement Learning Benchmarks for Traffic Signal Control (Ault & Sharon, 2021)](https://openreview.net/forum?id=LqRSh6V0vR)
- [EcoLight: Reward Shaping in Deep Reinforcement Learning for Ergonomic Traffic Signal Control (Agand et al., 2021)](https://s3.us-east-1.amazonaws.com/climate-change-ai/papers/neurips2021/43/paper.pdf)

<!-- end list of publications -->

            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "sumo-rl",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.7",
    "maintainer_email": null,
    "keywords": "Reinforcement Learning, Traffic Signal Control, SUMO, RL, PettingZoo, gymnasium",
    "author": null,
    "author_email": "Lucas Alegre <lucasnale@gmail.com>",
    "download_url": "https://files.pythonhosted.org/packages/e9/76/892e7b2e314887f21958a9c5b215c689a2e50cc134e49ec6654efef66751/sumo_rl-1.4.5.tar.gz",
    "platform": null,
    "description": "<img src=\"docs/_static/logo.png\" align=\"right\" width=\"30%\"/>\n\n[![DOI](https://zenodo.org/badge/161216111.svg)](https://zenodo.org/doi/10.5281/zenodo.10869789)\n[![tests](https://github.com/LucasAlegre/sumo-rl/actions/workflows/linux-test.yml/badge.svg)](https://github.com/LucasAlegre/sumo-rl/actions/workflows/linux-test.yml)\n[![PyPI version](https://badge.fury.io/py/sumo-rl.svg)](https://badge.fury.io/py/sumo-rl)\n[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://pre-commit.com/)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n[![License](http://img.shields.io/badge/license-MIT-brightgreen.svg?style=flat)](https://github.com/LucasAlegre/sumo-rl/blob/main/LICENSE)\n\n# SUMO-RL\n\n<!-- start intro -->\n\nSUMO-RL provides a simple interface to instantiate Reinforcement Learning (RL) environments with [SUMO](https://github.com/eclipse/sumo) for Traffic Signal Control.\n\nGoals of this repository:\n- Provide a simple interface to work with Reinforcement Learning for Traffic Signal Control using SUMO\n- Support Multiagent RL\n- Compatibility with gymnasium.Env and popular RL libraries such as [stable-baselines3](https://github.com/DLR-RM/stable-baselines3) and [RLlib](https://docs.ray.io/en/main/rllib.html)\n- Easy customisation: state and reward definitions are easily modifiable\n\nThe main class is [SumoEnvironment](https://github.com/LucasAlegre/sumo-rl/blob/main/sumo_rl/environment/env.py).\nIf instantiated with parameter 'single-agent=True', it behaves like a regular [Gymnasium Env](https://github.com/Farama-Foundation/Gymnasium).\nFor multiagent environments, use [env](https://github.com/LucasAlegre/sumo-rl/blob/main/sumo_rl/environment/env.py) or [parallel_env](https://github.com/LucasAlegre/sumo-rl/blob/main/sumo_rl/environment/env.py) to instantiate a [PettingZoo](https://github.com/PettingZoo-Team/PettingZoo) environment with AEC or Parallel API, respectively.\n[TrafficSignal](https://github.com/LucasAlegre/sumo-rl/blob/main/sumo_rl/environment/traffic_signal.py) is responsible for retrieving information and actuating on traffic lights using [TraCI](https://sumo.dlr.de/wiki/TraCI) API.\n\nFor more details, check the [documentation online](https://lucasalegre.github.io/sumo-rl/).\n\n<!-- end intro -->\n\n## Install\n\n<!-- start install -->\n\n### Install SUMO latest version:\n\n```bash\nsudo add-apt-repository ppa:sumo/stable\nsudo apt-get update\nsudo apt-get install sumo sumo-tools sumo-doc\n```\nDon't forget to set SUMO_HOME variable (default sumo installation path is /usr/share/sumo)\n```bash\necho 'export SUMO_HOME=\"/usr/share/sumo\"' >> ~/.bashrc\nsource ~/.bashrc\n```\nImportant: for a huge performance boost (~8x) with Libsumo, you can declare the variable:\n```bash\nexport LIBSUMO_AS_TRACI=1\n```\nNotice that you will not be able to run with sumo-gui or with multiple simulations in parallel if this is active ([more details](https://sumo.dlr.de/docs/Libsumo.html)).\n\n### Install SUMO-RL\n\nStable release version is available through pip\n```bash\npip install sumo-rl\n```\n\nAlternatively, you can install using the latest (unreleased) version\n```bash\ngit clone https://github.com/LucasAlegre/sumo-rl\ncd sumo-rl\npip install -e .\n```\n\n<!-- end install -->\n\n## MDP - Observations, Actions and Rewards\n\n### Observation\n\n<!-- start observation -->\n\nThe default observation for each traffic signal agent is a vector:\n```python\n    obs = [phase_one_hot, min_green, lane_1_density,...,lane_n_density, lane_1_queue,...,lane_n_queue]\n```\n- ```phase_one_hot``` is a one-hot encoded vector indicating the current active green phase\n- ```min_green``` is a binary variable indicating whether min_green seconds have already passed in the current phase\n- ```lane_i_density``` is the number of vehicles in incoming lane i dividided by the total capacity of the lane\n- ```lane_i_queue```is the number of queued (speed below 0.1 m/s) vehicles in incoming lane i divided by the total capacity of the lane\n\nYou can define your own observation by implementing a class that inherits from [ObservationFunction](https://github.com/LucasAlegre/sumo-rl/blob/main/sumo_rl/environment/observations.py) and passing it to the environment constructor.\n\n<!-- end observation -->\n\n### Action\n\n<!-- start action -->\n\nThe action space is discrete.\nEvery 'delta_time' seconds, each traffic signal agent can choose the next green phase configuration.\n\nE.g.: In the [2-way single intersection](https://github.com/LucasAlegre/sumo-rl/blob/main/experiments/dqn_2way-single-intersection.py) there are |A| = 4 discrete actions, corresponding to the following green phase configurations:\n\n<p align=\"center\">\n<img src=\"docs/_static/actions.png\" align=\"center\" width=\"75%\"/>\n</p>\n\nImportant: every time a phase change occurs, the next phase is preeceded by a yellow phase lasting ```yellow_time``` seconds.\n\n<!-- end action -->\n\n### Rewards\n\n<!-- start reward -->\n\nThe default reward function is the change in cumulative vehicle delay:\n\n<p align=\"center\">\n<img src=\"docs/_static/reward.png\" align=\"center\" width=\"25%\"/>\n</p>\n\nThat is, the reward is how much the total delay (sum of the waiting times of all approaching vehicles) changed in relation to the previous time-step.\n\nYou can choose a different reward function (see the ones implemented in [TrafficSignal](https://github.com/LucasAlegre/sumo-rl/blob/main/sumo_rl/environment/traffic_signal.py)) with the parameter `reward_fn` in the [SumoEnvironment](https://github.com/LucasAlegre/sumo-rl/blob/main/sumo_rl/environment/env.py) constructor.\n\nIt is also possible to implement your own reward function:\n\n```python\ndef my_reward_fn(traffic_signal):\n    return traffic_signal.get_average_speed()\n\nenv = SumoEnvironment(..., reward_fn=my_reward_fn)\n```\n\n<!-- end reward -->\n\n## API's (Gymnasium and PettingZoo)\n\n### Gymnasium Single-Agent API\n\n<!-- start gymnasium -->\n\nIf your network only has ONE traffic light, then you can instantiate a standard Gymnasium env (see [Gymnasium API](https://gymnasium.farama.org/api/env/)):\n```python\nimport gymnasium as gym\nimport sumo_rl\nenv = gym.make('sumo-rl-v0',\n                net_file='path_to_your_network.net.xml',\n                route_file='path_to_your_routefile.rou.xml',\n                out_csv_name='path_to_output.csv',\n                use_gui=True,\n                num_seconds=100000)\nobs, info = env.reset()\ndone = False\nwhile not done:\n    next_obs, reward, terminated, truncated, info = env.step(env.action_space.sample())\n    done = terminated or truncated\n```\n\n<!-- end gymnasium -->\n\n### PettingZoo Multi-Agent API\n\n<!-- start pettingzoo -->\n\nFor multi-agent environments, you can use the PettingZoo API (see [Petting Zoo API](https://pettingzoo.farama.org/api/parallel/)):\n\n```python\nimport sumo_rl\nenv = sumo_rl.parallel_env(net_file='nets/RESCO/grid4x4/grid4x4.net.xml',\n                  route_file='nets/RESCO/grid4x4/grid4x4_1.rou.xml',\n                  use_gui=True,\n                  num_seconds=3600)\nobservations = env.reset()\nwhile env.agents:\n    actions = {agent: env.action_space(agent).sample() for agent in env.agents}  # this is where you would insert your policy\n    observations, rewards, terminations, truncations, infos = env.step(actions)\n```\n\n<!-- end pettingzoo -->\n\n### RESCO Benchmarks\n\nIn the folder [nets/RESCO](https://github.com/LucasAlegre/sumo-rl/tree/main/nets/RESCO) you can find the network and route files from [RESCO](https://github.com/jault/RESCO) (Reinforcement Learning Benchmarks for Traffic Signal Control), which was built on top of SUMO-RL. See their [paper](https://people.engr.tamu.edu/guni/Papers/NeurIPS-signals.pdf) for results.\n\n<p align=\"center\">\n<img src=\"nets/RESCO/maps.png\" align=\"center\" width=\"60%\"/>\n</p>\n\n### Experiments\n\nCheck [experiments](https://github.com/LucasAlegre/sumo-rl/tree/main/experiments) for examples on how to instantiate an environment and train your RL agent.\n\n### [Q-learning](https://github.com/LucasAlegre/sumo-rl/blob/main/agents/ql_agent.py) in a one-way single intersection:\n```bash\npython experiments/ql_single-intersection.py\n```\n\n### [RLlib PPO](https://docs.ray.io/en/latest/_modules/ray/rllib/algorithms/ppo/ppo.html) multiagent in a 4x4 grid:\n```bash\npython experiments/ppo_4x4grid.py\n```\n\n### [stable-baselines3 DQN](https://github.com/DLR-RM/stable-baselines3/blob/master/stable_baselines3/dqn/dqn.py) in a 2-way single intersection:\nObs: you need to install stable-baselines3 with ```pip install \"stable_baselines3[extra]>=2.0.0a9\"``` for [Gymnasium compatibility](https://stable-baselines3.readthedocs.io/en/master/guide/install.html).\n```bash\npython experiments/dqn_2way-single-intersection.py\n```\n\n### Plotting results:\n```bash\npython outputs/plot.py -f outputs/4x4grid/ppo_conn0_ep2\n```\n<p align=\"center\">\n<img src=\"outputs/result.png\" align=\"center\" width=\"50%\"/>\n</p>\n\n## Citing\n\n<!-- start citation -->\n\nIf you use this repository in your research, please cite:\n```bibtex\n@misc{sumorl,\n    author = {Lucas N. Alegre},\n    title = {{SUMO-RL}},\n    year = {2019},\n    publisher = {GitHub},\n    journal = {GitHub repository},\n    howpublished = {\\url{https://github.com/LucasAlegre/sumo-rl}},\n}\n```\n\n<!-- end citation -->\n\n<!-- start list of publications -->\n\nList of publications that use SUMO-RL (please open a pull request to add missing entries):\n- [Quantifying the impact of non-stationarity in reinforcement learning-based traffic signal control (Alegre et al., 2021)](https://peerj.com/articles/cs-575/)\n- [Information-Theoretic State Space Model for Multi-View Reinforcement Learning (Hwang et al., 2023)](https://openreview.net/forum?id=jwy77xkyPt)\n- [A citywide TD-learning based intelligent traffic signal control for autonomous vehicles: Performance evaluation using SUMO (Reza et al., 2023)](https://onlinelibrary.wiley.com/doi/full/10.1111/exsy.13301)\n- [Handling uncertainty in self-adaptive systems: an ontology-based reinforcement learning model (Ghanadbashi et al., 2023)](https://link.springer.com/article/10.1007/s40860-022-00198-x)\n- [Multiagent Reinforcement Learning for Traffic Signal Control: a k-Nearest Neighbors Based Approach (Almeida et al., 2022)](https://ceur-ws.org/Vol-3173/3.pdf)\n- [From Local to Global: A Curriculum Learning Approach for Reinforcement Learning-based Traffic Signal Control (Zheng et al., 2022)](https://ieeexplore.ieee.org/abstract/document/9832372)\n- [Poster: Reliable On-Ramp Merging via Multimodal Reinforcement Learning (Bagwe et al., 2022)](https://ieeexplore.ieee.org/abstract/document/9996639)\n- [Using ontology to guide reinforcement learning agents in unseen situations (Ghanadbashi & Golpayegani, 2022)](https://link.springer.com/article/10.1007/s10489-021-02449-5)\n- [Information upwards, recommendation downwards: reinforcement learning with hierarchy for traffic signal control (Antes et al., 2022)](https://www.sciencedirect.com/science/article/pii/S1877050922004185)\n- [A Comparative Study of Algorithms for Intelligent Traffic Signal Control (Chaudhuri et al., 2022)](https://link.springer.com/chapter/10.1007/978-981-16-7996-4_19)\n- [An Ontology-Based Intelligent Traffic Signal Control Model (Ghanadbashi & Golpayegani, 2021)](https://ieeexplore.ieee.org/abstract/document/9564962)\n- [Reinforcement Learning Benchmarks for Traffic Signal Control (Ault & Sharon, 2021)](https://openreview.net/forum?id=LqRSh6V0vR)\n- [EcoLight: Reward Shaping in Deep Reinforcement Learning for Ergonomic Traffic Signal Control (Agand et al., 2021)](https://s3.us-east-1.amazonaws.com/climate-change-ai/papers/neurips2021/43/paper.pdf)\n\n<!-- end list of publications -->\n",
    "bugtrack_url": null,
    "license": "MIT License",
    "summary": "RL environments and learning code for traffic signal control in SUMO.",
    "version": "1.4.5",
    "project_urls": {
        "Bug Report": "https://github.com/LucasAlegre/sumo-rl/issues",
        "Documentation": "https://lucasalegre.github.io/sumo-rl",
        "Homepage": "https://lucasalegre.github.io/sumo-rl",
        "Repository": "https://github.com/LucasAlegre/sumo-rl"
    },
    "split_keywords": [
        "reinforcement learning",
        " traffic signal control",
        " sumo",
        " rl",
        " pettingzoo",
        " gymnasium"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "3ed30cf60a0398f14d508b5bc8b80ab0dac73c6fd3a14a8751b37f6c19d27c30",
                "md5": "e184db326873e8f788838c3bbd93ed24",
                "sha256": "322d090e6a66d97cc9b265c0b6fa092687bc6b35965fe5f53f678878d1987660"
            },
            "downloads": -1,
            "filename": "sumo_rl-1.4.5-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "e184db326873e8f788838c3bbd93ed24",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.7",
            "size": 1145381,
            "upload_time": "2024-05-07T14:51:30",
            "upload_time_iso_8601": "2024-05-07T14:51:30.372900Z",
            "url": "https://files.pythonhosted.org/packages/3e/d3/0cf60a0398f14d508b5bc8b80ab0dac73c6fd3a14a8751b37f6c19d27c30/sumo_rl-1.4.5-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "e976892e7b2e314887f21958a9c5b215c689a2e50cc134e49ec6654efef66751",
                "md5": "fd912e9246d6fd03c86f6a3c37683c70",
                "sha256": "230979fb77969a76839cd36027db2ac1aea0e2a41d9b38000f592287c3d12c11"
            },
            "downloads": -1,
            "filename": "sumo_rl-1.4.5.tar.gz",
            "has_sig": false,
            "md5_digest": "fd912e9246d6fd03c86f6a3c37683c70",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.7",
            "size": 1092820,
            "upload_time": "2024-05-07T14:51:35",
            "upload_time_iso_8601": "2024-05-07T14:51:35.294229Z",
            "url": "https://files.pythonhosted.org/packages/e9/76/892e7b2e314887f21958a9c5b215c689a2e50cc134e49ec6654efef66751/sumo_rl-1.4.5.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-05-07 14:51:35",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "LucasAlegre",
    "github_project": "sumo-rl",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "sumo-rl"
}
        
Elapsed time: 0.29393s