# JoyRL
[](https://pypi.org/project/joyrl/) [](https://github.com/datawhalechina/joyrl/issues) [](https://github.com/datawhalechina/joyrl/stargazers) [](https://github.com/datawhalechina/joyrl/network) [](https://github.com/datawhalechina/joyrl/blob/master/LICENSE)
`JoyRL` is a parallel reinforcement learning library based on PyTorch and Ray. Unlike existing RL libraries, `JoyRL` is helping users to release the burden of implementing algorithms with tough details, unfriendly APIs, and etc. JoyRL is designed for users to train and test RL algorithms with **only hyperparameters configuration**, which is mush easier for beginners to learn and use. Also, JoyRL supports plenties of state-of-art RL algorithms including **RLHF(core of ChatGPT)**(See algorithms below). JoyRL provides a **modularized framework** for users as well to customize their own algorithms and environments.
## Install
⚠️ Note that donot install JoyRL through any mirror image!!!
```bash
# you need to install Anaconda first
conda create -n joyrl python=3.8
conda activate joyrl
pip install -U joyrl
```
Torch install:
Pip install is recommended, but if you encounter network error, you can try conda install or pip install with mirrors.
```bash
# pip CPU only
pip install torch==1.10.0 torchvision==0.11.0 torchaudio==0.10.0
# if network error, then GPU with mirror image
pip install torch==1.10.0+cu113 torchvision==0.11.0+cu113 torchaudio==0.10.0 --extra-index-url https://download.pytorch.org/whl/cu113
# CPU only
conda install pytorch==1.10.0 torchvision==0.11.0 torchaudio==0.10.0 cpuonly -c pytorch
# GPU
conda install pytorch==1.10.0 torchvision==0.11.0 torchaudio==0.10.0 cudatoolkit=11.3 -c pytorch -c conda-forge
```
## Usage
### Quick Start
the following presents a demo to use joyrl. As you can see, first create a yaml file to **config hyperparameters**, then run the command as below in your terminal. That's all you need to do to train a DQN agent on CartPole-v1 environment.
```bash
joyrl --yaml ./presets/ClassControl/CartPole-v1/CartPole-v1_DQN.yaml
```
or you can run the following code in your python file.
```python
import joyrl
if __name__ == "__main__":
print(joyrl.__version__)
yaml_path = "./presets/ClassControl/CartPole-v1/CartPole-v1_DQN.yaml"
joyrl.run(yaml_path = yaml_path)
```
## Documentation
More tutorials and API documentation are hosted on [JoyRL docs](https://datawhalechina.github.io/joyrl/) or [JoyRL 中文文档](https://datawhalechina.github.io/joyrl-book/#/joyrl_docs/main).
## Algorithms
| Name | Reference | Author | Notes |
| :--------------: | :----------------------------------------------------------: | :-------------------------------------------: | :---: |
| Q-learning | [RL introduction](https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf) | [johnjim0816](https://github.com/johnjim0816) | |
| Sarsa | [RL introduction](https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf) | [johnjim0816](https://github.com/johnjim0816) | |
| DQN | [DQN Paper](https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf) | [johnjim0816](https://github.com/johnjim0816) | |
| Double DQN | [DoubleDQN Paper](https://arxiv.org/abs/1509.06461) | [johnjim0816](https://github.com/johnjim0816) | |
| Dueling DQN | [DuelingDQN Paper](https://arxiv.org/abs/1511.06581) | [johnjim0816](https://github.com/johnjim0816) | |
| NoisyDQN | [NoisyDQN Paper](https://arxiv.org/pdf/1706.10295.pdf) | [johnjim0816](https://github.com/johnjim0816) | |
| DDPG | [DDPG Paper](https://arxiv.org/abs/1509.02971) | [johnjim0816](https://github.com/johnjim0816) | |
| TD3 | [TD3 Paper](https://arxiv.org/pdf/1802.09477) | [johnjim0816](https://github.com/johnjim0816) | |
| PPO | [PPO Paper](https://arxiv.org/abs/1707.06347) | [johnjim0816](https://github.com/johnjim0816) | |
## Why JoyRL?
| RL Platform | GitHub Stars | # of Alg. <sup>(1)</sup> | Custom Env | Async Training | RNN Support | Multi-Head Observation | Backend |
| ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------ | ------------------------------ | ------------------ | ------------------ | ---------------------- | ------------------------------------------------- |
| [Baselines](https://github.com/openai/baselines) | [](https://github.com/openai/baselines/stargazers) | 9 | :heavy_check_mark: (gym) | :x: | :heavy_check_mark: | :x: | TF1 |
| [Stable-Baselines](https://github.com/hill-a/stable-baselines) | [](https://github.com/hill-a/stable-baselines/stargazers) | 11 | :heavy_check_mark: (gym) | :x: | :heavy_check_mark: | :x: | TF1 |
| [Stable-Baselines3](https://github.com/DLR-RM/stable-baselines3) | [](https://github.com/DLR-RM/stable-baselines3/stargazers) | 7 | :heavy_check_mark: (gym) | :x: | :x: | :heavy_check_mark: | PyTorch |
| [Ray/RLlib](https://github.com/ray-project/ray/tree/master/rllib/) | [](https://github.com/ray-project/ray/stargazers) | 16 | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | TF/PyTorch |
| [SpinningUp](https://github.com/openai/spinningup) | [](https://github.com/openai/spinningupstargazers) | 6 | :heavy_check_mark: (gym) | :x: | :x: | :x: | PyTorch |
| [Dopamine](https://github.com/google/dopamine) | [](https://github.com/google/dopamine/stargazers) | 7 | :x: | :x: | :x: | :x: | TF/JAX |
| [ACME](https://github.com/deepmind/acme) | [](https://github.com/deepmind/acme/stargazers) | 14 | :heavy_check_mark: (dm_env) | :x: | :heavy_check_mark: | :heavy_check_mark: | TF/JAX |
| [keras-rl](https://github.com/keras-rl/keras-rl) | [](https://github.com/keras-rl/keras-rlstargazers) | 7 | :heavy_check_mark: (gym) | :x: | :x: | :x: | Keras |
| [cleanrl](https://github.com/vwxyzjn/cleanrl) |  | 9 | :heavy_check_mark: (gym) | :x: | :x: | :x: | [poetry](https://github.com/python-poetry/poetry) |
| [rlpyt](https://github.com/astooke/rlpyt) | [](https://github.com/astooke/rlpyt/stargazers) | 11 | :x: | :x: | :heavy_check_mark: | :heavy_check_mark: | PyTorch |
| [ChainerRL](https://github.com/chainer/chainerrl) | [](https://github.com/chainer/chainerrl/stargazers) | 18 | :heavy_check_mark: (gym) | :x: | :heavy_check_mark: | :x: | Chainer |
| [Tianshou](https://github.com/thu-ml/tianshou) | [](https://github.com/thu-ml/tianshou/stargazers) | 20 | :heavy_check_mark: (Gymnasium) | :x: | :heavy_check_mark: | :heavy_check_mark: | PyTorch |
| [JoyRL](https://github.com/datawhalechina/joyrl) |  | 9 | :heavy_check_mark: (Gymnasium) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | PyTorch |
Here are some other highlghts of JoyRL:
* Provide a series of Chinese courses [JoyRL Book](https://github.com/datawhalechina/joyrl-book) (with the English version in progress), suitable for beginners to start with a combination of theory
## Contributors
<table border="0">
<tbody>
<tr align="center" >
<td>
<a href="https://github.com/JohnJim0816"><img width="70" height="70" src="https://github.com/JohnJim0816.png?s=40" alt="pic"></a><br>
<a href="https://github.com/JohnJim0816">John Jim</a>
<p>Peking University</p>
</td>
<td>
<a href="https://github.com/qiwang067"><img width="70" height="70" src="https://github.com/qiwang067.png?s=40" alt="pic"></a><br>
<a href="https://github.com/qiwang067">Qi Wang</a>
<p>Shanghai Jiao Tong University</p>
</td>
<td>
<a href="https://github.com/yyysjz1997"><img width="70" height="70" src="https://github.com/yyysjz1997.png?s=40" alt="pic"></a><br>
<a href="https://github.com/yyysjz1997">Yiyuan Yang</a>
<p>University of Oxford</p>
</td>
</tr>
</tbody>
</table>
Raw data
{
"_id": null,
"home_page": "https://github.com/datawhalechina/joyrl",
"name": "joyrl",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "reinforcement learning platform pytorch",
"author": "johnjim0816",
"author_email": "johnjim0816@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/12/50/a82f201abe8f3aa38a51e80dafaabc0ab50580e2e029354fe68ae6862d11/joyrl-0.5.2.tar.gz",
"platform": "any",
"description": "# JoyRL\n\n[](https://pypi.org/project/joyrl/) [](https://github.com/datawhalechina/joyrl/issues) [](https://github.com/datawhalechina/joyrl/stargazers) [](https://github.com/datawhalechina/joyrl/network) [](https://github.com/datawhalechina/joyrl/blob/master/LICENSE)\n\n`JoyRL` is a parallel reinforcement learning library based on PyTorch and Ray. Unlike existing RL libraries, `JoyRL` is helping users to release the burden of implementing algorithms with tough details, unfriendly APIs, and etc. JoyRL is designed for users to train and test RL algorithms with **only hyperparameters configuration**, which is mush easier for beginners to learn and use. Also, JoyRL supports plenties of state-of-art RL algorithms including **RLHF(core of ChatGPT)**(See algorithms below). JoyRL provides a **modularized framework** for users as well to customize their own algorithms and environments. \n\n## Install\n\n\u26a0\ufe0f Note that donot install JoyRL through any mirror image!!!\n\n```bash\n# you need to install Anaconda first\nconda create -n joyrl python=3.8\nconda activate joyrl\npip install -U joyrl\n```\n\nTorch install:\n\nPip install is recommended, but if you encounter network error, you can try conda install or pip install with mirrors.\n\n```bash\n# pip CPU only\npip install torch==1.10.0 torchvision==0.11.0 torchaudio==0.10.0\n# if network error, then GPU with mirror image\npip install torch==1.10.0+cu113 torchvision==0.11.0+cu113 torchaudio==0.10.0 --extra-index-url https://download.pytorch.org/whl/cu113\n# CPU only\nconda install pytorch==1.10.0 torchvision==0.11.0 torchaudio==0.10.0 cpuonly -c pytorch\n# GPU \nconda install pytorch==1.10.0 torchvision==0.11.0 torchaudio==0.10.0 cudatoolkit=11.3 -c pytorch -c conda-forge\n```\n\n## Usage\n\n### Quick Start\n\nthe following presents a demo to use joyrl. As you can see, first create a yaml file to **config hyperparameters**, then run the command as below in your terminal. That's all you need to do to train a DQN agent on CartPole-v1 environment.\n\n```bash\njoyrl --yaml ./presets/ClassControl/CartPole-v1/CartPole-v1_DQN.yaml\n```\nor you can run the following code in your python file. \n\n```python\nimport joyrl\nif __name__ == \"__main__\":\n print(joyrl.__version__)\n yaml_path = \"./presets/ClassControl/CartPole-v1/CartPole-v1_DQN.yaml\"\n joyrl.run(yaml_path = yaml_path)\n```\n\n\n\n## Documentation\n\nMore tutorials and API documentation are hosted on [JoyRL docs](https://datawhalechina.github.io/joyrl/) or [JoyRL \u4e2d\u6587\u6587\u6863](https://datawhalechina.github.io/joyrl-book/#/joyrl_docs/main).\n\n## Algorithms\n\n| Name | Reference | Author | Notes |\n| :--------------: | :----------------------------------------------------------: | :-------------------------------------------: | :---: |\n| Q-learning | [RL introduction](https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf) | [johnjim0816](https://github.com/johnjim0816) | |\n| Sarsa | [RL introduction](https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf) | [johnjim0816](https://github.com/johnjim0816) | |\n| DQN | [DQN Paper](https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf) | [johnjim0816](https://github.com/johnjim0816) | |\n| Double DQN | [DoubleDQN Paper](https://arxiv.org/abs/1509.06461) | [johnjim0816](https://github.com/johnjim0816) | |\n| Dueling DQN | [DuelingDQN Paper](https://arxiv.org/abs/1511.06581) | [johnjim0816](https://github.com/johnjim0816) | |\n| NoisyDQN | [NoisyDQN Paper](https://arxiv.org/pdf/1706.10295.pdf) | [johnjim0816](https://github.com/johnjim0816) | |\n| DDPG | [DDPG Paper](https://arxiv.org/abs/1509.02971) | [johnjim0816](https://github.com/johnjim0816) | |\n| TD3 | [TD3 Paper](https://arxiv.org/pdf/1802.09477) | [johnjim0816](https://github.com/johnjim0816) | |\n| PPO | [PPO Paper](https://arxiv.org/abs/1707.06347) | [johnjim0816](https://github.com/johnjim0816) | |\n\n## Why JoyRL?\n\n| RL Platform | GitHub Stars | # of Alg. <sup>(1)</sup> | Custom Env | Async Training | RNN Support | Multi-Head Observation | Backend |\n| ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------ | ------------------------------ | ------------------ | ------------------ | ---------------------- | ------------------------------------------------- |\n| [Baselines](https://github.com/openai/baselines) | [](https://github.com/openai/baselines/stargazers) | 9 | :heavy_check_mark: (gym) | :x: | :heavy_check_mark: | :x: | TF1 |\n| [Stable-Baselines](https://github.com/hill-a/stable-baselines) | [](https://github.com/hill-a/stable-baselines/stargazers) | 11 | :heavy_check_mark: (gym) | :x: | :heavy_check_mark: | :x: | TF1 |\n| [Stable-Baselines3](https://github.com/DLR-RM/stable-baselines3) | [](https://github.com/DLR-RM/stable-baselines3/stargazers) | 7 | :heavy_check_mark: (gym) | :x: | :x: | :heavy_check_mark: | PyTorch |\n| [Ray/RLlib](https://github.com/ray-project/ray/tree/master/rllib/) | [](https://github.com/ray-project/ray/stargazers) | 16 | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | TF/PyTorch |\n| [SpinningUp](https://github.com/openai/spinningup) | [](https://github.com/openai/spinningupstargazers) | 6 | :heavy_check_mark: (gym) | :x: | :x: | :x: | PyTorch |\n| [Dopamine](https://github.com/google/dopamine) | [](https://github.com/google/dopamine/stargazers) | 7 | :x: | :x: | :x: | :x: | TF/JAX |\n| [ACME](https://github.com/deepmind/acme) | [](https://github.com/deepmind/acme/stargazers) | 14 | :heavy_check_mark: (dm_env) | :x: | :heavy_check_mark: | :heavy_check_mark: | TF/JAX |\n| [keras-rl](https://github.com/keras-rl/keras-rl) | [](https://github.com/keras-rl/keras-rlstargazers) | 7 | :heavy_check_mark: (gym) | :x: | :x: | :x: | Keras |\n| [cleanrl](https://github.com/vwxyzjn/cleanrl) |  | 9 | :heavy_check_mark: (gym) | :x: | :x: | :x: | [poetry](https://github.com/python-poetry/poetry) |\n| [rlpyt](https://github.com/astooke/rlpyt) | [](https://github.com/astooke/rlpyt/stargazers) | 11 | :x: | :x: | :heavy_check_mark: | :heavy_check_mark: | PyTorch |\n| [ChainerRL](https://github.com/chainer/chainerrl) | [](https://github.com/chainer/chainerrl/stargazers) | 18 | :heavy_check_mark: (gym) | :x: | :heavy_check_mark: | :x: | Chainer |\n| [Tianshou](https://github.com/thu-ml/tianshou) | [](https://github.com/thu-ml/tianshou/stargazers) | 20 | :heavy_check_mark: (Gymnasium) | :x: | :heavy_check_mark: | :heavy_check_mark: | PyTorch |\n| [JoyRL](https://github.com/datawhalechina/joyrl) |  | 9 | :heavy_check_mark: (Gymnasium) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | PyTorch |\n\nHere are some other highlghts of JoyRL:\n\n* Provide a series of Chinese courses [JoyRL Book](https://github.com/datawhalechina/joyrl-book) (with the English version in progress), suitable for beginners to start with a combination of theory\n\n## Contributors\n\n<table border=\"0\">\n <tbody>\n <tr align=\"center\" >\n <td>\n <a href=\"https://github.com/JohnJim0816\"><img width=\"70\" height=\"70\" src=\"https://github.com/JohnJim0816.png?s=40\" alt=\"pic\"></a><br>\n <a href=\"https://github.com/JohnJim0816\">John Jim</a>\n <p>Peking University</p>\n </td>\n <td>\n <a href=\"https://github.com/qiwang067\"><img width=\"70\" height=\"70\" src=\"https://github.com/qiwang067.png?s=40\" alt=\"pic\"></a><br>\n <a href=\"https://github.com/qiwang067\">Qi Wang</a> \n <p>Shanghai Jiao Tong University</p>\n </td>\n <td>\n <a href=\"https://github.com/yyysjz1997\"><img width=\"70\" height=\"70\" src=\"https://github.com/yyysjz1997.png?s=40\" alt=\"pic\"></a><br>\n <a href=\"https://github.com/yyysjz1997\">Yiyuan Yang</a> \n <p>University of Oxford</p>\n </td>\n </tr>\n </tbody>\n</table>\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A Library for Deep Reinforcement Learning",
"version": "0.5.2",
"project_urls": {
"Homepage": "https://github.com/datawhalechina/joyrl"
},
"split_keywords": [
"reinforcement",
"learning",
"platform",
"pytorch"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0c04917daec9639e3b729fbbeffd232cb9dc37139689dba2f95d511812a228ed",
"md5": "0809c305f3e1794d0badc45b28d873bc",
"sha256": "8eadceca2604105b7d9c69d8a9f4bb90e8b3e5c8d67130c4d9a89d23e8e46b00"
},
"downloads": -1,
"filename": "joyrl-0.5.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "0809c305f3e1794d0badc45b28d873bc",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 92157,
"upload_time": "2024-01-27T14:58:10",
"upload_time_iso_8601": "2024-01-27T14:58:10.943137Z",
"url": "https://files.pythonhosted.org/packages/0c/04/917daec9639e3b729fbbeffd232cb9dc37139689dba2f95d511812a228ed/joyrl-0.5.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "1250a82f201abe8f3aa38a51e80dafaabc0ab50580e2e029354fe68ae6862d11",
"md5": "81ff0da80db6bafa4c1043a95cad11fc",
"sha256": "87ca7746abb7ea1d9bc44b525720f6b6a3b2e5211119031e92c10c3aa8d292bf"
},
"downloads": -1,
"filename": "joyrl-0.5.2.tar.gz",
"has_sig": false,
"md5_digest": "81ff0da80db6bafa4c1043a95cad11fc",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 66693,
"upload_time": "2024-01-27T14:58:12",
"upload_time_iso_8601": "2024-01-27T14:58:12.846938Z",
"url": "https://files.pythonhosted.org/packages/12/50/a82f201abe8f3aa38a51e80dafaabc0ab50580e2e029354fe68ae6862d11/joyrl-0.5.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-27 14:58:12",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "datawhalechina",
"github_project": "joyrl",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "joyrl"
}
JSON
