| Name | cusrl JSON |
| Version |
1.1.0
JSON |
| download |
| home_page | None |
| Summary | Customizable and modular RL algorithms implemented in PyTorch |
| upload_time | 2025-08-03 15:37:28 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.10 |
| license | None |
| keywords |
reinforcement-learning
pytorch
rl
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# CusRL: Customizable Reinforcement Learning
CusRL is a flexible and modular reinforcement-learning framework designed for customization.
By breaking down complex algorithms into minimal components, it allows users to easily modify
or integrate components instead of rebuilding the entire algorithm from scratch, making it
particularly well-suited for advancing robotics learning.
> **Note:** This project is under **active development**, which means the interface is unstable
and breaking changes are likely to occur frequently.
## Setup
CusRL requires Python 3.10 or later. It can be installed via PyPI with:
```bash
# Choose one of the following:
# 1. Minimal installation
pip install cusrl
# 2. Install with export and logging utilities
pip install cusrl[all]
```
or by cloning this repository and installing it with:
```bash
git clone https://github.com/chengruiz/cusrl.git
# Choose one of the following:
# 1. Minimal installation
pip install -e . --config-settings editable_mode=strict
# 2. Install with optional dependencies
pip install -e .[all] --config-settings editable_mode=strict
# 3. Install dependencies for development
pip install -e .[dev] --config-settings editable_mode=strict
pre-commit install
```
## Quick Start
List all available experiments:
```bash
python -m cusrl list-experiments
```
Train a PPO agent and evaluate it:
```bash
python -m cusrl train -env MountainCar-v0 -alg ppo --logger tensorboard --seed 42
python -m cusrl play --checkpoint logs/MountainCar-v0:ppo
```
Or if you have [IssacLab](https://github.com/isaac-sim/IsaacLab) installed:
```bash
python -m cusrl train -env Isaac-Velocity-Rough-Anymal-C-v0 -alg ppo \
--logger tensorboard --environment-args="--headless"
python -m cusrl play --checkpoint logs/Isaac-Velocity-Rough-Anymal-C-v0:ppo
```
Try distributed training:
```bash
torchrun --nproc-per-node=2 -m cusrl train -env Isaac-Velocity-Rough-Anymal-C-v0 \
-alg ppo --logger tensorboard --environment-args="--headless"
```
## Highlights
CusRL provides a modular and extensible framework for RL with the following key features:
- **Modular Design**: Components are highly decoupled, allowing for free combination and customization
- **Diverse Network Architectures**: Support for MLP, CNN, RNNs, Transformer and custom architectures
- **Modern Training Techniques**: Built-in support for distributed and mixed-precision training
## Implemented Algorithms
- [Adversarial Motion Prior (AMP)](https://dl.acm.org/doi/10.1145/3450626.3459670)
- [Generalized Advantage Estimation (GAE)](https://arxiv.org/abs/1506.02438)
with [distinct lambda values](https://proceedings.neurips.cc/paper_files/paper/2022/hash/e95475f5fb8edb9075bf9e25670d4013-Abstract-Conference.html)
- [Preserving Outputs Precisely, while Adaptively Rescaling Targets (Pop-Art)](https://proceedings.neurips.cc/paper/2016/hash/5227b6aaf294f5f027273aebf16015f2-Abstract.html)
- [Proximal Policy Optimization (PPO)](https://arxiv.org/abs/1707.06347) with recurrent policy support
- [Random Network Distillation (RND)](https://arxiv.org/abs/1810.12894)
- Symmetry Augmentations:
[Symmetric Architecture](https://dl.acm.org/doi/abs/10.1145/3359566.3360070),
[Symmetric Data Augmentation](https://ieeexplore.ieee.org/abstract/document/10611493),
[Symmetry Loss](https://dl.acm.org/doi/abs/10.1145/3197517.3201397)
## Cite
If you find this framework useful for your research, please consider citing our work on legged locomotion:
- [Efficient Learning of A Unified Policy For Whole-body Manipulation and Locomotion Skills](https://www.arxiv.org/abs/2507.04229), IROS 2025
- [Learning Accurate and Robust Velocity Tracking for Quadrupedal Robots](https://onlinelibrary.wiley.com/doi/10.1002/rob.70028), JFR 2025
- [Learning Safe Locomotion for Quadrupedal Robots by Derived-Action Optimization](https://ieeexplore.ieee.org/abstract/document/10802725), IROS 2024
## Acknowledgement
CusRL is based on or inspired by these projects:
- [Stable Baselines3](https://github.com/DLR-RM/stable-baselines3): Reliable implementations of reinforcement learning algorithms
- [RSL RL](https://github.com/leggedrobotics/rsl_rl): Fast and simple implementation of RL algorithms
- [IsaacLab](https://github.com/isaac-sim/IsaacLab): GPU-accelerated simulation environments for robot research
- [robot_lab](https://github.com/fan-ziqi/robot_lab): RL extension library for robots, based on IsaacLab
Raw data
{
"_id": null,
"home_page": null,
"name": "cusrl",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "reinforcement-learning, pytorch, rl",
"author": null,
"author_email": "Chengrui Zhu <jewel@zju.edu.cn>",
"download_url": "https://files.pythonhosted.org/packages/7d/61/ff7a978e2385b93f04291c500c3c88cf04b11d7a938eb9c951d6d8af6223/cusrl-1.1.0.tar.gz",
"platform": null,
"description": "# CusRL: Customizable Reinforcement Learning\n\nCusRL is a flexible and modular reinforcement-learning framework designed for customization.\nBy breaking down complex algorithms into minimal components, it allows users to easily modify\nor integrate components instead of rebuilding the entire algorithm from scratch, making it\nparticularly well-suited for advancing robotics learning.\n\n> **Note:** This project is under **active development**, which means the interface is unstable\nand breaking changes are likely to occur frequently.\n\n## Setup\n\nCusRL requires Python 3.10 or later. It can be installed via PyPI with:\n\n```bash\n# Choose one of the following:\n# 1. Minimal installation\npip install cusrl\n# 2. Install with export and logging utilities\npip install cusrl[all]\n```\n\nor by cloning this repository and installing it with:\n\n```bash\ngit clone https://github.com/chengruiz/cusrl.git\n# Choose one of the following:\n# 1. Minimal installation\npip install -e . --config-settings editable_mode=strict\n# 2. Install with optional dependencies\npip install -e .[all] --config-settings editable_mode=strict\n# 3. Install dependencies for development\npip install -e .[dev] --config-settings editable_mode=strict\npre-commit install\n```\n\n## Quick Start\n\nList all available experiments:\n\n```bash\npython -m cusrl list-experiments\n```\n\nTrain a PPO agent and evaluate it:\n\n```bash\npython -m cusrl train -env MountainCar-v0 -alg ppo --logger tensorboard --seed 42\npython -m cusrl play --checkpoint logs/MountainCar-v0:ppo\n```\n\nOr if you have [IssacLab](https://github.com/isaac-sim/IsaacLab) installed:\n\n```bash\npython -m cusrl train -env Isaac-Velocity-Rough-Anymal-C-v0 -alg ppo \\\n --logger tensorboard --environment-args=\"--headless\"\npython -m cusrl play --checkpoint logs/Isaac-Velocity-Rough-Anymal-C-v0:ppo\n```\n\nTry distributed training:\n\n```bash\ntorchrun --nproc-per-node=2 -m cusrl train -env Isaac-Velocity-Rough-Anymal-C-v0 \\\n -alg ppo --logger tensorboard --environment-args=\"--headless\"\n```\n\n## Highlights\n\nCusRL provides a modular and extensible framework for RL with the following key features:\n\n- **Modular Design**: Components are highly decoupled, allowing for free combination and customization\n- **Diverse Network Architectures**: Support for MLP, CNN, RNNs, Transformer and custom architectures\n- **Modern Training Techniques**: Built-in support for distributed and mixed-precision training\n\n## Implemented Algorithms\n\n- [Adversarial Motion Prior (AMP)](https://dl.acm.org/doi/10.1145/3450626.3459670)\n- [Generalized Advantage Estimation (GAE)](https://arxiv.org/abs/1506.02438)\n with [distinct lambda values](https://proceedings.neurips.cc/paper_files/paper/2022/hash/e95475f5fb8edb9075bf9e25670d4013-Abstract-Conference.html)\n- [Preserving Outputs Precisely, while Adaptively Rescaling Targets (Pop-Art)](https://proceedings.neurips.cc/paper/2016/hash/5227b6aaf294f5f027273aebf16015f2-Abstract.html)\n- [Proximal Policy Optimization (PPO)](https://arxiv.org/abs/1707.06347) with recurrent policy support\n- [Random Network Distillation (RND)](https://arxiv.org/abs/1810.12894)\n- Symmetry Augmentations:\n [Symmetric Architecture](https://dl.acm.org/doi/abs/10.1145/3359566.3360070),\n [Symmetric Data Augmentation](https://ieeexplore.ieee.org/abstract/document/10611493),\n [Symmetry Loss](https://dl.acm.org/doi/abs/10.1145/3197517.3201397)\n\n## Cite\n\nIf you find this framework useful for your research, please consider citing our work on legged locomotion:\n\n- [Efficient Learning of A Unified Policy For Whole-body Manipulation and Locomotion Skills](https://www.arxiv.org/abs/2507.04229), IROS 2025\n- [Learning Accurate and Robust Velocity Tracking for Quadrupedal Robots](https://onlinelibrary.wiley.com/doi/10.1002/rob.70028), JFR 2025\n- [Learning Safe Locomotion for Quadrupedal Robots by Derived-Action Optimization](https://ieeexplore.ieee.org/abstract/document/10802725), IROS 2024\n\n## Acknowledgement\n\nCusRL is based on or inspired by these projects:\n\n- [Stable Baselines3](https://github.com/DLR-RM/stable-baselines3): Reliable implementations of reinforcement learning algorithms\n- [RSL RL](https://github.com/leggedrobotics/rsl_rl): Fast and simple implementation of RL algorithms\n- [IsaacLab](https://github.com/isaac-sim/IsaacLab): GPU-accelerated simulation environments for robot research\n- [robot_lab](https://github.com/fan-ziqi/robot_lab): RL extension library for robots, based on IsaacLab\n",
"bugtrack_url": null,
"license": null,
"summary": "Customizable and modular RL algorithms implemented in PyTorch",
"version": "1.1.0",
"project_urls": null,
"split_keywords": [
"reinforcement-learning",
" pytorch",
" rl"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "b335de6a594f23aa2c075f55c344b4b958c401a93207912cc3d925a0d75d12ce",
"md5": "51b7eb14f91dd8a52f861cfb856211d7",
"sha256": "58149020672c48f028ec3d0f4ae69d07d6d71c9df0a310b297c889ac3611d3d6"
},
"downloads": -1,
"filename": "cusrl-1.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "51b7eb14f91dd8a52f861cfb856211d7",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 123976,
"upload_time": "2025-08-03T15:37:26",
"upload_time_iso_8601": "2025-08-03T15:37:26.880188Z",
"url": "https://files.pythonhosted.org/packages/b3/35/de6a594f23aa2c075f55c344b4b958c401a93207912cc3d925a0d75d12ce/cusrl-1.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "7d61ff7a978e2385b93f04291c500c3c88cf04b11d7a938eb9c951d6d8af6223",
"md5": "a263ed2869884a35516f8a45fb4fa35b",
"sha256": "8b13f2ac01350cc7aa97d981a99c3fa2b64f239c5a9251b6c46cce763920f55e"
},
"downloads": -1,
"filename": "cusrl-1.1.0.tar.gz",
"has_sig": false,
"md5_digest": "a263ed2869884a35516f8a45fb4fa35b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 105078,
"upload_time": "2025-08-03T15:37:28",
"upload_time_iso_8601": "2025-08-03T15:37:28.564439Z",
"url": "https://files.pythonhosted.org/packages/7d/61/ff7a978e2385b93f04291c500c3c88cf04b11d7a938eb9c951d6d8af6223/cusrl-1.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-03 15:37:28",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "cusrl"
}
