[](https://github.com/ahmedheakl/Mutli_Level_RL_Robotics/actions)
# Highrl: Multi-level Reinforcement Learning for Robotics Navigation
Highrl is a library for training robots using RL under the scheme of multi-level RL. The library has numerous features from generating random environment, training agents to generate curriculum learning schemes, or train robots in pre-defined environments.
The robot can explore a synthetic [gym](https://www.gymlibrary.dev/) environment using lidar vision (either in flat or rings format). The robot is trained to reach the goal with obstacles in between to hinder its movement and simulate the real-world environment. A teacher (an RL agent) will be trained to synthesize the perfect curriculum learning for the robot, so that the robot will solve maps with certain difficulties in minimal time.
The robot model is implemented with a [CNN](https://en.wikipedia.org/wiki/Convolutional_neural_network) for the feature extractor and an [MLP](https://en.wikipedia.org/wiki/Multilayer_perceptron) for both the value and policy networks. The teacher model is implemented with an LSTM network for the feature extractor and an MLP for value/policy network. The robot model is fed with the lidar data and outputs the velocity ```(vx, vy)``` of the robot. The teacher model is fed with data of the last session for the robot that the teacher is training, and outputs the configurations for the next environment to train the robot. At each step of the teacher, a new robot will be generated with probability of 10% and will be trained for a fixed number of steps. You can find the models in ```src/highrl/policy```.
## Installation
Please note that the library is **only tested on Linux distributions**. If you want to install it on Windows, you can use [WSL](https://learn.microsoft.com/en-us/windows/wsl/install).
Use the package manager [pip](https://pip.pypa.io/en/stable/) to install higrl library.
```bash
pip install highrl
```
## Usage
```bash
highrl -h # get available arguments
```
### Configurations
**--robot-config**: path of configuration file of robot environment (relative path).
**--teacher-config**: path of configuration file of teacher environment (relative path).
**--mode**: choose train or test mode.
**--env-mode**: choose whether to train the robot alone or in the presence of a teacher to generate curriculum learning.
**--render-each**: the frequency of rendering for robot environment (integer).
**--output-path**: relative path to output results for robot mode.
**--lidar-mode**: mode to process lidar flat=1D, rings=2D.
### Example
```bash
highrl --render-each=50 --output-dir=~/Desktop
```
<p align="center">
<img height="300" width="500" src="https://github.com/ahmedheakl/multi-level-rl-for-robotics/blob/main/imgs/highrl-img.png">
</p>
## Contributing
Pull requests are welcome. For major changes, please open an issue first
to discuss what you would like to change.
Please make sure to update tests as appropriate.
## License
[MIT](https://choosealicense.com/licenses/mit/)
Raw data
{
"_id": null,
"home_page": "",
"name": "highrl",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "reinforcement learning,robots,robot navigation,hierarchical reinforcement learning",
"author": "",
"author_email": "Ahmed Heakl <ahmed.heakl@ejust.edu.eg>",
"download_url": "https://files.pythonhosted.org/packages/0d/0b/9bf2b1597ce10185480cabe2c026d72a23dc55605474aecfa34e9ff341c5/highrl-0.1.5.tar.gz",
"platform": null,
"description": "[](https://github.com/ahmedheakl/Mutli_Level_RL_Robotics/actions)\n\n# Highrl: Multi-level Reinforcement Learning for Robotics Navigation\n\nHighrl is a library for training robots using RL under the scheme of multi-level RL. The library has numerous features from generating random environment, training agents to generate curriculum learning schemes, or train robots in pre-defined environments.\n\nThe robot can explore a synthetic [gym](https://www.gymlibrary.dev/) environment using lidar vision (either in flat or rings format). The robot is trained to reach the goal with obstacles in between to hinder its movement and simulate the real-world environment. A teacher (an RL agent) will be trained to synthesize the perfect curriculum learning for the robot, so that the robot will solve maps with certain difficulties in minimal time. \n\nThe robot model is implemented with a [CNN](https://en.wikipedia.org/wiki/Convolutional_neural_network) for the feature extractor and an [MLP](https://en.wikipedia.org/wiki/Multilayer_perceptron) for both the value and policy networks. The teacher model is implemented with an LSTM network for the feature extractor and an MLP for value/policy network. The robot model is fed with the lidar data and outputs the velocity ```(vx, vy)``` of the robot. The teacher model is fed with data of the last session for the robot that the teacher is training, and outputs the configurations for the next environment to train the robot. At each step of the teacher, a new robot will be generated with probability of 10% and will be trained for a fixed number of steps. You can find the models in ```src/highrl/policy```. \n\n\n## Installation\n\nPlease note that the library is **only tested on Linux distributions**. If you want to install it on Windows, you can use [WSL](https://learn.microsoft.com/en-us/windows/wsl/install).\n\nUse the package manager [pip](https://pip.pypa.io/en/stable/) to install higrl library.\n\n```bash\npip install highrl\n```\n\n## Usage\n\n```bash\nhighrl -h # get available arguments\n```\n### Configurations\n**--robot-config**: path of configuration file of robot environment (relative path). \n\n**--teacher-config**: path of configuration file of teacher environment (relative path). \n\n**--mode**: choose train or test mode.\n\n**--env-mode**: choose whether to train the robot alone or in the presence of a teacher to generate curriculum learning. \n\n**--render-each**: the frequency of rendering for robot environment (integer).\n\n**--output-path**: relative path to output results for robot mode. \n\n**--lidar-mode**: mode to process lidar flat=1D, rings=2D.\n\n### Example\n```bash\nhighrl --render-each=50 --output-dir=~/Desktop\n```\n\n<p align=\"center\">\n <img height=\"300\" width=\"500\" src=\"https://github.com/ahmedheakl/multi-level-rl-for-robotics/blob/main/imgs/highrl-img.png\">\n</p>\n\n\n## Contributing\n\nPull requests are welcome. For major changes, please open an issue first\nto discuss what you would like to change.\n\nPlease make sure to update tests as appropriate.\n\n## License\n\n[MIT](https://choosealicense.com/licenses/mit/)\n",
"bugtrack_url": null,
"license": "",
"summary": "Two level RL for robots navigation",
"version": "0.1.5",
"split_keywords": [
"reinforcement learning",
"robots",
"robot navigation",
"hierarchical reinforcement learning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "775f61635e57c1f8dbf40afe51192414fbb81af0b091be16a1e9d41a23c4ed3c",
"md5": "4901b2431d196bc74c75e65537fdaa3d",
"sha256": "28eee5e2f04b421533bec7d440215307e0b0d8684441f6a12d14ef2be6567cf4"
},
"downloads": -1,
"filename": "highrl-0.1.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
"has_sig": false,
"md5_digest": "4901b2431d196bc74c75e65537fdaa3d",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.7",
"size": 449587,
"upload_time": "2023-01-09T10:34:25",
"upload_time_iso_8601": "2023-01-09T10:34:25.324665Z",
"url": "https://files.pythonhosted.org/packages/77/5f/61635e57c1f8dbf40afe51192414fbb81af0b091be16a1e9d41a23c4ed3c/highrl-0.1.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "0d0b9bf2b1597ce10185480cabe2c026d72a23dc55605474aecfa34e9ff341c5",
"md5": "a7db5cf3101a7e49035c761eb4f1bd80",
"sha256": "45642b3f9d27c22fbad9fd80ba922271d434d5cbaaf5b67f5c093281a05a89ba"
},
"downloads": -1,
"filename": "highrl-0.1.5.tar.gz",
"has_sig": false,
"md5_digest": "a7db5cf3101a7e49035c761eb4f1bd80",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 152041,
"upload_time": "2023-01-09T10:34:26",
"upload_time_iso_8601": "2023-01-09T10:34:26.946165Z",
"url": "https://files.pythonhosted.org/packages/0d/0b/9bf2b1597ce10185480cabe2c026d72a23dc55605474aecfa34e9ff341c5/highrl-0.1.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-01-09 10:34:26",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "highrl"
}
JSON