<h2 align="center">Stable Reinforcement Learning for Simulink</h2>
<p align="center">
<a href="https://stablerls.readthedocs.io/en/latest/?badge=latest"><img alt="Documentation Status" src="https://readthedocs.org/projects/stablerls/badge/?version=latest"></a>
<a href="https://github.com/psf/black"><img alt="Code style: black" src="https://img.shields.io/badge/code%20style-black-000000.svg"></a>
<a href="https://joss.theoj.org/papers/5d136ccade14ad7e9e1a9c8172725647"><img src="https://joss.theoj.org/papers/5d136ccade14ad7e9e1a9c8172725647/status.svg"></a>
</p>
StableRLS is a software package that lets you use your existing MATLAB Simulink models in Python for reinforcement learning. Basically, your simulation is wrapped in a Python [Gymnasium](https://gymnasium.farama.org/) environment. The package provides the following features:
- automatic generation of input and output signals for your model
- automatic compilation of the Simulink model into a functional-mockup-unit (FMU) to enable fast simulation
- flexible post-processing implementation
- easy-to-read code
**And the best part is** that the only thing you need to do is:
- define a reward function to train your agent
## General Information
Reinforcement Learning (RL) is a rapidly changing and innovative field. The main purpose of this package is to combine the easy-to-use MATLAB Simulink modeling interface with the flexible and state-of-the-art Gymnasium interface. Therefore, the RL algorithm and the learning interface are out of scope for this package. However, we make the interface between Matlab and Python as easy as possible.
## Installation
This package is currently tested with Python 3.9.
To install the package, run `pip install StableRLS`. **You have to install the MATLAB engine seperately** because each MATLAB release has some specific requirements (see below).
You can also clone this repository and run `pip install -e StableRLS/` from the main directory. This will also install the main dependencies, which are listed in `requirements.txt`. To actively contribute, you should also install the `optional-requirements.txt`, which also includes the dependencies for building the documentation, by running `pip install -r optional-requirements.txt`.
We decided to exclude the typical machine learning frameworks (PyTorch, Tensorflow) from the requirements, because everyone has their own preferences and we want to keep this package small. But some of our example are based on PyTorch, so you need to run `pip install torch` if you want to run them locally. This will also be mentioned in the examples. To compile the documentation locally, you need to have Pandoc installed on your computer.
### Matlab Version
The MATLAB engine Python package is a requirement to compile a given MATLAB Simulink model. the MATLAB release R2022b it was inconvenient to install the engine, see the [instructions](https://de.mathworks.com/help/matlab/matlab_external/install-the-matlab-engine-for-python.html). After the release, it's possible to install the engine as a pip package. StableRLS won't try to install the MATLAB engine as dependency because the pip package only supports the newest MATLAB release. Currently, you can run `pip install matlabengine` if you have MATLAB 2023a installed, if you have MATLAB 2022b installed run `pip install matlabengine==9.13.7`. For other releases refer to the documentation mentioned.
## Get Started
Check out our examples (/examples) or the documentation, which also contains the examples.
## Contribution and other issues
We are researchers in the field of electrical engineering, but this package is also useful for other engineers who use MATLAB Simulink as part of their research. If you would like to contribute to the development of this tool, please create issues or pull requests.
If you have problems installing or using this tool, you can also create an issue. For more information about contributions and issues, take a look at "HOW_TO_CONTRIBUTE.md" located at the top of the repository.
## Building the documentation
Run `sphinx-autobuild docs/ docs/build/html` from the main directory and open the documentation `localhost:8000`. The page is updated automatically after any file in the documentation is changed.
## More information
To install clean use `pip freeze | grep -v -f requirements.txt - | grep -v '^#' | grep -v '^-e ' | xargs pip uninstall -y` to remove all packages except the one with -e flag. Afterwards `pip install -r requirements.txt` or `pip install -r optional-requirements.txt`.
## Make this public
- [ ] [https://joss.theoj.org/about](https://joss.theoj.org/about) Submit
Raw data
{
"_id": null,
"home_page": "https://github.com/rAnnuth/stablerls",
"name": "StableRLS",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "simulation RL reinforcement learning Simulink matlab FMU",
"author": "Robert Annuth",
"author_email": "robert.annuth@tuhh.de",
"download_url": "https://files.pythonhosted.org/packages/2c/b1/ab90d580c47afa5b8117a09377df4666ebde892a598f5a77609d42ff5599/StableRLS-1.0.4.tar.gz",
"platform": null,
"description": "\n\n<h2 align=\"center\">Stable Reinforcement Learning for Simulink</h2>\n\n<p align=\"center\">\n<a href=\"https://stablerls.readthedocs.io/en/latest/?badge=latest\"><img alt=\"Documentation Status\" src=\"https://readthedocs.org/projects/stablerls/badge/?version=latest\"></a>\n<a href=\"https://github.com/psf/black\"><img alt=\"Code style: black\" src=\"https://img.shields.io/badge/code%20style-black-000000.svg\"></a>\n<a href=\"https://joss.theoj.org/papers/5d136ccade14ad7e9e1a9c8172725647\"><img src=\"https://joss.theoj.org/papers/5d136ccade14ad7e9e1a9c8172725647/status.svg\"></a>\n</p>\n\n\nStableRLS is a software package that lets you use your existing MATLAB Simulink models in Python for reinforcement learning. Basically, your simulation is wrapped in a Python [Gymnasium](https://gymnasium.farama.org/) environment. The package provides the following features:\n- automatic generation of input and output signals for your model\n- automatic compilation of the Simulink model into a functional-mockup-unit (FMU) to enable fast simulation\n- flexible post-processing implementation\n- easy-to-read code\n\n**And the best part is** that the only thing you need to do is:\n- define a reward function to train your agent\n\n\n## General Information\nReinforcement Learning (RL) is a rapidly changing and innovative field. The main purpose of this package is to combine the easy-to-use MATLAB Simulink modeling interface with the flexible and state-of-the-art Gymnasium interface. Therefore, the RL algorithm and the learning interface are out of scope for this package. However, we make the interface between Matlab and Python as easy as possible.\n\n## Installation\nThis package is currently tested with Python 3.9.\nTo install the package, run `pip install StableRLS`. **You have to install the MATLAB engine seperately** because each MATLAB release has some specific requirements (see below).\n\nYou can also clone this repository and run `pip install -e StableRLS/` from the main directory. This will also install the main dependencies, which are listed in `requirements.txt`. To actively contribute, you should also install the `optional-requirements.txt`, which also includes the dependencies for building the documentation, by running `pip install -r optional-requirements.txt`.\n\nWe decided to exclude the typical machine learning frameworks (PyTorch, Tensorflow) from the requirements, because everyone has their own preferences and we want to keep this package small. But some of our example are based on PyTorch, so you need to run `pip install torch` if you want to run them locally. This will also be mentioned in the examples. To compile the documentation locally, you need to have Pandoc installed on your computer.\n\n\n### Matlab Version\nThe MATLAB engine Python package is a requirement to compile a given MATLAB Simulink model. the MATLAB release R2022b it was inconvenient to install the engine, see the [instructions](https://de.mathworks.com/help/matlab/matlab_external/install-the-matlab-engine-for-python.html). After the release, it's possible to install the engine as a pip package. StableRLS won't try to install the MATLAB engine as dependency because the pip package only supports the newest MATLAB release. Currently, you can run `pip install matlabengine` if you have MATLAB 2023a installed, if you have MATLAB 2022b installed run `pip install matlabengine==9.13.7`. For other releases refer to the documentation mentioned.\n\n## Get Started\nCheck out our examples (/examples) or the documentation, which also contains the examples.\n\n## Contribution and other issues\nWe are researchers in the field of electrical engineering, but this package is also useful for other engineers who use MATLAB Simulink as part of their research. If you would like to contribute to the development of this tool, please create issues or pull requests.\nIf you have problems installing or using this tool, you can also create an issue. For more information about contributions and issues, take a look at \"HOW_TO_CONTRIBUTE.md\" located at the top of the repository.\n\n## Building the documentation\nRun `sphinx-autobuild docs/ docs/build/html` from the main directory and open the documentation `localhost:8000`. The page is updated automatically after any file in the documentation is changed.\n\n## More information\nTo install clean use `pip freeze | grep -v -f requirements.txt - | grep -v '^#' | grep -v '^-e ' | xargs pip uninstall -y` to remove all packages except the one with -e flag. Afterwards `pip install -r requirements.txt` or `pip install -r optional-requirements.txt`.\n\n## Make this public\n- [ ] [https://joss.theoj.org/about](https://joss.theoj.org/about) Submit\n",
"bugtrack_url": null,
"license": "LICENSE.txt",
"summary": "Simulate Simulink FMUs as Gymnasium environment",
"version": "1.0.4",
"project_urls": {
"Homepage": "https://github.com/rAnnuth/stablerls"
},
"split_keywords": [
"simulation",
"rl",
"reinforcement",
"learning",
"simulink",
"matlab",
"fmu"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2cb1ab90d580c47afa5b8117a09377df4666ebde892a598f5a77609d42ff5599",
"md5": "45a2a3bd6f5dd7f124a1f0e65800c457",
"sha256": "c53738d8470a81f2e1c8b0eb144235458bc9d8ddb3484a57d887aa64d7d83f20"
},
"downloads": -1,
"filename": "StableRLS-1.0.4.tar.gz",
"has_sig": false,
"md5_digest": "45a2a3bd6f5dd7f124a1f0e65800c457",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 27352,
"upload_time": "2023-08-11T10:39:20",
"upload_time_iso_8601": "2023-08-11T10:39:20.832534Z",
"url": "https://files.pythonhosted.org/packages/2c/b1/ab90d580c47afa5b8117a09377df4666ebde892a598f5a77609d42ff5599/StableRLS-1.0.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-08-11 10:39:20",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "rAnnuth",
"github_project": "stablerls",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "attrs",
"specs": [
[
"==",
"22.2.0"
]
]
},
{
"name": "cloudpickle",
"specs": [
[
"==",
"2.2.1"
]
]
},
{
"name": "exceptiongroup",
"specs": [
[
"==",
"1.1.1"
]
]
},
{
"name": "Farama-Notifications",
"specs": [
[
"==",
"0.0.4"
]
]
},
{
"name": "FMPy",
"specs": [
[
"==",
"0.3.15"
]
]
},
{
"name": "gymnasium",
"specs": [
[
"==",
"0.28.1"
]
]
},
{
"name": "importlib-metadata",
"specs": [
[
"==",
"6.1.0"
]
]
},
{
"name": "iniconfig",
"specs": [
[
"==",
"2.0.0"
]
]
},
{
"name": "jax-jumpy",
"specs": [
[
"==",
"1.0.0"
]
]
},
{
"name": "Jinja2",
"specs": [
[
"==",
"3.1.2"
]
]
},
{
"name": "lark",
"specs": [
[
"==",
"1.1.5"
]
]
},
{
"name": "lxml",
"specs": [
[
"==",
"4.9.2"
]
]
},
{
"name": "MarkupSafe",
"specs": [
[
"==",
"2.1.2"
]
]
},
{
"name": "msgpack",
"specs": [
[
"==",
"1.0.5"
]
]
},
{
"name": "numpy",
"specs": [
[
"==",
"1.24.2"
]
]
},
{
"name": "packaging",
"specs": [
[
"==",
"23.0"
]
]
},
{
"name": "pluggy",
"specs": [
[
"==",
"1.0.0"
]
]
},
{
"name": "pytest",
"specs": [
[
"==",
"7.2.2"
]
]
},
{
"name": "pytest-ordering",
"specs": [
[
"==",
"0.6"
]
]
},
{
"name": "pytz",
"specs": [
[
"==",
"2023.3"
]
]
},
{
"name": "tomli",
"specs": [
[
"==",
"2.0.1"
]
]
},
{
"name": "typing_extensions",
"specs": [
[
"==",
"4.5.0"
]
]
},
{
"name": "zipp",
"specs": [
[
"==",
"3.15.0"
]
]
}
],
"lcname": "stablerls"
}
JSON
