# Sinergym
<div align="center">
<img src="images/logo.png" width=40%><br><br>
</div>
</p>
<p align="center">
<a href="https://github.com/ugr-sail/sinergym/releases">
<img alt="Github latest release" src="https://img.shields.io/github/release-date/ugr-sail/sinergym" />
</a>
<a href="https://github.com/ugr-sail/sinergym/commits/main">
<img alt="Github last commit" src="https://img.shields.io/github/last-commit/ugr-sail/sinergym" />
</a>
<a href="https://pypi.org/project/sinergym/">
<img alt="Pypi version" src="https://img.shields.io/pypi/v/sinergym" />
</a>
<a href="https://github.com/ugr-sail/sinergym/stargazers">
<img alt="Pypi downloads" src="https://img.shields.io/pypi/dm/sinergym" />
</a>
<a href="https://codecov.io/gh/ugr-sail/sinergym">
<img src="https://codecov.io/gh/ugr-sail/sinergym/branch/main/graph/badge.svg" />
</a>
<a href="https://github.com/ugr-sail/sinergym/graphs/contributors">
<img alt="GitHub Contributors" src="https://img.shields.io/github/contributors/ugr-sail/sinergym" />
</a>
<a href="https://github.com/ugr-sail/sinergym/issues">
<img alt="Github issues" src="https://img.shields.io/github/issues/ugr-sail/sinergym?color=0088ff" />
</a>
<a href="https://github.com/ugr-sail/sinergym/pulls">
<img alt="GitHub pull requests" src="https://img.shields.io/github/issues-pr/ugr-sail/sinergym?color=0088ff" />
</a>
<a href="https://github.com/ugr-sail/sinergym/blob/main/LICENSE">
<img alt="Github License" src="https://img.shields.io/github/license/ugr-sail/sinergym" />
</a>
<a href="https://www.python.org/downloads/release/python-3100/">
<img alt="Pypi Python version" src="https://img.shields.io/pypi/pyversions/sinergym" />
</a>
<br />
<br />
<a href="https://code.visualstudio.com/">
<img src="https://img.shields.io/badge/Supported%20by-VSCode%20Power%20User%20%E2%86%92-gray.svg?colorA=655BE1&colorB=4F44D6&style=for-the-badge"/>
</a>
</p>
**Welcome to Sinergym!**
<div align="center">
<img src="images/general_blueprint.png" width=80%><br><br>
</div>
The goal of this project is to create an environment following *Gymnasium* interface for wrapping
simulation engines (*EnergyPlus*) for building control using **deep reinforcement learning**
or any external control.
For more information about Sinergym, please visit our [documentation](https://ugr-sail.github.io/sinergym/compilation/main/index.html).
To report questions and issues, please use our [issue tracker](https://github.com/ugr-sail/sinergym/issues). We appreciate your feedback and contributions. Check out our [CONTRIBUTING.md](https://github.com/ugr-sail/sinergym/blob/main/CONTRIBUTING.md) for more details on how to contribute.
The main functionalities of *Sinergym* are the following:
- **Simulation Engine Compatibility**: Uses [EnergyPlus Python API](https://energyplus.readthedocs.io/en/latest/api.html) for Python-EnergyPlus communication. Future plans include more engines like [OpenModelica](https://openmodelica.org/).
- **Benchmark Environments**: Designs environments for benchmarking and testing deep RL algorithms or other external strategies, similar to *Atari* or *Mujoco*.
- **Customizable Environments**: Allows easy modification of experimental settings. Users can create their own environments or modify pre-configured ones in *Sinergym*.
- **Customizable Components**: Enables creation of new custom components for new environments, making *Sinergym* scalable, such as function rewards, wrappers, controllers, etc.
- **Automatic Building Model Adaptation**: *Sinergym* automates the process of adapting the building model to user changes in the environment definition.
- **Automatic Actuators Control**: Controls actuators through the Gymnasium interface based on user specification, only actuators names are required and *Sinergym* will do the rest.
- **Extensive Environment Information**: Provides comprehensive information about *Sinergym* background components from the environment interface.
- **Stable Baseline 3 Integration**: Customizes functionalities for easy testing of environments with SB3 algorithms, such as callbacks and customizable training real-time logging. However, *Sinergym* is agnostic to any DRL algorithm.
- **Google Cloud Integration**: Offers guidance on using *Sinergym* with Google Cloud infrastructure.
- **Weights & Biases Compatibility**: Automates and facilitates training, reproducibility, and comparison of agents in simulation-based building control problems. [WandB](https://wandb.ai/site) assists in managing and monitoring model lifecycle.
- **Notebook Examples**: Provides code in notebook format for user familiarity with the tool.
- **Extensive Documentation, Unit Tests, and GitHub Actions Workflows**: Ensures *Sinergym* is an efficient ecosystem for understanding and development.
- And much more!
_This is a project in active development. Stay tuned for upcoming releases._
<div align="center">
<img src="images/operation_diagram.png"><br><br>
</div>
## Project Structure
This repository is organized into the following directories:
- `sinergym/`: Contains the source code for *Sinergym*, including the environment, modeling, simulator, and tools such as wrappers and reward functions.
- `docs/`: Online documentation generated with Sphinx and using Restructured Text (RST).
- `examples/`: Jupyter notebooks illustrating use cases with Sinergym.
- `tests/`: Unit tests for Sinergym to ensure stability.
- `scripts/`: Scripts for various tasks such as agent training and performance checks, allowing configuration using JSON format.
## Available Environments
For a complete and up-to-date list of available environments, please refer to [our documentation](https://ugr-sail.github.io/sinergym/compilation/main/pages/environments.html#).
## Installation
Please visit [INSTALL.md](https://github.com/ugr-sail/sinergym/blob/main/INSTALL.md) for detailed installation instructions.
## Usage example
If you used our Dockerfile during installation, you should have the *try_env.py* file in your workspace as soon as you enter in. In case you have installed everything on your local machine directly, place it inside our cloned repository. In any case, we start from the point that you have at your disposal a terminal with the appropriate python version and *Sinergym* running correctly.
*Sinergym* uses the standard Gymnasium API. So a basic loop should look like:
```python
import gymnasium as gym
import sinergym
# Create the environment
env = gym.make('Eplus-datacenter-mixed-continuous-stochastic-v1')
# Initialize the episode
obs, info = env.reset()
truncated = terminated = False
R = 0.0
while not (terminated or truncated):
a = env.action_space.sample() # random action selection
obs, reward, terminated, truncated, info = env.step(a) # get new observation and reward
R += reward
print('Total reward for the episode: %.4f' % R)
env.close()
```
A folder will be created in the working directory after creating the environment. It will contain the Sinergym outputs produced during the simulation.
For more examples and details, please visit our [usage examples](https://ugr-sail.github.io/sinergym/compilation/main/pages/notebooks/basic_example.html#Basic-example) documentation section.
## Google Cloud Platform support
For more information about this functionality, please, visit our documentation [here](https://ugr-sail.github.io/sinergym/compilation/main/pages/gcloudAPI.html#sinergym-with-google-cloud).
## Projects using Sinergym
The following are some of the projects benefiting from the advantages of Sinergym:
- [Demosthen/ActiveRL](https://github.com/Demosthen/ActiveRL)
- [VectorInstitute/HV-Ai-C](https://github.com/VectorInstitute/HV-Ai-C)
- [rdnfn/beobench](https://github.com/rdnfn/beobench)
:pencil: If you want to appear in this list, do not hesitate to send us a PR and include the following badge in your repository:
<p align="center">
<a href="https://github.com/ugr-sail/sinergym">
<img src="https://img.shields.io/badge/Powered%20by-Sinergym%20%E2%86%92-gray.svg?colorA=00BABF&colorB=4BF2F7&style=for-the-badge"/>
</a>
</p>
## Repo Activity
## Citing Sinergym
If you use *Sinergym* in your work, please cite our [paper](https://dl.acm.org/doi/abs/10.1145/3486611.3488729):
```bibtex
@inproceedings{2021sinergym,
title={Sinergym: A Building Simulation and Control Framework for Training Reinforcement Learning Agents},
author={Jiménez-Raboso, Javier and Campoy-Nieves, Alejandro and Manjavacas-Lucas, Antonio and Gómez-Romero, Juan and Molina-Solana, Miguel},
year={2021},
isbn = {9781450391146},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3486611.3488729},
doi = {10.1145/3486611.3488729},
booktitle = {Proceedings of the 8th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation},
pages = {319–323},
numpages = {5},
}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/ugr-sail/sinergym",
"name": "sinergym",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "control reinforcement-learning buildings reinforcement-learning-environments",
"author": "J. Jim\u00e9nez, J. G\u00f3mez, M. Molina, A. Manjavacas, A. Campoy",
"author_email": "alejandroac79@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/59/c8/435aee34a038d8a7743b7bd3bb52b17d1eba06d3041325767ccfe3e1517a/sinergym-3.3.0.tar.gz",
"platform": null,
"description": "# Sinergym\n\n<div align=\"center\">\n <img src=\"images/logo.png\" width=40%><br><br>\n</div>\n\n</p>\n <p align=\"center\">\n <a href=\"https://github.com/ugr-sail/sinergym/releases\">\n <img alt=\"Github latest release\" src=\"https://img.shields.io/github/release-date/ugr-sail/sinergym\" />\n </a>\n <a href=\"https://github.com/ugr-sail/sinergym/commits/main\">\n <img alt=\"Github last commit\" src=\"https://img.shields.io/github/last-commit/ugr-sail/sinergym\" />\n </a>\n <a href=\"https://pypi.org/project/sinergym/\">\n <img alt=\"Pypi version\" src=\"https://img.shields.io/pypi/v/sinergym\" />\n </a>\n <a href=\"https://github.com/ugr-sail/sinergym/stargazers\">\n <img alt=\"Pypi downloads\" src=\"https://img.shields.io/pypi/dm/sinergym\" />\n </a>\n <a href=\"https://codecov.io/gh/ugr-sail/sinergym\">\n <img src=\"https://codecov.io/gh/ugr-sail/sinergym/branch/main/graph/badge.svg\" />\n </a>\n <a href=\"https://github.com/ugr-sail/sinergym/graphs/contributors\">\n <img alt=\"GitHub Contributors\" src=\"https://img.shields.io/github/contributors/ugr-sail/sinergym\" />\n </a>\n <a href=\"https://github.com/ugr-sail/sinergym/issues\">\n <img alt=\"Github issues\" src=\"https://img.shields.io/github/issues/ugr-sail/sinergym?color=0088ff\" />\n </a>\n <a href=\"https://github.com/ugr-sail/sinergym/pulls\">\n <img alt=\"GitHub pull requests\" src=\"https://img.shields.io/github/issues-pr/ugr-sail/sinergym?color=0088ff\" />\n </a>\n <a href=\"https://github.com/ugr-sail/sinergym/blob/main/LICENSE\">\n <img alt=\"Github License\" src=\"https://img.shields.io/github/license/ugr-sail/sinergym\" />\n </a>\n <a href=\"https://www.python.org/downloads/release/python-3100/\">\n <img alt=\"Pypi Python version\" src=\"https://img.shields.io/pypi/pyversions/sinergym\" />\n </a>\n <br />\n <br />\n <a href=\"https://code.visualstudio.com/\">\n <img src=\"https://img.shields.io/badge/Supported%20by-VSCode%20Power%20User%20%E2%86%92-gray.svg?colorA=655BE1&colorB=4F44D6&style=for-the-badge\"/>\n </a>\n </p>\n\n**Welcome to Sinergym!**\n\n<div align=\"center\">\n <img src=\"images/general_blueprint.png\" width=80%><br><br>\n</div>\n\nThe goal of this project is to create an environment following *Gymnasium* interface for wrapping \nsimulation engines (*EnergyPlus*) for building control using **deep reinforcement learning** \nor any external control.\n\nFor more information about Sinergym, please visit our [documentation](https://ugr-sail.github.io/sinergym/compilation/main/index.html).\n\nTo report questions and issues, please use our [issue tracker](https://github.com/ugr-sail/sinergym/issues). We appreciate your feedback and contributions. Check out our [CONTRIBUTING.md](https://github.com/ugr-sail/sinergym/blob/main/CONTRIBUTING.md) for more details on how to contribute.\n\nThe main functionalities of *Sinergym* are the following:\n\n- **Simulation Engine Compatibility**: Uses [EnergyPlus Python API](https://energyplus.readthedocs.io/en/latest/api.html) for Python-EnergyPlus communication. Future plans include more engines like [OpenModelica](https://openmodelica.org/).\n\n- **Benchmark Environments**: Designs environments for benchmarking and testing deep RL algorithms or other external strategies, similar to *Atari* or *Mujoco*.\n\n- **Customizable Environments**: Allows easy modification of experimental settings. Users can create their own environments or modify pre-configured ones in *Sinergym*.\n\n- **Customizable Components**: Enables creation of new custom components for new environments, making *Sinergym* scalable, such as function rewards, wrappers, controllers, etc.\n\n- **Automatic Building Model Adaptation**: *Sinergym* automates the process of adapting the building model to user changes in the environment definition.\n\n- **Automatic Actuators Control**: Controls actuators through the Gymnasium interface based on user specification, only actuators names are required and *Sinergym* will do the rest.\n\n- **Extensive Environment Information**: Provides comprehensive information about *Sinergym* background components from the environment interface.\n\n- **Stable Baseline 3 Integration**: Customizes functionalities for easy testing of environments with SB3 algorithms, such as callbacks and customizable training real-time logging. However, *Sinergym* is agnostic to any DRL algorithm.\n\n- **Google Cloud Integration**: Offers guidance on using *Sinergym* with Google Cloud infrastructure.\n\n- **Weights & Biases Compatibility**: Automates and facilitates training, reproducibility, and comparison of agents in simulation-based building control problems. [WandB](https://wandb.ai/site) assists in managing and monitoring model lifecycle.\n\n- **Notebook Examples**: Provides code in notebook format for user familiarity with the tool.\n\n- **Extensive Documentation, Unit Tests, and GitHub Actions Workflows**: Ensures *Sinergym* is an efficient ecosystem for understanding and development.\n\n- And much more!\n\n_This is a project in active development. Stay tuned for upcoming releases._\n\n<div align=\"center\">\n <img src=\"images/operation_diagram.png\"><br><br>\n</div>\n\n## Project Structure\n\nThis repository is organized into the following directories:\n\n- `sinergym/`: Contains the source code for *Sinergym*, including the environment, modeling, simulator, and tools such as wrappers and reward functions.\n- `docs/`: Online documentation generated with Sphinx and using Restructured Text (RST).\n- `examples/`: Jupyter notebooks illustrating use cases with Sinergym.\n- `tests/`: Unit tests for Sinergym to ensure stability.\n- `scripts/`: Scripts for various tasks such as agent training and performance checks, allowing configuration using JSON format.\n\n## Available Environments\n\nFor a complete and up-to-date list of available environments, please refer to [our documentation](https://ugr-sail.github.io/sinergym/compilation/main/pages/environments.html#).\n\n## Installation\n\nPlease visit [INSTALL.md](https://github.com/ugr-sail/sinergym/blob/main/INSTALL.md) for detailed installation instructions.\n\n## Usage example\n\n\nIf you used our Dockerfile during installation, you should have the *try_env.py* file in your workspace as soon as you enter in. In case you have installed everything on your local machine directly, place it inside our cloned repository. In any case, we start from the point that you have at your disposal a terminal with the appropriate python version and *Sinergym* running correctly.\n\n*Sinergym* uses the standard Gymnasium API. So a basic loop should look like:\n\n```python\n\nimport gymnasium as gym\nimport sinergym\n# Create the environment\nenv = gym.make('Eplus-datacenter-mixed-continuous-stochastic-v1')\n# Initialize the episode\nobs, info = env.reset()\ntruncated = terminated = False\nR = 0.0\nwhile not (terminated or truncated):\n a = env.action_space.sample() # random action selection\n obs, reward, terminated, truncated, info = env.step(a) # get new observation and reward\n R += reward\nprint('Total reward for the episode: %.4f' % R)\nenv.close()\n```\n\nA folder will be created in the working directory after creating the environment. It will contain the Sinergym outputs produced during the simulation.\n\nFor more examples and details, please visit our [usage examples](https://ugr-sail.github.io/sinergym/compilation/main/pages/notebooks/basic_example.html#Basic-example) documentation section.\n\n\n## Google Cloud Platform support\n\nFor more information about this functionality, please, visit our documentation [here](https://ugr-sail.github.io/sinergym/compilation/main/pages/gcloudAPI.html#sinergym-with-google-cloud).\n\n## Projects using Sinergym\n\nThe following are some of the projects benefiting from the advantages of Sinergym:\n\n- [Demosthen/ActiveRL](https://github.com/Demosthen/ActiveRL)\n- [VectorInstitute/HV-Ai-C](https://github.com/VectorInstitute/HV-Ai-C)\n- [rdnfn/beobench](https://github.com/rdnfn/beobench)\n\n:pencil: If you want to appear in this list, do not hesitate to send us a PR and include the following badge in your repository:\n\n<p align=\"center\">\n <a href=\"https://github.com/ugr-sail/sinergym\">\n <img src=\"https://img.shields.io/badge/Powered%20by-Sinergym%20%E2%86%92-gray.svg?colorA=00BABF&colorB=4BF2F7&style=for-the-badge\"/>\n </a>\n</p>\n\n## Repo Activity\n\n![Alt](https://repobeats.axiom.co/api/embed/d8dc96d423d6996351e728a2412dba2551f99cca.svg \"Repobeats analytics image\")\n\n## Citing Sinergym\n\nIf you use *Sinergym* in your work, please cite our [paper](https://dl.acm.org/doi/abs/10.1145/3486611.3488729):\n\n```bibtex\n@inproceedings{2021sinergym,\n title={Sinergym: A Building Simulation and Control Framework for Training Reinforcement Learning Agents}, \n author={Jim\u00e9nez-Raboso, Javier and Campoy-Nieves, Alejandro and Manjavacas-Lucas, Antonio and G\u00f3mez-Romero, Juan and Molina-Solana, Miguel},\n year={2021},\n isbn = {9781450391146},\n publisher = {Association for Computing Machinery},\n address = {New York, NY, USA},\n url = {https://doi.org/10.1145/3486611.3488729},\n doi = {10.1145/3486611.3488729},\n booktitle = {Proceedings of the 8th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation},\n pages = {319\u2013323},\n numpages = {5},\n}\n```\n\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "The goal of sinergym is to create an environment following OpenAI Gym interface for wrapping simulation engines for building control using deep reinforcement learning.",
"version": "3.3.0",
"project_urls": {
"Homepage": "https://github.com/ugr-sail/sinergym"
},
"split_keywords": [
"control",
"reinforcement-learning",
"buildings",
"reinforcement-learning-environments"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "59c8435aee34a038d8a7743b7bd3bb52b17d1eba06d3041325767ccfe3e1517a",
"md5": "3fa8ea8ae468ef1c2e1abcb5b7a45380",
"sha256": "f96f564944933a5c4f2254a8f864d3da059ab5062b2f566ac23ce1bab166e955"
},
"downloads": -1,
"filename": "sinergym-3.3.0.tar.gz",
"has_sig": false,
"md5_digest": "3fa8ea8ae468ef1c2e1abcb5b7a45380",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 3210931,
"upload_time": "2024-04-03T08:18:54",
"upload_time_iso_8601": "2024-04-03T08:18:54.235694Z",
"url": "https://files.pythonhosted.org/packages/59/c8/435aee34a038d8a7743b7bd3bb52b17d1eba06d3041325767ccfe3e1517a/sinergym-3.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-03 08:18:54",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ugr-sail",
"github_project": "sinergym",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"requirements": [
{
"name": "autopep8",
"specs": []
},
{
"name": "eppy",
"specs": []
},
{
"name": "gymnasium",
"specs": []
},
{
"name": "isort",
"specs": []
},
{
"name": "numpy",
"specs": []
},
{
"name": "opyplus",
"specs": []
},
{
"name": "pandas",
"specs": []
},
{
"name": "pytype",
"specs": []
},
{
"name": "twine",
"specs": []
},
{
"name": "xlsxwriter",
"specs": []
},
{
"name": "pipdeptree",
"specs": []
}
],
"lcname": "sinergym"
}
JSON
