| Name | SAC-pytorch JSON |
| Version |
0.0.15
JSON |
| download |
| home_page | None |
| Summary | Soft Actor Critic - Pytorch |
| upload_time | 2025-02-13 14:04:27 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.9 |
| license | MIT License
Copyright (c) 2024 Phil Wang
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE. |
| keywords |
artificial intelligence
deep learning
reinforcement learning
soft actor critic
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
## SAC (Soft Actor Critic) - Pytorch (wip)
Implementation of Soft Actor Critic and some of its improvements in Pytorch. Interest comes from watching <a href="https://www.youtube.com/watch?v=17NrtKHdPDw">this lecture</a>
```python
import torch
from SAC_pytorch import (
SAC,
Actor,
Critic,
MultipleCritics
)
critic1 = Critic(
dim_state = 5,
num_cont_actions = 2,
num_discrete_actions = (5, 5),
dim_out = 3
)
critic2 = Critic(
dim_state = 5,
num_cont_actions = 2,
num_discrete_actions = (5, 5),
dim_out = 3
)
actor = Actor(
dim_state = 5,
num_cont_actions = 2,
num_discrete_actions = (5, 5)
)
agent = SAC(
actor = actor,
critics = [critic1, critic2],
quantiled_critics = True
)
state = torch.randn(3, 5)
cont_actions, discrete, cont_logprob, discrete_logprob = actor(state, sample = True)
agent(
states = state,
cont_actions = cont_actions,
discrete_actions = discrete,
rewards = torch.randn(1),
done = torch.zeros(1).bool(),
next_states = state + 1
)
```
## Citations
```bibtex
@article{Haarnoja2018SoftAA,
title = {Soft Actor-Critic Algorithms and Applications},
author = {Tuomas Haarnoja and Aurick Zhou and Kristian Hartikainen and G. Tucker and Sehoon Ha and Jie Tan and Vikash Kumar and Henry Zhu and Abhishek Gupta and P. Abbeel and Sergey Levine},
journal = {ArXiv},
year = {2018},
volume = {abs/1812.05905},
url = {https://api.semanticscholar.org/CorpusID:55703664}
}
```
```bibtex
@article{Hiraoka2021DropoutQF,
title = {Dropout Q-Functions for Doubly Efficient Reinforcement Learning},
author = {Takuya Hiraoka and Takahisa Imagawa and Taisei Hashimoto and Takashi Onishi and Yoshimasa Tsuruoka},
journal = {ArXiv},
year = {2021},
volume = {abs/2110.02034},
url = {https://api.semanticscholar.org/CorpusID:238353966}
}
```
```bibtex
@inproceedings{ObandoCeron2024MixturesOE,
title = {Mixtures of Experts Unlock Parameter Scaling for Deep RL},
author = {Johan S. Obando-Ceron and Ghada Sokar and Timon Willi and Clare Lyle and Jesse Farebrother and Jakob Foerster and Gintare Karolina Dziugaite and Doina Precup and Pablo Samuel Castro},
year = {2024},
url = {https://api.semanticscholar.org/CorpusID:267637059}
}
```
```bibtex
@inproceedings{Kumar2023MaintainingPI,
title = {Maintaining Plasticity in Continual Learning via Regenerative Regularization},
author = {Saurabh Kumar and Henrik Marklund and Benjamin Van Roy},
year = {2023},
url = {https://api.semanticscholar.org/CorpusID:261076021}
}
```
```bibtex
@inproceedings{Kuznetsov2020ControllingOB,
title = {Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Critics},
author = {Arsenii Kuznetsov and Pavel Shvechikov and Alexander Grishin and Dmitry P. Vetrov},
booktitle = {International Conference on Machine Learning},
year = {2020},
url = {https://api.semanticscholar.org/CorpusID:218581840}
}
```
```bibtex
@article{Zagoruyko2017DiracNetsTV,
title = {DiracNets: Training Very Deep Neural Networks Without Skip-Connections},
author={Sergey Zagoruyko and Nikos Komodakis},
journal = {ArXiv},
year = {2017},
volume = {abs/1706.00388},
url = {https://api.semanticscholar.org/CorpusID:1086822}
}
```
```bibtex
@article{Abbas2023LossOP,
title = {Loss of Plasticity in Continual Deep Reinforcement Learning},
author = {Zaheer Abbas and Rosie Zhao and Joseph Modayil and Adam White and Marlos C. Machado},
journal = {ArXiv},
year = {2023},
volume = {abs/2303.07507},
url = {https://api.semanticscholar.org/CorpusID:257504763}
}
```
```bibtex
@article{Zhang2024ReLU2WD,
title = {ReLU2 Wins: Discovering Efficient Activation Functions for Sparse LLMs},
author = {Zhengyan Zhang and Yixin Song and Guanghui Yu and Xu Han and Yankai Lin and Chaojun Xiao and Chenyang Song and Zhiyuan Liu and Zeyu Mi and Maosong Sun},
journal = {ArXiv},
year = {2024},
volume = {abs/2402.03804},
url = {https://api.semanticscholar.org/CorpusID:267499856}
}
```
```bibtex
@inproceedings{Lee2024SimBaSB,
title = {SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning},
author = {Hojoon Lee and Dongyoon Hwang and Donghu Kim and Hyunseung Kim and Jun Jet Tai and Kaushik Subramanian and Peter R. Wurman and Jaegul Choo and Peter Stone and Takuma Seno},
year = {2024},
url = {https://api.semanticscholar.org/CorpusID:273346233}
}
```
```bibtex
@article{Farebrother2024StopRT,
title = {Stop Regressing: Training Value Functions via Classification for Scalable Deep RL},
author = {Jesse Farebrother and Jordi Orbay and Quan Ho Vuong and Adrien Ali Taiga and Yevgen Chebotar and Ted Xiao and Alex Irpan and Sergey Levine and Pablo Samuel Castro and Aleksandra Faust and Aviral Kumar and Rishabh Agarwal},
journal = {ArXiv},
year = {2024},
volume = {abs/2403.03950},
url = {https://api.semanticscholar.org/CorpusID:268253088}
}
```
Raw data
{
"_id": null,
"home_page": null,
"name": "SAC-pytorch",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "artificial intelligence, deep learning, reinforcement learning, soft actor critic",
"author": null,
"author_email": "Phil Wang <lucidrains@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/54/2a/09ae0802f7c49ac8824e3b2eacdbae7d8eba9888c2bb2c4b62d40da9a542/sac_pytorch-0.0.15.tar.gz",
"platform": null,
"description": "## SAC (Soft Actor Critic) - Pytorch (wip)\n\nImplementation of Soft Actor Critic and some of its improvements in Pytorch. Interest comes from watching <a href=\"https://www.youtube.com/watch?v=17NrtKHdPDw\">this lecture</a>\n\n```python\nimport torch\n\nfrom SAC_pytorch import (\n SAC,\n Actor,\n Critic,\n MultipleCritics\n)\n\ncritic1 = Critic(\n dim_state = 5,\n num_cont_actions = 2,\n num_discrete_actions = (5, 5),\n dim_out = 3\n)\n\ncritic2 = Critic(\n dim_state = 5,\n num_cont_actions = 2,\n num_discrete_actions = (5, 5),\n dim_out = 3\n)\n\nactor = Actor(\n dim_state = 5,\n num_cont_actions = 2,\n num_discrete_actions = (5, 5)\n)\n\nagent = SAC(\n actor = actor,\n critics = [critic1, critic2],\n quantiled_critics = True\n)\n\nstate = torch.randn(3, 5)\ncont_actions, discrete, cont_logprob, discrete_logprob = actor(state, sample = True)\n\nagent(\n states = state,\n cont_actions = cont_actions,\n discrete_actions = discrete,\n rewards = torch.randn(1),\n done = torch.zeros(1).bool(),\n next_states = state + 1\n)\n```\n\n## Citations\n\n```bibtex\n@article{Haarnoja2018SoftAA,\n title = {Soft Actor-Critic Algorithms and Applications},\n author = {Tuomas Haarnoja and Aurick Zhou and Kristian Hartikainen and G. Tucker and Sehoon Ha and Jie Tan and Vikash Kumar and Henry Zhu and Abhishek Gupta and P. Abbeel and Sergey Levine},\n journal = {ArXiv},\n year = {2018},\n volume = {abs/1812.05905},\n url = {https://api.semanticscholar.org/CorpusID:55703664}\n}\n```\n\n```bibtex\n@article{Hiraoka2021DropoutQF,\n title = {Dropout Q-Functions for Doubly Efficient Reinforcement Learning},\n author = {Takuya Hiraoka and Takahisa Imagawa and Taisei Hashimoto and Takashi Onishi and Yoshimasa Tsuruoka},\n journal = {ArXiv},\n year = {2021},\n volume = {abs/2110.02034},\n url = {https://api.semanticscholar.org/CorpusID:238353966}\n}\n```\n\n```bibtex\n@inproceedings{ObandoCeron2024MixturesOE,\n title = {Mixtures of Experts Unlock Parameter Scaling for Deep RL},\n author = {Johan S. Obando-Ceron and Ghada Sokar and Timon Willi and Clare Lyle and Jesse Farebrother and Jakob Foerster and Gintare Karolina Dziugaite and Doina Precup and Pablo Samuel Castro},\n year = {2024},\n url = {https://api.semanticscholar.org/CorpusID:267637059}\n}\n```\n\n```bibtex\n@inproceedings{Kumar2023MaintainingPI,\n title = {Maintaining Plasticity in Continual Learning via Regenerative Regularization},\n author = {Saurabh Kumar and Henrik Marklund and Benjamin Van Roy},\n year = {2023},\n url = {https://api.semanticscholar.org/CorpusID:261076021}\n}\n```\n\n```bibtex\n@inproceedings{Kuznetsov2020ControllingOB,\n title = {Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Critics},\n author = {Arsenii Kuznetsov and Pavel Shvechikov and Alexander Grishin and Dmitry P. Vetrov},\n booktitle = {International Conference on Machine Learning},\n year = {2020},\n url = {https://api.semanticscholar.org/CorpusID:218581840}\n}\n```\n\n```bibtex\n@article{Zagoruyko2017DiracNetsTV,\n title = {DiracNets: Training Very Deep Neural Networks Without Skip-Connections},\n author={Sergey Zagoruyko and Nikos Komodakis},\n journal = {ArXiv},\n year = {2017},\n volume = {abs/1706.00388},\n url = {https://api.semanticscholar.org/CorpusID:1086822}\n}\n```\n\n```bibtex\n@article{Abbas2023LossOP,\n title = {Loss of Plasticity in Continual Deep Reinforcement Learning},\n author = {Zaheer Abbas and Rosie Zhao and Joseph Modayil and Adam White and Marlos C. Machado},\n journal = {ArXiv},\n year = {2023},\n volume = {abs/2303.07507},\n url = {https://api.semanticscholar.org/CorpusID:257504763}\n}\n```\n\n```bibtex\n@article{Zhang2024ReLU2WD,\n title = {ReLU2 Wins: Discovering Efficient Activation Functions for Sparse LLMs},\n author = {Zhengyan Zhang and Yixin Song and Guanghui Yu and Xu Han and Yankai Lin and Chaojun Xiao and Chenyang Song and Zhiyuan Liu and Zeyu Mi and Maosong Sun},\n journal = {ArXiv},\n year = {2024},\n volume = {abs/2402.03804},\n url = {https://api.semanticscholar.org/CorpusID:267499856}\n}\n```\n\n```bibtex\n@inproceedings{Lee2024SimBaSB,\n title = {SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning},\n author = {Hojoon Lee and Dongyoon Hwang and Donghu Kim and Hyunseung Kim and Jun Jet Tai and Kaushik Subramanian and Peter R. Wurman and Jaegul Choo and Peter Stone and Takuma Seno},\n year = {2024},\n url = {https://api.semanticscholar.org/CorpusID:273346233}\n}\n```\n\n```bibtex\n@article{Farebrother2024StopRT,\n title = {Stop Regressing: Training Value Functions via Classification for Scalable Deep RL},\n author = {Jesse Farebrother and Jordi Orbay and Quan Ho Vuong and Adrien Ali Taiga and Yevgen Chebotar and Ted Xiao and Alex Irpan and Sergey Levine and Pablo Samuel Castro and Aleksandra Faust and Aviral Kumar and Rishabh Agarwal},\n journal = {ArXiv},\n year = {2024},\n volume = {abs/2403.03950},\n url = {https://api.semanticscholar.org/CorpusID:268253088}\n}\n```\n",
"bugtrack_url": null,
"license": "MIT License\n \n Copyright (c) 2024 Phil Wang\n \n Permission is hereby granted, free of charge, to any person obtaining a copy\n of this software and associated documentation files (the \"Software\"), to deal\n in the Software without restriction, including without limitation the rights\n to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n copies of the Software, and to permit persons to whom the Software is\n furnished to do so, subject to the following conditions:\n \n The above copyright notice and this permission notice shall be included in all\n copies or substantial portions of the Software.\n \n THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n SOFTWARE.",
"summary": "Soft Actor Critic - Pytorch",
"version": "0.0.15",
"project_urls": {
"Homepage": "https://pypi.org/project/SAC-pytorch/",
"Repository": "https://github.com/lucidrains/SAC-pytorch"
},
"split_keywords": [
"artificial intelligence",
" deep learning",
" reinforcement learning",
" soft actor critic"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "589ebb5d8e73b3878a18997ded116aa455af3e68b0b4db14eefce9469cf0dad1",
"md5": "fbaf1ac9f76de743d551886c9f923631",
"sha256": "a7fe528e4e9ca1e7ecee70bb7ec11a300f1333187e6dc918738e0749634a8716"
},
"downloads": -1,
"filename": "sac_pytorch-0.0.15-py3-none-any.whl",
"has_sig": false,
"md5_digest": "fbaf1ac9f76de743d551886c9f923631",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 11462,
"upload_time": "2025-02-13T14:04:25",
"upload_time_iso_8601": "2025-02-13T14:04:25.685817Z",
"url": "https://files.pythonhosted.org/packages/58/9e/bb5d8e73b3878a18997ded116aa455af3e68b0b4db14eefce9469cf0dad1/sac_pytorch-0.0.15-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "542a09ae0802f7c49ac8824e3b2eacdbae7d8eba9888c2bb2c4b62d40da9a542",
"md5": "b0a62ca8030f0d6430843ba998310476",
"sha256": "65706e7c3802435d2a8296b8d2015761e44554752a5b7cdbc8832d3a02e176a5"
},
"downloads": -1,
"filename": "sac_pytorch-0.0.15.tar.gz",
"has_sig": false,
"md5_digest": "b0a62ca8030f0d6430843ba998310476",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 12341,
"upload_time": "2025-02-13T14:04:27",
"upload_time_iso_8601": "2025-02-13T14:04:27.499429Z",
"url": "https://files.pythonhosted.org/packages/54/2a/09ae0802f7c49ac8824e3b2eacdbae7d8eba9888c2bb2c4b62d40da9a542/sac_pytorch-0.0.15.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-13 14:04:27",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "lucidrains",
"github_project": "SAC-pytorch",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "sac-pytorch"
}
