# ModularRL
ModularRL is a Python library for creating and training reinforcement learning agents using various algorithms. The library is designed to be easily customizable and modular, allowing users to quickly set up and train agents for various environments without being limited to a specific algorithm.
## Installation
```powershell
pip install modular_rl
```
## Features
- Implementations of various reinforcement learning algorithms,
such as Proximal Policy Optimization (PPO), Monte Carlo Tree Search (MCTS), Monte Carlo Information Set (MCIS), and Modular's sIMulator (MIM)
- Customizable agent settings and network architectures
- Modular structure for easy adaptation and extension across different algorithms
- Model saving and loading functionality for easy reuse of trained models
## Supported Algorithms
- Proximal Policy Optimization (PPO)
- Monte Carlo Tree Search (MCTS)
- Monte Carlo Information Set (MCIS)
- Modular's sIMulator (MIM)
Refer to the respective agent classes for each algorithm:
- AgentPPO (+ Modular)
- AgentMCTS (+ Modular)
- AgentMCIS (+ Modular)
- AgentMIM (+ Modular)
## Example Usage
You can use the tester.py script provided in the library to create and train an instance of an agent with default or modified settings:
```python
import modular_rl.tester as tester
tester.init_ppo()
# or
tester.init_ppo_modular()
tester.init_mcts()
```
As more algorithms are added, the tester functions will follow the naming convention init*[algorithm_name] or init*[algorithm_name]\_modular.
Please note that not all algorithms support modular training due to the nature of their design.
For such algorithms, you will need to use the non-modular training method provided by the respective agent class.
You can refer to the list of supported algorithms to determine which training method is appropriate.
Alternatively, you can create and train an instance of the AgentPPO(example) class directly in your code:
```python
from modular_rl.agents.agent_ppo import AgentPPO
from modular_rl.settings import AgentSettings
def init():
env = AgentPPO(env=None, setting=AgentSettings.default)
env.learn()
init()
```
To create and train an instance of the AgentPPO(example) class with modified settings, use the following code:
```python
from modular_rl.agents.agent_ppo import AgentPPO
from modular_rl.settings import AgentSettings
def init_modular():
# Semi-automatic (defined record usage)
# Implement your environment and pass it to 'env' parameter.
env = AgentPPO(env=None, setting=AgentSettings.default_modular)
env.reset()
env.learn_reset()
action, reward, is_done = env.learn_next()
env.learn_check()
env.update()
# Proceed with the learning manually.
env.reset()
# Implement the 'reset' method in your environment.
'''
def reset(self):
...
return initial_state
'''
env.learn_reset()
initial_state = env.learn_reset()
action, dist = env.select_action(initial_state)
'''
Note:
Please implement the resulting state of update_step in the step function of your environment.
For example:
def step(self, action):
...
return next_state, reward, is_done, _
'''
env.update_step(initial_state, dist, action, -1)
env.learn_check()
env.update()
env.learn_close()
init_modular()
```
## Saving and Loading Models
Agents can save and load their models using the save_model(file_name) and load_model(file_name) methods.
The file_name parameter should be the name of the file to save or load the model to/from.
Example:
```python
agent = AgentPPO(env, setting)
agent.train()
agent.save_model("my_saved_model.pth")
loaded_agent = AgentPPO(env, setting)
loaded_agent.load_model("my_saved_model.pth")
```
## Key Classes
- AgentPPO, AgentMCTS, AgentMCIS, AgentMIM: The main agent classes implementing various reinforcement learning algorithms.
- PolicyNetwork, ValueNetwork, ActorCriticNetwork: Customizable neural networks for the agent's policy and value functions.
- AgentSettings: A configuration class for setting up the agents.
## License
MIT License
Raw data
{
"_id": null,
"home_page": "https://github.com/horrible-gh/ModularRL",
"name": "modular-rl",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "ModularRL,Modular-RL,Modular_RL,Modular RL,modularrl,modular_rl,modular-rl,modular rl,mrl,modular,learn,learning,pytorch learn,pytorch learning,tensorflow learn,tensorflow learning,ai",
"author": "sjm",
"author_email": "horrible <shinjpn1@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/fe/e9/84e5af1c63f2a9542300d895d40001f20fab5a11dd549baa9c88ca8fe03b/modular_rl-0.4.3.tar.gz",
"platform": null,
"description": "# ModularRL\r\n\r\nModularRL is a Python library for creating and training reinforcement learning agents using various algorithms. The library is designed to be easily customizable and modular, allowing users to quickly set up and train agents for various environments without being limited to a specific algorithm.\r\n\r\n## Installation\r\n\r\n```powershell\r\npip install modular_rl\r\n```\r\n\r\n## Features\r\n\r\n- Implementations of various reinforcement learning algorithms,\r\n such as Proximal Policy Optimization (PPO), Monte Carlo Tree Search (MCTS), Monte Carlo Information Set (MCIS), and Modular's sIMulator (MIM)\r\n- Customizable agent settings and network architectures\r\n- Modular structure for easy adaptation and extension across different algorithms\r\n- Model saving and loading functionality for easy reuse of trained models\r\n\r\n## Supported Algorithms\r\n\r\n- Proximal Policy Optimization (PPO)\r\n- Monte Carlo Tree Search (MCTS)\r\n- Monte Carlo Information Set (MCIS)\r\n- Modular's sIMulator (MIM)\r\n\r\nRefer to the respective agent classes for each algorithm:\r\n\r\n- AgentPPO (+ Modular)\r\n- AgentMCTS (+ Modular)\r\n- AgentMCIS (+ Modular)\r\n- AgentMIM (+ Modular)\r\n\r\n## Example Usage\r\n\r\nYou can use the tester.py script provided in the library to create and train an instance of an agent with default or modified settings:\r\n\r\n```python\r\nimport modular_rl.tester as tester\r\n\r\ntester.init_ppo()\r\n# or\r\ntester.init_ppo_modular()\r\n\r\ntester.init_mcts()\r\n```\r\n\r\nAs more algorithms are added, the tester functions will follow the naming convention init*[algorithm_name] or init*[algorithm_name]\\_modular.\r\n\r\nPlease note that not all algorithms support modular training due to the nature of their design.\r\nFor such algorithms, you will need to use the non-modular training method provided by the respective agent class.\r\nYou can refer to the list of supported algorithms to determine which training method is appropriate.\r\n\r\nAlternatively, you can create and train an instance of the AgentPPO(example) class directly in your code:\r\n\r\n```python\r\nfrom modular_rl.agents.agent_ppo import AgentPPO\r\nfrom modular_rl.settings import AgentSettings\r\n\r\ndef init():\r\n env = AgentPPO(env=None, setting=AgentSettings.default)\r\n env.learn()\r\n\r\ninit()\r\n```\r\n\r\nTo create and train an instance of the AgentPPO(example) class with modified settings, use the following code:\r\n\r\n```python\r\nfrom modular_rl.agents.agent_ppo import AgentPPO\r\nfrom modular_rl.settings import AgentSettings\r\n\r\ndef init_modular():\r\n # Semi-automatic (defined record usage)\r\n # Implement your environment and pass it to 'env' parameter.\r\n env = AgentPPO(env=None, setting=AgentSettings.default_modular)\r\n env.reset()\r\n env.learn_reset()\r\n action, reward, is_done = env.learn_next()\r\n env.learn_check()\r\n env.update()\r\n\r\n # Proceed with the learning manually.\r\n env.reset()\r\n # Implement the 'reset' method in your environment.\r\n '''\r\n def reset(self):\r\n ...\r\n return initial_state\r\n '''\r\n env.learn_reset()\r\n initial_state = env.learn_reset()\r\n action, dist = env.select_action(initial_state)\r\n\r\n '''\r\n Note:\r\n Please implement the resulting state of update_step in the step function of your environment.\r\n\r\n For example:\r\n\r\n def step(self, action):\r\n ...\r\n return next_state, reward, is_done, _\r\n '''\r\n\r\n env.update_step(initial_state, dist, action, -1)\r\n\r\n env.learn_check()\r\n env.update()\r\n\r\n env.learn_close()\r\n\r\ninit_modular()\r\n```\r\n\r\n## Saving and Loading Models\r\n\r\nAgents can save and load their models using the save_model(file_name) and load_model(file_name) methods.\r\nThe file_name parameter should be the name of the file to save or load the model to/from.\r\n\r\nExample:\r\n\r\n```python\r\nagent = AgentPPO(env, setting)\r\nagent.train()\r\n\r\nagent.save_model(\"my_saved_model.pth\")\r\n\r\nloaded_agent = AgentPPO(env, setting)\r\nloaded_agent.load_model(\"my_saved_model.pth\")\r\n```\r\n\r\n## Key Classes\r\n\r\n- AgentPPO, AgentMCTS, AgentMCIS, AgentMIM: The main agent classes implementing various reinforcement learning algorithms.\r\n- PolicyNetwork, ValueNetwork, ActorCriticNetwork: Customizable neural networks for the agent's policy and value functions.\r\n- AgentSettings: A configuration class for setting up the agents.\r\n\r\n## License\r\n\r\nMIT License\r\n",
"bugtrack_url": null,
"license": "",
"summary": "ModularRL is a Python library for creating and training reinforcement learning agents using various algorithms. The library is designed to be easily customizable and modular, allowing users to quickly set up and train agents for various environments without being limited to a specific algorithm.",
"version": "0.4.3",
"project_urls": {
"Bug Tracker": "https://github.com/horrible-gh/ModularRL/issues",
"Homepage": "https://github.com/horrible-gh/ModularRL"
},
"split_keywords": [
"modularrl",
"modular-rl",
"modular_rl",
"modular rl",
"modularrl",
"modular_rl",
"modular-rl",
"modular rl",
"mrl",
"modular",
"learn",
"learning",
"pytorch learn",
"pytorch learning",
"tensorflow learn",
"tensorflow learning",
"ai"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3ed00450ac324c574c0ca2bf221fcb01809a785cef4aa77124fac673792d82b8",
"md5": "6e740e3b0e896f22c944784dbe76fd4e",
"sha256": "249fb8a34786595a6471b5313e993aea32204f96514594a59fc2932b0c4f55cc"
},
"downloads": -1,
"filename": "modular_rl-0.4.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6e740e3b0e896f22c944784dbe76fd4e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 72689,
"upload_time": "2023-09-22T13:57:55",
"upload_time_iso_8601": "2023-09-22T13:57:55.626429Z",
"url": "https://files.pythonhosted.org/packages/3e/d0/0450ac324c574c0ca2bf221fcb01809a785cef4aa77124fac673792d82b8/modular_rl-0.4.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "fee984e5af1c63f2a9542300d895d40001f20fab5a11dd549baa9c88ca8fe03b",
"md5": "8a7f9eb2ec8e22828bb8b8d4aad2b98e",
"sha256": "e9495fc0d28bb9786067e84794a70e9b57430d07b114430fe861048e4c85b77f"
},
"downloads": -1,
"filename": "modular_rl-0.4.3.tar.gz",
"has_sig": false,
"md5_digest": "8a7f9eb2ec8e22828bb8b8d4aad2b98e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 38002,
"upload_time": "2023-09-22T13:57:57",
"upload_time_iso_8601": "2023-09-22T13:57:57.107065Z",
"url": "https://files.pythonhosted.org/packages/fe/e9/84e5af1c63f2a9542300d895d40001f20fab5a11dd549baa9c88ca8fe03b/modular_rl-0.4.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-09-22 13:57:57",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "horrible-gh",
"github_project": "ModularRL",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "gym",
"specs": [
[
"==",
"0.26.2"
]
]
},
{
"name": "numpy",
"specs": [
[
">=",
"1.22"
],
[
"<",
"1.24"
]
]
},
{
"name": "torch",
"specs": [
[
"==",
"2.0.1"
]
]
},
{
"name": "LogAssist",
"specs": [
[
"==",
"1.0.7"
]
]
},
{
"name": "tensorflow",
"specs": [
[
"==",
"2.13.0"
]
]
},
{
"name": "tensorflow_probability",
"specs": [
[
"==",
"0.20.1"
]
]
}
],
"lcname": "modular-rl"
}
JSON
