[](https://www.repostatus.org/#active)
[](https://pre-commit.com/)
[](https://github.com/psf/black)
<p align="center">
<img src="docs/_static/img/momaland-text.png" width="500px"/>
</p>
<!-- start elevator-pitch -->
MOMAland is an open source Python library for developing and comparing multi-objective multi-agent reinforcement learning algorithms by providing a standard API to communicate between learning algorithms and environments, as well as a standard set of environments compliant with that API. Essentially, the environments follow the standard [PettingZoo APIs](https://github.com/Farama-Foundation/PettingZoo), but return vectorized rewards as numpy arrays instead of scalar values.
The documentation website is at https://momaland.farama.org/, and we have a public discord server (which we also use to coordinate development work) that you can join [here](https://discord.gg/bnJ6kubTg6).
<!-- end elevator-pitch -->
## Environments
MOMAland includes environments taken from the MOMARL literature, as well as multi-objective version of classical environments, such as SISL or Butterfly.
The full list of environments is available at https://momaland.farama.org/environments/all-envs/.
## Installation
<!-- start install -->
To install MOMAland, use:
```bash
pip install momaland
```
This does not include dependencies for all components of MOMAland (not everything is required for the basic usage, and some can be problematic to install on certain systems).
- `pip install "momaland[testing]"` to install dependencies for API testing.
- `pip install "momaland[learning]"` to install dependencies for the supplied learning algorithms.
- `pip install "momaland[all]"` for all dependencies for all components.
<!-- end install -->
## API
<!-- start snippet-usage -->
Similar to [PettingZoo](https://pettingzoo.farama.org), the MOMAland API models environments as simple Python `env` classes. Creating environment instances and interacting with them is very simple - here's an example using the "momultiwalker_stability_v0" environment:
```python
from momaland.envs.momultiwalker_stability import momultiwalker_stability_v0 as _env
import numpy as np
# .env() function will return an AEC environment, as per PZ standard
env = _env.env(render_mode="human")
env.reset(seed=42)
for agent in env.agent_iter():
# vec_reward is a numpy array
observation, vec_reward, termination, truncation, info = env.last()
if termination or truncation:
action = None
else:
action = env.action_space(agent).sample() # this is where you would insert your policy
env.step(action)
env.close()
# optionally, you can scalarize the reward with weights
# Making the vector reward a scalar reward to shift to single-objective multi-agent (aka PettingZoo)
# We can assign different weights to the objectives of each agent.
weights = {
"walker_0": np.array([0.7, 0.3]),
"walker_1": np.array([0.5, 0.5]),
"walker_2": np.array([0.2, 0.8]),
}
env = LinearizeReward(env, weights)
```
For details on multi-objective multi-agent RL definitions, see [Multi-Objective Multi-Agent Decision Making: A Utility-based Analysis and Survey](https://arxiv.org/abs/1909.02964).
You can also check more examples in this colab notebook! [](https://colab.research.google.com/github/Farama-Foundation/momaland/blob/doc/notebook/momaland_demo.ipynb)
<!-- end snippet-usage -->
## Learning Algorithms
<!-- start learning-algorithms -->
We provide a set of learning algorithms that are compatible with the MOMAland environments. The learning algorithms are implemented in the [learning/](https://github.com/Farama-Foundation/momaland/tree/main/momaland/learning) directory. To keep everything as self-contained as possible, each algorithm is implemented as a single-file (close to [cleanRL's philosophy](https://github.com/vwxyzjn/cleanrl/tree/master)).
Nevertheless, we reuse tools provided by other libraries, like multi-objective evaluations and performance indicators from [MORL-Baselines](https://github.com/LucasAlegre/morl-baselines).
Here is a list of algorithms that are currently implemented:
| **Name** | Single/Multi-policy | Reward | Utility | Observation space | Action space | Paper |
|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------|------------|---------------------|-------------------|--------------|-------|
| MOMAPPO (OLS) [continuous](https://github.com/Farama-Foundation/momaland/blob/main/momaland/learning/continuous/cooperative_momappo.py),<br/> [discrete](https://github.com/Farama-Foundation/momaland/blob/main/momaland/learning/discrete/cooperative_momappo.py) | Multi | Team | Team / Linear | Any | Any | |
| [Scalarized IQL](https://github.com/Farama-Foundation/momaland/tree/main/momaland/learning/iql) | Single | Individual | Individual / Linear | Discrete | Discrete | |
| [Centralization wrapper](https://github.com/Farama-Foundation/momaland/blob/main/momaland/utils/parallel_wrappers.py#L149) | Any | Team | Team / Any | Discrete | Discrete | |
| [Linearization wrapper](https://github.com/Farama-Foundation/momaland/blob/main/momaland/utils/parallel_wrappers.py#L49) | Single | Any | Individual / Linear | Any | Any | |
<!-- end learning-algorithms -->
## Environment Versioning
MOMAland keeps strict versioning for reproducibility reasons. All environments end in a suffix like "_v0". When changes are made to environments that might impact learning results, the number is increased by one to prevent potential confusion.
## Development Roadmap
We have a roadmap for future development available [here](https://github.com/Farama-Foundation/momaland/issues/56).
## Project Maintainers
Project Managers: Florian Felten (@ffelten)
Maintenance for this project is also contributed by the broader Farama team: [farama.org/team](https://farama.org/team).
## Citing
<!-- start citation -->
If you use this repository in your research, please cite:
```bibtex
@inproceedings{TODO}
```
<!-- end citation -->
## Development
### Setup pre-commit
Clone the repo and run `pre-commit install` to setup the pre-commit hooks.
Raw data
{
"_id": null,
"home_page": null,
"name": "momaland",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "Reinforcement Learning, Multi-Objective, Multi-Agent, RL, AI, gymnasium, pettingzoo",
"author": "Umut Ucak, Hicham Azmani",
"author_email": "Florian Felten <florian.felten@uni.lu>, Roxana Radulescu <roxana.radulescu@vub.be>, \"Hendrik J. S. Baier\" <h.j.s.baier@tue.nl>, Willem R\u00f6pke <willem.ropke@vub.be>, Patrick Mannion <patrick.mannion@universityofgalway.ie>, \"Diederik M. Roijers\" <diederik.roijers@vub.be>",
"download_url": "https://files.pythonhosted.org/packages/ab/c6/4bbc737ca56a1b911d31f755e2e031c5bd1061797810cade0ab65b523921/momaland-0.1.0.tar.gz",
"platform": null,
"description": "[](https://www.repostatus.org/#active)\n\n[](https://pre-commit.com/)\n[](https://github.com/psf/black)\n\n<p align=\"center\">\n <img src=\"docs/_static/img/momaland-text.png\" width=\"500px\"/>\n</p>\n\n<!-- start elevator-pitch -->\nMOMAland is an open source Python library for developing and comparing multi-objective multi-agent reinforcement learning algorithms by providing a standard API to communicate between learning algorithms and environments, as well as a standard set of environments compliant with that API. Essentially, the environments follow the standard [PettingZoo APIs](https://github.com/Farama-Foundation/PettingZoo), but return vectorized rewards as numpy arrays instead of scalar values.\n\nThe documentation website is at https://momaland.farama.org/, and we have a public discord server (which we also use to coordinate development work) that you can join [here](https://discord.gg/bnJ6kubTg6).\n<!-- end elevator-pitch -->\n\n## Environments\nMOMAland includes environments taken from the MOMARL literature, as well as multi-objective version of classical environments, such as SISL or Butterfly.\nThe full list of environments is available at https://momaland.farama.org/environments/all-envs/.\n\n## Installation\n<!-- start install -->\nTo install MOMAland, use:\n```bash\npip install momaland\n```\nThis does not include dependencies for all components of MOMAland (not everything is required for the basic usage, and some can be problematic to install on certain systems).\n- `pip install \"momaland[testing]\"` to install dependencies for API testing.\n- `pip install \"momaland[learning]\"` to install dependencies for the supplied learning algorithms.\n- `pip install \"momaland[all]\"` for all dependencies for all components.\n<!-- end install -->\n\n## API\n<!-- start snippet-usage -->\nSimilar to [PettingZoo](https://pettingzoo.farama.org), the MOMAland API models environments as simple Python `env` classes. Creating environment instances and interacting with them is very simple - here's an example using the \"momultiwalker_stability_v0\" environment:\n\n```python\nfrom momaland.envs.momultiwalker_stability import momultiwalker_stability_v0 as _env\nimport numpy as np\n\n# .env() function will return an AEC environment, as per PZ standard\nenv = _env.env(render_mode=\"human\")\n\nenv.reset(seed=42)\nfor agent in env.agent_iter():\n # vec_reward is a numpy array\n observation, vec_reward, termination, truncation, info = env.last()\n\n if termination or truncation:\n action = None\n else:\n action = env.action_space(agent).sample() # this is where you would insert your policy\n\n env.step(action)\nenv.close()\n\n# optionally, you can scalarize the reward with weights\n# Making the vector reward a scalar reward to shift to single-objective multi-agent (aka PettingZoo)\n# We can assign different weights to the objectives of each agent.\nweights = {\n \"walker_0\": np.array([0.7, 0.3]),\n \"walker_1\": np.array([0.5, 0.5]),\n \"walker_2\": np.array([0.2, 0.8]),\n}\nenv = LinearizeReward(env, weights)\n```\n\nFor details on multi-objective multi-agent RL definitions, see [Multi-Objective Multi-Agent Decision Making: A Utility-based Analysis and Survey](https://arxiv.org/abs/1909.02964).\n\nYou can also check more examples in this colab notebook! [](https://colab.research.google.com/github/Farama-Foundation/momaland/blob/doc/notebook/momaland_demo.ipynb)\n<!-- end snippet-usage -->\n\n## Learning Algorithms\n<!-- start learning-algorithms -->\nWe provide a set of learning algorithms that are compatible with the MOMAland environments. The learning algorithms are implemented in the [learning/](https://github.com/Farama-Foundation/momaland/tree/main/momaland/learning) directory. To keep everything as self-contained as possible, each algorithm is implemented as a single-file (close to [cleanRL's philosophy](https://github.com/vwxyzjn/cleanrl/tree/master)).\n\nNevertheless, we reuse tools provided by other libraries, like multi-objective evaluations and performance indicators from [MORL-Baselines](https://github.com/LucasAlegre/morl-baselines).\n\nHere is a list of algorithms that are currently implemented:\n\n| **Name** | Single/Multi-policy | Reward | Utility | Observation space | Action space | Paper |\n|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------|------------|---------------------|-------------------|--------------|-------|\n| MOMAPPO (OLS) [continuous](https://github.com/Farama-Foundation/momaland/blob/main/momaland/learning/continuous/cooperative_momappo.py),<br/> [discrete](https://github.com/Farama-Foundation/momaland/blob/main/momaland/learning/discrete/cooperative_momappo.py) | Multi | Team | Team / Linear | Any | Any | |\n| [Scalarized IQL](https://github.com/Farama-Foundation/momaland/tree/main/momaland/learning/iql) | Single | Individual | Individual / Linear | Discrete | Discrete | |\n| [Centralization wrapper](https://github.com/Farama-Foundation/momaland/blob/main/momaland/utils/parallel_wrappers.py#L149) | Any | Team | Team / Any | Discrete | Discrete | |\n| [Linearization wrapper](https://github.com/Farama-Foundation/momaland/blob/main/momaland/utils/parallel_wrappers.py#L49) | Single | Any | Individual / Linear | Any | Any | |\n\n\n<!-- end learning-algorithms -->\n\n## Environment Versioning\nMOMAland keeps strict versioning for reproducibility reasons. All environments end in a suffix like \"_v0\". When changes are made to environments that might impact learning results, the number is increased by one to prevent potential confusion.\n\n## Development Roadmap\nWe have a roadmap for future development available [here](https://github.com/Farama-Foundation/momaland/issues/56).\n\n## Project Maintainers\nProject Managers: Florian Felten (@ffelten)\n\nMaintenance for this project is also contributed by the broader Farama team: [farama.org/team](https://farama.org/team).\n\n## Citing\n<!-- start citation -->\nIf you use this repository in your research, please cite:\n```bibtex\n@inproceedings{TODO}\n```\n<!-- end citation -->\n\n## Development\n### Setup pre-commit\nClone the repo and run `pre-commit install` to setup the pre-commit hooks.\n",
"bugtrack_url": null,
"license": "GNU General Public License v3.0",
"summary": "A standard API for Multi-Objective Multi-Agent Decision making and a diverse set of reference environments.",
"version": "0.1.0",
"project_urls": {
"Bug Report": "https://github.com/rradules/momaland/tree/main/issues",
"Documentation": "https://github.com/rradules/momaland/tree/main",
"Homepage": "https://github.com/rradules/momaland/tree/main",
"Repository": "https://github.com/rradules/momaland/tree/main"
},
"split_keywords": [
"reinforcement learning",
" multi-objective",
" multi-agent",
" rl",
" ai",
" gymnasium",
" pettingzoo"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3ecb0a4c46b8caf8ee701234adfde99853877a3d19a44d49ffde5a3b82b8c4fc",
"md5": "4e56282c0471daf921493088f0fe6163",
"sha256": "e2376d6673c24038961a2f04125a9c25f371acc39005ff3aafbf97d898f4a7bb"
},
"downloads": -1,
"filename": "momaland-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4e56282c0471daf921493088f0fe6163",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 152448,
"upload_time": "2024-05-03T17:03:36",
"upload_time_iso_8601": "2024-05-03T17:03:36.254959Z",
"url": "https://files.pythonhosted.org/packages/3e/cb/0a4c46b8caf8ee701234adfde99853877a3d19a44d49ffde5a3b82b8c4fc/momaland-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "abc64bbc737ca56a1b911d31f755e2e031c5bd1061797810cade0ab65b523921",
"md5": "09141efa2b4c317140fbd90cc027890e",
"sha256": "dcac14dcc66a850c9228938547fe64b8bef876d334e603277b3536618a7e9e75"
},
"downloads": -1,
"filename": "momaland-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "09141efa2b4c317140fbd90cc027890e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 114372,
"upload_time": "2024-05-03T17:03:37",
"upload_time_iso_8601": "2024-05-03T17:03:37.911596Z",
"url": "https://files.pythonhosted.org/packages/ab/c6/4bbc737ca56a1b911d31f755e2e031c5bd1061797810cade0ab65b523921/momaland-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-03 17:03:37",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "rradules",
"github_project": "momaland",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "momaland"
}
JSON
