# Chicken Coop
* [Full paper](https://r.rachum.com/dh-paper)
* Coming soon: Extended abstract
* Coming soon: Poster
* Coming soon: Deck
* Coming soon: Talk video
This is the accompanying code for our paper **Emergent Dominance Hierarchies in Reinforcement Learning Agents** presented in AAMAS 2024.
Abstract:
> Modern Reinforcement Learning (RL) algorithms are able to outperform humans in a wide variety of tasks. Multi-agent reinforcement learning (MARL) settings present additional challenges, and successful cooperation in mixed-motive groups of agents depends on a delicate balancing act between individual and group objectives. Social conventions and norms, often inspired by human institutions, are used as tools for striking this balance.
>
> In this paper, we examine a fundamental, well-studied social convention that underlies cooperation in both animal and human societies: Dominance hierarchies.
>
> We adapt the ethological theory of dominance hierarchies to artificial agents, borrowing the established terminology and definitions with as few amendments as possible. We demonstrate that populations of RL agents, operating without explicit programming or intrinsic rewards, can invent, learn, enforce, and transmit a dominance hierarchy to new populations. The dominance hierarchies that emerge have a similar structure to those studied in chickens, mice, fish, and other species.
Chicken Coop is a multi-agent environment in which agents learn to form dominance hierarchies with each other, like those that form in animal societies. The environment is a multi-player variant of the classic [game of chicken](https://en.wikipedia.org/wiki/Chicken_(game)).
The figure above shows the aggressiveness levels of each of the six agents (depicted as differently-colored lines) in four sample Chicken Coop populations. The two populations in the top row form transitive dominance hierarchies, while the two populations in the bottom row form intransitive dominance hierarchies.
## Installation
Tested on Python 3.11.
```
# Set up virtualenv if needed:
python -m venv chicken-coop-env
source chicken-coop-env/bin/activate
pip install chicken-coop
# Run tests
_test_chicken_coop
```
## Usage
Basic run:
```
chicken_coop run
```
Main run used in the paper:
```
chicken_coop run --moniker paper-run --use-tune --n-tune-samples 300
```
Transplant experienced agents into a naive population:
```
chicken_coop transplant --moniker paper-transplant --visitor-trek \
~/.chicken_coop/<YOUR_PREVIOUS_RUN>
```
Ablate opponent perception accuracy:
```
chicken_coop run --moniker paper-ablate-observation --use-tune --n-tune-samples 10 \
--observation-accuracy 0.0 \
--observation-accuracy 0.1 \
--observation-accuracy 0.2 \
--observation-accuracy 0.3 \
--observation-accuracy 0.4 \
--observation-accuracy 0.5 \
--observation-accuracy 0.6 \
--observation-accuracy 0.7 \
--observation-accuracy 0.8 \
--observation-accuracy 0.9 \
--observation-accuracy 1.0
```
Investigate non-linearity:
```
chicken_coop run --moniker paper-cycles --use-tune --n-tune-samples 30 --n-agents 12 \
--learning-rate 3e-05
```
## Citing
If you used either the Chicken Coop module, or any insights from our paper, please cite our paper:
```
@inproceedings{rachum2024dh,
title = {Emergent Dominance Hierarchies in Reinforcement Learning Agents},
author = {Rachum, Ram and Nakar, Yonatan and Tomlinson, Bill and Alon, Nitay and Mirsky, Reuth},
year = 2024,
publisher = {International Foundation for Autonomous Agents and Multiagent Systems},
booktitle = {Proceedings of the 2024 International Conference on Autonomous Agents and Multiagent Systems},
location = {Auckland, New Zealand},
series = {AAMAS '24}
}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/cool-RR/chicken_coop",
"name": "chicken-coop",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "Ram Rachum",
"author_email": "ram@rachum.com",
"download_url": "https://files.pythonhosted.org/packages/a7/f2/9cf4620da50d41aa2f742835f041367e246c97e6d3ff2d00d5f20350e857/chicken_coop-0.0.5.tar.gz",
"platform": null,
"description": "# Chicken Coop\n\n* [Full paper](https://r.rachum.com/dh-paper)\n* Coming soon: Extended abstract\n* Coming soon: Poster\n* Coming soon: Deck\n* Coming soon: Talk video\n\nThis is the accompanying code for our paper **Emergent Dominance Hierarchies in Reinforcement Learning Agents** presented in AAMAS 2024.\n\nAbstract:\n\n> Modern Reinforcement Learning (RL) algorithms are able to outperform humans in a wide variety of tasks. Multi-agent reinforcement learning (MARL) settings present additional challenges, and successful cooperation in mixed-motive groups of agents depends on a delicate balancing act between individual and group objectives. Social conventions and norms, often inspired by human institutions, are used as tools for striking this balance.\n>\n> In this paper, we examine a fundamental, well-studied social convention that underlies cooperation in both animal and human societies: Dominance hierarchies.\n>\n> We adapt the ethological theory of dominance hierarchies to artificial agents, borrowing the established terminology and definitions with as few amendments as possible. We demonstrate that populations of RL agents, operating without explicit programming or intrinsic rewards, can invent, learn, enforce, and transmit a dominance hierarchy to new populations. The dominance hierarchies that emerge have a similar structure to those studied in chickens, mice, fish, and other species.\n\nChicken Coop is a multi-agent environment in which agents learn to form dominance hierarchies with each other, like those that form in animal societies. The environment is a multi-player variant of the classic [game of chicken](https://en.wikipedia.org/wiki/Chicken_(game)).\n\n\n\nThe figure above shows the aggressiveness levels of each of the six agents (depicted as differently-colored lines) in four sample Chicken Coop populations. The two populations in the top row form transitive dominance hierarchies, while the two populations in the bottom row form intransitive dominance hierarchies.\n\n## Installation\n\nTested on Python 3.11.\n\n```\n# Set up virtualenv if needed:\npython -m venv chicken-coop-env\nsource chicken-coop-env/bin/activate\n\npip install chicken-coop\n\n# Run tests\n_test_chicken_coop\n```\n\n## Usage\n\n\nBasic run:\n\n```\nchicken_coop run\n```\n\nMain run used in the paper:\n\n```\nchicken_coop run --moniker paper-run --use-tune --n-tune-samples 300\n```\n\nTransplant experienced agents into a naive population:\n\n```\nchicken_coop transplant --moniker paper-transplant --visitor-trek \\\n ~/.chicken_coop/<YOUR_PREVIOUS_RUN>\n```\n\nAblate opponent perception accuracy:\n\n```\nchicken_coop run --moniker paper-ablate-observation --use-tune --n-tune-samples 10 \\\n --observation-accuracy 0.0 \\\n --observation-accuracy 0.1 \\\n --observation-accuracy 0.2 \\\n --observation-accuracy 0.3 \\\n --observation-accuracy 0.4 \\\n --observation-accuracy 0.5 \\\n --observation-accuracy 0.6 \\\n --observation-accuracy 0.7 \\\n --observation-accuracy 0.8 \\\n --observation-accuracy 0.9 \\\n --observation-accuracy 1.0\n```\n\nInvestigate non-linearity:\n\n```\nchicken_coop run --moniker paper-cycles --use-tune --n-tune-samples 30 --n-agents 12 \\\n --learning-rate 3e-05\n```\n\n## Citing\n\nIf you used either the Chicken Coop module, or any insights from our paper, please cite our paper:\n\n```\n@inproceedings{rachum2024dh,\n title = {Emergent Dominance Hierarchies in Reinforcement Learning Agents},\n author = {Rachum, Ram and Nakar, Yonatan and Tomlinson, Bill and Alon, Nitay and Mirsky, Reuth},\n year = 2024,\n publisher = {International Foundation for Autonomous Agents and Multiagent Systems},\n booktitle = {Proceedings of the 2024 International Conference on Autonomous Agents and Multiagent Systems},\n location = {Auckland, New Zealand},\n series = {AAMAS '24}\n}\n```\n\n",
"bugtrack_url": null,
"license": "",
"summary": "An environment for reproducing dominance hierarchies in RL agents",
"version": "0.0.5",
"project_urls": {
"Homepage": "https://github.com/cool-RR/chicken_coop"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "bcdda187b3fdea1302b3927ef5187292ef33b3982c8db81b7fde934e37bc2bee",
"md5": "0f2a1318050d175fda9db5c5bd5d3bdc",
"sha256": "2a91f2d2a085fae9b5ec96b4c669131717a523ff0c9f31139efc54bf36390d19"
},
"downloads": -1,
"filename": "chicken_coop-0.0.5-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "0f2a1318050d175fda9db5c5bd5d3bdc",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 63833,
"upload_time": "2023-12-25T18:38:32",
"upload_time_iso_8601": "2023-12-25T18:38:32.002612Z",
"url": "https://files.pythonhosted.org/packages/bc/dd/a187b3fdea1302b3927ef5187292ef33b3982c8db81b7fde934e37bc2bee/chicken_coop-0.0.5-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a7f29cf4620da50d41aa2f742835f041367e246c97e6d3ff2d00d5f20350e857",
"md5": "e17628cbacd4018a855b7e061a9f637c",
"sha256": "6db73da96fd6db4082aff68c96d60f0a799c40573ca0e8d839f0ec30b02ef7be"
},
"downloads": -1,
"filename": "chicken_coop-0.0.5.tar.gz",
"has_sig": false,
"md5_digest": "e17628cbacd4018a855b7e061a9f637c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 50817,
"upload_time": "2023-12-25T18:38:34",
"upload_time_iso_8601": "2023-12-25T18:38:34.477759Z",
"url": "https://files.pythonhosted.org/packages/a7/f2/9cf4620da50d41aa2f742835f041367e246c97e6d3ff2d00d5f20350e857/chicken_coop-0.0.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-12-25 18:38:34",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cool-RR",
"github_project": "chicken_coop",
"github_not_found": true,
"lcname": "chicken-coop"
}
JSON
