| Name | chatarena JSON |
| Version |
0.1.18
JSON |
| download |
| home_page | |
| Summary | Multi-Agent Language Game Environments for Large Language Models |
| upload_time | 2023-12-21 00:42:32 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.7 |
| license | |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
<!--
Title: Chat Arena
Description: Chat Arena (or ChatArena) is a language game environment for Large Language Models (LLMs) like GPT-3, GPT-4, ChatGPT, etc.
Author: Yuxiang Wu
-->
<h1 align="center"> 🏟 <span style="color:orange"><a href="https://www.chatarena.org/">ChatArena</a></span> </h1>
<h3 align="center">
<p>Multi-Agent Language Game Environments for LLMs</p>
</h3>
[](https://github.com/chatarena/chatarena/blob/main/LICENSE)
[](https://pypi.org/project/chatarena/)
[](https://www.python.org/downloads/release/python-390/)
[](https://twitter.com/_chatarena)
[](https://join.slack.com/t/chatarena/shared_invite/zt-1t5fpbiep-CbKucEHdJ5YeDLEpKWxDOg)
[](https://colab.research.google.com/drive/1vKaskNMBtuGOVgn8fQxMgjCevn2wp1Ml?authuser=0#scrollTo=P5DCC0Y0Zbxi)
[](https://chatarena-chatarena-demo.hf.space)
---
ChatArena is a library that provides multi-agent language game environments and facilitates research about autonomous
LLM agents and their social interactions.
It provides the following features:
- **Abstraction**: it provides a flexible framework to define multiple players, environments and the interactions
between them, based on Markov Decision Process.
- **Language Game Environments**: it provides a set of environments that can help understanding, benchmarking or
training agent LLMs.
- **User-friendly Interfaces**: it provides both Web UI and CLI to develop/prompt engineer your LLM agents to act in
environments.
## Getting Started
**Try our online demo:**
[](https://chatarena-chatarena-demo.hf.space)
[](https://vimeo.com/816979419)
### Installation
Requirements:
- Python >= 3. 7
- OpenAI API key (optional, for using GPT-3.5-turbo or GPT-4 as an LLM agent)
Install with pip:
```bash
pip install chatarena
```
or install from source:
```bash
pip install git+https://github.com/chatarena/chatarena
```
To use GPT-3 as an LLM agent, set your OpenAI API key:
```bash
export OPENAI_API_KEY="your_api_key_here"
```
#### Optional Dependencies
By default `pip install chatarena` will only install dependencies necessary for ChatArena's core functionalities.
You can install optional dependencies with the following commands:
```bash
pip install chatarena[all_backends] # install dependencies for all supported backends: anthropic, cohere, huggingface, etc.
pip install chatarena[all_envs] # install dependencies for all environments, such as pettingzoo
pip install chatarena[all] # install all optional dependencies for full functionality
```
### Launch the Demo Locally
The quickest way to see ChatArena in action is via the demo Web UI.
To launch the demo on your local machine, you first pip install chatarena with extra gradio dependency, then git clone
this repository to your local folder, and finally call the `app.py` in the root directory of the repository:
```shell
pip install chatarena[gradio]
git clone https://github.com/chatarena/chatarena.git
cd chatarena
gradio app.py
```
This will launch a demo server for ChatArena, and you can access it from your browser (port 8080).
[//]: # (The interface looks like this:)
[//]: # ()
Check out this video to learn how to use Web UI: [](https://vimeo.com/816979419)
## For Developers
For an introduction to the ChatArena framework, please refer to [this document](docs/devdoc/design.md).
For a walkthrough of building a new environment, check [](https://colab.research.google.com/drive/1vKaskNMBtuGOVgn8fQxMgjCevn2wp1Ml?authuser=0#scrollTo=P5DCC0Y0Zbxi)
Here we provide a compact guide on minimal setup to run the game and some general advice on customization.
### Key Concepts
1. **Arena**: Arena encapsulates an environment and a collection of players. It drives the main loop of the game and
provides HCI utilities like webUI, CLI, configuration loading and data storage.
2. **Environment**: The environment stores the game state and executes game logics to make transitions between game
states. It also renders observations for players, the observations are natural languages.
1. The game state is not directly visible to the players. Players can only see the observations.
3. **Language Backend**: Language backends are the source of language intelligence. It takes text (or collection of
text) as input and returns text in response.
4. **Player**: The player is an agent that plays the game. In RL terminology, it’s a policy, a stateless function
mapping from observations to actions.
### Run the Game with Python API
Load `Arena` from a config file -- here we use `examples/nlp-classroom-3players.json` in this repository as an example:
```python
arena = Arena.from_config("examples/nlp-classroom-3players.json")
arena.run(num_steps=10)
```
Run the game in an interactive CLI interface:
```python
arena.launch_cli()
```
Check out this video to learn how to use
CLI: [](https://vimeo.com/816989884)
A more detailed guide about how to run the main interaction loop with finer-grained control can be
found [here](docs/devdoc/mainloop.md)
### General Customization Advice
1. **Arena**: Overriding Arena basically means one is going to write their own main loop. This can allow different
interaction interfaces or drive games in a more automated manner, for example, running an online RL training loop
2. **Environment**: A new environment corresponds to a new game, one can define the game dynamics here with hard-coded
rules or a mixture of rules and language backend.
3. **Backend**: If one needs to change the way of formatting observations (in terms of messages) into queries for the
language model, the backend should be overridden.
4. **Player**: By default, when a new observation is fed, players will query the language backend and return the
response as actions. But one can also customize the way that players are interacting with the language backend.
### Creating your Custom Environment
You can define your own environment by extending the `Environment` class. Here are the general steps:
1. Define the class by inheriting from a base class and setting `type_name`, then add the class
to [`ALL_ENVIRONMENTS`](chatarena/environments/__init__.py#L17)
2. Initialize the class by defining `__init__` method (its arguments will define the corresponding config) and
initializing class attributes
3. Implement game mechanics in methods `step`
4. Handle game states and rewards by implementing methods such as `reset`, `get_observation`, `is_terminal`,
and `get_rewards`
5. Develop role description prompts (and a global prompt if necessary) for players using CLI or Web UI and save them to
a
config file.
We provide [a detailed tutorial](docs/tutorials/create_your_environment.md) to demonstrate how to define a custom
environment,
using the [`Chameleon` environment](chatarena/environments/chameleon.py) as example.
If you want to port an existing library's environment to ChatArena, check
out [`PettingzooChess` environment](chatarena/environments/pettingzoo_chess.py) as an example.
## List of Environments
### [Conversation](chatarena/environments/conversation.py)
A multi-player language game environment that simulates a
conversation.
* [NLP Classroom](examples/nlp-classroom-3players.json): a 3-player language game environment that simulates a
classroom
setting. The game is played in turns, and each turn a player can either ask a question or answer a question.
The game ends when all players have asked and answered all questions.
### [Moderator Conversation](chatarena/environments/conversation.py)
Based on conversation, but with a moderator that controls the game dynamics.
* [Rock-paper-scissors](examples/rock-paper-scissors.json): a 2-player language game environment that simulates a
rock-paper-scissors game with moderator conversation.
Both player will act in parallel, and the game ends when one player wins 2 rounds.
* [Tic-tac-toe](examples/tic-tac-toe.json): a 2-player language game environment that simulates a tic-tac-toe
game with moderator conversation.
The game is played in turns, and each turn a player can either ask for a move or make a move. The game ends when
one
player wins or the board is full.
### [Chameleon](chatarena/environments/chameleon.py)
A multi-player social deduction game. There are two roles in the game, chameleon and non-chameleon.
The topic of the secret word will be first revealed to all the players.
Then the secret word will be revealed to non-chameleons.
The chameleon does not know the secret word.
The objective in the game depends on the role of the player:
- If you are not a chameleon, your goal is to reveal the chameleon without exposing the secret word.
- If you are a chameleon, your aim is to blend in with other players, avoid being caught, and figure out the secret
word.
There are three stages in the game:
1. The giving clues stage: each player will describe the clues about the secret word.
2. The accusation stage: In this stage, each player will vote for another player who is most likely the chameleon. The
chameleon should vote for other players.
3. The guess stage: If the accusation is correct, the chameleon should guess the secret word given the clues revealed by
other players.
### [PettingZooChess](chatarena/environments/pettingzoo_chess.py)
A two-player chess game environment that uses the PettingZoo Chess environment.
### [PettingZooTicTacTeo](chatarena/environments/pettingzoo_tictactoe.py)
A two-player tic-tac-toe game environment that uses the PettingZoo TicTacToe environment. Differing from the
`Moderator Conversation` environment, this environment is driven by hard-coded rules rather than a LLM moderator.
## Contributing
We welcome contributions to improve and extend ChatArena. Please follow these steps to contribute:
1. Fork the repository.
2. Create a new branch for your feature or bugfix.
3. Commit your changes to the new branch.
4. Create a pull request describing your changes.
5. We will review your pull request and provide feedback or merge your changes.
Please ensure your code follows the existing style and structure.
## Citation
If you find ChatArena useful for your research, please cite our repository (our arxiv paper is coming soon):
```bibtex
@software{ChatArena,
author = {Yuxiang Wu, Zhengyao Jiang, Akbir Khan, Yao Fu, Laura Ruis, Edward Grefenstette, and Tim Rocktäschel},
title = {ChatArena: Multi-Agent Language Game Environments for Large Language Models},
year = {2023},
publisher = {GitHub},
journal = {GitHub repository},
version = {0.1},
howpublished = {\url{https://github.com/chatarena/chatarena}},
}
```
## Contact
If you have any questions or suggestions, feel free to open an issue or submit a pull request.
You can also contact us on the Farama discord server- https://discord.gg/Vrtdmu9Y8Q
Happy chatting!
## Sponsors
We would like to thank our sponsors for supporting this project:
- [SEQUOIA](https://www.sequoiacap.com/)
- [Shixiang Capital](https://sx.shixiangcap.com/home)
Raw data
{
"_id": null,
"home_page": "",
"name": "chatarena",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "",
"author": "",
"author_email": "Yuxiang Wu <yuxiang.cs@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/7a/11/ea5fe0fce7e09ededc29e5db59bffc3d47b4235fa123714dc7a61752e809/chatarena-0.1.18.tar.gz",
"platform": null,
"description": "<!--\n Title: Chat Arena\n Description: Chat Arena (or ChatArena) is a language game environment for Large Language Models (LLMs) like GPT-3, GPT-4, ChatGPT, etc.\n Author: Yuxiang Wu\n -->\n\n<h1 align=\"center\"> \ud83c\udfdf <span style=\"color:orange\"><a href=\"https://www.chatarena.org/\">ChatArena</a></span> </h1>\n\n<h3 align=\"center\">\n <p>Multi-Agent Language Game Environments for LLMs</p>\n</h3>\n\n\n[](https://github.com/chatarena/chatarena/blob/main/LICENSE)\n[](https://pypi.org/project/chatarena/)\n[](https://www.python.org/downloads/release/python-390/)\n[](https://twitter.com/_chatarena)\n[](https://join.slack.com/t/chatarena/shared_invite/zt-1t5fpbiep-CbKucEHdJ5YeDLEpKWxDOg)\n[](https://colab.research.google.com/drive/1vKaskNMBtuGOVgn8fQxMgjCevn2wp1Ml?authuser=0#scrollTo=P5DCC0Y0Zbxi)\n[](https://chatarena-chatarena-demo.hf.space)\n\n---\n\nChatArena is a library that provides multi-agent language game environments and facilitates research about autonomous\nLLM agents and their social interactions.\nIt provides the following features:\n\n- **Abstraction**: it provides a flexible framework to define multiple players, environments and the interactions\n between them, based on Markov Decision Process.\n- **Language Game Environments**: it provides a set of environments that can help understanding, benchmarking or\n training agent LLMs.\n- **User-friendly Interfaces**: it provides both Web UI and CLI to develop/prompt engineer your LLM agents to act in\n environments.\n\n\n\n## Getting Started\n\n**Try our online demo:**\n[](https://chatarena-chatarena-demo.hf.space)\n[](https://vimeo.com/816979419)\n\n### Installation\n\nRequirements:\n\n- Python >= 3. 7\n- OpenAI API key (optional, for using GPT-3.5-turbo or GPT-4 as an LLM agent)\n\nInstall with pip:\n\n```bash\npip install chatarena\n```\n\nor install from source:\n\n```bash\npip install git+https://github.com/chatarena/chatarena\n```\n\nTo use GPT-3 as an LLM agent, set your OpenAI API key:\n\n```bash\nexport OPENAI_API_KEY=\"your_api_key_here\"\n```\n\n#### Optional Dependencies\n\nBy default `pip install chatarena` will only install dependencies necessary for ChatArena's core functionalities.\nYou can install optional dependencies with the following commands:\n```bash\npip install chatarena[all_backends] # install dependencies for all supported backends: anthropic, cohere, huggingface, etc.\npip install chatarena[all_envs] # install dependencies for all environments, such as pettingzoo\npip install chatarena[all] # install all optional dependencies for full functionality\n```\n\n### Launch the Demo Locally\n\nThe quickest way to see ChatArena in action is via the demo Web UI.\nTo launch the demo on your local machine, you first pip install chatarena with extra gradio dependency, then git clone\nthis repository to your local folder, and finally call the `app.py` in the root directory of the repository:\n\n```shell\npip install chatarena[gradio]\ngit clone https://github.com/chatarena/chatarena.git\ncd chatarena\ngradio app.py\n```\n\nThis will launch a demo server for ChatArena, and you can access it from your browser (port 8080).\n\n[//]: # (The interface looks like this:)\n\n[//]: # ()\n\nCheck out this video to learn how to use Web UI: [](https://vimeo.com/816979419)\n\n## For Developers\n\nFor an introduction to the ChatArena framework, please refer to [this document](docs/devdoc/design.md).\nFor a walkthrough of building a new environment, check [](https://colab.research.google.com/drive/1vKaskNMBtuGOVgn8fQxMgjCevn2wp1Ml?authuser=0#scrollTo=P5DCC0Y0Zbxi)\n\nHere we provide a compact guide on minimal setup to run the game and some general advice on customization.\n\n### Key Concepts\n\n1. **Arena**: Arena encapsulates an environment and a collection of players. It drives the main loop of the game and\n provides HCI utilities like webUI, CLI, configuration loading and data storage.\n2. **Environment**: The environment stores the game state and executes game logics to make transitions between game\n states. It also renders observations for players, the observations are natural languages.\n 1. The game state is not directly visible to the players. Players can only see the observations.\n3. **Language Backend**: Language backends are the source of language intelligence. It takes text (or collection of\n text) as input and returns text in response.\n4. **Player**: The player is an agent that plays the game. In RL terminology, it\u2019s a policy, a stateless function\n mapping from observations to actions.\n\n### Run the Game with Python API\n\nLoad `Arena` from a config file -- here we use `examples/nlp-classroom-3players.json` in this repository as an example:\n\n```python\narena = Arena.from_config(\"examples/nlp-classroom-3players.json\")\narena.run(num_steps=10)\n```\n\nRun the game in an interactive CLI interface:\n\n```python\narena.launch_cli()\n```\n\nCheck out this video to learn how to use\nCLI: [](https://vimeo.com/816989884)\nA more detailed guide about how to run the main interaction loop with finer-grained control can be\nfound [here](docs/devdoc/mainloop.md)\n\n### General Customization Advice\n\n1. **Arena**: Overriding Arena basically means one is going to write their own main loop. This can allow different\n interaction interfaces or drive games in a more automated manner, for example, running an online RL training loop\n2. **Environment**: A new environment corresponds to a new game, one can define the game dynamics here with hard-coded\n rules or a mixture of rules and language backend.\n3. **Backend**: If one needs to change the way of formatting observations (in terms of messages) into queries for the\n language model, the backend should be overridden.\n4. **Player**: By default, when a new observation is fed, players will query the language backend and return the\n response as actions. But one can also customize the way that players are interacting with the language backend.\n\n### Creating your Custom Environment\n\nYou can define your own environment by extending the `Environment` class. Here are the general steps:\n\n1. Define the class by inheriting from a base class and setting `type_name`, then add the class\n to [`ALL_ENVIRONMENTS`](chatarena/environments/__init__.py#L17)\n2. Initialize the class by defining `__init__` method (its arguments will define the corresponding config) and\n initializing class attributes\n3. Implement game mechanics in methods `step`\n4. Handle game states and rewards by implementing methods such as `reset`, `get_observation`, `is_terminal`,\n and `get_rewards`\n5. Develop role description prompts (and a global prompt if necessary) for players using CLI or Web UI and save them to\n a\n config file.\n\nWe provide [a detailed tutorial](docs/tutorials/create_your_environment.md) to demonstrate how to define a custom\nenvironment,\nusing the [`Chameleon` environment](chatarena/environments/chameleon.py) as example.\n\nIf you want to port an existing library's environment to ChatArena, check\nout [`PettingzooChess` environment](chatarena/environments/pettingzoo_chess.py) as an example.\n\n## List of Environments\n\n### [Conversation](chatarena/environments/conversation.py)\n\nA multi-player language game environment that simulates a\nconversation.\n\n* [NLP Classroom](examples/nlp-classroom-3players.json): a 3-player language game environment that simulates a\n classroom\n setting. The game is played in turns, and each turn a player can either ask a question or answer a question.\n The game ends when all players have asked and answered all questions.\n\n### [Moderator Conversation](chatarena/environments/conversation.py)\n\nBased on conversation, but with a moderator that controls the game dynamics.\n\n* [Rock-paper-scissors](examples/rock-paper-scissors.json): a 2-player language game environment that simulates a\n rock-paper-scissors game with moderator conversation.\n Both player will act in parallel, and the game ends when one player wins 2 rounds.\n* [Tic-tac-toe](examples/tic-tac-toe.json): a 2-player language game environment that simulates a tic-tac-toe\n game with moderator conversation.\n The game is played in turns, and each turn a player can either ask for a move or make a move. The game ends when\n one\n player wins or the board is full.\n\n### [Chameleon](chatarena/environments/chameleon.py)\n\nA multi-player social deduction game. There are two roles in the game, chameleon and non-chameleon.\nThe topic of the secret word will be first revealed to all the players.\nThen the secret word will be revealed to non-chameleons.\nThe chameleon does not know the secret word.\nThe objective in the game depends on the role of the player:\n\n- If you are not a chameleon, your goal is to reveal the chameleon without exposing the secret word.\n- If you are a chameleon, your aim is to blend in with other players, avoid being caught, and figure out the secret\n word.\n There are three stages in the game:\n\n1. The giving clues stage: each player will describe the clues about the secret word.\n2. The accusation stage: In this stage, each player will vote for another player who is most likely the chameleon. The\n chameleon should vote for other players.\n3. The guess stage: If the accusation is correct, the chameleon should guess the secret word given the clues revealed by\n other players.\n\n### [PettingZooChess](chatarena/environments/pettingzoo_chess.py)\n\nA two-player chess game environment that uses the PettingZoo Chess environment.\n\n### [PettingZooTicTacTeo](chatarena/environments/pettingzoo_tictactoe.py)\n\nA two-player tic-tac-toe game environment that uses the PettingZoo TicTacToe environment. Differing from the\n`Moderator Conversation` environment, this environment is driven by hard-coded rules rather than a LLM moderator.\n\n## Contributing\n\nWe welcome contributions to improve and extend ChatArena. Please follow these steps to contribute:\n\n1. Fork the repository.\n2. Create a new branch for your feature or bugfix.\n3. Commit your changes to the new branch.\n4. Create a pull request describing your changes.\n5. We will review your pull request and provide feedback or merge your changes.\n\nPlease ensure your code follows the existing style and structure.\n\n## Citation\n\nIf you find ChatArena useful for your research, please cite our repository (our arxiv paper is coming soon):\n\n```bibtex\n@software{ChatArena,\n author = {Yuxiang Wu, Zhengyao Jiang, Akbir Khan, Yao Fu, Laura Ruis, Edward Grefenstette, and Tim Rockt\u00e4schel},\n title = {ChatArena: Multi-Agent Language Game Environments for Large Language Models},\n year = {2023},\n publisher = {GitHub},\n journal = {GitHub repository},\n version = {0.1},\n howpublished = {\\url{https://github.com/chatarena/chatarena}},\n}\n```\n\n## Contact\n\nIf you have any questions or suggestions, feel free to open an issue or submit a pull request.\nYou can also contact us on the Farama discord server- https://discord.gg/Vrtdmu9Y8Q\n\nHappy chatting!\n\n## Sponsors\n\nWe would like to thank our sponsors for supporting this project:\n\n- [SEQUOIA](https://www.sequoiacap.com/)\n- [Shixiang Capital](https://sx.shixiangcap.com/home)\n",
"bugtrack_url": null,
"license": "",
"summary": "Multi-Agent Language Game Environments for Large Language Models",
"version": "0.1.18",
"project_urls": {
"Bug Tracker": "https://github.com/chatarena/chatarena/issues",
"Homepage": "https://github.com/chatarena/chatarena"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "df8cf43df468507d1df2d4de13fe51db5f5e82b2ed712656f29fc8a3ff886e25",
"md5": "8584ff230c3befa60320f8fa60528136",
"sha256": "e412f48884312fa7e60afe8619e6719c9234383919f57da16d25e29bb1f1274d"
},
"downloads": -1,
"filename": "chatarena-0.1.18-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8584ff230c3befa60320f8fa60528136",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 79122,
"upload_time": "2023-12-21T00:42:30",
"upload_time_iso_8601": "2023-12-21T00:42:30.449997Z",
"url": "https://files.pythonhosted.org/packages/df/8c/f43df468507d1df2d4de13fe51db5f5e82b2ed712656f29fc8a3ff886e25/chatarena-0.1.18-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7a11ea5fe0fce7e09ededc29e5db59bffc3d47b4235fa123714dc7a61752e809",
"md5": "7378875c3009d9d03cbdd40a46e04d23",
"sha256": "f3280f25bf855ae9ea41db3d3869d5e18026091ca0422d9a8e03c4109ecb2d80"
},
"downloads": -1,
"filename": "chatarena-0.1.18.tar.gz",
"has_sig": false,
"md5_digest": "7378875c3009d9d03cbdd40a46e04d23",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 59651,
"upload_time": "2023-12-21T00:42:32",
"upload_time_iso_8601": "2023-12-21T00:42:32.656458Z",
"url": "https://files.pythonhosted.org/packages/7a/11/ea5fe0fce7e09ededc29e5db59bffc3d47b4235fa123714dc7a61752e809/chatarena-0.1.18.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-12-21 00:42:32",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "chatarena",
"github_project": "chatarena",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "chatarena"
}
