<picture>
<source media="(prefers-color-scheme: dark)" srcset="docs/cfrainbow_readme_banner_dark.png">
<source media="(prefers-color-scheme: light)" srcset="docs/cfrainbow_readme_banner_light.png">
<img alt="Readme banner." src="docs/cfrainbow_readme_banner_light.png">
</picture>
---
# <p align=center> CFRainbow
<p align=center> Implementations Of <i>Counterfactual Regret Minimization</i> In Its Many Shapes & Colors
[](https://github.com/maichmueller/cfrainbow/actions/workflows/python.yml)
[](https://codecov.io/gh/maichmueller/cfrainbow)
CFRainbow provides implementations of the basic CFR algorithm and some of the improved variants of CFR
for computing <b>Nash Equilibria</b> in the 2-player 0-sum case as well as <b>Correlated Equilibria</b> in the general-sum case.
The pacakge is loosely built for modularity and general applicability by building on the Openspiel framework by Deepmind. CFRainbow is <b>not</b> built
for performance and will not scale well to large game implementations. Most algorithms are implemented with basic python data structures.
The package is in the early WIP phase.
# Usage
The easiest way to use CFRainbow is to import the desired algorithm from the package, a regret minimizer, and the `run` method with a pyspiel game object or its OpenSpiel name as input.
For example, to use the Vanilla CFR algorithm:
```python
import cfrainbow
from cfrainbow import cfr, rm
cfrainbow.run(
cfr.VanillaCFR,
n_iter=1000,
game="kuhn_poker",
regret_minimizer=rm.RegretMatcher
)
```
This will run the algorithm for the number of iterations given and compute the exploitability.
You can also use the solver object directly and call `iterate` on it after instantiating it with the correct arguments.
For example:
```python
from cfrainbow import cfr, rm
from cfrainbow.utils import load_game, KuhnPolicyPrinter, normalize_policy_profile
root_state = load_game("kuhn_poker").new_initial_states()
solver = cfr.VanillaCFR(
root_state,
regret_minimizer_type=rm.RegretMatcher,
alternating=True, # whether to do alternating or simultaneous updates
)
# iterate for a given number of iterations.
for i in range(1000):
solver.iterate()
avg_policy = normalize_policy_profile(solver.average_policy())
print(
KuhnPolicyPrinter(digits=2).print_profile(avg_policy)
)
```
Output (Infostate --> Action Policy):
```
PLAYER 0:
P1: Jack | P2: ? | cb --> ['check: 1.00', 'bet: 0.00']
P1: Jack | P2: ? --> ['check: 0.81', 'bet: 0.19']
P1: Queen | P2: ? | cb --> ['check: 0.48', 'bet: 0.52']
P1: Queen | P2: ? --> ['check: 0.99', 'bet: 0.01']
P1: King | P2: ? | cb --> ['check: 0.00', 'bet: 1.00']
P1: King | P2: ? --> ['check: 0.42', 'bet: 0.58']
PLAYER 1:
P1: ? | P2: Queen | c --> ['check: 0.99', 'bet: 0.01']
P1: ? | P2: Queen | b --> ['check: 0.66', 'bet: 0.34']
P1: ? | P2: King | c --> ['check: 0.00', 'bet: 1.00']
P1: ? | P2: King | b --> ['check: 0.00', 'bet: 1.00']
P1: ? | P2: Jack | c --> ['check: 0.67', 'bet: 0.33']
P1: ? | P2: Jack | b --> ['check: 1.00', 'bet: 0.00']
```
Note that in <i>alternating</i> updates each iteration is a single player's policy update.
In <i>simultaneous</i> updates both players' policy updates constitute one iteration.
## Available Algorithms
The following list shows the available algorithms that have been implemented (✔️), those that are still work in progress (🔨👷♂️),
and those that are planned to be implemented in the future (📅):
| Algorithm | Status | Convergence Tests | Paper Results Reproducing |
|-----------|:--------:|:-------------------:|:--------------------------:|
| [Best-Response CFR](https://www.cs.cmu.edu/~kwaugh/publications/johanson11.pdf) | ✔️ | ✔️ | 🔨👷 |
| [Discounted CFR](https://arxiv.org/abs/1809.04040) | ✔️ | ✔️ | 🔨👷 |
| [Exponential CFR](https://arxiv.org/abs/2008.02679) | ✔️ | ✔️ | ❌ |
| [Internal CFR](https://proceedings.neurips.cc/paper/2020/file/5763abe87ed1938799203fb6e8650025-Paper.pdf) | 🔨👷♂️ | 🔨👷 | 🔨👷 |
| [Joint-Reconstruction CFR](https://proceedings.neurips.cc/paper/2019/file/525b8410cc8612283c9ecaf9a319f8ed-Paper.pdf) | 🔨👷♂️ | 🔨👷 | 🔨👷 |
| [Chance Sampling Monte Carlo CFR](http://mlanctot.info/files/papers/nips09mccfr.pdf) | ✔️ | ✔️ | 🔨👷 |
| [External Sampling Monte Carlo CFR](http://mlanctot.info/files/papers/nips09mccfr.pdf) | ✔️ | ✔️ | 🔨👷 |
| [Outcome Sampling Monte Carlo CFR](http://mlanctot.info/files/papers/nips09mccfr.pdf) | ✔️ | ✔️ | 🔨👷 |
| [Predictive Plus CFR](https://arxiv.org/abs/1902.04982) | ✔️ | ✔️ | 🔨👷 |
| [Pure CFR](https://richardggibson.appspot.com/static/work/thesis-phd/thesis-phd-paper.pdf) | ✔️ | ✔️ | 🔨👷 |
| [Sampling CFR](https://proceedings.neurips.cc/paper/2019/file/525b8410cc8612283c9ecaf9a319f8ed-Paper.pdf) | ✔️ | 🔨👷 | 🔨👷 |
| [Vanilla CFR](https://proceedings.neurips.cc/paper/2007/file/08d98638c6fcd194a4b1e6992063e944-Paper.pdf) | ✔️ | ✔️ | 🔨👷 |
| [Lazy CFR](https://arxiv.org/pdf/1810.04433v3.pdf) | 📅 | 📅 | 📅 |
# Installation
### <b> Web Install </b>
<b> Pip </b>
To install CFRainbow from PyPi using pip, run the following command:
```bash
pip install cfrainbow
```
<b> Poetry </b>
If you're using Poetry to manage your Python packages, you can add CFRainbow to your project by adding the following to your pyproject.toml file:
```toml
[tool.poetry.dependencies]
cfrainbow = "*"
```
Then run:
```bash
poetry install
```
to install your own package. Poetry will then install `cfrainbow` as dependency alongside your own package.
### <b> Source Install </b>
To install CFRainbow from master, please follow these steps:
```bash
git clone https://github.com/maichmueller/cfrainbow.git
cd cfrainbow
```
and install the package with pip
```bash
pip install .
```
or poetry
```bash
poetry install
```
use the option `--no-dev` to ensure a non-editable installation.
Raw data
{
"_id": null,
"home_page": "https://github.com/maichmueller/cfrainbow",
"name": "cfrainbow",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "counterfactual regret minimization,regret minimization,extensive form games,cfr,cfr+,discounted cfr,linear,monte-carlo,openspiel",
"author": "Michael Aichmueller",
"author_email": "m.aichmueller@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/3b/f6/4b04747aa3c58b95d04ed06302aa53d86374945cab3be40992bdd3e2ad4a/cfrainbow-0.1.1.tar.gz",
"platform": null,
"description": "<picture>\n <source media=\"(prefers-color-scheme: dark)\" srcset=\"docs/cfrainbow_readme_banner_dark.png\">\n <source media=\"(prefers-color-scheme: light)\" srcset=\"docs/cfrainbow_readme_banner_light.png\">\n <img alt=\"Readme banner.\" src=\"docs/cfrainbow_readme_banner_light.png\">\n</picture>\n\n---\n\n# <p align=center> CFRainbow\n<p align=center> Implementations Of <i>Counterfactual Regret Minimization</i> In Its Many Shapes & Colors\n\n[](https://github.com/maichmueller/cfrainbow/actions/workflows/python.yml)\n[](https://codecov.io/gh/maichmueller/cfrainbow)\n\nCFRainbow provides implementations of the basic CFR algorithm and some of the improved variants of CFR\nfor computing <b>Nash Equilibria</b> in the 2-player 0-sum case as well as <b>Correlated Equilibria</b> in the general-sum case.\n\nThe pacakge is loosely built for modularity and general applicability by building on the Openspiel framework by Deepmind. CFRainbow is <b>not</b> built\nfor performance and will not scale well to large game implementations. Most algorithms are implemented with basic python data structures.\n\nThe package is in the early WIP phase.\n\n\n# Usage\n\nThe easiest way to use CFRainbow is to import the desired algorithm from the package, a regret minimizer, and the `run` method with a pyspiel game object or its OpenSpiel name as input.\nFor example, to use the Vanilla CFR algorithm:\n\n```python\n\nimport cfrainbow\nfrom cfrainbow import cfr, rm\n\ncfrainbow.run(\n cfr.VanillaCFR,\n n_iter=1000,\n game=\"kuhn_poker\",\n regret_minimizer=rm.RegretMatcher\n)\n```\nThis will run the algorithm for the number of iterations given and compute the exploitability.\n\nYou can also use the solver object directly and call `iterate` on it after instantiating it with the correct arguments.\nFor example:\n\n```python\nfrom cfrainbow import cfr, rm\nfrom cfrainbow.utils import load_game, KuhnPolicyPrinter, normalize_policy_profile\n\nroot_state = load_game(\"kuhn_poker\").new_initial_states()\n\nsolver = cfr.VanillaCFR(\n root_state,\n regret_minimizer_type=rm.RegretMatcher,\n alternating=True, # whether to do alternating or simultaneous updates\n)\n# iterate for a given number of iterations.\nfor i in range(1000):\n solver.iterate()\n\navg_policy = normalize_policy_profile(solver.average_policy())\n\nprint(\n KuhnPolicyPrinter(digits=2).print_profile(avg_policy)\n)\n```\nOutput (Infostate --> Action Policy):\n```\nPLAYER 0:\nP1: Jack | P2: ? | cb --> ['check: 1.00', 'bet: 0.00']\nP1: Jack | P2: ? --> ['check: 0.81', 'bet: 0.19']\nP1: Queen | P2: ? | cb --> ['check: 0.48', 'bet: 0.52']\nP1: Queen | P2: ? --> ['check: 0.99', 'bet: 0.01']\nP1: King | P2: ? | cb --> ['check: 0.00', 'bet: 1.00']\nP1: King | P2: ? --> ['check: 0.42', 'bet: 0.58']\nPLAYER 1:\nP1: ? | P2: Queen | c --> ['check: 0.99', 'bet: 0.01']\nP1: ? | P2: Queen | b --> ['check: 0.66', 'bet: 0.34']\nP1: ? | P2: King | c --> ['check: 0.00', 'bet: 1.00']\nP1: ? | P2: King | b --> ['check: 0.00', 'bet: 1.00']\nP1: ? | P2: Jack | c --> ['check: 0.67', 'bet: 0.33']\nP1: ? | P2: Jack | b --> ['check: 1.00', 'bet: 0.00']\n```\nNote that in <i>alternating</i> updates each iteration is a single player's policy update.\nIn <i>simultaneous</i> updates both players' policy updates constitute one iteration.\n\n## Available Algorithms\n\nThe following list shows the available algorithms that have been implemented (\u2714\ufe0f), those that are still work in progress (\ud83d\udd28\ud83d\udc77\u200d\u2642\ufe0f),\nand those that are planned to be implemented in the future (\ud83d\udcc5):\n\n| Algorithm | Status | Convergence Tests | Paper Results Reproducing |\n|-----------|:--------:|:-------------------:|:--------------------------:|\n| [Best-Response CFR](https://www.cs.cmu.edu/~kwaugh/publications/johanson11.pdf) | \u2714\ufe0f | \u2714\ufe0f | \ud83d\udd28\ud83d\udc77\u200d |\n| [Discounted CFR](https://arxiv.org/abs/1809.04040) | \u2714\ufe0f | \u2714\ufe0f | \ud83d\udd28\ud83d\udc77\u200d |\n| [Exponential CFR](https://arxiv.org/abs/2008.02679) | \u2714\ufe0f | \u2714\ufe0f | \u274c |\n| [Internal CFR](https://proceedings.neurips.cc/paper/2020/file/5763abe87ed1938799203fb6e8650025-Paper.pdf) | \ud83d\udd28\ud83d\udc77\u200d\u2642\ufe0f | \ud83d\udd28\ud83d\udc77\u200d | \ud83d\udd28\ud83d\udc77\u200d |\n| [Joint-Reconstruction CFR](https://proceedings.neurips.cc/paper/2019/file/525b8410cc8612283c9ecaf9a319f8ed-Paper.pdf) | \ud83d\udd28\ud83d\udc77\u200d\u2642\ufe0f | \ud83d\udd28\ud83d\udc77\u200d | \ud83d\udd28\ud83d\udc77\u200d |\n| [Chance Sampling Monte Carlo CFR](http://mlanctot.info/files/papers/nips09mccfr.pdf) | \u2714\ufe0f | \u2714\ufe0f | \ud83d\udd28\ud83d\udc77\u200d |\n| [External Sampling Monte Carlo CFR](http://mlanctot.info/files/papers/nips09mccfr.pdf) | \u2714\ufe0f | \u2714\ufe0f | \ud83d\udd28\ud83d\udc77\u200d |\n| [Outcome Sampling Monte Carlo CFR](http://mlanctot.info/files/papers/nips09mccfr.pdf) | \u2714\ufe0f | \u2714\ufe0f | \ud83d\udd28\ud83d\udc77\u200d |\n| [Predictive Plus CFR](https://arxiv.org/abs/1902.04982) | \u2714\ufe0f | \u2714\ufe0f | \ud83d\udd28\ud83d\udc77\u200d |\n| [Pure CFR](https://richardggibson.appspot.com/static/work/thesis-phd/thesis-phd-paper.pdf) | \u2714\ufe0f | \u2714\ufe0f | \ud83d\udd28\ud83d\udc77\u200d |\n| [Sampling CFR](https://proceedings.neurips.cc/paper/2019/file/525b8410cc8612283c9ecaf9a319f8ed-Paper.pdf) | \u2714\ufe0f | \ud83d\udd28\ud83d\udc77\u200d | \ud83d\udd28\ud83d\udc77\u200d |\n| [Vanilla CFR](https://proceedings.neurips.cc/paper/2007/file/08d98638c6fcd194a4b1e6992063e944-Paper.pdf) | \u2714\ufe0f | \u2714\ufe0f | \ud83d\udd28\ud83d\udc77\u200d |\n| [Lazy CFR](https://arxiv.org/pdf/1810.04433v3.pdf) | \ud83d\udcc5 | \ud83d\udcc5 | \ud83d\udcc5 |\n\n# Installation\n\n### <b> Web Install </b>\n\n<b> Pip </b>\n\nTo install CFRainbow from PyPi using pip, run the following command:\n```bash\npip install cfrainbow\n```\n\n<b> Poetry </b>\n\nIf you're using Poetry to manage your Python packages, you can add CFRainbow to your project by adding the following to your pyproject.toml file:\n\n```toml\n[tool.poetry.dependencies]\ncfrainbow = \"*\"\n```\n\nThen run:\n\n```bash\npoetry install\n```\n\nto install your own package. Poetry will then install `cfrainbow` as dependency alongside your own package.\n### <b> Source Install </b>\n\nTo install CFRainbow from master, please follow these steps:\n\n```bash\ngit clone https://github.com/maichmueller/cfrainbow.git\ncd cfrainbow\n```\nand install the package with pip\n```bash\npip install .\n```\nor poetry\n```bash\npoetry install\n```\nuse the option `--no-dev` to ensure a non-editable installation.\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Counterfactual Regret Minimization Open-Source Implementations",
"version": "0.1.1",
"project_urls": {
"Homepage": "https://github.com/maichmueller/cfrainbow"
},
"split_keywords": [
"counterfactual regret minimization",
"regret minimization",
"extensive form games",
"cfr",
"cfr+",
"discounted cfr",
"linear",
"monte-carlo",
"openspiel"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5f9ead8d1029211591a5f5e3c8d5572f754625e788da852de3ec8fb8845c1c54",
"md5": "62f824496f77c40cb515e3b4b5843a87",
"sha256": "c9ef4375179f898d7d086d9febc344c43d99bf56d7f571174c5d67f8e1cb81a3"
},
"downloads": -1,
"filename": "cfrainbow-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "62f824496f77c40cb515e3b4b5843a87",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 44410,
"upload_time": "2023-11-04T19:17:14",
"upload_time_iso_8601": "2023-11-04T19:17:14.346493Z",
"url": "https://files.pythonhosted.org/packages/5f/9e/ad8d1029211591a5f5e3c8d5572f754625e788da852de3ec8fb8845c1c54/cfrainbow-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3bf64b04747aa3c58b95d04ed06302aa53d86374945cab3be40992bdd3e2ad4a",
"md5": "a7cc528ce6a485eddd3b23c6f431df45",
"sha256": "a884e9f8f7e030662cd6d64a11fbd0a0d7211b4116ad080b7afc053e5612ddd7"
},
"downloads": -1,
"filename": "cfrainbow-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "a7cc528ce6a485eddd3b23c6f431df45",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 34189,
"upload_time": "2023-11-04T19:17:15",
"upload_time_iso_8601": "2023-11-04T19:17:15.879748Z",
"url": "https://files.pythonhosted.org/packages/3b/f6/4b04747aa3c58b95d04ed06302aa53d86374945cab3be40992bdd3e2ad4a/cfrainbow-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-04 19:17:15",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "maichmueller",
"github_project": "cfrainbow",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "cfrainbow"
}
JSON
