# Human + AI = ❤️
## <a href="https://cogment.ai/cogment_lab"><strong>Docs</strong></a> | <a href="https://ai-r.com/blog"><strong>Blog</strong></a> | <a href="https://discord.gg/kh3t6esJRy"><strong> Discord </strong></a>
[](https://pypi.org/project/cogment-lab)
[](https://pepy.tech/project/cogment-lab)
[](https://pypi.org/project/cogment-lab)
[](https://github.com/cogment-lab/blob/main/LICENSE)
[](https://twitter.com/AI_Redefined)
[](https://pre-commit.com/)
# Introduction
Cogment Lab is a toolkit for doing HILL RL -- that is human-in-the-loop learning, with an emphasis on reinforcement learning.
It is based on [cogment](https://cogment.ai), a low-level framework for exchanging messages between
environments, AI agents and humans.
It's the perfect tool for when you want to interact with your environment yourself, and maybe even trained AI agents.
# Installation guide
1. Activate your venv, conda env, or whatever you use to keep your python environment clean.
2. Install cogment_lab with `pip install cogment_lab`
3. Install cogment in `COGMENT_LAB_HOME` folder with `cogmentlab install` (this environment variable defaults to `~/.cogment_lab`)
4. In a separate terminal, run `cogmentlab launch base` to start the orchestrator and datastore. Keep it open.
5. Run the tutorials, examples, or whatever you want to do.
# Cogment interaction model
While it typically isn't necessary to interact with Cogment directly to use Cogment Lab, it is useful to understand the principles on which it operates.
Cogment exchanges messages between environments and actor. These messages contain the observations, actions, rewards, and anything
else that you might want to keep track of.
Interactions are split into Trials, which correspond to the typical notion of an episode in RL. Each trial has a unique ID, and
## Cogment Lab at a glance
Cogment Lab (as well as Cogment in general) follows a microservice-based architecture.
Each environment, agent, and human interface (collectively: service) is launched as a subprocess, and exchanges messages with the orchestrator,
which in turn ensures synchronization and correct routing of messages.
Generally speaking, you don't need to worry about any of that - Cogment Lab conveniently covers up all the rough edges,
allowing you to do your research without worries.
Cogment Lab is inherently asynchronous - but if you're not familiar with async python, don't worry about it.
The only things you need to remember are:
- Wrap your code in `async def main()`
- Run it with `asyncio.run(main())`
- When calling certain functions use the `await` keyword, e.g. `data = await cog.get_episode_data(...)`
If you are familiar with async programming, there's a lot of interesting things you can do with it - go crazy.
## Terminology
- A `service` is anything that interacts with the Cogment orchestrator. It can be an environment or an actor, including human actors.
- An `actor` in particular is the service that interacts with an environment, and often wraps an `agent`. The internal structure of an actor is entirely up to the user
- An `agent` is what we typically think of as an agent in RL - something that perceives its environment and acts upon it. We do not attempt to solve the agent foundation problem in this documentation.
- An `agent` is simultaneously the part of the environment that's taking an action - multiagent environments may have several agents, so we need to assign an actor to each agent.
## Known rough edges
- When running the web UI, you can open the tab only once per launched process. So if you open the UI, you can run however many trials you want, as long as you don't close it. If you do close it, you should kill the process and start a new one.
## Local installation
- Requires Python 3.10
- Install requirements in a virtual env with something similar to the following
```console
$ python -m venv .venv
$ source .venv/bin/activate
$ pip install -r requirements.txt
$ pip install -e .
```
- For the examples you'll need to install the additional `examples_requirements.txt`.
### Apple silicon installation
To run on M1/2/3 macs, you'll need to perform those additional steps
```
pip uninstall grpcio grpcio-tools
export GRPC_PYTHON_LDFLAGS=" -framework CoreFoundation"
pip install grpcio==1.60.0 grpcio-tools==1.60.0 --no-binary :all:
```
Adjust the version (here 1.60.0) to whatever you have installed.
## Usage
Run `cogmentlab launch base`.
Then, run whatever scripts or notebooks.
Terminology:
- Model: a relatively raw PyTorch (or other?) model, inheriting from `nn.Module`
- Agent: a model wrapped in some utility class to interact with np arrays
- Actor: a cogment service that may involve models and/or actors
## Release process
People having maintainers rights of the repository can follow these steps to release a version **MAJOR.MINOR.PATCH**. The versioning scheme follows [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
1. Run `./scripts/create_release_branch.sh MAJOR.MINOR.PATCH`, this will automatically:
- update the version of the package, in `cogment_lab/version.py`,
- create a release branch with the changes at `release/vMAJOR.MINOR.PATCH` and push it.
2. On the release branch:
- Make sure the changelog, at `CHANGELOG.md`, reflects the changes since the last release,
- Fix any issue, making sure that the build passes on CI,
- Commit and push any changes.
3. Run `./scripts/tag_release.sh MAJOR.MINOR.PATCH`, this will automatically:
- create the specific version section in the changelog and push it to the release branch,
- merge the release branch in `main`,
- create the release tag and,
- update the `develop` to match the latest release.
4. The CI will automatically publish the package to PyPI.
Raw data
{
"_id": null,
"home_page": null,
"name": "cogment_lab",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "AI, HILL, RL, Reinforcement Learning, human-in-the-loop",
"author": null,
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/13/2e/a23e9dbf52b1a8bb08deda7181304c5ed948ba025fbbb88b5ea4bd28d9d0/cogment_lab-0.1.5.tar.gz",
"platform": null,
"description": "\n\n# Human + AI = \u2764\ufe0f\n\n## <a href=\"https://cogment.ai/cogment_lab\"><strong>Docs</strong></a> | <a href=\"https://ai-r.com/blog\"><strong>Blog</strong></a> | <a href=\"https://discord.gg/kh3t6esJRy\"><strong> Discord </strong></a>\n\n[](https://pypi.org/project/cogment-lab)\n[](https://pepy.tech/project/cogment-lab)\n[](https://pypi.org/project/cogment-lab)\n[](https://github.com/cogment-lab/blob/main/LICENSE)\n[](https://twitter.com/AI_Redefined)\n[](https://pre-commit.com/)\n\n# Introduction\n\nCogment Lab is a toolkit for doing HILL RL -- that is human-in-the-loop learning, with an emphasis on reinforcement learning.\nIt is based on [cogment](https://cogment.ai), a low-level framework for exchanging messages between\nenvironments, AI agents and humans.\nIt's the perfect tool for when you want to interact with your environment yourself, and maybe even trained AI agents.\n\n# Installation guide\n\n1. Activate your venv, conda env, or whatever you use to keep your python environment clean.\n2. Install cogment_lab with `pip install cogment_lab`\n3. Install cogment in `COGMENT_LAB_HOME` folder with `cogmentlab install` (this environment variable defaults to `~/.cogment_lab`)\n4. In a separate terminal, run `cogmentlab launch base` to start the orchestrator and datastore. Keep it open.\n5. Run the tutorials, examples, or whatever you want to do.\n\n# Cogment interaction model\n\nWhile it typically isn't necessary to interact with Cogment directly to use Cogment Lab, it is useful to understand the principles on which it operates.\n\nCogment exchanges messages between environments and actor. These messages contain the observations, actions, rewards, and anything\nelse that you might want to keep track of.\n\nInteractions are split into Trials, which correspond to the typical notion of an episode in RL. Each trial has a unique ID, and\n\n## Cogment Lab at a glance\n\nCogment Lab (as well as Cogment in general) follows a microservice-based architecture.\nEach environment, agent, and human interface (collectively: service) is launched as a subprocess, and exchanges messages with the orchestrator,\nwhich in turn ensures synchronization and correct routing of messages.\n\nGenerally speaking, you don't need to worry about any of that - Cogment Lab conveniently covers up all the rough edges,\nallowing you to do your research without worries.\n\nCogment Lab is inherently asynchronous - but if you're not familiar with async python, don't worry about it.\nThe only things you need to remember are:\n\n- Wrap your code in `async def main()`\n- Run it with `asyncio.run(main())`\n- When calling certain functions use the `await` keyword, e.g. `data = await cog.get_episode_data(...)`\n\nIf you are familiar with async programming, there's a lot of interesting things you can do with it - go crazy.\n\n## Terminology\n\n- A `service` is anything that interacts with the Cogment orchestrator. It can be an environment or an actor, including human actors.\n- An `actor` in particular is the service that interacts with an environment, and often wraps an `agent`. The internal structure of an actor is entirely up to the user\n- An `agent` is what we typically think of as an agent in RL - something that perceives its environment and acts upon it. We do not attempt to solve the agent foundation problem in this documentation.\n- An `agent` is simultaneously the part of the environment that's taking an action - multiagent environments may have several agents, so we need to assign an actor to each agent.\n\n## Known rough edges\n\n- When running the web UI, you can open the tab only once per launched process. So if you open the UI, you can run however many trials you want, as long as you don't close it. If you do close it, you should kill the process and start a new one.\n\n## Local installation\n\n- Requires Python 3.10\n- Install requirements in a virtual env with something similar to the following\n\n ```console\n $ python -m venv .venv\n $ source .venv/bin/activate\n $ pip install -r requirements.txt\n $ pip install -e .\n ```\n\n- For the examples you'll need to install the additional `examples_requirements.txt`.\n\n### Apple silicon installation\n\nTo run on M1/2/3 macs, you'll need to perform those additional steps\n\n```\npip uninstall grpcio grpcio-tools\nexport GRPC_PYTHON_LDFLAGS=\" -framework CoreFoundation\"\npip install grpcio==1.60.0 grpcio-tools==1.60.0 --no-binary :all:\n```\n\nAdjust the version (here 1.60.0) to whatever you have installed.\n\n## Usage\n\nRun `cogmentlab launch base`.\n\nThen, run whatever scripts or notebooks.\n\nTerminology:\n\n- Model: a relatively raw PyTorch (or other?) model, inheriting from `nn.Module`\n- Agent: a model wrapped in some utility class to interact with np arrays\n- Actor: a cogment service that may involve models and/or actors\n\n## Release process\n\nPeople having maintainers rights of the repository can follow these steps to release a version **MAJOR.MINOR.PATCH**. The versioning scheme follows [Semantic Versioning](http://semver.org/spec/v2.0.0.html).\n\n1. Run `./scripts/create_release_branch.sh MAJOR.MINOR.PATCH`, this will automatically:\n - update the version of the package, in `cogment_lab/version.py`,\n - create a release branch with the changes at `release/vMAJOR.MINOR.PATCH` and push it.\n2. On the release branch:\n - Make sure the changelog, at `CHANGELOG.md`, reflects the changes since the last release,\n - Fix any issue, making sure that the build passes on CI,\n - Commit and push any changes.\n3. Run `./scripts/tag_release.sh MAJOR.MINOR.PATCH`, this will automatically:\n - create the specific version section in the changelog and push it to the release branch,\n - merge the release branch in `main`,\n - create the release tag and,\n - update the `develop` to match the latest release.\n4. The CI will automatically publish the package to PyPI.\n",
"bugtrack_url": null,
"license": null,
"summary": null,
"version": "0.1.5",
"project_urls": {
"Bug Report": "https://github.com/cogment/cogment-lab/issues",
"Documentation": "https://cogment.ai/lab",
"Homepage": "https://cogment.ai/lab",
"Repository": "https://github.com/cogment/cogment-lab"
},
"split_keywords": [
"ai",
" hill",
" rl",
" reinforcement learning",
" human-in-the-loop"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "62f248b670e4c88f10a045b75b5f5ccfbfbd92494eb71eb712f5c57579280739",
"md5": "469bc90bd1a33b18ff8a17198df1d265",
"sha256": "09e5628aeeb5604d0ddfe937680f945739c0e25978252241935eef36c5f75cc2"
},
"downloads": -1,
"filename": "cogment_lab-0.1.5-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "469bc90bd1a33b18ff8a17198df1d265",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 150538,
"upload_time": "2024-03-25T22:12:46",
"upload_time_iso_8601": "2024-03-25T22:12:46.285892Z",
"url": "https://files.pythonhosted.org/packages/62/f2/48b670e4c88f10a045b75b5f5ccfbfbd92494eb71eb712f5c57579280739/cogment_lab-0.1.5-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "132ea23e9dbf52b1a8bb08deda7181304c5ed948ba025fbbb88b5ea4bd28d9d0",
"md5": "8cb257904d64c2b045413e0ee440b13c",
"sha256": "2f67343899ce776eaa6f87a8c216863575865d91dbe2e51ad3c9e4176033bea6"
},
"downloads": -1,
"filename": "cogment_lab-0.1.5.tar.gz",
"has_sig": false,
"md5_digest": "8cb257904d64c2b045413e0ee440b13c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 118412,
"upload_time": "2024-03-25T22:12:48",
"upload_time_iso_8601": "2024-03-25T22:12:48.090443Z",
"url": "https://files.pythonhosted.org/packages/13/2e/a23e9dbf52b1a8bb08deda7181304c5ed948ba025fbbb88b5ea4bd28d9d0/cogment_lab-0.1.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-25 22:12:48",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cogment",
"github_project": "cogment-lab",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "cogment_lab"
}
JSON
