| Name | interlab JSON |
| Version |
0.4.1
JSON |
| download |
| home_page | |
| Summary | A research framework for artificial agent interactions |
| upload_time | 2024-01-21 08:04:56 |
| maintainer | |
| docs_url | None |
| author | Tomas Gavenciak |
| requires_python | >=3.10,<4.0 |
| license | |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
<p align="center"><img src="docs/assets/logo3-cut.webp" width="350" height="350"/></p>
# InterLab - toolkit for multi-agent interactions
<p align="center">
<a href="https://github.com/acsresearch/interlab/releases/latest/"><img alt="GitHub Release" src="https://img.shields.io/github/v/release/acsresearch/interlab" /></a>
<a href="https://pypi.org/project/interlab/"><img alt="PyPI Release" src="https://img.shields.io/pypi/v/interlab"></a>
<a href="https://acsresearch.org/interlab/stable/"><img alt="Documentation" src="https://img.shields.io/badge/Documentation-blue" /></a>
<a href="https://github.com/acsresearch/interlab/actions/workflows/ci.yaml"><img alt="Build status" src="https://img.shields.io/github/actions/workflow/status/acsresearch/interlab/ci.yaml" /></a>
<a href="https://colab.research.google.com/github/acsresearch/interlab/blob/main/notebooks/car_negotiation_colab.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
<a href="https://github.com/acsresearch/interlab/blob/main/LICENSE"><img alt="GitHub License" src="https://img.shields.io/github/license/acsresearch/interlab"></a>
</p>
Welcome to InterLab, a research-focused toolkit created to facilitate study and experimentation in the realm of agent interactions, particularly those based on Language Learning Models (LLMs). Our primary objective is to simplify the process of crafting, deploying, and inspecting complex and structured queries within the context of agent interactions, while also providing robust support for interaction logging, UI and visualization. While we maintain a broad scope and plan to include game theoretic agents and a variety of scenarios, our main emphasis lies in the sphere of LLM interactions.
InterLab is developed at the [Alignment of Complex Systems Research Group](https://acsresearch.org/) and distributed under MIT license (see `LICENSE`).
**Current status: InterLab is in open beta.** InterLab is also active development, we use it internally for our experiments, and we want to make it available to the wider research community. Although we aim to limit the breaking changes between major versions, the API may change substantially as we refine the design and gather information about usage.
If you use InterLab, want to share ideas, feedback or have any questions, please email us at `gavento@acsresearch.org` - we'd be happy to hear from you.
## Overview
InterLab is composed of three main packages:
* `interlab` contains the core functionality and common abstractions of actors, environments, memory, and language model interactions, along with a few helpful scaffolds. The main modules there are:
- `actor` provides framework for actor interactions, including a generic LLM single-shot agent and a web console for playing as an actor, along with actor memory systems. The agents may be queried for structured (typed JSON-like) or unstuctured actions.
- `environment` providing abstraction over partially observable environments.
- `queries` contains powerful helpers for querying the models, in particular querying for any dataclass or Pydantic model with robust response parsing and retries, summarization tools etc.
* `treetrace` offers comprehensive structured logging of nested `TracingNodes`s, storage for traing nodes, and custom visualization of content such as Images, generic HTML, and tracking f-string-like field substitutions in larger text.
* `interlab_zoo` serves as a repository for specific and opinionated implementations of actors, scenarios, actor memory systems, tracing post-processing, and other tools that enhance the InterLab project. Its mission is to gather code that is both widely applicable and useful, while maintaining a compact and focused core package.
Beyond that, some less-tested and experimental code can be found in `experimental` submodules accross the core package.
### Structured interaction log browser
In-notebook or independent browser for the structured logs, with live updates, support for JSON-like structured data and inline visualizations. Captures both high-level interaction structure and the low-level API calls for easy inspection and debugging.
Example screenshots (click to zoom in):
<p align="center"><img src="docs/assets/imgs/context-browser-2.png" alt= "Tracing browser screenshot" width="100%" ></p>
<p align="center"><img src="docs/assets/imgs/context-browser-1.png" alt= "Tracing browser screenshot" width="90%" ></p>
### Example notebooks
You can find Jupyter notebooks with a few worked-out examples [here](https://github.com/acsresearch/interlab/tree/main/notebooks). The notebook [car_negotiation.ipynb](https://github.com/acsresearch/interlab/blob/main/notebooks/car_negotiation.ipynb) is a good starting point for a simple bargaining simulation.
## Installation
You can install the package `interlab` from PyPI using `pip` or any other package manager.
```commandline
pip install interlab
```
### Installation with development tools
This repository utilizes [**Poetry**](https://python-poetry.org/) package management, which is recommended for dependency installation and is required for InterLab development. Poetry automatically generates and manages a virtual environment for you, and also installs `interlab` module itself. If you have poetry installed, running the following command inside this repository will install InterLab:
```commandline
poetry install
```
**pip and `requirements.txt`.** Alternatively, `pip` can be used to install dependencies with `pip install -r requirements.txt` (core requirements) or `pip install -r requirements-full.txt` (including development tools, Jupyter Lab, etc.; equivalent to `poetry install`). To use InterLab in Google Colab, we use the requirements in `requirements-colab.txt` to get aroung some Colab versioning conflicts.
## Development and experiments
Jupyter Lab provides the simplest way to interact with the code and design experiments:
```commandline
poetry run jupyter lab
# Or without poetry, in the project root folder:
jupyter lab
```
After running the command, open the provided link in your browser. `notebooks/car_negotiation.ipynb` is a recommended starting point.
### Google Colab
<a target="_blank" href="https://colab.research.google.com/github/acsresearch/interlab/blob/main/notebooks/car_negotiation_colab.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
Google Colab often offers a lightweight alternative to setting up InterLab locally on your computer. Interlab comes with built-in colab compatibility and we have prepared a [Example InterLab Colab experiment](https://colab.research.google.com/github/acsresearch/interlab/blob/main/notebooks/car_negotiation_colab.ipynb) with common setup and a simple example experiment with two LLMs interacting on behalf of their users.
### Note: API Keys
In order to use LLM provider serveics and APIs, you need to generate and provide the corresponding API keys. You can provide the keys as environment variables, via `.env` file, or interactively on every run (e.g. in the colab). Storing keys in the notebook is possible but not recommended (as they easily leak into git or while sharing the code).
API keys can be stored in a `.env` file located in the notebook directory or your home directory. (This file is ignored by `git` by default, providing an additional security measure.) The file is a simple text file with key=value pairs, for example:
```text
OPENAI_API_KEY=sk-...
OPENAI_API_ORG=org-...
ANTHROPIC_API_KEY=sk-ant-...
...
```
You can then import these variables from the `.env` file into a Jupyter notebook using the [dotenv](https://github.com/theskumar/python-dotenv) package:
```python
import dotenv
dotenv.load_dotenv()
```
## Contributions and development
To contribute, please submit a pull request with your proposed changes based on the latest `main` branch.
By submitting a pull request for this project, you agree to license your contribution under the MIT license to this project as written in the `LICENSE` file.
## Citing InterLab
If you use InterLab in your researh, please cite it in your work using the "Cite this repository" github gadget or with:
Tomáš Gavenčiak, Ada Böhm: *InterLab [Computer software].* 2023. https://github.com/acsresearch/interlab
Raw data
{
"_id": null,
"home_page": "",
"name": "interlab",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10,<4.0",
"maintainer_email": "",
"keywords": "",
"author": "Tomas Gavenciak",
"author_email": "gavento@acsresearch.org",
"download_url": "https://files.pythonhosted.org/packages/de/08/13717b70227e81826525225ec8411714ab605184431f27499da4e9154c21/interlab-0.4.1.tar.gz",
"platform": null,
"description": "<p align=\"center\"><img src=\"docs/assets/logo3-cut.webp\" width=\"350\" height=\"350\"/></p>\n\n# InterLab - toolkit for multi-agent interactions\n\n<p align=\"center\">\n<a href=\"https://github.com/acsresearch/interlab/releases/latest/\"><img alt=\"GitHub Release\" src=\"https://img.shields.io/github/v/release/acsresearch/interlab\" /></a>\n<a href=\"https://pypi.org/project/interlab/\"><img alt=\"PyPI Release\" src=\"https://img.shields.io/pypi/v/interlab\"></a>\n<a href=\"https://acsresearch.org/interlab/stable/\"><img alt=\"Documentation\" src=\"https://img.shields.io/badge/Documentation-blue\" /></a>\n<a href=\"https://github.com/acsresearch/interlab/actions/workflows/ci.yaml\"><img alt=\"Build status\" src=\"https://img.shields.io/github/actions/workflow/status/acsresearch/interlab/ci.yaml\" /></a>\n<a href=\"https://colab.research.google.com/github/acsresearch/interlab/blob/main/notebooks/car_negotiation_colab.ipynb\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n<a href=\"https://github.com/acsresearch/interlab/blob/main/LICENSE\"><img alt=\"GitHub License\" src=\"https://img.shields.io/github/license/acsresearch/interlab\"></a>\n</p>\n\nWelcome to InterLab, a research-focused toolkit created to facilitate study and experimentation in the realm of agent interactions, particularly those based on Language Learning Models (LLMs). Our primary objective is to simplify the process of crafting, deploying, and inspecting complex and structured queries within the context of agent interactions, while also providing robust support for interaction logging, UI and visualization. While we maintain a broad scope and plan to include game theoretic agents and a variety of scenarios, our main emphasis lies in the sphere of LLM interactions.\n\nInterLab is developed at the [Alignment of Complex Systems Research Group](https://acsresearch.org/) and distributed under MIT license (see `LICENSE`).\n\n**Current status: InterLab is in open beta.** InterLab is also active development, we use it internally for our experiments, and we want to make it available to the wider research community. Although we aim to limit the breaking changes between major versions, the API may change substantially as we refine the design and gather information about usage.\n\nIf you use InterLab, want to share ideas, feedback or have any questions, please email us at `gavento@acsresearch.org` - we'd be happy to hear from you.\n\n## Overview\n\nInterLab is composed of three main packages:\n\n* `interlab` contains the core functionality and common abstractions of actors, environments, memory, and language model interactions, along with a few helpful scaffolds. The main modules there are:\n - `actor` provides framework for actor interactions, including a generic LLM single-shot agent and a web console for playing as an actor, along with actor memory systems. The agents may be queried for structured (typed JSON-like) or unstuctured actions.\n - `environment` providing abstraction over partially observable environments.\n - `queries` contains powerful helpers for querying the models, in particular querying for any dataclass or Pydantic model with robust response parsing and retries, summarization tools etc.\n* `treetrace` offers comprehensive structured logging of nested `TracingNodes`s, storage for traing nodes, and custom visualization of content such as Images, generic HTML, and tracking f-string-like field substitutions in larger text.\n* `interlab_zoo` serves as a repository for specific and opinionated implementations of actors, scenarios, actor memory systems, tracing post-processing, and other tools that enhance the InterLab project. Its mission is to gather code that is both widely applicable and useful, while maintaining a compact and focused core package.\n\nBeyond that, some less-tested and experimental code can be found in `experimental` submodules accross the core package.\n\n### Structured interaction log browser\n\nIn-notebook or independent browser for the structured logs, with live updates, support for JSON-like structured data and inline visualizations. Captures both high-level interaction structure and the low-level API calls for easy inspection and debugging.\nExample screenshots (click to zoom in):\n\n<p align=\"center\"><img src=\"docs/assets/imgs/context-browser-2.png\" alt= \"Tracing browser screenshot\" width=\"100%\" ></p>\n\n<p align=\"center\"><img src=\"docs/assets/imgs/context-browser-1.png\" alt= \"Tracing browser screenshot\" width=\"90%\" ></p>\n\n### Example notebooks\n\nYou can find Jupyter notebooks with a few worked-out examples [here](https://github.com/acsresearch/interlab/tree/main/notebooks). The notebook [car_negotiation.ipynb](https://github.com/acsresearch/interlab/blob/main/notebooks/car_negotiation.ipynb) is a good starting point for a simple bargaining simulation.\n\n## Installation\n\nYou can install the package `interlab` from PyPI using `pip` or any other package manager.\n\n```commandline\npip install interlab\n```\n\n### Installation with development tools\n\nThis repository utilizes [**Poetry**](https://python-poetry.org/) package management, which is recommended for dependency installation and is required for InterLab development. Poetry automatically generates and manages a virtual environment for you, and also installs `interlab` module itself. If you have poetry installed, running the following command inside this repository will install InterLab:\n\n```commandline\npoetry install\n```\n\n**pip and `requirements.txt`.** Alternatively, `pip` can be used to install dependencies with `pip install -r requirements.txt` (core requirements) or `pip install -r requirements-full.txt` (including development tools, Jupyter Lab, etc.; equivalent to `poetry install`). To use InterLab in Google Colab, we use the requirements in `requirements-colab.txt` to get aroung some Colab versioning conflicts.\n\n## Development and experiments\n\nJupyter Lab provides the simplest way to interact with the code and design experiments:\n\n```commandline\npoetry run jupyter lab\n# Or without poetry, in the project root folder:\njupyter lab\n```\n\nAfter running the command, open the provided link in your browser. `notebooks/car_negotiation.ipynb` is a recommended starting point.\n\n### Google Colab\n\n<a target=\"_blank\" href=\"https://colab.research.google.com/github/acsresearch/interlab/blob/main/notebooks/car_negotiation_colab.ipynb\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n\nGoogle Colab often offers a lightweight alternative to setting up InterLab locally on your computer. Interlab comes with built-in colab compatibility and we have prepared a [Example InterLab Colab experiment](https://colab.research.google.com/github/acsresearch/interlab/blob/main/notebooks/car_negotiation_colab.ipynb) with common setup and a simple example experiment with two LLMs interacting on behalf of their users.\n\n### Note: API Keys\n\nIn order to use LLM provider serveics and APIs, you need to generate and provide the corresponding API keys. You can provide the keys as environment variables, via `.env` file, or interactively on every run (e.g. in the colab). Storing keys in the notebook is possible but not recommended (as they easily leak into git or while sharing the code).\n\nAPI keys can be stored in a `.env` file located in the notebook directory or your home directory. (This file is ignored by `git` by default, providing an additional security measure.) The file is a simple text file with key=value pairs, for example:\n\n```text\nOPENAI_API_KEY=sk-...\nOPENAI_API_ORG=org-...\nANTHROPIC_API_KEY=sk-ant-...\n...\n```\n\nYou can then import these variables from the `.env` file into a Jupyter notebook using the [dotenv](https://github.com/theskumar/python-dotenv) package:\n\n```python\nimport dotenv\ndotenv.load_dotenv()\n```\n\n## Contributions and development\n\nTo contribute, please submit a pull request with your proposed changes based on the latest `main` branch.\n\nBy submitting a pull request for this project, you agree to license your contribution under the MIT license to this project as written in the `LICENSE` file.\n\n## Citing InterLab\n\nIf you use InterLab in your researh, please cite it in your work using the \"Cite this repository\" github gadget or with:\n\nTom\u00e1\u0161 Gaven\u010diak, Ada B\u00f6hm: *InterLab [Computer software].* 2023. https://github.com/acsresearch/interlab\n",
"bugtrack_url": null,
"license": "",
"summary": "A research framework for artificial agent interactions",
"version": "0.4.1",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "dff972d25d5dfdd57990f3f60d51cdc8d023a3918d8c0d8f5426aaf7c269dd0b",
"md5": "bc6b4056be0ba9c342115df3ccb36a24",
"sha256": "96dadd090c09238eb76ef651025755a7f8142d97cf27585715d8f37130f27eb4"
},
"downloads": -1,
"filename": "interlab-0.4.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "bc6b4056be0ba9c342115df3ccb36a24",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10,<4.0",
"size": 746606,
"upload_time": "2024-01-21T08:04:52",
"upload_time_iso_8601": "2024-01-21T08:04:52.169456Z",
"url": "https://files.pythonhosted.org/packages/df/f9/72d25d5dfdd57990f3f60d51cdc8d023a3918d8c0d8f5426aaf7c269dd0b/interlab-0.4.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "de0813717b70227e81826525225ec8411714ab605184431f27499da4e9154c21",
"md5": "02a37636426ac0bdac42867cb3c6a158",
"sha256": "9f69c60976cd1e53a515d2605906e7a333aa41cd1c177cf4d63506451532b10a"
},
"downloads": -1,
"filename": "interlab-0.4.1.tar.gz",
"has_sig": false,
"md5_digest": "02a37636426ac0bdac42867cb3c6a158",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10,<4.0",
"size": 731207,
"upload_time": "2024-01-21T08:04:56",
"upload_time_iso_8601": "2024-01-21T08:04:56.259548Z",
"url": "https://files.pythonhosted.org/packages/de/08/13717b70227e81826525225ec8411714ab605184431f27499da4e9154c21/interlab-0.4.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-21 08:04:56",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "interlab"
}
