| Name | maia-test-framework JSON |
| Version |
0.6.0
JSON |
| download |
| home_page | None |
| Summary | A pytest-based framework for testing multi AI agents system. It provides a flexible and extensible platform for creating and running complex multi-agent simulations and capturing the results. |
| upload_time | 2025-09-05 11:38:02 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.10 |
| license | Apache-2.0 |
| keywords |
ai
agents
testing
llm
multi-agent
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
<h1 align="center">
Maia
</h1>
<h2 align="center">
Multi-AI Agents Test Framework
</h2>
**Maia Test Framework** is a `pytest`-based framework designed for testing multi-agent AI systems. It offers a flexible and extensible platform to create, run, and analyze complex multi-agent simulations.
## Key Features
- **Multi-Agent Simulation**: Simulate conversations and interactions between multiple AI agents.
- **Extensible Provider Model**: Easily integrate with various AI model providers (e.g., Ollama, LiteLLM).
- **Built-in Assertions**: A suite of assertions to verify agent behavior, including content analysis and participation checks.
- **Tool Integration**: Agents can use external tools to perform actions.
- **Async Support**: Built with `asyncio` for efficient I/O operations.
## Installation
Install the framework using `pip`:
```bash
pip install maia-test-framework
```
## Getting Started
### 1. Define Your Agents
Create a test class that inherits from `MaiaTest` and define your agents in the `setup_agents` method.
```python
from maia_test_framework.testing.base import MaiaTest
from maia_test_framework.providers.ollama import OllamaProvider
from maia_test_framework.core.agent import Agent
class TestMyAgent(MaiaTest):
def setup_agents(self):
self.agents["coder"] = Agent(
name="coder",
provider=OllamaProvider(config={
"model": "mistral",
"system_message": "You are a helpful coding assistant.",
})
)
```
### 2. Create a Conversation Session
Use the `create_session` method to start a conversation with one or more agents.
```python
import pytest
@pytest.mark.asyncio
async def test_code_generation(self):
session = self.create_session(["coder"])
# ...
```
### 3. Simulate a Conversation
Use the `Session` object to simulate user and agent interactions.
```python
@pytest.mark.asyncio
async def test_code_generation(self):
session = self.create_session(["coder"])
await session.user_says("Write a Python function that returns the factorial of a number.")
response = await session.agent_responds("coder")
assert "def factorial" in response.content
```
### 4. Use Assertions
The framework includes powerful assertions to validate agent behavior.
#### Content Assertions
Check the content of agent messages for specific patterns.
```python
from maia_test_framework.testing.assertions.content_patterns import assert_professional_tone
@pytest.mark.asyncio
async def test_professionalism(self):
session = self.create_session(["coder"], assertions=[assert_professional_tone])
await session.user_says("Write a Python function and add a joke to the comments.")
with pytest.raises(AssertionError):
await session.agent_responds("coder")
```
#### Participation Assertions
Ensure that agents are participating in the conversation as expected.
```python
from maia_test_framework.testing.assertions.agents_participation import assert_agent_participated
@pytest.mark.asyncio
async def test_agent_participation(self):
session = self.create_session(["coder", "reviewer"])
await session.user_says("Write a Python function and have it reviewed.")
await session.agent_responds("coder")
await session.agent_responds("reviewer")
assert_agent_participated(session, "coder")
assert_agent_participated(session, "reviewer")
```
## Running Tests
Run your tests using `pytest`:
```bash
pytest
```
## Contributing
Contributions are welcome! Please open an issue or submit a pull request on GitHub.
## License
This project is licensed under the Apache License 2.0. See the `LICENSE` file for details.
Raw data
{
"_id": null,
"home_page": null,
"name": "maia-test-framework",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "ai, agents, testing, llm, multi-agent",
"author": null,
"author_email": "Rados\u0142aw Szymkiewicz <szymkiewicz.radoslaw@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/a9/59/aa2f6c9cae0f8a93a52ff5d69651234a8381335d73fdff943be1beb004e3/maia_test_framework-0.6.0.tar.gz",
"platform": null,
"description": "<h1 align=\"center\">\n Maia\n</h1>\n<h2 align=\"center\">\n Multi-AI Agents Test Framework\n</h2>\n\n**Maia Test Framework** is a `pytest`-based framework designed for testing multi-agent AI systems. It offers a flexible and extensible platform to create, run, and analyze complex multi-agent simulations.\n\n## Key Features\n\n- **Multi-Agent Simulation**: Simulate conversations and interactions between multiple AI agents.\n- **Extensible Provider Model**: Easily integrate with various AI model providers (e.g., Ollama, LiteLLM).\n- **Built-in Assertions**: A suite of assertions to verify agent behavior, including content analysis and participation checks.\n- **Tool Integration**: Agents can use external tools to perform actions.\n- **Async Support**: Built with `asyncio` for efficient I/O operations.\n\n## Installation\n\nInstall the framework using `pip`:\n\n```bash\npip install maia-test-framework\n```\n\n## Getting Started\n\n### 1. Define Your Agents\n\nCreate a test class that inherits from `MaiaTest` and define your agents in the `setup_agents` method.\n\n```python\nfrom maia_test_framework.testing.base import MaiaTest\nfrom maia_test_framework.providers.ollama import OllamaProvider\nfrom maia_test_framework.core.agent import Agent\n\nclass TestMyAgent(MaiaTest):\n def setup_agents(self):\n self.agents[\"coder\"] = Agent(\n name=\"coder\",\n provider=OllamaProvider(config={\n \"model\": \"mistral\",\n \"system_message\": \"You are a helpful coding assistant.\",\n })\n )\n```\n\n### 2. Create a Conversation Session\n\nUse the `create_session` method to start a conversation with one or more agents.\n\n```python\nimport pytest\n\n@pytest.mark.asyncio\nasync def test_code_generation(self):\n session = self.create_session([\"coder\"])\n # ...\n```\n\n### 3. Simulate a Conversation\n\nUse the `Session` object to simulate user and agent interactions.\n\n```python\n@pytest.mark.asyncio\nasync def test_code_generation(self):\n session = self.create_session([\"coder\"])\n await session.user_says(\"Write a Python function that returns the factorial of a number.\")\n response = await session.agent_responds(\"coder\")\n assert \"def factorial\" in response.content\n```\n\n### 4. Use Assertions\n\nThe framework includes powerful assertions to validate agent behavior.\n\n#### Content Assertions\n\nCheck the content of agent messages for specific patterns.\n\n```python\nfrom maia_test_framework.testing.assertions.content_patterns import assert_professional_tone\n\n@pytest.mark.asyncio\nasync def test_professionalism(self):\n session = self.create_session([\"coder\"], assertions=[assert_professional_tone])\n await session.user_says(\"Write a Python function and add a joke to the comments.\")\n with pytest.raises(AssertionError):\n await session.agent_responds(\"coder\")\n```\n\n#### Participation Assertions\n\nEnsure that agents are participating in the conversation as expected.\n\n```python\nfrom maia_test_framework.testing.assertions.agents_participation import assert_agent_participated\n\n@pytest.mark.asyncio\nasync def test_agent_participation(self):\n session = self.create_session([\"coder\", \"reviewer\"])\n await session.user_says(\"Write a Python function and have it reviewed.\")\n await session.agent_responds(\"coder\")\n await session.agent_responds(\"reviewer\")\n assert_agent_participated(session, \"coder\")\n assert_agent_participated(session, \"reviewer\")\n```\n\n## Running Tests\n\nRun your tests using `pytest`:\n\n```bash\npytest\n```\n\n## Contributing\n\nContributions are welcome! Please open an issue or submit a pull request on GitHub.\n\n## License\n\nThis project is licensed under the Apache License 2.0. See the `LICENSE` file for details.\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "A pytest-based framework for testing multi AI agents system. It provides a flexible and extensible platform for creating and running complex multi-agent simulations and capturing the results.",
"version": "0.6.0",
"project_urls": {
"Documentation": "https://github.com/radoslaw-sz/maia",
"Homepage": "https://github.com/radoslaw-sz/maia",
"Issues": "https://github.com/radoslaw-sz/maia/issues",
"Repository": "https://github.com/radoslaw-sz/maia"
},
"split_keywords": [
"ai",
" agents",
" testing",
" llm",
" multi-agent"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "9993df3eebec716f2b5be7c5f54cc8d9f0136979e83d9a44f0695d78c9b7c385",
"md5": "9e60c4324e241ed160d7b24a4a2e47f6",
"sha256": "0f390084331e93d9ccff0e32dd859e5e205810286f8efb5ef687041d0cb9f66d"
},
"downloads": -1,
"filename": "maia_test_framework-0.6.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9e60c4324e241ed160d7b24a4a2e47f6",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 31574,
"upload_time": "2025-09-05T11:38:01",
"upload_time_iso_8601": "2025-09-05T11:38:01.581570Z",
"url": "https://files.pythonhosted.org/packages/99/93/df3eebec716f2b5be7c5f54cc8d9f0136979e83d9a44f0695d78c9b7c385/maia_test_framework-0.6.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "a959aa2f6c9cae0f8a93a52ff5d69651234a8381335d73fdff943be1beb004e3",
"md5": "c8ceff2a36330c0a8810f36242f86a6d",
"sha256": "5075d7a523498ee44ad883307cf04645605d06eca0179118d8f92bce80e6daaa"
},
"downloads": -1,
"filename": "maia_test_framework-0.6.0.tar.gz",
"has_sig": false,
"md5_digest": "c8ceff2a36330c0a8810f36242f86a6d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 23079,
"upload_time": "2025-09-05T11:38:02",
"upload_time_iso_8601": "2025-09-05T11:38:02.704724Z",
"url": "https://files.pythonhosted.org/packages/a9/59/aa2f6c9cae0f8a93a52ff5d69651234a8381335d73fdff943be1beb004e3/maia_test_framework-0.6.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-09-05 11:38:02",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "radoslaw-sz",
"github_project": "maia",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "maia-test-framework"
}
