agentops

Name	agentops JSON
Version	0.3.26 JSON
	download
home_page	None
Summary	Observability and DevTool Platform for AI Agents
upload_time	2025-01-24 23:44:08
maintainer	None
docs_url	None
author	None
requires_python	<3.14,>=3.9
license	None
keywords
VCS
bugtrack_url
requirements	No requirements were recorded.
Travis-CI	No Travis.
coveralls test coverage	No coveralls.

            <div align="center">
  <a href="https://agentops.ai?ref=gh">
    <img src="docs/images/external/logo/github-banner.png" alt="Logo">
  </a>
</div>

<div align="center">
  <em>Observability and DevTool platform for AI Agents</em>
</div>

<br />

<div align="center">
  <a href="https://pepy.tech/project/agentops">
    <img src="https://static.pepy.tech/badge/agentops/month" alt="Downloads">
  </a>
  <a href="https://github.com/agentops-ai/agentops/issues">
  <img src="https://img.shields.io/github/commit-activity/m/agentops-ai/agentops" alt="git commit activity">
  </a>
  <img src="https://img.shields.io/pypi/v/agentops?&color=3670A0" alt="PyPI - Version">
  <a href="https://opensource.org/licenses/MIT">
    <img src="https://img.shields.io/badge/License-MIT-yellow.svg?&color=3670A0" alt="License: MIT">
  </a>
</div>

<p align="center">
  <a href="https://twitter.com/agentopsai/">
    <img src="https://img.shields.io/twitter/follow/agentopsai?style=social" alt="Twitter" style="height: 20px;">
  </a>
  <a href="https://discord.gg/FagdcwwXRR">
    <img src="https://img.shields.io/badge/discord-7289da.svg?style=flat-square&logo=discord" alt="Discord" style="height: 20px;">
  </a>
  <a href="https://app.agentops.ai/?ref=gh">
    <img src="https://img.shields.io/badge/Dashboard-blue.svg?style=flat-square" alt="Dashboard" style="height: 20px;">
  </a>
  <a href="https://docs.agentops.ai/introduction">
    <img src="https://img.shields.io/badge/Documentation-orange.svg?style=flat-square" alt="Documentation" style="height: 20px;">
  </a>
  <a href="https://entelligence.ai/AgentOps-AI&agentops">
    <img src="https://img.shields.io/badge/Chat%20with%20Docs-green.svg?style=flat-square" alt="Chat with Docs" style="height: 20px;">
  </a>
</p>



<div style="justify-content: center">
  <img src="docs/images/external/app_screenshots/dashboard-banner.png" alt="Dashboard Banner">
</div>

<br/>


AgentOps helps developers build, evaluate, and monitor AI agents. From prototype to production.

|                                       |                                                               |
| ------------------------------------- | ------------------------------------------------------------- |
| 📊 **Replay Analytics and Debugging** | Step-by-step agent execution graphs                           |
| 💸 **LLM Cost Management**            | Track spend with LLM foundation model providers               |
| 🧪 **Agent Benchmarking**             | Test your agents against 1,000+ evals                         |
| 🔐 **Compliance and Security**        | Detect common prompt injection and data exfiltration exploits |
| 🤝 **Framework Integrations**         | Native Integrations with CrewAI, AG2(AutoGen), Camel AI, & LangChain         |

## Quick Start ⌨️

```bash
pip install agentops
```


#### Session replays in 2 lines of code

Initialize the AgentOps client and automatically get analytics on all your LLM calls.

[Get an API key](https://app.agentops.ai/settings/projects)

```python
import agentops

# Beginning of your program (i.e. main.py, __init__.py)
agentops.init( < INSERT YOUR API KEY HERE >)

...

# End of program
agentops.end_session('Success')
```

All your sessions can be viewed on the [AgentOps dashboard](https://app.agentops.ai?ref=gh)
<br/>

<details>
  <summary>Agent Debugging</summary>
  <a href="https://app.agentops.ai?ref=gh">
    <img src="docs/images/external/app_screenshots/session-drilldown-metadata.png" style="width: 90%;" alt="Agent Metadata"/>
  </a>
  <a href="https://app.agentops.ai?ref=gh">
    <img src="docs/images/external/app_screenshots/chat-viewer.png" style="width: 90%;" alt="Chat Viewer"/>
  </a>
  <a href="https://app.agentops.ai?ref=gh">
    <img src="docs/images/external/app_screenshots/session-drilldown-graphs.png" style="width: 90%;" alt="Event Graphs"/>
  </a>
</details>

<details>
  <summary>Session Replays</summary>
  <a href="https://app.agentops.ai?ref=gh">
    <img src="docs/images/external/app_screenshots/session-replay.png" style="width: 90%;" alt="Session Replays"/>
  </a>
</details>

<details open>
  <summary>Summary Analytics</summary>
  <a href="https://app.agentops.ai?ref=gh">
   <img src="docs/images/external/app_screenshots/overview.png" style="width: 90%;" alt="Summary Analytics"/>
  </a>
  <a href="https://app.agentops.ai?ref=gh">
   <img src="docs/images/external/app_screenshots/overview-charts.png" style="width: 90%;" alt="Summary Analytics Charts"/>
  </a>
</details>


### First class Developer Experience
Add powerful observability to your agents, tools, and functions with as little code as possible: one line at a time.
<br/>
Refer to our [documentation](http://docs.agentops.ai)

```python
# Automatically associate all Events with the agent that originated them
from agentops import track_agent

@track_agent(name='SomeCustomName')
class MyAgent:
  ...
```

```python
# Automatically create ToolEvents for tools that agents will use
from agentops import record_tool

@record_tool('SampleToolName')
def sample_tool(...):
  ...
```

```python
# Automatically create ActionEvents for other functions.
from agentops import record_action

@agentops.record_action('sample function being record')
def sample_function(...):
  ...
```

```python
# Manually record any other Events
from agentops import record, ActionEvent

record(ActionEvent("received_user_input"))
```

## Integrations 🦾

### CrewAI 🛶

Build Crew agents with observability with only 2 lines of code. Simply set an `AGENTOPS_API_KEY` in your environment, and your crews will get automatic monitoring on the AgentOps dashboard.

```bash
pip install 'crewai[agentops]'
```

- [AgentOps integration example](https://docs.agentops.ai/v1/integrations/crewai)
- [Official CrewAI documentation](https://docs.crewai.com/how-to/AgentOps-Observability)

### AG2 🤖
With only two lines of code, add full observability and monitoring to AG2 (formerly AutoGen) agents. Set an `AGENTOPS_API_KEY` in your environment and call `agentops.init()`

- [AG2 Observability Example](https://docs.ag2.ai/notebooks/agentchat_agentops)
- [AG2 - AgentOps Documentation](https://docs.ag2.ai/docs/ecosystem/agentops)

### Camel AI 🐪

Track and analyze CAMEL agents with full observability. Set an `AGENTOPS_API_KEY` in your environment and initialize AgentOps to get started.

- [Camel AI](https://www.camel-ai.org/) - Advanced agent communication framework
- [AgentOps integration example](https://docs.agentops.ai/v1/integrations/camel)
- [Official Camel AI documentation](https://docs.camel-ai.org/cookbooks/agents_tracking.html)

<details>
  <summary>Installation</summary>

```bash
pip install "camel-ai[all]==0.2.11"
pip install agentops
```

```python
import os
import agentops
from camel.agents import ChatAgent
from camel.messages import BaseMessage
from camel.models import ModelFactory
from camel.types import ModelPlatformType, ModelType

# Initialize AgentOps
agentops.init(os.getenv("AGENTOPS_API_KEY"), default_tags=["CAMEL Example"])

# Import toolkits after AgentOps init for tracking
from camel.toolkits import SearchToolkit

# Set up the agent with search tools
sys_msg = BaseMessage.make_assistant_message(
    role_name='Tools calling operator',
    content='You are a helpful assistant'
)

# Configure tools and model
tools = [*SearchToolkit().get_tools()]
model = ModelFactory.create(
    model_platform=ModelPlatformType.OPENAI,
    model_type=ModelType.GPT_4O_MINI,
)

# Create and run the agent
camel_agent = ChatAgent(
    system_message=sys_msg,
    model=model,
    tools=tools,
)

response = camel_agent.step("What is AgentOps?")
print(response)

agentops.end_session("Success")
```

Check out our [Camel integration guide](https://docs.agentops.ai/v1/integrations/camel) for more examples including multi-agent scenarios.
</details>

### Langchain 🦜🔗

AgentOps works seamlessly with applications built using Langchain. To use the handler, install Langchain as an optional dependency:

<details>
  <summary>Installation</summary>
  
```shell
pip install agentops[langchain]
```

To use the handler, import and set

```python
import os
from langchain.chat_models import ChatOpenAI
from langchain.agents import initialize_agent, AgentType
from agentops.partners.langchain_callback_handler import LangchainCallbackHandler

AGENTOPS_API_KEY = os.environ['AGENTOPS_API_KEY']
handler = LangchainCallbackHandler(api_key=AGENTOPS_API_KEY, tags=['Langchain Example'])

llm = ChatOpenAI(openai_api_key=OPENAI_API_KEY,
                 callbacks=[handler],
                 model='gpt-3.5-turbo')

agent = initialize_agent(tools,
                         llm,
                         agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,
                         verbose=True,
                         callbacks=[handler], # You must pass in a callback handler to record your agent
                         handle_parsing_errors=True)
```

Check out the [Langchain Examples Notebook](./examples/langchain_examples.ipynb) for more details including Async handlers.

</details>

### Cohere ⌨️

First class support for Cohere(>=5.4.0). This is a living integration, should you need any added functionality please message us on Discord!

- [AgentOps integration example](https://docs.agentops.ai/v1/integrations/cohere)
- [Official Cohere documentation](https://docs.cohere.com/reference/about)

<details>
  <summary>Installation</summary>
  
```bash
pip install cohere
```

```python python
import cohere
import agentops

# Beginning of program's code (i.e. main.py, __init__.py)
agentops.init(<INSERT YOUR API KEY HERE>)
co = cohere.Client()

chat = co.chat(
    message="Is it pronounced ceaux-hear or co-hehray?"
)

print(chat)

agentops.end_session('Success')
```

```python python
import cohere
import agentops

# Beginning of program's code (i.e. main.py, __init__.py)
agentops.init(<INSERT YOUR API KEY HERE>)

co = cohere.Client()

stream = co.chat_stream(
    message="Write me a haiku about the synergies between Cohere and AgentOps"
)

for event in stream:
    if event.event_type == "text-generation":
        print(event.text, end='')

agentops.end_session('Success')
```
</details>


### Anthropic ﹨

Track agents built with the Anthropic Python SDK (>=0.32.0).

- [AgentOps integration guide](https://docs.agentops.ai/v1/integrations/anthropic)
- [Official Anthropic documentation](https://docs.anthropic.com/en/docs/welcome)

<details>
  <summary>Installation</summary>
  
```bash
pip install anthropic
```

```python python
import anthropic
import agentops

# Beginning of program's code (i.e. main.py, __init__.py)
agentops.init(<INSERT YOUR API KEY HERE>)

client = anthropic.Anthropic(
    # This is the default and can be omitted
    api_key=os.environ.get("ANTHROPIC_API_KEY"),
)

message = client.messages.create(
        max_tokens=1024,
        messages=[
            {
                "role": "user",
                "content": "Tell me a cool fact about AgentOps",
            }
        ],
        model="claude-3-opus-20240229",
    )
print(message.content)

agentops.end_session('Success')
```

Streaming
```python python
import anthropic
import agentops

# Beginning of program's code (i.e. main.py, __init__.py)
agentops.init(<INSERT YOUR API KEY HERE>)

client = anthropic.Anthropic(
    # This is the default and can be omitted
    api_key=os.environ.get("ANTHROPIC_API_KEY"),
)

stream = client.messages.create(
    max_tokens=1024,
    model="claude-3-opus-20240229",
    messages=[
        {
            "role": "user",
            "content": "Tell me something cool about streaming agents",
        }
    ],
    stream=True,
)

response = ""
for event in stream:
    if event.type == "content_block_delta":
        response += event.delta.text
    elif event.type == "message_stop":
        print("\n")
        print(response)
        print("\n")
```

Async

```python python
import asyncio
from anthropic import AsyncAnthropic

client = AsyncAnthropic(
    # This is the default and can be omitted
    api_key=os.environ.get("ANTHROPIC_API_KEY"),
)


async def main() -> None:
    message = await client.messages.create(
        max_tokens=1024,
        messages=[
            {
                "role": "user",
                "content": "Tell me something interesting about async agents",
            }
        ],
        model="claude-3-opus-20240229",
    )
    print(message.content)


await main()
```
</details>

### Mistral 〽️

Track agents built with the Anthropic Python SDK (>=0.32.0).

- [AgentOps integration example](./examples/mistral//mistral_example.ipynb)
- [Official Mistral documentation](https://docs.mistral.ai)

<details>
  <summary>Installation</summary>
  
```bash
pip install mistralai
```

Sync

```python python
from mistralai import Mistral
import agentops

# Beginning of program's code (i.e. main.py, __init__.py)
agentops.init(<INSERT YOUR API KEY HERE>)

client = Mistral(
    # This is the default and can be omitted
    api_key=os.environ.get("MISTRAL_API_KEY"),
)

message = client.chat.complete(
        messages=[
            {
                "role": "user",
                "content": "Tell me a cool fact about AgentOps",
            }
        ],
        model="open-mistral-nemo",
    )
print(message.choices[0].message.content)

agentops.end_session('Success')
```

Streaming

```python python
from mistralai import Mistral
import agentops

# Beginning of program's code (i.e. main.py, __init__.py)
agentops.init(<INSERT YOUR API KEY HERE>)

client = Mistral(
    # This is the default and can be omitted
    api_key=os.environ.get("MISTRAL_API_KEY"),
)

message = client.chat.stream(
        messages=[
            {
                "role": "user",
                "content": "Tell me something cool about streaming agents",
            }
        ],
        model="open-mistral-nemo",
    )

response = ""
for event in message:
    if event.data.choices[0].finish_reason == "stop":
        print("\n")
        print(response)
        print("\n")
    else:
        response += event.text

agentops.end_session('Success')
```

Async

```python python
import asyncio
from mistralai import Mistral

client = Mistral(
    # This is the default and can be omitted
    api_key=os.environ.get("MISTRAL_API_KEY"),
)


async def main() -> None:
    message = await client.chat.complete_async(
        messages=[
            {
                "role": "user",
                "content": "Tell me something interesting about async agents",
            }
        ],
        model="open-mistral-nemo",
    )
    print(message.choices[0].message.content)


await main()
```

Async Streaming

```python python
import asyncio
from mistralai import Mistral

client = Mistral(
    # This is the default and can be omitted
    api_key=os.environ.get("MISTRAL_API_KEY"),
)


async def main() -> None:
    message = await client.chat.stream_async(
        messages=[
            {
                "role": "user",
                "content": "Tell me something interesting about async streaming agents",
            }
        ],
        model="open-mistral-nemo",
    )

    response = ""
    async for event in message:
        if event.data.choices[0].finish_reason == "stop":
            print("\n")
            print(response)
            print("\n")
        else:
            response += event.text


await main()
```
</details>



### CamelAI ﹨

Track agents built with the CamelAI Python SDK (>=0.32.0).

- [CamelAI integration guide](https://docs.camel-ai.org/cookbooks/agents_tracking.html#)
- [Official CamelAI documentation](https://docs.camel-ai.org/index.html)

<details>
  <summary>Installation</summary>
  
```bash
pip install camel-ai[all]
pip install agentops
```

```python python
#Import Dependencies
import agentops
import os
from getpass import getpass
from dotenv import load_dotenv

#Set Keys
load_dotenv()
openai_api_key = os.getenv("OPENAI_API_KEY") or "<your openai key here>"
agentops_api_key = os.getenv("AGENTOPS_API_KEY") or "<your agentops key here>"



```
</details>

[You can find usage examples here!](examples/camelai_examples/README.md).



### LiteLLM 🚅

AgentOps provides support for LiteLLM(>=1.3.1), allowing you to call 100+ LLMs using the same Input/Output Format. 

- [AgentOps integration example](https://docs.agentops.ai/v1/integrations/litellm)
- [Official LiteLLM documentation](https://docs.litellm.ai/docs/providers)

<details>
  <summary>Installation</summary>
  
```bash
pip install litellm
```

```python python
# Do not use LiteLLM like this
# from litellm import completion
# ...
# response = completion(model="claude-3", messages=messages)

# Use LiteLLM like this
import litellm
...
response = litellm.completion(model="claude-3", messages=messages)
# or
response = await litellm.acompletion(model="claude-3", messages=messages)
```
</details>

### LlamaIndex 🦙


AgentOps works seamlessly with applications built using LlamaIndex, a framework for building context-augmented generative AI applications with LLMs.

<details>
  <summary>Installation</summary>
  
```shell
pip install llama-index-instrumentation-agentops
```

To use the handler, import and set

```python
from llama_index.core import set_global_handler

# NOTE: Feel free to set your AgentOps environment variables (e.g., 'AGENTOPS_API_KEY')
# as outlined in the AgentOps documentation, or pass the equivalent keyword arguments
# anticipated by AgentOps' AOClient as **eval_params in set_global_handler.

set_global_handler("agentops")
```

Check out the [LlamaIndex docs](https://docs.llamaindex.ai/en/stable/module_guides/observability/?h=agentops#agentops) for more details.

</details>

### Llama Stack 🦙🥞

AgentOps provides support for Llama Stack Python Client(>=0.0.53), allowing you to monitor your Agentic applications. 

- [AgentOps integration example 1](https://github.com/AgentOps-AI/agentops/pull/530/files/65a5ab4fdcf310326f191d4b870d4f553591e3ea#diff-fdddf65549f3714f8f007ce7dfd1cde720329fe54155d54389dd50fbd81813cb)
- [AgentOps integration example 2](https://github.com/AgentOps-AI/agentops/pull/530/files/65a5ab4fdcf310326f191d4b870d4f553591e3ea#diff-6688ff4fb7ab1ce7b1cc9b8362ca27264a3060c16737fb1d850305787a6e3699)
- [Official Llama Stack Python Client](https://github.com/meta-llama/llama-stack-client-python)

### SwarmZero AI 🐝

Track and analyze SwarmZero agents with full observability. Set an `AGENTOPS_API_KEY` in your environment and initialize AgentOps to get started.

- [SwarmZero](https://swarmzero.ai) - Advanced multi-agent framework
- [AgentOps integration example](https://docs.agentops.ai/v1/integrations/swarmzero)
- [SwarmZero AI integration example](https://docs.swarmzero.ai/examples/ai-agents/build-and-monitor-a-web-search-agent)
- [SwarmZero AI - AgentOps documentation](https://docs.swarmzero.ai/sdk/observability/agentops)
- [Official SwarmZero Python SDK](https://github.com/swarmzero/swarmzero)

<details>
  <summary>Installation</summary>

```bash
pip install swarmzero
pip install agentops
```

```python
from dotenv import load_dotenv
load_dotenv()

import agentops
agentops.init(<INSERT YOUR API KEY HERE>)

from swarmzero import Agent, Swarm
# ...
```
</details>

## Time travel debugging 🔮

<div style="justify-content: center">
  <img src="docs/images/external/app_screenshots/time_travel_banner.png" alt="Time Travel Banner">
</div>

<br />

[Try it out!](https://app.agentops.ai/timetravel)

## Agent Arena 🥊

(coming soon!)

## Evaluations Roadmap 🧭

| Platform                                                                     | Dashboard                                  | Evals                                  |
| ---------------------------------------------------------------------------- | ------------------------------------------ | -------------------------------------- |
| ✅ Python SDK                                                                | ✅ Multi-session and Cross-session metrics | ✅ Custom eval metrics                 |
| 🚧 Evaluation builder API                                                    | ✅ Custom event tag tracking               | 🔜 Agent scorecards                    |
| ✅ [Javascript/Typescript SDK](https://github.com/AgentOps-AI/agentops-node) | ✅ Session replays                         | 🔜 Evaluation playground + leaderboard |

## Debugging Roadmap 🧭

| Performance testing                       | Environments                                                                        | LLM Testing                                 | Reasoning and execution testing                   |
| ----------------------------------------- | ----------------------------------------------------------------------------------- | ------------------------------------------- | ------------------------------------------------- |
| ✅ Event latency analysis                 | 🔜 Non-stationary environment testing                                               | 🔜 LLM non-deterministic function detection | 🚧 Infinite loops and recursive thought detection |
| ✅ Agent workflow execution pricing       | 🔜 Multi-modal environments                                                         | 🚧 Token limit overflow flags               | 🔜 Faulty reasoning detection                     |
| 🚧 Success validators (external)          | 🔜 Execution containers                                                             | 🔜 Context limit overflow flags             | 🔜 Generative code validators                     |
| 🔜 Agent controllers/skill tests          | ✅ Honeypot and prompt injection detection ([PromptArmor](https://promptarmor.com)) | 🔜 API bill tracking                        | 🔜 Error breakpoint analysis                      |
| 🔜 Information context constraint testing | 🔜 Anti-agent roadblocks (i.e. Captchas)                                            | 🔜 CI/CD integration checks                 |                                                   |
| 🔜 Regression testing                     | 🔜 Multi-agent framework visualization                                              |                                             |                                                   |

### Why AgentOps? 🤔

Without the right tools, AI agents are slow, expensive, and unreliable. Our mission is to bring your agent from prototype to production. Here's why AgentOps stands out:

- **Comprehensive Observability**: Track your AI agents' performance, user interactions, and API usage.
- **Real-Time Monitoring**: Get instant insights with session replays, metrics, and live monitoring tools.
- **Cost Control**: Monitor and manage your spend on LLM and API calls.
- **Failure Detection**: Quickly identify and respond to agent failures and multi-agent interaction issues.
- **Tool Usage Statistics**: Understand how your agents utilize external tools with detailed analytics.
- **Session-Wide Metrics**: Gain a holistic view of your agents' sessions with comprehensive statistics.

AgentOps is designed to make agent observability, testing, and monitoring easy.


## Star History

Check out our growth in the community:

<img src="https://api.star-history.com/svg?repos=AgentOps-AI/agentops&type=Date" style="max-width: 500px" width="50%" alt="Logo">

## Popular projects using AgentOps


| Repository | Stars  |
| :--------  | -----: |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/2707039?s=40&v=4" width="20" height="20" alt="">  &nbsp; [geekan](https://github.com/geekan) / [MetaGPT](https://github.com/geekan/MetaGPT) | 42787 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/130722866?s=40&v=4" width="20" height="20" alt="">  &nbsp; [run-llama](https://github.com/run-llama) / [llama_index](https://github.com/run-llama/llama_index) | 34446 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/170677839?s=40&v=4" width="20" height="20" alt="">  &nbsp; [crewAIInc](https://github.com/crewAIInc) / [crewAI](https://github.com/crewAIInc/crewAI) | 18287 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/134388954?s=40&v=4" width="20" height="20" alt="">  &nbsp; [camel-ai](https://github.com/camel-ai) / [camel](https://github.com/camel-ai/camel) | 5166 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/152537519?s=40&v=4" width="20" height="20" alt="">  &nbsp; [superagent-ai](https://github.com/superagent-ai) / [superagent](https://github.com/superagent-ai/superagent) | 5050 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/30197649?s=40&v=4" width="20" height="20" alt="">  &nbsp; [iyaja](https://github.com/iyaja) / [llama-fs](https://github.com/iyaja/llama-fs) | 4713 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/162546372?s=40&v=4" width="20" height="20" alt="">  &nbsp; [BasedHardware](https://github.com/BasedHardware) / [Omi](https://github.com/BasedHardware/Omi) | 2723 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/454862?s=40&v=4" width="20" height="20" alt="">  &nbsp; [MervinPraison](https://github.com/MervinPraison) / [PraisonAI](https://github.com/MervinPraison/PraisonAI) | 2007 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/140554352?s=40&v=4" width="20" height="20" alt="">  &nbsp; [AgentOps-AI](https://github.com/AgentOps-AI) / [Jaiqu](https://github.com/AgentOps-AI/Jaiqu) | 272 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/173542722?s=48&v=4" width="20" height="20" alt="">  &nbsp; [swarmzero](https://github.com/swarmzero) / [swarmzero](https://github.com/swarmzero/swarmzero) | 195 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/3074263?s=40&v=4" width="20" height="20" alt="">  &nbsp; [strnad](https://github.com/strnad) / [CrewAI-Studio](https://github.com/strnad/CrewAI-Studio) | 134 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/18406448?s=40&v=4" width="20" height="20" alt="">  &nbsp; [alejandro-ao](https://github.com/alejandro-ao) / [exa-crewai](https://github.com/alejandro-ao/exa-crewai) | 55 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/64493665?s=40&v=4" width="20" height="20" alt="">  &nbsp; [tonykipkemboi](https://github.com/tonykipkemboi) / [youtube_yapper_trapper](https://github.com/tonykipkemboi/youtube_yapper_trapper) | 47 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/17598928?s=40&v=4" width="20" height="20" alt="">  &nbsp; [sethcoast](https://github.com/sethcoast) / [cover-letter-builder](https://github.com/sethcoast/cover-letter-builder) | 27 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/109994880?s=40&v=4" width="20" height="20" alt="">  &nbsp; [bhancockio](https://github.com/bhancockio) / [chatgpt4o-analysis](https://github.com/bhancockio/chatgpt4o-analysis) | 19 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/14105911?s=40&v=4" width="20" height="20" alt="">  &nbsp; [breakstring](https://github.com/breakstring) / [Agentic_Story_Book_Workflow](https://github.com/breakstring/Agentic_Story_Book_Workflow) | 14 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/124134656?s=40&v=4" width="20" height="20" alt="">  &nbsp; [MULTI-ON](https://github.com/MULTI-ON) / [multion-python](https://github.com/MULTI-ON/multion-python) | 13 |


_Generated using [github-dependents-info](https://github.com/nvuillam/github-dependents-info), by [Nicolas Vuillamy](https://github.com/nvuillam)_

            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "agentops",
    "maintainer": null,
    "docs_url": null,
    "requires_python": "<3.14,>=3.9",
    "maintainer_email": null,
    "keywords": null,
    "author": null,
    "author_email": "Alex Reibman <areibman@gmail.com>, Shawn Qiu <siyangqiu@gmail.com>, Braelyn Boynton <bboynton97@gmail.com>, Howard Gil <howardbgil@gmail.com>, Constantin Teodorescu <teocns@gmail.com>, Pratyush Shukla <ps4534@nyu.edu>",
    "download_url": "https://files.pythonhosted.org/packages/76/a1/b03c6348a77798e750bde4eec03b4af620d71b9e4b64ff7dcf0860025a2d/agentops-0.3.26.tar.gz",
    "platform": null,
    "description": "<div align=\"center\">\n  <a href=\"https://agentops.ai?ref=gh\">\n    <img src=\"docs/images/external/logo/github-banner.png\" alt=\"Logo\">\n  </a>\n</div>\n\n<div align=\"center\">\n  <em>Observability and DevTool platform for AI Agents</em>\n</div>\n\n<br />\n\n<div align=\"center\">\n  <a href=\"https://pepy.tech/project/agentops\">\n    <img src=\"https://static.pepy.tech/badge/agentops/month\" alt=\"Downloads\">\n  </a>\n  <a href=\"https://github.com/agentops-ai/agentops/issues\">\n  <img src=\"https://img.shields.io/github/commit-activity/m/agentops-ai/agentops\" alt=\"git commit activity\">\n  </a>\n  <img src=\"https://img.shields.io/pypi/v/agentops?&color=3670A0\" alt=\"PyPI - Version\">\n  <a href=\"https://opensource.org/licenses/MIT\">\n    <img src=\"https://img.shields.io/badge/License-MIT-yellow.svg?&color=3670A0\" alt=\"License: MIT\">\n  </a>\n</div>\n\n<p align=\"center\">\n  <a href=\"https://twitter.com/agentopsai/\">\n    <img src=\"https://img.shields.io/twitter/follow/agentopsai?style=social\" alt=\"Twitter\" style=\"height: 20px;\">\n  </a>\n  <a href=\"https://discord.gg/FagdcwwXRR\">\n    <img src=\"https://img.shields.io/badge/discord-7289da.svg?style=flat-square&logo=discord\" alt=\"Discord\" style=\"height: 20px;\">\n  </a>\n  <a href=\"https://app.agentops.ai/?ref=gh\">\n    <img src=\"https://img.shields.io/badge/Dashboard-blue.svg?style=flat-square\" alt=\"Dashboard\" style=\"height: 20px;\">\n  </a>\n  <a href=\"https://docs.agentops.ai/introduction\">\n    <img src=\"https://img.shields.io/badge/Documentation-orange.svg?style=flat-square\" alt=\"Documentation\" style=\"height: 20px;\">\n  </a>\n  <a href=\"https://entelligence.ai/AgentOps-AI&agentops\">\n    <img src=\"https://img.shields.io/badge/Chat%20with%20Docs-green.svg?style=flat-square\" alt=\"Chat with Docs\" style=\"height: 20px;\">\n  </a>\n</p>\n\n\n\n<div style=\"justify-content: center\">\n  <img src=\"docs/images/external/app_screenshots/dashboard-banner.png\" alt=\"Dashboard Banner\">\n</div>\n\n<br/>\n\n\nAgentOps helps developers build, evaluate, and monitor AI agents. From prototype to production.\n\n|                                       |                                                               |\n| ------------------------------------- | ------------------------------------------------------------- |\n| \ud83d\udcca **Replay Analytics and Debugging** | Step-by-step agent execution graphs                           |\n| \ud83d\udcb8 **LLM Cost Management**            | Track spend with LLM foundation model providers               |\n| \ud83e\uddea **Agent Benchmarking**             | Test your agents against 1,000+ evals                         |\n| \ud83d\udd10 **Compliance and Security**        | Detect common prompt injection and data exfiltration exploits |\n| \ud83e\udd1d **Framework Integrations**         | Native Integrations with CrewAI, AG2(AutoGen), Camel AI, & LangChain         |\n\n## Quick Start \u2328\ufe0f\n\n```bash\npip install agentops\n```\n\n\n#### Session replays in 2 lines of code\n\nInitialize the AgentOps client and automatically get analytics on all your LLM calls.\n\n[Get an API key](https://app.agentops.ai/settings/projects)\n\n```python\nimport agentops\n\n# Beginning of your program (i.e. main.py, __init__.py)\nagentops.init( < INSERT YOUR API KEY HERE >)\n\n...\n\n# End of program\nagentops.end_session('Success')\n```\n\nAll your sessions can be viewed on the [AgentOps dashboard](https://app.agentops.ai?ref=gh)\n<br/>\n\n<details>\n  <summary>Agent Debugging</summary>\n  <a href=\"https://app.agentops.ai?ref=gh\">\n    <img src=\"docs/images/external/app_screenshots/session-drilldown-metadata.png\" style=\"width: 90%;\" alt=\"Agent Metadata\"/>\n  </a>\n  <a href=\"https://app.agentops.ai?ref=gh\">\n    <img src=\"docs/images/external/app_screenshots/chat-viewer.png\" style=\"width: 90%;\" alt=\"Chat Viewer\"/>\n  </a>\n  <a href=\"https://app.agentops.ai?ref=gh\">\n    <img src=\"docs/images/external/app_screenshots/session-drilldown-graphs.png\" style=\"width: 90%;\" alt=\"Event Graphs\"/>\n  </a>\n</details>\n\n<details>\n  <summary>Session Replays</summary>\n  <a href=\"https://app.agentops.ai?ref=gh\">\n    <img src=\"docs/images/external/app_screenshots/session-replay.png\" style=\"width: 90%;\" alt=\"Session Replays\"/>\n  </a>\n</details>\n\n<details open>\n  <summary>Summary Analytics</summary>\n  <a href=\"https://app.agentops.ai?ref=gh\">\n   <img src=\"docs/images/external/app_screenshots/overview.png\" style=\"width: 90%;\" alt=\"Summary Analytics\"/>\n  </a>\n  <a href=\"https://app.agentops.ai?ref=gh\">\n   <img src=\"docs/images/external/app_screenshots/overview-charts.png\" style=\"width: 90%;\" alt=\"Summary Analytics Charts\"/>\n  </a>\n</details>\n\n\n### First class Developer Experience\nAdd powerful observability to your agents, tools, and functions with as little code as possible: one line at a time.\n<br/>\nRefer to our [documentation](http://docs.agentops.ai)\n\n```python\n# Automatically associate all Events with the agent that originated them\nfrom agentops import track_agent\n\n@track_agent(name='SomeCustomName')\nclass MyAgent:\n  ...\n```\n\n```python\n# Automatically create ToolEvents for tools that agents will use\nfrom agentops import record_tool\n\n@record_tool('SampleToolName')\ndef sample_tool(...):\n  ...\n```\n\n```python\n# Automatically create ActionEvents for other functions.\nfrom agentops import record_action\n\n@agentops.record_action('sample function being record')\ndef sample_function(...):\n  ...\n```\n\n```python\n# Manually record any other Events\nfrom agentops import record, ActionEvent\n\nrecord(ActionEvent(\"received_user_input\"))\n```\n\n## Integrations \ud83e\uddbe\n\n### CrewAI \ud83d\udef6\n\nBuild Crew agents with observability with only 2 lines of code. Simply set an `AGENTOPS_API_KEY` in your environment, and your crews will get automatic monitoring on the AgentOps dashboard.\n\n```bash\npip install 'crewai[agentops]'\n```\n\n- [AgentOps integration example](https://docs.agentops.ai/v1/integrations/crewai)\n- [Official CrewAI documentation](https://docs.crewai.com/how-to/AgentOps-Observability)\n\n### AG2 \ud83e\udd16\nWith only two lines of code, add full observability and monitoring to AG2 (formerly AutoGen) agents. Set an `AGENTOPS_API_KEY` in your environment and call `agentops.init()`\n\n- [AG2 Observability Example](https://docs.ag2.ai/notebooks/agentchat_agentops)\n- [AG2 - AgentOps Documentation](https://docs.ag2.ai/docs/ecosystem/agentops)\n\n### Camel AI \ud83d\udc2a\n\nTrack and analyze CAMEL agents with full observability. Set an `AGENTOPS_API_KEY` in your environment and initialize AgentOps to get started.\n\n- [Camel AI](https://www.camel-ai.org/) - Advanced agent communication framework\n- [AgentOps integration example](https://docs.agentops.ai/v1/integrations/camel)\n- [Official Camel AI documentation](https://docs.camel-ai.org/cookbooks/agents_tracking.html)\n\n<details>\n  <summary>Installation</summary>\n\n```bash\npip install \"camel-ai[all]==0.2.11\"\npip install agentops\n```\n\n```python\nimport os\nimport agentops\nfrom camel.agents import ChatAgent\nfrom camel.messages import BaseMessage\nfrom camel.models import ModelFactory\nfrom camel.types import ModelPlatformType, ModelType\n\n# Initialize AgentOps\nagentops.init(os.getenv(\"AGENTOPS_API_KEY\"), default_tags=[\"CAMEL Example\"])\n\n# Import toolkits after AgentOps init for tracking\nfrom camel.toolkits import SearchToolkit\n\n# Set up the agent with search tools\nsys_msg = BaseMessage.make_assistant_message(\n    role_name='Tools calling operator',\n    content='You are a helpful assistant'\n)\n\n# Configure tools and model\ntools = [*SearchToolkit().get_tools()]\nmodel = ModelFactory.create(\n    model_platform=ModelPlatformType.OPENAI,\n    model_type=ModelType.GPT_4O_MINI,\n)\n\n# Create and run the agent\ncamel_agent = ChatAgent(\n    system_message=sys_msg,\n    model=model,\n    tools=tools,\n)\n\nresponse = camel_agent.step(\"What is AgentOps?\")\nprint(response)\n\nagentops.end_session(\"Success\")\n```\n\nCheck out our [Camel integration guide](https://docs.agentops.ai/v1/integrations/camel) for more examples including multi-agent scenarios.\n</details>\n\n### Langchain \ud83e\udd9c\ud83d\udd17\n\nAgentOps works seamlessly with applications built using Langchain. To use the handler, install Langchain as an optional dependency:\n\n<details>\n  <summary>Installation</summary>\n  \n```shell\npip install agentops[langchain]\n```\n\nTo use the handler, import and set\n\n```python\nimport os\nfrom langchain.chat_models import ChatOpenAI\nfrom langchain.agents import initialize_agent, AgentType\nfrom agentops.partners.langchain_callback_handler import LangchainCallbackHandler\n\nAGENTOPS_API_KEY = os.environ['AGENTOPS_API_KEY']\nhandler = LangchainCallbackHandler(api_key=AGENTOPS_API_KEY, tags=['Langchain Example'])\n\nllm = ChatOpenAI(openai_api_key=OPENAI_API_KEY,\n                 callbacks=[handler],\n                 model='gpt-3.5-turbo')\n\nagent = initialize_agent(tools,\n                         llm,\n                         agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,\n                         verbose=True,\n                         callbacks=[handler], # You must pass in a callback handler to record your agent\n                         handle_parsing_errors=True)\n```\n\nCheck out the [Langchain Examples Notebook](./examples/langchain_examples.ipynb) for more details including Async handlers.\n\n</details>\n\n### Cohere \u2328\ufe0f\n\nFirst class support for Cohere(>=5.4.0). This is a living integration, should you need any added functionality please message us on Discord!\n\n- [AgentOps integration example](https://docs.agentops.ai/v1/integrations/cohere)\n- [Official Cohere documentation](https://docs.cohere.com/reference/about)\n\n<details>\n  <summary>Installation</summary>\n  \n```bash\npip install cohere\n```\n\n```python python\nimport cohere\nimport agentops\n\n# Beginning of program's code (i.e. main.py, __init__.py)\nagentops.init(<INSERT YOUR API KEY HERE>)\nco = cohere.Client()\n\nchat = co.chat(\n    message=\"Is it pronounced ceaux-hear or co-hehray?\"\n)\n\nprint(chat)\n\nagentops.end_session('Success')\n```\n\n```python python\nimport cohere\nimport agentops\n\n# Beginning of program's code (i.e. main.py, __init__.py)\nagentops.init(<INSERT YOUR API KEY HERE>)\n\nco = cohere.Client()\n\nstream = co.chat_stream(\n    message=\"Write me a haiku about the synergies between Cohere and AgentOps\"\n)\n\nfor event in stream:\n    if event.event_type == \"text-generation\":\n        print(event.text, end='')\n\nagentops.end_session('Success')\n```\n</details>\n\n\n### Anthropic \ufe68\n\nTrack agents built with the Anthropic Python SDK (>=0.32.0).\n\n- [AgentOps integration guide](https://docs.agentops.ai/v1/integrations/anthropic)\n- [Official Anthropic documentation](https://docs.anthropic.com/en/docs/welcome)\n\n<details>\n  <summary>Installation</summary>\n  \n```bash\npip install anthropic\n```\n\n```python python\nimport anthropic\nimport agentops\n\n# Beginning of program's code (i.e. main.py, __init__.py)\nagentops.init(<INSERT YOUR API KEY HERE>)\n\nclient = anthropic.Anthropic(\n    # This is the default and can be omitted\n    api_key=os.environ.get(\"ANTHROPIC_API_KEY\"),\n)\n\nmessage = client.messages.create(\n        max_tokens=1024,\n        messages=[\n            {\n                \"role\": \"user\",\n                \"content\": \"Tell me a cool fact about AgentOps\",\n            }\n        ],\n        model=\"claude-3-opus-20240229\",\n    )\nprint(message.content)\n\nagentops.end_session('Success')\n```\n\nStreaming\n```python python\nimport anthropic\nimport agentops\n\n# Beginning of program's code (i.e. main.py, __init__.py)\nagentops.init(<INSERT YOUR API KEY HERE>)\n\nclient = anthropic.Anthropic(\n    # This is the default and can be omitted\n    api_key=os.environ.get(\"ANTHROPIC_API_KEY\"),\n)\n\nstream = client.messages.create(\n    max_tokens=1024,\n    model=\"claude-3-opus-20240229\",\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": \"Tell me something cool about streaming agents\",\n        }\n    ],\n    stream=True,\n)\n\nresponse = \"\"\nfor event in stream:\n    if event.type == \"content_block_delta\":\n        response += event.delta.text\n    elif event.type == \"message_stop\":\n        print(\"\\n\")\n        print(response)\n        print(\"\\n\")\n```\n\nAsync\n\n```python python\nimport asyncio\nfrom anthropic import AsyncAnthropic\n\nclient = AsyncAnthropic(\n    # This is the default and can be omitted\n    api_key=os.environ.get(\"ANTHROPIC_API_KEY\"),\n)\n\n\nasync def main() -> None:\n    message = await client.messages.create(\n        max_tokens=1024,\n        messages=[\n            {\n                \"role\": \"user\",\n                \"content\": \"Tell me something interesting about async agents\",\n            }\n        ],\n        model=\"claude-3-opus-20240229\",\n    )\n    print(message.content)\n\n\nawait main()\n```\n</details>\n\n### Mistral \u303d\ufe0f\n\nTrack agents built with the Anthropic Python SDK (>=0.32.0).\n\n- [AgentOps integration example](./examples/mistral//mistral_example.ipynb)\n- [Official Mistral documentation](https://docs.mistral.ai)\n\n<details>\n  <summary>Installation</summary>\n  \n```bash\npip install mistralai\n```\n\nSync\n\n```python python\nfrom mistralai import Mistral\nimport agentops\n\n# Beginning of program's code (i.e. main.py, __init__.py)\nagentops.init(<INSERT YOUR API KEY HERE>)\n\nclient = Mistral(\n    # This is the default and can be omitted\n    api_key=os.environ.get(\"MISTRAL_API_KEY\"),\n)\n\nmessage = client.chat.complete(\n        messages=[\n            {\n                \"role\": \"user\",\n                \"content\": \"Tell me a cool fact about AgentOps\",\n            }\n        ],\n        model=\"open-mistral-nemo\",\n    )\nprint(message.choices[0].message.content)\n\nagentops.end_session('Success')\n```\n\nStreaming\n\n```python python\nfrom mistralai import Mistral\nimport agentops\n\n# Beginning of program's code (i.e. main.py, __init__.py)\nagentops.init(<INSERT YOUR API KEY HERE>)\n\nclient = Mistral(\n    # This is the default and can be omitted\n    api_key=os.environ.get(\"MISTRAL_API_KEY\"),\n)\n\nmessage = client.chat.stream(\n        messages=[\n            {\n                \"role\": \"user\",\n                \"content\": \"Tell me something cool about streaming agents\",\n            }\n        ],\n        model=\"open-mistral-nemo\",\n    )\n\nresponse = \"\"\nfor event in message:\n    if event.data.choices[0].finish_reason == \"stop\":\n        print(\"\\n\")\n        print(response)\n        print(\"\\n\")\n    else:\n        response += event.text\n\nagentops.end_session('Success')\n```\n\nAsync\n\n```python python\nimport asyncio\nfrom mistralai import Mistral\n\nclient = Mistral(\n    # This is the default and can be omitted\n    api_key=os.environ.get(\"MISTRAL_API_KEY\"),\n)\n\n\nasync def main() -> None:\n    message = await client.chat.complete_async(\n        messages=[\n            {\n                \"role\": \"user\",\n                \"content\": \"Tell me something interesting about async agents\",\n            }\n        ],\n        model=\"open-mistral-nemo\",\n    )\n    print(message.choices[0].message.content)\n\n\nawait main()\n```\n\nAsync Streaming\n\n```python python\nimport asyncio\nfrom mistralai import Mistral\n\nclient = Mistral(\n    # This is the default and can be omitted\n    api_key=os.environ.get(\"MISTRAL_API_KEY\"),\n)\n\n\nasync def main() -> None:\n    message = await client.chat.stream_async(\n        messages=[\n            {\n                \"role\": \"user\",\n                \"content\": \"Tell me something interesting about async streaming agents\",\n            }\n        ],\n        model=\"open-mistral-nemo\",\n    )\n\n    response = \"\"\n    async for event in message:\n        if event.data.choices[0].finish_reason == \"stop\":\n            print(\"\\n\")\n            print(response)\n            print(\"\\n\")\n        else:\n            response += event.text\n\n\nawait main()\n```\n</details>\n\n\n\n### CamelAI \ufe68\n\nTrack agents built with the CamelAI Python SDK (>=0.32.0).\n\n- [CamelAI integration guide](https://docs.camel-ai.org/cookbooks/agents_tracking.html#)\n- [Official CamelAI documentation](https://docs.camel-ai.org/index.html)\n\n<details>\n  <summary>Installation</summary>\n  \n```bash\npip install camel-ai[all]\npip install agentops\n```\n\n```python python\n#Import Dependencies\nimport agentops\nimport os\nfrom getpass import getpass\nfrom dotenv import load_dotenv\n\n#Set Keys\nload_dotenv()\nopenai_api_key = os.getenv(\"OPENAI_API_KEY\") or \"<your openai key here>\"\nagentops_api_key = os.getenv(\"AGENTOPS_API_KEY\") or \"<your agentops key here>\"\n\n\n\n```\n</details>\n\n[You can find usage examples here!](examples/camelai_examples/README.md).\n\n\n\n### LiteLLM \ud83d\ude85\n\nAgentOps provides support for LiteLLM(>=1.3.1), allowing you to call 100+ LLMs using the same Input/Output Format. \n\n- [AgentOps integration example](https://docs.agentops.ai/v1/integrations/litellm)\n- [Official LiteLLM documentation](https://docs.litellm.ai/docs/providers)\n\n<details>\n  <summary>Installation</summary>\n  \n```bash\npip install litellm\n```\n\n```python python\n# Do not use LiteLLM like this\n# from litellm import completion\n# ...\n# response = completion(model=\"claude-3\", messages=messages)\n\n# Use LiteLLM like this\nimport litellm\n...\nresponse = litellm.completion(model=\"claude-3\", messages=messages)\n# or\nresponse = await litellm.acompletion(model=\"claude-3\", messages=messages)\n```\n</details>\n\n### LlamaIndex \ud83e\udd99\n\n\nAgentOps works seamlessly with applications built using LlamaIndex, a framework for building context-augmented generative AI applications with LLMs.\n\n<details>\n  <summary>Installation</summary>\n  \n```shell\npip install llama-index-instrumentation-agentops\n```\n\nTo use the handler, import and set\n\n```python\nfrom llama_index.core import set_global_handler\n\n# NOTE: Feel free to set your AgentOps environment variables (e.g., 'AGENTOPS_API_KEY')\n# as outlined in the AgentOps documentation, or pass the equivalent keyword arguments\n# anticipated by AgentOps' AOClient as **eval_params in set_global_handler.\n\nset_global_handler(\"agentops\")\n```\n\nCheck out the [LlamaIndex docs](https://docs.llamaindex.ai/en/stable/module_guides/observability/?h=agentops#agentops) for more details.\n\n</details>\n\n### Llama Stack \ud83e\udd99\ud83e\udd5e\n\nAgentOps provides support for Llama Stack Python Client(>=0.0.53), allowing you to monitor your Agentic applications. \n\n- [AgentOps integration example 1](https://github.com/AgentOps-AI/agentops/pull/530/files/65a5ab4fdcf310326f191d4b870d4f553591e3ea#diff-fdddf65549f3714f8f007ce7dfd1cde720329fe54155d54389dd50fbd81813cb)\n- [AgentOps integration example 2](https://github.com/AgentOps-AI/agentops/pull/530/files/65a5ab4fdcf310326f191d4b870d4f553591e3ea#diff-6688ff4fb7ab1ce7b1cc9b8362ca27264a3060c16737fb1d850305787a6e3699)\n- [Official Llama Stack Python Client](https://github.com/meta-llama/llama-stack-client-python)\n\n### SwarmZero AI \ud83d\udc1d\n\nTrack and analyze SwarmZero agents with full observability. Set an `AGENTOPS_API_KEY` in your environment and initialize AgentOps to get started.\n\n- [SwarmZero](https://swarmzero.ai) - Advanced multi-agent framework\n- [AgentOps integration example](https://docs.agentops.ai/v1/integrations/swarmzero)\n- [SwarmZero AI integration example](https://docs.swarmzero.ai/examples/ai-agents/build-and-monitor-a-web-search-agent)\n- [SwarmZero AI - AgentOps documentation](https://docs.swarmzero.ai/sdk/observability/agentops)\n- [Official SwarmZero Python SDK](https://github.com/swarmzero/swarmzero)\n\n<details>\n  <summary>Installation</summary>\n\n```bash\npip install swarmzero\npip install agentops\n```\n\n```python\nfrom dotenv import load_dotenv\nload_dotenv()\n\nimport agentops\nagentops.init(<INSERT YOUR API KEY HERE>)\n\nfrom swarmzero import Agent, Swarm\n# ...\n```\n</details>\n\n## Time travel debugging \ud83d\udd2e\n\n<div style=\"justify-content: center\">\n  <img src=\"docs/images/external/app_screenshots/time_travel_banner.png\" alt=\"Time Travel Banner\">\n</div>\n\n<br />\n\n[Try it out!](https://app.agentops.ai/timetravel)\n\n## Agent Arena \ud83e\udd4a\n\n(coming soon!)\n\n## Evaluations Roadmap \ud83e\udded\n\n| Platform                                                                     | Dashboard                                  | Evals                                  |\n| ---------------------------------------------------------------------------- | ------------------------------------------ | -------------------------------------- |\n| \u2705 Python SDK                                                                | \u2705 Multi-session and Cross-session metrics | \u2705 Custom eval metrics                 |\n| \ud83d\udea7 Evaluation builder API                                                    | \u2705 Custom event tag tracking\u00a0              | \ud83d\udd1c Agent scorecards                    |\n| \u2705 [Javascript/Typescript SDK](https://github.com/AgentOps-AI/agentops-node) | \u2705 Session replays                         | \ud83d\udd1c Evaluation playground + leaderboard |\n\n## Debugging Roadmap \ud83e\udded\n\n| Performance testing                       | Environments                                                                        | LLM Testing                                 | Reasoning and execution testing                   |\n| ----------------------------------------- | ----------------------------------------------------------------------------------- | ------------------------------------------- | ------------------------------------------------- |\n| \u2705 Event latency analysis                 | \ud83d\udd1c Non-stationary environment testing                                               | \ud83d\udd1c LLM non-deterministic function detection | \ud83d\udea7 Infinite loops and recursive thought detection |\n| \u2705 Agent workflow execution pricing       | \ud83d\udd1c Multi-modal environments                                                         | \ud83d\udea7 Token limit overflow flags               | \ud83d\udd1c Faulty reasoning detection                     |\n| \ud83d\udea7 Success validators (external)          | \ud83d\udd1c Execution containers                                                             | \ud83d\udd1c Context limit overflow flags             | \ud83d\udd1c Generative code validators                     |\n| \ud83d\udd1c Agent controllers/skill tests          | \u2705 Honeypot and prompt injection detection ([PromptArmor](https://promptarmor.com)) | \ud83d\udd1c API bill tracking                        | \ud83d\udd1c Error breakpoint analysis                      |\n| \ud83d\udd1c Information context constraint testing | \ud83d\udd1c Anti-agent roadblocks (i.e. Captchas)                                            | \ud83d\udd1c CI/CD integration checks                 |                                                   |\n| \ud83d\udd1c Regression testing                     | \ud83d\udd1c Multi-agent framework visualization                                              |                                             |                                                   |\n\n### Why AgentOps? \ud83e\udd14\n\nWithout the right tools, AI agents are slow, expensive, and unreliable. Our mission is to bring your agent from prototype to production. Here's why AgentOps stands out:\n\n- **Comprehensive Observability**: Track your AI agents' performance, user interactions, and API usage.\n- **Real-Time Monitoring**: Get instant insights with session replays, metrics, and live monitoring tools.\n- **Cost Control**: Monitor and manage your spend on LLM and API calls.\n- **Failure Detection**: Quickly identify and respond to agent failures and multi-agent interaction issues.\n- **Tool Usage Statistics**: Understand how your agents utilize external tools with detailed analytics.\n- **Session-Wide Metrics**: Gain a holistic view of your agents' sessions with comprehensive statistics.\n\nAgentOps is designed to make agent observability, testing, and monitoring easy.\n\n\n## Star History\n\nCheck out our growth in the community:\n\n<img src=\"https://api.star-history.com/svg?repos=AgentOps-AI/agentops&type=Date\" style=\"max-width: 500px\" width=\"50%\" alt=\"Logo\">\n\n## Popular projects using AgentOps\n\n\n| Repository | Stars  |\n| :--------  | -----: |\n|<img class=\"avatar mr-2\" src=\"https://avatars.githubusercontent.com/u/2707039?s=40&v=4\" width=\"20\" height=\"20\" alt=\"\">  &nbsp; [geekan](https://github.com/geekan) / [MetaGPT](https://github.com/geekan/MetaGPT) | 42787 |\n|<img class=\"avatar mr-2\" src=\"https://avatars.githubusercontent.com/u/130722866?s=40&v=4\" width=\"20\" height=\"20\" alt=\"\">  &nbsp; [run-llama](https://github.com/run-llama) / [llama_index](https://github.com/run-llama/llama_index) | 34446 |\n|<img class=\"avatar mr-2\" src=\"https://avatars.githubusercontent.com/u/170677839?s=40&v=4\" width=\"20\" height=\"20\" alt=\"\">  &nbsp; [crewAIInc](https://github.com/crewAIInc) / [crewAI](https://github.com/crewAIInc/crewAI) | 18287 |\n|<img class=\"avatar mr-2\" src=\"https://avatars.githubusercontent.com/u/134388954?s=40&v=4\" width=\"20\" height=\"20\" alt=\"\">  &nbsp; [camel-ai](https://github.com/camel-ai) / [camel](https://github.com/camel-ai/camel) | 5166 |\n|<img class=\"avatar mr-2\" src=\"https://avatars.githubusercontent.com/u/152537519?s=40&v=4\" width=\"20\" height=\"20\" alt=\"\">  &nbsp; [superagent-ai](https://github.com/superagent-ai) / [superagent](https://github.com/superagent-ai/superagent) | 5050 |\n|<img class=\"avatar mr-2\" src=\"https://avatars.githubusercontent.com/u/30197649?s=40&v=4\" width=\"20\" height=\"20\" alt=\"\">  &nbsp; [iyaja](https://github.com/iyaja) / [llama-fs](https://github.com/iyaja/llama-fs) | 4713 |\n|<img class=\"avatar mr-2\" src=\"https://avatars.githubusercontent.com/u/162546372?s=40&v=4\" width=\"20\" height=\"20\" alt=\"\">  &nbsp; [BasedHardware](https://github.com/BasedHardware) / [Omi](https://github.com/BasedHardware/Omi) | 2723 |\n|<img class=\"avatar mr-2\" src=\"https://avatars.githubusercontent.com/u/454862?s=40&v=4\" width=\"20\" height=\"20\" alt=\"\">  &nbsp; [MervinPraison](https://github.com/MervinPraison) / [PraisonAI](https://github.com/MervinPraison/PraisonAI) | 2007 |\n|<img class=\"avatar mr-2\" src=\"https://avatars.githubusercontent.com/u/140554352?s=40&v=4\" width=\"20\" height=\"20\" alt=\"\">  &nbsp; [AgentOps-AI](https://github.com/AgentOps-AI) / [Jaiqu](https://github.com/AgentOps-AI/Jaiqu) | 272 |\n|<img class=\"avatar mr-2\" src=\"https://avatars.githubusercontent.com/u/173542722?s=48&v=4\" width=\"20\" height=\"20\" alt=\"\">  &nbsp; [swarmzero](https://github.com/swarmzero) / [swarmzero](https://github.com/swarmzero/swarmzero) | 195 |\n|<img class=\"avatar mr-2\" src=\"https://avatars.githubusercontent.com/u/3074263?s=40&v=4\" width=\"20\" height=\"20\" alt=\"\">  &nbsp; [strnad](https://github.com/strnad) / [CrewAI-Studio](https://github.com/strnad/CrewAI-Studio) | 134 |\n|<img class=\"avatar mr-2\" src=\"https://avatars.githubusercontent.com/u/18406448?s=40&v=4\" width=\"20\" height=\"20\" alt=\"\">  &nbsp; [alejandro-ao](https://github.com/alejandro-ao) / [exa-crewai](https://github.com/alejandro-ao/exa-crewai) | 55 |\n|<img class=\"avatar mr-2\" src=\"https://avatars.githubusercontent.com/u/64493665?s=40&v=4\" width=\"20\" height=\"20\" alt=\"\">  &nbsp; [tonykipkemboi](https://github.com/tonykipkemboi) / [youtube_yapper_trapper](https://github.com/tonykipkemboi/youtube_yapper_trapper) | 47 |\n|<img class=\"avatar mr-2\" src=\"https://avatars.githubusercontent.com/u/17598928?s=40&v=4\" width=\"20\" height=\"20\" alt=\"\">  &nbsp; [sethcoast](https://github.com/sethcoast) / [cover-letter-builder](https://github.com/sethcoast/cover-letter-builder) | 27 |\n|<img class=\"avatar mr-2\" src=\"https://avatars.githubusercontent.com/u/109994880?s=40&v=4\" width=\"20\" height=\"20\" alt=\"\">  &nbsp; [bhancockio](https://github.com/bhancockio) / [chatgpt4o-analysis](https://github.com/bhancockio/chatgpt4o-analysis) | 19 |\n|<img class=\"avatar mr-2\" src=\"https://avatars.githubusercontent.com/u/14105911?s=40&v=4\" width=\"20\" height=\"20\" alt=\"\">  &nbsp; [breakstring](https://github.com/breakstring) / [Agentic_Story_Book_Workflow](https://github.com/breakstring/Agentic_Story_Book_Workflow) | 14 |\n|<img class=\"avatar mr-2\" src=\"https://avatars.githubusercontent.com/u/124134656?s=40&v=4\" width=\"20\" height=\"20\" alt=\"\">  &nbsp; [MULTI-ON](https://github.com/MULTI-ON) / [multion-python](https://github.com/MULTI-ON/multion-python) | 13 |\n\n\n_Generated using [github-dependents-info](https://github.com/nvuillam/github-dependents-info), by [Nicolas Vuillamy](https://github.com/nvuillam)_\n",
    "bugtrack_url": null,
    "license": null,
    "summary": "Observability and DevTool Platform for AI Agents",
    "version": "0.3.26",
    "project_urls": {
        "Homepage": "https://github.com/AgentOps-AI/agentops",
        "Issues": "https://github.com/AgentOps-AI/agentops/issues"
    },
    "split_keywords": [],
    "urls": [
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "f521671c458951850bd3a445aa09eafd2793aae1104fa68351a5c3976cdf762b",
                "md5": "c3f8fa92ff5a94a37516e774c7f58b9a",
                "sha256": "20948f52e3ffb4ba1d52301c3a82e59490182c4dad22774ad831dce0181eb5c2"
            },
            "downloads": -1,
            "filename": "agentops-0.3.26-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "c3f8fa92ff5a94a37516e774c7f58b9a",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": "<3.14,>=3.9",
            "size": 72090,
            "upload_time": "2025-01-24T23:44:06",
            "upload_time_iso_8601": "2025-01-24T23:44:06.828461Z",
            "url": "https://files.pythonhosted.org/packages/f5/21/671c458951850bd3a445aa09eafd2793aae1104fa68351a5c3976cdf762b/agentops-0.3.26-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "76a1b03c6348a77798e750bde4eec03b4af620d71b9e4b64ff7dcf0860025a2d",
                "md5": "ba4d0f2411ec72828677b38a395465cc",
                "sha256": "bc824bf8727332f59bf803cf84440d13e9e398406222ab29f45909ac1e39f815"
            },
            "downloads": -1,
            "filename": "agentops-0.3.26.tar.gz",
            "has_sig": false,
            "md5_digest": "ba4d0f2411ec72828677b38a395465cc",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": "<3.14,>=3.9",
            "size": 234235,
            "upload_time": "2025-01-24T23:44:08",
            "upload_time_iso_8601": "2025-01-24T23:44:08.541961Z",
            "url": "https://files.pythonhosted.org/packages/76/a1/b03c6348a77798e750bde4eec03b4af620d71b9e4b64ff7dcf0860025a2d/agentops-0.3.26.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-01-24 23:44:08",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "AgentOps-AI",
    "github_project": "agentops",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "agentops"
}