# AI Monitor - Plug & Play AI Agent Monitoring
A comprehensive, zero-configuration monitoring solution for AI agents that requires no source code changes. Simply import and use decorators or context managers.
## 🚀 Quick Start
### Installation (No Cloning Required!)
**One command installs everything:**
```bash
pip install ai-monitor
```
**That's it!** No cloning, no source code changes, no complex setup. Your AI monitoring is ready to use immediately.
### Basic Usage (Plug & Play)
#### 1. Ultra Simple - One Line Setup
```python
from ai_monitor import ultra_simple_setup
# Add this ONE line to enable comprehensive monitoring
ultra_simple_setup()
# Your existing AI code works unchanged!
import openai
response = openai.ChatCompletion.create(
model="gpt-4",
messages=[{"role": "user", "content": "Hello!"}]
)
# 🎉 Automatically monitored: latency, tokens, quality, traces
```
#### 2. Enterprise Setup with Traceloop
```python
from ai_monitor import setup_with_traceloop
# Enable enterprise-grade tracing
setup_with_traceloop("my_app_name")
# Your AI code - now with distributed tracing!
```
#### 3. Flask App Integration
```python
from flask import Flask
from ai_monitor import flask_plug_and_play
app = Flask(__name__)
# One line to monitor entire Flask app
flask_plug_and_play(app)
# All your routes are now monitored automatically!
```
### What You Get Immediately
After `pip install ai-monitor`, you get:
- ✅ **HTTP Interception** - Automatic OpenAI API monitoring
- ✅ **Prometheus Metrics** - Real-time metrics at `http://localhost:8000/metrics`
- ✅ **Quality Analysis** - Hallucination detection & drift analysis
- ✅ **OpenTelemetry Traces** - Ready for Jaeger/Traceloop integration
- ✅ **Zero Code Changes** - Works with your existing AI applications
#### 1. Decorator Style (Easiest)
```python
from ai_monitor import monitor_llm_call, monitor_agent, monitor_tool_use
# Monitor LLM calls automatically
@monitor_llm_call(model="gpt-4")
def call_openai(prompt):
# Your existing LLM code - no changes needed!
response = openai.ChatCompletion.create(...)
return response
# Monitor agent sessions
@monitor_agent(name="my_assistant")
def run_agent():
# Your agent code
return agent_result
# Monitor tool usage
@monitor_tool_use(tool_name="web_search")
def search_web(query):
# Your tool code
return search_results
```
#### 2. Context Manager Style
```python
from ai_monitor import monitor_llm_call, monitor_agent_session
# Monitor individual LLM calls
with monitor_llm_call("gpt-4", "Hello world") as monitor:
response = call_llm("Hello world")
monitor.record_response(
response="Hello! How can I help?",
input_tokens=2,
output_tokens=5,
cost=0.001
)
# Monitor complete agent sessions
with monitor_agent_session("customer_service") as session:
# Record LLM calls within session
session.record_llm_call("gpt-4", "Help request", "I can help with that", 10, 15, 0.002)
# Record tool usage
session.record_tool_use("knowledge_base", "search query", "results", success=True, duration=0.5)
```
#### 3. Global Monitor Instance
```python
from ai_monitor import get_monitor
# Get the default monitor instance
monitor = get_monitor()
# Record data manually
call_id = monitor.record_llm_call(
model="gpt-3.5-turbo",
prompt="What is AI?",
response="AI is artificial intelligence...",
input_tokens=4,
output_tokens=20,
latency=1.2,
cost=0.001
)
# Get monitoring data
stats = monitor.get_summary_stats()
metrics = monitor.get_metrics()
recent_calls = monitor.get_llm_calls(limit=10)
```
## 📊 What Gets Monitored
### All OpenAI API Endpoints (Enhanced Coverage)
- **Chat Completions**: `client.chat.completions.create()` ✅
- **Text Completions**: `client.completions.create()` ✅
- **Embeddings**: `client.embeddings.create()` ✅
- **Image Generation**: `client.images.generate()` (DALL-E) ✅
- **Audio Transcription**: `client.audio.transcriptions.create()` (Whisper) ✅
- **Audio Translation**: `client.audio.translations.create()` (Whisper) ✅
- **Content Moderation**: `client.moderations.create()` ✅
- **Model Operations**: `client.models.list()`, file operations, fine-tuning ✅
- **Azure OpenAI**: All deployment endpoints with enhanced model detection ✅
### Automatic Metrics (All Endpoints)
- **Latency & Performance**: Response time, tokens per second, throughput
- **Token Usage**: Input/output tokens, token costs, efficiency ratios
- **Quality Indicators**: Response completeness, structure, consistency
- **Error Tracking**: Failure rates, error types, recovery times
- **Resource Usage**: Memory, CPU (when psutil installed)
### Advanced Detection
- **Hallucination Detection**: Pattern-based detection of uncertain or fabricated content
- **Model Drift**: Automatic detection of performance degradation over time
- **Cost Monitoring**: Real-time cost tracking and budgeting
### Trace & Context
- **Request Tracing**: End-to-end request flows with Jaeger integration
- **Agent Sessions**: Multi-step agent conversations and decision paths
- **Tool Usage**: External API calls and tool performance
## 📈 Data Export Options
### Built-in Exporters
- **Prometheus**: Metrics collection with built-in HTTP server
- **Jaeger**: Distributed tracing for request flows
- **Structured Logs**: JSON logs for analysis
- **Console**: Real-time monitoring output
- **JSON Files**: Local data storage
### Configuration Example
```python
from ai_monitor import init_monitoring, MonitoringConfig
config = MonitoringConfig(
# Exporters
enable_prometheus=True,
enable_jaeger=True,
enable_logging=True,
# Prometheus
prometheus_port=8000,
# Features
detect_hallucination=True,
detect_drift=True,
track_costs=True,
# Sampling
trace_sampling_rate=1.0
)
monitor = init_monitoring(config)
```
## 🔍 Monitoring Features
### Hallucination Detection
```python
from ai_monitor.detectors import HallucinationDetector
detector = HallucinationDetector()
result = detector.detect(
prompt="What is the capital of France?",
response="I think it might be Paris, but I'm not entirely sure...",
context="France is a country in Europe."
)
print(f"Is hallucination: {result.is_hallucination}")
print(f"Confidence: {result.confidence_score}")
print(f"Reasons: {result.reasons}")
```
### Drift Detection
```python
from ai_monitor.detectors import DriftDetector
drift_detector = DriftDetector()
# Update with each LLM call
drift_detector.update(
latency=1.2,
input_tokens=10,
output_tokens=50,
cost=0.002,
response="AI response text"
)
# Check for drift
drift_results = drift_detector.detect_drift()
for drift in drift_results:
if drift.has_drift:
print(f"Drift detected in {drift.drift_type}")
print(f"Baseline: {drift.baseline_value}, Current: {drift.current_value}")
```
## 📋 Dashboard & Visualization
### Prometheus Metrics (localhost:8000)
```
ai_monitor_llm_calls_total{model="gpt-4"} 150
ai_monitor_llm_latency_seconds_bucket{model="gpt-4",le="1.0"} 120
ai_monitor_tokens_total{model="gpt-4",type="input"} 5000
ai_monitor_tokens_total{model="gpt-4",type="output"} 15000
ai_monitor_cost_total{model="gpt-4"} 2.50
```
### Getting Summary Statistics
```python
from ai_monitor import get_monitor
monitor = get_monitor()
stats = monitor.get_summary_stats()
print(f"Total LLM calls: {stats['total_calls']}")
print(f"Total tokens: {stats['total_tokens']}")
print(f"Total cost: ${stats['total_cost']:.4f}")
print(f"Average latency: {stats['average_latency']:.2f}s")
print(f"Tokens per second: {stats['tokens_per_second']:.1f}")
```
## 🛠️ Integration Examples
### OpenAI Integration
```python
from ai_monitor import monitor_llm_call
import openai
@monitor_llm_call() # Automatically detects OpenAI format
def chat_with_gpt(messages):
response = openai.ChatCompletion.create(
model="gpt-4",
messages=messages
)
return response
# Usage - monitoring happens automatically!
result = chat_with_gpt([{"role": "user", "content": "Hello"}])
```
### Anthropic Claude Integration
```python
@monitor_llm_call(model="claude-3")
def chat_with_claude(prompt):
response = anthropic.completions.create(
model="claude-3-sonnet-20240229",
prompt=prompt,
max_tokens_to_sample=100
)
return response
```
### Custom Agent Integration
```python
@monitor_agent(name="research_assistant")
def research_agent(query):
with monitor_agent_session("research_session") as session:
# Search step
search_results = search_web(query)
session.record_tool_use("web_search", query, search_results, True, 2.1)
# Analysis step
analysis = analyze_with_llm(search_results)
session.record_llm_call("gpt-4", f"Analyze: {search_results}", analysis, 100, 200, 0.01)
# Summary step
summary = summarize_findings(analysis)
session.record_llm_call("gpt-3.5-turbo", f"Summarize: {analysis}", summary, 200, 50, 0.002)
return summary
```
## 📊 Grafana Dashboard Setup
1. **Install Grafana and Prometheus**
2. **Configure Prometheus** to scrape `localhost:8000/metrics`
3. **Import dashboard** with these key metrics:
- LLM call rate and latency
- Token usage and costs
- Error rates and drift detection
- Agent session success rates
## 🚨 Alerting
### Prometheus Alerting Rules
```yaml
groups:
- name: ai_monitor_alerts
rules:
- alert: HighLatency
expr: ai_monitor_llm_latency_seconds > 5
labels:
severity: warning
annotations:
summary: "High LLM latency detected"
- alert: CostThreshold
expr: increase(ai_monitor_cost_total[1h]) > 10
labels:
severity: critical
annotations:
summary: "Hourly cost exceeds $10"
```
## 🔧 Advanced Configuration
### Custom Exporters
```python
from ai_monitor import configure_exporters, get_monitor
monitor = get_monitor()
configure_exporters(monitor, [
{'type': 'console'},
{'type': 'json_file', 'file_prefix': 'my_app_monitor'}
])
```
### Performance Profiling
```python
from ai_monitor.utils import PerformanceProfiler
profiler = PerformanceProfiler()
with profiler.profile_operation("llm_inference"):
result = call_llm("prompt")
summary = profiler.get_performance_summary()
```
## 🎯 Key Benefits
- **Zero Code Changes**: Drop-in decorators and context managers
- **Comprehensive Monitoring**: Tracks everything from tokens to traces
- **Production Ready**: Prometheus, Jaeger, and structured logging
- **Intelligent Detection**: Automated hallucination and drift detection
- **Cost Awareness**: Real-time cost tracking and optimization
- **Performance Insights**: Detailed latency and throughput analysis
## 📝 License
MIT License - feel free to use in your projects!
---
**Ready to monitor your AI agents?** Just `pip install` the dependencies and add a single decorator! 🚀
Raw data
{
"_id": null,
"home_page": "https://github.com/yourusername/ai-monitor",
"name": "ai-monitor",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "AI Monitor Team <ai-monitor@example.com>",
"keywords": "ai, monitoring, llm, agent, observability, prometheus, opentelemetry, tracing, metrics, quality-analysis, hallucination-detection, drift-detection",
"author": "AI Monitor Team",
"author_email": "AI Monitor Team <ai-monitor@example.com>",
"download_url": "https://files.pythonhosted.org/packages/3b/4c/d58c6d4b36b6b9334e4ab760bf9f2ba4e5d5a5c938c3ae92e03142bec311/ai_monitor-1.0.11.tar.gz",
"platform": null,
"description": "# AI Monitor - Plug & Play AI Agent Monitoring\r\n\r\nA comprehensive, zero-configuration monitoring solution for AI agents that requires no source code changes. Simply import and use decorators or context managers.\r\n\r\n## \ud83d\ude80 Quick Start\r\n\r\n### Installation (No Cloning Required!)\r\n\r\n**One command installs everything:**\r\n```bash\r\npip install ai-monitor\r\n```\r\n\r\n**That's it!** No cloning, no source code changes, no complex setup. Your AI monitoring is ready to use immediately.\r\n\r\n### Basic Usage (Plug & Play)\r\n\r\n#### 1. Ultra Simple - One Line Setup\r\n```python\r\nfrom ai_monitor import ultra_simple_setup\r\n\r\n# Add this ONE line to enable comprehensive monitoring\r\nultra_simple_setup()\r\n\r\n# Your existing AI code works unchanged!\r\nimport openai\r\nresponse = openai.ChatCompletion.create(\r\n model=\"gpt-4\",\r\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\r\n)\r\n# \ud83c\udf89 Automatically monitored: latency, tokens, quality, traces\r\n```\r\n\r\n#### 2. Enterprise Setup with Traceloop\r\n```python\r\nfrom ai_monitor import setup_with_traceloop\r\n\r\n# Enable enterprise-grade tracing\r\nsetup_with_traceloop(\"my_app_name\")\r\n\r\n# Your AI code - now with distributed tracing!\r\n```\r\n\r\n#### 3. Flask App Integration\r\n```python\r\nfrom flask import Flask\r\nfrom ai_monitor import flask_plug_and_play\r\n\r\napp = Flask(__name__)\r\n\r\n# One line to monitor entire Flask app\r\nflask_plug_and_play(app)\r\n\r\n# All your routes are now monitored automatically!\r\n```\r\n\r\n### What You Get Immediately\r\n\r\nAfter `pip install ai-monitor`, you get:\r\n- \u2705 **HTTP Interception** - Automatic OpenAI API monitoring\r\n- \u2705 **Prometheus Metrics** - Real-time metrics at `http://localhost:8000/metrics`\r\n- \u2705 **Quality Analysis** - Hallucination detection & drift analysis\r\n- \u2705 **OpenTelemetry Traces** - Ready for Jaeger/Traceloop integration\r\n- \u2705 **Zero Code Changes** - Works with your existing AI applications\r\n\r\n#### 1. Decorator Style (Easiest)\r\n```python\r\nfrom ai_monitor import monitor_llm_call, monitor_agent, monitor_tool_use\r\n\r\n# Monitor LLM calls automatically\r\n@monitor_llm_call(model=\"gpt-4\")\r\ndef call_openai(prompt):\r\n # Your existing LLM code - no changes needed!\r\n response = openai.ChatCompletion.create(...)\r\n return response\r\n\r\n# Monitor agent sessions \r\n@monitor_agent(name=\"my_assistant\")\r\ndef run_agent():\r\n # Your agent code\r\n return agent_result\r\n\r\n# Monitor tool usage\r\n@monitor_tool_use(tool_name=\"web_search\")\r\ndef search_web(query):\r\n # Your tool code\r\n return search_results\r\n```\r\n\r\n#### 2. Context Manager Style\r\n```python\r\nfrom ai_monitor import monitor_llm_call, monitor_agent_session\r\n\r\n# Monitor individual LLM calls\r\nwith monitor_llm_call(\"gpt-4\", \"Hello world\") as monitor:\r\n response = call_llm(\"Hello world\")\r\n monitor.record_response(\r\n response=\"Hello! How can I help?\",\r\n input_tokens=2,\r\n output_tokens=5,\r\n cost=0.001\r\n )\r\n\r\n# Monitor complete agent sessions\r\nwith monitor_agent_session(\"customer_service\") as session:\r\n # Record LLM calls within session\r\n session.record_llm_call(\"gpt-4\", \"Help request\", \"I can help with that\", 10, 15, 0.002)\r\n \r\n # Record tool usage\r\n session.record_tool_use(\"knowledge_base\", \"search query\", \"results\", success=True, duration=0.5)\r\n```\r\n\r\n#### 3. Global Monitor Instance\r\n```python\r\nfrom ai_monitor import get_monitor\r\n\r\n# Get the default monitor instance\r\nmonitor = get_monitor()\r\n\r\n# Record data manually\r\ncall_id = monitor.record_llm_call(\r\n model=\"gpt-3.5-turbo\",\r\n prompt=\"What is AI?\",\r\n response=\"AI is artificial intelligence...\",\r\n input_tokens=4,\r\n output_tokens=20,\r\n latency=1.2,\r\n cost=0.001\r\n)\r\n\r\n# Get monitoring data\r\nstats = monitor.get_summary_stats()\r\nmetrics = monitor.get_metrics()\r\nrecent_calls = monitor.get_llm_calls(limit=10)\r\n```\r\n\r\n## \ud83d\udcca What Gets Monitored\r\n\r\n### All OpenAI API Endpoints (Enhanced Coverage)\r\n- **Chat Completions**: `client.chat.completions.create()` \u2705\r\n- **Text Completions**: `client.completions.create()` \u2705 \r\n- **Embeddings**: `client.embeddings.create()` \u2705\r\n- **Image Generation**: `client.images.generate()` (DALL-E) \u2705\r\n- **Audio Transcription**: `client.audio.transcriptions.create()` (Whisper) \u2705\r\n- **Audio Translation**: `client.audio.translations.create()` (Whisper) \u2705\r\n- **Content Moderation**: `client.moderations.create()` \u2705\r\n- **Model Operations**: `client.models.list()`, file operations, fine-tuning \u2705\r\n- **Azure OpenAI**: All deployment endpoints with enhanced model detection \u2705\r\n\r\n### Automatic Metrics (All Endpoints)\r\n- **Latency & Performance**: Response time, tokens per second, throughput \r\n- **Token Usage**: Input/output tokens, token costs, efficiency ratios\r\n- **Quality Indicators**: Response completeness, structure, consistency\r\n- **Error Tracking**: Failure rates, error types, recovery times\r\n- **Resource Usage**: Memory, CPU (when psutil installed)\r\n\r\n### Advanced Detection\r\n- **Hallucination Detection**: Pattern-based detection of uncertain or fabricated content\r\n- **Model Drift**: Automatic detection of performance degradation over time\r\n- **Cost Monitoring**: Real-time cost tracking and budgeting\r\n\r\n### Trace & Context\r\n- **Request Tracing**: End-to-end request flows with Jaeger integration\r\n- **Agent Sessions**: Multi-step agent conversations and decision paths\r\n- **Tool Usage**: External API calls and tool performance\r\n\r\n## \ud83d\udcc8 Data Export Options\r\n\r\n### Built-in Exporters\r\n- **Prometheus**: Metrics collection with built-in HTTP server\r\n- **Jaeger**: Distributed tracing for request flows\r\n- **Structured Logs**: JSON logs for analysis\r\n- **Console**: Real-time monitoring output\r\n- **JSON Files**: Local data storage\r\n\r\n### Configuration Example\r\n```python\r\nfrom ai_monitor import init_monitoring, MonitoringConfig\r\n\r\nconfig = MonitoringConfig(\r\n # Exporters\r\n enable_prometheus=True,\r\n enable_jaeger=True, \r\n enable_logging=True,\r\n \r\n # Prometheus\r\n prometheus_port=8000,\r\n \r\n # Features\r\n detect_hallucination=True,\r\n detect_drift=True,\r\n track_costs=True,\r\n \r\n # Sampling\r\n trace_sampling_rate=1.0\r\n)\r\n\r\nmonitor = init_monitoring(config)\r\n```\r\n\r\n## \ud83d\udd0d Monitoring Features\r\n\r\n### Hallucination Detection\r\n```python\r\nfrom ai_monitor.detectors import HallucinationDetector\r\n\r\ndetector = HallucinationDetector()\r\nresult = detector.detect(\r\n prompt=\"What is the capital of France?\",\r\n response=\"I think it might be Paris, but I'm not entirely sure...\",\r\n context=\"France is a country in Europe.\"\r\n)\r\n\r\nprint(f\"Is hallucination: {result.is_hallucination}\")\r\nprint(f\"Confidence: {result.confidence_score}\")\r\nprint(f\"Reasons: {result.reasons}\")\r\n```\r\n\r\n### Drift Detection\r\n```python\r\nfrom ai_monitor.detectors import DriftDetector\r\n\r\ndrift_detector = DriftDetector()\r\n\r\n# Update with each LLM call\r\ndrift_detector.update(\r\n latency=1.2,\r\n input_tokens=10,\r\n output_tokens=50,\r\n cost=0.002,\r\n response=\"AI response text\"\r\n)\r\n\r\n# Check for drift\r\ndrift_results = drift_detector.detect_drift()\r\nfor drift in drift_results:\r\n if drift.has_drift:\r\n print(f\"Drift detected in {drift.drift_type}\")\r\n print(f\"Baseline: {drift.baseline_value}, Current: {drift.current_value}\")\r\n```\r\n\r\n## \ud83d\udccb Dashboard & Visualization\r\n\r\n### Prometheus Metrics (localhost:8000)\r\n```\r\nai_monitor_llm_calls_total{model=\"gpt-4\"} 150\r\nai_monitor_llm_latency_seconds_bucket{model=\"gpt-4\",le=\"1.0\"} 120\r\nai_monitor_tokens_total{model=\"gpt-4\",type=\"input\"} 5000\r\nai_monitor_tokens_total{model=\"gpt-4\",type=\"output\"} 15000\r\nai_monitor_cost_total{model=\"gpt-4\"} 2.50\r\n```\r\n\r\n### Getting Summary Statistics\r\n```python\r\nfrom ai_monitor import get_monitor\r\n\r\nmonitor = get_monitor() \r\nstats = monitor.get_summary_stats()\r\n\r\nprint(f\"Total LLM calls: {stats['total_calls']}\")\r\nprint(f\"Total tokens: {stats['total_tokens']}\")\r\nprint(f\"Total cost: ${stats['total_cost']:.4f}\")\r\nprint(f\"Average latency: {stats['average_latency']:.2f}s\")\r\nprint(f\"Tokens per second: {stats['tokens_per_second']:.1f}\")\r\n```\r\n\r\n## \ud83d\udee0\ufe0f Integration Examples\r\n\r\n### OpenAI Integration\r\n```python\r\nfrom ai_monitor import monitor_llm_call\r\nimport openai\r\n\r\n@monitor_llm_call() # Automatically detects OpenAI format\r\ndef chat_with_gpt(messages):\r\n response = openai.ChatCompletion.create(\r\n model=\"gpt-4\",\r\n messages=messages\r\n )\r\n return response\r\n\r\n# Usage - monitoring happens automatically!\r\nresult = chat_with_gpt([{\"role\": \"user\", \"content\": \"Hello\"}])\r\n```\r\n\r\n### Anthropic Claude Integration\r\n```python\r\n@monitor_llm_call(model=\"claude-3\")\r\ndef chat_with_claude(prompt):\r\n response = anthropic.completions.create(\r\n model=\"claude-3-sonnet-20240229\",\r\n prompt=prompt,\r\n max_tokens_to_sample=100\r\n )\r\n return response\r\n```\r\n\r\n### Custom Agent Integration\r\n```python\r\n@monitor_agent(name=\"research_assistant\")\r\ndef research_agent(query):\r\n with monitor_agent_session(\"research_session\") as session:\r\n \r\n # Search step\r\n search_results = search_web(query)\r\n session.record_tool_use(\"web_search\", query, search_results, True, 2.1)\r\n \r\n # Analysis step\r\n analysis = analyze_with_llm(search_results)\r\n session.record_llm_call(\"gpt-4\", f\"Analyze: {search_results}\", analysis, 100, 200, 0.01)\r\n \r\n # Summary step\r\n summary = summarize_findings(analysis)\r\n session.record_llm_call(\"gpt-3.5-turbo\", f\"Summarize: {analysis}\", summary, 200, 50, 0.002)\r\n \r\n return summary\r\n```\r\n\r\n## \ud83d\udcca Grafana Dashboard Setup\r\n\r\n1. **Install Grafana and Prometheus**\r\n2. **Configure Prometheus** to scrape `localhost:8000/metrics`\r\n3. **Import dashboard** with these key metrics:\r\n - LLM call rate and latency\r\n - Token usage and costs\r\n - Error rates and drift detection\r\n - Agent session success rates\r\n\r\n## \ud83d\udea8 Alerting\r\n\r\n### Prometheus Alerting Rules\r\n```yaml\r\ngroups:\r\n- name: ai_monitor_alerts\r\n rules:\r\n - alert: HighLatency\r\n expr: ai_monitor_llm_latency_seconds > 5\r\n labels:\r\n severity: warning\r\n annotations:\r\n summary: \"High LLM latency detected\"\r\n \r\n - alert: CostThreshold\r\n expr: increase(ai_monitor_cost_total[1h]) > 10\r\n labels:\r\n severity: critical\r\n annotations:\r\n summary: \"Hourly cost exceeds $10\"\r\n```\r\n\r\n## \ud83d\udd27 Advanced Configuration\r\n\r\n### Custom Exporters\r\n```python\r\nfrom ai_monitor import configure_exporters, get_monitor\r\n\r\nmonitor = get_monitor()\r\nconfigure_exporters(monitor, [\r\n {'type': 'console'},\r\n {'type': 'json_file', 'file_prefix': 'my_app_monitor'}\r\n])\r\n```\r\n\r\n### Performance Profiling\r\n```python\r\nfrom ai_monitor.utils import PerformanceProfiler\r\n\r\nprofiler = PerformanceProfiler()\r\n\r\nwith profiler.profile_operation(\"llm_inference\"):\r\n result = call_llm(\"prompt\")\r\n\r\nsummary = profiler.get_performance_summary()\r\n```\r\n\r\n## \ud83c\udfaf Key Benefits\r\n\r\n- **Zero Code Changes**: Drop-in decorators and context managers\r\n- **Comprehensive Monitoring**: Tracks everything from tokens to traces\r\n- **Production Ready**: Prometheus, Jaeger, and structured logging\r\n- **Intelligent Detection**: Automated hallucination and drift detection \r\n- **Cost Awareness**: Real-time cost tracking and optimization\r\n- **Performance Insights**: Detailed latency and throughput analysis\r\n\r\n## \ud83d\udcdd License\r\n\r\nMIT License - feel free to use in your projects!\r\n\r\n---\r\n\r\n**Ready to monitor your AI agents?** Just `pip install` the dependencies and add a single decorator! \ud83d\ude80\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Plug & Play AI Agent Monitoring - Zero-configuration monitoring for AI agents with no source code changes",
"version": "1.0.11",
"project_urls": {
"Changelog": "https://github.com/yourusername/ai-monitor/blob/main/CHANGELOG.md",
"Documentation": "https://ai-monitor.readthedocs.io/",
"Homepage": "https://github.com/yourusername/ai-monitor",
"Issues": "https://github.com/yourusername/ai-monitor/issues",
"Repository": "https://github.com/yourusername/ai-monitor"
},
"split_keywords": [
"ai",
" monitoring",
" llm",
" agent",
" observability",
" prometheus",
" opentelemetry",
" tracing",
" metrics",
" quality-analysis",
" hallucination-detection",
" drift-detection"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "b21309a3ea8271f040195b4106ceda9532e738451fa30d6ddcfbfad62c0c4991",
"md5": "ab3fe33094452e2d2dd1c931021b340d",
"sha256": "b905d8f1e7044da321cad9041974c2ed1fd4f0161b741febf6b1a512d46781bd"
},
"downloads": -1,
"filename": "ai_monitor-1.0.11-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ab3fe33094452e2d2dd1c931021b340d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 51306,
"upload_time": "2025-10-26T18:02:49",
"upload_time_iso_8601": "2025-10-26T18:02:49.730283Z",
"url": "https://files.pythonhosted.org/packages/b2/13/09a3ea8271f040195b4106ceda9532e738451fa30d6ddcfbfad62c0c4991/ai_monitor-1.0.11-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "3b4cd58c6d4b36b6b9334e4ab760bf9f2ba4e5d5a5c938c3ae92e03142bec311",
"md5": "9c59853ca139ed97c1686fa5334c4045",
"sha256": "9cb7d7a545b9cd667f126f3b579801c0fd659e7aae1000bf4a23fd1ce44584fe"
},
"downloads": -1,
"filename": "ai_monitor-1.0.11.tar.gz",
"has_sig": false,
"md5_digest": "9c59853ca139ed97c1686fa5334c4045",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 53060,
"upload_time": "2025-10-26T18:02:51",
"upload_time_iso_8601": "2025-10-26T18:02:51.341553Z",
"url": "https://files.pythonhosted.org/packages/3b/4c/d58c6d4b36b6b9334e4ab760bf9f2ba4e5d5a5c938c3ae92e03142bec311/ai_monitor-1.0.11.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-26 18:02:51",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "yourusername",
"github_project": "ai-monitor",
"github_not_found": true,
"lcname": "ai-monitor"
}
JSON
