# Observicia SDK
Observicia is a Cloud Native observability and policy control SDK for LLM applications. It provides seamless integration with CNCF native observability stack while offering comprehensive token tracking, policy enforcement, and PII protection capabilities.
[](https://observicia.readthedocs.io/en/latest/)
[](LICENSE)
[](https://opentelemetry.io/)
[](https://www.openpolicyagent.org/)
[](https://badge.fury.io/py/observicia)
[](https://pypi.python.org/pypi/observicia/)
## Features
- **Token Tracking and Management**
- Real-time token usage monitoring across providers
- Stream-aware token counting
- Token usage retention and cleanup
- Per-session token tracking
- Configurable data retention policies
- **LLM Backend Support**
- OpenAI
- Chat completions (sync/async)
- Text completions (sync/async)
- Embeddings
- Image generation
- File operations
- Streaming support
- Ollama
- Local model deployment
- Chat completions
- Text generation
- Embeddings
- Streaming support
- WatsonX
- Foundation models integration
- Text generation
- Chat completions
- Parameter controls
- Basic scaffolding for:
- Anthropic
- LiteLLM
- **Transaction Tracking**
- Multi-round conversation tracking
- Transaction lifecycle management
- Metadata and state tracking
- Parent-child transaction relationships
- Transaction performance metrics
- **Chat Logging and Analytics**
- Structured chat history logging
- Conversation flow analysis
- Interaction metrics
- Policy compliance logging
- Chat completion tracking
- **Telemetry Storage and Export**
- SQLite exporter for persistent telemetry storage
- Structured schema for token usage and metrics
- Transaction and trace correlation
- Query-friendly format for analytics
- Redis exporter with configurable retention
- Time-based data retention policies
- Real-time metrics access
- Distributed telemetry storage
- OpenTelemetry integration
- Standard OTLP export support
- Custom attribute mapping
- Span context preservation
- **Policy Enforcement**
- Integration with Open Policy Agent (OPA)
- Support for multiple policy evaluation levels
- Risk level assessment (low, medium, high, critical)
- Custom policy definition support
- Synchronous and asynchronous policy evaluation
- **Framework Integration**
- LangChain support
- Conversation chain monitoring
- Chain metrics
- Token usage across abstractions
- **Observability Features**
- OpenTelemetry integration
- Span-based tracing for all LLM operations
- Configurable logging (console, file, OTLP)
- Mermaid diagram generation from telemetry data
- Detailed request/response tracing
- Custom attribute tracking
## Quick Start
1. Install the SDK:
```bash
pip install observicia
```
2. Create a configuration file (`observicia_config.yaml`):
```yaml
service_name: my-service
otel_endpoint: http://localhost:4317
opa_endpoint: http://localhost:8181/
policies:
- name: pii_check
path: policies/pii
description: Check for PII in responses
required_trace_level: enhanced
risk_level: high
logging:
file: "app.json"
telemetry:
enabled: true
format: "json"
redis:
enabled: true
host: "localhost"
port: 6379
db: 0
key_prefix: "observicia:telemetry:"
retention_hours: 24
messages:
enabled: true
level: "INFO"
chat:
enabled: true
level: "both"
file: "chat.log"
```
3. Initialize in your code:
```python
from observicia import init
from observicia.core.context_manager import ObservabilityContext
# Required - Initialize Observicia
init()
# Optional - Set user ID for tracking
ObservabilityContext.set_user_id("user123")
# Optional - Start a conversation transaction
transaction_id = ObservabilityContext.start_transaction(
metadata={"conversation_type": "chat"}
)
# Use with OpenAI
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": "Hello!"}]
)
# Or use with Ollama
import ollama
response = ollama.chat(
model="llama2",
messages=[{"role": "user", "content": "Hello!"}]
)
# Optional - End the transaction
ObservabilityContext.end_transaction(
transaction_id,
metadata={"resolution": "completed"}
)
```
## Architecture
```mermaid
flowchart TB
App[Application] --> SDK[Observicia SDK]
subgraph LLM Backends
OpenAI[OpenAI API]
Ollama[Ollama Local]
Anthropic[Anthropic API]
LiteLLM[LiteLLM]
WatsonX[WatsonX]
end
SDK --> OpenAI
SDK --> Ollama
SDK --> Anthropic
SDK --> LiteLLM
SDK --> WatsonX
SDK --> OPA[Open Policy Agent]
SDK --> OTEL[OpenTelemetry Collector]
SDK --> SQLite[(SQLite)]
SDK --> Redis[(Redis)]
OTEL --> Jaeger[Jaeger]
OTEL --> Prom[Prometheus]
OPA --> PII[PII Detection Service]
OPA --> Compliance[Prompt Compliance Service]
subgraph Telemetry Storage
SQLite
Redis
end
style OpenAI fill:#85e,color:#fff
style Ollama fill:#85e,color:#fff
style WatsonX fill:#85e,color:#fff
style Anthropic fill:#ccc,color:#666
style LiteLLM fill:#ccc,color:#666
```
## Example Applications
The SDK includes three example applications demonstrating different use cases:
## Example Applications
The SDK includes the following example applications demonstrating different use cases:
1. **Simple Chat Application** ([examples/simple-chat](examples/simple-chat))
- Basic chat interface using OpenAI
- Demonstrates token tracking and tracing
- Shows streaming response handling
- Includes transaction management
2. **RAG Application** ([examples/rag-app](examples/rag-app))
- Retrieval-Augmented Generation example
- Shows policy enforcement for PII protection
- Demonstrates context tracking
- Includes secure document retrieval
3. **LangChain Chat** ([examples/langchain-chat](examples/langchain-chat))
- Integration with LangChain framework
- Shows conversation chain tracking
- Token tracking across abstractions
4. **WatsonX Generation** ([examples/watsonx-generate](examples/watsonx-generate))
- Integration with IBM WatsonX.ai Foundation Models
- Demonstrates model inference with parameters
- Shows token tracking for WatsonX models
- Includes chat and generation examples
- Policy enforcement for enterprise use cases
5. **Ollama Generation** ([examples/ollama-generate](examples/ollama-generate))
- Integration with local Ollama models
- Shows local model deployment monitoring
- Demonstrates both chat and generation modes
- Includes embedding tracking
- Token usage tracking for local models
- Support for multiple model formats
## Deployment
### Prerequisites
- Kubernetes cluster with:
- OpenTelemetry Collector
- Open Policy Agent
- Jaeger (optional)
- Prometheus (optional)
### Example Kubernetes Deployment
See the [deploy/k8s](deploy/k8s) directory for complete deployment manifests.
## Core Components
- **Context Manager**: Manages trace context, transactions and session tracking
- **Policy Engine**: Handles policy evaluation and enforcement
- **Token Tracker**: Monitors token usage across providers
- **Patch Manager**: Manages LLM provider SDK instrumentation
- **Tracing Manager**: Handles OpenTelemetry integration
## Token Usage Visualization
The SDK includes sample tools to visualize token usage metrics through Grafana dashboards.
[](https://www.youtube.com/watch?v=IkYBNWHVIXQ)
## Development Status
- ✅ Core Framework
- ✅ OpenAI Integration
- ✅ Basic Policy Engine
- ✅ Token Tracking
- ✅ OpenTelemetry Integration
- ✅ Transaction Management
- ✅ Chat Logging
- ✅ LangChain Support
- 🚧 Additional Provider Support
- 🚧 Advanced Policy Features
- 🚧 UI Components
## License
This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.
Raw data
{
"_id": null,
"home_page": "https://github.com/observicia/observicia",
"name": "observicia",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "llm, observability, monitoring, tracing, opentelemetry, opa",
"author": null,
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/f3/8c/3294c4d5f1695850df41a04321393b19eda10ce4d41779c12de4eedfd1bc/observicia-0.1.13.tar.gz",
"platform": "any",
"description": "# Observicia SDK\n\nObservicia is a Cloud Native observability and policy control SDK for LLM applications. It provides seamless integration with CNCF native observability stack while offering comprehensive token tracking, policy enforcement, and PII protection capabilities.\n\n[](https://observicia.readthedocs.io/en/latest/)\n[](LICENSE)\n[](https://opentelemetry.io/)\n[](https://www.openpolicyagent.org/)\n[](https://badge.fury.io/py/observicia)\n[](https://pypi.python.org/pypi/observicia/)\n\n## Features\n\n- **Token Tracking and Management**\n - Real-time token usage monitoring across providers\n - Stream-aware token counting\n - Token usage retention and cleanup\n - Per-session token tracking\n - Configurable data retention policies\n\n- **LLM Backend Support**\n - OpenAI\n - Chat completions (sync/async)\n - Text completions (sync/async)\n - Embeddings\n - Image generation\n - File operations\n - Streaming support\n - Ollama\n - Local model deployment\n - Chat completions\n - Text generation\n - Embeddings\n - Streaming support\n - WatsonX\n - Foundation models integration\n - Text generation\n - Chat completions\n - Parameter controls\n - Basic scaffolding for:\n - Anthropic\n - LiteLLM\n\n- **Transaction Tracking**\n - Multi-round conversation tracking\n - Transaction lifecycle management\n - Metadata and state tracking\n - Parent-child transaction relationships\n - Transaction performance metrics\n\n- **Chat Logging and Analytics**\n - Structured chat history logging\n - Conversation flow analysis\n - Interaction metrics\n - Policy compliance logging\n - Chat completion tracking\n\n- **Telemetry Storage and Export**\n - SQLite exporter for persistent telemetry storage\n - Structured schema for token usage and metrics\n - Transaction and trace correlation\n - Query-friendly format for analytics\n - Redis exporter with configurable retention\n - Time-based data retention policies\n - Real-time metrics access\n - Distributed telemetry storage\n - OpenTelemetry integration\n - Standard OTLP export support\n - Custom attribute mapping\n - Span context preservation\n\n- **Policy Enforcement**\n - Integration with Open Policy Agent (OPA)\n - Support for multiple policy evaluation levels\n - Risk level assessment (low, medium, high, critical)\n - Custom policy definition support\n - Synchronous and asynchronous policy evaluation\n\n- **Framework Integration**\n - LangChain support\n - Conversation chain monitoring\n - Chain metrics\n - Token usage across abstractions\n\n- **Observability Features**\n - OpenTelemetry integration\n - Span-based tracing for all LLM operations\n - Configurable logging (console, file, OTLP)\n - Mermaid diagram generation from telemetry data\n - Detailed request/response tracing\n - Custom attribute tracking\n\n## Quick Start\n\n1. Install the SDK:\n```bash\npip install observicia\n```\n\n2. Create a configuration file (`observicia_config.yaml`):\n```yaml\nservice_name: my-service\notel_endpoint: http://localhost:4317\nopa_endpoint: http://localhost:8181/\npolicies:\n - name: pii_check\n path: policies/pii\n description: Check for PII in responses\n required_trace_level: enhanced\n risk_level: high\nlogging:\n file: \"app.json\"\n telemetry:\n enabled: true\n format: \"json\"\n redis:\n enabled: true\n host: \"localhost\"\n port: 6379\n db: 0\n key_prefix: \"observicia:telemetry:\"\n retention_hours: 24\n messages:\n enabled: true\n level: \"INFO\"\n chat:\n enabled: true\n level: \"both\"\n file: \"chat.log\"\n```\n\n3. Initialize in your code:\n```python\nfrom observicia import init\nfrom observicia.core.context_manager import ObservabilityContext\n\n# Required - Initialize Observicia\ninit()\n\n# Optional - Set user ID for tracking\nObservabilityContext.set_user_id(\"user123\")\n\n# Optional - Start a conversation transaction\ntransaction_id = ObservabilityContext.start_transaction(\n metadata={\"conversation_type\": \"chat\"}\n)\n\n# Use with OpenAI\nfrom openai import OpenAI\nclient = OpenAI()\nresponse = client.chat.completions.create(\n model=\"gpt-4\",\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n\n# Or use with Ollama\nimport ollama\nresponse = ollama.chat(\n model=\"llama2\",\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n\n# Optional - End the transaction\nObservabilityContext.end_transaction(\n transaction_id,\n metadata={\"resolution\": \"completed\"}\n)\n```\n\n## Architecture\n```mermaid\nflowchart TB\n App[Application] --> SDK[Observicia SDK]\n subgraph LLM Backends\n OpenAI[OpenAI API]\n Ollama[Ollama Local]\n Anthropic[Anthropic API]\n LiteLLM[LiteLLM]\n WatsonX[WatsonX]\n end\n\n SDK --> OpenAI\n SDK --> Ollama\n SDK --> Anthropic\n SDK --> LiteLLM\n SDK --> WatsonX\n\n SDK --> OPA[Open Policy Agent]\n SDK --> OTEL[OpenTelemetry Collector]\n SDK --> SQLite[(SQLite)]\n SDK --> Redis[(Redis)]\n\n OTEL --> Jaeger[Jaeger]\n OTEL --> Prom[Prometheus]\n\n OPA --> PII[PII Detection Service]\n OPA --> Compliance[Prompt Compliance Service]\n\n subgraph Telemetry Storage\n SQLite\n Redis\n end\n\n style OpenAI fill:#85e,color:#fff\n style Ollama fill:#85e,color:#fff\n style WatsonX fill:#85e,color:#fff\n style Anthropic fill:#ccc,color:#666\n style LiteLLM fill:#ccc,color:#666\n```\n\n## Example Applications\n\nThe SDK includes three example applications demonstrating different use cases:\n\n## Example Applications\n\nThe SDK includes the following example applications demonstrating different use cases:\n\n1. **Simple Chat Application** ([examples/simple-chat](examples/simple-chat))\n - Basic chat interface using OpenAI\n - Demonstrates token tracking and tracing\n - Shows streaming response handling\n - Includes transaction management\n\n2. **RAG Application** ([examples/rag-app](examples/rag-app))\n - Retrieval-Augmented Generation example\n - Shows policy enforcement for PII protection\n - Demonstrates context tracking\n - Includes secure document retrieval\n\n3. **LangChain Chat** ([examples/langchain-chat](examples/langchain-chat))\n - Integration with LangChain framework\n - Shows conversation chain tracking\n - Token tracking across abstractions\n\n4. **WatsonX Generation** ([examples/watsonx-generate](examples/watsonx-generate))\n - Integration with IBM WatsonX.ai Foundation Models\n - Demonstrates model inference with parameters\n - Shows token tracking for WatsonX models\n - Includes chat and generation examples\n - Policy enforcement for enterprise use cases\n\n5. **Ollama Generation** ([examples/ollama-generate](examples/ollama-generate))\n - Integration with local Ollama models\n - Shows local model deployment monitoring\n - Demonstrates both chat and generation modes\n - Includes embedding tracking\n - Token usage tracking for local models\n - Support for multiple model formats\n\n## Deployment\n\n### Prerequisites\n\n- Kubernetes cluster with:\n - OpenTelemetry Collector\n - Open Policy Agent\n - Jaeger (optional)\n - Prometheus (optional)\n\n### Example Kubernetes Deployment\n\nSee the [deploy/k8s](deploy/k8s) directory for complete deployment manifests.\n\n## Core Components\n\n- **Context Manager**: Manages trace context, transactions and session tracking\n- **Policy Engine**: Handles policy evaluation and enforcement\n- **Token Tracker**: Monitors token usage across providers\n- **Patch Manager**: Manages LLM provider SDK instrumentation\n- **Tracing Manager**: Handles OpenTelemetry integration\n\n## Token Usage Visualization\n\nThe SDK includes sample tools to visualize token usage metrics through Grafana dashboards.\n\n[](https://www.youtube.com/watch?v=IkYBNWHVIXQ)\n\n\n\n## Development Status\n\n- \u2705 Core Framework\n- \u2705 OpenAI Integration\n- \u2705 Basic Policy Engine\n- \u2705 Token Tracking\n- \u2705 OpenTelemetry Integration\n- \u2705 Transaction Management\n- \u2705 Chat Logging\n- \u2705 LangChain Support\n- \ud83d\udea7 Additional Provider Support\n- \ud83d\udea7 Advanced Policy Features\n- \ud83d\udea7 UI Components\n\n## License\n\nThis project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.\n",
"bugtrack_url": null,
"license": null,
"summary": "Cloud Native Observability SDK for LLM applications",
"version": "0.1.13",
"project_urls": {
"Bug Reports": "https://github.com/observicia/observicia/issues",
"Discussion": "https://github.com/observicia/observicia/discussions",
"Documentation": "https://observicia.readthedocs.io/",
"Homepage": "https://github.com/observicia/observicia",
"Source": "https://github.com/observicia/observicia"
},
"split_keywords": [
"llm",
" observability",
" monitoring",
" tracing",
" opentelemetry",
" opa"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "07b24b3417838b988f01a0454e92abf6b1dfa6a6655eab5ab43f794368edbc22",
"md5": "f26de86c8d978170ff822c5c71e72b1a",
"sha256": "1a0cccb7c5ed242c9ba4a7bbdb7c496eeabd1e9ec794602fa7e1a788ae54eb84"
},
"downloads": -1,
"filename": "observicia-0.1.13-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f26de86c8d978170ff822c5c71e72b1a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 52569,
"upload_time": "2024-12-17T18:58:06",
"upload_time_iso_8601": "2024-12-17T18:58:06.820065Z",
"url": "https://files.pythonhosted.org/packages/07/b2/4b3417838b988f01a0454e92abf6b1dfa6a6655eab5ab43f794368edbc22/observicia-0.1.13-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f38c3294c4d5f1695850df41a04321393b19eda10ce4d41779c12de4eedfd1bc",
"md5": "3f3adabe5937ffde08e513a1e0876401",
"sha256": "5747a2a42fa76962ef0fee35e38826d3bc693acc37d4dc7644553ffa3a0f8b92"
},
"downloads": -1,
"filename": "observicia-0.1.13.tar.gz",
"has_sig": false,
"md5_digest": "3f3adabe5937ffde08e513a1e0876401",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 43629,
"upload_time": "2024-12-17T18:58:09",
"upload_time_iso_8601": "2024-12-17T18:58:09.196744Z",
"url": "https://files.pythonhosted.org/packages/f3/8c/3294c4d5f1695850df41a04321393b19eda10ce4d41779c12de4eedfd1bc/observicia-0.1.13.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-17 18:58:09",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "observicia",
"github_project": "observicia",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "observicia"
}
JSON
