# Context Reference Store
[](https://badge.fury.io/py/context-reference-store)
[](https://www.python.org/downloads/)
[](https://opensource.org/licenses/Apache-2.0)
**Efficient Large Context Window Management for AI Agents and Frameworks**
Context Reference Store is a high-performance Python library designed to solve the challenge of managing large context windows in Agentic AI applications. It provides intelligent caching, compression, and retrieval mechanisms that significantly reduce memory usage and improve response times for AI agents and frameworks.
## Table of Contents
- [Context Reference Store](#context-reference-store)
- [Table of Contents](#table-of-contents)
- [Key Features](#key-features)
- [Core Capabilities](#core-capabilities)
- [Framework Integrations](#framework-integrations)
- [Advanced Features](#advanced-features)
- [Architecture](#architecture)
- [Quick Start](#quick-start)
- [Installation](#installation)
- [Basic Usage](#basic-usage)
- [Async Operations](#async-operations)
- [Multimodal Content](#multimodal-content)
- [Building AI Agents](#building-ai-agents)
- [Simple Agent Example](#simple-agent-example)
- [Multi-Agent System](#multi-agent-system)
- [Agent with Tool Integration](#agent-with-tool-integration)
- [Framework Integration Examples](#framework-integration-examples)
- [Agent Development Kit (ADK) Integration](#agent-development-kit-adk-integration)
- [LangChain Integration](#langchain-integration)
- [LangGraph Integration](#langgraph-integration)
- [LlamaIndex Integration](#llamaindex-integration)
- [Composio Integration](#composio-integration)
- [Performance Benchmarks](#performance-benchmarks)
- [Configuration Options](#configuration-options)
- [Cache Policies](#cache-policies)
- [Compression Settings](#compression-settings)
- [Storage Configuration](#storage-configuration)
- [Monitoring and Analytics](#monitoring-and-analytics)
- [Real-time Dashboard](#real-time-dashboard)
- [Performance Metrics](#performance-metrics)
- [Custom Monitoring](#custom-monitoring)
- [Advanced Features](#advanced-features-1)
- [Semantic Analysis](#semantic-analysis)
- [Token Optimization](#token-optimization)
- [API Reference](#api-reference)
- [Development](#development)
- [Installation for Development](#installation-for-development)
- [Running Tests](#running-tests)
- [Code Quality](#code-quality)
- [Optional Dependencies](#optional-dependencies)
- [Documentation](#documentation)
- [Contributing](#contributing)
- [Quick Contribution Steps](#quick-contribution-steps)
- [License](#license)
- [Acknowledgments](#acknowledgments)
- [Support](#support)
## Key Features
### Core Capabilities
- **Intelligent Context Caching**: LRU, LFU, and TTL-based eviction policies
- **Advanced Compression**: 625x faster serialization with 99.99% storage reduction
- **Async/Await Support**: Non-blocking operations for modern applications
- **Multimodal Content**: Handle text, images, audio, and video efficiently
- **High Performance**: Sub-100ms retrieval times for large contexts
### Framework Integrations
- **🤖 Agent Development Kit (ADK)**: Native support for ADK agent workflows and state management
- **🦜 LangChain**: Seamless integration with chat and retrieval chains
- **🕸️ LangGraph**: Native support for graph-based agent workflows
- **🦙 LlamaIndex**: Vector store and query engine implementations
- **🔧 Composio**: Tool integration with secure authentication
### Advanced Features
- **Performance Monitoring**: Real-time metrics and dashboard
- **Semantic Analysis**: Content similarity and clustering
- **Token Optimization**: Intelligent context window management
- **Persistent Storage**: Disk-based caching for large datasets
## Architecture
The Context Reference Store follows a clean, optimized workflow that transforms large context inputs into efficiently managed references:
The architecture provides:
1. **Large Context Input**: Handles 1M-2M tokens, multimodal content (images, audio, video), and structured data
2. **Smart Optimization**: Multiple processing engines for compression, deduplication, and hashing
3. **Reference Storage**: Centralized store with metadata tracking and multi-tier storage management
4. **Fast Retrieval**: Agent cockpit with framework adapters delivering 625x faster performance
**Key Performance Benefits:**
- **625x Faster** serialization and retrieval
- **49x Memory Reduction** for multi-agent scenarios
- **99.55% Storage Savings** through intelligent compression
- **Zero Quality Loss** with perfect content preservation
## Quick Start
### Installation
```bash
# Basic installation
pip install context-reference-store
# With framework integrations
pip install context-reference-store[adk,langchain,langgraph,llamaindex]
# Full installation with all features
pip install context-reference-store[full]
```
### Basic Usage
```python
from context_store import ContextReferenceStore
# Initialize the store
store = ContextReferenceStore(cache_size=100)
# Store context content
context_id = store.store("Your long context content here...")
# Retrieve when needed
content = store.retrieve(context_id)
# Get performance statistics
stats = store.get_cache_stats()
print(f"Hit rate: {stats['hit_rate']:.2%}")
```
### Async Operations
```python
from context_store import AsyncContextReferenceStore
async def main():
async with AsyncContextReferenceStore() as store:
# Store multiple contexts concurrently
context_ids = await store.batch_store_async([
"Context 1", "Context 2", "Context 3"
])
# Retrieve all at once
contents = await store.batch_retrieve_async(context_ids)
```
### Multimodal Content
```python
from context_store import MultimodalContent, MultimodalPart
# Create multimodal content
text_part = MultimodalPart.from_text("Describe this image:")
image_part = MultimodalPart.from_file("path/to/image.jpg")
content = MultimodalContent(parts=[text_part, image_part])
# Store and retrieve
context_id = store.store_multimodal_content(content)
retrieved = store.retrieve_multimodal_content(context_id)
```
## Building AI Agents
### Simple Agent Example
```python
from context_store import ContextReferenceStore
from context_store.adapters import ADKAdapter
class SimpleAgent:
def __init__(self):
self.store = ContextReferenceStore(cache_size=1000)
self.adk_adapter = ADKAdapter(self.store)
self.conversation_history = []
def process_message(self, user_message: str) -> str:
# Store user message in context
user_context_id = self.store.store({
"type": "user_message",
"content": user_message,
"timestamp": time.time()
})
# Retrieve relevant conversation history
context = self.adk_adapter.get_conversation_context(
limit=10,
include_multimodal=True
)
# Process with your LLM
response = self.generate_response(context, user_message)
# Store response
response_context_id = self.store.store({
"type": "agent_response",
"content": response,
"timestamp": time.time()
})
return response
def generate_response(self, context, message):
# Your LLM processing logic here
return f"Processed: {message}"
# Usage
agent = SimpleAgent()
response = agent.process_message("Hello, how can you help me?")
```
### Multi-Agent System
```python
from context_store import ContextReferenceStore
from context_store.adapters import ADKAdapter
class MultiAgentSystem:
def __init__(self):
self.shared_store = ContextReferenceStore(
cache_size=5000,
use_compression=True
)
self.agents = {}
self.coordinator = AgentCoordinator(self.shared_store)
def add_agent(self, agent_id: str, agent_type: str):
"""Add an agent to the system"""
self.agents[agent_id] = {
"type": agent_type,
"adapter": ADKAdapter(self.shared_store),
"state": {},
"tools": []
}
def route_message(self, message: str, target_agent: str = None):
"""Route message to appropriate agent"""
if target_agent:
return self.process_with_agent(message, target_agent)
# Use coordinator to determine best agent
agent_id = self.coordinator.select_agent(message, self.agents.keys())
return self.process_with_agent(message, agent_id)
def process_with_agent(self, message: str, agent_id: str):
"""Process message with specific agent"""
agent = self.agents[agent_id]
adapter = agent["adapter"]
# Get agent-specific context
context = adapter.get_agent_context(
agent_id=agent_id,
message_count=20,
include_shared_memory=True
)
# Process and update shared context
response = self.generate_agent_response(message, context, agent)
# Store interaction in shared memory
interaction_id = self.shared_store.store({
"agent_id": agent_id,
"user_message": message,
"agent_response": response,
"timestamp": time.time(),
"context_used": len(context)
})
return response
# Usage
system = MultiAgentSystem()
system.add_agent("researcher", "research_agent")
system.add_agent("writer", "content_agent")
system.add_agent("analyst", "data_agent")
response = system.route_message("Research the latest AI trends")
```
### Agent with Tool Integration
```python
from context_store import ContextReferenceStore
from context_store.adapters import ADKAdapter, ComposioAdapter
class ToolIntegratedAgent:
def __init__(self):
self.store = ContextReferenceStore()
self.adk_adapter = ADKAdapter(self.store)
self.composio_adapter = ComposioAdapter(self.store)
# Initialize tools
self.available_tools = {
"search": self.web_search,
"calculate": self.calculate,
"send_email": self.send_email,
"file_operations": self.file_operations
}
def process_with_tools(self, user_message: str):
"""Process message and use tools as needed"""
# Analyze message to determine needed tools
required_tools = self.analyze_tool_requirements(user_message)
# Store initial context
context_id = self.store.store({
"user_message": user_message,
"required_tools": required_tools,
"status": "processing"
})
# Execute tools and gather results
tool_results = {}
for tool_name in required_tools:
if tool_name in self.available_tools:
try:
result = self.available_tools[tool_name](user_message)
tool_results[tool_name] = result
# Store tool result in context
self.store.store({
"context_id": context_id,
"tool": tool_name,
"result": result,
"timestamp": time.time()
})
except Exception as e:
tool_results[tool_name] = f"Error: {str(e)}"
# Generate final response using tool results
final_response = self.generate_final_response(
user_message,
tool_results,
context_id
)
return final_response
def web_search(self, query: str):
"""Web search using Composio integration"""
return self.composio_adapter.execute_tool(
app="googlesearch",
action="search",
params={"query": query, "num_results": 5}
)
def calculate(self, expression: str):
"""Mathematical calculations"""
# Safe calculation logic
import ast
import operator
# Simplified calculator - extend as needed
operators = {
ast.Add: operator.add,
ast.Sub: operator.sub,
ast.Mult: operator.mul,
ast.Div: operator.truediv,
ast.USub: operator.neg,
}
try:
tree = ast.parse(expression, mode='eval')
result = self._eval_node(tree.body, operators)
return {"result": result, "expression": expression}
except Exception as e:
return {"error": str(e), "expression": expression}
# Usage
agent = ToolIntegratedAgent()
response = agent.process_with_tools(
"Search for the latest Python releases and calculate the time difference"
)
```
## Framework Integration Examples
### Agent Development Kit (ADK) Integration
```python
from context_store.adapters import ADKAdapter
from adk import Agent, Workflow
# Create ADK-integrated agent
class ADKContextAgent(Agent):
def __init__(self, name: str):
super().__init__(name)
self.context_adapter = ADKAdapter()
def setup(self):
# Initialize context store for this agent
self.context_store = self.context_adapter.create_agent_store(
agent_id=self.name,
cache_size=1000,
use_compression=True
)
def process_step(self, input_data):
# Store step context
step_context_id = self.context_store.store({
"step": self.current_step,
"input": input_data,
"agent_id": self.name,
"timestamp": time.time()
})
# Get relevant historical context
context = self.context_adapter.get_step_context(
agent_id=self.name,
step_type=self.current_step,
limit=5
)
# Process with context
result = self.execute_with_context(input_data, context)
# Store result
self.context_store.store({
"step_context_id": step_context_id,
"result": result,
"success": True
})
return result
# Workflow with context management
workflow = Workflow("data_processing")
workflow.add_agent(ADKContextAgent("preprocessor"))
workflow.add_agent(ADKContextAgent("analyzer"))
workflow.add_agent(ADKContextAgent("reporter"))
# Context is automatically shared between agents
workflow.run(input_data="large_dataset.csv")
```
**[Complete ADK Integration Guide →](docs/integrations/adk.md)**
### LangChain Integration
```python
from context_store.adapters import LangChainAdapter
from langchain.schema import HumanMessage, AIMessage
from langchain.memory import ConversationBufferMemory
adapter = LangChainAdapter()
# Enhanced conversation memory
class ContextAwareMemory(ConversationBufferMemory):
def __init__(self, context_adapter: LangChainAdapter):
super().__init__()
self.context_adapter = context_adapter
def save_context(self, inputs, outputs):
# Save to both LangChain memory and Context Store
super().save_context(inputs, outputs)
# Store in context store for advanced retrieval
self.context_adapter.store_conversation_turn(
inputs=inputs,
outputs=outputs,
session_id=getattr(self, 'session_id', 'default')
)
# Usage with chains
memory = ContextAwareMemory(adapter)
conversation_chain = ConversationChain(
llm=your_llm,
memory=memory
)
# Store conversation with metadata
messages = [
HumanMessage(content="What's the weather like?"),
AIMessage(content="I can help you check the weather. What's your location?")
]
session_id = adapter.store_messages(
messages,
session_id="weather_chat",
metadata={"topic": "weather", "user_intent": "information"}
)
# Retrieve with semantic search
similar_conversations = adapter.find_similar_conversations(
query="weather information",
limit=3
)
```
**[Complete LangChain Integration Guide →](docs/integrations/langchain.md)**
### LangGraph Integration
```python
from context_store.adapters import LangGraphAdapter
from langgraph import StateGraph, START, END
adapter = LangGraphAdapter()
# Define state with context integration
class AgentState(TypedDict):
messages: list
context_id: str
step_history: list
def context_aware_node(state: AgentState):
# Store current state
context_id = adapter.store_graph_state(
state=state,
graph_id="analysis_workflow",
node_name="analysis"
)
# Get relevant context from previous executions
historical_context = adapter.get_node_context(
graph_id="analysis_workflow",
node_name="analysis",
limit=5
)
# Process with context
result = process_with_historical_context(state, historical_context)
# Update state with context reference
state["context_id"] = context_id
state["step_history"].append({
"node": "analysis",
"context_id": context_id,
"timestamp": time.time()
})
return state
# Build graph with context integration
graph = StateGraph(AgentState)
graph.add_node("analysis", context_aware_node)
graph.add_edge(START, "analysis")
graph.add_edge("analysis", END)
compiled_graph = graph.compile()
# Run with context persistence
result = compiled_graph.invoke({
"messages": ["Analyze this data"],
"context_id": "",
"step_history": []
})
```
**[Complete LangGraph Integration Guide →](docs/integrations/langgraph.md)**
### LlamaIndex Integration
```python
from context_store.adapters import LlamaIndexAdapter
from llama_index import Document, VectorStoreIndex, ServiceContext
adapter = LlamaIndexAdapter()
# Enhanced document store with context management
class ContextAwareDocumentStore:
def __init__(self):
self.adapter = LlamaIndexAdapter()
self.indexes = {}
def add_documents(self, documents: list[Document], collection: str):
# Store documents with enhanced metadata
doc_contexts = []
for doc in documents:
# Create context entry for each document
context_id = self.adapter.store_document_context(
document=doc,
collection=collection,
metadata={
"added_timestamp": time.time(),
"source": doc.metadata.get("source", "unknown"),
"document_type": doc.metadata.get("type", "text")
}
)
doc_contexts.append(context_id)
# Create or update index
if collection not in self.indexes:
self.indexes[collection] = VectorStoreIndex.from_documents(documents)
else:
for doc in documents:
self.indexes[collection].insert(doc)
return doc_contexts
def query_with_context(self, query: str, collection: str, include_history: bool = True):
# Get query context if requested
query_context = None
if include_history:
query_context = self.adapter.get_query_context(
query=query,
collection=collection,
limit=5
)
# Perform vector search
query_engine = self.indexes[collection].as_query_engine()
response = query_engine.query(query)
# Store query and response
self.adapter.store_query_response(
query=query,
response=str(response),
collection=collection,
context_used=query_context,
source_nodes=[str(node.id_) for node in response.source_nodes]
)
return response
# Usage
doc_store = ContextAwareDocumentStore()
# Add documents with automatic context tracking
documents = [
Document(text="AI research paper content...", metadata={"source": "arxiv", "type": "research"}),
Document(text="Technical documentation...", metadata={"source": "github", "type": "documentation"})
]
doc_store.add_documents(documents, "ai_research")
# Query with context awareness
response = doc_store.query_with_context(
"What are the latest AI research trends?",
"ai_research",
include_history=True
)
```
**[Complete LlamaIndex Integration Guide →](docs/integrations/llamaindex.md)**
### Composio Integration
```python
from context_store.adapters import ComposioAdapter
from composio import ComposioToolSet, App
adapter = ComposioAdapter()
# Context-aware tool execution
class ContextAwareToolAgent:
def __init__(self):
self.composio_adapter = ComposioAdapter()
self.toolset = ComposioToolSet()
def execute_tool_with_context(self, app: str, action: str, params: dict, session_id: str = None):
# Get execution context
execution_context = self.composio_adapter.get_execution_context(
app=app,
action=action,
session_id=session_id
)
# Store execution intent
execution_id = self.composio_adapter.store_execution_intent(
app=app,
action=action,
params=params,
context=execution_context,
session_id=session_id
)
try:
# Execute tool
result = self.toolset.execute_action(
app=app,
action=action,
params=params
)
# Store successful result
self.composio_adapter.store_execution_result(
execution_id=execution_id,
result=result,
status="success"
)
return result
except Exception as e:
# Store error for learning
self.composio_adapter.store_execution_result(
execution_id=execution_id,
result=None,
status="error",
error=str(e)
)
raise
def get_tool_recommendations(self, user_intent: str, session_id: str = None):
"""Get tool recommendations based on context and history"""
return self.composio_adapter.recommend_tools(
intent=user_intent,
session_id=session_id,
limit=5
)
# Usage
tool_agent = ContextAwareToolAgent()
# Execute with context tracking
result = tool_agent.execute_tool_with_context(
app="gmail",
action="send_email",
params={
"to": "recipient@example.com",
"subject": "Context-aware email",
"body": "This email was sent with context awareness"
},
session_id="email_session_1"
)
# Get recommendations based on context
recommendations = tool_agent.get_tool_recommendations(
"I need to schedule a meeting",
session_id="productivity_session"
)
```
**[Complete Composio Integration Guide →](docs/integrations/composio.md)**
## Performance Benchmarks
Our benchmarks show significant improvements over standard approaches:
| Metric | Standard | Context Store | Improvement |
| ------------------- | -------- | ------------- | ----------------- |
| Serialization Speed | 2.5s | 4ms | **625x faster** |
| Memory Usage | 1.2GB | 24MB | **49x reduction** |
| Storage Size | 450MB | 900KB | **99.8% smaller** |
| Retrieval Time | 250ms | 15ms | **16x faster** |
| Agent State Sync | 1.2s | 25ms | **48x faster** |
| Multi-Agent Memory | 2.8GB | 57MB | **49x reduction** |
## Configuration Options
### Cache Policies
```python
from context_store import CacheEvictionPolicy
# LRU (Least Recently Used)
store = ContextReferenceStore(
cache_size=100,
eviction_policy=CacheEvictionPolicy.LRU
)
# LFU (Least Frequently Used)
store = ContextReferenceStore(
eviction_policy=CacheEvictionPolicy.LFU
)
# TTL (Time To Live)
store = ContextReferenceStore(
eviction_policy=CacheEvictionPolicy.TTL,
ttl_seconds=3600 # 1 hour
)
```
### Compression Settings
```python
# Enable compression for better storage efficiency
store = ContextReferenceStore(
use_compression=True,
compression_algorithm="lz4", # or "zstd"
compression_level=3
)
```
### Storage Configuration
```python
# Configure disk storage for large datasets
store = ContextReferenceStore(
use_disk_storage=True,
disk_cache_dir="/path/to/cache",
memory_threshold_mb=500
)
```
## Monitoring and Analytics
### Real-time Dashboard
The Context Reference Store includes a beautiful terminal-based dashboard for real-time monitoring of performance metrics, compression analytics, and system health.
```python
from context_store.monitoring import create_dashboard
# Create and launch interactive dashboard
store = ContextReferenceStore(enable_compression=True)
dashboard = create_dashboard(store)
dashboard.start() # Opens interactive TUI in terminal
```
**Dashboard Features:**
- **Live Performance Metrics**: Real-time cache hit rates, compression ratios, and efficiency multipliers
- **Compression Analytics**: Detailed breakdown of compression algorithms and space savings
- **Cache Management**: Memory usage, eviction policies, and hit rate history
- **Interactive Navigation**: Tabbed interface with keyboard controls (←/→ arrows, Q to quit)
- **Color-coded Alerts**: Visual indicators for performance thresholds and system health
### Performance Metrics
```python
# Get detailed statistics
stats = store.get_detailed_stats()
print(f"""
Performance Metrics:
- Cache Hit Rate: {stats['hit_rate']:.2%}
- Average Retrieval Time: {stats['avg_retrieval_time_ms']}ms
- Memory Usage: {stats['memory_usage_mb']}MB
- Compression Ratio: {stats['compression_ratio']:.2f}x
""")
```
### Custom Monitoring
```python
from context_store.monitoring import PerformanceMonitor
monitor = PerformanceMonitor()
store.add_monitor(monitor)
# Access real-time metrics
print(monitor.get_current_metrics())
```
## Advanced Features
### Semantic Analysis
```python
from context_store.semantic import SemanticAnalyzer
analyzer = SemanticAnalyzer(store)
# Find similar contexts
similar = analyzer.find_similar_contexts(
"query text",
threshold=0.8,
limit=5
)
# Cluster related contexts
clusters = analyzer.cluster_contexts(method="kmeans", n_clusters=5)
```
### Token Optimization
```python
from context_store.optimization import TokenManager
token_manager = TokenManager(store)
# Optimize context for token limits
optimized = token_manager.optimize_context(
context_id,
max_tokens=4000,
strategy="importance_ranking"
)
```
## API Reference
Detailed API documentation is available in the following files:
- [Core API Reference](docs/api/core.md)
- [Adapters API Reference](docs/api/adapters.md)
- [Monitoring API Reference](docs/api/monitoring.md)
- [Utilities API Reference](docs/api/utils.md)
## Development
### Installation for Development
```bash
git clone https://github.com/Adewale-1/Context_reference_store.git
cd Context_reference_store
pip install -e ".[dev]"
```
### Running Tests
```bash
# Run all tests
pytest
# Run with coverage
pytest --cov=context_store
# Run performance benchmarks
pytest -m benchmark
```
### Code Quality
```bash
# Format code
black .
isort .
# Lint code
flake8 context_store/
mypy context_store/
```
## Optional Dependencies
The library supports various optional dependencies for enhanced functionality:
```bash
# Framework integrations
pip install context-reference-store[adk] # Agent Development Kit support
pip install context-reference-store[langchain] # LangChain support
pip install context-reference-store[langgraph] # LangGraph support
pip install context-reference-store[llamaindex] # LlamaIndex support
pip install context-reference-store[composio] # Composio support
# Performance enhancements
pip install context-reference-store[compression] # Advanced compression
pip install context-reference-store[async] # Async optimizations
# Development tools
pip install context-reference-store[dev] # Testing and linting
pip install context-reference-store[docs] # Documentation tools
# Everything included
pip install context-reference-store[full] # All features
```
## Documentation
Comprehensive documentation is available:
- [Getting Started Guide](docs/getting-started.md)
- [Agent Building Tutorial](docs/tutorials/building-agents.md)
- [Framework Integration Guides](docs/integrations/)
- [ADK Integration Guide](docs/integrations/adk.md)
- [LangChain Integration Guide](docs/integrations/langchain.md)
- [LangGraph Integration Guide](docs/integrations/langgraph.md)
- [LlamaIndex Integration Guide](docs/integrations/llamaindex.md)
- [Composio Integration Guide](docs/integrations/composio.md)
- [Performance Optimization Guide](docs/guides/performance.md)
- [Deployment Guide](docs/guides/deployment.md)
- [API Reference](docs/api/)
## Contributing
We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.
### Quick Contribution Steps
1. Fork the repository
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request
## License
This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.
## Acknowledgments
- Built for Google Summer of Code 2025 with Google DeepMind
- Inspired by the need for efficient context management in modern AI applications
- Thanks to the open-source AI community for feedback and contributions
## Support
- **Documentation**: [https://context-reference-store.readthedocs.io/](https://context-reference-store.readthedocs.io/)
- **Issues**: [GitHub Issues](https://github.com/Adewale-1/Context_reference_store/issues)
- **Discussions**: [GitHub Discussions](https://github.com/Adewale-1/Context_reference_store/discussions)
---
**Made with ❤️ for the AI community**
Raw data
{
"_id": null,
"home_page": null,
"name": "context-reference-store",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "Adewale Adenle <waleadenle1@gmail.com>",
"keywords": "ai, agents, context, memory, multimodal, adk, langchain, langgraph, llamaindex, composio, performance",
"author": null,
"author_email": "Adewale Adenle <waleadenle1@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/62/9e/c57dba074cfce3863eb4c00f6b0d217e57b2ded86184cec3231ad09e14d9/context_reference_store-1.0.8.tar.gz",
"platform": null,
"description": "# Context Reference Store\n\n[](https://badge.fury.io/py/context-reference-store)\n[](https://www.python.org/downloads/)\n[](https://opensource.org/licenses/Apache-2.0)\n\n**Efficient Large Context Window Management for AI Agents and Frameworks**\n\nContext Reference Store is a high-performance Python library designed to solve the challenge of managing large context windows in Agentic AI applications. It provides intelligent caching, compression, and retrieval mechanisms that significantly reduce memory usage and improve response times for AI agents and frameworks.\n\n## Table of Contents\n\n- [Context Reference Store](#context-reference-store)\n - [Table of Contents](#table-of-contents)\n - [Key Features](#key-features)\n - [Core Capabilities](#core-capabilities)\n - [Framework Integrations](#framework-integrations)\n - [Advanced Features](#advanced-features)\n - [Architecture](#architecture)\n - [Quick Start](#quick-start)\n - [Installation](#installation)\n - [Basic Usage](#basic-usage)\n - [Async Operations](#async-operations)\n - [Multimodal Content](#multimodal-content)\n - [Building AI Agents](#building-ai-agents)\n - [Simple Agent Example](#simple-agent-example)\n - [Multi-Agent System](#multi-agent-system)\n - [Agent with Tool Integration](#agent-with-tool-integration)\n - [Framework Integration Examples](#framework-integration-examples)\n - [Agent Development Kit (ADK) Integration](#agent-development-kit-adk-integration)\n - [LangChain Integration](#langchain-integration)\n - [LangGraph Integration](#langgraph-integration)\n - [LlamaIndex Integration](#llamaindex-integration)\n - [Composio Integration](#composio-integration)\n - [Performance Benchmarks](#performance-benchmarks)\n - [Configuration Options](#configuration-options)\n - [Cache Policies](#cache-policies)\n - [Compression Settings](#compression-settings)\n - [Storage Configuration](#storage-configuration)\n - [Monitoring and Analytics](#monitoring-and-analytics)\n - [Real-time Dashboard](#real-time-dashboard)\n - [Performance Metrics](#performance-metrics)\n - [Custom Monitoring](#custom-monitoring)\n - [Advanced Features](#advanced-features-1)\n - [Semantic Analysis](#semantic-analysis)\n - [Token Optimization](#token-optimization)\n - [API Reference](#api-reference)\n - [Development](#development)\n - [Installation for Development](#installation-for-development)\n - [Running Tests](#running-tests)\n - [Code Quality](#code-quality)\n - [Optional Dependencies](#optional-dependencies)\n - [Documentation](#documentation)\n - [Contributing](#contributing)\n - [Quick Contribution Steps](#quick-contribution-steps)\n - [License](#license)\n - [Acknowledgments](#acknowledgments)\n - [Support](#support)\n\n## Key Features\n\n### Core Capabilities\n\n- **Intelligent Context Caching**: LRU, LFU, and TTL-based eviction policies\n- **Advanced Compression**: 625x faster serialization with 99.99% storage reduction\n- **Async/Await Support**: Non-blocking operations for modern applications\n- **Multimodal Content**: Handle text, images, audio, and video efficiently\n- **High Performance**: Sub-100ms retrieval times for large contexts\n\n### Framework Integrations\n\n- **\ud83e\udd16 Agent Development Kit (ADK)**: Native support for ADK agent workflows and state management\n- **\ud83e\udd9c LangChain**: Seamless integration with chat and retrieval chains\n- **\ud83d\udd78\ufe0f LangGraph**: Native support for graph-based agent workflows\n- **\ud83e\udd99 LlamaIndex**: Vector store and query engine implementations\n- **\ud83d\udd27 Composio**: Tool integration with secure authentication\n\n### Advanced Features\n\n- **Performance Monitoring**: Real-time metrics and dashboard\n- **Semantic Analysis**: Content similarity and clustering\n- **Token Optimization**: Intelligent context window management\n- **Persistent Storage**: Disk-based caching for large datasets\n\n## Architecture\n\nThe Context Reference Store follows a clean, optimized workflow that transforms large context inputs into efficiently managed references:\n\n\n\nThe architecture provides:\n\n1. **Large Context Input**: Handles 1M-2M tokens, multimodal content (images, audio, video), and structured data\n2. **Smart Optimization**: Multiple processing engines for compression, deduplication, and hashing\n3. **Reference Storage**: Centralized store with metadata tracking and multi-tier storage management\n4. **Fast Retrieval**: Agent cockpit with framework adapters delivering 625x faster performance\n\n**Key Performance Benefits:**\n\n- **625x Faster** serialization and retrieval\n- **49x Memory Reduction** for multi-agent scenarios\n- **99.55% Storage Savings** through intelligent compression\n- **Zero Quality Loss** with perfect content preservation\n\n## Quick Start\n\n### Installation\n\n```bash\n# Basic installation\npip install context-reference-store\n\n# With framework integrations\npip install context-reference-store[adk,langchain,langgraph,llamaindex]\n\n# Full installation with all features\npip install context-reference-store[full]\n```\n\n### Basic Usage\n\n```python\nfrom context_store import ContextReferenceStore\n\n# Initialize the store\nstore = ContextReferenceStore(cache_size=100)\n\n# Store context content\ncontext_id = store.store(\"Your long context content here...\")\n\n# Retrieve when needed\ncontent = store.retrieve(context_id)\n\n# Get performance statistics\nstats = store.get_cache_stats()\nprint(f\"Hit rate: {stats['hit_rate']:.2%}\")\n```\n\n### Async Operations\n\n```python\nfrom context_store import AsyncContextReferenceStore\n\nasync def main():\n async with AsyncContextReferenceStore() as store:\n # Store multiple contexts concurrently\n context_ids = await store.batch_store_async([\n \"Context 1\", \"Context 2\", \"Context 3\"\n ])\n\n # Retrieve all at once\n contents = await store.batch_retrieve_async(context_ids)\n```\n\n### Multimodal Content\n\n```python\nfrom context_store import MultimodalContent, MultimodalPart\n\n# Create multimodal content\ntext_part = MultimodalPart.from_text(\"Describe this image:\")\nimage_part = MultimodalPart.from_file(\"path/to/image.jpg\")\ncontent = MultimodalContent(parts=[text_part, image_part])\n\n# Store and retrieve\ncontext_id = store.store_multimodal_content(content)\nretrieved = store.retrieve_multimodal_content(context_id)\n```\n\n## Building AI Agents\n\n### Simple Agent Example\n\n```python\nfrom context_store import ContextReferenceStore\nfrom context_store.adapters import ADKAdapter\n\nclass SimpleAgent:\n def __init__(self):\n self.store = ContextReferenceStore(cache_size=1000)\n self.adk_adapter = ADKAdapter(self.store)\n self.conversation_history = []\n\n def process_message(self, user_message: str) -> str:\n # Store user message in context\n user_context_id = self.store.store({\n \"type\": \"user_message\",\n \"content\": user_message,\n \"timestamp\": time.time()\n })\n\n # Retrieve relevant conversation history\n context = self.adk_adapter.get_conversation_context(\n limit=10,\n include_multimodal=True\n )\n\n # Process with your LLM\n response = self.generate_response(context, user_message)\n\n # Store response\n response_context_id = self.store.store({\n \"type\": \"agent_response\",\n \"content\": response,\n \"timestamp\": time.time()\n })\n\n return response\n\n def generate_response(self, context, message):\n # Your LLM processing logic here\n return f\"Processed: {message}\"\n\n# Usage\nagent = SimpleAgent()\nresponse = agent.process_message(\"Hello, how can you help me?\")\n```\n\n### Multi-Agent System\n\n```python\nfrom context_store import ContextReferenceStore\nfrom context_store.adapters import ADKAdapter\n\nclass MultiAgentSystem:\n def __init__(self):\n self.shared_store = ContextReferenceStore(\n cache_size=5000,\n use_compression=True\n )\n self.agents = {}\n self.coordinator = AgentCoordinator(self.shared_store)\n\n def add_agent(self, agent_id: str, agent_type: str):\n \"\"\"Add an agent to the system\"\"\"\n self.agents[agent_id] = {\n \"type\": agent_type,\n \"adapter\": ADKAdapter(self.shared_store),\n \"state\": {},\n \"tools\": []\n }\n\n def route_message(self, message: str, target_agent: str = None):\n \"\"\"Route message to appropriate agent\"\"\"\n if target_agent:\n return self.process_with_agent(message, target_agent)\n\n # Use coordinator to determine best agent\n agent_id = self.coordinator.select_agent(message, self.agents.keys())\n return self.process_with_agent(message, agent_id)\n\n def process_with_agent(self, message: str, agent_id: str):\n \"\"\"Process message with specific agent\"\"\"\n agent = self.agents[agent_id]\n adapter = agent[\"adapter\"]\n\n # Get agent-specific context\n context = adapter.get_agent_context(\n agent_id=agent_id,\n message_count=20,\n include_shared_memory=True\n )\n\n # Process and update shared context\n response = self.generate_agent_response(message, context, agent)\n\n # Store interaction in shared memory\n interaction_id = self.shared_store.store({\n \"agent_id\": agent_id,\n \"user_message\": message,\n \"agent_response\": response,\n \"timestamp\": time.time(),\n \"context_used\": len(context)\n })\n\n return response\n\n# Usage\nsystem = MultiAgentSystem()\nsystem.add_agent(\"researcher\", \"research_agent\")\nsystem.add_agent(\"writer\", \"content_agent\")\nsystem.add_agent(\"analyst\", \"data_agent\")\n\nresponse = system.route_message(\"Research the latest AI trends\")\n```\n\n### Agent with Tool Integration\n\n```python\nfrom context_store import ContextReferenceStore\nfrom context_store.adapters import ADKAdapter, ComposioAdapter\n\nclass ToolIntegratedAgent:\n def __init__(self):\n self.store = ContextReferenceStore()\n self.adk_adapter = ADKAdapter(self.store)\n self.composio_adapter = ComposioAdapter(self.store)\n\n # Initialize tools\n self.available_tools = {\n \"search\": self.web_search,\n \"calculate\": self.calculate,\n \"send_email\": self.send_email,\n \"file_operations\": self.file_operations\n }\n\n def process_with_tools(self, user_message: str):\n \"\"\"Process message and use tools as needed\"\"\"\n\n # Analyze message to determine needed tools\n required_tools = self.analyze_tool_requirements(user_message)\n\n # Store initial context\n context_id = self.store.store({\n \"user_message\": user_message,\n \"required_tools\": required_tools,\n \"status\": \"processing\"\n })\n\n # Execute tools and gather results\n tool_results = {}\n for tool_name in required_tools:\n if tool_name in self.available_tools:\n try:\n result = self.available_tools[tool_name](user_message)\n tool_results[tool_name] = result\n\n # Store tool result in context\n self.store.store({\n \"context_id\": context_id,\n \"tool\": tool_name,\n \"result\": result,\n \"timestamp\": time.time()\n })\n except Exception as e:\n tool_results[tool_name] = f\"Error: {str(e)}\"\n\n # Generate final response using tool results\n final_response = self.generate_final_response(\n user_message,\n tool_results,\n context_id\n )\n\n return final_response\n\n def web_search(self, query: str):\n \"\"\"Web search using Composio integration\"\"\"\n return self.composio_adapter.execute_tool(\n app=\"googlesearch\",\n action=\"search\",\n params={\"query\": query, \"num_results\": 5}\n )\n\n def calculate(self, expression: str):\n \"\"\"Mathematical calculations\"\"\"\n # Safe calculation logic\n import ast\n import operator\n\n # Simplified calculator - extend as needed\n operators = {\n ast.Add: operator.add,\n ast.Sub: operator.sub,\n ast.Mult: operator.mul,\n ast.Div: operator.truediv,\n ast.USub: operator.neg,\n }\n\n try:\n tree = ast.parse(expression, mode='eval')\n result = self._eval_node(tree.body, operators)\n return {\"result\": result, \"expression\": expression}\n except Exception as e:\n return {\"error\": str(e), \"expression\": expression}\n\n# Usage\nagent = ToolIntegratedAgent()\nresponse = agent.process_with_tools(\n \"Search for the latest Python releases and calculate the time difference\"\n)\n```\n\n## Framework Integration Examples\n\n### Agent Development Kit (ADK) Integration\n\n```python\nfrom context_store.adapters import ADKAdapter\nfrom adk import Agent, Workflow\n\n# Create ADK-integrated agent\nclass ADKContextAgent(Agent):\n def __init__(self, name: str):\n super().__init__(name)\n self.context_adapter = ADKAdapter()\n\n def setup(self):\n # Initialize context store for this agent\n self.context_store = self.context_adapter.create_agent_store(\n agent_id=self.name,\n cache_size=1000,\n use_compression=True\n )\n\n def process_step(self, input_data):\n # Store step context\n step_context_id = self.context_store.store({\n \"step\": self.current_step,\n \"input\": input_data,\n \"agent_id\": self.name,\n \"timestamp\": time.time()\n })\n\n # Get relevant historical context\n context = self.context_adapter.get_step_context(\n agent_id=self.name,\n step_type=self.current_step,\n limit=5\n )\n\n # Process with context\n result = self.execute_with_context(input_data, context)\n\n # Store result\n self.context_store.store({\n \"step_context_id\": step_context_id,\n \"result\": result,\n \"success\": True\n })\n\n return result\n\n# Workflow with context management\nworkflow = Workflow(\"data_processing\")\nworkflow.add_agent(ADKContextAgent(\"preprocessor\"))\nworkflow.add_agent(ADKContextAgent(\"analyzer\"))\nworkflow.add_agent(ADKContextAgent(\"reporter\"))\n\n# Context is automatically shared between agents\nworkflow.run(input_data=\"large_dataset.csv\")\n```\n\n**[Complete ADK Integration Guide \u2192](docs/integrations/adk.md)**\n\n### LangChain Integration\n\n```python\nfrom context_store.adapters import LangChainAdapter\nfrom langchain.schema import HumanMessage, AIMessage\nfrom langchain.memory import ConversationBufferMemory\n\nadapter = LangChainAdapter()\n\n# Enhanced conversation memory\nclass ContextAwareMemory(ConversationBufferMemory):\n def __init__(self, context_adapter: LangChainAdapter):\n super().__init__()\n self.context_adapter = context_adapter\n\n def save_context(self, inputs, outputs):\n # Save to both LangChain memory and Context Store\n super().save_context(inputs, outputs)\n\n # Store in context store for advanced retrieval\n self.context_adapter.store_conversation_turn(\n inputs=inputs,\n outputs=outputs,\n session_id=getattr(self, 'session_id', 'default')\n )\n\n# Usage with chains\nmemory = ContextAwareMemory(adapter)\nconversation_chain = ConversationChain(\n llm=your_llm,\n memory=memory\n)\n\n# Store conversation with metadata\nmessages = [\n HumanMessage(content=\"What's the weather like?\"),\n AIMessage(content=\"I can help you check the weather. What's your location?\")\n]\nsession_id = adapter.store_messages(\n messages,\n session_id=\"weather_chat\",\n metadata={\"topic\": \"weather\", \"user_intent\": \"information\"}\n)\n\n# Retrieve with semantic search\nsimilar_conversations = adapter.find_similar_conversations(\n query=\"weather information\",\n limit=3\n)\n```\n\n**[Complete LangChain Integration Guide \u2192](docs/integrations/langchain.md)**\n\n### LangGraph Integration\n\n```python\nfrom context_store.adapters import LangGraphAdapter\nfrom langgraph import StateGraph, START, END\n\nadapter = LangGraphAdapter()\n\n# Define state with context integration\nclass AgentState(TypedDict):\n messages: list\n context_id: str\n step_history: list\n\ndef context_aware_node(state: AgentState):\n # Store current state\n context_id = adapter.store_graph_state(\n state=state,\n graph_id=\"analysis_workflow\",\n node_name=\"analysis\"\n )\n\n # Get relevant context from previous executions\n historical_context = adapter.get_node_context(\n graph_id=\"analysis_workflow\",\n node_name=\"analysis\",\n limit=5\n )\n\n # Process with context\n result = process_with_historical_context(state, historical_context)\n\n # Update state with context reference\n state[\"context_id\"] = context_id\n state[\"step_history\"].append({\n \"node\": \"analysis\",\n \"context_id\": context_id,\n \"timestamp\": time.time()\n })\n\n return state\n\n# Build graph with context integration\ngraph = StateGraph(AgentState)\ngraph.add_node(\"analysis\", context_aware_node)\ngraph.add_edge(START, \"analysis\")\ngraph.add_edge(\"analysis\", END)\n\ncompiled_graph = graph.compile()\n\n# Run with context persistence\nresult = compiled_graph.invoke({\n \"messages\": [\"Analyze this data\"],\n \"context_id\": \"\",\n \"step_history\": []\n})\n```\n\n**[Complete LangGraph Integration Guide \u2192](docs/integrations/langgraph.md)**\n\n### LlamaIndex Integration\n\n```python\nfrom context_store.adapters import LlamaIndexAdapter\nfrom llama_index import Document, VectorStoreIndex, ServiceContext\n\nadapter = LlamaIndexAdapter()\n\n# Enhanced document store with context management\nclass ContextAwareDocumentStore:\n def __init__(self):\n self.adapter = LlamaIndexAdapter()\n self.indexes = {}\n\n def add_documents(self, documents: list[Document], collection: str):\n # Store documents with enhanced metadata\n doc_contexts = []\n for doc in documents:\n # Create context entry for each document\n context_id = self.adapter.store_document_context(\n document=doc,\n collection=collection,\n metadata={\n \"added_timestamp\": time.time(),\n \"source\": doc.metadata.get(\"source\", \"unknown\"),\n \"document_type\": doc.metadata.get(\"type\", \"text\")\n }\n )\n doc_contexts.append(context_id)\n\n # Create or update index\n if collection not in self.indexes:\n self.indexes[collection] = VectorStoreIndex.from_documents(documents)\n else:\n for doc in documents:\n self.indexes[collection].insert(doc)\n\n return doc_contexts\n\n def query_with_context(self, query: str, collection: str, include_history: bool = True):\n # Get query context if requested\n query_context = None\n if include_history:\n query_context = self.adapter.get_query_context(\n query=query,\n collection=collection,\n limit=5\n )\n\n # Perform vector search\n query_engine = self.indexes[collection].as_query_engine()\n response = query_engine.query(query)\n\n # Store query and response\n self.adapter.store_query_response(\n query=query,\n response=str(response),\n collection=collection,\n context_used=query_context,\n source_nodes=[str(node.id_) for node in response.source_nodes]\n )\n\n return response\n\n# Usage\ndoc_store = ContextAwareDocumentStore()\n\n# Add documents with automatic context tracking\ndocuments = [\n Document(text=\"AI research paper content...\", metadata={\"source\": \"arxiv\", \"type\": \"research\"}),\n Document(text=\"Technical documentation...\", metadata={\"source\": \"github\", \"type\": \"documentation\"})\n]\n\ndoc_store.add_documents(documents, \"ai_research\")\n\n# Query with context awareness\nresponse = doc_store.query_with_context(\n \"What are the latest AI research trends?\",\n \"ai_research\",\n include_history=True\n)\n```\n\n**[Complete LlamaIndex Integration Guide \u2192](docs/integrations/llamaindex.md)**\n\n### Composio Integration\n\n```python\nfrom context_store.adapters import ComposioAdapter\nfrom composio import ComposioToolSet, App\n\nadapter = ComposioAdapter()\n\n# Context-aware tool execution\nclass ContextAwareToolAgent:\n def __init__(self):\n self.composio_adapter = ComposioAdapter()\n self.toolset = ComposioToolSet()\n\n def execute_tool_with_context(self, app: str, action: str, params: dict, session_id: str = None):\n # Get execution context\n execution_context = self.composio_adapter.get_execution_context(\n app=app,\n action=action,\n session_id=session_id\n )\n\n # Store execution intent\n execution_id = self.composio_adapter.store_execution_intent(\n app=app,\n action=action,\n params=params,\n context=execution_context,\n session_id=session_id\n )\n\n try:\n # Execute tool\n result = self.toolset.execute_action(\n app=app,\n action=action,\n params=params\n )\n\n # Store successful result\n self.composio_adapter.store_execution_result(\n execution_id=execution_id,\n result=result,\n status=\"success\"\n )\n\n return result\n\n except Exception as e:\n # Store error for learning\n self.composio_adapter.store_execution_result(\n execution_id=execution_id,\n result=None,\n status=\"error\",\n error=str(e)\n )\n raise\n\n def get_tool_recommendations(self, user_intent: str, session_id: str = None):\n \"\"\"Get tool recommendations based on context and history\"\"\"\n return self.composio_adapter.recommend_tools(\n intent=user_intent,\n session_id=session_id,\n limit=5\n )\n\n# Usage\ntool_agent = ContextAwareToolAgent()\n\n# Execute with context tracking\nresult = tool_agent.execute_tool_with_context(\n app=\"gmail\",\n action=\"send_email\",\n params={\n \"to\": \"recipient@example.com\",\n \"subject\": \"Context-aware email\",\n \"body\": \"This email was sent with context awareness\"\n },\n session_id=\"email_session_1\"\n)\n\n# Get recommendations based on context\nrecommendations = tool_agent.get_tool_recommendations(\n \"I need to schedule a meeting\",\n session_id=\"productivity_session\"\n)\n```\n\n**[Complete Composio Integration Guide \u2192](docs/integrations/composio.md)**\n\n## Performance Benchmarks\n\nOur benchmarks show significant improvements over standard approaches:\n\n| Metric | Standard | Context Store | Improvement |\n| ------------------- | -------- | ------------- | ----------------- |\n| Serialization Speed | 2.5s | 4ms | **625x faster** |\n| Memory Usage | 1.2GB | 24MB | **49x reduction** |\n| Storage Size | 450MB | 900KB | **99.8% smaller** |\n| Retrieval Time | 250ms | 15ms | **16x faster** |\n| Agent State Sync | 1.2s | 25ms | **48x faster** |\n| Multi-Agent Memory | 2.8GB | 57MB | **49x reduction** |\n\n## Configuration Options\n\n### Cache Policies\n\n```python\nfrom context_store import CacheEvictionPolicy\n\n# LRU (Least Recently Used)\nstore = ContextReferenceStore(\n cache_size=100,\n eviction_policy=CacheEvictionPolicy.LRU\n)\n\n# LFU (Least Frequently Used)\nstore = ContextReferenceStore(\n eviction_policy=CacheEvictionPolicy.LFU\n)\n\n# TTL (Time To Live)\nstore = ContextReferenceStore(\n eviction_policy=CacheEvictionPolicy.TTL,\n ttl_seconds=3600 # 1 hour\n)\n```\n\n### Compression Settings\n\n```python\n# Enable compression for better storage efficiency\nstore = ContextReferenceStore(\n use_compression=True,\n compression_algorithm=\"lz4\", # or \"zstd\"\n compression_level=3\n)\n```\n\n### Storage Configuration\n\n```python\n# Configure disk storage for large datasets\nstore = ContextReferenceStore(\n use_disk_storage=True,\n disk_cache_dir=\"/path/to/cache\",\n memory_threshold_mb=500\n)\n```\n\n## Monitoring and Analytics\n\n### Real-time Dashboard\n\nThe Context Reference Store includes a beautiful terminal-based dashboard for real-time monitoring of performance metrics, compression analytics, and system health.\n\n\n\n```python\nfrom context_store.monitoring import create_dashboard\n\n# Create and launch interactive dashboard\nstore = ContextReferenceStore(enable_compression=True)\ndashboard = create_dashboard(store)\ndashboard.start() # Opens interactive TUI in terminal\n```\n\n**Dashboard Features:**\n\n- **Live Performance Metrics**: Real-time cache hit rates, compression ratios, and efficiency multipliers\n- **Compression Analytics**: Detailed breakdown of compression algorithms and space savings\n- **Cache Management**: Memory usage, eviction policies, and hit rate history\n- **Interactive Navigation**: Tabbed interface with keyboard controls (\u2190/\u2192 arrows, Q to quit)\n- **Color-coded Alerts**: Visual indicators for performance thresholds and system health\n\n### Performance Metrics\n\n```python\n# Get detailed statistics\nstats = store.get_detailed_stats()\nprint(f\"\"\"\nPerformance Metrics:\n- Cache Hit Rate: {stats['hit_rate']:.2%}\n- Average Retrieval Time: {stats['avg_retrieval_time_ms']}ms\n- Memory Usage: {stats['memory_usage_mb']}MB\n- Compression Ratio: {stats['compression_ratio']:.2f}x\n\"\"\")\n```\n\n### Custom Monitoring\n\n```python\nfrom context_store.monitoring import PerformanceMonitor\n\nmonitor = PerformanceMonitor()\nstore.add_monitor(monitor)\n\n# Access real-time metrics\nprint(monitor.get_current_metrics())\n```\n\n## Advanced Features\n\n### Semantic Analysis\n\n```python\nfrom context_store.semantic import SemanticAnalyzer\n\nanalyzer = SemanticAnalyzer(store)\n\n# Find similar contexts\nsimilar = analyzer.find_similar_contexts(\n \"query text\",\n threshold=0.8,\n limit=5\n)\n\n# Cluster related contexts\nclusters = analyzer.cluster_contexts(method=\"kmeans\", n_clusters=5)\n```\n\n### Token Optimization\n\n```python\nfrom context_store.optimization import TokenManager\n\ntoken_manager = TokenManager(store)\n\n# Optimize context for token limits\noptimized = token_manager.optimize_context(\n context_id,\n max_tokens=4000,\n strategy=\"importance_ranking\"\n)\n```\n\n## API Reference\n\nDetailed API documentation is available in the following files:\n\n- [Core API Reference](docs/api/core.md)\n- [Adapters API Reference](docs/api/adapters.md)\n- [Monitoring API Reference](docs/api/monitoring.md)\n- [Utilities API Reference](docs/api/utils.md)\n\n## Development\n\n### Installation for Development\n\n```bash\ngit clone https://github.com/Adewale-1/Context_reference_store.git\ncd Context_reference_store\npip install -e \".[dev]\"\n```\n\n### Running Tests\n\n```bash\n# Run all tests\npytest\n\n# Run with coverage\npytest --cov=context_store\n\n# Run performance benchmarks\npytest -m benchmark\n```\n\n### Code Quality\n\n```bash\n# Format code\nblack .\nisort .\n\n# Lint code\nflake8 context_store/\nmypy context_store/\n```\n\n## Optional Dependencies\n\nThe library supports various optional dependencies for enhanced functionality:\n\n```bash\n# Framework integrations\npip install context-reference-store[adk] # Agent Development Kit support\npip install context-reference-store[langchain] # LangChain support\npip install context-reference-store[langgraph] # LangGraph support\npip install context-reference-store[llamaindex] # LlamaIndex support\npip install context-reference-store[composio] # Composio support\n\n# Performance enhancements\npip install context-reference-store[compression] # Advanced compression\npip install context-reference-store[async] # Async optimizations\n\n# Development tools\npip install context-reference-store[dev] # Testing and linting\npip install context-reference-store[docs] # Documentation tools\n\n# Everything included\npip install context-reference-store[full] # All features\n```\n\n## Documentation\n\nComprehensive documentation is available:\n\n- [Getting Started Guide](docs/getting-started.md)\n- [Agent Building Tutorial](docs/tutorials/building-agents.md)\n- [Framework Integration Guides](docs/integrations/)\n - [ADK Integration Guide](docs/integrations/adk.md)\n - [LangChain Integration Guide](docs/integrations/langchain.md)\n - [LangGraph Integration Guide](docs/integrations/langgraph.md)\n - [LlamaIndex Integration Guide](docs/integrations/llamaindex.md)\n - [Composio Integration Guide](docs/integrations/composio.md)\n- [Performance Optimization Guide](docs/guides/performance.md)\n- [Deployment Guide](docs/guides/deployment.md)\n- [API Reference](docs/api/)\n\n## Contributing\n\nWe welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.\n\n### Quick Contribution Steps\n\n1. Fork the repository\n2. Create your feature branch (`git checkout -b feature/amazing-feature`)\n3. Commit your changes (`git commit -m 'Add amazing feature'`)\n4. Push to the branch (`git push origin feature/amazing-feature`)\n5. Open a Pull Request\n\n## License\n\nThis project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.\n\n## Acknowledgments\n\n- Built for Google Summer of Code 2025 with Google DeepMind\n- Inspired by the need for efficient context management in modern AI applications\n- Thanks to the open-source AI community for feedback and contributions\n\n## Support\n\n- **Documentation**: [https://context-reference-store.readthedocs.io/](https://context-reference-store.readthedocs.io/)\n- **Issues**: [GitHub Issues](https://github.com/Adewale-1/Context_reference_store/issues)\n- **Discussions**: [GitHub Discussions](https://github.com/Adewale-1/Context_reference_store/discussions)\n\n---\n\n**Made with \u2764\ufe0f for the AI community**\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Efficient Large Context Window Management for AI Agents and Frameworks",
"version": "1.0.8",
"project_urls": {
"Changelog": "https://github.com/Adewale-1/Context_reference_store/blob/main/CHANGELOG.md",
"Documentation": "https://context-reference-store.readthedocs.io/",
"Homepage": "https://github.com/Adewale-1/Context_reference_store",
"Issues": "https://github.com/Adewale-1/Context_reference_store/issues",
"Repository": "https://github.com/Adewale-1/Context_reference_store.git"
},
"split_keywords": [
"ai",
" agents",
" context",
" memory",
" multimodal",
" adk",
" langchain",
" langgraph",
" llamaindex",
" composio",
" performance"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "96568e7ff4dac9a8351014ad2e2c6758dd8b5b3b6964b8ffe10646788fb4682d",
"md5": "9329196e036041154d2a5010e9dc289b",
"sha256": "3b70387b2a5a3ef5d70ade754ba7615381e6e58b0e7cb79c32c82ed798592ed9"
},
"downloads": -1,
"filename": "context_reference_store-1.0.8-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9329196e036041154d2a5010e9dc289b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 102090,
"upload_time": "2025-08-14T19:37:25",
"upload_time_iso_8601": "2025-08-14T19:37:25.830106Z",
"url": "https://files.pythonhosted.org/packages/96/56/8e7ff4dac9a8351014ad2e2c6758dd8b5b3b6964b8ffe10646788fb4682d/context_reference_store-1.0.8-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "629ec57dba074cfce3863eb4c00f6b0d217e57b2ded86184cec3231ad09e14d9",
"md5": "758ab50e5722a615eec5b9d87a2a7ee1",
"sha256": "944de1ce4ec424e7a352337a295a626691d2f29cf5e686045bf82a9d19650f71"
},
"downloads": -1,
"filename": "context_reference_store-1.0.8.tar.gz",
"has_sig": false,
"md5_digest": "758ab50e5722a615eec5b9d87a2a7ee1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 342900,
"upload_time": "2025-08-14T19:37:27",
"upload_time_iso_8601": "2025-08-14T19:37:27.339821Z",
"url": "https://files.pythonhosted.org/packages/62/9e/c57dba074cfce3863eb4c00f6b0d217e57b2ded86184cec3231ad09e14d9/context_reference_store-1.0.8.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-14 19:37:27",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Adewale-1",
"github_project": "Context_reference_store",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "context-reference-store"
}
JSON
