| Name | definable-llms JSON |
| Version |
0.1.4
JSON |
| download |
| home_page | None |
| Summary | Scalable multi-provider LLM library with unified interface for OpenAI, Gemini, and Anthropic |
| upload_time | 2025-10-15 18:38:03 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.10 |
| license | MIT |
| keywords |
ai
anthropic
api
chatbot
gemini
llm
openai
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# LLM Library - Scalable Multi-Provider LLM Library
A production-ready, scalable multi-provider Large Language Model (LLM) library designed for **definable.ai**. This library provides a unified interface for multiple LLM providers including OpenAI, Gemini, and Anthropic, with support for chat completions, image generation, file processing, and advanced capabilities.
## ✨ Features
### Core Capabilities
- **Multi-Provider Support**: OpenAI, Gemini, Anthropic (extensible architecture)
- **Unified Interface**: Consistent API across all providers
- **Session Management**: Persistent conversation sessions with context
- **File Processing**: Support for PDF, DOCX, PPTX, XLSX, images, and text files
- **Streaming Responses**: Real-time streaming for chat completions
- **Rate Limiting**: Built-in token bucket rate limiting
- **Retry Logic**: Exponential backoff with circuit breaker patterns
- **FastAPI Integration**: Production-ready REST API
### Advanced Features
- **Provider Switching**: Change providers mid-conversation
- **Image Processing**: OCR, analysis, and multimodal support
- **Chunking**: Smart text chunking for large documents
- **Error Handling**: Comprehensive exception hierarchy
- **Configuration**: Environment-based configuration management
- **Monitoring**: Structured logging and health checks
## 📦 Installation
```bash
# Clone the repository
git clone <repository-url>
cd llms_lib
# Install dependencies using uv
uv sync
# Or with pip
pip install -e .
```
## ⚙️ Configuration
Create a `.env` file in your project root:
```env
# API Keys
OPENAI_API_KEY=your_openai_api_key_here
GEMINI_API_KEY=your_gemini_api_key_here
ANTHROPIC_API_KEY=your_anthropic_api_key_here
# Provider Settings
DEFAULT_PROVIDER=openai
OPENAI_DEFAULT_MODEL=gpt-4-turbo-preview
OPENAI_TEMPERATURE=0.7
# Rate Limiting
RATE_LIMIT_ENABLED=true
RATE_LIMIT_REQUESTS_PER_MINUTE=60
RATE_LIMIT_TOKENS_PER_MINUTE=90000
# Session Management
SESSION_STORE_TYPE=memory # or redis
SESSION_TTL_SECONDS=3600
REDIS_URL=redis://localhost:6379/0
# API Configuration
API_HOST=0.0.0.0
API_PORT=8000
CORS_ENABLED=true
```
## 🚀 Quick Start
### 1. Basic Chat Completion
```python
import asyncio
from definable.llms import provider_factory
async def basic_chat():
# Get an OpenAI provider
provider = provider_factory.get_provider("openai")
# Create a chat request
from definable.llms.base.types import ChatRequest, Message, MessageRole
messages = [
Message(role=MessageRole.USER, content="Hello, how are you?")
]
request = ChatRequest(messages=messages, model="gpt-4-turbo-preview")
response = await provider.chat(request)
print(response.choices[0].message.content)
# Run the example
asyncio.run(basic_chat())
```
### 2. Session-Based Conversation
```python
import asyncio
from definable.llms import session_manager
async def session_chat():
# Create a new session
session = await session_manager.create_session(
provider="openai",
model="gpt-4-turbo-preview"
)
# Send messages in the session
response1 = await session_manager.chat(
session_id=session.session_id,
message="My name is Alice. Please remember this."
)
print("Assistant:", response1.choices[0].message.content)
response2 = await session_manager.chat(
session_id=session.session_id,
message="What's my name?"
)
print("Assistant:", response2.choices[0].message.content)
asyncio.run(session_chat())
```
### 3. File Processing
```python
import asyncio
from definable.llms import file_processor
async def process_document():
# Process a PDF file
processed_file = await file_processor.process_file(
filename="document.pdf",
file_path="/path/to/document.pdf"
)
print(f"Extracted text length: {len(processed_file.processed_text)}")
print(f"Number of chunks: {len(processed_file.chunks)}")
print(f"Metadata: {processed_file.metadata}")
asyncio.run(process_document())
```
### 4. Streaming Responses
```python
import asyncio
from definable.llms import session_manager
async def streaming_chat():
session = await session_manager.create_session(
provider="openai",
model="gpt-4-turbo-preview"
)
response_stream = await session_manager.chat(
session_id=session.session_id,
message="Tell me a story about AI",
stream=True
)
async for chunk in response_stream:
if chunk.choices and chunk.choices[0].get("delta", {}).get("content"):
print(chunk.choices[0]["delta"]["content"], end="")
asyncio.run(streaming_chat())
```
## 🌐 FastAPI Server
### Running the Server
```python
from definable.llms.api import run_server
# Run with default settings
run_server()
# Or with custom settings
run_server(host="0.0.0.0", port=8080, reload=True)
```
### API Endpoints
The FastAPI server provides the following endpoints:
- **Health**: `GET /api/v1/health` - System health check
- **Providers**: `GET /api/v1/providers` - List available providers
- **Sessions**: `POST /api/v1/sessions` - Create conversation session
- **Chat**: `POST /api/v1/chat` - Send chat messages
- **Files**: `POST /api/v1/files/process` - Process uploaded files
### Example API Usage
```bash
# Create a session
curl -X POST "http://localhost:8000/api/v1/sessions" \
-H "Content-Type: application/json" \
-d '{
"provider": "openai",
"model": "gpt-4-turbo-preview"
}'
# Send a chat message
curl -X POST "http://localhost:8000/api/v1/chat" \
-H "Content-Type: application/json" \
-d '{
"message": "Hello, world!",
"session_id": "your-session-id"
}'
# Process a file
curl -X POST "http://localhost:8000/api/v1/files/process" \
-F "file=@document.pdf"
```
## 🔌 Adding New Providers
The library is designed for easy extension. Here's how to add a new provider:
```python
from definable.llms.base import BaseProvider, ProviderCapabilities
from definable.llms.base.types import ChatRequest, ChatResponse
class CustomProvider(BaseProvider):
def _initialize(self, **kwargs):
# Initialize your provider
pass
def get_capabilities(self) -> ProviderCapabilities:
return ProviderCapabilities(
chat=True,
streaming=False,
# ... other capabilities
)
async def chat(self, request: ChatRequest) -> ChatResponse:
# Implement chat functionality
pass
async def validate_model(self, model: str) -> bool:
# Validate model support
pass
# Register the provider
from definable.llms import provider_factory
provider_factory.register_provider("custom", CustomProvider)
```
## 🏗️ Architecture
The library follows a modular, plugin-based architecture:
```
src/libs/llms/
├── base/ # Base classes and types
├── providers/ # Provider implementations
├── sessions/ # Session management
├── processors/ # File processing
├── utils/ # Utilities (rate limiting, retry, etc.)
├── api/ # FastAPI integration
└── config.py # Configuration management
```
### Key Components
- **BaseProvider**: Abstract base class for all providers
- **SessionManager**: Manages conversation sessions
- **FileProcessor**: Handles document processing
- **RateLimiter**: Token bucket rate limiting
- **RetryStrategy**: Exponential backoff retry logic
## 🧪 Testing
```bash
# Run tests
python -m pytest tests/
# Run with coverage
python -m pytest tests/ --cov=src/libs/llms
# Run specific test category
python -m pytest tests/unit/
python -m pytest tests/integration/
```
## 📊 Monitoring and Observability
The library includes comprehensive logging and monitoring:
```python
# Configure structured logging
from definable.llms.utils import configure_logging
configure_logging(log_level="INFO", json_logs=True)
# Health checks
from definable.llms.api.routes.health import health_check
health_status = await health_check()
```
## 🔒 Security Considerations
- **API Keys**: Stored securely in environment variables
- **Rate Limiting**: Prevents abuse and quota exhaustion
- **Input Validation**: All inputs are validated and sanitized
- **Error Handling**: Sensitive information is not exposed in errors
## 🚢 Production Deployment
### Docker Deployment
```dockerfile
FROM python:3.10-slim
WORKDIR /app
COPY . .
RUN pip install uv && uv sync
EXPOSE 8000
CMD ["python", "-m", "definable.llms.api.main"]
```
### Environment Configuration
For production, ensure you set:
- `DEBUG=false`
- `LOG_LEVEL=INFO`
- Appropriate rate limits
- Redis for session storage
- Proper CORS origins
## 📚 Documentation
- **API Documentation**: Available at `/docs` when running the server
- **Provider Guide**: See `docs/providers.md`
- **Configuration Reference**: See `docs/configuration.md`
- **Deployment Guide**: See `docs/deployment.md`
## 🤝 Contributing
1. Fork the repository
2. Create a feature branch
3. Add tests for new functionality
4. Ensure all tests pass
5. Submit a pull request
## 📄 License
This project is proprietary to definable.ai.
## 💬 Support
For support and questions, please contact the definable.ai team.
---
Built with ❤️ for scalable AI applications.
Raw data
{
"_id": null,
"home_page": null,
"name": "definable-llms",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": "\"Definable.ai Team\" <info@definable.ai>",
"keywords": "ai, anthropic, api, chatbot, gemini, llm, openai",
"author": null,
"author_email": "\"Definable.ai Team\" <info@definable.ai>",
"download_url": "https://files.pythonhosted.org/packages/95/63/6a8a5e263204d4721501fb93a9e5fc8c305dd602c66a3d99f0e8a293e4e1/definable_llms-0.1.4.tar.gz",
"platform": null,
"description": "# LLM Library - Scalable Multi-Provider LLM Library\n\nA production-ready, scalable multi-provider Large Language Model (LLM) library designed for **definable.ai**. This library provides a unified interface for multiple LLM providers including OpenAI, Gemini, and Anthropic, with support for chat completions, image generation, file processing, and advanced capabilities.\n\n## \u2728 Features\n\n### Core Capabilities\n- **Multi-Provider Support**: OpenAI, Gemini, Anthropic (extensible architecture)\n- **Unified Interface**: Consistent API across all providers\n- **Session Management**: Persistent conversation sessions with context\n- **File Processing**: Support for PDF, DOCX, PPTX, XLSX, images, and text files\n- **Streaming Responses**: Real-time streaming for chat completions\n- **Rate Limiting**: Built-in token bucket rate limiting\n- **Retry Logic**: Exponential backoff with circuit breaker patterns\n- **FastAPI Integration**: Production-ready REST API\n\n### Advanced Features\n- **Provider Switching**: Change providers mid-conversation\n- **Image Processing**: OCR, analysis, and multimodal support\n- **Chunking**: Smart text chunking for large documents\n- **Error Handling**: Comprehensive exception hierarchy\n- **Configuration**: Environment-based configuration management\n- **Monitoring**: Structured logging and health checks\n\n## \ud83d\udce6 Installation\n\n```bash\n# Clone the repository\ngit clone <repository-url>\ncd llms_lib\n\n# Install dependencies using uv\nuv sync\n\n# Or with pip\npip install -e .\n```\n\n## \u2699\ufe0f Configuration\n\nCreate a `.env` file in your project root:\n\n```env\n# API Keys\nOPENAI_API_KEY=your_openai_api_key_here\nGEMINI_API_KEY=your_gemini_api_key_here\nANTHROPIC_API_KEY=your_anthropic_api_key_here\n\n# Provider Settings\nDEFAULT_PROVIDER=openai\nOPENAI_DEFAULT_MODEL=gpt-4-turbo-preview\nOPENAI_TEMPERATURE=0.7\n\n# Rate Limiting\nRATE_LIMIT_ENABLED=true\nRATE_LIMIT_REQUESTS_PER_MINUTE=60\nRATE_LIMIT_TOKENS_PER_MINUTE=90000\n\n# Session Management\nSESSION_STORE_TYPE=memory # or redis\nSESSION_TTL_SECONDS=3600\nREDIS_URL=redis://localhost:6379/0\n\n# API Configuration\nAPI_HOST=0.0.0.0\nAPI_PORT=8000\nCORS_ENABLED=true\n```\n\n## \ud83d\ude80 Quick Start\n\n### 1. Basic Chat Completion\n\n```python\nimport asyncio\nfrom definable.llms import provider_factory\n\nasync def basic_chat():\n # Get an OpenAI provider\n provider = provider_factory.get_provider(\"openai\")\n \n # Create a chat request\n from definable.llms.base.types import ChatRequest, Message, MessageRole\n \n messages = [\n Message(role=MessageRole.USER, content=\"Hello, how are you?\")\n ]\n \n request = ChatRequest(messages=messages, model=\"gpt-4-turbo-preview\")\n response = await provider.chat(request)\n \n print(response.choices[0].message.content)\n\n# Run the example\nasyncio.run(basic_chat())\n```\n\n### 2. Session-Based Conversation\n\n```python\nimport asyncio\nfrom definable.llms import session_manager\n\nasync def session_chat():\n # Create a new session\n session = await session_manager.create_session(\n provider=\"openai\",\n model=\"gpt-4-turbo-preview\"\n )\n \n # Send messages in the session\n response1 = await session_manager.chat(\n session_id=session.session_id,\n message=\"My name is Alice. Please remember this.\"\n )\n print(\"Assistant:\", response1.choices[0].message.content)\n \n response2 = await session_manager.chat(\n session_id=session.session_id,\n message=\"What's my name?\"\n )\n print(\"Assistant:\", response2.choices[0].message.content)\n\nasyncio.run(session_chat())\n```\n\n### 3. File Processing\n\n```python\nimport asyncio\nfrom definable.llms import file_processor\n\nasync def process_document():\n # Process a PDF file\n processed_file = await file_processor.process_file(\n filename=\"document.pdf\",\n file_path=\"/path/to/document.pdf\"\n )\n \n print(f\"Extracted text length: {len(processed_file.processed_text)}\")\n print(f\"Number of chunks: {len(processed_file.chunks)}\")\n print(f\"Metadata: {processed_file.metadata}\")\n\nasyncio.run(process_document())\n```\n\n### 4. Streaming Responses\n\n```python\nimport asyncio\nfrom definable.llms import session_manager\n\nasync def streaming_chat():\n session = await session_manager.create_session(\n provider=\"openai\",\n model=\"gpt-4-turbo-preview\"\n )\n \n response_stream = await session_manager.chat(\n session_id=session.session_id,\n message=\"Tell me a story about AI\",\n stream=True\n )\n \n async for chunk in response_stream:\n if chunk.choices and chunk.choices[0].get(\"delta\", {}).get(\"content\"):\n print(chunk.choices[0][\"delta\"][\"content\"], end=\"\")\n\nasyncio.run(streaming_chat())\n```\n\n## \ud83c\udf10 FastAPI Server\n\n### Running the Server\n\n```python\nfrom definable.llms.api import run_server\n\n# Run with default settings\nrun_server()\n\n# Or with custom settings\nrun_server(host=\"0.0.0.0\", port=8080, reload=True)\n```\n\n### API Endpoints\n\nThe FastAPI server provides the following endpoints:\n\n- **Health**: `GET /api/v1/health` - System health check\n- **Providers**: `GET /api/v1/providers` - List available providers\n- **Sessions**: `POST /api/v1/sessions` - Create conversation session\n- **Chat**: `POST /api/v1/chat` - Send chat messages\n- **Files**: `POST /api/v1/files/process` - Process uploaded files\n\n### Example API Usage\n\n```bash\n# Create a session\ncurl -X POST \"http://localhost:8000/api/v1/sessions\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"provider\": \"openai\",\n \"model\": \"gpt-4-turbo-preview\"\n }'\n\n# Send a chat message\ncurl -X POST \"http://localhost:8000/api/v1/chat\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"message\": \"Hello, world!\",\n \"session_id\": \"your-session-id\"\n }'\n\n# Process a file\ncurl -X POST \"http://localhost:8000/api/v1/files/process\" \\\n -F \"file=@document.pdf\"\n```\n\n## \ud83d\udd0c Adding New Providers\n\nThe library is designed for easy extension. Here's how to add a new provider:\n\n```python\nfrom definable.llms.base import BaseProvider, ProviderCapabilities\nfrom definable.llms.base.types import ChatRequest, ChatResponse\n\nclass CustomProvider(BaseProvider):\n def _initialize(self, **kwargs):\n # Initialize your provider\n pass\n \n def get_capabilities(self) -> ProviderCapabilities:\n return ProviderCapabilities(\n chat=True,\n streaming=False,\n # ... other capabilities\n )\n \n async def chat(self, request: ChatRequest) -> ChatResponse:\n # Implement chat functionality\n pass\n \n async def validate_model(self, model: str) -> bool:\n # Validate model support\n pass\n\n# Register the provider\nfrom definable.llms import provider_factory\nprovider_factory.register_provider(\"custom\", CustomProvider)\n```\n\n## \ud83c\udfd7\ufe0f Architecture\n\nThe library follows a modular, plugin-based architecture:\n\n```\nsrc/libs/llms/\n\u251c\u2500\u2500 base/ # Base classes and types\n\u251c\u2500\u2500 providers/ # Provider implementations\n\u251c\u2500\u2500 sessions/ # Session management\n\u251c\u2500\u2500 processors/ # File processing\n\u251c\u2500\u2500 utils/ # Utilities (rate limiting, retry, etc.)\n\u251c\u2500\u2500 api/ # FastAPI integration\n\u2514\u2500\u2500 config.py # Configuration management\n```\n\n### Key Components\n\n- **BaseProvider**: Abstract base class for all providers\n- **SessionManager**: Manages conversation sessions\n- **FileProcessor**: Handles document processing\n- **RateLimiter**: Token bucket rate limiting\n- **RetryStrategy**: Exponential backoff retry logic\n\n## \ud83e\uddea Testing\n\n```bash\n# Run tests\npython -m pytest tests/\n\n# Run with coverage\npython -m pytest tests/ --cov=src/libs/llms\n\n# Run specific test category\npython -m pytest tests/unit/\npython -m pytest tests/integration/\n```\n\n## \ud83d\udcca Monitoring and Observability\n\nThe library includes comprehensive logging and monitoring:\n\n```python\n# Configure structured logging\nfrom definable.llms.utils import configure_logging\nconfigure_logging(log_level=\"INFO\", json_logs=True)\n\n# Health checks\nfrom definable.llms.api.routes.health import health_check\nhealth_status = await health_check()\n```\n\n## \ud83d\udd12 Security Considerations\n\n- **API Keys**: Stored securely in environment variables\n- **Rate Limiting**: Prevents abuse and quota exhaustion\n- **Input Validation**: All inputs are validated and sanitized\n- **Error Handling**: Sensitive information is not exposed in errors\n\n## \ud83d\udea2 Production Deployment\n\n### Docker Deployment\n\n```dockerfile\nFROM python:3.10-slim\n\nWORKDIR /app\nCOPY . .\nRUN pip install uv && uv sync\n\nEXPOSE 8000\nCMD [\"python\", \"-m\", \"definable.llms.api.main\"]\n```\n\n### Environment Configuration\n\nFor production, ensure you set:\n- `DEBUG=false`\n- `LOG_LEVEL=INFO`\n- Appropriate rate limits\n- Redis for session storage\n- Proper CORS origins\n\n## \ud83d\udcda Documentation\n\n- **API Documentation**: Available at `/docs` when running the server\n- **Provider Guide**: See `docs/providers.md`\n- **Configuration Reference**: See `docs/configuration.md`\n- **Deployment Guide**: See `docs/deployment.md`\n\n## \ud83e\udd1d Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Add tests for new functionality\n4. Ensure all tests pass\n5. Submit a pull request\n\n## \ud83d\udcc4 License\n\nThis project is proprietary to definable.ai.\n\n## \ud83d\udcac Support\n\nFor support and questions, please contact the definable.ai team.\n\n---\n\nBuilt with \u2764\ufe0f for scalable AI applications.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Scalable multi-provider LLM library with unified interface for OpenAI, Gemini, and Anthropic",
"version": "0.1.4",
"project_urls": {
"Documentation": "https://github.com/definable-ai/llms-lib#readme",
"Homepage": "https://definable.ai",
"Issues": "https://github.com/definable-ai/llms-lib/issues",
"Repository": "https://github.com/definable-ai/llms-lib"
},
"split_keywords": [
"ai",
" anthropic",
" api",
" chatbot",
" gemini",
" llm",
" openai"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "3e9d06df39ab549675826608400a647912f1ccd519193fec7b7d34d620d92d7a",
"md5": "49fa857b33169b506a372c54c434544c",
"sha256": "c78a3bab17d1b513636230ae712a1cf6d594cb82deaf763f3131cf3b902f7a55"
},
"downloads": -1,
"filename": "definable_llms-0.1.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "49fa857b33169b506a372c54c434544c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 125841,
"upload_time": "2025-10-15T18:38:02",
"upload_time_iso_8601": "2025-10-15T18:38:02.120985Z",
"url": "https://files.pythonhosted.org/packages/3e/9d/06df39ab549675826608400a647912f1ccd519193fec7b7d34d620d92d7a/definable_llms-0.1.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "95636a8a5e263204d4721501fb93a9e5fc8c305dd602c66a3d99f0e8a293e4e1",
"md5": "3ef6dbc6c25e2212635bf898b6f31ed9",
"sha256": "cc4cbf8135862e1ce396a926517ec50271f676d56ccc6cb52568a240c4f2b256"
},
"downloads": -1,
"filename": "definable_llms-0.1.4.tar.gz",
"has_sig": false,
"md5_digest": "3ef6dbc6c25e2212635bf898b6f31ed9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 355491,
"upload_time": "2025-10-15T18:38:03",
"upload_time_iso_8601": "2025-10-15T18:38:03.910931Z",
"url": "https://files.pythonhosted.org/packages/95/63/6a8a5e263204d4721501fb93a9e5fc8c305dd602c66a3d99f0e8a293e4e1/definable_llms-0.1.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-15 18:38:03",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "definable-ai",
"github_project": "llms-lib#readme",
"github_not_found": true,
"lcname": "definable-llms"
}
