| Name | chuk-llm JSON |
| Version |
0.11
JSON |
| download |
| home_page | None |
| Summary | A unified, production-ready Python library for Large Language Model (LLM) providers with real-time streaming, function calling, middleware support, automatic session tracking, dynamic model discovery, and intelligent system prompt generation. |
| upload_time | 2025-08-31 15:00:02 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.11 |
| license | MIT |
| keywords |
llm
ai
openai
anthropic
claude
gpt
gemini
ollama
streaming
async
machine-learning
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# chuk_llm
A unified, production-ready Python library for Large Language Model (LLM) providers with real-time streaming, function calling, middleware support, automatic session tracking, dynamic model discovery, intelligent system prompt generation, and a powerful CLI.
## 🌟 Why ChukLLM?
✅ **🔌 OpenAI-Compatible Everything** - Support for 100+ providers via OpenAI API compatibility
✅ **🎯 Dynamic Provider Registration** - Add new providers at runtime without config files
✅ **🛠️ Advanced Tool Streaming** - Real-time tool calls with incremental JSON parsing
✅ **200+ Auto-Generated Functions** - Every provider & model + discovered models
✅ **🚀 GPT-5 & Reasoning Models** - Full support for GPT-5, O1, O3+ series, Claude 4, and GPT-OSS
✅ **🤖 Smart Sync/Async Detection** - Functions auto-detect context, no more coroutine confusion
✅ **3-7x Performance Boost** - Concurrent requests vs sequential
✅ **Real-time Streaming** - Token-by-token output as it's generated
✅ **Memory Management** - Stateful conversations with context
✅ **Automatic Session Tracking** - Zero-config usage analytics & cost monitoring
✅ **✨ Dynamic Model Discovery** - Automatically detect and generate functions for new models
✅ **🧠 Intelligent System Prompts** - Provider-optimized prompts with tool integration
✅ **🖥️ Enhanced CLI** - Terminal access with streaming, discovery, and convenience functions
✅ **🏢 Enterprise Ready** - Error handling, retries, connection pooling, compliance features
✅ **👨💻 Developer Friendly** - Simple sync for scripts, powerful async for apps
## 🚀 QuickStart
### Installation
```bash
# Install uv (if you haven't already)
curl -LsSf https://astral.sh/uv/install.sh | sh
# Core functionality with session tracking (memory storage)
uv add chuk_llm
# With Redis for persistent sessions
uv add chuk_llm[redis]
# With enhanced CLI experience
uv add chuk_llm[cli]
# Full installation
uv add chuk_llm[all]
# Alternative: use pip if you prefer
pip install chuk_llm[all]
```
### 30-Second Demo
```bash
# Zero installation required - try it instantly with uv!
uvx chuk-llm stream_ollama_gpt_oss "What is Python?"
```
Or in Python:
```python
from chuk_llm import quick_question
# Ultra-simple one-liner
answer = quick_question("What is 2+2?")
print(answer) # "2 + 2 equals 4."
```
## 🤖 NEW: Smart Sync/Async Auto-Detection
ChukLLM's `ask_*` functions now automatically detect whether they're being called from sync or async context:
```python
from chuk_llm import ask_ollama_granite
# ✅ Sync context - works without _sync suffix!
result = ask_ollama_granite("What is Python?")
print(result) # Direct result, not a coroutine!
# ✅ Async context - same function with await
async def my_async_function():
result = await ask_ollama_granite("What is Python?")
return result
# No more confusion! The same function works in both contexts
```
### Before vs After
```python
# ❌ OLD WAY - Confusing coroutine errors
from chuk_llm import ask_ollama_granite
print(ask_ollama_granite("Hello")) # <coroutine object...> 😕
# ❌ OLD WAY - Need to remember _sync suffix
from chuk_llm import ask_ollama_granite_sync
print(ask_ollama_granite_sync("Hello")) # Works but verbose
# ✅ NEW WAY - Just works!
from chuk_llm import ask_ollama_granite
print(ask_ollama_granite("Hello")) # "Hello! How can I help?" 🎉
```
**Note:** Streaming functions (`stream_*`) remain async-only as streaming is inherently asynchronous.
## 🔌 NEW: OpenAI-Compatible Providers
ChukLLM now supports **ANY OpenAI-compatible API** with zero configuration changes. This includes gateways, proxies, and self-hosted services.
### Built-in OpenAI-Compatible Providers
```python
from chuk_llm import ask_sync
# LiteLLM Gateway - Universal LLM Gateway (100+ providers)
response = ask_sync("Hello!", provider="litellm", model="gpt-3.5-turbo")
response = ask_sync("Hello!", provider="litellm", model="claude-3-opus")
response = ask_sync("Hello!", provider="litellm", model="gemini-pro")
# OpenRouter - Unified API for LLMs
response = ask_sync("Hello!", provider="openrouter", model="openai/gpt-4")
response = ask_sync("Hello!", provider="openrouter", model="anthropic/claude-3-opus")
response = ask_sync("Hello!", provider="openrouter", model="meta-llama/llama-3-70b-instruct")
# vLLM - High-performance inference
response = ask_sync("Hello!", provider="vllm", model="meta-llama/Llama-3-70b-hf")
# Together AI - Scalable inference
response = ask_sync("Hello!", provider="togetherai", model="deepseek-ai/deepseek-v3")
response = ask_sync("Hello!", provider="togetherai", model="meta-llama/Llama-3.3-70B-Instruct-Turbo")
# Generic OpenAI-compatible endpoint
response = ask_sync("Hello!", provider="openai_compatible",
base_url="https://your-service.com/v1",
api_key="your-key")
```
### Environment Configuration
```bash
# LiteLLM Gateway
export LITELLM_API_BASE=http://localhost:4000
export LITELLM_API_KEY=your-litellm-key
# OpenRouter
export OPENROUTER_API_KEY=sk-or-...
# vLLM Server
export VLLM_API_BASE=http://localhost:8000/v1
# Together AI
export TOGETHER_API_KEY=...
export TOGETHERAI_API_BASE=https://api.together.xyz/v1
# Generic OpenAI-compatible
export OPENAI_COMPATIBLE_API_BASE=https://your-service.com/v1
export OPENAI_COMPATIBLE_API_KEY=your-key
```
## 🎯 NEW: Dynamic Provider Registration
Register new providers at runtime without modifying configuration files:
### Simple Registration
```python
from chuk_llm import register_provider, register_openai_compatible
# Register a simple OpenAI-compatible provider
register_openai_compatible(
name="my_service",
api_base="https://api.myservice.com/v1",
api_key="sk-abc123",
models=["model-a", "model-b"],
default_model="model-a"
)
# Now use it immediately!
from chuk_llm import ask_sync
response = ask_sync("Hello!", provider="my_service")
```
### Advanced Registration
```python
# Register with environment variables
register_provider(
name="custom_llm",
api_key_env="CUSTOM_API_KEY", # Uses environment variable
api_base_env="CUSTOM_API_BASE", # Uses environment variable
models=["gpt-3.5-turbo", "gpt-4"],
client_class="chuk_llm.llm.providers.openai_client.OpenAILLMClient",
features=["streaming", "tools", "vision", "json_mode"]
)
# Inherit from existing provider
register_provider(
name="my_openai",
inherits_from="openai", # Inherit OpenAI's configuration
api_base="https://proxy.company.com/v1",
api_key="company-key"
)
# Register LocalAI
register_openai_compatible(
name="localai",
api_base="http://localhost:8080/v1",
models=["llama", "mistral", "phi"]
)
# Register FastChat
register_openai_compatible(
name="fastchat",
api_base="http://localhost:8000/v1",
models=["vicuna-13b", "chatglm2-6b"]
)
# Register LM Studio
register_openai_compatible(
name="lmstudio",
api_base="http://localhost:1234/v1",
models=["*"] # Accept any model
)
# Register Anyscale Endpoints
register_openai_compatible(
name="anyscale",
api_base="https://api.endpoints.anyscale.com/v1",
api_key_env="ANYSCALE_API_KEY",
models=["meta-llama/Llama-2-70b-chat-hf", "mistralai/Mixtral-8x7B-Instruct-v0.1"]
)
# Register Ollama with OpenAI compatibility
register_openai_compatible(
name="ollama_openai",
api_base="http://localhost:11434/v1",
models=["llama3.3", "mistral", "phi3"]
)
```
### Managing Dynamic Providers
```python
from chuk_llm import (
update_provider,
unregister_provider,
list_dynamic_providers,
get_provider_config,
provider_exists
)
# Update a provider's configuration
update_provider(
"my_service",
api_base="https://new-endpoint.com/v1",
models=["model-a", "model-b", "model-c"]
)
# Check if a provider exists
if provider_exists("my_service"):
response = ask_sync("Hello", provider="my_service")
# List all dynamically registered providers
dynamic = list_dynamic_providers()
print(f"Dynamic providers: {dynamic}")
# Get provider configuration
config = get_provider_config("my_service")
print(f"API base: {config.api_base}")
print(f"Models: {config.models}")
# Remove a dynamic provider
success = unregister_provider("my_service")
```
## 🖥️ Enhanced CLI with Dynamic Configuration
The CLI now supports dynamic provider configuration on-the-fly:
```bash
# Use any provider with custom endpoints
chuk-llm ask "Hello" --provider openai --base-url https://api.custom.com/v1 --api-key sk-custom-key
chuk-llm ask "Test" --provider ollama --base-url http://remote-server:11434
chuk-llm ask "Query" --provider anthropic --api-key sk-test-key
# Use OpenAI-compatible providers
chuk-llm ask "Hello" --provider litellm --model claude-3-opus
chuk-llm ask "Hello" --provider openrouter --model openai/gpt-4
chuk-llm ask "Hello" --provider vllm --model meta-llama/Llama-3-70b-hf
chuk-llm ask "Hello" --provider togetherai --model deepseek-ai/deepseek-v3
# Quick questions using global aliases
chuk-llm ask_granite "What is Python?"
chuk-llm ask_claude "Explain quantum computing"
chuk-llm ask_gpt "Write a haiku about code"
# Convenience functions for discovered models
chuk-llm ask_ollama_gpt_oss "Think through this step by step"
chuk-llm ask_ollama_mistral_small_latest "Tell me a joke"
chuk-llm stream_ollama_llama3_2 "Write a long explanation"
# Dot notation is automatically converted to underscores
chuk-llm ask_ollama_granite3.3 "What is AI?" # Works with dots!
chuk-llm ask_ollama_llama3.2 "Explain quantum computing"
# JSON responses for structured output
chuk-llm ask "List 3 Python libraries as json " --json --provider openai --model gpt-5
# Set system prompts to control AI personality
chuk-llm ask "What is coding?" -p ollama -m granite3.3:latest -s "You are a pirate. Use pirate speak."
chuk-llm ask "Explain databases" --provider openai --system-prompt "You are a 5-year-old. Use simple words."
chuk-llm ask "Review my code" -p anthropic -s "Be extremely critical and thorough."
# Provider and model management
chuk-llm providers # Show all available providers
chuk-llm models openai # Show models for OpenAI
chuk-llm test openai # Test OpenAI connection
chuk-llm discover ollama # Discover new Ollama models
# Configuration and diagnostics
chuk-llm config # Show current configuration
chuk-llm functions # List all auto-generated functions
chuk-llm functions ollama # Filter functions by provider
chuk-llm help # Comprehensive help
# Use with uv for zero-install usage
uvx chuk-llm ask "What is AI?" --provider openai
uvx chuk-llm ask_ollama_gpt_oss "Reasoning problem"
uvx chuk-llm ask "Test GPT-5" --provider openai --model gpt-5
```
## 📊 Supported Providers
### Major Cloud Providers
| Provider | Models | Special Features |
|----------|---------|------------------|
| **OpenAI** | GPT-5, GPT-4o, GPT-3.5-turbo, O1/O3 series | Reasoning models, function calling, JSON mode |
| **Azure OpenAI** | Enterprise GPT-5, GPT-4, Custom deployments | Private endpoints, compliance, auto-discovery |
| **Anthropic** | Claude 4.1 Opus, Claude 4 Sonnet, Claude 3.7 | Advanced reasoning, 200K context, vision |
| **Google Gemini** | Gemini 2.5 Flash/Pro, 2.0 Flash, 1.5 Pro | Multimodal, 2M context, thinking capabilities |
| **Groq** | Llama 3.3, Mixtral, GPT-OSS | Ultra-fast inference (245+ tokens/sec), 131K context |
| **Perplexity** | Sonar models | Real-time web search, citations |
| **Mistral AI** | Magistral (reasoning), Codestral, Pixtral | European, <think> tags, vision models |
| **DeepSeek** | DeepSeek-Reasoner, DeepSeek-Chat | Complex reasoning (30-60s), OpenAI-compatible |
| **IBM watsonx** | Granite 3.3, Llama 4, Custom models | Enterprise compliance, IBM Cloud integration |
### OpenAI-Compatible Gateways
| Provider | Description | Use Case |
|----------|-------------|----------|
| **LiteLLM** | Universal gateway for 100+ providers | Multi-provider apps |
| **OpenRouter** | Unified API for all major LLMs | Provider switching |
| **vLLM** | High-performance OpenAI-compatible | Self-hosted inference |
| **Together AI** | Scalable inference platform | Production workloads |
| **openai_compatible** | Generic OpenAI API | Any compatible service |
### Local/Self-Hosted
| Provider | Description | Use Case |
|----------|-------------|----------|
| **Ollama** | Local models with discovery | Privacy, offline |
| **LocalAI** | OpenAI-compatible local API | Self-hosted |
| **FastChat** | Multi-model serving | Research |
| **LM Studio** | Desktop model server | Personal use |
| **Text Generation WebUI** | Gradio-based UI | Experimentation |
## 🛠️ Advanced Features
### Real-time Tool Streaming
ChukLLM implements cutting-edge real-time tool call streaming:
```python
import asyncio
from chuk_llm import stream
async def stream_with_tools():
tools = [
{
"type": "function",
"function": {
"name": "calculate",
"description": "Perform mathematical calculations",
"parameters": {
"type": "object",
"properties": {
"expression": {"type": "string"}
}
}
}
}
]
print("🛠️ Streaming with tool calls:")
async for chunk in stream(
"Calculate 15% of 2,847",
provider="openai",
model="gpt-5",
tools=tools
):
if chunk.get("tool_calls"):
for tool_call in chunk["tool_calls"]:
func_name = tool_call["function"]["name"]
args = tool_call["function"]["arguments"]
print(f"🔧 TOOL CALL: {func_name}({args})")
if chunk.get("response"):
print(chunk["response"], end="", flush=True)
asyncio.run(stream_with_tools())
```
### Conversations with Memory
```python
from chuk_llm import conversation
async def chat_example():
# Conversation with automatic session tracking
async with conversation(provider="openai", model="gpt-5") as chat:
await chat.say("My name is Alice")
response = await chat.say("What's my name?")
# Remembers: "Your name is Alice"
# Save conversation
conversation_id = await chat.save()
# Resume later
async with conversation(resume_from=conversation_id) as chat:
response = await chat.say("Do you remember me?")
# Still remembers the context!
asyncio.run(chat_example())
```
### Dynamic Model Discovery
ChukLLM automatically discovers and generates functions for available models:
```python
# Ollama models are discovered automatically
# ollama pull gpt-oss
# ollama pull llama3.2
from chuk_llm import (
ask_ollama_gpt_oss, # Auto-generated with smart detection!
ask_ollama_llama3_2, # Works in both sync and async!
)
# No need for _sync suffix anymore!
response = ask_ollama_gpt_oss("Think through this problem")
# Trigger discovery manually
from chuk_llm.api.providers import trigger_ollama_discovery_and_refresh
new_functions = trigger_ollama_discovery_and_refresh()
print(f"Discovered {len(new_functions)} new functions")
```
#### Provider Discovery Examples
Run the discovery examples to see available models for each provider:
```bash
# Discover OpenAI models
uv run examples/llm_provider_examples/openai_usage_examples.py
# Discover Anthropic models
uv run examples/llm_provider_examples/anthropic_usage_examples.py
# Discover local Ollama models
uv run examples/llm_provider_examples/ollama_usage_examples.py
# Discover Azure OpenAI deployments
uv run examples/llm_provider_examples/azure_usage_examples.py
```
Available provider examples:
- `anthropic_usage_examples.py` - Claude models with Opus/Sonnet/Haiku families
- `azure_usage_examples.py` - Azure OpenAI deployments with auto-discovery
- `deepseek_usage_examples.py` - DeepSeek reasoning and chat models
- `gemini_usage_examples.py` - Google Gemini models with multimodal support
- `groq_usage_examples.py` - Groq's ultra-fast inference models
- `mistral_usage_examples.py` - Mistral AI models including Magistral
- `openai_usage_examples.py` - OpenAI GPT models including O1/O3 reasoning
- `openrouter_usage_examples.py` - OpenRouter's model marketplace
- `perplexity_usage_examples.py` - Perplexity's search-enhanced models
- `watsonx_usage_examples.py` - IBM Watsonx Granite models
### Session Analytics
```python
from chuk_llm import ask, get_session_stats, get_session_history
# All calls are automatically tracked
await ask("What's the capital of France?")
await ask("What's 2+2?")
# Get comprehensive analytics
stats = await get_session_stats()
print(f"📊 Tracked {stats['total_messages']} messages")
print(f"💰 Total cost: ${stats['estimated_cost']:.6f}")
print(f"🔤 Total tokens: {stats['total_tokens']}")
# View complete history
history = await get_session_history()
for msg in history:
print(f"{msg['role']}: {msg['content'][:50]}...")
```
## 🔧 Configuration
### Environment Variables
```bash
# Core API Keys
export OPENAI_API_KEY="your-openai-key"
export AZURE_OPENAI_API_KEY="your-azure-key"
export AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com"
export ANTHROPIC_API_KEY="your-anthropic-key"
export GEMINI_API_KEY="your-google-key"
export GROQ_API_KEY="your-groq-key"
# OpenAI-Compatible Services
export LITELLM_API_BASE="http://localhost:4000"
export OPENROUTER_API_KEY="sk-or-..."
export VLLM_API_BASE="http://localhost:8000/v1"
export TOGETHER_API_KEY="..."
# Custom endpoints (override defaults)
export OPENAI_API_BASE="https://api.openai.com/v1"
export PERPLEXITY_API_BASE="https://api.perplexity.ai"
export OLLAMA_API_BASE="http://localhost:11434"
# Session tracking
export CHUK_LLM_DISABLE_SESSIONS="false"
export SESSION_PROVIDER="redis" # or "memory"
export SESSION_REDIS_URL="redis://localhost:6379/0"
# Discovery settings
export CHUK_LLM_DISCOVERY_ENABLED="true"
export CHUK_LLM_OLLAMA_DISCOVERY="true"
export CHUK_LLM_AUTO_DISCOVER="true"
export CHUK_LLM_DISCOVERY_TIMEOUT="5"
```
### Programmatic Configuration
```python
from chuk_llm import configure, get_current_config
# Simple configuration
configure(
provider="openai",
model="gpt-5",
temperature=0.7
)
# Check current configuration
config = get_current_config()
print(f"Using {config['provider']} with {config['model']}")
# Quick setup helpers
from chuk_llm import quick_setup, switch_provider
# Setup a provider quickly
quick_setup("openai", model="gpt-5")
# Switch between providers
switch_provider("anthropic", model="claude-4-sonnet")
```
## 📦 Installation Options
| Command | Features | Use Case |
|---------|----------|----------|
| `uv add chuk_llm` | Core + Memory sessions | Development |
| `uv add chuk_llm[redis]` | Core + Redis sessions | Production |
| `uv add chuk_llm[cli]` | Core + Enhanced CLI | CLI tools |
| `uv add chuk_llm[all]` | Everything | Full features |
| `pip install chuk_llm[all]` | Everything (alternative) | If not using uv |
## 🤝 Contributing
We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.
## 📄 License
MIT License - see [LICENSE](LICENSE) for details.
## 🚀 What's Next?
- **🔌 More Providers**: Cohere, AI21, Replicate integration
- **🧠 Advanced Reasoning**: Support for O6, O7 series and Claude 5
- **🌐 Multi-Modal**: Enhanced image, audio, and document processing
- **🔧 Tool Orchestration**: Advanced workflows with tool dependencies
- **📊 Analytics Dashboard**: Web UI for session analytics
- **🏢 Enterprise Features**: SSO, audit logs, compliance tools
- **⚡ Performance**: WebSocket streaming, connection pooling optimizations
- **🔐 Security**: End-to-end encryption for sensitive workloads
## 📊 Performance Benchmarks
```python
# Concurrent vs Sequential Performance
import asyncio
import time
from chuk_llm import ask
async def benchmark():
questions = ["Question 1", "Question 2", "Question 3"]
# Sequential
start = time.time()
for q in questions:
await ask(q, provider="openai")
sequential_time = time.time() - start
# Concurrent
start = time.time()
await asyncio.gather(*[ask(q, provider="openai") for q in questions])
concurrent_time = time.time() - start
print(f"Sequential: {sequential_time:.2f}s")
print(f"Concurrent: {concurrent_time:.2f}s")
print(f"Speedup: {sequential_time/concurrent_time:.1f}x")
asyncio.run(benchmark())
```
Typical results:
- Sequential: 6.2s
- Concurrent: 1.8s
- **Speedup: 3.4x faster!**
## 🧪 Testing
```python
# Test provider connectivity
from chuk_llm import test_connection_sync, test_all_providers_sync
# Test single provider
result = test_connection_sync("openai", model="gpt-4o")
print(f"Response time: {result['duration']:.2f}s")
# Test all configured providers
results = test_all_providers_sync()
for provider, result in results.items():
if result["success"]:
print(f"✅ {provider}: {result['duration']:.2f}s")
else:
print(f"❌ {provider}: {result['error']}")
```
## 📚 Examples
### Compare Multiple Providers
```python
from chuk_llm import compare_providers
results = compare_providers(
"Explain quantum computing in one sentence",
["openai", "anthropic", "gemini", "groq"]
)
for provider, response in results.items():
print(f"{provider}: {response}")
```
### Stream with Multiple Providers
```python
import asyncio
from chuk_llm import stream
async def multi_stream():
providers = ["openai", "anthropic", "ollama"]
prompt = "Write a haiku about coding"
async def stream_provider(provider):
print(f"\n{provider.upper()}:")
async for chunk in stream(prompt, provider=provider):
print(chunk, end="", flush=True)
await asyncio.gather(*[stream_provider(p) for p in providers])
asyncio.run(multi_stream())
```
### Use with Pandas DataFrames
```python
import pandas as pd
from chuk_llm import ask_sync
# Process DataFrame with LLM
df = pd.DataFrame({
'product': ['laptop', 'phone', 'tablet'],
'review': ['Great device!', 'Battery issues', 'Perfect for reading']
})
# Add sentiment analysis
df['sentiment'] = df['review'].apply(
lambda x: ask_sync(f"Sentiment of '{x}' (positive/negative/neutral):",
provider="openai", model="gpt-4o-mini")
)
print(df)
```
### Build a Simple Chatbot
```python
from chuk_llm import conversation
import asyncio
async def chatbot():
print("Chatbot ready! Type 'quit' to exit.\n")
async with conversation(provider="openai", model="gpt-4o") as chat:
while True:
user_input = input("You: ")
if user_input.lower() == 'quit':
break
response = await chat.say(user_input)
print(f"Bot: {response}\n")
asyncio.run(chatbot())
```
### Use System Prompts for Custom Personalities
```python
from chuk_llm import ask_sync, ask_ollama_granite
# Make the AI respond with different personalities
pirate_prompt = "You are a pirate captain. Speak in pirate dialect with 'arr' and 'matey'."
response = ask_sync("What is Python?", provider="ollama", model="granite3.3:latest",
system_prompt=pirate_prompt)
print(f"🏴☠️ {response}")
# Works with auto-detection too!
professor_prompt = "You are a university professor. Be academic and thorough."
response = ask_ollama_granite("Explain recursion", system_prompt=professor_prompt)
print(f"👨🏫 {response}")
# CLI examples
# chuk-llm ask "What is AI?" -p ollama -m granite3.3:latest -s "Explain like I'm 5"
# chuk-llm ask "Debug this" --provider openai --system-prompt "You are a senior developer"
```
### Function Calling Example
```python
from chuk_llm import ask_sync
import json
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string"},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
},
"required": ["location"]
}
}
}
]
response = ask_sync(
"What's the weather in Paris?",
provider="openai",
model="gpt-4o",
tools=tools
)
print(response)
# The model will call the get_weather function with location="Paris"
```
## 🛠️ Troubleshooting
### Common Issues
**Issue: "No API key found"**
```bash
# Set your API key
export OPENAI_API_KEY="sk-..."
# Or use a .env file
echo "OPENAI_API_KEY=sk-..." > .env
```
**Issue: "Provider not found"**
```python
# Check available providers
from chuk_llm import list_available_providers
print(list_available_providers())
# Register a new provider
from chuk_llm import register_openai_compatible
register_openai_compatible(
name="my_provider",
api_base="https://api.example.com/v1",
api_key="your-key"
)
```
**Issue: "Connection timeout"**
```python
# Use a different endpoint
from chuk_llm import ask_sync
response = ask_sync(
"Hello",
provider="ollama",
base_url="http://192.168.1.100:11434" # Custom Ollama server
)
```
**Issue: "Model not available"**
```python
# Discover available models
from chuk_llm.api.providers import trigger_ollama_discovery_and_refresh
trigger_ollama_discovery_and_refresh()
# Or check provider models
from chuk_llm import get_provider_config
config = get_provider_config("ollama")
print(config.models)
```
## 📞 Support
- **Documentation**: [docs.chukai.io](https://docs.chuk-llm.dev)
- **Issues**: [GitHub Issues](https://github.com/chuk-llm/chuk-llm/issues)
- **Discussions**: [GitHub Discussions](https://github.com/chuk-llm/chuk-llm/discussions)
- **Email**: support@chukai.io
## 🌟 Why Choose ChukLLM?
1. **Universal Compatibility**: Works with 100+ LLM providers through OpenAI-compatible APIs
2. **Zero Lock-in**: Switch providers with one line of code
3. **Production Ready**: Built-in retries, connection pooling, error handling
4. **Developer Friendly**: Auto-generated functions, great documentation
5. **Cost Tracking**: Automatic session analytics and cost estimation
6. **Enterprise Features**: Azure OpenAI support, compliance, audit logs
7. **Active Development**: Regular updates with new providers and features
8. **Community Driven**: Open source with active community contributions
---
**⭐ Star us on GitHub if ChukLLM helps your AI projects!**
Built with ❤️ by the ChukLLM team
Raw data
{
"_id": null,
"home_page": null,
"name": "chuk-llm",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.11",
"maintainer_email": "Chris Hay <chrishayuk@somejunkmailbox.com>",
"keywords": "llm, ai, openai, anthropic, claude, gpt, gemini, ollama, streaming, async, machine-learning",
"author": null,
"author_email": "Chris Hay <chrishayuk@somejunkmailbox.com>",
"download_url": "https://files.pythonhosted.org/packages/1a/f7/5bfe6199bc914b728f2349ce61b35f5f1fc8c2491aa2d9d06445ab137142/chuk_llm-0.11.tar.gz",
"platform": null,
"description": "# chuk_llm\n\nA unified, production-ready Python library for Large Language Model (LLM) providers with real-time streaming, function calling, middleware support, automatic session tracking, dynamic model discovery, intelligent system prompt generation, and a powerful CLI.\n\n## \ud83c\udf1f Why ChukLLM?\n\n\u2705 **\ud83d\udd0c OpenAI-Compatible Everything** - Support for 100+ providers via OpenAI API compatibility \n\u2705 **\ud83c\udfaf Dynamic Provider Registration** - Add new providers at runtime without config files \n\u2705 **\ud83d\udee0\ufe0f Advanced Tool Streaming** - Real-time tool calls with incremental JSON parsing \n\u2705 **200+ Auto-Generated Functions** - Every provider & model + discovered models \n\u2705 **\ud83d\ude80 GPT-5 & Reasoning Models** - Full support for GPT-5, O1, O3+ series, Claude 4, and GPT-OSS \n\u2705 **\ud83e\udd16 Smart Sync/Async Detection** - Functions auto-detect context, no more coroutine confusion \n\u2705 **3-7x Performance Boost** - Concurrent requests vs sequential \n\u2705 **Real-time Streaming** - Token-by-token output as it's generated \n\u2705 **Memory Management** - Stateful conversations with context \n\u2705 **Automatic Session Tracking** - Zero-config usage analytics & cost monitoring \n\u2705 **\u2728 Dynamic Model Discovery** - Automatically detect and generate functions for new models \n\u2705 **\ud83e\udde0 Intelligent System Prompts** - Provider-optimized prompts with tool integration \n\u2705 **\ud83d\udda5\ufe0f Enhanced CLI** - Terminal access with streaming, discovery, and convenience functions \n\u2705 **\ud83c\udfe2 Enterprise Ready** - Error handling, retries, connection pooling, compliance features \n\u2705 **\ud83d\udc68\u200d\ud83d\udcbb Developer Friendly** - Simple sync for scripts, powerful async for apps \n\n## \ud83d\ude80 QuickStart\n\n### Installation\n\n```bash\n# Install uv (if you haven't already)\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n\n# Core functionality with session tracking (memory storage)\nuv add chuk_llm\n\n# With Redis for persistent sessions\nuv add chuk_llm[redis]\n\n# With enhanced CLI experience\nuv add chuk_llm[cli]\n\n# Full installation\nuv add chuk_llm[all]\n\n# Alternative: use pip if you prefer\npip install chuk_llm[all]\n```\n\n### 30-Second Demo\n\n```bash\n# Zero installation required - try it instantly with uv!\nuvx chuk-llm stream_ollama_gpt_oss \"What is Python?\"\n```\n\nOr in Python:\n```python\nfrom chuk_llm import quick_question\n\n# Ultra-simple one-liner\nanswer = quick_question(\"What is 2+2?\")\nprint(answer) # \"2 + 2 equals 4.\"\n```\n\n## \ud83e\udd16 NEW: Smart Sync/Async Auto-Detection\n\nChukLLM's `ask_*` functions now automatically detect whether they're being called from sync or async context:\n\n```python\nfrom chuk_llm import ask_ollama_granite\n\n# \u2705 Sync context - works without _sync suffix!\nresult = ask_ollama_granite(\"What is Python?\")\nprint(result) # Direct result, not a coroutine!\n\n# \u2705 Async context - same function with await\nasync def my_async_function():\n result = await ask_ollama_granite(\"What is Python?\")\n return result\n\n# No more confusion! The same function works in both contexts\n```\n\n### Before vs After\n\n```python\n# \u274c OLD WAY - Confusing coroutine errors\nfrom chuk_llm import ask_ollama_granite\nprint(ask_ollama_granite(\"Hello\")) # <coroutine object...> \ud83d\ude15\n\n# \u274c OLD WAY - Need to remember _sync suffix\nfrom chuk_llm import ask_ollama_granite_sync\nprint(ask_ollama_granite_sync(\"Hello\")) # Works but verbose\n\n# \u2705 NEW WAY - Just works!\nfrom chuk_llm import ask_ollama_granite\nprint(ask_ollama_granite(\"Hello\")) # \"Hello! How can I help?\" \ud83c\udf89\n```\n\n**Note:** Streaming functions (`stream_*`) remain async-only as streaming is inherently asynchronous.\n\n## \ud83d\udd0c NEW: OpenAI-Compatible Providers\n\nChukLLM now supports **ANY OpenAI-compatible API** with zero configuration changes. This includes gateways, proxies, and self-hosted services.\n\n### Built-in OpenAI-Compatible Providers\n\n```python\nfrom chuk_llm import ask_sync\n\n# LiteLLM Gateway - Universal LLM Gateway (100+ providers)\nresponse = ask_sync(\"Hello!\", provider=\"litellm\", model=\"gpt-3.5-turbo\")\nresponse = ask_sync(\"Hello!\", provider=\"litellm\", model=\"claude-3-opus\")\nresponse = ask_sync(\"Hello!\", provider=\"litellm\", model=\"gemini-pro\")\n\n# OpenRouter - Unified API for LLMs\nresponse = ask_sync(\"Hello!\", provider=\"openrouter\", model=\"openai/gpt-4\")\nresponse = ask_sync(\"Hello!\", provider=\"openrouter\", model=\"anthropic/claude-3-opus\")\nresponse = ask_sync(\"Hello!\", provider=\"openrouter\", model=\"meta-llama/llama-3-70b-instruct\")\n\n# vLLM - High-performance inference\nresponse = ask_sync(\"Hello!\", provider=\"vllm\", model=\"meta-llama/Llama-3-70b-hf\")\n\n# Together AI - Scalable inference\nresponse = ask_sync(\"Hello!\", provider=\"togetherai\", model=\"deepseek-ai/deepseek-v3\")\nresponse = ask_sync(\"Hello!\", provider=\"togetherai\", model=\"meta-llama/Llama-3.3-70B-Instruct-Turbo\")\n\n# Generic OpenAI-compatible endpoint\nresponse = ask_sync(\"Hello!\", provider=\"openai_compatible\", \n base_url=\"https://your-service.com/v1\",\n api_key=\"your-key\")\n```\n\n### Environment Configuration\n\n```bash\n# LiteLLM Gateway\nexport LITELLM_API_BASE=http://localhost:4000\nexport LITELLM_API_KEY=your-litellm-key\n\n# OpenRouter\nexport OPENROUTER_API_KEY=sk-or-...\n\n# vLLM Server\nexport VLLM_API_BASE=http://localhost:8000/v1\n\n# Together AI\nexport TOGETHER_API_KEY=...\nexport TOGETHERAI_API_BASE=https://api.together.xyz/v1\n\n# Generic OpenAI-compatible\nexport OPENAI_COMPATIBLE_API_BASE=https://your-service.com/v1\nexport OPENAI_COMPATIBLE_API_KEY=your-key\n```\n\n## \ud83c\udfaf NEW: Dynamic Provider Registration\n\nRegister new providers at runtime without modifying configuration files:\n\n### Simple Registration\n\n```python\nfrom chuk_llm import register_provider, register_openai_compatible\n\n# Register a simple OpenAI-compatible provider\nregister_openai_compatible(\n name=\"my_service\",\n api_base=\"https://api.myservice.com/v1\",\n api_key=\"sk-abc123\",\n models=[\"model-a\", \"model-b\"],\n default_model=\"model-a\"\n)\n\n# Now use it immediately!\nfrom chuk_llm import ask_sync\nresponse = ask_sync(\"Hello!\", provider=\"my_service\")\n```\n\n### Advanced Registration\n\n```python\n# Register with environment variables\nregister_provider(\n name=\"custom_llm\",\n api_key_env=\"CUSTOM_API_KEY\", # Uses environment variable\n api_base_env=\"CUSTOM_API_BASE\", # Uses environment variable\n models=[\"gpt-3.5-turbo\", \"gpt-4\"],\n client_class=\"chuk_llm.llm.providers.openai_client.OpenAILLMClient\",\n features=[\"streaming\", \"tools\", \"vision\", \"json_mode\"]\n)\n\n# Inherit from existing provider\nregister_provider(\n name=\"my_openai\",\n inherits_from=\"openai\", # Inherit OpenAI's configuration\n api_base=\"https://proxy.company.com/v1\",\n api_key=\"company-key\"\n)\n\n# Register LocalAI\nregister_openai_compatible(\n name=\"localai\",\n api_base=\"http://localhost:8080/v1\",\n models=[\"llama\", \"mistral\", \"phi\"]\n)\n\n# Register FastChat\nregister_openai_compatible(\n name=\"fastchat\",\n api_base=\"http://localhost:8000/v1\",\n models=[\"vicuna-13b\", \"chatglm2-6b\"]\n)\n\n# Register LM Studio\nregister_openai_compatible(\n name=\"lmstudio\",\n api_base=\"http://localhost:1234/v1\",\n models=[\"*\"] # Accept any model\n)\n\n# Register Anyscale Endpoints\nregister_openai_compatible(\n name=\"anyscale\",\n api_base=\"https://api.endpoints.anyscale.com/v1\",\n api_key_env=\"ANYSCALE_API_KEY\",\n models=[\"meta-llama/Llama-2-70b-chat-hf\", \"mistralai/Mixtral-8x7B-Instruct-v0.1\"]\n)\n\n# Register Ollama with OpenAI compatibility\nregister_openai_compatible(\n name=\"ollama_openai\",\n api_base=\"http://localhost:11434/v1\",\n models=[\"llama3.3\", \"mistral\", \"phi3\"]\n)\n```\n\n### Managing Dynamic Providers\n\n```python\nfrom chuk_llm import (\n update_provider,\n unregister_provider,\n list_dynamic_providers,\n get_provider_config,\n provider_exists\n)\n\n# Update a provider's configuration\nupdate_provider(\n \"my_service\",\n api_base=\"https://new-endpoint.com/v1\",\n models=[\"model-a\", \"model-b\", \"model-c\"]\n)\n\n# Check if a provider exists\nif provider_exists(\"my_service\"):\n response = ask_sync(\"Hello\", provider=\"my_service\")\n\n# List all dynamically registered providers\ndynamic = list_dynamic_providers()\nprint(f\"Dynamic providers: {dynamic}\")\n\n# Get provider configuration\nconfig = get_provider_config(\"my_service\")\nprint(f\"API base: {config.api_base}\")\nprint(f\"Models: {config.models}\")\n\n# Remove a dynamic provider\nsuccess = unregister_provider(\"my_service\")\n```\n\n## \ud83d\udda5\ufe0f Enhanced CLI with Dynamic Configuration\n\nThe CLI now supports dynamic provider configuration on-the-fly:\n\n```bash\n# Use any provider with custom endpoints\nchuk-llm ask \"Hello\" --provider openai --base-url https://api.custom.com/v1 --api-key sk-custom-key\nchuk-llm ask \"Test\" --provider ollama --base-url http://remote-server:11434\nchuk-llm ask \"Query\" --provider anthropic --api-key sk-test-key\n\n# Use OpenAI-compatible providers\nchuk-llm ask \"Hello\" --provider litellm --model claude-3-opus\nchuk-llm ask \"Hello\" --provider openrouter --model openai/gpt-4\nchuk-llm ask \"Hello\" --provider vllm --model meta-llama/Llama-3-70b-hf\nchuk-llm ask \"Hello\" --provider togetherai --model deepseek-ai/deepseek-v3\n\n# Quick questions using global aliases\nchuk-llm ask_granite \"What is Python?\"\nchuk-llm ask_claude \"Explain quantum computing\"\nchuk-llm ask_gpt \"Write a haiku about code\"\n\n# Convenience functions for discovered models\nchuk-llm ask_ollama_gpt_oss \"Think through this step by step\"\nchuk-llm ask_ollama_mistral_small_latest \"Tell me a joke\"\nchuk-llm stream_ollama_llama3_2 \"Write a long explanation\"\n\n# Dot notation is automatically converted to underscores\nchuk-llm ask_ollama_granite3.3 \"What is AI?\" # Works with dots!\nchuk-llm ask_ollama_llama3.2 \"Explain quantum computing\"\n\n# JSON responses for structured output\nchuk-llm ask \"List 3 Python libraries as json \" --json --provider openai --model gpt-5\n\n# Set system prompts to control AI personality\nchuk-llm ask \"What is coding?\" -p ollama -m granite3.3:latest -s \"You are a pirate. Use pirate speak.\"\nchuk-llm ask \"Explain databases\" --provider openai --system-prompt \"You are a 5-year-old. Use simple words.\"\nchuk-llm ask \"Review my code\" -p anthropic -s \"Be extremely critical and thorough.\"\n\n# Provider and model management\nchuk-llm providers # Show all available providers\nchuk-llm models openai # Show models for OpenAI\nchuk-llm test openai # Test OpenAI connection\nchuk-llm discover ollama # Discover new Ollama models\n\n# Configuration and diagnostics\nchuk-llm config # Show current configuration\nchuk-llm functions # List all auto-generated functions\nchuk-llm functions ollama # Filter functions by provider\nchuk-llm help # Comprehensive help\n\n# Use with uv for zero-install usage\nuvx chuk-llm ask \"What is AI?\" --provider openai\nuvx chuk-llm ask_ollama_gpt_oss \"Reasoning problem\"\nuvx chuk-llm ask \"Test GPT-5\" --provider openai --model gpt-5\n```\n\n## \ud83d\udcca Supported Providers\n\n### Major Cloud Providers\n| Provider | Models | Special Features |\n|----------|---------|------------------|\n| **OpenAI** | GPT-5, GPT-4o, GPT-3.5-turbo, O1/O3 series | Reasoning models, function calling, JSON mode |\n| **Azure OpenAI** | Enterprise GPT-5, GPT-4, Custom deployments | Private endpoints, compliance, auto-discovery |\n| **Anthropic** | Claude 4.1 Opus, Claude 4 Sonnet, Claude 3.7 | Advanced reasoning, 200K context, vision |\n| **Google Gemini** | Gemini 2.5 Flash/Pro, 2.0 Flash, 1.5 Pro | Multimodal, 2M context, thinking capabilities |\n| **Groq** | Llama 3.3, Mixtral, GPT-OSS | Ultra-fast inference (245+ tokens/sec), 131K context |\n| **Perplexity** | Sonar models | Real-time web search, citations |\n| **Mistral AI** | Magistral (reasoning), Codestral, Pixtral | European, <think> tags, vision models |\n| **DeepSeek** | DeepSeek-Reasoner, DeepSeek-Chat | Complex reasoning (30-60s), OpenAI-compatible |\n| **IBM watsonx** | Granite 3.3, Llama 4, Custom models | Enterprise compliance, IBM Cloud integration |\n\n### OpenAI-Compatible Gateways\n| Provider | Description | Use Case |\n|----------|-------------|----------|\n| **LiteLLM** | Universal gateway for 100+ providers | Multi-provider apps |\n| **OpenRouter** | Unified API for all major LLMs | Provider switching |\n| **vLLM** | High-performance OpenAI-compatible | Self-hosted inference |\n| **Together AI** | Scalable inference platform | Production workloads |\n| **openai_compatible** | Generic OpenAI API | Any compatible service |\n\n### Local/Self-Hosted\n| Provider | Description | Use Case |\n|----------|-------------|----------|\n| **Ollama** | Local models with discovery | Privacy, offline |\n| **LocalAI** | OpenAI-compatible local API | Self-hosted |\n| **FastChat** | Multi-model serving | Research |\n| **LM Studio** | Desktop model server | Personal use |\n| **Text Generation WebUI** | Gradio-based UI | Experimentation |\n\n## \ud83d\udee0\ufe0f Advanced Features\n\n### Real-time Tool Streaming\n\nChukLLM implements cutting-edge real-time tool call streaming:\n\n```python\nimport asyncio\nfrom chuk_llm import stream\n\nasync def stream_with_tools():\n tools = [\n {\n \"type\": \"function\",\n \"function\": {\n \"name\": \"calculate\",\n \"description\": \"Perform mathematical calculations\",\n \"parameters\": {\n \"type\": \"object\",\n \"properties\": {\n \"expression\": {\"type\": \"string\"}\n }\n }\n }\n }\n ]\n \n print(\"\ud83d\udee0\ufe0f Streaming with tool calls:\")\n async for chunk in stream(\n \"Calculate 15% of 2,847\",\n provider=\"openai\",\n model=\"gpt-5\",\n tools=tools\n ):\n if chunk.get(\"tool_calls\"):\n for tool_call in chunk[\"tool_calls\"]:\n func_name = tool_call[\"function\"][\"name\"]\n args = tool_call[\"function\"][\"arguments\"]\n print(f\"\ud83d\udd27 TOOL CALL: {func_name}({args})\")\n \n if chunk.get(\"response\"):\n print(chunk[\"response\"], end=\"\", flush=True)\n\nasyncio.run(stream_with_tools())\n```\n\n### Conversations with Memory\n\n```python\nfrom chuk_llm import conversation\n\nasync def chat_example():\n # Conversation with automatic session tracking\n async with conversation(provider=\"openai\", model=\"gpt-5\") as chat:\n await chat.say(\"My name is Alice\")\n response = await chat.say(\"What's my name?\")\n # Remembers: \"Your name is Alice\"\n \n # Save conversation\n conversation_id = await chat.save()\n \n # Resume later\n async with conversation(resume_from=conversation_id) as chat:\n response = await chat.say(\"Do you remember me?\")\n # Still remembers the context!\n\nasyncio.run(chat_example())\n```\n\n### Dynamic Model Discovery\n\nChukLLM automatically discovers and generates functions for available models:\n\n```python\n# Ollama models are discovered automatically\n# ollama pull gpt-oss\n# ollama pull llama3.2\n\nfrom chuk_llm import (\n ask_ollama_gpt_oss, # Auto-generated with smart detection!\n ask_ollama_llama3_2, # Works in both sync and async!\n)\n\n# No need for _sync suffix anymore!\nresponse = ask_ollama_gpt_oss(\"Think through this problem\")\n\n# Trigger discovery manually\nfrom chuk_llm.api.providers import trigger_ollama_discovery_and_refresh\nnew_functions = trigger_ollama_discovery_and_refresh()\nprint(f\"Discovered {len(new_functions)} new functions\")\n```\n\n#### Provider Discovery Examples\n\nRun the discovery examples to see available models for each provider:\n\n```bash\n# Discover OpenAI models\nuv run examples/llm_provider_examples/openai_usage_examples.py\n\n# Discover Anthropic models \nuv run examples/llm_provider_examples/anthropic_usage_examples.py\n\n# Discover local Ollama models\nuv run examples/llm_provider_examples/ollama_usage_examples.py\n\n# Discover Azure OpenAI deployments\nuv run examples/llm_provider_examples/azure_usage_examples.py\n```\n\nAvailable provider examples:\n- `anthropic_usage_examples.py` - Claude models with Opus/Sonnet/Haiku families\n- `azure_usage_examples.py` - Azure OpenAI deployments with auto-discovery\n- `deepseek_usage_examples.py` - DeepSeek reasoning and chat models\n- `gemini_usage_examples.py` - Google Gemini models with multimodal support\n- `groq_usage_examples.py` - Groq's ultra-fast inference models\n- `mistral_usage_examples.py` - Mistral AI models including Magistral\n- `openai_usage_examples.py` - OpenAI GPT models including O1/O3 reasoning\n- `openrouter_usage_examples.py` - OpenRouter's model marketplace\n- `perplexity_usage_examples.py` - Perplexity's search-enhanced models\n- `watsonx_usage_examples.py` - IBM Watsonx Granite models\n\n### Session Analytics\n\n```python\nfrom chuk_llm import ask, get_session_stats, get_session_history\n\n# All calls are automatically tracked\nawait ask(\"What's the capital of France?\")\nawait ask(\"What's 2+2?\")\n\n# Get comprehensive analytics\nstats = await get_session_stats()\nprint(f\"\ud83d\udcca Tracked {stats['total_messages']} messages\")\nprint(f\"\ud83d\udcb0 Total cost: ${stats['estimated_cost']:.6f}\")\nprint(f\"\ud83d\udd24 Total tokens: {stats['total_tokens']}\")\n\n# View complete history\nhistory = await get_session_history()\nfor msg in history:\n print(f\"{msg['role']}: {msg['content'][:50]}...\")\n```\n\n## \ud83d\udd27 Configuration\n\n### Environment Variables\n\n```bash\n# Core API Keys\nexport OPENAI_API_KEY=\"your-openai-key\"\nexport AZURE_OPENAI_API_KEY=\"your-azure-key\"\nexport AZURE_OPENAI_ENDPOINT=\"https://your-resource.openai.azure.com\"\nexport ANTHROPIC_API_KEY=\"your-anthropic-key\"\nexport GEMINI_API_KEY=\"your-google-key\"\nexport GROQ_API_KEY=\"your-groq-key\"\n\n# OpenAI-Compatible Services\nexport LITELLM_API_BASE=\"http://localhost:4000\"\nexport OPENROUTER_API_KEY=\"sk-or-...\"\nexport VLLM_API_BASE=\"http://localhost:8000/v1\"\nexport TOGETHER_API_KEY=\"...\"\n\n# Custom endpoints (override defaults)\nexport OPENAI_API_BASE=\"https://api.openai.com/v1\"\nexport PERPLEXITY_API_BASE=\"https://api.perplexity.ai\"\nexport OLLAMA_API_BASE=\"http://localhost:11434\"\n\n# Session tracking\nexport CHUK_LLM_DISABLE_SESSIONS=\"false\"\nexport SESSION_PROVIDER=\"redis\" # or \"memory\"\nexport SESSION_REDIS_URL=\"redis://localhost:6379/0\"\n\n# Discovery settings\nexport CHUK_LLM_DISCOVERY_ENABLED=\"true\"\nexport CHUK_LLM_OLLAMA_DISCOVERY=\"true\"\nexport CHUK_LLM_AUTO_DISCOVER=\"true\"\nexport CHUK_LLM_DISCOVERY_TIMEOUT=\"5\"\n```\n\n### Programmatic Configuration\n\n```python\nfrom chuk_llm import configure, get_current_config\n\n# Simple configuration\nconfigure(\n provider=\"openai\",\n model=\"gpt-5\",\n temperature=0.7\n)\n\n# Check current configuration\nconfig = get_current_config()\nprint(f\"Using {config['provider']} with {config['model']}\")\n\n# Quick setup helpers\nfrom chuk_llm import quick_setup, switch_provider\n\n# Setup a provider quickly\nquick_setup(\"openai\", model=\"gpt-5\")\n\n# Switch between providers\nswitch_provider(\"anthropic\", model=\"claude-4-sonnet\")\n```\n\n## \ud83d\udce6 Installation Options\n\n| Command | Features | Use Case |\n|---------|----------|----------|\n| `uv add chuk_llm` | Core + Memory sessions | Development |\n| `uv add chuk_llm[redis]` | Core + Redis sessions | Production |\n| `uv add chuk_llm[cli]` | Core + Enhanced CLI | CLI tools |\n| `uv add chuk_llm[all]` | Everything | Full features |\n| `pip install chuk_llm[all]` | Everything (alternative) | If not using uv |\n\n## \ud83e\udd1d Contributing\n\nWe welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.\n\n## \ud83d\udcc4 License\n\nMIT License - see [LICENSE](LICENSE) for details.\n\n## \ud83d\ude80 What's Next?\n\n- **\ud83d\udd0c More Providers**: Cohere, AI21, Replicate integration\n- **\ud83e\udde0 Advanced Reasoning**: Support for O6, O7 series and Claude 5\n- **\ud83c\udf10 Multi-Modal**: Enhanced image, audio, and document processing\n- **\ud83d\udd27 Tool Orchestration**: Advanced workflows with tool dependencies\n- **\ud83d\udcca Analytics Dashboard**: Web UI for session analytics\n- **\ud83c\udfe2 Enterprise Features**: SSO, audit logs, compliance tools\n- **\u26a1 Performance**: WebSocket streaming, connection pooling optimizations\n- **\ud83d\udd10 Security**: End-to-end encryption for sensitive workloads\n\n## \ud83d\udcca Performance Benchmarks\n\n```python\n# Concurrent vs Sequential Performance\nimport asyncio\nimport time\nfrom chuk_llm import ask\n\nasync def benchmark():\n questions = [\"Question 1\", \"Question 2\", \"Question 3\"]\n \n # Sequential\n start = time.time()\n for q in questions:\n await ask(q, provider=\"openai\")\n sequential_time = time.time() - start\n \n # Concurrent\n start = time.time()\n await asyncio.gather(*[ask(q, provider=\"openai\") for q in questions])\n concurrent_time = time.time() - start\n \n print(f\"Sequential: {sequential_time:.2f}s\")\n print(f\"Concurrent: {concurrent_time:.2f}s\")\n print(f\"Speedup: {sequential_time/concurrent_time:.1f}x\")\n\nasyncio.run(benchmark())\n```\n\nTypical results:\n- Sequential: 6.2s\n- Concurrent: 1.8s \n- **Speedup: 3.4x faster!**\n\n## \ud83e\uddea Testing\n\n```python\n# Test provider connectivity\nfrom chuk_llm import test_connection_sync, test_all_providers_sync\n\n# Test single provider\nresult = test_connection_sync(\"openai\", model=\"gpt-4o\")\nprint(f\"Response time: {result['duration']:.2f}s\")\n\n# Test all configured providers\nresults = test_all_providers_sync()\nfor provider, result in results.items():\n if result[\"success\"]:\n print(f\"\u2705 {provider}: {result['duration']:.2f}s\")\n else:\n print(f\"\u274c {provider}: {result['error']}\")\n```\n\n## \ud83d\udcda Examples\n\n### Compare Multiple Providers\n\n```python\nfrom chuk_llm import compare_providers\n\nresults = compare_providers(\n \"Explain quantum computing in one sentence\",\n [\"openai\", \"anthropic\", \"gemini\", \"groq\"]\n)\n\nfor provider, response in results.items():\n print(f\"{provider}: {response}\")\n```\n\n### Stream with Multiple Providers\n\n```python\nimport asyncio\nfrom chuk_llm import stream\n\nasync def multi_stream():\n providers = [\"openai\", \"anthropic\", \"ollama\"]\n prompt = \"Write a haiku about coding\"\n \n async def stream_provider(provider):\n print(f\"\\n{provider.upper()}:\")\n async for chunk in stream(prompt, provider=provider):\n print(chunk, end=\"\", flush=True)\n \n await asyncio.gather(*[stream_provider(p) for p in providers])\n\nasyncio.run(multi_stream())\n```\n\n### Use with Pandas DataFrames\n\n```python\nimport pandas as pd\nfrom chuk_llm import ask_sync\n\n# Process DataFrame with LLM\ndf = pd.DataFrame({\n 'product': ['laptop', 'phone', 'tablet'],\n 'review': ['Great device!', 'Battery issues', 'Perfect for reading']\n})\n\n# Add sentiment analysis\ndf['sentiment'] = df['review'].apply(\n lambda x: ask_sync(f\"Sentiment of '{x}' (positive/negative/neutral):\", \n provider=\"openai\", model=\"gpt-4o-mini\")\n)\n\nprint(df)\n```\n\n### Build a Simple Chatbot\n\n```python\nfrom chuk_llm import conversation\nimport asyncio\n\nasync def chatbot():\n print(\"Chatbot ready! Type 'quit' to exit.\\n\")\n \n async with conversation(provider=\"openai\", model=\"gpt-4o\") as chat:\n while True:\n user_input = input(\"You: \")\n if user_input.lower() == 'quit':\n break\n \n response = await chat.say(user_input)\n print(f\"Bot: {response}\\n\")\n\nasyncio.run(chatbot())\n```\n\n### Use System Prompts for Custom Personalities\n\n```python\nfrom chuk_llm import ask_sync, ask_ollama_granite\n\n# Make the AI respond with different personalities\npirate_prompt = \"You are a pirate captain. Speak in pirate dialect with 'arr' and 'matey'.\"\nresponse = ask_sync(\"What is Python?\", provider=\"ollama\", model=\"granite3.3:latest\", \n system_prompt=pirate_prompt)\nprint(f\"\ud83c\udff4\u200d\u2620\ufe0f {response}\")\n\n# Works with auto-detection too!\nprofessor_prompt = \"You are a university professor. Be academic and thorough.\"\nresponse = ask_ollama_granite(\"Explain recursion\", system_prompt=professor_prompt)\nprint(f\"\ud83d\udc68\u200d\ud83c\udfeb {response}\")\n\n# CLI examples\n# chuk-llm ask \"What is AI?\" -p ollama -m granite3.3:latest -s \"Explain like I'm 5\"\n# chuk-llm ask \"Debug this\" --provider openai --system-prompt \"You are a senior developer\"\n```\n\n### Function Calling Example\n\n```python\nfrom chuk_llm import ask_sync\nimport json\n\ntools = [\n {\n \"type\": \"function\",\n \"function\": {\n \"name\": \"get_weather\",\n \"description\": \"Get weather for a location\",\n \"parameters\": {\n \"type\": \"object\",\n \"properties\": {\n \"location\": {\"type\": \"string\"},\n \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]}\n },\n \"required\": [\"location\"]\n }\n }\n }\n]\n\nresponse = ask_sync(\n \"What's the weather in Paris?\",\n provider=\"openai\",\n model=\"gpt-4o\",\n tools=tools\n)\n\nprint(response)\n# The model will call the get_weather function with location=\"Paris\"\n```\n\n## \ud83d\udee0\ufe0f Troubleshooting\n\n### Common Issues\n\n**Issue: \"No API key found\"**\n```bash\n# Set your API key\nexport OPENAI_API_KEY=\"sk-...\"\n# Or use a .env file\necho \"OPENAI_API_KEY=sk-...\" > .env\n```\n\n**Issue: \"Provider not found\"**\n```python\n# Check available providers\nfrom chuk_llm import list_available_providers\nprint(list_available_providers())\n\n# Register a new provider\nfrom chuk_llm import register_openai_compatible\nregister_openai_compatible(\n name=\"my_provider\",\n api_base=\"https://api.example.com/v1\",\n api_key=\"your-key\"\n)\n```\n\n**Issue: \"Connection timeout\"**\n```python\n# Use a different endpoint\nfrom chuk_llm import ask_sync\nresponse = ask_sync(\n \"Hello\",\n provider=\"ollama\",\n base_url=\"http://192.168.1.100:11434\" # Custom Ollama server\n)\n```\n\n**Issue: \"Model not available\"**\n```python\n# Discover available models\nfrom chuk_llm.api.providers import trigger_ollama_discovery_and_refresh\ntrigger_ollama_discovery_and_refresh()\n\n# Or check provider models\nfrom chuk_llm import get_provider_config\nconfig = get_provider_config(\"ollama\")\nprint(config.models)\n```\n\n## \ud83d\udcde Support\n\n- **Documentation**: [docs.chukai.io](https://docs.chuk-llm.dev)\n- **Issues**: [GitHub Issues](https://github.com/chuk-llm/chuk-llm/issues)\n- **Discussions**: [GitHub Discussions](https://github.com/chuk-llm/chuk-llm/discussions)\n- **Email**: support@chukai.io\n\n## \ud83c\udf1f Why Choose ChukLLM?\n\n1. **Universal Compatibility**: Works with 100+ LLM providers through OpenAI-compatible APIs\n2. **Zero Lock-in**: Switch providers with one line of code\n3. **Production Ready**: Built-in retries, connection pooling, error handling\n4. **Developer Friendly**: Auto-generated functions, great documentation\n5. **Cost Tracking**: Automatic session analytics and cost estimation\n6. **Enterprise Features**: Azure OpenAI support, compliance, audit logs\n7. **Active Development**: Regular updates with new providers and features\n8. **Community Driven**: Open source with active community contributions\n\n---\n\n**\u2b50 Star us on GitHub if ChukLLM helps your AI projects!**\n\nBuilt with \u2764\ufe0f by the ChukLLM team\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A unified, production-ready Python library for Large Language Model (LLM) providers with real-time streaming, function calling, middleware support, automatic session tracking, dynamic model discovery, and intelligent system prompt generation.",
"version": "0.11",
"project_urls": {
"Changelog": "https://github.com/chrishayuk/chuk-llm/releases",
"Documentation": "https://github.com/chrishayuk/chuk-llm#readme",
"Homepage": "https://github.com/chrishayuk/chuk-llm",
"Issues": "https://github.com/chrishayuk/chuk-llm/issues",
"Repository": "https://github.com/chrishayuk/chuk-llm.git"
},
"split_keywords": [
"llm",
" ai",
" openai",
" anthropic",
" claude",
" gpt",
" gemini",
" ollama",
" streaming",
" async",
" machine-learning"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "c32f41b5d5dc675725742540b31a080795639226f87392856e7caea7e1037c85",
"md5": "10f074852fd3f4579e94059c5e9028b9",
"sha256": "5df747b41ffa1f152e0c08b1e81ab080dd0d311d6bba585b89e18e873c0376e1"
},
"downloads": -1,
"filename": "chuk_llm-0.11-py3-none-any.whl",
"has_sig": false,
"md5_digest": "10f074852fd3f4579e94059c5e9028b9",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11",
"size": 260868,
"upload_time": "2025-08-31T15:00:01",
"upload_time_iso_8601": "2025-08-31T15:00:01.291164Z",
"url": "https://files.pythonhosted.org/packages/c3/2f/41b5d5dc675725742540b31a080795639226f87392856e7caea7e1037c85/chuk_llm-0.11-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "1af75bfe6199bc914b728f2349ce61b35f5f1fc8c2491aa2d9d06445ab137142",
"md5": "9444208831f27e1952adf9114f50ea85",
"sha256": "314362e0dd6a026078113152796264e9a21a2099ecfeb7537c51d1257d77be4d"
},
"downloads": -1,
"filename": "chuk_llm-0.11.tar.gz",
"has_sig": false,
"md5_digest": "9444208831f27e1952adf9114f50ea85",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11",
"size": 244379,
"upload_time": "2025-08-31T15:00:02",
"upload_time_iso_8601": "2025-08-31T15:00:02.857478Z",
"url": "https://files.pythonhosted.org/packages/1a/f7/5bfe6199bc914b728f2349ce61b35f5f1fc8c2491aa2d9d06445ab137142/chuk_llm-0.11.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-31 15:00:02",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "chrishayuk",
"github_project": "chuk-llm",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "chuk-llm"
}
