united-llm

Name	united-llm JSON
Version	0.1.3 JSON
	download
home_page	https://github.com/xychenmsn/united-llm
Summary	United LLM client with search capabilities and FastAPI server
upload_time	2025-08-18 01:35:13
maintainer	None
docs_url	None
author	United LLM Team
requires_python	>=3.8
license	None
keywords	llm openai anthropic google ollama fastapi search structured-output ai
VCS
bugtrack_url
requirements	No requirements were recorded.
Travis-CI	No Travis.
coveralls test coverage	No coveralls.

            # 🚀 United LLM

A comprehensive, production-ready LLM client with multi-provider support, web search integration, structured outputs, and comprehensive logging system.

## ✨ Key Features

- **🔄 Multi-Provider Support**: OpenAI (GPT-4o, GPT-4o-mini), Anthropic (Claude Sonnet 4), Google (Gemini 1.5), and Ollama (Qwen3)
- **🔍 Advanced Web Search**:
  - Anthropic native web search integration
  - DuckDuckGo 3-step search with intelligent query optimization
- **🌐 FastAPI Admin Interface**: RESTful API with automatic documentation and database management
- **📋 Structured Outputs**: Pydantic models with instructor library integration
- **🗄️ Comprehensive Logging**:
  - SQLite database logging with detailed metrics
  - TXT file logging with daily organization
  - JSON structured logs for programmatic analysis
  - Rotating application and server logs
- **⚙️ Smart Configuration**: Environment-based settings with automatic model detection and smart config merging
- **🔧 Production Ready**: Rate limiting, error handling, fallback strategies

## 🏗️ Architecture

```
united_llm/
├── client.py                  # Enhanced LLMClient with search & logging
├── config/
│   ├── settings.py           # Environment-based configuration
│   ├── bootstrap.py          # Bootstrap configuration loader
│   └── logging_config.py     # Comprehensive logging setup
├── search/
│   ├── anthropic_search.py   # Anthropic web search integration
│   └── duckduckgo_search.py  # DuckDuckGo 3-step search
├── utils/
│   ├── database.py           # SQLite logging database
│   ├── schema_converter.py   # JSON Schema ↔ Pydantic conversion

│   ├── schema_utils.py       # Local schema utilities (compatibility layer)
│   └── model_manager.py      # Model detection and management
├── api/
│   ├── server.py             # FastAPI server with admin interface
│   ├── admin.py              # Admin interface implementation
│   └── schemas.py            # Request/response models
└── tests/                    # Comprehensive test suite
```

## 🚀 Quick Start

### 1. Installation

```bash
# Clone the repository
git clone https://github.com/xychenmsn/united_llm.git
cd united_llm

# Create virtual environment (recommended)
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Or install in development mode (recommended for development)
pip install -e .

# Optional: Install development dependencies
pip install -e ".[dev]"  # Includes pytest, black, flake8, mypy
```

### 2. Configuration

Create a `.env` file with your API keys:

```bash
# Copy example configuration
cp env_example.txt .env

# Edit with your API keys
LLM__OPENAI_API_KEY=your_openai_key_here
LLM__ANTHROPIC_API_KEY=your_anthropic_key_here
LLM__GOOGLE_API_KEY=your_google_key_here
LLM__OLLAMA_BASE_URL=http://localhost:11434/v1

# Optional: Configure default model and logging
LLM__DEFAULT_MODEL=gpt-4o-mini
LLM__LOG_TO_DB=true
LLM__LOG_LEVEL=INFO
```

### 3. Basic Usage

```python
from united_llm import LLMClient
from pydantic import BaseModel

# Recommended: Use automatic configuration
client = LLMClient()  # Automatically loads from environment/config files

# Or override specific settings only (Smart Config Merging)
client = LLMClient(config={'log_calls': True})  # Just enable logging

# Define output structure
class ArticleSummary(BaseModel):
    title: str
    summary: str
    key_points: list[str]
    confidence: float

# Generate structured output
result = client.generate_structured(
    prompt="Summarize the latest developments in AI",
    output_model=ArticleSummary,
    model="gpt-4o-mini"
)

print(result.title)     # "Latest AI Developments"
print(result.summary)   # "Recent advances include..."
```

### 4. Web Search Integration

```python
# DuckDuckGo search with any model (3-step process)
result = client.generate_structured(
    prompt="What are the latest AI breakthroughs in 2024?",
    output_model=ArticleSummary,
    model="gpt-4o-mini",
    duckduckgo_search=True  # Enables intelligent search
)

# Anthropic web search (Anthropic models only)
result = client.generate_structured(
    prompt="Current AI research trends",
    output_model=ArticleSummary,
    model="claude-3-5-sonnet-20241022",
    anthropic_web_search=True
)
```

## 🎯 Smart Configuration System

### Key Insight: Override Only What You Need

The `LLMClient` features **smart config merging** - when you pass a config dictionary, it merges with bootstrap defaults instead of replacing them. This means you only need to specify the settings you want to override!

**Before (old behavior):**

```python
# User had to specify EVERYTHING
config = {
    'openai_api_key': 'your-key',
    'anthropic_api_key': 'your-other-key',
    'google_api_key': 'your-google-key',
    'ollama_base_url': 'http://localhost:11434/v1',
    'log_calls': True,  # This is all they wanted to change!
    'log_to_db': True,
    'db_path': 'custom_path.db'
}
client = LLMClient(config=config)
```

**After (new behavior):**

```python
# User only specifies what they want to change
client = LLMClient(config={'log_calls': True})  # That's it!
# All other settings (API keys, etc.) come from bootstrap automatically
```

### How Config Merging Works

1. **Bootstrap loads first** - API keys, paths, defaults from environment/config files
2. **User config overlays** - Only the keys you specify override bootstrap
3. **Everything else preserved** - Model lists, paths, other API keys stay intact

### Common Use Cases

```python
# Enable logging only
client = LLMClient(config={'log_calls': True})

# Test with different API key
client = LLMClient(config={'openai_api_key': 'test-key-12345'})

# Use remote Ollama
client = LLMClient(config={'ollama_base_url': 'http://remote-server:11434/v1'})

# Development vs Production
dev_client = LLMClient(config={'log_calls': True, 'log_to_db': True})
prod_client = LLMClient(config={'log_calls': False})

# Multiple overrides
client = LLMClient(config={
    'openai_api_key': 'test-key',
    'log_calls': True,
    'log_to_db': False,
    'ollama_base_url': 'http://custom-ollama:11434/v1'
})
```

### Configuration Debugging

```python
from united_llm import setup_environment, get_config

# 🔍 DEBUGGING: Inspect zero-config configuration
setup_environment()  # Optional - LLMClient does this automatically
config = get_config()

print(f"API key: {config.get('openai_api_key', '')[:20]}...")
print(f"Log calls: {config.get('log_calls')}")
print(f"All config keys: {list(config.to_dict().keys())}")

# Check specific configuration values
print(f"Default model: {config.get('default_model')}")
print(f"Database path: {config.get_db_path()}")
print(f"Data path: {config.data_path()}")
print(f"Logs path: {config.logs_path()}")
```

**Configuration Priority Order:**

1. **`.env.united_llm` file** (highest priority)
2. **Environment variables with aliases** (LLM**\*, ADMIN**\_, API\_\_\_, etc.)
3. **Standard API key environment variables** (OPENAI_API_KEY, etc.)
4. **TOML config files** (lowest priority)

## 🌐 FastAPI Admin Interface

### Start the Server

```bash
# Development mode
source .venv/bin/activate
python -m united_llm.api.server

# Or use the console script (after installation)
united-llm-server

# Or use the restart script (kills existing processes and restarts)
./restart.sh

# Production mode
uvicorn united_llm.api.server:app --host 0.0.0.0 --port 8818 --workers 4
```

### Admin Interface Features

Visit `http://localhost:8818` for the admin interface:

- **📊 Enhanced Dashboard**: Real-time statistics, provider charts, model analytics
- **📋 Request History**: Comprehensive LLM call history with advanced filtering
- **📄 Export Functionality**: CSV and JSON export capabilities
- **🔐 Authentication**: HTTP Basic Auth protection (admin:admin by default)
- **🎨 Modern UI**: Responsive design with FontAwesome icons
- **🔍 Search Testing**: Test search capabilities with different models
- **📈 Analytics**: Token usage, response times, error rates
- **🎯 Send Request Dialog**: Interactive form with model selection, schema input, and tabbed output
- **💻 Code Generation**: Generate Python code examples for your specific prompts and schemas
- **📱 Responsive Design**: Works on desktop and mobile devices

### API Endpoints

#### Core Generation Endpoints

- `POST /generate/dict` - Generate plain dictionaries with string schemas
- `POST /generate/united` - **🆕 United schema endpoint with auto-detection** (supports both JSON Schema and string schemas)

#### Admin & Management

- `GET /` - Admin dashboard interface with real-time analytics
- `GET /admin` - Redirect to main dashboard
- `GET /admin/llm_calls` - Enhanced request history with filtering
- `GET /admin/export/csv` - Export call history as CSV
- `GET /admin/export/json` - Export call history as JSON
- `GET /admin/logout` - Admin logout endpoint

#### System Information

- `GET /models` - List available models and their status
- `GET /health` - System health check with provider status
- `GET /stats` - Usage statistics and metrics

#### Validation & Testing

- `POST /validate-schema` - Validate JSON schemas
- `POST /schema/validate-string` - Validate string schemas with optimization
- `POST /test-search` - **🆕 Test search functionality with different providers**

#### Authentication

- `POST /admin/login` - Admin login (HTTP Basic Auth)

### Admin Interface Usage

**Dashboard**: Monitor system health, view statistics, and access quick actions

**Requests Page**:

- View complete request history with filtering by model, provider, date range
- Click "View" to see full request details including schema and response
- Click "Send Request" to open the interactive request dialog
- Export data as CSV or JSON for analysis

**Send Request Dialog**:

- **Model Selection**: Choose from available models with intelligent defaults
- **Schema Input**: Use JSON Schema or string schema syntax (supports curly brace format)
- **Web Search**: Enable DuckDuckGo or Anthropic web search
- **Tabbed Output**: Switch between "Output" (LLM response) and "Code" (Python examples)
- **Generate Code**: Get Python code examples for your specific prompt and schema

**Navigation**: Use the sidebar to switch between Dashboard and Requests pages

## 🆕 New Schema Syntax Features

### United Schema API

The new `/generate/united` endpoint accepts either JSON Schema or string schema definitions with auto-detection:

```python
# Works with JSON Schema
result = client.generate_structured(
    prompt="Extract user info: John Doe, 30, from NYC",
    schema={
        "type": "object",
        "properties": {
            "name": {"type": "string"},
            "age": {"type": "integer"},
            "city": {"type": "string"}
        }
    }
)

# Works with string schema
result = client.generate_structured(
    prompt="Extract user info: John Doe, 30, from NYC",
    schema="name, age:int, city"
)
```

### Curly Brace Syntax (NEW!)

JSON-consistent syntax with `{}` for objects and `[]` for arrays:

```python
# Simple object
result = client.generate_dict(
    prompt="Create user: Alice, 25, alice@example.com",
    schema="{name, age:int, email}",
    model="claude-sonnet-4-20250514"
)
# Returns: {"name": "Alice", "age": 25, "email": "alice@example.com"}

# Array of objects
result = client.generate_dict(
    prompt="Create contact list with 2 people",
    schema="[{name, email}]",
    model="claude-sonnet-4-20250514"
)

# Nested structure
team = client.generate_dict(
    prompt="Engineering team with 2 developers",
    schema="{team, members:[{name, role}]}",
    model="claude-sonnet-4-20250514"
)
```

### Benefits of New Syntax

- **50% less code** for common use cases
- **Direct JSON serialization** (no `.model_dump()` needed)
- **Perfect for web APIs** and microservices
- **Same validation** as Pydantic models
- **JSON consistency**: `{}` = objects, `[]` = arrays
- **100% backward compatibility**: All legacy syntax still works

## 📊 Comprehensive Logging System

The system provides multi-level logging for complete observability:

### 1. Database Logging (SQLite)

- **Location**: `llm_calls.db`
- **Content**: Complete request/response records with metadata
- **Features**: Searchable, exportable, with usage statistics
- **Schema**: Timestamps, models, providers, tokens, duration, errors

### 2. TXT File Logging

- **Location**: `logs/llm_calls/YYYY-MM-DD.txt`
- **Content**: Human-readable call logs organized by date
- **Format**: Structured text with timestamps and metadata
- **Rotation**: Daily log files with automatic cleanup

### 3. JSON Structured Logs

- **Location**: `logs/llm_calls/YYYY-MM-DD.json`
- **Content**: Machine-readable JSON logs for analysis
- **Features**: Programmatic log analysis and monitoring
- **Integration**: Perfect for log aggregation systems

### 4. Application Logs

- **Location**: `logs/api.log`, `logs/api.log.bak`
- **Content**: FastAPI server logs and application events
- **Features**: Automatic backup and rotation
- **Levels**: INFO, DEBUG, WARNING, ERROR

## 🧪 Examples and Testing

### Examples Directory (`examples/`)

Learn how to use the library with organized examples:

- **`basic_usage.py`** - Core functionality and getting started
- **`web_search.py`** - DuckDuckGo and Anthropic web search integration
- **`advanced_features.py`** - Configuration, fallbacks, and advanced patterns

### Tests Directory (`tests/`)

Verify functionality with comprehensive tests:

- **`test_integration.py`** - Real-world integration testing
- **`test_comprehensive.py`** - Complete system test with all providers

See `examples/README.md` and `tests/README.md` for detailed descriptions.

### Running Examples and Tests

```bash
# Activate virtual environment
source .venv/bin/activate

# Run examples (start here!)
python examples/basic_usage.py
python examples/web_search.py
python examples/advanced_features.py

# Run tests (verify functionality)
python tests/test_integration.py
python tests/test_comprehensive.py
```

## 🔧 Development and Production

### Development Workflow

```bash
# Start development server manually
source .venv/bin/activate
python -m united_llm.api.server

# Or use the restart script
./restart.sh

# View logs in logs/ directory
tail -f logs/api.log
```

### Production Deployment

```bash
# Production mode with uvicorn
source .venv/bin/activate
uvicorn united_llm.api.server:app --host 0.0.0.0 --port 8080 --workers 4

# Or use restart script on different port
API_PORT=8080 ./restart.sh
```

### Environment Variables

The system supports multiple environment variable formats:

#### Standard Format (Recommended)

```bash
# API Keys
OPENAI_API_KEY=your_openai_key
ANTHROPIC_API_KEY=your_anthropic_key
GOOGLE_API_KEY=your_google_key

# Configuration
LLM__DEFAULT_MODEL=gpt-4o-mini
LLM__LOG_TO_DB=true
LLM__LOG_CALLS=true
```

#### Domain-Specific Format (`.env.united_llm`)

```bash
# Clean format without prefixes
DEFAULT_MODEL=gpt-4o-mini
LOG_TO_DB=true
LOG_CALLS=true
OPENAI_API_KEY=your_openai_key
```

#### Namespaced Format

```bash
# Full namespace
UNITED_LLM_DEFAULT_MODEL=gpt-4o-mini
UNITED_LLM_LOG_TO_DB=true
UNITED_LLM_OPENAI_API_KEY=your_openai_key
```

## 🚀 Advanced Features

### Chinese Text Optimization

The system includes specialized Chinese text processing:

```python
# Automatic Chinese query optimization
result = client.generate_structured(
    prompt="分析最新的人工智能技术趋势",
    output_model=TechTrend,
    duckduckgo_search=True  # Optimizes Chinese search automatically
)
```

### Model Fallback

Automatic fallback to working models when preferred models are unavailable:

```python
# Will try gpt-4 first, fall back to gpt-3.5-turbo if unavailable
client = LLMClient(config={
    'default_model': 'gpt-4',
    'fallback_models': ['gpt-3.5-turbo', 'claude-3-haiku']
})
```

### Rate Limiting and Error Handling

- Automatic retry with exponential backoff
- Rate limit detection and handling
- Comprehensive error logging and reporting
- Graceful degradation strategies

## 📦 Dependencies & Installation Options

### Core Dependencies

Core dependencies are automatically installed with the base package:

```bash
# Core LLM libraries
instructor>=1.4.3
pydantic>=2.0.0
pydantic-settings>=2.0.0

# LLM Provider SDKs
openai>=1.12.0
anthropic>=0.21.0
google-generativeai>=0.8.0

# Web framework
fastapi>=0.104.0
uvicorn[standard]>=0.24.0

# Search and utilities
duckduckgo-search>=6.1.0
requests>=2.31.0
httpx>=0.27.0
python-dotenv>=1.0.0
```

### Installation Options

```bash
# Basic installation
pip install -e .

# Development installation (includes testing tools)
pip install -e ".[dev]"
# Includes: pytest, pytest-asyncio, pytest-cov, black, flake8, mypy, pre-commit

# Documentation installation
pip install -e ".[docs]"
# Includes: mkdocs, mkdocs-material, mkdocstrings

# Complete installation (everything)
pip install -e ".[all]"
```

### Console Scripts

After installation, you can use these command-line tools:

```bash
# Start the API server
united-llm-server

# CLI interface (if implemented)
united-llm --help
```

## 🏆 Production Readiness

The united LLM project is **production-ready** with:

- **📋 Comprehensive Documentation**: Complete API docs and examples
- **⚙️ Flexible Configuration**: Environment-based settings with smart merging
- **🔧 Error Handling**: Robust error handling and fallbacks
- **📊 Monitoring**: Statistics and logging capabilities
- **🌐 API Server**: Production-ready FastAPI server
- **🔍 Search Integration**: Both Anthropic and DuckDuckGo search
- **🇨🇳 Chinese Support**: Specialized Chinese text processing
- **🔄 Multi-Provider**: Support for all major LLM providers
- **🛡️ Security**: HTTP Basic Auth and input validation
- **📈 Scalability**: Handles thousands of requests efficiently

## 🎉 Recent Updates

### v2.1 - Enhanced Features & Reliability (Current)

- ✅ **Ollama Function Detection**: Automatic detection of function calling capabilities
- ✅ **Search Testing Endpoint**: `/test-search` for validating search functionality
- ✅ **Enhanced Error Handling**: Better fallback strategies and error reporting
- ✅ **Console Scripts**: `united-llm-server` command-line tool
- ✅ **Improved Documentation**: Updated README and development guide
- ✅ **Better Type Hints**: Enhanced type safety throughout codebase

### v2.0 - Schema Unification & Config Improvements

- ✅ **United Schema API**: Single endpoint handles both JSON Schema and string definitions
- ✅ **Curly Brace Syntax**: JSON-consistent `{}` and `[]` syntax
- ✅ **Smart Config Merging**: Override only what you need
- ✅ **Enhanced Admin Interface**: Modern UI with real-time statistics
- ✅ **Improved Debugging**: `get_effective_config()` for final configuration inspection
- ✅ **Namespace Consistency**: All imports from `united_llm`
- ✅ **Pydantic v2 Compatibility**: Eliminated schema field conflicts

### v1.5 - Admin Interface & Logging

- ✅ **Admin Dashboard**: Beautiful web interface with analytics
- ✅ **Database Logging**: SQLite with comprehensive metrics
- ✅ **Export Functionality**: CSV and JSON export capabilities
- ✅ **Search Integration**: DuckDuckGo and Anthropic web search
- ✅ **Multi-Provider Support**: OpenAI, Anthropic, Google, Ollama

## 📞 Support

For issues, questions, or contributions:

1. Check the comprehensive test files for usage examples
2. Review the admin interface for debugging tools
3. Use `get_effective_config()` for configuration debugging
4. Check logs in the `logs/` directory for detailed error information

---

**💡 Pro Tip**: Start with `LLMClient()` for automatic configuration, then use `config={'key': 'value'}` to override only specific settings you need to change!

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/xychenmsn/united-llm",
    "name": "united-llm",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.8",
    "maintainer_email": null,
    "keywords": "llm, openai, anthropic, google, ollama, fastapi, search, structured-output, ai",
    "author": "United LLM Team",
    "author_email": "contact@united-llm.com",
    "download_url": "https://files.pythonhosted.org/packages/95/84/abe21f429a87da3c8b4e926ab69c72b85324afbe60056b192008df3bb1ab/united_llm-0.1.3.tar.gz",
    "platform": null,
    "description": "# \ud83d\ude80 United LLM\n\nA comprehensive, production-ready LLM client with multi-provider support, web search integration, structured outputs, and comprehensive logging system.\n\n## \u2728 Key Features\n\n- **\ud83d\udd04 Multi-Provider Support**: OpenAI (GPT-4o, GPT-4o-mini), Anthropic (Claude Sonnet 4), Google (Gemini 1.5), and Ollama (Qwen3)\n- **\ud83d\udd0d Advanced Web Search**:\n  - Anthropic native web search integration\n  - DuckDuckGo 3-step search with intelligent query optimization\n- **\ud83c\udf10 FastAPI Admin Interface**: RESTful API with automatic documentation and database management\n- **\ud83d\udccb Structured Outputs**: Pydantic models with instructor library integration\n- **\ud83d\uddc4\ufe0f Comprehensive Logging**:\n  - SQLite database logging with detailed metrics\n  - TXT file logging with daily organization\n  - JSON structured logs for programmatic analysis\n  - Rotating application and server logs\n- **\u2699\ufe0f Smart Configuration**: Environment-based settings with automatic model detection and smart config merging\n- **\ud83d\udd27 Production Ready**: Rate limiting, error handling, fallback strategies\n\n## \ud83c\udfd7\ufe0f Architecture\n\n```\nunited_llm/\n\u251c\u2500\u2500 client.py                  # Enhanced LLMClient with search & logging\n\u251c\u2500\u2500 config/\n\u2502   \u251c\u2500\u2500 settings.py           # Environment-based configuration\n\u2502   \u251c\u2500\u2500 bootstrap.py          # Bootstrap configuration loader\n\u2502   \u2514\u2500\u2500 logging_config.py     # Comprehensive logging setup\n\u251c\u2500\u2500 search/\n\u2502   \u251c\u2500\u2500 anthropic_search.py   # Anthropic web search integration\n\u2502   \u2514\u2500\u2500 duckduckgo_search.py  # DuckDuckGo 3-step search\n\u251c\u2500\u2500 utils/\n\u2502   \u251c\u2500\u2500 database.py           # SQLite logging database\n\u2502   \u251c\u2500\u2500 schema_converter.py   # JSON Schema \u2194 Pydantic conversion\n\n\u2502   \u251c\u2500\u2500 schema_utils.py       # Local schema utilities (compatibility layer)\n\u2502   \u2514\u2500\u2500 model_manager.py      # Model detection and management\n\u251c\u2500\u2500 api/\n\u2502   \u251c\u2500\u2500 server.py             # FastAPI server with admin interface\n\u2502   \u251c\u2500\u2500 admin.py              # Admin interface implementation\n\u2502   \u2514\u2500\u2500 schemas.py            # Request/response models\n\u2514\u2500\u2500 tests/                    # Comprehensive test suite\n```\n\n## \ud83d\ude80 Quick Start\n\n### 1. Installation\n\n```bash\n# Clone the repository\ngit clone https://github.com/xychenmsn/united_llm.git\ncd united_llm\n\n# Create virtual environment (recommended)\npython -m venv .venv\nsource .venv/bin/activate  # On Windows: .venv\\Scripts\\activate\n\n# Install dependencies\npip install -r requirements.txt\n\n# Or install in development mode (recommended for development)\npip install -e .\n\n# Optional: Install development dependencies\npip install -e \".[dev]\"  # Includes pytest, black, flake8, mypy\n```\n\n### 2. Configuration\n\nCreate a `.env` file with your API keys:\n\n```bash\n# Copy example configuration\ncp env_example.txt .env\n\n# Edit with your API keys\nLLM__OPENAI_API_KEY=your_openai_key_here\nLLM__ANTHROPIC_API_KEY=your_anthropic_key_here\nLLM__GOOGLE_API_KEY=your_google_key_here\nLLM__OLLAMA_BASE_URL=http://localhost:11434/v1\n\n# Optional: Configure default model and logging\nLLM__DEFAULT_MODEL=gpt-4o-mini\nLLM__LOG_TO_DB=true\nLLM__LOG_LEVEL=INFO\n```\n\n### 3. Basic Usage\n\n```python\nfrom united_llm import LLMClient\nfrom pydantic import BaseModel\n\n# Recommended: Use automatic configuration\nclient = LLMClient()  # Automatically loads from environment/config files\n\n# Or override specific settings only (Smart Config Merging)\nclient = LLMClient(config={'log_calls': True})  # Just enable logging\n\n# Define output structure\nclass ArticleSummary(BaseModel):\n    title: str\n    summary: str\n    key_points: list[str]\n    confidence: float\n\n# Generate structured output\nresult = client.generate_structured(\n    prompt=\"Summarize the latest developments in AI\",\n    output_model=ArticleSummary,\n    model=\"gpt-4o-mini\"\n)\n\nprint(result.title)     # \"Latest AI Developments\"\nprint(result.summary)   # \"Recent advances include...\"\n```\n\n### 4. Web Search Integration\n\n```python\n# DuckDuckGo search with any model (3-step process)\nresult = client.generate_structured(\n    prompt=\"What are the latest AI breakthroughs in 2024?\",\n    output_model=ArticleSummary,\n    model=\"gpt-4o-mini\",\n    duckduckgo_search=True  # Enables intelligent search\n)\n\n# Anthropic web search (Anthropic models only)\nresult = client.generate_structured(\n    prompt=\"Current AI research trends\",\n    output_model=ArticleSummary,\n    model=\"claude-3-5-sonnet-20241022\",\n    anthropic_web_search=True\n)\n```\n\n## \ud83c\udfaf Smart Configuration System\n\n### Key Insight: Override Only What You Need\n\nThe `LLMClient` features **smart config merging** - when you pass a config dictionary, it merges with bootstrap defaults instead of replacing them. This means you only need to specify the settings you want to override!\n\n**Before (old behavior):**\n\n```python\n# User had to specify EVERYTHING\nconfig = {\n    'openai_api_key': 'your-key',\n    'anthropic_api_key': 'your-other-key',\n    'google_api_key': 'your-google-key',\n    'ollama_base_url': 'http://localhost:11434/v1',\n    'log_calls': True,  # This is all they wanted to change!\n    'log_to_db': True,\n    'db_path': 'custom_path.db'\n}\nclient = LLMClient(config=config)\n```\n\n**After (new behavior):**\n\n```python\n# User only specifies what they want to change\nclient = LLMClient(config={'log_calls': True})  # That's it!\n# All other settings (API keys, etc.) come from bootstrap automatically\n```\n\n### How Config Merging Works\n\n1. **Bootstrap loads first** - API keys, paths, defaults from environment/config files\n2. **User config overlays** - Only the keys you specify override bootstrap\n3. **Everything else preserved** - Model lists, paths, other API keys stay intact\n\n### Common Use Cases\n\n```python\n# Enable logging only\nclient = LLMClient(config={'log_calls': True})\n\n# Test with different API key\nclient = LLMClient(config={'openai_api_key': 'test-key-12345'})\n\n# Use remote Ollama\nclient = LLMClient(config={'ollama_base_url': 'http://remote-server:11434/v1'})\n\n# Development vs Production\ndev_client = LLMClient(config={'log_calls': True, 'log_to_db': True})\nprod_client = LLMClient(config={'log_calls': False})\n\n# Multiple overrides\nclient = LLMClient(config={\n    'openai_api_key': 'test-key',\n    'log_calls': True,\n    'log_to_db': False,\n    'ollama_base_url': 'http://custom-ollama:11434/v1'\n})\n```\n\n### Configuration Debugging\n\n```python\nfrom united_llm import setup_environment, get_config\n\n# \ud83d\udd0d DEBUGGING: Inspect zero-config configuration\nsetup_environment()  # Optional - LLMClient does this automatically\nconfig = get_config()\n\nprint(f\"API key: {config.get('openai_api_key', '')[:20]}...\")\nprint(f\"Log calls: {config.get('log_calls')}\")\nprint(f\"All config keys: {list(config.to_dict().keys())}\")\n\n# Check specific configuration values\nprint(f\"Default model: {config.get('default_model')}\")\nprint(f\"Database path: {config.get_db_path()}\")\nprint(f\"Data path: {config.data_path()}\")\nprint(f\"Logs path: {config.logs_path()}\")\n```\n\n**Configuration Priority Order:**\n\n1. **`.env.united_llm` file** (highest priority)\n2. **Environment variables with aliases** (LLM**\\*, ADMIN**\\_, API\\_\\_\\_, etc.)\n3. **Standard API key environment variables** (OPENAI_API_KEY, etc.)\n4. **TOML config files** (lowest priority)\n\n## \ud83c\udf10 FastAPI Admin Interface\n\n### Start the Server\n\n```bash\n# Development mode\nsource .venv/bin/activate\npython -m united_llm.api.server\n\n# Or use the console script (after installation)\nunited-llm-server\n\n# Or use the restart script (kills existing processes and restarts)\n./restart.sh\n\n# Production mode\nuvicorn united_llm.api.server:app --host 0.0.0.0 --port 8818 --workers 4\n```\n\n### Admin Interface Features\n\nVisit `http://localhost:8818` for the admin interface:\n\n- **\ud83d\udcca Enhanced Dashboard**: Real-time statistics, provider charts, model analytics\n- **\ud83d\udccb Request History**: Comprehensive LLM call history with advanced filtering\n- **\ud83d\udcc4 Export Functionality**: CSV and JSON export capabilities\n- **\ud83d\udd10 Authentication**: HTTP Basic Auth protection (admin:admin by default)\n- **\ud83c\udfa8 Modern UI**: Responsive design with FontAwesome icons\n- **\ud83d\udd0d Search Testing**: Test search capabilities with different models\n- **\ud83d\udcc8 Analytics**: Token usage, response times, error rates\n- **\ud83c\udfaf Send Request Dialog**: Interactive form with model selection, schema input, and tabbed output\n- **\ud83d\udcbb Code Generation**: Generate Python code examples for your specific prompts and schemas\n- **\ud83d\udcf1 Responsive Design**: Works on desktop and mobile devices\n\n### API Endpoints\n\n#### Core Generation Endpoints\n\n- `POST /generate/dict` - Generate plain dictionaries with string schemas\n- `POST /generate/united` - **\ud83c\udd95 United schema endpoint with auto-detection** (supports both JSON Schema and string schemas)\n\n#### Admin & Management\n\n- `GET /` - Admin dashboard interface with real-time analytics\n- `GET /admin` - Redirect to main dashboard\n- `GET /admin/llm_calls` - Enhanced request history with filtering\n- `GET /admin/export/csv` - Export call history as CSV\n- `GET /admin/export/json` - Export call history as JSON\n- `GET /admin/logout` - Admin logout endpoint\n\n#### System Information\n\n- `GET /models` - List available models and their status\n- `GET /health` - System health check with provider status\n- `GET /stats` - Usage statistics and metrics\n\n#### Validation & Testing\n\n- `POST /validate-schema` - Validate JSON schemas\n- `POST /schema/validate-string` - Validate string schemas with optimization\n- `POST /test-search` - **\ud83c\udd95 Test search functionality with different providers**\n\n#### Authentication\n\n- `POST /admin/login` - Admin login (HTTP Basic Auth)\n\n### Admin Interface Usage\n\n**Dashboard**: Monitor system health, view statistics, and access quick actions\n\n**Requests Page**:\n\n- View complete request history with filtering by model, provider, date range\n- Click \"View\" to see full request details including schema and response\n- Click \"Send Request\" to open the interactive request dialog\n- Export data as CSV or JSON for analysis\n\n**Send Request Dialog**:\n\n- **Model Selection**: Choose from available models with intelligent defaults\n- **Schema Input**: Use JSON Schema or string schema syntax (supports curly brace format)\n- **Web Search**: Enable DuckDuckGo or Anthropic web search\n- **Tabbed Output**: Switch between \"Output\" (LLM response) and \"Code\" (Python examples)\n- **Generate Code**: Get Python code examples for your specific prompt and schema\n\n**Navigation**: Use the sidebar to switch between Dashboard and Requests pages\n\n## \ud83c\udd95 New Schema Syntax Features\n\n### United Schema API\n\nThe new `/generate/united` endpoint accepts either JSON Schema or string schema definitions with auto-detection:\n\n```python\n# Works with JSON Schema\nresult = client.generate_structured(\n    prompt=\"Extract user info: John Doe, 30, from NYC\",\n    schema={\n        \"type\": \"object\",\n        \"properties\": {\n            \"name\": {\"type\": \"string\"},\n            \"age\": {\"type\": \"integer\"},\n            \"city\": {\"type\": \"string\"}\n        }\n    }\n)\n\n# Works with string schema\nresult = client.generate_structured(\n    prompt=\"Extract user info: John Doe, 30, from NYC\",\n    schema=\"name, age:int, city\"\n)\n```\n\n### Curly Brace Syntax (NEW!)\n\nJSON-consistent syntax with `{}` for objects and `[]` for arrays:\n\n```python\n# Simple object\nresult = client.generate_dict(\n    prompt=\"Create user: Alice, 25, alice@example.com\",\n    schema=\"{name, age:int, email}\",\n    model=\"claude-sonnet-4-20250514\"\n)\n# Returns: {\"name\": \"Alice\", \"age\": 25, \"email\": \"alice@example.com\"}\n\n# Array of objects\nresult = client.generate_dict(\n    prompt=\"Create contact list with 2 people\",\n    schema=\"[{name, email}]\",\n    model=\"claude-sonnet-4-20250514\"\n)\n\n# Nested structure\nteam = client.generate_dict(\n    prompt=\"Engineering team with 2 developers\",\n    schema=\"{team, members:[{name, role}]}\",\n    model=\"claude-sonnet-4-20250514\"\n)\n```\n\n### Benefits of New Syntax\n\n- **50% less code** for common use cases\n- **Direct JSON serialization** (no `.model_dump()` needed)\n- **Perfect for web APIs** and microservices\n- **Same validation** as Pydantic models\n- **JSON consistency**: `{}` = objects, `[]` = arrays\n- **100% backward compatibility**: All legacy syntax still works\n\n## \ud83d\udcca Comprehensive Logging System\n\nThe system provides multi-level logging for complete observability:\n\n### 1. Database Logging (SQLite)\n\n- **Location**: `llm_calls.db`\n- **Content**: Complete request/response records with metadata\n- **Features**: Searchable, exportable, with usage statistics\n- **Schema**: Timestamps, models, providers, tokens, duration, errors\n\n### 2. TXT File Logging\n\n- **Location**: `logs/llm_calls/YYYY-MM-DD.txt`\n- **Content**: Human-readable call logs organized by date\n- **Format**: Structured text with timestamps and metadata\n- **Rotation**: Daily log files with automatic cleanup\n\n### 3. JSON Structured Logs\n\n- **Location**: `logs/llm_calls/YYYY-MM-DD.json`\n- **Content**: Machine-readable JSON logs for analysis\n- **Features**: Programmatic log analysis and monitoring\n- **Integration**: Perfect for log aggregation systems\n\n### 4. Application Logs\n\n- **Location**: `logs/api.log`, `logs/api.log.bak`\n- **Content**: FastAPI server logs and application events\n- **Features**: Automatic backup and rotation\n- **Levels**: INFO, DEBUG, WARNING, ERROR\n\n## \ud83e\uddea Examples and Testing\n\n### Examples Directory (`examples/`)\n\nLearn how to use the library with organized examples:\n\n- **`basic_usage.py`** - Core functionality and getting started\n- **`web_search.py`** - DuckDuckGo and Anthropic web search integration\n- **`advanced_features.py`** - Configuration, fallbacks, and advanced patterns\n\n### Tests Directory (`tests/`)\n\nVerify functionality with comprehensive tests:\n\n- **`test_integration.py`** - Real-world integration testing\n- **`test_comprehensive.py`** - Complete system test with all providers\n\nSee `examples/README.md` and `tests/README.md` for detailed descriptions.\n\n### Running Examples and Tests\n\n```bash\n# Activate virtual environment\nsource .venv/bin/activate\n\n# Run examples (start here!)\npython examples/basic_usage.py\npython examples/web_search.py\npython examples/advanced_features.py\n\n# Run tests (verify functionality)\npython tests/test_integration.py\npython tests/test_comprehensive.py\n```\n\n## \ud83d\udd27 Development and Production\n\n### Development Workflow\n\n```bash\n# Start development server manually\nsource .venv/bin/activate\npython -m united_llm.api.server\n\n# Or use the restart script\n./restart.sh\n\n# View logs in logs/ directory\ntail -f logs/api.log\n```\n\n### Production Deployment\n\n```bash\n# Production mode with uvicorn\nsource .venv/bin/activate\nuvicorn united_llm.api.server:app --host 0.0.0.0 --port 8080 --workers 4\n\n# Or use restart script on different port\nAPI_PORT=8080 ./restart.sh\n```\n\n### Environment Variables\n\nThe system supports multiple environment variable formats:\n\n#### Standard Format (Recommended)\n\n```bash\n# API Keys\nOPENAI_API_KEY=your_openai_key\nANTHROPIC_API_KEY=your_anthropic_key\nGOOGLE_API_KEY=your_google_key\n\n# Configuration\nLLM__DEFAULT_MODEL=gpt-4o-mini\nLLM__LOG_TO_DB=true\nLLM__LOG_CALLS=true\n```\n\n#### Domain-Specific Format (`.env.united_llm`)\n\n```bash\n# Clean format without prefixes\nDEFAULT_MODEL=gpt-4o-mini\nLOG_TO_DB=true\nLOG_CALLS=true\nOPENAI_API_KEY=your_openai_key\n```\n\n#### Namespaced Format\n\n```bash\n# Full namespace\nUNITED_LLM_DEFAULT_MODEL=gpt-4o-mini\nUNITED_LLM_LOG_TO_DB=true\nUNITED_LLM_OPENAI_API_KEY=your_openai_key\n```\n\n## \ud83d\ude80 Advanced Features\n\n### Chinese Text Optimization\n\nThe system includes specialized Chinese text processing:\n\n```python\n# Automatic Chinese query optimization\nresult = client.generate_structured(\n    prompt=\"\u5206\u6790\u6700\u65b0\u7684\u4eba\u5de5\u667a\u80fd\u6280\u672f\u8d8b\u52bf\",\n    output_model=TechTrend,\n    duckduckgo_search=True  # Optimizes Chinese search automatically\n)\n```\n\n### Model Fallback\n\nAutomatic fallback to working models when preferred models are unavailable:\n\n```python\n# Will try gpt-4 first, fall back to gpt-3.5-turbo if unavailable\nclient = LLMClient(config={\n    'default_model': 'gpt-4',\n    'fallback_models': ['gpt-3.5-turbo', 'claude-3-haiku']\n})\n```\n\n### Rate Limiting and Error Handling\n\n- Automatic retry with exponential backoff\n- Rate limit detection and handling\n- Comprehensive error logging and reporting\n- Graceful degradation strategies\n\n## \ud83d\udce6 Dependencies & Installation Options\n\n### Core Dependencies\n\nCore dependencies are automatically installed with the base package:\n\n```bash\n# Core LLM libraries\ninstructor>=1.4.3\npydantic>=2.0.0\npydantic-settings>=2.0.0\n\n# LLM Provider SDKs\nopenai>=1.12.0\nanthropic>=0.21.0\ngoogle-generativeai>=0.8.0\n\n# Web framework\nfastapi>=0.104.0\nuvicorn[standard]>=0.24.0\n\n# Search and utilities\nduckduckgo-search>=6.1.0\nrequests>=2.31.0\nhttpx>=0.27.0\npython-dotenv>=1.0.0\n```\n\n### Installation Options\n\n```bash\n# Basic installation\npip install -e .\n\n# Development installation (includes testing tools)\npip install -e \".[dev]\"\n# Includes: pytest, pytest-asyncio, pytest-cov, black, flake8, mypy, pre-commit\n\n# Documentation installation\npip install -e \".[docs]\"\n# Includes: mkdocs, mkdocs-material, mkdocstrings\n\n# Complete installation (everything)\npip install -e \".[all]\"\n```\n\n### Console Scripts\n\nAfter installation, you can use these command-line tools:\n\n```bash\n# Start the API server\nunited-llm-server\n\n# CLI interface (if implemented)\nunited-llm --help\n```\n\n## \ud83c\udfc6 Production Readiness\n\nThe united LLM project is **production-ready** with:\n\n- **\ud83d\udccb Comprehensive Documentation**: Complete API docs and examples\n- **\u2699\ufe0f Flexible Configuration**: Environment-based settings with smart merging\n- **\ud83d\udd27 Error Handling**: Robust error handling and fallbacks\n- **\ud83d\udcca Monitoring**: Statistics and logging capabilities\n- **\ud83c\udf10 API Server**: Production-ready FastAPI server\n- **\ud83d\udd0d Search Integration**: Both Anthropic and DuckDuckGo search\n- **\ud83c\udde8\ud83c\uddf3 Chinese Support**: Specialized Chinese text processing\n- **\ud83d\udd04 Multi-Provider**: Support for all major LLM providers\n- **\ud83d\udee1\ufe0f Security**: HTTP Basic Auth and input validation\n- **\ud83d\udcc8 Scalability**: Handles thousands of requests efficiently\n\n## \ud83c\udf89 Recent Updates\n\n### v2.1 - Enhanced Features & Reliability (Current)\n\n- \u2705 **Ollama Function Detection**: Automatic detection of function calling capabilities\n- \u2705 **Search Testing Endpoint**: `/test-search` for validating search functionality\n- \u2705 **Enhanced Error Handling**: Better fallback strategies and error reporting\n- \u2705 **Console Scripts**: `united-llm-server` command-line tool\n- \u2705 **Improved Documentation**: Updated README and development guide\n- \u2705 **Better Type Hints**: Enhanced type safety throughout codebase\n\n### v2.0 - Schema Unification & Config Improvements\n\n- \u2705 **United Schema API**: Single endpoint handles both JSON Schema and string definitions\n- \u2705 **Curly Brace Syntax**: JSON-consistent `{}` and `[]` syntax\n- \u2705 **Smart Config Merging**: Override only what you need\n- \u2705 **Enhanced Admin Interface**: Modern UI with real-time statistics\n- \u2705 **Improved Debugging**: `get_effective_config()` for final configuration inspection\n- \u2705 **Namespace Consistency**: All imports from `united_llm`\n- \u2705 **Pydantic v2 Compatibility**: Eliminated schema field conflicts\n\n### v1.5 - Admin Interface & Logging\n\n- \u2705 **Admin Dashboard**: Beautiful web interface with analytics\n- \u2705 **Database Logging**: SQLite with comprehensive metrics\n- \u2705 **Export Functionality**: CSV and JSON export capabilities\n- \u2705 **Search Integration**: DuckDuckGo and Anthropic web search\n- \u2705 **Multi-Provider Support**: OpenAI, Anthropic, Google, Ollama\n\n## \ud83d\udcde Support\n\nFor issues, questions, or contributions:\n\n1. Check the comprehensive test files for usage examples\n2. Review the admin interface for debugging tools\n3. Use `get_effective_config()` for configuration debugging\n4. Check logs in the `logs/` directory for detailed error information\n\n---\n\n**\ud83d\udca1 Pro Tip**: Start with `LLMClient()` for automatic configuration, then use `config={'key': 'value'}` to override only specific settings you need to change!\n",
    "bugtrack_url": null,
    "license": null,
    "summary": "United LLM client with search capabilities and FastAPI server",
    "version": "0.1.3",
    "project_urls": {
        "Bug Reports": "https://github.com/xychenmsn/united-llm/issues",
        "Documentation": "https://github.com/xychenmsn/united-llm#readme",
        "Homepage": "https://github.com/xychenmsn/united-llm",
        "Source": "https://github.com/xychenmsn/united-llm"
    },
    "split_keywords": [
        "llm",
        " openai",
        " anthropic",
        " google",
        " ollama",
        " fastapi",
        " search",
        " structured-output",
        " ai"
    ],
    "urls": [
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "f1e90496295e80e6a0f4f364b9aca7bedb40788107b9bddef5366a7d3ea35b88",
                "md5": "c728adef915020055c067b04dc398d06",
                "sha256": "e7ff35a2230c6ff5716d8633e26fd63064ea58c68f4271104003c35533a560c9"
            },
            "downloads": -1,
            "filename": "united_llm-0.1.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "c728adef915020055c067b04dc398d06",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.8",
            "size": 97404,
            "upload_time": "2025-08-18T01:35:12",
            "upload_time_iso_8601": "2025-08-18T01:35:12.050245Z",
            "url": "https://files.pythonhosted.org/packages/f1/e9/0496295e80e6a0f4f364b9aca7bedb40788107b9bddef5366a7d3ea35b88/united_llm-0.1.3-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "9584abe21f429a87da3c8b4e926ab69c72b85324afbe60056b192008df3bb1ab",
                "md5": "ae8cf11541953f9aa6dc495ba48c6281",
                "sha256": "23f530b30547a4306ca7392af8c3ff3fc76e8136084aa78d4885e432b8d88ac0"
            },
            "downloads": -1,
            "filename": "united_llm-0.1.3.tar.gz",
            "has_sig": false,
            "md5_digest": "ae8cf11541953f9aa6dc495ba48c6281",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.8",
            "size": 104897,
            "upload_time": "2025-08-18T01:35:13",
            "upload_time_iso_8601": "2025-08-18T01:35:13.189954Z",
            "url": "https://files.pythonhosted.org/packages/95/84/abe21f429a87da3c8b4e926ab69c72b85324afbe60056b192008df3bb1ab/united_llm-0.1.3.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-08-18 01:35:13",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "xychenmsn",
    "github_project": "united-llm",
    "github_not_found": true,
    "lcname": "united-llm"
}