| Name | model-forge-llm JSON |
| Version |
2.3.0
JSON |
| download |
| home_page | None |
| Summary | A reusable library for managing LLM providers, authentication, and model selection. |
| upload_time | 2025-07-28 15:34:34 |
| maintainer | None |
| docs_url | None |
| author | ModelForge Contributors |
| requires_python | <4.0,>=3.11 |
| license | MIT License
Copyright (c) 2025 Shuhai Miao
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
|
| keywords |
llm
ai
langchain
openai
ollama
providers
authentication
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# ModelForge
A Python library for managing LLM providers, authentication, and model selection with seamless LangChain integration.
[](https://badge.fury.io/py/model-forge-llm)
[](https://www.python.org/downloads/)
[](LICENSE)
**🚀 Version 2.2.0 - Enhanced Model Metadata, Improved Telemetry, and Quiet Mode!**
## Installation
### Recommended: Virtual Environment
```bash
# Create and activate virtual environment
python -m venv model-forge-env
source model-forge-env/bin/activate # On Windows: model-forge-env\Scripts\activate
# Install package
pip install model-forge-llm
# Verify installation
modelforge --help
```
### Quick Install (System-wide)
```bash
pip install model-forge-llm
```
## Quick Start
### 🚀 NEW: Interactive Configuration Wizard (Easiest)
```bash
# Launch the interactive wizard
modelforge config
# The wizard will guide you through:
# 1. Choosing a provider (with recommendations)
# 2. Setting up authentication
# 3. Selecting a model
# 4. Testing your configuration
```
### Option 1: GitHub Copilot via Device Authentication Flow
```bash
# Discover GitHub Copilot models
modelforge models list --provider github_copilot
# Set up GitHub Copilot with device authentication
modelforge auth login --provider github_copilot
# Select Claude 3.7 Sonnet via GitHub Copilot
modelforge config use --provider github_copilot --model claude-3.7-sonnet
# Test your setup
modelforge test --prompt "Write a Python function to reverse a string"
```
### Option 2: OpenAI (API Key Required)
```bash
# Add OpenAI with your API key
modelforge auth login --provider openai --api-key YOUR_API_KEY
# Select GPT-4o-mini
modelforge config use --provider openai --model gpt-4o-mini
# Test your setup
modelforge test --prompt "Hello, world!"
```
### Option 3: Local Ollama (No API Key Needed)
```bash
# Make sure Ollama is running locally
# Then add a local model
modelforge config add --provider ollama --model qwen3:1.7b
# Select the local model
modelforge config use --provider ollama --model qwen3:1.7b
# Test your setup
modelforge test --prompt "What is machine learning?"
```
### Common Commands - Complete Lifecycle
```bash
# Installation & Setup
modelforge --help # Verify installation
modelforge config show # View current config
# Model Discovery & Selection
modelforge models list # List all available models
modelforge models search "claude" # Search models by name
modelforge models info --provider openai --model gpt-4o # Get model details
# Authentication Management
modelforge auth login --provider openai --api-key KEY # API key auth
modelforge auth login --provider github_copilot # Device flow auth
modelforge auth status # Check auth status
modelforge auth logout --provider openai # Remove credentials
# Configuration Management
# Interactive Configuration Wizard (NEW in v2.2.2)
modelforge config # Launch interactive wizard
modelforge config --wizard # Alternative wizard launch
modelforge config --wizard --verbose # Debug mode with logs
# Manual Configuration
modelforge config add --provider openai --model gpt-4o-mini --api-key KEY
modelforge config add --provider ollama --model qwen3:1.7b --local
modelforge config use --provider openai --model gpt-4o-mini
modelforge config remove --provider openai --model gpt-4o-mini
# Testing & Usage (NEW in v2.2: Quiet mode for automation)
modelforge test --prompt "Hello, how are you?" # Test current model
modelforge test --prompt "Explain quantum computing" --verbose # Debug mode
modelforge test --input-file prompt.txt --output-file response.txt # File I/O
echo "What is AI?" | modelforge test # Stdin input
modelforge test --prompt "Hello" --no-telemetry # Disable telemetry
modelforge test --prompt "What is 2+2?" --quiet # Minimal output (v2.2)
echo "Hello" | modelforge test --quiet > output.txt # Perfect for piping
# Cache & Maintenance
modelforge models list --refresh # Force refresh from models.dev
# Telemetry Settings (NEW in v2.0)
modelforge settings telemetry on # Enable telemetry display
modelforge settings telemetry off # Disable telemetry display
modelforge settings telemetry status # Check current setting
```
## What's New
### v2.2.2 Features
#### 🧙 Interactive Configuration Wizard
- **Easy setup**: Run `modelforge config` to launch the interactive wizard
- **Visual guidance**: Arrow key navigation with clear indicators
- ⭐ = Recommended providers
- ✅ = Already configured
- **Smart defaults**: Automatically selects current provider/model
- **Flexible authentication**: Choose to use existing credentials or re-authenticate
- **Live testing**: Test configuration with a sample prompt before saving
- **Quiet by default**: No verbose logs unless `--verbose` flag is used
### v2.2.0 Features
#### 🤫 Quiet Mode for Automation
- **`--quiet` flag**: Minimal output showing only the model response
- **Perfect for piping**: Clean output for scripts and automation
- **Automatic telemetry suppression**: No metadata in quiet mode
- **Conflict prevention**: Cannot use with `--verbose` flag
#### 📊 Enhanced Telemetry Display
- **Context window tracking**: See how much of the model's context you're using
- **Token estimation**: Automatic estimation for providers that don't report usage
- **Capability display**: Shows if model supports functions, vision, etc.
- **Improved formatting**: Cleaner, more informative telemetry output
#### 🎯 Enhanced Model Metadata (Opt-in)
- **Model capabilities**: Access context length, max tokens, supported features
- **Cost estimation**: Calculate costs before making API calls
- **Parameter validation**: Automatic validation against model limits
- **Backward compatible**: Opt-in feature with `enhanced=True`
#### 🔧 Developer Experience
- **Logging control**: Suppress logs without `--verbose` flag
- **Better error messages**: More context and helpful suggestions
- **Improved callback handling**: Fixed telemetry in enhanced mode
### v2.1.0 Features
#### 🔐 Environment Variable Authentication
- Zero-touch auth for CI/CD pipelines
- Support for all providers via env vars
- Automatic token handling
#### 🌊 Streaming Support
- Real-time response streaming
- Automatic auth refresh during streams
- CLI and API streaming capabilities
### v2.0 Features
### 🎯 Telemetry & Cost Tracking
- **Token usage monitoring**: See exactly how many tokens each request uses
- **Cost estimation**: Real-time cost calculation for supported providers
- For GitHub Copilot: Shows reference costs based on equivalent OpenAI models (subscription-based service)
- **Performance metrics**: Request duration and model response times
- **Configurable display**: Enable/disable telemetry output globally or per-command
### 📥 Flexible Input/Output
- **Multiple input sources**: Command line, files, or stdin
- **File output**: Save responses directly to files
- **Streaming support**: Pipe commands together for automation
- **Q&A formatting**: Clean, readable output for interactive use
### 🏗️ Simplified Architecture
- **Cleaner codebase**: Removed complex decorators and factory patterns
- **Direct error handling**: Clear, actionable error messages
- **Improved test coverage**: Comprehensive test suite with 90%+ coverage
- **Better maintainability**: Simplified patterns for easier contribution
### 🔧 Enhanced CLI
- **Settings management**: Global configuration for telemetry and preferences
- **Improved error messages**: Context and suggestions for common issues
- **Better help text**: More descriptive command documentation
- **Consistent output**: Unified formatting across all commands
- **Provider name flexibility**: Both `github-copilot` and `github_copilot` formats supported
## Python API
### Basic Usage
```python
from modelforge.registry import ModelForgeRegistry
# Initialize registry
registry = ModelForgeRegistry()
# Get currently configured model
llm = registry.get_llm()
# Use directly with LangChain
from langchain_core.prompts import ChatPromptTemplate
prompt = ChatPromptTemplate.from_messages([("human", "{input}")])
chain = prompt | llm
response = chain.invoke({"input": "Tell me a joke"})
print(response)
```
### Advanced Usage with Telemetry (NEW in v2.0)
```python
from modelforge.registry import ModelForgeRegistry
from modelforge.telemetry import TelemetryCallback
# Initialize with debug logging
registry = ModelForgeRegistry(verbose=True)
# Create telemetry callback
telemetry = TelemetryCallback(provider="openai", model="gpt-4o-mini")
# Get model with telemetry tracking
llm = registry.get_llm(
provider_name="openai",
model_alias="gpt-4o-mini",
callbacks=[telemetry]
)
# Use with full LangChain features
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
# Create complex chains
prompt = ChatPromptTemplate.from_template("Explain {topic} in simple terms")
chain = prompt | llm | StrOutputParser()
# Use with streaming
for chunk in chain.stream({"topic": "quantum computing"}):
print(chunk, end="", flush=True)
# Batch processing
questions = [
"What is machine learning?",
"Explain neural networks",
"How does backpropagation work?"
]
responses = chain.batch([{"topic": q} for q in questions])
# Access telemetry data after execution
print(f"Tokens used: {telemetry.metrics.token_usage.total_tokens}")
print(f"Duration: {telemetry.metrics.duration_ms:.0f}ms")
print(f"Estimated cost: ${telemetry.metrics.estimated_cost:.6f}")
# Format telemetry for display
from modelforge.telemetry import format_metrics
print(format_metrics(telemetry.metrics))
```
### Enhanced Model Metadata (v2.2.0 - Opt-in Feature)
```python
from modelforge import ModelForgeRegistry
# Enable enhanced features
registry = ModelForgeRegistry()
llm = registry.get_llm("openai", "gpt-4o", enhanced=True)
# Access model metadata
print(f"Context window: {llm.context_length:,} tokens")
print(f"Max output: {llm.max_output_tokens:,} tokens")
print(f"Supports functions: {llm.supports_function_calling}")
print(f"Supports vision: {llm.supports_vision}")
# Get pricing information
pricing = llm.pricing_info
print(f"Input cost: ${pricing['input_per_1m']}/1M tokens")
print(f"Output cost: ${pricing['output_per_1m']}/1M tokens")
# Estimate costs before making calls
estimated_cost = llm.estimate_cost(input_tokens=5000, output_tokens=1000)
print(f"Estimated cost for this request: ${estimated_cost:.4f}")
# Configure parameters with validation
llm.temperature = 0.7 # Validated against model limits
llm.max_tokens = 2000 # Checked against model's max_output_tokens
# Note: This is opt-in for now. In future versions, enhanced=True may become default
# To maintain current behavior, explicitly use enhanced=False
```
### Configuration Management
```python
from modelforge import config
# Get current model selection
current = config.get_current_model()
print(f"Current: {current.get('provider')}/{current.get('model')}")
# Check if models are configured
if not current:
print("No model selected. Configure with:")
print("modelforge config add --provider openai --model gpt-4o-mini")
# Manage settings (NEW in v2.0)
settings = config.get_settings()
print(f"Telemetry enabled: {settings.get('show_telemetry', True)}")
# Update settings
config.update_setting("show_telemetry", False) # Disable telemetry
```
### Error Handling
```python
from modelforge.registry import ModelForgeRegistry
from modelforge.exceptions import ConfigurationError, ProviderError
try:
registry = ModelForgeRegistry()
llm = registry.get_llm()
response = llm.invoke("Hello world")
except ConfigurationError as e:
print(f"Configuration issue: {e}")
print("Run: modelforge config add --provider PROVIDER --model MODEL")
except ProviderError as e:
print(f"Provider error: {e}")
print("Check: modelforge auth status")
```
### Streaming Support (v2.1+)
ModelForge provides enhanced streaming capabilities with automatic authentication handling:
```python
from modelforge.registry import ModelForgeRegistry
from modelforge.streaming import stream
# Initialize with your LLM
registry = ModelForgeRegistry()
llm = registry.get_llm()
# Stream responses with auth handling
async for chunk in stream(llm, "Write a story about AI",
provider_name="openai",
provider_data=registry._config.get("providers", {}).get("openai")):
print(chunk, end="", flush=True)
# Stream to file with automatic token refresh
from modelforge.streaming import stream_to_file
from pathlib import Path
await stream_to_file(llm, "Explain quantum computing",
Path("output.txt"),
provider_name="github_copilot",
provider_data=provider_data)
```
**CLI Streaming:**
```bash
# Stream responses in real-time
modelforge test --prompt "Write a story" --stream
# Stream to file
modelforge test --prompt "Explain AI" --stream --output-file response.txt
```
**Key Features:**
- Automatic token refresh for OAuth providers during long streams
- Environment variable authentication support
- Retry on authentication errors
- Progress callbacks and buffering options
**Note on Streaming Behavior:**
The actual streaming granularity depends on the provider's API implementation. Some providers (like GitHub Copilot) may return responses in larger chunks rather than token-by-token streaming, while others (like Ollama) support finer-grained streaming.
## Supported Providers
- **OpenAI**: GPT-4, GPT-4o, GPT-3.5-turbo
- **Google**: Gemini Pro, Gemini Flash
- **Ollama**: Local models (Llama, Qwen, Mistral)
- **GitHub Copilot**: Claude, GPT models via GitHub (use `github_copilot` or `github-copilot`)
## Authentication
ModelForge supports multiple authentication methods:
- **API Keys**: Store securely in configuration
- **Device Flow**: Browser-based OAuth for GitHub Copilot
- **No Auth**: For local models like Ollama
- **Environment Variables**: Zero-touch authentication for CI/CD (NEW in v2.1)
### Authentication Methods
```bash
# API Key authentication
modelforge auth login --provider openai --api-key YOUR_KEY
# Device flow (GitHub Copilot)
modelforge auth login --provider github_copilot
# Check auth status
modelforge auth status
```
### Environment Variable Support (v2.1+)
For CI/CD and production deployments, you can use environment variables to provide credentials without storing them in configuration files:
```bash
# API Key providers
export MODELFORGE_OPENAI_API_KEY="sk-..."
export MODELFORGE_ANTHROPIC_API_KEY="sk-ant-..."
export MODELFORGE_GOOGLE_API_KEY="..."
# OAuth providers
export MODELFORGE_GITHUB_COPILOT_ACCESS_TOKEN="ghu_..."
# Use models without manual authentication
modelforge test --prompt "Hello"
```
Environment variables take precedence over stored credentials and eliminate the need for interactive authentication.
## Configuration
ModelForge uses a two-tier configuration system:
- **Global**: `~/.config/model-forge/config.json` (user-wide)
- **Local**: `./.model-forge/config.json` (project-specific)
Local config takes precedence over global when both exist.
## Model Discovery
```bash
# List all available models
modelforge models list
# Search models by name or capability
modelforge models search "gpt"
# Get detailed model info
modelforge models info --provider openai --model gpt-4o
```
## Development Setup
For contributors and developers:
```bash
git clone https://github.com/smiao-icims/model-forge.git
cd model-forge
# Quick setup with uv (recommended)
./setup.sh
# Or manual setup
uv sync --extra dev
uv run pytest
```
**Requirements:**
- Python 3.11+
- [uv](https://docs.astral.sh/uv/) (modern Python package manager)
See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed development guidelines.
## Documentation
- [Models.dev](https://models.dev) - Comprehensive model reference
- [GitHub Issues](https://github.com/smiao-icims/model-forge/issues) - Support and bug reports
## License
MIT License - see LICENSE file for details.
Raw data
{
"_id": null,
"home_page": null,
"name": "model-forge-llm",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.11",
"maintainer_email": null,
"keywords": "llm, ai, langchain, openai, ollama, providers, authentication",
"author": "ModelForge Contributors",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/20/ad/298a01c584da1981b00a577c3aaf5344edc441088c2aeb83ba9cde5f7500/model_forge_llm-2.3.0.tar.gz",
"platform": null,
"description": "# ModelForge\n\nA Python library for managing LLM providers, authentication, and model selection with seamless LangChain integration.\n\n[](https://badge.fury.io/py/model-forge-llm)\n[](https://www.python.org/downloads/)\n[](LICENSE)\n\n**\ud83d\ude80 Version 2.2.0 - Enhanced Model Metadata, Improved Telemetry, and Quiet Mode!**\n\n## Installation\n\n### Recommended: Virtual Environment\n```bash\n# Create and activate virtual environment\npython -m venv model-forge-env\nsource model-forge-env/bin/activate # On Windows: model-forge-env\\Scripts\\activate\n\n# Install package\npip install model-forge-llm\n\n# Verify installation\nmodelforge --help\n```\n\n### Quick Install (System-wide)\n```bash\npip install model-forge-llm\n```\n\n## Quick Start\n\n### \ud83d\ude80 NEW: Interactive Configuration Wizard (Easiest)\n```bash\n# Launch the interactive wizard\nmodelforge config\n\n# The wizard will guide you through:\n# 1. Choosing a provider (with recommendations)\n# 2. Setting up authentication\n# 3. Selecting a model\n# 4. Testing your configuration\n```\n\n### Option 1: GitHub Copilot via Device Authentication Flow\n```bash\n# Discover GitHub Copilot models\nmodelforge models list --provider github_copilot\n\n# Set up GitHub Copilot with device authentication\nmodelforge auth login --provider github_copilot\n\n# Select Claude 3.7 Sonnet via GitHub Copilot\nmodelforge config use --provider github_copilot --model claude-3.7-sonnet\n\n# Test your setup\nmodelforge test --prompt \"Write a Python function to reverse a string\"\n```\n\n### Option 2: OpenAI (API Key Required)\n```bash\n# Add OpenAI with your API key\nmodelforge auth login --provider openai --api-key YOUR_API_KEY\n\n# Select GPT-4o-mini\nmodelforge config use --provider openai --model gpt-4o-mini\n\n# Test your setup\nmodelforge test --prompt \"Hello, world!\"\n```\n\n### Option 3: Local Ollama (No API Key Needed)\n```bash\n# Make sure Ollama is running locally\n# Then add a local model\nmodelforge config add --provider ollama --model qwen3:1.7b\n\n# Select the local model\nmodelforge config use --provider ollama --model qwen3:1.7b\n\n# Test your setup\nmodelforge test --prompt \"What is machine learning?\"\n```\n\n### Common Commands - Complete Lifecycle\n```bash\n# Installation & Setup\nmodelforge --help # Verify installation\nmodelforge config show # View current config\n\n# Model Discovery & Selection\nmodelforge models list # List all available models\nmodelforge models search \"claude\" # Search models by name\nmodelforge models info --provider openai --model gpt-4o # Get model details\n\n# Authentication Management\nmodelforge auth login --provider openai --api-key KEY # API key auth\nmodelforge auth login --provider github_copilot # Device flow auth\nmodelforge auth status # Check auth status\nmodelforge auth logout --provider openai # Remove credentials\n\n# Configuration Management\n# Interactive Configuration Wizard (NEW in v2.2.2)\nmodelforge config # Launch interactive wizard\nmodelforge config --wizard # Alternative wizard launch\nmodelforge config --wizard --verbose # Debug mode with logs\n\n# Manual Configuration\nmodelforge config add --provider openai --model gpt-4o-mini --api-key KEY\nmodelforge config add --provider ollama --model qwen3:1.7b --local\nmodelforge config use --provider openai --model gpt-4o-mini\nmodelforge config remove --provider openai --model gpt-4o-mini\n\n# Testing & Usage (NEW in v2.2: Quiet mode for automation)\nmodelforge test --prompt \"Hello, how are you?\" # Test current model\nmodelforge test --prompt \"Explain quantum computing\" --verbose # Debug mode\nmodelforge test --input-file prompt.txt --output-file response.txt # File I/O\necho \"What is AI?\" | modelforge test # Stdin input\nmodelforge test --prompt \"Hello\" --no-telemetry # Disable telemetry\nmodelforge test --prompt \"What is 2+2?\" --quiet # Minimal output (v2.2)\necho \"Hello\" | modelforge test --quiet > output.txt # Perfect for piping\n\n# Cache & Maintenance\nmodelforge models list --refresh # Force refresh from models.dev\n\n# Telemetry Settings (NEW in v2.0)\nmodelforge settings telemetry on # Enable telemetry display\nmodelforge settings telemetry off # Disable telemetry display\nmodelforge settings telemetry status # Check current setting\n```\n\n## What's New\n\n### v2.2.2 Features\n\n#### \ud83e\uddd9 Interactive Configuration Wizard\n- **Easy setup**: Run `modelforge config` to launch the interactive wizard\n- **Visual guidance**: Arrow key navigation with clear indicators\n - \u2b50 = Recommended providers\n - \u2705 = Already configured\n- **Smart defaults**: Automatically selects current provider/model\n- **Flexible authentication**: Choose to use existing credentials or re-authenticate\n- **Live testing**: Test configuration with a sample prompt before saving\n- **Quiet by default**: No verbose logs unless `--verbose` flag is used\n\n### v2.2.0 Features\n\n#### \ud83e\udd2b Quiet Mode for Automation\n- **`--quiet` flag**: Minimal output showing only the model response\n- **Perfect for piping**: Clean output for scripts and automation\n- **Automatic telemetry suppression**: No metadata in quiet mode\n- **Conflict prevention**: Cannot use with `--verbose` flag\n\n#### \ud83d\udcca Enhanced Telemetry Display\n- **Context window tracking**: See how much of the model's context you're using\n- **Token estimation**: Automatic estimation for providers that don't report usage\n- **Capability display**: Shows if model supports functions, vision, etc.\n- **Improved formatting**: Cleaner, more informative telemetry output\n\n#### \ud83c\udfaf Enhanced Model Metadata (Opt-in)\n- **Model capabilities**: Access context length, max tokens, supported features\n- **Cost estimation**: Calculate costs before making API calls\n- **Parameter validation**: Automatic validation against model limits\n- **Backward compatible**: Opt-in feature with `enhanced=True`\n\n#### \ud83d\udd27 Developer Experience\n- **Logging control**: Suppress logs without `--verbose` flag\n- **Better error messages**: More context and helpful suggestions\n- **Improved callback handling**: Fixed telemetry in enhanced mode\n\n### v2.1.0 Features\n\n#### \ud83d\udd10 Environment Variable Authentication\n- Zero-touch auth for CI/CD pipelines\n- Support for all providers via env vars\n- Automatic token handling\n\n#### \ud83c\udf0a Streaming Support\n- Real-time response streaming\n- Automatic auth refresh during streams\n- CLI and API streaming capabilities\n\n### v2.0 Features\n\n### \ud83c\udfaf Telemetry & Cost Tracking\n- **Token usage monitoring**: See exactly how many tokens each request uses\n- **Cost estimation**: Real-time cost calculation for supported providers\n - For GitHub Copilot: Shows reference costs based on equivalent OpenAI models (subscription-based service)\n- **Performance metrics**: Request duration and model response times\n- **Configurable display**: Enable/disable telemetry output globally or per-command\n\n### \ud83d\udce5 Flexible Input/Output\n- **Multiple input sources**: Command line, files, or stdin\n- **File output**: Save responses directly to files\n- **Streaming support**: Pipe commands together for automation\n- **Q&A formatting**: Clean, readable output for interactive use\n\n### \ud83c\udfd7\ufe0f Simplified Architecture\n- **Cleaner codebase**: Removed complex decorators and factory patterns\n- **Direct error handling**: Clear, actionable error messages\n- **Improved test coverage**: Comprehensive test suite with 90%+ coverage\n- **Better maintainability**: Simplified patterns for easier contribution\n\n### \ud83d\udd27 Enhanced CLI\n- **Settings management**: Global configuration for telemetry and preferences\n- **Improved error messages**: Context and suggestions for common issues\n- **Better help text**: More descriptive command documentation\n- **Consistent output**: Unified formatting across all commands\n- **Provider name flexibility**: Both `github-copilot` and `github_copilot` formats supported\n\n## Python API\n\n### Basic Usage\n\n```python\nfrom modelforge.registry import ModelForgeRegistry\n\n# Initialize registry\nregistry = ModelForgeRegistry()\n\n# Get currently configured model\nllm = registry.get_llm()\n\n# Use directly with LangChain\nfrom langchain_core.prompts import ChatPromptTemplate\n\nprompt = ChatPromptTemplate.from_messages([(\"human\", \"{input}\")])\nchain = prompt | llm\nresponse = chain.invoke({\"input\": \"Tell me a joke\"})\nprint(response)\n```\n\n### Advanced Usage with Telemetry (NEW in v2.0)\n\n```python\nfrom modelforge.registry import ModelForgeRegistry\nfrom modelforge.telemetry import TelemetryCallback\n\n# Initialize with debug logging\nregistry = ModelForgeRegistry(verbose=True)\n\n# Create telemetry callback\ntelemetry = TelemetryCallback(provider=\"openai\", model=\"gpt-4o-mini\")\n\n# Get model with telemetry tracking\nllm = registry.get_llm(\n provider_name=\"openai\",\n model_alias=\"gpt-4o-mini\",\n callbacks=[telemetry]\n)\n\n# Use with full LangChain features\nfrom langchain_core.prompts import ChatPromptTemplate\nfrom langchain_core.output_parsers import StrOutputParser\n\n# Create complex chains\nprompt = ChatPromptTemplate.from_template(\"Explain {topic} in simple terms\")\nchain = prompt | llm | StrOutputParser()\n\n# Use with streaming\nfor chunk in chain.stream({\"topic\": \"quantum computing\"}):\n print(chunk, end=\"\", flush=True)\n\n# Batch processing\nquestions = [\n \"What is machine learning?\",\n \"Explain neural networks\",\n \"How does backpropagation work?\"\n]\nresponses = chain.batch([{\"topic\": q} for q in questions])\n\n# Access telemetry data after execution\nprint(f\"Tokens used: {telemetry.metrics.token_usage.total_tokens}\")\nprint(f\"Duration: {telemetry.metrics.duration_ms:.0f}ms\")\nprint(f\"Estimated cost: ${telemetry.metrics.estimated_cost:.6f}\")\n\n# Format telemetry for display\nfrom modelforge.telemetry import format_metrics\nprint(format_metrics(telemetry.metrics))\n```\n\n### Enhanced Model Metadata (v2.2.0 - Opt-in Feature)\n\n```python\nfrom modelforge import ModelForgeRegistry\n\n# Enable enhanced features\nregistry = ModelForgeRegistry()\nllm = registry.get_llm(\"openai\", \"gpt-4o\", enhanced=True)\n\n# Access model metadata\nprint(f\"Context window: {llm.context_length:,} tokens\")\nprint(f\"Max output: {llm.max_output_tokens:,} tokens\")\nprint(f\"Supports functions: {llm.supports_function_calling}\")\nprint(f\"Supports vision: {llm.supports_vision}\")\n\n# Get pricing information\npricing = llm.pricing_info\nprint(f\"Input cost: ${pricing['input_per_1m']}/1M tokens\")\nprint(f\"Output cost: ${pricing['output_per_1m']}/1M tokens\")\n\n# Estimate costs before making calls\nestimated_cost = llm.estimate_cost(input_tokens=5000, output_tokens=1000)\nprint(f\"Estimated cost for this request: ${estimated_cost:.4f}\")\n\n# Configure parameters with validation\nllm.temperature = 0.7 # Validated against model limits\nllm.max_tokens = 2000 # Checked against model's max_output_tokens\n\n# Note: This is opt-in for now. In future versions, enhanced=True may become default\n# To maintain current behavior, explicitly use enhanced=False\n```\n\n### Configuration Management\n\n```python\nfrom modelforge import config\n\n# Get current model selection\ncurrent = config.get_current_model()\nprint(f\"Current: {current.get('provider')}/{current.get('model')}\")\n\n# Check if models are configured\nif not current:\n print(\"No model selected. Configure with:\")\n print(\"modelforge config add --provider openai --model gpt-4o-mini\")\n\n# Manage settings (NEW in v2.0)\nsettings = config.get_settings()\nprint(f\"Telemetry enabled: {settings.get('show_telemetry', True)}\")\n\n# Update settings\nconfig.update_setting(\"show_telemetry\", False) # Disable telemetry\n```\n\n### Error Handling\n\n```python\nfrom modelforge.registry import ModelForgeRegistry\nfrom modelforge.exceptions import ConfigurationError, ProviderError\n\ntry:\n registry = ModelForgeRegistry()\n llm = registry.get_llm()\n response = llm.invoke(\"Hello world\")\nexcept ConfigurationError as e:\n print(f\"Configuration issue: {e}\")\n print(\"Run: modelforge config add --provider PROVIDER --model MODEL\")\nexcept ProviderError as e:\n print(f\"Provider error: {e}\")\n print(\"Check: modelforge auth status\")\n```\n\n### Streaming Support (v2.1+)\n\nModelForge provides enhanced streaming capabilities with automatic authentication handling:\n\n```python\nfrom modelforge.registry import ModelForgeRegistry\nfrom modelforge.streaming import stream\n\n# Initialize with your LLM\nregistry = ModelForgeRegistry()\nllm = registry.get_llm()\n\n# Stream responses with auth handling\nasync for chunk in stream(llm, \"Write a story about AI\",\n provider_name=\"openai\",\n provider_data=registry._config.get(\"providers\", {}).get(\"openai\")):\n print(chunk, end=\"\", flush=True)\n\n# Stream to file with automatic token refresh\nfrom modelforge.streaming import stream_to_file\nfrom pathlib import Path\n\nawait stream_to_file(llm, \"Explain quantum computing\",\n Path(\"output.txt\"),\n provider_name=\"github_copilot\",\n provider_data=provider_data)\n```\n\n**CLI Streaming:**\n```bash\n# Stream responses in real-time\nmodelforge test --prompt \"Write a story\" --stream\n\n# Stream to file\nmodelforge test --prompt \"Explain AI\" --stream --output-file response.txt\n```\n\n**Key Features:**\n- Automatic token refresh for OAuth providers during long streams\n- Environment variable authentication support\n- Retry on authentication errors\n- Progress callbacks and buffering options\n\n**Note on Streaming Behavior:**\nThe actual streaming granularity depends on the provider's API implementation. Some providers (like GitHub Copilot) may return responses in larger chunks rather than token-by-token streaming, while others (like Ollama) support finer-grained streaming.\n\n## Supported Providers\n\n- **OpenAI**: GPT-4, GPT-4o, GPT-3.5-turbo\n- **Google**: Gemini Pro, Gemini Flash\n- **Ollama**: Local models (Llama, Qwen, Mistral)\n- **GitHub Copilot**: Claude, GPT models via GitHub (use `github_copilot` or `github-copilot`)\n\n## Authentication\n\nModelForge supports multiple authentication methods:\n\n- **API Keys**: Store securely in configuration\n- **Device Flow**: Browser-based OAuth for GitHub Copilot\n- **No Auth**: For local models like Ollama\n- **Environment Variables**: Zero-touch authentication for CI/CD (NEW in v2.1)\n\n### Authentication Methods\n\n```bash\n# API Key authentication\nmodelforge auth login --provider openai --api-key YOUR_KEY\n\n# Device flow (GitHub Copilot)\nmodelforge auth login --provider github_copilot\n\n# Check auth status\nmodelforge auth status\n```\n\n### Environment Variable Support (v2.1+)\n\nFor CI/CD and production deployments, you can use environment variables to provide credentials without storing them in configuration files:\n\n```bash\n# API Key providers\nexport MODELFORGE_OPENAI_API_KEY=\"sk-...\"\nexport MODELFORGE_ANTHROPIC_API_KEY=\"sk-ant-...\"\nexport MODELFORGE_GOOGLE_API_KEY=\"...\"\n\n# OAuth providers\nexport MODELFORGE_GITHUB_COPILOT_ACCESS_TOKEN=\"ghu_...\"\n\n# Use models without manual authentication\nmodelforge test --prompt \"Hello\"\n```\n\nEnvironment variables take precedence over stored credentials and eliminate the need for interactive authentication.\n\n## Configuration\n\nModelForge uses a two-tier configuration system:\n\n- **Global**: `~/.config/model-forge/config.json` (user-wide)\n- **Local**: `./.model-forge/config.json` (project-specific)\n\nLocal config takes precedence over global when both exist.\n\n## Model Discovery\n\n```bash\n# List all available models\nmodelforge models list\n\n# Search models by name or capability\nmodelforge models search \"gpt\"\n\n# Get detailed model info\nmodelforge models info --provider openai --model gpt-4o\n```\n\n## Development Setup\n\nFor contributors and developers:\n\n```bash\ngit clone https://github.com/smiao-icims/model-forge.git\ncd model-forge\n\n# Quick setup with uv (recommended)\n./setup.sh\n\n# Or manual setup\nuv sync --extra dev\nuv run pytest\n```\n\n**Requirements:**\n- Python 3.11+\n- [uv](https://docs.astral.sh/uv/) (modern Python package manager)\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for detailed development guidelines.\n\n## Documentation\n\n- [Models.dev](https://models.dev) - Comprehensive model reference\n- [GitHub Issues](https://github.com/smiao-icims/model-forge/issues) - Support and bug reports\n\n## License\n\nMIT License - see LICENSE file for details.\n",
"bugtrack_url": null,
"license": "MIT License\n \n Copyright (c) 2025 Shuhai Miao\n \n Permission is hereby granted, free of charge, to any person obtaining a copy\n of this software and associated documentation files (the \"Software\"), to deal\n in the Software without restriction, including without limitation the rights\n to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n copies of the Software, and to permit persons to whom the Software is\n furnished to do so, subject to the following conditions:\n \n The above copyright notice and this permission notice shall be included in all\n copies or substantial portions of the Software.\n \n THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n SOFTWARE.\n ",
"summary": "A reusable library for managing LLM providers, authentication, and model selection.",
"version": "2.3.0",
"project_urls": {
"Documentation": "https://model-forge.readthedocs.io",
"Homepage": "https://github.com/smiao-icims/model-forge",
"Issues": "https://github.com/smiao-icims/model-forge/issues",
"Repository": "https://github.com/smiao-icims/model-forge"
},
"split_keywords": [
"llm",
" ai",
" langchain",
" openai",
" ollama",
" providers",
" authentication"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "a414889887f4dd38e791c549db8e2164b76b4b3cc5ee60b3c31875b8a65418c3",
"md5": "7c0eebc38f6ee32c6afb210e664dec0b",
"sha256": "bfc200b96e3d54e8cb1bd447c891391edcb6e64d3b7de9537969b506351abb55"
},
"downloads": -1,
"filename": "model_forge_llm-2.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7c0eebc38f6ee32c6afb210e664dec0b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.11",
"size": 60221,
"upload_time": "2025-07-28T15:34:32",
"upload_time_iso_8601": "2025-07-28T15:34:32.977669Z",
"url": "https://files.pythonhosted.org/packages/a4/14/889887f4dd38e791c549db8e2164b76b4b3cc5ee60b3c31875b8a65418c3/model_forge_llm-2.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "20ad298a01c584da1981b00a577c3aaf5344edc441088c2aeb83ba9cde5f7500",
"md5": "09ddcb889dd835a6c573974b84e777ba",
"sha256": "195435998acc09935a78fbe49876868475d106a75aa89ef60d3a2ae7cb3c1509"
},
"downloads": -1,
"filename": "model_forge_llm-2.3.0.tar.gz",
"has_sig": false,
"md5_digest": "09ddcb889dd835a6c573974b84e777ba",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.11",
"size": 63038,
"upload_time": "2025-07-28T15:34:34",
"upload_time_iso_8601": "2025-07-28T15:34:34.131427Z",
"url": "https://files.pythonhosted.org/packages/20/ad/298a01c584da1981b00a577c3aaf5344edc441088c2aeb83ba9cde5f7500/model_forge_llm-2.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-28 15:34:34",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "smiao-icims",
"github_project": "model-forge",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "model-forge-llm"
}
