# Ollama to OpenAI Proxy
[](https://github.com/eyalrot/ollama_openai/releases)
[](https://pypi.org/project/ollama-openai-proxy/)
[](https://pypi.org/project/ollama-openai-proxy/)
[](https://github.com/eyalrot/ollama_openai/actions/workflows/ci.yml)
[](https://codecov.io/gh/eyalrot/ollama_openai)
[](https://github.com/eyalrot/ollama_openai/actions/workflows/security.yml)
[](https://github.com/eyalrot/ollama_openai/actions/workflows/docker.yml)
[](https://github.com/eyalrot/ollama_openai/pkgs/container/ollama_openai)
[](https://hub.docker.com/r/eyalrot2/ollama-openai-proxy)
[](https://opensource.org/licenses/MIT)
A transparent proxy service that allows applications to use both Ollama and OpenAI API formats seamlessly with OpenAI-compatible LLM servers like **OpenAI**, **vLLM**, **LiteLLM**, **OpenRouter**, **Ollama**, and any other OpenAI-compatible API provider.
**Perfect for N8N**: Enables using N8N's Ollama model node against OpenAI-compatible API servers. N8N's OpenAI models only support the hardcoded OpenAI URL (https://api.openai.com/v1) and cannot be configured, but the Ollama model node allows custom endpoints - making this proxy ideal for connecting N8N to any OpenAI-compatible LLM provider.
## Features
- ✅ Drop-in replacement for Ollama server
- ✅ Zero changes required to existing code
- ✅ **Dual API format support**: Both Ollama and OpenAI endpoints
- ✅ Supports text generation and chat endpoints
- ✅ Streaming and non-streaming responses
- ✅ Model listing from backend
- ✅ Configurable model name mapping
- ✅ Docker and standalone deployment
- ✅ Automatic retry with exponential backoff
- ✅ Comprehensive logging and monitoring
- ✅ Request ID tracking for debugging
- ✅ Phase 1: Text-only chat and embeddings (completed)
- ✅ Phase 2: Tool calling support (completed)
- ✅ Phase 2: Image input support (completed)
## Table of Contents
- [Quick Start](#quick-start)
- [Docker Images](#docker-images)
- [Configuration](#configuration)
- [API Compatibility](#api-compatibility)
- [Model Mapping](#model-mapping)
- [Deployment](#deployment)
- [Testing](#testing)
- [Troubleshooting](#troubleshooting)
- [Development](#development)
- [Documentation](#documentation)
- [Security & Compliance](#security--compliance)
- [Contributing](#contributing)
- [License](#license)
## Quick Start
Get started in under 5 minutes! See the [Quick Start Guide](docs/QUICK_START.md) for detailed instructions.
### Using Docker (Recommended)
```bash
# Clone and configure
git clone https://github.com/eyalrot/ollama_openai.git
cd ollama_openai
cp .env.example .env
# Edit .env with your API details
nano .env
# Start the proxy
docker-compose up -d
# Verify it's working
curl http://localhost:11434/health
```
### Using PyPI Package (Recommended)
```bash
# Install from PyPI
pip install ollama-openai-proxy
# Create configuration file
cat > .env << EOF
OPENAI_API_BASE_URL=https://api.openai.com/v1
OPENAI_API_KEY=your-api-key-here
EOF
# Run the proxy (method 1: using installed command)
ollama-openai-proxy
# Or run using Python module (method 2)
python -c "from src.main import main; main()"
```
### Using Python Source
```bash
# Setup
git clone https://github.com/eyalrot/ollama_openai.git
cd ollama_openai
pip install -r requirements.txt
# Configure
cp .env.example .env
nano .env
# Run
python -m uvicorn src.main:app --host 0.0.0.0 --port 11434
```
### Quick Test
```bash
# Check version and health
curl http://localhost:11434/v1/version
curl http://localhost:11434/v1/health
```
```python
# Option 1: Use Ollama client (existing code works unchanged)
from ollama import Client
client = Client(host='http://localhost:11434')
response = client.generate(model='gpt-3.5-turbo', prompt='Hello!')
print(response['response'])
# Option 2: Use OpenAI client (new in v0.6.0!)
import openai
openai.api_base = "http://localhost:11434/v1"
openai.api_key = "your-api-key"
response = openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)
```
For more examples and detailed setup instructions, see the [Quick Start Guide](docs/QUICK_START.md).
## Docker Images
### Pre-built Docker Images
Ready-to-use production images are available on both Docker Hub and GitHub Container Registry:
#### Docker Hub 🐳 (Recommended)
```bash
# Pull and run latest
docker pull eyalrot2/ollama-openai-proxy:latest
docker run -d -p 11434:11434 \
-e OPENAI_API_BASE_URL=https://openrouter.ai/api/v1 \
-e OPENAI_API_KEY=your_key \
eyalrot2/ollama-openai-proxy:latest
# Or use specific version
docker pull eyalrot2/ollama-openai-proxy:0.6.3
# Available tags: latest, 0.6.3, 0.6, 0
```
#### GitHub Container Registry 📦
```bash
# Pull and run latest
docker pull ghcr.io/eyalrot/ollama_openai:latest
docker run -d -p 11434:11434 \
-e OPENAI_API_BASE_URL=https://openrouter.ai/api/v1 \
-e OPENAI_API_KEY=your_key \
ghcr.io/eyalrot/ollama_openai:latest
# Or use specific version
docker pull ghcr.io/eyalrot/ollama_openai:0.6.3
# Available tags: latest, 0.6.3, 0.6, 0
```
#### Multi-Architecture Support 🏗️
- **linux/amd64** (Intel/AMD processors)
- **linux/arm64** (ARM processors, Apple Silicon, Raspberry Pi)
### Docker Compose with Pre-built Images
```yaml
services:
ollama-proxy:
# Use Docker Hub (recommended)
image: eyalrot2/ollama-openai-proxy:latest
# Or use GitHub Container Registry
# image: ghcr.io/eyalrot/ollama_openai:latest
ports:
- "11434:11434"
environment:
- OPENAI_API_BASE_URL=https://openrouter.ai/api/v1
- OPENAI_API_KEY=your_openrouter_key
- LOG_LEVEL=INFO
restart: unless-stopped
```
### Image Features
- **Size**: 271MB (optimized production build)
- **Security**: Non-root user, read-only filesystem, no-new-privileges
- **Performance**: Multi-stage build with optimized dependencies
- **Compatibility**: Supports **OpenAI**, **vLLM**, **LiteLLM**, **OpenRouter**, **Ollama**, and any OpenAI-compatible API provider
- **SSL Support**: System SSL certificates included for private endpoints
### Available Tags
| Tag | Description | Docker Hub | GitHub Container Registry |
|-----|-------------|------------|---------------------------|
| `latest` | Latest stable build | `eyalrot2/ollama-openai-proxy:latest` | `ghcr.io/eyalrot/ollama_openai:latest` |
| `0.6.3` | Specific version | `eyalrot2/ollama-openai-proxy:0.6.3` | `ghcr.io/eyalrot/ollama_openai:0.6.3` |
| `0.6` | Major.minor version | `eyalrot2/ollama-openai-proxy:0.6` | `ghcr.io/eyalrot/ollama_openai:0.6` |
| `0` | Major version | `eyalrot2/ollama-openai-proxy:0` | `ghcr.io/eyalrot/ollama_openai:0` |
### Quick Test with Pre-built Image
```bash
# Start with OpenRouter free models (using Docker Hub)
docker run -d --name ollama-proxy -p 11434:11434 \
-e OPENAI_API_BASE_URL=https://openrouter.ai/api/v1 \
-e OPENAI_API_KEY=your_key \
eyalrot2/ollama-openai-proxy:latest
# Or using GitHub Container Registry
# docker run -d --name ollama-proxy -p 11434:11434 \
# -e OPENAI_API_BASE_URL=https://openrouter.ai/api/v1 \
# -e OPENAI_API_KEY=your_key \
# ghcr.io/eyalrot/ollama_openai:latest
# Test with free model (Ollama format)
curl -X POST http://localhost:11434/api/generate \
-H "Content-Type: application/json" \
-d '{"model": "google/gemma-2-9b-it:free", "prompt": "Hello!"}'
# Or test with OpenAI format
curl -X POST http://localhost:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer your_key" \
-d '{"model": "google/gemma-2-9b-it:free", "messages": [{"role": "user", "content": "Hello!"}]}'
```
## Configuration
See the [Configuration Guide](docs/CONFIGURATION.md) for detailed setup instructions.
### Required Environment Variables
| Variable | Description | Example |
|----------|-------------|---------|
| `OPENAI_API_BASE_URL` | URL of your OpenAI-compatible server | `https://api.openai.com/v1` |
| `OPENAI_API_KEY` | API key for authentication | `sk-...` |
### Key Optional Settings
| Variable | Description | Default |
|----------|-------------|---------|
| `PROXY_PORT` | Port to run proxy on | `11434` |
| `LOG_LEVEL` | Logging verbosity | `INFO` |
| `REQUEST_TIMEOUT` | Request timeout in seconds | `60` |
| `MODEL_MAPPING_FILE` | **Optional**: Path to model mapping JSON. When not set, model names pass through unchanged to your provider | `None` (recommended) |
For all configuration options, validation rules, and examples, see the [Configuration Guide](docs/CONFIGURATION.md).
### Quick Testing with Different Providers
#### OpenRouter (Free Models Available)
```env
OPENAI_API_BASE_URL=https://openrouter.ai/api/v1
OPENAI_API_KEY=sk-or-v1-your-key
```
Free models: `google/gemma-2-9b-it:free`, `meta-llama/llama-3.2-3b-instruct:free`
#### OpenAI
```env
OPENAI_API_BASE_URL=https://api.openai.com/v1
OPENAI_API_KEY=sk-proj-your-key
```
#### vLLM Server
```env
OPENAI_API_BASE_URL=http://your-vllm-server:8000/v1
OPENAI_API_KEY=your-api-key-or-none
```
#### LiteLLM Proxy
```env
OPENAI_API_BASE_URL=http://your-litellm-proxy:4000
OPENAI_API_KEY=your-litellm-key
```
#### Local Ollama Server
```env
OPENAI_API_BASE_URL=http://localhost:11434/v1
OPENAI_API_KEY=ollama # or any value
```
## API Compatibility
See the [API Compatibility Matrix](docs/API_COMPATIBILITY.md) for detailed endpoint mappings and parameter translations.
### Supported Endpoints
| Endpoint | Method | Status | Description |
|----------|--------|---------|-------------|
| `/api/generate` | POST | ✅ Full Support | Text generation (Ollama-style) |
| `/api/chat` | POST | ✅ Full Support | Chat completion (Ollama-style) |
| `/api/tags` | GET | ✅ Full Support | List models |
| `/api/embeddings` | POST | ✅ Full Support | Generate embeddings (Ollama-style) |
### Dual API Format Support ✨
The proxy now supports **both Ollama and OpenAI API formats simultaneously**:
#### Ollama-Style Endpoints
- `/api/generate` - Text generation
- `/api/chat` - Chat completion
- `/api/embeddings` - Generate embeddings
#### OpenAI-Style Endpoints
- `/v1/chat/completions` - Chat completions
- `/v1/models` - List models
- `/v1/embeddings` - Generate embeddings
**Choose the format that works best for your application!** The proxy automatically detects the API format based on the URL path (`/api/*` vs `/v1/*`) and routes accordingly.
For detailed parameter mappings, response formats, and examples, see the [API Compatibility Matrix](docs/API_COMPATIBILITY.md).
## Phase 2 Features
### Tool Calling Support ✅
The proxy now supports full tool/function calling capabilities, allowing your AI models to execute tools and functions. This enables:
- **Function Definitions**: Define functions with JSON schema parameters
- **Tool Invocation**: Models can request to call tools during conversation
- **Bidirectional Translation**: Seamless translation between Ollama and OpenAI tool formats
- **Streaming Support**: Tool calls work with both streaming and non-streaming responses
```python
from ollama import Client
client = Client(host='http://localhost:11434')
# Define tools
tools = [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get weather information for a location",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string", "description": "City name"}
},
"required": ["location"]
}
}
}]
# Chat with tool support
response = client.chat(
model='gpt-4',
messages=[{"role": "user", "content": "What's the weather in Paris?"}],
tools=tools
)
```
### Image Input Support ✅
The proxy supports multimodal inputs, allowing you to send images along with text messages:
- **Base64 Images**: Send images as base64-encoded strings
- **Data URLs**: Support for data URL formatted images
- **Multiple Images**: Send multiple images in a single message
- **Mixed Content**: Combine text and images in conversations
```python
from ollama import Client
import base64
client = Client(host='http://localhost:11434')
# Load and encode image
with open("image.jpg", "rb") as f:
image_data = base64.b64encode(f.read()).decode()
# Send multimodal message
response = client.chat(
model='gpt-4-vision-preview',
messages=[{
"role": "user",
"content": "What do you see in this image?",
"images": [image_data]
}]
)
```
For comprehensive Phase 2 examples and integration guides, see the [examples/phase2/](examples/phase2/) directory.
## Examples
See the [examples/](examples/) directory for:
- Python client examples (Ollama SDK, OpenAI SDK, streaming, batch processing, LangChain)
- JavaScript/Node.js examples (both Ollama and OpenAI formats)
- Configuration templates
- Docker and Nginx setup examples
- Dual API format usage patterns
## Model Mapping
**Model mapping is completely optional.** By default, the proxy passes all model names through unchanged to your OpenAI-compatible provider, allowing direct use of provider-specific model names.
### Default Behavior: No Mapping Required ✅
**When `MODEL_MAPPING_FILE` is not configured (recommended for most users):**
- Model names are passed directly to your provider as-is
- No configuration needed - just use your provider's exact model names
- Perfect for **OpenAI**, **vLLM**, **LiteLLM**, **OpenRouter**, **Ollama**, and any OpenAI-compatible API
```bash
# Direct model usage (no mapping file needed)
# Ollama format:
curl -X POST http://localhost:11434/api/generate \
-H "Content-Type: application/json" \
-d '{"model": "google/gemma-2-9b-it:free", "prompt": "Hello!"}'
# OpenAI format:
curl -X POST http://localhost:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer your_key" \
-d '{"model": "gpt-4", "messages": [{"role": "user", "content": "Hello!"}]}'
# Both send model names directly to your OpenAI-compatible provider
```
### Optional: Custom Model Mapping
**Only configure model mapping if you want to create custom aliases:**
```json
{
"model_mappings": {
"llama2": "meta-llama/Llama-2-7b-chat-hf",
"gpt4": "gpt-4",
"free-gemma": "google/gemma-2-9b-it:free"
},
"default_model": "gpt-3.5-turbo"
}
```
Then set in environment:
```env
MODEL_MAPPING_FILE=./config/model_mapping.json
```
With mapping enabled, you can use aliases in both formats:
```bash
# Ollama format with alias "free-gemma" -> maps to "google/gemma-2-9b-it:free"
curl -X POST http://localhost:11434/api/generate \
-H "Content-Type: application/json" \
-d '{"model": "free-gemma", "prompt": "Hello!"}'
# OpenAI format with same alias
curl -X POST http://localhost:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer your_key" \
-d '{"model": "free-gemma", "messages": [{"role": "user", "content": "Hello!"}]}'
```
### When to Use Model Mapping
✅ **Use model mapping when:**
- You want shorter, memorable aliases for long model names
- Migrating from Ollama and want to keep existing model names
- Need consistent model names across different environments
❌ **Skip model mapping when:**
- Using **OpenAI**, **vLLM**, **LiteLLM**, **OpenRouter**, **Ollama**, or similar APIs directly (most common)
- You prefer using the provider's exact model names
- You want simpler configuration
For advanced mapping strategies and examples, see the [Model Mapping Guide](docs/MODEL_MAPPING.md).
## Deployment
### Docker Deployment
Using the provided `docker-compose.yml`:
```yaml
services:
ollama-proxy:
build: .
ports:
- "11434:11434"
env_file:
- .env
restart: unless-stopped
volumes:
- ./config:/app/config:ro
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:11434/health"]
interval: 30s
timeout: 10s
retries: 3
```
### Production Considerations
1. **Reverse Proxy**: Use nginx/traefik for SSL termination
2. **Rate Limiting**: Implement rate limiting to prevent abuse
3. **Monitoring**: Enable Prometheus metrics (coming soon)
4. **Logging**: Configure structured logging with log aggregation
5. **High Availability**: Run multiple replicas behind a load balancer
## Testing
This project maintains comprehensive test coverage across unit, integration, and performance tests. For detailed testing documentation, see our **[Testing Guide](docs/TESTING.md)**.
### Quick Testing
```bash
# Install dev dependencies
pip install -r requirements-dev.txt
# Run all tests
pytest tests/ -v
# Run with coverage report
pytest tests/ --cov=src --cov-report=html
# Run specific test categories
pytest tests/unit/ -v # Unit tests
pytest tests/performance/ -v # Performance tests
```
### Test Categories
- **Unit Tests**: 290+ tests covering individual components with comprehensive coverage
- **Integration Tests**: End-to-end API testing with mock backends
- **Performance Tests**: Load testing and benchmarking with metrics validation
- **Security Tests**: Input validation and error handling verification
### Current Test Status (Updated: 2025-07-15)
✅ **All tests passing**: 290 tests passed, 1 skipped, 0 failed
✅ **Code coverage**: 65.40% (exceeds minimum 10% requirement)
✅ **Performance validated**: All benchmarks within thresholds
✅ **Zero failing tests**: Complete test suite reliability
### Coverage Requirements
Our coverage standards ensure code quality and reliability:
- **Current Coverage**: 65.40% (minimum 10% requirement exceeded)
- **Target Coverage**: Working toward 85% overall coverage
- **New Code Coverage**: ≥85% (enforced on PRs)
- **Critical Components**: ≥90% (config, models, translators)
- **Quality Gates**: Automatic PR blocking below thresholds
```bash
# Generate coverage reports
make coverage # All formats
make coverage-html # HTML report only
pytest --cov=src --cov-fail-under=80 # With threshold check
```
### CI/CD Testing
All tests run automatically on:
- Pull requests and commits to main branch
- Nightly scheduled runs for regression detection
- Docker image builds for container testing
For complete testing instructions, coverage reports, and test strategy details, see the **[Testing Guide](docs/TESTING.md)**.
## Troubleshooting
See the [Troubleshooting Guide](docs/TROUBLESHOOTING.md) for comprehensive debugging help.
### Quick Fixes
#### Connection Issues
- **Connection refused**: Check if proxy is running on port 11434
- **Backend unreachable**: Verify `OPENAI_API_BASE_URL` is correct
- **Authentication failed**: Ensure `OPENAI_API_KEY` is valid
#### Common Problems
- **Model not found**: Add model mapping or use exact name
- **Timeout errors**: Increase `REQUEST_TIMEOUT`
- **CORS errors**: Proxy includes CORS headers by default
### Debug Mode
```env
LOG_LEVEL=DEBUG
DEBUG=true
```
For detailed solutions and error codes, see the [Troubleshooting Guide](docs/TROUBLESHOOTING.md).
## Development
### Makefile Usage
This project includes a comprehensive Makefile with 50+ commands for development, testing, deployment, and CI/CD workflows. The Makefile automatically detects your environment and provides intelligent defaults.
#### Quick Start with Makefile
```bash
# Get help and see all available commands
make help
# First-time setup (creates venv, installs deps, sets up hooks)
make first-time-setup
# Daily development workflow
make dev # Setup development environment
make run-local # Start development server
make test # Run all tests
make lint # Code quality checks
```
#### Key Command Categories
- 🏗️ **Environment**: `make dev`, `make venv-create`, `make reset-env`
- 🧪 **Testing**: `make test`, `make coverage`, `make test-watch`
- ✨ **Code Quality**: `make lint`, `make format`, `make typecheck`
- 🐳 **Docker**: `make build`, `make run`, `make compose-dev`
- 🐳 **Docker Compose**: `make compose-ci`, `make compose-prod`, `make compose-cluster`
- 📚 **Documentation**: `make docs-serve`, `make openapi-spec`
- 🚀 **CI/CD**: `make ci`, `make all-checks`, `make dist`
#### Smart Environment Detection
The Makefile automatically detects:
- **Virtual Environment**: Finds and uses venv, .venv, env, or active environment
- **Docker Compose**: Auto-detects v1 (`docker-compose`) vs v2 (`docker compose`)
- **Python/Pip**: Uses python3/pip3 or python/pip based on availability
#### Docker Compose Integration
```bash
# Development environments
make compose-dev # Start development with hot-reload
make compose-prod # Start production environment
make compose-ssl # Start with SSL support
make compose-debug # Start with debugging tools
# CI/CD workflows
make compose-ci # Run full CI pipeline
make compose-test # Run tests via Docker
make compose-lint # Run linting via Docker
# Multi-stack deployments
make compose-cluster # Load-balanced cluster
make compose-full-stack # Production + monitoring
make compose-down-all # Stop all environments
```
For complete Makefile documentation with examples and troubleshooting, see **[docs/Makefile.md](docs/Makefile.md)**.
### Project Structure
```
ollama_openai/
├── src/
│ ├── main.py # FastAPI application
│ ├── models.py # Pydantic models
│ ├── config.py # Configuration management
│ ├── routers/ # API endpoints
│ │ ├── chat.py
│ │ ├── models.py
│ │ └── embeddings.py
│ ├── translators/ # Format converters
│ │ ├── chat.py
│ │ └── embeddings.py
│ ├── middleware/ # Request/response processing
│ └── utils/ # Utilities
├── tests/ # Test suite
├── docker/ # Docker configurations
├── deployment/ # Deployment manifests
├── docs/ # Additional documentation
└── Makefile # Development automation
```
### Code Style
This project uses:
- `black` for code formatting
- `isort` for import sorting
- `mypy` for type checking
- `pylint` for linting
Run all checks:
```bash
make lint
```
### Adding New Features
1. Create a feature branch
2. Write tests first
3. Implement the feature
4. Ensure all tests pass
5. Update documentation
6. Submit a pull request
## Documentation
### Comprehensive Guides
- 📚 **[Architecture](ARCHITECTURE.md)** - System design and implementation details
- 🧪 **[Testing Guide](docs/TESTING.md)** - Comprehensive testing documentation and coverage reports
- 🔒 **[Security](docs/SECURITY.md)** - Security standards, best practices, and vulnerability reporting
- 📊 **[Performance Benchmarks](docs/PERFORMANCE_BENCHMARKS.md)** - Performance testing and optimization guide
- 🔧 **[Monitoring Integration](docs/MONITORING_INTEGRATION.md)** - Prometheus/Grafana setup and metrics
### Quick Reference
- [Quick Start Guide](docs/QUICK_START.md) - Get running in 5 minutes
- [Configuration Guide](docs/CONFIGURATION.md) - Environment variables and settings
- [API Compatibility Matrix](docs/API_COMPATIBILITY.md) - Supported endpoints and parameters
- [Model Mapping Guide](docs/MODEL_MAPPING.md) - Custom model name configuration
- [Makefile Guide](docs/Makefile.md) - Complete development automation documentation
- [Troubleshooting Guide](docs/TROUBLESHOOTING.md) - Common issues and solutions
## Security & Compliance
This project follows industry security standards and best practices:
### 🔒 Security Standards
- **OWASP Compliance**: Follows [OWASP Top 10](https://owasp.org/www-project-top-ten/) and [OWASP API Security Top 10](https://owasp.org/www-project-api-security/) guidelines
- **Input Validation**: All API inputs validated using Pydantic models with strict type checking
- **Secure Configuration**: Environment-based configuration with no hardcoded credentials
- **Error Handling**: Generic error messages prevent information leakage
### 🛡️ Security Features
- API key validation and secure forwarding
- Request size limits and timeout enforcement
- Connection pooling with configurable limits
- Graceful degradation under load
- Comprehensive audit logging with request IDs
### 📋 Security Scanning
- **Trivy**: Container vulnerability scanning
- **Bandit**: Python security linting
- **TruffleHog**: Secret detection in code
- **GitHub Security**: Automated dependency scanning
For detailed security information, see our [Security Policy](docs/SECURITY.md).
### 🚨 Vulnerability Reporting
Please report security vulnerabilities responsibly by following our [Security Policy](docs/SECURITY.md#vulnerability-reporting).
## Contributing
We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
### Areas for Contribution
- 📊 Prometheus metrics integration
- 🔐 Additional authentication methods
- 🌐 Multi-language SDK examples
- 📚 Additional documentation and tutorials
- 🔄 Phase 3: Advanced features and optimizations
- 🧪 Additional testing and benchmarking
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
## Acknowledgments
- Built for seamless integration between Ollama and OpenAI API formats
- Supports major LLM providers: **OpenAI**, **vLLM**, **LiteLLM**, **OpenRouter**, **Ollama**
- Inspired by the need to preserve existing codebases during infrastructure changes
- Thanks to all contributors and users providing feedback
---
For more detailed documentation, see the [docs/](docs/) directory.
Raw data
{
"_id": null,
"home_page": null,
"name": "ollama-openai-proxy",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "ollama, openai, proxy, api, translation",
"author": null,
"author_email": "Development Team <dev@example.com>",
"download_url": "https://files.pythonhosted.org/packages/2c/31/ac2590c0e99165e447385033912c32d674705d0a80d8d762e7970b68279e/ollama_openai_proxy-0.7.0.tar.gz",
"platform": null,
"description": "# Ollama to OpenAI Proxy\n\n[](https://github.com/eyalrot/ollama_openai/releases)\n[](https://pypi.org/project/ollama-openai-proxy/)\n[](https://pypi.org/project/ollama-openai-proxy/)\n[](https://github.com/eyalrot/ollama_openai/actions/workflows/ci.yml)\n[](https://codecov.io/gh/eyalrot/ollama_openai)\n[](https://github.com/eyalrot/ollama_openai/actions/workflows/security.yml)\n[](https://github.com/eyalrot/ollama_openai/actions/workflows/docker.yml)\n[](https://github.com/eyalrot/ollama_openai/pkgs/container/ollama_openai)\n[](https://hub.docker.com/r/eyalrot2/ollama-openai-proxy)\n[](https://opensource.org/licenses/MIT)\n\nA transparent proxy service that allows applications to use both Ollama and OpenAI API formats seamlessly with OpenAI-compatible LLM servers like **OpenAI**, **vLLM**, **LiteLLM**, **OpenRouter**, **Ollama**, and any other OpenAI-compatible API provider.\n\n**Perfect for N8N**: Enables using N8N's Ollama model node against OpenAI-compatible API servers. N8N's OpenAI models only support the hardcoded OpenAI URL (https://api.openai.com/v1) and cannot be configured, but the Ollama model node allows custom endpoints - making this proxy ideal for connecting N8N to any OpenAI-compatible LLM provider.\n\n## Features\n\n- \u2705 Drop-in replacement for Ollama server\n- \u2705 Zero changes required to existing code\n- \u2705 **Dual API format support**: Both Ollama and OpenAI endpoints\n- \u2705 Supports text generation and chat endpoints\n- \u2705 Streaming and non-streaming responses\n- \u2705 Model listing from backend\n- \u2705 Configurable model name mapping\n- \u2705 Docker and standalone deployment\n- \u2705 Automatic retry with exponential backoff\n- \u2705 Comprehensive logging and monitoring\n- \u2705 Request ID tracking for debugging\n- \u2705 Phase 1: Text-only chat and embeddings (completed)\n- \u2705 Phase 2: Tool calling support (completed)\n- \u2705 Phase 2: Image input support (completed)\n\n## Table of Contents\n\n- [Quick Start](#quick-start)\n- [Docker Images](#docker-images)\n- [Configuration](#configuration)\n- [API Compatibility](#api-compatibility)\n- [Model Mapping](#model-mapping)\n- [Deployment](#deployment)\n- [Testing](#testing)\n- [Troubleshooting](#troubleshooting)\n- [Development](#development)\n- [Documentation](#documentation)\n- [Security & Compliance](#security--compliance)\n- [Contributing](#contributing)\n- [License](#license)\n\n## Quick Start\n\nGet started in under 5 minutes! See the [Quick Start Guide](docs/QUICK_START.md) for detailed instructions.\n\n### Using Docker (Recommended)\n\n```bash\n# Clone and configure\ngit clone https://github.com/eyalrot/ollama_openai.git\ncd ollama_openai\ncp .env.example .env\n\n# Edit .env with your API details\nnano .env\n\n# Start the proxy\ndocker-compose up -d\n\n# Verify it's working\ncurl http://localhost:11434/health\n```\n\n### Using PyPI Package (Recommended)\n\n```bash\n# Install from PyPI\npip install ollama-openai-proxy\n\n# Create configuration file\ncat > .env << EOF\nOPENAI_API_BASE_URL=https://api.openai.com/v1\nOPENAI_API_KEY=your-api-key-here\nEOF\n\n# Run the proxy (method 1: using installed command)\nollama-openai-proxy\n\n# Or run using Python module (method 2)\npython -c \"from src.main import main; main()\"\n```\n\n### Using Python Source\n\n```bash\n# Setup\ngit clone https://github.com/eyalrot/ollama_openai.git\ncd ollama_openai\npip install -r requirements.txt\n\n# Configure\ncp .env.example .env\nnano .env\n\n# Run\npython -m uvicorn src.main:app --host 0.0.0.0 --port 11434\n```\n\n### Quick Test\n\n```bash\n# Check version and health\ncurl http://localhost:11434/v1/version\ncurl http://localhost:11434/v1/health\n```\n\n```python\n# Option 1: Use Ollama client (existing code works unchanged)\nfrom ollama import Client\nclient = Client(host='http://localhost:11434')\n\nresponse = client.generate(model='gpt-3.5-turbo', prompt='Hello!')\nprint(response['response'])\n\n# Option 2: Use OpenAI client (new in v0.6.0!)\nimport openai\nopenai.api_base = \"http://localhost:11434/v1\"\nopenai.api_key = \"your-api-key\"\n\nresponse = openai.ChatCompletion.create(\n model=\"gpt-3.5-turbo\",\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\nprint(response.choices[0].message.content)\n```\n\nFor more examples and detailed setup instructions, see the [Quick Start Guide](docs/QUICK_START.md).\n\n## Docker Images\n\n### Pre-built Docker Images\n\nReady-to-use production images are available on both Docker Hub and GitHub Container Registry:\n\n#### Docker Hub \ud83d\udc33 (Recommended)\n```bash\n# Pull and run latest\ndocker pull eyalrot2/ollama-openai-proxy:latest\ndocker run -d -p 11434:11434 \\\n -e OPENAI_API_BASE_URL=https://openrouter.ai/api/v1 \\\n -e OPENAI_API_KEY=your_key \\\n eyalrot2/ollama-openai-proxy:latest\n\n# Or use specific version\ndocker pull eyalrot2/ollama-openai-proxy:0.6.3\n# Available tags: latest, 0.6.3, 0.6, 0\n```\n\n#### GitHub Container Registry \ud83d\udce6\n```bash\n# Pull and run latest\ndocker pull ghcr.io/eyalrot/ollama_openai:latest\ndocker run -d -p 11434:11434 \\\n -e OPENAI_API_BASE_URL=https://openrouter.ai/api/v1 \\\n -e OPENAI_API_KEY=your_key \\\n ghcr.io/eyalrot/ollama_openai:latest\n\n# Or use specific version\ndocker pull ghcr.io/eyalrot/ollama_openai:0.6.3\n# Available tags: latest, 0.6.3, 0.6, 0\n```\n\n#### Multi-Architecture Support \ud83c\udfd7\ufe0f\n- **linux/amd64** (Intel/AMD processors)\n- **linux/arm64** (ARM processors, Apple Silicon, Raspberry Pi)\n\n### Docker Compose with Pre-built Images\n\n```yaml\nservices:\n ollama-proxy:\n # Use Docker Hub (recommended)\n image: eyalrot2/ollama-openai-proxy:latest\n # Or use GitHub Container Registry\n # image: ghcr.io/eyalrot/ollama_openai:latest\n ports:\n - \"11434:11434\"\n environment:\n - OPENAI_API_BASE_URL=https://openrouter.ai/api/v1\n - OPENAI_API_KEY=your_openrouter_key\n - LOG_LEVEL=INFO\n restart: unless-stopped\n```\n\n### Image Features\n\n- **Size**: 271MB (optimized production build)\n- **Security**: Non-root user, read-only filesystem, no-new-privileges\n- **Performance**: Multi-stage build with optimized dependencies\n- **Compatibility**: Supports **OpenAI**, **vLLM**, **LiteLLM**, **OpenRouter**, **Ollama**, and any OpenAI-compatible API provider\n- **SSL Support**: System SSL certificates included for private endpoints\n\n### Available Tags\n\n| Tag | Description | Docker Hub | GitHub Container Registry |\n|-----|-------------|------------|---------------------------|\n| `latest` | Latest stable build | `eyalrot2/ollama-openai-proxy:latest` | `ghcr.io/eyalrot/ollama_openai:latest` |\n| `0.6.3` | Specific version | `eyalrot2/ollama-openai-proxy:0.6.3` | `ghcr.io/eyalrot/ollama_openai:0.6.3` |\n| `0.6` | Major.minor version | `eyalrot2/ollama-openai-proxy:0.6` | `ghcr.io/eyalrot/ollama_openai:0.6` |\n| `0` | Major version | `eyalrot2/ollama-openai-proxy:0` | `ghcr.io/eyalrot/ollama_openai:0` |\n\n### Quick Test with Pre-built Image\n\n```bash\n# Start with OpenRouter free models (using Docker Hub)\ndocker run -d --name ollama-proxy -p 11434:11434 \\\n -e OPENAI_API_BASE_URL=https://openrouter.ai/api/v1 \\\n -e OPENAI_API_KEY=your_key \\\n eyalrot2/ollama-openai-proxy:latest\n\n# Or using GitHub Container Registry\n# docker run -d --name ollama-proxy -p 11434:11434 \\\n# -e OPENAI_API_BASE_URL=https://openrouter.ai/api/v1 \\\n# -e OPENAI_API_KEY=your_key \\\n# ghcr.io/eyalrot/ollama_openai:latest\n\n# Test with free model (Ollama format)\ncurl -X POST http://localhost:11434/api/generate \\\n -H \"Content-Type: application/json\" \\\n -d '{\"model\": \"google/gemma-2-9b-it:free\", \"prompt\": \"Hello!\"}'\n\n# Or test with OpenAI format\ncurl -X POST http://localhost:11434/v1/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -H \"Authorization: Bearer your_key\" \\\n -d '{\"model\": \"google/gemma-2-9b-it:free\", \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}]}'\n```\n\n## Configuration\n\nSee the [Configuration Guide](docs/CONFIGURATION.md) for detailed setup instructions.\n\n### Required Environment Variables\n\n| Variable | Description | Example |\n|----------|-------------|---------|\n| `OPENAI_API_BASE_URL` | URL of your OpenAI-compatible server | `https://api.openai.com/v1` |\n| `OPENAI_API_KEY` | API key for authentication | `sk-...` |\n\n### Key Optional Settings\n\n| Variable | Description | Default |\n|----------|-------------|---------|\n| `PROXY_PORT` | Port to run proxy on | `11434` |\n| `LOG_LEVEL` | Logging verbosity | `INFO` |\n| `REQUEST_TIMEOUT` | Request timeout in seconds | `60` |\n| `MODEL_MAPPING_FILE` | **Optional**: Path to model mapping JSON. When not set, model names pass through unchanged to your provider | `None` (recommended) |\n\nFor all configuration options, validation rules, and examples, see the [Configuration Guide](docs/CONFIGURATION.md).\n\n### Quick Testing with Different Providers\n\n#### OpenRouter (Free Models Available)\n```env\nOPENAI_API_BASE_URL=https://openrouter.ai/api/v1\nOPENAI_API_KEY=sk-or-v1-your-key\n```\nFree models: `google/gemma-2-9b-it:free`, `meta-llama/llama-3.2-3b-instruct:free`\n\n#### OpenAI\n```env\nOPENAI_API_BASE_URL=https://api.openai.com/v1\nOPENAI_API_KEY=sk-proj-your-key\n```\n\n#### vLLM Server\n```env\nOPENAI_API_BASE_URL=http://your-vllm-server:8000/v1\nOPENAI_API_KEY=your-api-key-or-none\n```\n\n#### LiteLLM Proxy\n```env\nOPENAI_API_BASE_URL=http://your-litellm-proxy:4000\nOPENAI_API_KEY=your-litellm-key\n```\n\n#### Local Ollama Server\n```env\nOPENAI_API_BASE_URL=http://localhost:11434/v1\nOPENAI_API_KEY=ollama # or any value\n```\n\n## API Compatibility\n\nSee the [API Compatibility Matrix](docs/API_COMPATIBILITY.md) for detailed endpoint mappings and parameter translations.\n\n### Supported Endpoints\n\n| Endpoint | Method | Status | Description |\n|----------|--------|---------|-------------|\n| `/api/generate` | POST | \u2705 Full Support | Text generation (Ollama-style) |\n| `/api/chat` | POST | \u2705 Full Support | Chat completion (Ollama-style) |\n| `/api/tags` | GET | \u2705 Full Support | List models |\n| `/api/embeddings` | POST | \u2705 Full Support | Generate embeddings (Ollama-style) |\n\n### Dual API Format Support \u2728\n\nThe proxy now supports **both Ollama and OpenAI API formats simultaneously**:\n\n#### Ollama-Style Endpoints\n- `/api/generate` - Text generation\n- `/api/chat` - Chat completion \n- `/api/embeddings` - Generate embeddings\n\n#### OpenAI-Style Endpoints\n- `/v1/chat/completions` - Chat completions\n- `/v1/models` - List models \n- `/v1/embeddings` - Generate embeddings\n\n**Choose the format that works best for your application!** The proxy automatically detects the API format based on the URL path (`/api/*` vs `/v1/*`) and routes accordingly.\n\nFor detailed parameter mappings, response formats, and examples, see the [API Compatibility Matrix](docs/API_COMPATIBILITY.md).\n\n## Phase 2 Features\n\n### Tool Calling Support \u2705\n\nThe proxy now supports full tool/function calling capabilities, allowing your AI models to execute tools and functions. This enables:\n\n- **Function Definitions**: Define functions with JSON schema parameters\n- **Tool Invocation**: Models can request to call tools during conversation\n- **Bidirectional Translation**: Seamless translation between Ollama and OpenAI tool formats\n- **Streaming Support**: Tool calls work with both streaming and non-streaming responses\n\n```python\nfrom ollama import Client\n\nclient = Client(host='http://localhost:11434')\n\n# Define tools\ntools = [{\n \"type\": \"function\",\n \"function\": {\n \"name\": \"get_weather\",\n \"description\": \"Get weather information for a location\",\n \"parameters\": {\n \"type\": \"object\",\n \"properties\": {\n \"location\": {\"type\": \"string\", \"description\": \"City name\"}\n },\n \"required\": [\"location\"]\n }\n }\n}]\n\n# Chat with tool support\nresponse = client.chat(\n model='gpt-4',\n messages=[{\"role\": \"user\", \"content\": \"What's the weather in Paris?\"}],\n tools=tools\n)\n```\n\n### Image Input Support \u2705\n\nThe proxy supports multimodal inputs, allowing you to send images along with text messages:\n\n- **Base64 Images**: Send images as base64-encoded strings\n- **Data URLs**: Support for data URL formatted images\n- **Multiple Images**: Send multiple images in a single message\n- **Mixed Content**: Combine text and images in conversations\n\n```python\nfrom ollama import Client\nimport base64\n\nclient = Client(host='http://localhost:11434')\n\n# Load and encode image\nwith open(\"image.jpg\", \"rb\") as f:\n image_data = base64.b64encode(f.read()).decode()\n\n# Send multimodal message\nresponse = client.chat(\n model='gpt-4-vision-preview',\n messages=[{\n \"role\": \"user\", \n \"content\": \"What do you see in this image?\",\n \"images\": [image_data]\n }]\n)\n```\n\nFor comprehensive Phase 2 examples and integration guides, see the [examples/phase2/](examples/phase2/) directory.\n\n## Examples\n\nSee the [examples/](examples/) directory for:\n- Python client examples (Ollama SDK, OpenAI SDK, streaming, batch processing, LangChain)\n- JavaScript/Node.js examples (both Ollama and OpenAI formats)\n- Configuration templates\n- Docker and Nginx setup examples\n- Dual API format usage patterns\n\n## Model Mapping\n\n**Model mapping is completely optional.** By default, the proxy passes all model names through unchanged to your OpenAI-compatible provider, allowing direct use of provider-specific model names.\n\n### Default Behavior: No Mapping Required \u2705\n\n**When `MODEL_MAPPING_FILE` is not configured (recommended for most users):**\n- Model names are passed directly to your provider as-is\n- No configuration needed - just use your provider's exact model names\n- Perfect for **OpenAI**, **vLLM**, **LiteLLM**, **OpenRouter**, **Ollama**, and any OpenAI-compatible API\n\n```bash\n# Direct model usage (no mapping file needed)\n# Ollama format:\ncurl -X POST http://localhost:11434/api/generate \\\n -H \"Content-Type: application/json\" \\\n -d '{\"model\": \"google/gemma-2-9b-it:free\", \"prompt\": \"Hello!\"}'\n\n# OpenAI format:\ncurl -X POST http://localhost:11434/v1/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -H \"Authorization: Bearer your_key\" \\\n -d '{\"model\": \"gpt-4\", \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}]}'\n\n# Both send model names directly to your OpenAI-compatible provider\n```\n\n### Optional: Custom Model Mapping\n\n**Only configure model mapping if you want to create custom aliases:**\n\n```json\n{\n \"model_mappings\": {\n \"llama2\": \"meta-llama/Llama-2-7b-chat-hf\",\n \"gpt4\": \"gpt-4\",\n \"free-gemma\": \"google/gemma-2-9b-it:free\"\n },\n \"default_model\": \"gpt-3.5-turbo\"\n}\n```\n\nThen set in environment:\n```env\nMODEL_MAPPING_FILE=./config/model_mapping.json\n```\n\nWith mapping enabled, you can use aliases in both formats:\n```bash\n# Ollama format with alias \"free-gemma\" -> maps to \"google/gemma-2-9b-it:free\"\ncurl -X POST http://localhost:11434/api/generate \\\n -H \"Content-Type: application/json\" \\\n -d '{\"model\": \"free-gemma\", \"prompt\": \"Hello!\"}'\n\n# OpenAI format with same alias\ncurl -X POST http://localhost:11434/v1/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -H \"Authorization: Bearer your_key\" \\\n -d '{\"model\": \"free-gemma\", \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}]}'\n```\n\n### When to Use Model Mapping\n\n\u2705 **Use model mapping when:**\n- You want shorter, memorable aliases for long model names\n- Migrating from Ollama and want to keep existing model names\n- Need consistent model names across different environments\n\n\u274c **Skip model mapping when:**\n- Using **OpenAI**, **vLLM**, **LiteLLM**, **OpenRouter**, **Ollama**, or similar APIs directly (most common)\n- You prefer using the provider's exact model names\n- You want simpler configuration\n\nFor advanced mapping strategies and examples, see the [Model Mapping Guide](docs/MODEL_MAPPING.md).\n\n## Deployment\n\n### Docker Deployment\n\nUsing the provided `docker-compose.yml`:\n\n```yaml\nservices:\n ollama-proxy:\n build: .\n ports:\n - \"11434:11434\"\n env_file:\n - .env\n restart: unless-stopped\n volumes:\n - ./config:/app/config:ro\n healthcheck:\n test: [\"CMD\", \"curl\", \"-f\", \"http://localhost:11434/health\"]\n interval: 30s\n timeout: 10s\n retries: 3\n```\n\n### Production Considerations\n\n1. **Reverse Proxy**: Use nginx/traefik for SSL termination\n2. **Rate Limiting**: Implement rate limiting to prevent abuse\n3. **Monitoring**: Enable Prometheus metrics (coming soon)\n4. **Logging**: Configure structured logging with log aggregation\n5. **High Availability**: Run multiple replicas behind a load balancer\n\n## Testing\n\n\n\nThis project maintains comprehensive test coverage across unit, integration, and performance tests. For detailed testing documentation, see our **[Testing Guide](docs/TESTING.md)**.\n\n### Quick Testing\n\n```bash\n# Install dev dependencies\npip install -r requirements-dev.txt\n\n# Run all tests\npytest tests/ -v\n\n# Run with coverage report\npytest tests/ --cov=src --cov-report=html\n\n# Run specific test categories\npytest tests/unit/ -v # Unit tests\npytest tests/performance/ -v # Performance tests\n```\n\n### Test Categories\n\n- **Unit Tests**: 290+ tests covering individual components with comprehensive coverage\n- **Integration Tests**: End-to-end API testing with mock backends\n- **Performance Tests**: Load testing and benchmarking with metrics validation\n- **Security Tests**: Input validation and error handling verification\n\n### Current Test Status (Updated: 2025-07-15)\n\n\u2705 **All tests passing**: 290 tests passed, 1 skipped, 0 failed\n\u2705 **Code coverage**: 65.40% (exceeds minimum 10% requirement)\n\u2705 **Performance validated**: All benchmarks within thresholds\n\u2705 **Zero failing tests**: Complete test suite reliability\n\n### Coverage Requirements\n\nOur coverage standards ensure code quality and reliability:\n\n- **Current Coverage**: 65.40% (minimum 10% requirement exceeded)\n- **Target Coverage**: Working toward 85% overall coverage\n- **New Code Coverage**: \u226585% (enforced on PRs)\n- **Critical Components**: \u226590% (config, models, translators)\n- **Quality Gates**: Automatic PR blocking below thresholds\n\n```bash\n# Generate coverage reports\nmake coverage # All formats\nmake coverage-html # HTML report only\npytest --cov=src --cov-fail-under=80 # With threshold check\n```\n\n### CI/CD Testing\n\nAll tests run automatically on:\n- Pull requests and commits to main branch\n- Nightly scheduled runs for regression detection\n- Docker image builds for container testing\n\nFor complete testing instructions, coverage reports, and test strategy details, see the **[Testing Guide](docs/TESTING.md)**.\n\n## Troubleshooting\n\nSee the [Troubleshooting Guide](docs/TROUBLESHOOTING.md) for comprehensive debugging help.\n\n### Quick Fixes\n\n#### Connection Issues\n- **Connection refused**: Check if proxy is running on port 11434\n- **Backend unreachable**: Verify `OPENAI_API_BASE_URL` is correct\n- **Authentication failed**: Ensure `OPENAI_API_KEY` is valid\n\n#### Common Problems\n- **Model not found**: Add model mapping or use exact name\n- **Timeout errors**: Increase `REQUEST_TIMEOUT` \n- **CORS errors**: Proxy includes CORS headers by default\n\n### Debug Mode\n\n```env\nLOG_LEVEL=DEBUG\nDEBUG=true\n```\n\nFor detailed solutions and error codes, see the [Troubleshooting Guide](docs/TROUBLESHOOTING.md).\n\n## Development\n\n### Makefile Usage\n\nThis project includes a comprehensive Makefile with 50+ commands for development, testing, deployment, and CI/CD workflows. The Makefile automatically detects your environment and provides intelligent defaults.\n\n#### Quick Start with Makefile\n\n```bash\n# Get help and see all available commands\nmake help\n\n# First-time setup (creates venv, installs deps, sets up hooks)\nmake first-time-setup\n\n# Daily development workflow\nmake dev # Setup development environment\nmake run-local # Start development server\nmake test # Run all tests\nmake lint # Code quality checks\n```\n\n#### Key Command Categories\n\n- \ud83c\udfd7\ufe0f **Environment**: `make dev`, `make venv-create`, `make reset-env`\n- \ud83e\uddea **Testing**: `make test`, `make coverage`, `make test-watch`\n- \u2728 **Code Quality**: `make lint`, `make format`, `make typecheck`\n- \ud83d\udc33 **Docker**: `make build`, `make run`, `make compose-dev`\n- \ud83d\udc33 **Docker Compose**: `make compose-ci`, `make compose-prod`, `make compose-cluster`\n- \ud83d\udcda **Documentation**: `make docs-serve`, `make openapi-spec`\n- \ud83d\ude80 **CI/CD**: `make ci`, `make all-checks`, `make dist`\n\n#### Smart Environment Detection\n\nThe Makefile automatically detects:\n- **Virtual Environment**: Finds and uses venv, .venv, env, or active environment\n- **Docker Compose**: Auto-detects v1 (`docker-compose`) vs v2 (`docker compose`)\n- **Python/Pip**: Uses python3/pip3 or python/pip based on availability\n\n#### Docker Compose Integration\n\n```bash\n# Development environments\nmake compose-dev # Start development with hot-reload\nmake compose-prod # Start production environment\nmake compose-ssl # Start with SSL support\nmake compose-debug # Start with debugging tools\n\n# CI/CD workflows\nmake compose-ci # Run full CI pipeline\nmake compose-test # Run tests via Docker\nmake compose-lint # Run linting via Docker\n\n# Multi-stack deployments\nmake compose-cluster # Load-balanced cluster\nmake compose-full-stack # Production + monitoring\nmake compose-down-all # Stop all environments\n```\n\nFor complete Makefile documentation with examples and troubleshooting, see **[docs/Makefile.md](docs/Makefile.md)**.\n\n### Project Structure\n\n```\nollama_openai/\n\u251c\u2500\u2500 src/\n\u2502 \u251c\u2500\u2500 main.py # FastAPI application\n\u2502 \u251c\u2500\u2500 models.py # Pydantic models\n\u2502 \u251c\u2500\u2500 config.py # Configuration management\n\u2502 \u251c\u2500\u2500 routers/ # API endpoints\n\u2502 \u2502 \u251c\u2500\u2500 chat.py\n\u2502 \u2502 \u251c\u2500\u2500 models.py\n\u2502 \u2502 \u2514\u2500\u2500 embeddings.py\n\u2502 \u251c\u2500\u2500 translators/ # Format converters\n\u2502 \u2502 \u251c\u2500\u2500 chat.py\n\u2502 \u2502 \u2514\u2500\u2500 embeddings.py\n\u2502 \u251c\u2500\u2500 middleware/ # Request/response processing\n\u2502 \u2514\u2500\u2500 utils/ # Utilities\n\u251c\u2500\u2500 tests/ # Test suite\n\u251c\u2500\u2500 docker/ # Docker configurations\n\u251c\u2500\u2500 deployment/ # Deployment manifests\n\u251c\u2500\u2500 docs/ # Additional documentation\n\u2514\u2500\u2500 Makefile # Development automation\n```\n\n### Code Style\n\nThis project uses:\n- `black` for code formatting\n- `isort` for import sorting\n- `mypy` for type checking\n- `pylint` for linting\n\nRun all checks:\n```bash\nmake lint\n```\n\n### Adding New Features\n\n1. Create a feature branch\n2. Write tests first\n3. Implement the feature\n4. Ensure all tests pass\n5. Update documentation\n6. Submit a pull request\n\n## Documentation\n\n### Comprehensive Guides\n\n- \ud83d\udcda **[Architecture](ARCHITECTURE.md)** - System design and implementation details\n- \ud83e\uddea **[Testing Guide](docs/TESTING.md)** - Comprehensive testing documentation and coverage reports\n- \ud83d\udd12 **[Security](docs/SECURITY.md)** - Security standards, best practices, and vulnerability reporting\n- \ud83d\udcca **[Performance Benchmarks](docs/PERFORMANCE_BENCHMARKS.md)** - Performance testing and optimization guide\n- \ud83d\udd27 **[Monitoring Integration](docs/MONITORING_INTEGRATION.md)** - Prometheus/Grafana setup and metrics\n\n### Quick Reference\n\n- [Quick Start Guide](docs/QUICK_START.md) - Get running in 5 minutes\n- [Configuration Guide](docs/CONFIGURATION.md) - Environment variables and settings\n- [API Compatibility Matrix](docs/API_COMPATIBILITY.md) - Supported endpoints and parameters\n- [Model Mapping Guide](docs/MODEL_MAPPING.md) - Custom model name configuration\n- [Makefile Guide](docs/Makefile.md) - Complete development automation documentation\n- [Troubleshooting Guide](docs/TROUBLESHOOTING.md) - Common issues and solutions\n\n## Security & Compliance\n\nThis project follows industry security standards and best practices:\n\n### \ud83d\udd12 Security Standards\n- **OWASP Compliance**: Follows [OWASP Top 10](https://owasp.org/www-project-top-ten/) and [OWASP API Security Top 10](https://owasp.org/www-project-api-security/) guidelines\n- **Input Validation**: All API inputs validated using Pydantic models with strict type checking\n- **Secure Configuration**: Environment-based configuration with no hardcoded credentials\n- **Error Handling**: Generic error messages prevent information leakage\n\n### \ud83d\udee1\ufe0f Security Features\n- API key validation and secure forwarding\n- Request size limits and timeout enforcement\n- Connection pooling with configurable limits\n- Graceful degradation under load\n- Comprehensive audit logging with request IDs\n\n### \ud83d\udccb Security Scanning\n- **Trivy**: Container vulnerability scanning\n- **Bandit**: Python security linting\n- **TruffleHog**: Secret detection in code\n- **GitHub Security**: Automated dependency scanning\n\nFor detailed security information, see our [Security Policy](docs/SECURITY.md).\n\n### \ud83d\udea8 Vulnerability Reporting\nPlease report security vulnerabilities responsibly by following our [Security Policy](docs/SECURITY.md#vulnerability-reporting).\n\n## Contributing\n\nWe welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.\n\n### Areas for Contribution\n\n- \ud83d\udcca Prometheus metrics integration\n- \ud83d\udd10 Additional authentication methods\n- \ud83c\udf10 Multi-language SDK examples\n- \ud83d\udcda Additional documentation and tutorials\n- \ud83d\udd04 Phase 3: Advanced features and optimizations\n- \ud83e\uddea Additional testing and benchmarking\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## Acknowledgments\n\n- Built for seamless integration between Ollama and OpenAI API formats\n- Supports major LLM providers: **OpenAI**, **vLLM**, **LiteLLM**, **OpenRouter**, **Ollama**\n- Inspired by the need to preserve existing codebases during infrastructure changes\n- Thanks to all contributors and users providing feedback\n\n---\n\nFor more detailed documentation, see the [docs/](docs/) directory.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A proxy service that translates Ollama API requests to OpenAI-compatible backends",
"version": "0.7.0",
"project_urls": {
"Bug Reports": "https://github.com/eyalrot/ollama_openai/issues",
"Documentation": "https://github.com/eyalrot/ollama_openai#readme",
"Homepage": "https://github.com/eyalrot/ollama_openai",
"Source": "https://github.com/eyalrot/ollama_openai"
},
"split_keywords": [
"ollama",
" openai",
" proxy",
" api",
" translation"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "8811abdd82bbb04c28dd9ebbf9df5a4b45e5bdd5e2aadbf669daf61d6d446a56",
"md5": "b238df485bfc8327eb2f991ea3e59faf",
"sha256": "47f6ea529c7606a6a2327ded1e957023afb986fab03150e42d2aa6ea80929fdf"
},
"downloads": -1,
"filename": "ollama_openai_proxy-0.7.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b238df485bfc8327eb2f991ea3e59faf",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 73772,
"upload_time": "2025-07-22T12:20:41",
"upload_time_iso_8601": "2025-07-22T12:20:41.820567Z",
"url": "https://files.pythonhosted.org/packages/88/11/abdd82bbb04c28dd9ebbf9df5a4b45e5bdd5e2aadbf669daf61d6d446a56/ollama_openai_proxy-0.7.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "2c31ac2590c0e99165e447385033912c32d674705d0a80d8d762e7970b68279e",
"md5": "93fb520be78e99a0269d4e12db8abac7",
"sha256": "d367b25a51901f277e214a87fbc8e98dca2bf5fd0389c289826e0fab8e7334d1"
},
"downloads": -1,
"filename": "ollama_openai_proxy-0.7.0.tar.gz",
"has_sig": false,
"md5_digest": "93fb520be78e99a0269d4e12db8abac7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 75456,
"upload_time": "2025-07-22T12:20:42",
"upload_time_iso_8601": "2025-07-22T12:20:42.767184Z",
"url": "https://files.pythonhosted.org/packages/2c/31/ac2590c0e99165e447385033912c32d674705d0a80d8d762e7970b68279e/ollama_openai_proxy-0.7.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-22 12:20:42",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "eyalrot",
"github_project": "ollama_openai",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"requirements": [
{
"name": "fastapi",
"specs": [
[
"==",
"0.104.1"
]
]
},
{
"name": "uvicorn",
"specs": [
[
"==",
"0.24.0"
]
]
},
{
"name": "httpx",
"specs": [
[
"==",
"0.25.0"
]
]
},
{
"name": "langchain-openai",
"specs": [
[
"==",
"0.0.5"
]
]
},
{
"name": "pydantic",
"specs": [
[
"==",
"2.5.0"
]
]
},
{
"name": "pydantic-settings",
"specs": [
[
"==",
"2.1.0"
]
]
},
{
"name": "python-dotenv",
"specs": [
[
"==",
"1.0.0"
]
]
},
{
"name": "aiofiles",
"specs": [
[
"==",
"23.2.1"
]
]
},
{
"name": "python-multipart",
"specs": [
[
"==",
"0.0.6"
]
]
},
{
"name": "psutil",
"specs": [
[
"==",
"5.9.6"
]
]
}
],
"lcname": "ollama-openai-proxy"
}
JSON
