# LlamaIndex LLM Integration for Cloudflare AI Gateway
This package provides integration between LlamaIndex and Cloudflare AI Gateway, allowing you to use multiple AI models from different providers with automatic fallback.
## Features
- **Multiple Providers**: Use OpenAI, Anthropic, Mistral, Groq, and more
- **Automatic Fallback**: If one provider fails, automatically tries the next
- **Streaming Support**: Both chat and completion streaming
- **Async Support**: Full async/await support
- **Caching**: Built-in request caching with Cloudflare AI Gateway
## Installation
```bash
pip install llama-index-llms-cloudflare-ai-gateway
```
## Quick Start
```python
from llama_index.llms.cloudflare_ai_gateway import CloudflareAIGateway
from llama_index.llms.openai import OpenAI
from llama_index.llms.anthropic import Anthropic
from llama_index.core.llms import ChatMessage
# Create LLM instances
openai_llm = OpenAI(model="gpt-4o-mini", api_key="your-openai-key")
anthropic_llm = Anthropic(
model="claude-3-5-sonnet-20241022", api_key="your-anthropic-key"
)
# Create Cloudflare AI Gateway LLM
llm = CloudflareAIGateway(
llms=[openai_llm, anthropic_llm], # Try OpenAI first, then Anthropic
account_id="your-cloudflare-account-id",
gateway="your-ai-gateway-name",
api_key="your-cloudflare-api-key",
)
# Use the LLM
messages = [ChatMessage(role="user", content="What is 2+2?")]
response = llm.chat(messages)
print(response.message.content)
```
## Supported Providers
- OpenAI
- Anthropic
- Mistral AI
- Groq
- DeepSeek
- Perplexity
- Replicate
- Grok
- Azure OpenAI
## Testing
```bash
# Set environment variables
export OPENAI_API_KEY="your-key"
export ANTHROPIC_API_KEY="your-key"
export CLOUDFLARE_ACCOUNT_ID="your-id"
export CLOUDFLARE_API_KEY="your-key"
export CLOUDFLARE_GATEWAY="your-gateway"
# Run tests
uv run pytest tests/
```
Raw data
{
"_id": null,
"home_page": null,
"name": "llama-index-llms-cloudflare-ai-gateway",
"maintainer": "psiace",
"docs_url": null,
"requires_python": "<4.0,>=3.10",
"maintainer_email": null,
"keywords": "ai, cloudflare, gateway",
"author": null,
"author_email": "Chojan Shang <psiace@outlook.com>",
"download_url": "https://files.pythonhosted.org/packages/79/39/9bbdea1ec890998f9a3f5a278d4fca6cc6199f94ad5ca640d8382ddbdfca/llama_index_llms_cloudflare_ai_gateway-0.1.0.tar.gz",
"platform": null,
"description": "# LlamaIndex LLM Integration for Cloudflare AI Gateway\n\nThis package provides integration between LlamaIndex and Cloudflare AI Gateway, allowing you to use multiple AI models from different providers with automatic fallback.\n\n## Features\n\n- **Multiple Providers**: Use OpenAI, Anthropic, Mistral, Groq, and more\n- **Automatic Fallback**: If one provider fails, automatically tries the next\n- **Streaming Support**: Both chat and completion streaming\n- **Async Support**: Full async/await support\n- **Caching**: Built-in request caching with Cloudflare AI Gateway\n\n## Installation\n\n```bash\npip install llama-index-llms-cloudflare-ai-gateway\n```\n\n## Quick Start\n\n```python\nfrom llama_index.llms.cloudflare_ai_gateway import CloudflareAIGateway\nfrom llama_index.llms.openai import OpenAI\nfrom llama_index.llms.anthropic import Anthropic\nfrom llama_index.core.llms import ChatMessage\n\n# Create LLM instances\nopenai_llm = OpenAI(model=\"gpt-4o-mini\", api_key=\"your-openai-key\")\nanthropic_llm = Anthropic(\n model=\"claude-3-5-sonnet-20241022\", api_key=\"your-anthropic-key\"\n)\n\n# Create Cloudflare AI Gateway LLM\nllm = CloudflareAIGateway(\n llms=[openai_llm, anthropic_llm], # Try OpenAI first, then Anthropic\n account_id=\"your-cloudflare-account-id\",\n gateway=\"your-ai-gateway-name\",\n api_key=\"your-cloudflare-api-key\",\n)\n\n# Use the LLM\nmessages = [ChatMessage(role=\"user\", content=\"What is 2+2?\")]\nresponse = llm.chat(messages)\nprint(response.message.content)\n```\n\n## Supported Providers\n\n- OpenAI\n- Anthropic\n- Mistral AI\n- Groq\n- DeepSeek\n- Perplexity\n- Replicate\n- Grok\n- Azure OpenAI\n\n## Testing\n\n```bash\n# Set environment variables\nexport OPENAI_API_KEY=\"your-key\"\nexport ANTHROPIC_API_KEY=\"your-key\"\nexport CLOUDFLARE_ACCOUNT_ID=\"your-id\"\nexport CLOUDFLARE_API_KEY=\"your-key\"\nexport CLOUDFLARE_GATEWAY=\"your-gateway\"\n\n# Run tests\nuv run pytest tests/\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "llama-index llms cloudflare ai gateway integration",
"version": "0.1.0",
"project_urls": null,
"split_keywords": [
"ai",
" cloudflare",
" gateway"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "6a1421296ff2b0403a4c10df595642b926170b006c02092a043110130d102e9c",
"md5": "60bf7262511d4c04a4fa0b4dd78edf52",
"sha256": "dc05077c6a80e65b66b89e57fc17371b1bcf1c0c3fe01dbadd797e53e3c25a85"
},
"downloads": -1,
"filename": "llama_index_llms_cloudflare_ai_gateway-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "60bf7262511d4c04a4fa0b4dd78edf52",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.10",
"size": 8094,
"upload_time": "2025-07-17T09:29:07",
"upload_time_iso_8601": "2025-07-17T09:29:07.653492Z",
"url": "https://files.pythonhosted.org/packages/6a/14/21296ff2b0403a4c10df595642b926170b006c02092a043110130d102e9c/llama_index_llms_cloudflare_ai_gateway-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "79399bbdea1ec890998f9a3f5a278d4fca6cc6199f94ad5ca640d8382ddbdfca",
"md5": "381cc5dac16918aa4c5d701a198e7bec",
"sha256": "82b2a1a5e02ee6c0bad50484001aabc44b6513dd6e569969fb460318465562cd"
},
"downloads": -1,
"filename": "llama_index_llms_cloudflare_ai_gateway-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "381cc5dac16918aa4c5d701a198e7bec",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.10",
"size": 8080,
"upload_time": "2025-07-17T09:29:08",
"upload_time_iso_8601": "2025-07-17T09:29:08.835818Z",
"url": "https://files.pythonhosted.org/packages/79/39/9bbdea1ec890998f9a3f5a278d4fca6cc6199f94ad5ca640d8382ddbdfca/llama_index_llms_cloudflare_ai_gateway-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-17 09:29:08",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "llama-index-llms-cloudflare-ai-gateway"
}
JSON