| Name | llama-index-llms-openrouter JSON |
| Version |
0.3.0
JSON |
| download |
| home_page | None |
| Summary | llama-index llms openrouter integration |
| upload_time | 2024-11-18 01:29:02 |
| maintainer | None |
| docs_url | None |
| author | Your Name |
| requires_python | <4.0,>=3.9 |
| license | MIT |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# LlamaIndex Llms Integration: Openrouter
## Installation
To install the required packages, run:
```bash
%pip install llama-index-llms-openrouter
!pip install llama-index
```
## Setup
### Initialize OpenRouter
You need to set either the environment variable `OPENROUTER_API_KEY` or pass your API key directly in the class constructor. Replace `<your-api-key>` with your actual API key:
```python
from llama_index.llms.openrouter import OpenRouter
from llama_index.core.llms import ChatMessage
llm = OpenRouter(
api_key="<your-api-key>",
max_tokens=256,
context_window=4096,
model="gryphe/mythomax-l2-13b",
)
```
## Generate Chat Responses
You can generate a chat response by sending a list of `ChatMessage` instances:
```python
message = ChatMessage(role="user", content="Tell me a joke")
resp = llm.chat([message])
print(resp)
```
### Streaming Responses
To stream responses, use the `stream_chat` method:
```python
message = ChatMessage(role="user", content="Tell me a story in 250 words")
resp = llm.stream_chat([message])
for r in resp:
print(r.delta, end="")
```
### Complete with Prompt
You can also generate completions with a prompt using the `complete` method:
```python
resp = llm.complete("Tell me a joke")
print(resp)
```
### Streaming Completion
To stream completions, use the `stream_complete` method:
```python
resp = llm.stream_complete("Tell me a story in 250 words")
for r in resp:
print(r.delta, end="")
```
## Model Configuration
To use a specific model, you can specify it during initialization. For example, to use Mistral's Mixtral model, you can set it like this:
```python
llm = OpenRouter(model="mistralai/mixtral-8x7b-instruct")
resp = llm.complete("Write a story about a dragon who can code in Rust")
print(resp)
```
### LLM Implementation example
https://docs.llamaindex.ai/en/stable/examples/llm/openrouter/
Raw data
{
"_id": null,
"home_page": null,
"name": "llama-index-llms-openrouter",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.9",
"maintainer_email": null,
"keywords": null,
"author": "Your Name",
"author_email": "you@example.com",
"download_url": "https://files.pythonhosted.org/packages/7c/27/210827b02edf888e5d2b8e18d20c0e407b5de7bfc370e2b596e0602dcf24/llama_index_llms_openrouter-0.3.0.tar.gz",
"platform": null,
"description": "# LlamaIndex Llms Integration: Openrouter\n\n## Installation\n\nTo install the required packages, run:\n\n```bash\n%pip install llama-index-llms-openrouter\n!pip install llama-index\n```\n\n## Setup\n\n### Initialize OpenRouter\n\nYou need to set either the environment variable `OPENROUTER_API_KEY` or pass your API key directly in the class constructor. Replace `<your-api-key>` with your actual API key:\n\n```python\nfrom llama_index.llms.openrouter import OpenRouter\nfrom llama_index.core.llms import ChatMessage\n\nllm = OpenRouter(\n api_key=\"<your-api-key>\",\n max_tokens=256,\n context_window=4096,\n model=\"gryphe/mythomax-l2-13b\",\n)\n```\n\n## Generate Chat Responses\n\nYou can generate a chat response by sending a list of `ChatMessage` instances:\n\n```python\nmessage = ChatMessage(role=\"user\", content=\"Tell me a joke\")\nresp = llm.chat([message])\nprint(resp)\n```\n\n### Streaming Responses\n\nTo stream responses, use the `stream_chat` method:\n\n```python\nmessage = ChatMessage(role=\"user\", content=\"Tell me a story in 250 words\")\nresp = llm.stream_chat([message])\nfor r in resp:\n print(r.delta, end=\"\")\n```\n\n### Complete with Prompt\n\nYou can also generate completions with a prompt using the `complete` method:\n\n```python\nresp = llm.complete(\"Tell me a joke\")\nprint(resp)\n```\n\n### Streaming Completion\n\nTo stream completions, use the `stream_complete` method:\n\n```python\nresp = llm.stream_complete(\"Tell me a story in 250 words\")\nfor r in resp:\n print(r.delta, end=\"\")\n```\n\n## Model Configuration\n\nTo use a specific model, you can specify it during initialization. For example, to use Mistral's Mixtral model, you can set it like this:\n\n```python\nllm = OpenRouter(model=\"mistralai/mixtral-8x7b-instruct\")\nresp = llm.complete(\"Write a story about a dragon who can code in Rust\")\nprint(resp)\n```\n\n### LLM Implementation example\n\nhttps://docs.llamaindex.ai/en/stable/examples/llm/openrouter/\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "llama-index llms openrouter integration",
"version": "0.3.0",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "fa23c3cd899dc99b34d319764569a3efe33b30e6c7fd006a6242b54eb289a404",
"md5": "fba50a3e3ed746c722c4e61f0309696a",
"sha256": "2d8971b338cc86a6e3cab39a844954df9e017bf3706b0db4d1e7da6c02d94a49"
},
"downloads": -1,
"filename": "llama_index_llms_openrouter-0.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "fba50a3e3ed746c722c4e61f0309696a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.9",
"size": 3478,
"upload_time": "2024-11-18T01:29:01",
"upload_time_iso_8601": "2024-11-18T01:29:01.198334Z",
"url": "https://files.pythonhosted.org/packages/fa/23/c3cd899dc99b34d319764569a3efe33b30e6c7fd006a6242b54eb289a404/llama_index_llms_openrouter-0.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7c27210827b02edf888e5d2b8e18d20c0e407b5de7bfc370e2b596e0602dcf24",
"md5": "27ab1edf79e726374c84d7bce8d6aa16",
"sha256": "4dfa5b39f1d3bfef1b494c1c0d9341f7a05024d6b043e80cd5790bad25ceac86"
},
"downloads": -1,
"filename": "llama_index_llms_openrouter-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "27ab1edf79e726374c84d7bce8d6aa16",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.9",
"size": 3103,
"upload_time": "2024-11-18T01:29:02",
"upload_time_iso_8601": "2024-11-18T01:29:02.208380Z",
"url": "https://files.pythonhosted.org/packages/7c/27/210827b02edf888e5d2b8e18d20c0e407b5de7bfc370e2b596e0602dcf24/llama_index_llms_openrouter-0.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-18 01:29:02",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "llama-index-llms-openrouter"
}
