| Name | llama-index-llms-neutrino JSON |
| Version |
0.2.1
JSON |
| download |
| home_page | None |
| Summary | llama-index llms neutrino integration |
| upload_time | 2024-10-08 22:32:37 |
| maintainer | None |
| docs_url | None |
| author | Your Name |
| requires_python | <4.0,>=3.8.1 |
| license | MIT |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# LlamaIndex Llms Integration: Neutrino
## Installation
To install the required packages, run:
```bash
%pip install llama-index-llms-neutrino
!pip install llama-index
```
## Setup
### Create Neutrino API Key
You can create an API key by visiting [platform.neutrinoapp.com](https://platform.neutrinoapp.com). Once you have the API key, set it as an environment variable:
```python
import os
os.environ["NEUTRINO_API_KEY"] = "<your-neutrino-api-key>"
```
## Using Your Router
A router is a collection of LLMs that you can route queries to. You can create a router in the Neutrino dashboard or use the default router, which includes all supported models. You can treat a router as a single LLM.
### Initialize Neutrino
Create an instance of the Neutrino model:
```python
from llama_index.llms.neutrino import Neutrino
llm = Neutrino(
# api_key="<your-neutrino-api-key>",
# router="<your-router-id>" # Use 'default' for the default router
)
```
### Generate Completions
To generate a text completion for a prompt, use the `complete` method:
```python
response = llm.complete("In short, a Neutrino is")
print(f"Optimal model: {response.raw['model']}")
print(response)
```
### Chat Responses
To send a chat message and receive a response, create a `ChatMessage` and use the `chat` method:
```python
from llama_index.core.llms import ChatMessage
message = ChatMessage(
role="user",
content="Explain the difference between statically typed and dynamically typed languages.",
)
resp = llm.chat([message])
print(f"Optimal model: {resp.raw['model']}")
print(resp)
```
### Streaming Responses
To stream responses for a chat message, use the `stream_chat` method:
```python
message = ChatMessage(
role="user", content="What is the approximate population of Mexico?"
)
resp = llm.stream_chat([message])
for i, r in enumerate(resp):
if i == 0:
print(f"Optimal model: {r.raw['model']}")
print(r.delta, end="")
```
### LLM Implementation example
https://docs.llamaindex.ai/en/stable/examples/llm/neutrino/
Raw data
{
"_id": null,
"home_page": null,
"name": "llama-index-llms-neutrino",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8.1",
"maintainer_email": null,
"keywords": null,
"author": "Your Name",
"author_email": "you@example.com",
"download_url": "https://files.pythonhosted.org/packages/98/ef/0b3eb0b6d891408375c17e54afa73be609547fe03410322a01c0153f06be/llama_index_llms_neutrino-0.2.1.tar.gz",
"platform": null,
"description": "# LlamaIndex Llms Integration: Neutrino\n\n## Installation\n\nTo install the required packages, run:\n\n```bash\n%pip install llama-index-llms-neutrino\n!pip install llama-index\n```\n\n## Setup\n\n### Create Neutrino API Key\n\nYou can create an API key by visiting [platform.neutrinoapp.com](https://platform.neutrinoapp.com). Once you have the API key, set it as an environment variable:\n\n```python\nimport os\n\nos.environ[\"NEUTRINO_API_KEY\"] = \"<your-neutrino-api-key>\"\n```\n\n## Using Your Router\n\nA router is a collection of LLMs that you can route queries to. You can create a router in the Neutrino dashboard or use the default router, which includes all supported models. You can treat a router as a single LLM.\n\n### Initialize Neutrino\n\nCreate an instance of the Neutrino model:\n\n```python\nfrom llama_index.llms.neutrino import Neutrino\n\nllm = Neutrino(\n # api_key=\"<your-neutrino-api-key>\",\n # router=\"<your-router-id>\" # Use 'default' for the default router\n)\n```\n\n### Generate Completions\n\nTo generate a text completion for a prompt, use the `complete` method:\n\n```python\nresponse = llm.complete(\"In short, a Neutrino is\")\nprint(f\"Optimal model: {response.raw['model']}\")\nprint(response)\n```\n\n### Chat Responses\n\nTo send a chat message and receive a response, create a `ChatMessage` and use the `chat` method:\n\n```python\nfrom llama_index.core.llms import ChatMessage\n\nmessage = ChatMessage(\n role=\"user\",\n content=\"Explain the difference between statically typed and dynamically typed languages.\",\n)\n\nresp = llm.chat([message])\nprint(f\"Optimal model: {resp.raw['model']}\")\nprint(resp)\n```\n\n### Streaming Responses\n\nTo stream responses for a chat message, use the `stream_chat` method:\n\n```python\nmessage = ChatMessage(\n role=\"user\", content=\"What is the approximate population of Mexico?\"\n)\n\nresp = llm.stream_chat([message])\nfor i, r in enumerate(resp):\n if i == 0:\n print(f\"Optimal model: {r.raw['model']}\")\n print(r.delta, end=\"\")\n```\n\n### LLM Implementation example\n\nhttps://docs.llamaindex.ai/en/stable/examples/llm/neutrino/\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "llama-index llms neutrino integration",
"version": "0.2.1",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "03bee8709137c9f82c7c9a1af75b9f4d1c1fa135d4457af1a48d332ed1e35ebe",
"md5": "b5d22af2332dd4cd7e55ecb21dee3db5",
"sha256": "729836d4b9f737761ed6f00f6e9e9180f03753df427067c637a86b292b375f43"
},
"downloads": -1,
"filename": "llama_index_llms_neutrino-0.2.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b5d22af2332dd4cd7e55ecb21dee3db5",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8.1",
"size": 3543,
"upload_time": "2024-10-08T22:32:35",
"upload_time_iso_8601": "2024-10-08T22:32:35.520498Z",
"url": "https://files.pythonhosted.org/packages/03/be/e8709137c9f82c7c9a1af75b9f4d1c1fa135d4457af1a48d332ed1e35ebe/llama_index_llms_neutrino-0.2.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "98ef0b3eb0b6d891408375c17e54afa73be609547fe03410322a01c0153f06be",
"md5": "10a96b8ba3e862038ab5d0802751efd0",
"sha256": "3cb2e625604b11c20d16070b517d45299564fa14e7f8c21e750dbf0f64e8b614"
},
"downloads": -1,
"filename": "llama_index_llms_neutrino-0.2.1.tar.gz",
"has_sig": false,
"md5_digest": "10a96b8ba3e862038ab5d0802751efd0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8.1",
"size": 3038,
"upload_time": "2024-10-08T22:32:37",
"upload_time_iso_8601": "2024-10-08T22:32:37.040810Z",
"url": "https://files.pythonhosted.org/packages/98/ef/0b3eb0b6d891408375c17e54afa73be609547fe03410322a01c0153f06be/llama_index_llms_neutrino-0.2.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-08 22:32:37",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "llama-index-llms-neutrino"
}
