| Name | llama-index-llms-ollama JSON |
| Version |
0.3.4
JSON |
| download |
| home_page | None |
| Summary | llama-index llms ollama integration |
| upload_time | 2024-10-08 22:32:52 |
| maintainer | None |
| docs_url | None |
| author | Your Name |
| requires_python | <4.0,>=3.8.1 |
| license | MIT |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# LlamaIndex Llms Integration: Ollama
## Installation
To install the required package, run:
```bash
%pip install llama-index-llms-ollama
```
## Setup
1. Follow the [Ollama README](https://ollama.com) to set up and run a local Ollama instance.
2. When the Ollama app is running on your local machine, it will serve all of your local models on `localhost:11434`.
3. Select your model when creating the `Ollama` instance by specifying `model=":"`.
4. You can increase the default timeout (30 seconds) by setting `Ollama(..., request_timeout=300.0)`.
5. If you set `llm = Ollama(..., model="<model family>")` without a version, it will automatically look for the latest version.
## Usage
### Initialize Ollama
```python
from llama_index.llms.ollama import Ollama
llm = Ollama(model="llama3.1:latest", request_timeout=120.0)
```
### Generate Completions
To generate a text completion for a prompt, use the `complete` method:
```python
resp = llm.complete("Who is Paul Graham?")
print(resp)
```
### Chat Responses
To send a chat message and receive a response, create a list of `ChatMessage` instances and use the `chat` method:
```python
from llama_index.core.llms import ChatMessage
messages = [
ChatMessage(
role="system", content="You are a pirate with a colorful personality."
),
ChatMessage(role="user", content="What is your name?"),
]
resp = llm.chat(messages)
print(resp)
```
### Streaming Responses
#### Stream Complete
To stream responses for a prompt, use the `stream_complete` method:
```python
response = llm.stream_complete("Who is Paul Graham?")
for r in response:
print(r.delta, end="")
```
#### Stream Chat
To stream chat responses, use the `stream_chat` method:
```python
messages = [
ChatMessage(
role="system", content="You are a pirate with a colorful personality."
),
ChatMessage(role="user", content="What is your name?"),
]
resp = llm.stream_chat(messages)
for r in resp:
print(r.delta, end="")
```
### JSON Mode
Ollama supports a JSON mode to ensure all responses are valid JSON, which is useful for tools that need to parse structured outputs:
```python
llm = Ollama(model="llama3.1:latest", request_timeout=120.0, json_mode=True)
response = llm.complete(
"Who is Paul Graham? Output as a structured JSON object."
)
print(str(response))
```
### Structured Outputs
You can attach a Pydantic class to the LLM to ensure structured outputs:
```python
from llama_index.core.bridge.pydantic import BaseModel
from llama_index.core.tools import FunctionTool
class Song(BaseModel):
"""A song with name and artist."""
name: str
artist: str
llm = Ollama(model="llama3.1:latest", request_timeout=120.0)
sllm = llm.as_structured_llm(Song)
response = sllm.chat([ChatMessage(role="user", content="Name a random song!")])
print(
response.message.content
) # e.g., {"name": "Yesterday", "artist": "The Beatles"}
```
### Asynchronous Chat
You can also use asynchronous chat:
```python
response = await sllm.achat(
[ChatMessage(role="user", content="Name a random song!")]
)
print(response.message.content)
```
### LLM Implementation example
https://docs.llamaindex.ai/en/stable/examples/llm/ollama/
Raw data
{
"_id": null,
"home_page": null,
"name": "llama-index-llms-ollama",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8.1",
"maintainer_email": null,
"keywords": null,
"author": "Your Name",
"author_email": "you@example.com",
"download_url": "https://files.pythonhosted.org/packages/90/92/e7378d2af5ad8f19d16025d050161a44ef1d380ebfa313752f7fb8e36424/llama_index_llms_ollama-0.3.4.tar.gz",
"platform": null,
"description": "# LlamaIndex Llms Integration: Ollama\n\n## Installation\n\nTo install the required package, run:\n\n```bash\n%pip install llama-index-llms-ollama\n```\n\n## Setup\n\n1. Follow the [Ollama README](https://ollama.com) to set up and run a local Ollama instance.\n2. When the Ollama app is running on your local machine, it will serve all of your local models on `localhost:11434`.\n3. Select your model when creating the `Ollama` instance by specifying `model=\":\"`.\n4. You can increase the default timeout (30 seconds) by setting `Ollama(..., request_timeout=300.0)`.\n5. If you set `llm = Ollama(..., model=\"<model family>\")` without a version, it will automatically look for the latest version.\n\n## Usage\n\n### Initialize Ollama\n\n```python\nfrom llama_index.llms.ollama import Ollama\n\nllm = Ollama(model=\"llama3.1:latest\", request_timeout=120.0)\n```\n\n### Generate Completions\n\nTo generate a text completion for a prompt, use the `complete` method:\n\n```python\nresp = llm.complete(\"Who is Paul Graham?\")\nprint(resp)\n```\n\n### Chat Responses\n\nTo send a chat message and receive a response, create a list of `ChatMessage` instances and use the `chat` method:\n\n```python\nfrom llama_index.core.llms import ChatMessage\n\nmessages = [\n ChatMessage(\n role=\"system\", content=\"You are a pirate with a colorful personality.\"\n ),\n ChatMessage(role=\"user\", content=\"What is your name?\"),\n]\nresp = llm.chat(messages)\nprint(resp)\n```\n\n### Streaming Responses\n\n#### Stream Complete\n\nTo stream responses for a prompt, use the `stream_complete` method:\n\n```python\nresponse = llm.stream_complete(\"Who is Paul Graham?\")\nfor r in response:\n print(r.delta, end=\"\")\n```\n\n#### Stream Chat\n\nTo stream chat responses, use the `stream_chat` method:\n\n```python\nmessages = [\n ChatMessage(\n role=\"system\", content=\"You are a pirate with a colorful personality.\"\n ),\n ChatMessage(role=\"user\", content=\"What is your name?\"),\n]\nresp = llm.stream_chat(messages)\nfor r in resp:\n print(r.delta, end=\"\")\n```\n\n### JSON Mode\n\nOllama supports a JSON mode to ensure all responses are valid JSON, which is useful for tools that need to parse structured outputs:\n\n```python\nllm = Ollama(model=\"llama3.1:latest\", request_timeout=120.0, json_mode=True)\nresponse = llm.complete(\n \"Who is Paul Graham? Output as a structured JSON object.\"\n)\nprint(str(response))\n```\n\n### Structured Outputs\n\nYou can attach a Pydantic class to the LLM to ensure structured outputs:\n\n```python\nfrom llama_index.core.bridge.pydantic import BaseModel\nfrom llama_index.core.tools import FunctionTool\n\n\nclass Song(BaseModel):\n \"\"\"A song with name and artist.\"\"\"\n\n name: str\n artist: str\n\n\nllm = Ollama(model=\"llama3.1:latest\", request_timeout=120.0)\nsllm = llm.as_structured_llm(Song)\n\nresponse = sllm.chat([ChatMessage(role=\"user\", content=\"Name a random song!\")])\nprint(\n response.message.content\n) # e.g., {\"name\": \"Yesterday\", \"artist\": \"The Beatles\"}\n```\n\n### Asynchronous Chat\n\nYou can also use asynchronous chat:\n\n```python\nresponse = await sllm.achat(\n [ChatMessage(role=\"user\", content=\"Name a random song!\")]\n)\nprint(response.message.content)\n```\n\n### LLM Implementation example\n\nhttps://docs.llamaindex.ai/en/stable/examples/llm/ollama/\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "llama-index llms ollama integration",
"version": "0.3.4",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3883e8fc5234c19900a1bcca5dd64b98cb79a11f92526a8c741263e24d1279a7",
"md5": "7f7fd43478b056c6d2f9f4a6388abd39",
"sha256": "4c144551c59decd2a993eccde87e451fe61843e31480ebf541951581b43c08e4"
},
"downloads": -1,
"filename": "llama_index_llms_ollama-0.3.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7f7fd43478b056c6d2f9f4a6388abd39",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8.1",
"size": 5823,
"upload_time": "2024-10-08T22:32:50",
"upload_time_iso_8601": "2024-10-08T22:32:50.836994Z",
"url": "https://files.pythonhosted.org/packages/38/83/e8fc5234c19900a1bcca5dd64b98cb79a11f92526a8c741263e24d1279a7/llama_index_llms_ollama-0.3.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9092e7378d2af5ad8f19d16025d050161a44ef1d380ebfa313752f7fb8e36424",
"md5": "b4519e522ce9d3a2bc0c595ffa562033",
"sha256": "56df6e1887081af4522f9ec32be2d4632d8d4f2619f72aa71ea60966231e54a0"
},
"downloads": -1,
"filename": "llama_index_llms_ollama-0.3.4.tar.gz",
"has_sig": false,
"md5_digest": "b4519e522ce9d3a2bc0c595ffa562033",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8.1",
"size": 5489,
"upload_time": "2024-10-08T22:32:52",
"upload_time_iso_8601": "2024-10-08T22:32:52.400150Z",
"url": "https://files.pythonhosted.org/packages/90/92/e7378d2af5ad8f19d16025d050161a44ef1d380ebfa313752f7fb8e36424/llama_index_llms_ollama-0.3.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-08 22:32:52",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "llama-index-llms-ollama"
}
