| Name | llama-index-llms-openllm JSON |
| Version |
0.4.0
JSON |
| download |
| home_page | None |
| Summary | llama-index llms openllm integration |
| upload_time | 2024-11-18 01:38:01 |
| maintainer | None |
| docs_url | None |
| author | Aaron Pham |
| requires_python | <4.0,>=3.9 |
| license | MIT |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# LlamaIndex LLM Integration: OpenLLM
## Installation
To install the required packages, run:
```bash
%pip install llama-index-llms-openllm
!pip install llama-index
```
## Setup
### Initialize OpenLLM
First, import the necessary libraries and set up your `OpenLLM` instance. Replace `my-model`, `https://hostname.com/v1`, and `na` with your model name, API base URL, and API key, respectively:
```python
import os
from typing import List, Optional
from llama_index.llms.openllm import OpenLLM
from llama_index.core.llms import ChatMessage
llm = OpenLLM(
model="my-model", api_base="https://hostname.com/v1", api_key="na"
)
```
## Generate Completions
To generate a completion, use the `complete` method:
```python
completion_response = llm.complete("To infinity, and")
print(completion_response)
```
### Stream Completions
You can also stream completions using the `stream_complete` method:
```python
async for it in llm.stream_complete(
"The meaning of time is", max_new_tokens=128
):
print(it, end="", flush=True)
```
## Chat Functionality
OpenLLM supports chat APIs, allowing you to handle conversation-like interactions. Here’s how to use it:
### Synchronous Chat
You can perform a synchronous chat by constructing a list of `ChatMessage` instances:
```python
from llama_index.core.llms import ChatMessage
chat_messages = [
ChatMessage(role="system", content="You are acting as Ernest Hemmingway."),
ChatMessage(role="user", content="Hi there!"),
ChatMessage(role="assistant", content="Yes?"),
ChatMessage(role="user", content="What is the meaning of life?"),
]
for it in llm.chat(chat_messages):
print(it.message.content, flush=True, end="")
```
### Asynchronous Chat
To perform an asynchronous chat, use the `astream_chat` method:
```python
async for it in llm.astream_chat(chat_messages):
print(it.message.content, flush=True, end="")
```
### LLM Implementation example
https://docs.llamaindex.ai/en/stable/examples/llm/openllm/
Raw data
{
"_id": null,
"home_page": null,
"name": "llama-index-llms-openllm",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.9",
"maintainer_email": null,
"keywords": null,
"author": "Aaron Pham",
"author_email": "aarnphm@bentoml.com",
"download_url": "https://files.pythonhosted.org/packages/94/a5/2cc1ce926918f0a1f05c842fe2fd2d712a1c09f1e4860f9267d716b6afd7/llama_index_llms_openllm-0.4.0.tar.gz",
"platform": null,
"description": "# LlamaIndex LLM Integration: OpenLLM\n\n## Installation\n\nTo install the required packages, run:\n\n```bash\n%pip install llama-index-llms-openllm\n!pip install llama-index\n```\n\n## Setup\n\n### Initialize OpenLLM\n\nFirst, import the necessary libraries and set up your `OpenLLM` instance. Replace `my-model`, `https://hostname.com/v1`, and `na` with your model name, API base URL, and API key, respectively:\n\n```python\nimport os\nfrom typing import List, Optional\nfrom llama_index.llms.openllm import OpenLLM\nfrom llama_index.core.llms import ChatMessage\n\nllm = OpenLLM(\n model=\"my-model\", api_base=\"https://hostname.com/v1\", api_key=\"na\"\n)\n```\n\n## Generate Completions\n\nTo generate a completion, use the `complete` method:\n\n```python\ncompletion_response = llm.complete(\"To infinity, and\")\nprint(completion_response)\n```\n\n### Stream Completions\n\nYou can also stream completions using the `stream_complete` method:\n\n```python\nasync for it in llm.stream_complete(\n \"The meaning of time is\", max_new_tokens=128\n):\n print(it, end=\"\", flush=True)\n```\n\n## Chat Functionality\n\nOpenLLM supports chat APIs, allowing you to handle conversation-like interactions. Here\u2019s how to use it:\n\n### Synchronous Chat\n\nYou can perform a synchronous chat by constructing a list of `ChatMessage` instances:\n\n```python\nfrom llama_index.core.llms import ChatMessage\n\nchat_messages = [\n ChatMessage(role=\"system\", content=\"You are acting as Ernest Hemmingway.\"),\n ChatMessage(role=\"user\", content=\"Hi there!\"),\n ChatMessage(role=\"assistant\", content=\"Yes?\"),\n ChatMessage(role=\"user\", content=\"What is the meaning of life?\"),\n]\n\nfor it in llm.chat(chat_messages):\n print(it.message.content, flush=True, end=\"\")\n```\n\n### Asynchronous Chat\n\nTo perform an asynchronous chat, use the `astream_chat` method:\n\n```python\nasync for it in llm.astream_chat(chat_messages):\n print(it.message.content, flush=True, end=\"\")\n```\n\n### LLM Implementation example\n\nhttps://docs.llamaindex.ai/en/stable/examples/llm/openllm/\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "llama-index llms openllm integration",
"version": "0.4.0",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "09192f5bc90a41b23bb377d9836cf793c0fd7233bf59eb054688e87790e786a4",
"md5": "d58a2ae8acafee9f0d9450b9591f7b2e",
"sha256": "7f4772670c7950218e6341141937078e2dddf4fa321a302e3598467d75f621a2"
},
"downloads": -1,
"filename": "llama_index_llms_openllm-0.4.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d58a2ae8acafee9f0d9450b9591f7b2e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.9",
"size": 2709,
"upload_time": "2024-11-18T01:38:00",
"upload_time_iso_8601": "2024-11-18T01:38:00.309647Z",
"url": "https://files.pythonhosted.org/packages/09/19/2f5bc90a41b23bb377d9836cf793c0fd7233bf59eb054688e87790e786a4/llama_index_llms_openllm-0.4.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "94a52cc1ce926918f0a1f05c842fe2fd2d712a1c09f1e4860f9267d716b6afd7",
"md5": "17bf156a2dba73aca693083786cb928b",
"sha256": "133955f3ed7c80c34766c39b9b084b04b18616c5c70e6d75ba604d668eadfc20"
},
"downloads": -1,
"filename": "llama_index_llms_openllm-0.4.0.tar.gz",
"has_sig": false,
"md5_digest": "17bf156a2dba73aca693083786cb928b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.9",
"size": 2482,
"upload_time": "2024-11-18T01:38:01",
"upload_time_iso_8601": "2024-11-18T01:38:01.868220Z",
"url": "https://files.pythonhosted.org/packages/94/a5/2cc1ce926918f0a1f05c842fe2fd2d712a1c09f1e4860f9267d716b6afd7/llama_index_llms_openllm-0.4.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-18 01:38:01",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "llama-index-llms-openllm"
}
