| Name | llama-index-llms-litellm JSON |
| Version |
0.2.2
JSON |
| download |
| home_page | None |
| Summary | llama-index llms litellm integration |
| upload_time | 2024-10-08 22:29:49 |
| maintainer | None |
| docs_url | None |
| author | Your Name |
| requires_python | <4.0,>=3.8.1 |
| license | MIT |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# LlamaIndex Llms Integration: Litellm
## Installation
1. Install the required Python packages:
```bash
%pip install llama-index-llms-litellm
!pip install llama-index
```
## Usage
### Import Required Libraries
```python
import os
from llama_index.llms.litellm import LiteLLM
from llama_index.core.llms import ChatMessage
```
### Set Up Environment Variables
Set your API keys as environment variables:
```python
os.environ["OPENAI_API_KEY"] = "your-api-key"
os.environ["COHERE_API_KEY"] = "your-api-key"
```
### Example: OpenAI Call
To interact with the OpenAI model:
```python
message = ChatMessage(role="user", content="Hey! how's it going?")
llm = LiteLLM("gpt-3.5-turbo")
chat_response = llm.chat([message])
print(chat_response)
```
### Example: Cohere Call
To interact with the Cohere model:
```python
llm = LiteLLM("command-nightly")
chat_response = llm.chat([message])
print(chat_response)
```
### Example: Chat with System Message
To have a chat with a system role:
```python
messages = [
ChatMessage(
role="system", content="You are a pirate with a colorful personality"
),
ChatMessage(role="user", content="Tell me a story"),
]
resp = LiteLLM("gpt-3.5-turbo").chat(messages)
print(resp)
```
### Streaming Responses
To use the streaming feature with `stream_complete`:
```python
llm = LiteLLM("gpt-3.5-turbo")
resp = llm.stream_complete("Paul Graham is ")
for r in resp:
print(r.delta, end="")
```
### Streaming Chat Example
To stream chat messages:
```python
llm = LiteLLM("gpt-3.5-turbo")
resp = llm.stream_chat(messages)
for r in resp:
print(r.delta, end="")
```
### Asynchronous Example
For asynchronous calls, use:
```python
llm = LiteLLM("gpt-3.5-turbo")
resp = await llm.acomplete("Paul Graham is ")
print(resp)
```
### LLM Implementation example
https://docs.llamaindex.ai/en/stable/examples/llm/litellm/
Raw data
{
"_id": null,
"home_page": null,
"name": "llama-index-llms-litellm",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8.1",
"maintainer_email": null,
"keywords": null,
"author": "Your Name",
"author_email": "you@example.com",
"download_url": "https://files.pythonhosted.org/packages/51/93/0173dc0ef7e5adb62470ba1c6a5b19b77c804e4fac649da5a240cda904e7/llama_index_llms_litellm-0.2.2.tar.gz",
"platform": null,
"description": "# LlamaIndex Llms Integration: Litellm\n\n## Installation\n\n1. Install the required Python packages:\n\n ```bash\n %pip install llama-index-llms-litellm\n !pip install llama-index\n ```\n\n## Usage\n\n### Import Required Libraries\n\n```python\nimport os\nfrom llama_index.llms.litellm import LiteLLM\nfrom llama_index.core.llms import ChatMessage\n```\n\n### Set Up Environment Variables\n\nSet your API keys as environment variables:\n\n```python\nos.environ[\"OPENAI_API_KEY\"] = \"your-api-key\"\nos.environ[\"COHERE_API_KEY\"] = \"your-api-key\"\n```\n\n### Example: OpenAI Call\n\nTo interact with the OpenAI model:\n\n```python\nmessage = ChatMessage(role=\"user\", content=\"Hey! how's it going?\")\nllm = LiteLLM(\"gpt-3.5-turbo\")\nchat_response = llm.chat([message])\nprint(chat_response)\n```\n\n### Example: Cohere Call\n\nTo interact with the Cohere model:\n\n```python\nllm = LiteLLM(\"command-nightly\")\nchat_response = llm.chat([message])\nprint(chat_response)\n```\n\n### Example: Chat with System Message\n\nTo have a chat with a system role:\n\n```python\nmessages = [\n ChatMessage(\n role=\"system\", content=\"You are a pirate with a colorful personality\"\n ),\n ChatMessage(role=\"user\", content=\"Tell me a story\"),\n]\nresp = LiteLLM(\"gpt-3.5-turbo\").chat(messages)\nprint(resp)\n```\n\n### Streaming Responses\n\nTo use the streaming feature with `stream_complete`:\n\n```python\nllm = LiteLLM(\"gpt-3.5-turbo\")\nresp = llm.stream_complete(\"Paul Graham is \")\nfor r in resp:\n print(r.delta, end=\"\")\n```\n\n### Streaming Chat Example\n\nTo stream chat messages:\n\n```python\nllm = LiteLLM(\"gpt-3.5-turbo\")\nresp = llm.stream_chat(messages)\nfor r in resp:\n print(r.delta, end=\"\")\n```\n\n### Asynchronous Example\n\nFor asynchronous calls, use:\n\n```python\nllm = LiteLLM(\"gpt-3.5-turbo\")\nresp = await llm.acomplete(\"Paul Graham is \")\nprint(resp)\n```\n\n### LLM Implementation example\n\nhttps://docs.llamaindex.ai/en/stable/examples/llm/litellm/\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "llama-index llms litellm integration",
"version": "0.2.2",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "09e5e58e2cdf565e6462b06776ccc5045d57bada3a55e509dccfb9887d535c54",
"md5": "100408fbf3e014914d23a295059b9212",
"sha256": "85d53430ad2f7d0697ac55473aaf9628d97db856e40555bf479175f6f31508e4"
},
"downloads": -1,
"filename": "llama_index_llms_litellm-0.2.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "100408fbf3e014914d23a295059b9212",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8.1",
"size": 7876,
"upload_time": "2024-10-08T22:29:49",
"upload_time_iso_8601": "2024-10-08T22:29:49.007308Z",
"url": "https://files.pythonhosted.org/packages/09/e5/e58e2cdf565e6462b06776ccc5045d57bada3a55e509dccfb9887d535c54/llama_index_llms_litellm-0.2.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "51930173dc0ef7e5adb62470ba1c6a5b19b77c804e4fac649da5a240cda904e7",
"md5": "26611fd2a31ba6cd8f4a538c42134fc5",
"sha256": "955f8117170718b627bd48c88213bf9afb9fd70d91fca8cd2455bcdbdf83f3bf"
},
"downloads": -1,
"filename": "llama_index_llms_litellm-0.2.2.tar.gz",
"has_sig": false,
"md5_digest": "26611fd2a31ba6cd8f4a538c42134fc5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8.1",
"size": 7339,
"upload_time": "2024-10-08T22:29:49",
"upload_time_iso_8601": "2024-10-08T22:29:49.847692Z",
"url": "https://files.pythonhosted.org/packages/51/93/0173dc0ef7e5adb62470ba1c6a5b19b77c804e4fac649da5a240cda904e7/llama_index_llms_litellm-0.2.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-08 22:29:49",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "llama-index-llms-litellm"
}
