# ollama-instructor
`ollama-instructor` is a lightweight Python library that provides a convenient wrapper around the Client of the renowned Ollama repository, extending it with validation features for obtaining valid JSON responses from a Large Language Model (LLM). Utilizing Pydantic, `ollama-instructor` allows users to specify models for JSON schemas and data validation, ensuring that responses from LLMs adhere to the defined schema.
> **Note 1**: This library has a native support for the Ollamas Python client. If you want to have more flexibility with other providers like Groq, OpenAI, Perplexity and more, have a look into the great library of [instrutor](https://github.com/jxnl/instructor) of Jason Lui.
> **Note 2**: This library depends on having [Ollama](https://ollama.com) installed and running. For more information, please refer to the official website of Ollama.
## Features
- Easy integration with the Ollama repository for running open-source LLMs locally. See:
- https://github.com/ollama/ollama
- https://github.com/ollama/ollama-python
- Data validation using Pydantic models to ensure the JSON response from a LLM meets the specified schema. See:
- https://docs.pydantic.dev/latest/
- Retries with error guidance if the LLM returns invalid responses. You can set the maxium number of retries.
- Allow partial responses to be returned by setting the `allow_partial` flag to True. This will try to clean set invalid data within the response and set it to `None`. Unsetted data (not part of the Pydantic model) will be deleted from the response.
`ollama-instructor` can help you to get structured and reliable JSON from local LLMs like:
- llama3
- phi3
- mistral
- gemma
- ...
`ollama-instructor` can be your starting point to build agents by your self. Have full control over agent flows without relying on complex agent framework.
## Installation
To install `ollama-instructor`, run the following command in your terminal:
```
pip install ollama-instructor
```
## Quick Start
Here are quick examples to get you started with `ollama-instructor`:
**chat completion**:
```python
from ollama_instructor.ollama_instructor_client import OllamaInstructorClient
from pydantic import BaseModel
class Person(BaseModel):
name: str
age: int
client = OllamaInstructorClient(...)
response = client.chat_completion(
model='phi3',
pydantic_model=Person,
messages=[
{
'role': 'user',
'content': 'Jason is 30 years old.'
}
]
)
print(response['message']['content'])
```
Output:
```json
{"name": "Jason", "age": 30}
```
**asynchronous chat completion**:
```python
from pydantic import BaseModel, ConfigDict
from enum import Enum
from typing import List
import rich
import asyncio
from ollama_instructor.ollama_instructor_client import OllamaInstructorAsyncClient
class Gender(Enum):
MALE = 'male'
FEMALE = 'female'
class Person(BaseModel):
'''
This model defines a person.
'''
name: str
age: int
gender: Gender
friends: List[str] = []
model_config = ConfigDict(
extra='forbid'
)
async def main():
client = OllamaInstructorAsyncClient(...)
await client.async_init() # Wichtig: Die asynchrone Initialisierung aufrufen
response = await client.chat_completion(
model='phi3:instruct',
pydantic_model=Person,
messages=[
{
'role': 'user',
'content': 'Jason is 25 years old. Jason loves to play soccer with his friends Nick and Gabriel. His favorite food is pizza.'
}
],
)
rich.print(response['message']['content'])
if __name__ == "__main__":
asyncio.run(main())
```
**chat completion with streaming**:
```python
from ollama_instructor.ollama_instructor_client import OllamaInstructorClient
from pydantic import BaseModel
class Person(BaseModel):
name: str
age: int
client = OllamaInstructorClient(...)
response = client.chat_completion_with_stream(
model='phi3',
pydantic_model=Person,
messages=[
{
'role': 'user',
'content': 'Jason is 30 years old.'
}
]
)
for chunk in response:
print(chunk['message']['content'])
```
## OllamaInstructorClient and OllamaInstructorAsyncClient
The classes `OllamaInstructorClient` and `OllamaInstructorAsyncClient` are the main class of the `ollama-instructor` library. They are the wrapper around the `Ollama` client and contain the following arguments:
- `host`: the URL of the Ollama server (default: `http://localhost:11434`). See documentation of [Ollama](https://github.com/ollama/ollama)
- `debug`: a `bool` indicating whether to print debug messages (default: `False`).
> **Note**: I am currently working with `iceream` for the debug messages. Will try to improve that in further development of this library.
### chat_completion & chat_completion_with_stream
The `chat_completion` and `chat_completion_with_stream` methods are the main methods of the library. They are used to generate text completions from a given prompt.
`ollama-instructor` uses `chat_completion` and `chat_completion_with_stream` to expand the `chat` method of `Ollama`. For all available arguments of `chat` see the [Ollama documentation](https://github.com/ollama/ollama).
The following arguments are added to the `chat` method within `chat_completion` and `chat_completion_with_stream`:
- `pydantic_model`: a class of Pydantic's `BaseModel` class that is used to firstly instruct the LLM with the JSON schema of the `BaseModel` and secondly to validate the response of the LLM with the built-in validation of [Pydantic](https://docs.pydantic.dev/latest/).
- `retries`: the number of retries if the LLM fails to generate a valid response (default: `3`). If a LLM fails the retry will provide the last response of the LLM with the given `ValidationError` and insructs it to generate a valid response.
- `allow_partial`: If set to `True` `ollama-instructor` will modify the `BaseModel` to allow partial responses. In this case it makes sure to provide the correct instance of the JSON schema but with default or None values. Therefore, it is useful to provide default values within the `BaseModel`. With the improvement of this library you will find examples and best practice guides on that topic in the [docs](/docs/) folder.
## Documentation and examples
- It was always my goal to have a well documented library. Therefore, have a look into the repositorys code to get an idea how to use it.
- There will be a great bunch of how-to-use guides and examples in the [docs](/docs/) folder (coming soon).
- If you need more information about the library, please feel free to open an issue.
## License
`ollama-instructor` is released under the MIT License. See the [LICENSE](LICENSE) file for more details.
## Support and Community
If you need help or want to discuss `ollama-instructor`, feel free to open an issue on GitHub.
Raw data
{
"_id": null,
"home_page": "https://github.com/lennartpollvogt/ollama-instructor",
"name": "ollama-instructor",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8",
"maintainer_email": null,
"keywords": "ollama, pydantic, validation, json-schema, json, instructor, prompting, local-llm, llm",
"author": "Lennart Pollvogt",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/4a/88/67d51b06e55a3863e90169a21c2194cbdfd0d5063cfaf941e09c572e01ad/ollama_instructor-0.2.0.tar.gz",
"platform": null,
"description": "# ollama-instructor\n\n`ollama-instructor` is a lightweight Python library that provides a convenient wrapper around the Client of the renowned Ollama repository, extending it with validation features for obtaining valid JSON responses from a Large Language Model (LLM). Utilizing Pydantic, `ollama-instructor` allows users to specify models for JSON schemas and data validation, ensuring that responses from LLMs adhere to the defined schema.\n\n> **Note 1**: This library has a native support for the Ollamas Python client. If you want to have more flexibility with other providers like Groq, OpenAI, Perplexity and more, have a look into the great library of [instrutor](https://github.com/jxnl/instructor) of Jason Lui.\n\n> **Note 2**: This library depends on having [Ollama](https://ollama.com) installed and running. For more information, please refer to the official website of Ollama.\n\n## Features\n\n- Easy integration with the Ollama repository for running open-source LLMs locally. See: \n - https://github.com/ollama/ollama\n - https://github.com/ollama/ollama-python\n- Data validation using Pydantic models to ensure the JSON response from a LLM meets the specified schema. See:\n - https://docs.pydantic.dev/latest/\n- Retries with error guidance if the LLM returns invalid responses. You can set the maxium number of retries.\n- Allow partial responses to be returned by setting the `allow_partial` flag to True. This will try to clean set invalid data within the response and set it to `None`. Unsetted data (not part of the Pydantic model) will be deleted from the response.\n\n`ollama-instructor` can help you to get structured and reliable JSON from local LLMs like:\n- llama3\n- phi3\n- mistral\n- gemma\n- ...\n\n`ollama-instructor` can be your starting point to build agents by your self. Have full control over agent flows without relying on complex agent framework.\n\n\n## Installation\n\nTo install `ollama-instructor`, run the following command in your terminal:\n\n```\npip install ollama-instructor\n```\n\n\n## Quick Start\n\nHere are quick examples to get you started with `ollama-instructor`:\n\n**chat completion**:\n```python\nfrom ollama_instructor.ollama_instructor_client import OllamaInstructorClient\nfrom pydantic import BaseModel\n\nclass Person(BaseModel):\n name: str\n age: int\n\nclient = OllamaInstructorClient(...)\nresponse = client.chat_completion(\n model='phi3', \n pydantic_model=Person, \n messages=[\n {\n 'role': 'user',\n 'content': 'Jason is 30 years old.'\n }\n ]\n)\n\nprint(response['message']['content'])\n```\nOutput:\n```json\n{\"name\": \"Jason\", \"age\": 30}\n```\n\n**asynchronous chat completion**:\n```python\nfrom pydantic import BaseModel, ConfigDict\nfrom enum import Enum\nfrom typing import List\nimport rich\nimport asyncio\n\nfrom ollama_instructor.ollama_instructor_client import OllamaInstructorAsyncClient\n\nclass Gender(Enum):\n MALE = 'male'\n FEMALE = 'female'\n\nclass Person(BaseModel):\n '''\n This model defines a person.\n '''\n name: str\n age: int\n gender: Gender\n friends: List[str] = []\n\n model_config = ConfigDict(\n extra='forbid'\n )\n\nasync def main():\n client = OllamaInstructorAsyncClient(...)\n await client.async_init() # Wichtig: Die asynchrone Initialisierung aufrufen\n\n response = await client.chat_completion(\n model='phi3:instruct',\n pydantic_model=Person,\n messages=[\n {\n 'role': 'user',\n 'content': 'Jason is 25 years old. Jason loves to play soccer with his friends Nick and Gabriel. His favorite food is pizza.'\n }\n ],\n )\n rich.print(response['message']['content'])\n\nif __name__ == \"__main__\":\n asyncio.run(main())\n```\n\n**chat completion with streaming**:\n```python\nfrom ollama_instructor.ollama_instructor_client import OllamaInstructorClient\nfrom pydantic import BaseModel\n\nclass Person(BaseModel):\n name: str\n age: int\n\nclient = OllamaInstructorClient(...)\nresponse = client.chat_completion_with_stream(\n model='phi3', \n pydantic_model=Person, \n messages=[\n {\n 'role': 'user',\n 'content': 'Jason is 30 years old.'\n }\n ]\n)\n\nfor chunk in response:\n print(chunk['message']['content'])\n```\n\n## OllamaInstructorClient and OllamaInstructorAsyncClient\n\nThe classes `OllamaInstructorClient` and `OllamaInstructorAsyncClient` are the main class of the `ollama-instructor` library. They are the wrapper around the `Ollama` client and contain the following arguments:\n- `host`: the URL of the Ollama server (default: `http://localhost:11434`). See documentation of [Ollama](https://github.com/ollama/ollama)\n- `debug`: a `bool` indicating whether to print debug messages (default: `False`). \n\n> **Note**: I am currently working with `iceream` for the debug messages. Will try to improve that in further development of this library.\n\n### chat_completion & chat_completion_with_stream\n\nThe `chat_completion` and `chat_completion_with_stream` methods are the main methods of the library. They are used to generate text completions from a given prompt.\n\n`ollama-instructor` uses `chat_completion` and `chat_completion_with_stream` to expand the `chat` method of `Ollama`. For all available arguments of `chat` see the [Ollama documentation](https://github.com/ollama/ollama).\n\nThe following arguments are added to the `chat` method within `chat_completion` and `chat_completion_with_stream`:\n- `pydantic_model`: a class of Pydantic's `BaseModel` class that is used to firstly instruct the LLM with the JSON schema of the `BaseModel` and secondly to validate the response of the LLM with the built-in validation of [Pydantic](https://docs.pydantic.dev/latest/).\n- `retries`: the number of retries if the LLM fails to generate a valid response (default: `3`). If a LLM fails the retry will provide the last response of the LLM with the given `ValidationError` and insructs it to generate a valid response.\n- `allow_partial`: If set to `True` `ollama-instructor` will modify the `BaseModel` to allow partial responses. In this case it makes sure to provide the correct instance of the JSON schema but with default or None values. Therefore, it is useful to provide default values within the `BaseModel`. With the improvement of this library you will find examples and best practice guides on that topic in the [docs](/docs/) folder.\n\n\n## Documentation and examples\n- It was always my goal to have a well documented library. Therefore, have a look into the repositorys code to get an idea how to use it.\n- There will be a great bunch of how-to-use guides and examples in the [docs](/docs/) folder (coming soon).\n- If you need more information about the library, please feel free to open an issue.\n\n\n## License\n\n`ollama-instructor` is released under the MIT License. See the [LICENSE](LICENSE) file for more details.\n\n\n## Support and Community\n\nIf you need help or want to discuss `ollama-instructor`, feel free to open an issue on GitHub.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Instruct and validate structured outputs from LLMs with Ollama.",
"version": "0.2.0",
"project_urls": {
"Documentation": "https://github.com/lennartpollvogt/ollama-instructor/tree/main/docs",
"Homepage": "https://github.com/lennartpollvogt/ollama-instructor",
"Repository": "https://github.com/lennartpollvogt/ollama-instructor"
},
"split_keywords": [
"ollama",
" pydantic",
" validation",
" json-schema",
" json",
" instructor",
" prompting",
" local-llm",
" llm"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ceba5b6ad228aa92b6726ce12db449eaee4e8f340432f323eb281ec28f6538bc",
"md5": "3feace7f0493407fc62ed68d7080acfd",
"sha256": "e5f00f3486bfd8d3843656e6a13481aaeeb521f8e5b3bf5d11ddc8869fa05eeb"
},
"downloads": -1,
"filename": "ollama_instructor-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "3feace7f0493407fc62ed68d7080acfd",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8",
"size": 14055,
"upload_time": "2024-06-16T15:05:18",
"upload_time_iso_8601": "2024-06-16T15:05:18.925060Z",
"url": "https://files.pythonhosted.org/packages/ce/ba/5b6ad228aa92b6726ce12db449eaee4e8f340432f323eb281ec28f6538bc/ollama_instructor-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4a8867d51b06e55a3863e90169a21c2194cbdfd0d5063cfaf941e09c572e01ad",
"md5": "105151f0db493237567b8b46dcd56800",
"sha256": "bc022ce30ae779c092cb42e89cd2a04c3bb8117e5e6dc0a0bf70b7a2be40bb8d"
},
"downloads": -1,
"filename": "ollama_instructor-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "105151f0db493237567b8b46dcd56800",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8",
"size": 13692,
"upload_time": "2024-06-16T15:05:20",
"upload_time_iso_8601": "2024-06-16T15:05:20.304528Z",
"url": "https://files.pythonhosted.org/packages/4a/88/67d51b06e55a3863e90169a21c2194cbdfd0d5063cfaf941e09c572e01ad/ollama_instructor-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-06-16 15:05:20",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "lennartpollvogt",
"github_project": "ollama-instructor",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "ollama-instructor"
}
JSON
