# Zrb Ollama
Zrb Ollama is a [pypi](https://pypi.org) package that acts as Ollama's and LangChain's wrapper, allowing you to incorporate LLM into your workflow.
Zrb Ollama is a part of [Zrb](https://pypi.org/project/zrb) ecosystem, but you can install it independently from Zrb.
# Installation
You can install Zrb Ollama by invoking any of the following commands:
```bash
# From pypi
pip install zrb-ollama
# From github
pip install git+https://github.com/state-alchemists/zrb-ollama.git@main
# From directory
pip install --use-feature=in-tree-build path/to/this/directory
```
By default, Zrb Ollama uses Ollama-based LLM. You can install Ollama by visiting the official website: [`https://ollama.ai/`](https://ollama.ai/).
You can, however, change this behavior by setting `ZRB_LLM_PROVIDER` into `openai` or `bedrock`.
# Configuration
You can configure Zrb Ollama using a few environment variables.
## General Configuration
- `ZRB_LLM_PROVIDER`: LLM Provider (i.e., `ollama`, `openai`, `bedrock`). If not specified, Zrb Ollama will use `ollama`
- `ZRB_SYSTEM_PROMPT` Default system prompt
- `ZRB_CHAT_HISTORY_RETENTION`: Default: 3
- `ZRB_EMBEDDING_DB_DIR`: Directory to store document embedding. By default, it is `~/.zrb-embedding`.
- `ZRB_CHAT_HISTORY_FILE`: File to store past conversations. By default, it is `~/.zrb-ollama-history.txt`)
- `ZRB_DOCUMENT_DIRS`: List of directories for RAG, separated by `:`.
## Ollama Configuration
- `ZRB_OLLAMA_BASE_URL`: Default Ollama base URL. If not specified, Zrb Ollama will use `http://localhost:11434`.
- `ZRB_OLLAMA_MODEL`: Default Ollama model. If not specified, Zrb Ollama will use `mistral:latest`.
- `ZRB_OLLAMA_EMBEDDING_MODEL`: Default Ollama embedding model (Used for RAG). If not specified, Zrb Ollama will use `nomic-embed-text`.
## Open AI Configuration
- `OPENAI_API_KEY`: OpenAI API key
- `OPENAI_MODEL`: OpenAI model (Default to `gpt-3.5-turbo-instruct`)
- `OPENAI_API_BASE`: OpenAI base URL, default to empty.
## AWS Configuration
- `AWS_ACCESS_KEY`: AWS Access Key
- `AWS_SECRET_ACCESS_KEY`: AWS Secret Key
- `AWS_REGION_NAME`: AWS Region (By default, it is `us-east-1`)
- `BEDROCK_MODEL_ID`: Default bedrock model (By default, it is `anthropic.claude-v2`)
# Talk to Zrb Ollama
You can effortlessly enjoy RAG functionality.
```bash
export OPENAI_API_KEY=your-api-key
export ZRB_LLM_PROVIDER=openai
export ZRB_DOCUMENT_DIRS=${HOME}/borg/src/daily/notes
zrb-ollama
```
Let's see some conversation recaps.
## Who are you?
The AI can answer this question without using any tool.
```
Human: Who are you?
Assistant: I am Assistant, a large language model designed to assist humans in a wide variety of tasks. I am always here to help you with any questions or tasks you may have.
```
## What are you capable of?
Again, the AI can answer this question without using any tool.
```
Human: What are you capable of?
Assistant: I am capable of assisting you in a wide variety of tasks, such as searching for information, executing bash or python scripts, and providing you with notes. Is there something specific you would like me to help you with?
```
## What did I write in my notes?
Now, the AI needs to use an `RAG tool`. It will parse your `ZRB_DOCUMENT_DIRS`, turn that into a vector database, and answer the question for you.
```
Human: What did I write in my notes?
Assistant: You wrote about your experience with a text expander and how it has increased your productivity. You also mentioned using Zrb to define flows and the possibility of using a simple abbreviation for Zrb commands. Additionally, you discussed exploring the click framework and the current issues with Zrb commands, such as being slow and verbose.
```
## Show me the content of my notes!
To answer this question, the AI needs to use `Bash REPL`.
```
Human: I wrote my notes in /home/gofrendi/borg/src/daily/notes/ directory. The notes were written in markdown format (*.md). Show me the content of my notes as is.
Assistant: Here is the content of your notes in markdown format:
# Text expander
- It seems to be simple but highly increase productivity
- Zrb has handle this in some way, and I'm happy with how I can define flows with zrb (despite of the performance problem on my samsung A4 tablet).
- Probably what I need to provide is a simple abreviation for Zrb commands. It is painful already, since I already create a shell function named "ztadd", which is alias to "zrb todo add -t $*"
- Probably I should explore click framework. The problem with zrb commands are currently:
- Slow (This is because of PHP like mechanism. All tasks are rendered everytime Zrb is called)
- Verbose (i.e., `zrb todo add` instead of `ztadd`)
- Use can't add argument right away (i.e, `zrb todo add -t "message"`, rather than `zrb todo add "message"`)
```
## Who is the Founder of Bank Sinarmas?
Finally, the AI also capable to use a search engine.
```
Human: Who is the founder of Bank Sinarmas?
Assistant: Eka Tjipta Widjaja is the founder of Bank Sinarmas.
```
# Getting Creative
The Zrb Ollama CLI program is helpful on its own. You can, for example, ask the LLM model to explain a code for you and refactor it.
```bash
zrb-ollama "What this code do? $(cat fibo.py)"
zrb-ollama "Can you make it better?"
```
There are a lot of things you can do with Zrb Ollama.
# Creating Custom PromptTasks
Finally, you can incorporate Zrb Ollama into your [Zrb](https://pypi.org/project/zrb) project workflow. Zrb Ollama introduces a `PromptTask` class that you can use to create more customized LLM tasks.
Let's see an example:
```python
from zrb import runner
from zrb_ollama import PromptTask
chat = PromptTask(
name="chat",
input_prompt='echo {{ " ".join(input._args) if input._args | length > 0 else "tell me some fun fact" }}', # noqa
)
runner.register(chat)
```
## PromptTask Properties
Each PrompTask has the following properties:
- `name (str)`: The name of the task.
- `history_file (str | None)`: Optional file path for storing conversation history.
- `callback_handler_factories (Iterable[CallbackHandlerFactory])`: Factory for creating CallbackHandler.
- `tool_factories (Iterable[ToolFactory])`: Factory for creating tools.
- `llm_factory (LLMFactory | None)`: Factory for creating LLM.
- `prompt_factory (PromptFactory | None)`: Factory for creating prompt.
- `group (Group | None)`: The group to which this task belongs.
- `description (str)`: Description of the task.
- `inputs (List[AnyInput])`: List of inputs for the task.
- `envs (Iterable[Env])`: Iterable of environment variables for the task.
- `env_files (Iterable[EnvFile])`: Iterable of environment files for the task.
- `icon (str | None)`: Icon for the task.
- `color (str | None)`: Color associated with the task.
- `retry (int)`: Number of retries for the task.
- `retry_interval (float | int)`: Interval between retries.
- `upstreams (Iterable[AnyTask])`: Iterable of upstream tasks.
- `checkers (Iterable[AnyTask])`: Iterable of checker tasks.
- `checking_interval (float | int)`: Interval for checking task status.
- `on_triggered (OnTriggered | None)`: Callback for when the task is triggered.
- `on_waiting (OnWaiting | None)`: Callback for when the task is waiting.
- `on_skipped (OnSkipped | None)`: Callback for when the task is skipped.
- `on_started (OnStarted | None)`: Callback for when the task starts.
- `on_ready (OnReady | None)`: Callback for when the task is ready.
- `on_retry (OnRetry | None)`: Callback for when the task retries.
- `on_failed (OnFailed | None)`: Callback for when the task fails.
- `should_execute (bool | str | Callable[..., bool])`: Condition for executing the task.
- `return_upstream_result (bool)`: Flag to return the result of upstream tasks.
## Factories
To understand what factories are for, first, we need to see what a LangChain program looks like:
```python
import os
import sys
from typing import Any
from langchain import hub
from langchain.agents import AgentExecutor, Tool, create_react_agent
from langchain_community.chat_models import ChatOllama
from langchain_community.utilities.duckduckgo_search import DuckDuckGoSearchAPIWrapper
from langchain.prompts import PromptTemplate
tools = [
Tool(
name="Search",
func=DuckDuckGoSearchAPIWrapper().run,
description="Search engine to answer questions about current events",
)
]
prompt = hub.pull("hwchase17/react-chat")
llm = ChatOllama(
model="mistral",
temperature=0.9,
)
agent = create_react_agent(llm=llm, tools=tools, prompt=prompt)
agent_executor = AgentExecutor(
agent=agent,
tools=tools,
handle_parsing_errors=True,
)
result = agent_executor.invoke(
{
# "input": "Who am I?",
"input": "How many people live in Canada right now?",
"chat_history": "Human: Hi! My name is Bob\nAI: Hello Bob! Nice to meet you",
}
)
```
You can see a lot of things going on. But let's focus on the `agent`. You can see that you need a few other components to create an `agent`:
- `llm`
- `tools`
- `prompt`
LangChain allows you to swap the component with anything if the interface matches. For example, you can use bot `OpenAIChat` and `OllamaChat` as `llm`.
PromptTask handles this by allowing you to define how to create elements based on other existing components. Let's see the following pseudo-code:
```python
class PromptTask(AnyPromptTask, BaseTask):
def __init__(
self, user_prompt, llm_factory, prompt_factory, tool_factories
):
self.user_prompt = user_prompt
self.llm_factory = llm_factory
self.prompt_factory = prompt_factory
self.tool_factories = tool_factories
def run():
agent = self.get_agent()
agent_executor = AgentExecutor(
agent=agent,
tools=self.get_tools(),
handle_parsing_errors=True,
)
result = agent_executor.invoke(
{
# "input": "Who am I?",
"input": "How many people live in Canada right now?",
"chat_history": "Human: Hi! My name is Bob\nAI: Hello Bob! Nice to meet you",
}
)
return result["output"]
def get_agent(self):
return Agent(
llm=self.get_llm(),
prompt=self.get_prompt(),
tools=self.get_tools(),
)
@lru_cache(maxsize=1)
def get_llm(self):
return self.llm_factory(self)
@lru_cache(maxsize=1)
def get_prompt(self):
return self.prompt_factory(self)
@lru_cache(maxsize=1)
def get_tools(self):
return [
tool_factory(self)
for tool_factory in self.llm_chain_factories
]
```
Now, you can control how `get_llm`, `get_prompt`, and `get_tools` behave by setting up the factory properties.
The `lru_cache` also ensures that the getter method will only be called once or less, so you won't lose reference to the components (i.e., when you call `get_llm` twice, the result will refer to the same object).
### How Factories Work
Let's continue with factories:
```python
def ollama_llm_factory(model, temperature):
def create_ollama_llm(task)
return ChatOllama(
model=model,
callback_manager=CallbackManager([StreamingStdOutCallbackHandler()]),
temperature=temperature,
)
return create_ollama_llm
def openai_llm_factory(api_key, temperature):
def create_openai_llm(task)
return ChatOpenAI(
api_key,
callback_manager=CallbackManager([StreamingStdOutCallbackHandler()]),
streaming=True
temperature=temperature,
)
return create_openai_llm
prompt_task = PromptTask(
user_prompt='Why is the sky blue?',
llm_factory=ollama_llm_factory(),
# ...
)
```
We will see how things work in detail by focusing on `PromptTask`'s `run`, `get_agent`, and `get_llm` methods.
```python
class PromptTask(AnyPromptTask, BaseTask):
# ...
def run(self):
agent = self.get_agent()
agent_executor = AgentExecutor(
agent=agent,
tools=self.get_tools(),
handle_parsing_errors=True,
)
result = agent_executor.invoke(
{
# "input": "Who am I?",
"input": "How many people live in Canada right now?",
"chat_history": "Human: Hi! My name is Bob\nAI: Hello Bob! Nice to meet you",
}
)
return result["output"]
def get_agent(self):
return Agent(
llm=self.get_llm(),
prompt=self.get_prompt(),
tools=self.get_tools(),
)
@lru_cache(maxsize=1)
def get_llm(self):
return self.llm_factory(self)
# ...
```
When Zrb calls `prompt_task.run()`, PromptTask will invoke `get_llm_chain` to get the `llm_chain`.
### The Advantage
By using factories, we create a dependency inversion mechanism. The mechanism allows you to:
- Only create components whenever necessary
- Swap components painlessly
- Implement your custom factory without affecting the other components
# For maintainers
## Publish to pypi
To publish zrb-ollama, you need to have a `Pypi` account:
- Log in or register to [https://pypi.org/](https://pypi.org/)
- Create an API token
You can also create a `TestPypi` account:
- Log in or register to [https://test.pypi.org/](https://test.pypi.org/)
- Create an API token
Once you have your API token, you need to create a `~/.pypirc` file:
```
[distutils]
index-servers =
pypi
testpypi
[pypi]
repository = https://upload.pypi.org/legacy/
username = __token__
password = pypi-xxx-xxx
[testpypi]
repository = https://test.pypi.org/legacy/
username = __token__
password = pypi-xxx-xxx
```
To publish zrb-ollama, you can do the following command:
```bash
zrb plugin publish
```
## Updating version
You can update zrb-ollama version by modifying the following section in `pyproject.toml`:
```toml
[project]
version = "0.0.2"
```
## Adding dependencies
To add zrb-ollama dependencies, you can edit the following section in `pyproject.toml`:
```toml
[project]
dependencies = [
"Jinja2==3.1.2",
"jsons==1.6.3"
]
```
## Adding script
To make zrb-package-name executable, you can edit the following section in `pyproject.toml`:
```toml
[project-scripts]
zrb-ollama = "zrb-ollama.__main__:hello"
```
This will look for `hello` callable inside of your `__main__.py` file
Raw data
{
"_id": null,
"home_page": "https://github.com/state-alchemists/zrb-ollama",
"name": "zrb-ollama",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0.0,>=3.10.0",
"maintainer_email": null,
"keywords": "Ollama, OpenAI, Bedrock",
"author": "Go Frendi",
"author_email": "gofrendiasgard@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/61/7d/4c7c6c29fe200c6eccbdd7de77828258f5f93b40cb46d9e95f418aa4dfa2/zrb_ollama-0.0.31.tar.gz",
"platform": null,
"description": "\n\n# Zrb Ollama\n\nZrb Ollama is a [pypi](https://pypi.org) package that acts as Ollama's and LangChain's wrapper, allowing you to incorporate LLM into your workflow.\n\nZrb Ollama is a part of [Zrb](https://pypi.org/project/zrb) ecosystem, but you can install it independently from Zrb. \n\n# Installation\n\nYou can install Zrb Ollama by invoking any of the following commands:\n\n```bash\n# From pypi\npip install zrb-ollama\n# From github\npip install git+https://github.com/state-alchemists/zrb-ollama.git@main\n# From directory\npip install --use-feature=in-tree-build path/to/this/directory\n```\n\nBy default, Zrb Ollama uses Ollama-based LLM. You can install Ollama by visiting the official website: [`https://ollama.ai/`](https://ollama.ai/).\n\nYou can, however, change this behavior by setting `ZRB_LLM_PROVIDER` into `openai` or `bedrock`.\n\n# Configuration\n\nYou can configure Zrb Ollama using a few environment variables.\n\n## General Configuration\n\n- `ZRB_LLM_PROVIDER`: LLM Provider (i.e., `ollama`, `openai`, `bedrock`). If not specified, Zrb Ollama will use `ollama`\n- `ZRB_SYSTEM_PROMPT` Default system prompt\n- `ZRB_CHAT_HISTORY_RETENTION`: Default: 3\n- `ZRB_EMBEDDING_DB_DIR`: Directory to store document embedding. By default, it is `~/.zrb-embedding`.\n- `ZRB_CHAT_HISTORY_FILE`: File to store past conversations. By default, it is `~/.zrb-ollama-history.txt`)\n- `ZRB_DOCUMENT_DIRS`: List of directories for RAG, separated by `:`.\n\n## Ollama Configuration\n\n- `ZRB_OLLAMA_BASE_URL`: Default Ollama base URL. If not specified, Zrb Ollama will use `http://localhost:11434`.\n- `ZRB_OLLAMA_MODEL`: Default Ollama model. If not specified, Zrb Ollama will use `mistral:latest`.\n- `ZRB_OLLAMA_EMBEDDING_MODEL`: Default Ollama embedding model (Used for RAG). If not specified, Zrb Ollama will use `nomic-embed-text`.\n\n## Open AI Configuration\n\n- `OPENAI_API_KEY`: OpenAI API key\n- `OPENAI_MODEL`: OpenAI model (Default to `gpt-3.5-turbo-instruct`)\n- `OPENAI_API_BASE`: OpenAI base URL, default to empty.\n\n## AWS Configuration\n\n- `AWS_ACCESS_KEY`: AWS Access Key\n- `AWS_SECRET_ACCESS_KEY`: AWS Secret Key\n- `AWS_REGION_NAME`: AWS Region (By default, it is `us-east-1`)\n- `BEDROCK_MODEL_ID`: Default bedrock model (By default, it is `anthropic.claude-v2`)\n\n# Talk to Zrb Ollama\n\nYou can effortlessly enjoy RAG functionality.\n\n```bash\nexport OPENAI_API_KEY=your-api-key\nexport ZRB_LLM_PROVIDER=openai\nexport ZRB_DOCUMENT_DIRS=${HOME}/borg/src/daily/notes\nzrb-ollama\n```\n\nLet's see some conversation recaps.\n\n## Who are you?\n\nThe AI can answer this question without using any tool.\n\n```\nHuman: Who are you?\n\nAssistant: I am Assistant, a large language model designed to assist humans in a wide variety of tasks. I am always here to help you with any questions or tasks you may have.\n```\n\n## What are you capable of?\n\nAgain, the AI can answer this question without using any tool.\n\n```\nHuman: What are you capable of?\n\nAssistant: I am capable of assisting you in a wide variety of tasks, such as searching for information, executing bash or python scripts, and providing you with notes. Is there something specific you would like me to help you with?\n```\n\n## What did I write in my notes?\n\nNow, the AI needs to use an `RAG tool`. It will parse your `ZRB_DOCUMENT_DIRS`, turn that into a vector database, and answer the question for you.\n\n```\nHuman: What did I write in my notes?\n\nAssistant: You wrote about your experience with a text expander and how it has increased your productivity. You also mentioned using Zrb to define flows and the possibility of using a simple abbreviation for Zrb commands. Additionally, you discussed exploring the click framework and the current issues with Zrb commands, such as being slow and verbose.\n```\n\n## Show me the content of my notes!\n\nTo answer this question, the AI needs to use `Bash REPL`.\n\n```\nHuman: I wrote my notes in /home/gofrendi/borg/src/daily/notes/ directory. The notes were written in markdown format (*.md). Show me the content of my notes as is.\nAssistant: Here is the content of your notes in markdown format:\n\n# Text expander\n\n- It seems to be simple but highly increase productivity\n- Zrb has handle this in some way, and I'm happy with how I can define flows with zrb (despite of the performance problem on my samsung A4 tablet).\n- Probably what I need to provide is a simple abreviation for Zrb commands. It is painful already, since I already create a shell function named \"ztadd\", which is alias to \"zrb todo add -t $*\"\n- Probably I should explore click framework. The problem with zrb commands are currently:\n - Slow (This is because of PHP like mechanism. All tasks are rendered everytime Zrb is called)\n - Verbose (i.e., `zrb todo add` instead of `ztadd`)\n - Use can't add argument right away (i.e, `zrb todo add -t \"message\"`, rather than `zrb todo add \"message\"`)\n```\n\n## Who is the Founder of Bank Sinarmas?\n\nFinally, the AI also capable to use a search engine.\n\n```\nHuman: Who is the founder of Bank Sinarmas?\nAssistant: Eka Tjipta Widjaja is the founder of Bank Sinarmas.\n```\n\n\n# Getting Creative\n\nThe Zrb Ollama CLI program is helpful on its own. You can, for example, ask the LLM model to explain a code for you and refactor it.\n\n```bash\nzrb-ollama \"What this code do? $(cat fibo.py)\"\nzrb-ollama \"Can you make it better?\"\n```\n\nThere are a lot of things you can do with Zrb Ollama.\n\n# Creating Custom PromptTasks\n\nFinally, you can incorporate Zrb Ollama into your [Zrb](https://pypi.org/project/zrb) project workflow. Zrb Ollama introduces a `PromptTask` class that you can use to create more customized LLM tasks.\n\nLet's see an example:\n\n```python\nfrom zrb import runner\n\nfrom zrb_ollama import PromptTask\n\nchat = PromptTask(\n name=\"chat\",\n input_prompt='echo {{ \" \".join(input._args) if input._args | length > 0 else \"tell me some fun fact\" }}', # noqa\n)\nrunner.register(chat)\n```\n\n## PromptTask Properties\n\nEach PrompTask has the following properties:\n\n- `name (str)`: The name of the task.\n- `history_file (str | None)`: Optional file path for storing conversation history.\n- `callback_handler_factories (Iterable[CallbackHandlerFactory])`: Factory for creating CallbackHandler.\n- `tool_factories (Iterable[ToolFactory])`: Factory for creating tools.\n- `llm_factory (LLMFactory | None)`: Factory for creating LLM.\n- `prompt_factory (PromptFactory | None)`: Factory for creating prompt.\n- `group (Group | None)`: The group to which this task belongs.\n- `description (str)`: Description of the task.\n- `inputs (List[AnyInput])`: List of inputs for the task.\n- `envs (Iterable[Env])`: Iterable of environment variables for the task.\n- `env_files (Iterable[EnvFile])`: Iterable of environment files for the task.\n- `icon (str | None)`: Icon for the task.\n- `color (str | None)`: Color associated with the task.\n- `retry (int)`: Number of retries for the task.\n- `retry_interval (float | int)`: Interval between retries.\n- `upstreams (Iterable[AnyTask])`: Iterable of upstream tasks.\n- `checkers (Iterable[AnyTask])`: Iterable of checker tasks.\n- `checking_interval (float | int)`: Interval for checking task status.\n- `on_triggered (OnTriggered | None)`: Callback for when the task is triggered.\n- `on_waiting (OnWaiting | None)`: Callback for when the task is waiting.\n- `on_skipped (OnSkipped | None)`: Callback for when the task is skipped.\n- `on_started (OnStarted | None)`: Callback for when the task starts.\n- `on_ready (OnReady | None)`: Callback for when the task is ready.\n- `on_retry (OnRetry | None)`: Callback for when the task retries.\n- `on_failed (OnFailed | None)`: Callback for when the task fails.\n- `should_execute (bool | str | Callable[..., bool])`: Condition for executing the task.\n- `return_upstream_result (bool)`: Flag to return the result of upstream tasks.\n\n## Factories\n\nTo understand what factories are for, first, we need to see what a LangChain program looks like:\n\n```python\nimport os\nimport sys\nfrom typing import Any\n\nfrom langchain import hub\nfrom langchain.agents import AgentExecutor, Tool, create_react_agent\nfrom langchain_community.chat_models import ChatOllama\nfrom langchain_community.utilities.duckduckgo_search import DuckDuckGoSearchAPIWrapper\nfrom langchain.prompts import PromptTemplate\n\n\ntools = [\n Tool(\n name=\"Search\",\n func=DuckDuckGoSearchAPIWrapper().run,\n description=\"Search engine to answer questions about current events\",\n )\n]\n\nprompt = hub.pull(\"hwchase17/react-chat\")\n\nllm = ChatOllama(\n model=\"mistral\",\n temperature=0.9,\n)\n\nagent = create_react_agent(llm=llm, tools=tools, prompt=prompt)\n\nagent_executor = AgentExecutor(\n agent=agent,\n tools=tools,\n handle_parsing_errors=True,\n)\n\nresult = agent_executor.invoke(\n {\n # \"input\": \"Who am I?\",\n \"input\": \"How many people live in Canada right now?\",\n \"chat_history\": \"Human: Hi! My name is Bob\\nAI: Hello Bob! Nice to meet you\",\n }\n)\n```\n\nYou can see a lot of things going on. But let's focus on the `agent`. You can see that you need a few other components to create an `agent`:\n\n- `llm`\n- `tools`\n- `prompt`\n\nLangChain allows you to swap the component with anything if the interface matches. For example, you can use bot `OpenAIChat` and `OllamaChat` as `llm`.\n\nPromptTask handles this by allowing you to define how to create elements based on other existing components. Let's see the following pseudo-code:\n\n```python\nclass PromptTask(AnyPromptTask, BaseTask):\n def __init__(\n self, user_prompt, llm_factory, prompt_factory, tool_factories\n ):\n self.user_prompt = user_prompt\n self.llm_factory = llm_factory\n self.prompt_factory = prompt_factory\n self.tool_factories = tool_factories\n \n def run():\n agent = self.get_agent()\n agent_executor = AgentExecutor(\n agent=agent,\n tools=self.get_tools(),\n handle_parsing_errors=True,\n )\n result = agent_executor.invoke(\n {\n # \"input\": \"Who am I?\",\n \"input\": \"How many people live in Canada right now?\",\n \"chat_history\": \"Human: Hi! My name is Bob\\nAI: Hello Bob! Nice to meet you\",\n }\n )\n return result[\"output\"]\n \n def get_agent(self):\n return Agent(\n llm=self.get_llm(),\n prompt=self.get_prompt(),\n tools=self.get_tools(),\n )\n\n @lru_cache(maxsize=1)\n def get_llm(self):\n return self.llm_factory(self) \n\n @lru_cache(maxsize=1)\n def get_prompt(self):\n return self.prompt_factory(self)\n\n @lru_cache(maxsize=1)\n def get_tools(self):\n return [\n tool_factory(self)\n for tool_factory in self.llm_chain_factories\n ]\n```\n\nNow, you can control how `get_llm`, `get_prompt`, and `get_tools` behave by setting up the factory properties.\n\nThe `lru_cache` also ensures that the getter method will only be called once or less, so you won't lose reference to the components (i.e., when you call `get_llm` twice, the result will refer to the same object).\n\n### How Factories Work\n\nLet's continue with factories:\n\n```python\ndef ollama_llm_factory(model, temperature):\n def create_ollama_llm(task)\n return ChatOllama(\n model=model,\n callback_manager=CallbackManager([StreamingStdOutCallbackHandler()]),\n temperature=temperature,\n ) \n return create_ollama_llm\n\ndef openai_llm_factory(api_key, temperature):\n def create_openai_llm(task)\n return ChatOpenAI(\n api_key,\n callback_manager=CallbackManager([StreamingStdOutCallbackHandler()]),\n streaming=True\n temperature=temperature,\n ) \n return create_openai_llm\n\n\nprompt_task = PromptTask(\n user_prompt='Why is the sky blue?',\n llm_factory=ollama_llm_factory(),\n # ...\n)\n```\n\nWe will see how things work in detail by focusing on `PromptTask`'s `run`, `get_agent`, and `get_llm` methods.\n\n```python\nclass PromptTask(AnyPromptTask, BaseTask):\n # ...\n \n def run(self):\n agent = self.get_agent()\n agent_executor = AgentExecutor(\n agent=agent,\n tools=self.get_tools(),\n handle_parsing_errors=True,\n )\n result = agent_executor.invoke(\n {\n # \"input\": \"Who am I?\",\n \"input\": \"How many people live in Canada right now?\",\n \"chat_history\": \"Human: Hi! My name is Bob\\nAI: Hello Bob! Nice to meet you\",\n }\n )\n return result[\"output\"]\n\n def get_agent(self):\n return Agent(\n llm=self.get_llm(),\n prompt=self.get_prompt(),\n tools=self.get_tools(),\n )\n\n @lru_cache(maxsize=1)\n def get_llm(self):\n return self.llm_factory(self) \n\n # ...\n```\n\nWhen Zrb calls `prompt_task.run()`, PromptTask will invoke `get_llm_chain` to get the `llm_chain`.\n\n### The Advantage\n\nBy using factories, we create a dependency inversion mechanism. The mechanism allows you to:\n\n- Only create components whenever necessary\n- Swap components painlessly\n- Implement your custom factory without affecting the other components\n\n\n# For maintainers\n\n## Publish to pypi\n\nTo publish zrb-ollama, you need to have a `Pypi` account:\n\n- Log in or register to [https://pypi.org/](https://pypi.org/)\n- Create an API token\n\nYou can also create a `TestPypi` account:\n\n- Log in or register to [https://test.pypi.org/](https://test.pypi.org/)\n- Create an API token\n\nOnce you have your API token, you need to create a `~/.pypirc` file:\n\n```\n[distutils]\nindex-servers =\n pypi\n testpypi\n\n[pypi]\n repository = https://upload.pypi.org/legacy/\n username = __token__\n password = pypi-xxx-xxx\n[testpypi]\n repository = https://test.pypi.org/legacy/\n username = __token__\n password = pypi-xxx-xxx\n```\n\nTo publish zrb-ollama, you can do the following command:\n\n```bash\nzrb plugin publish\n```\n\n## Updating version\n\nYou can update zrb-ollama version by modifying the following section in `pyproject.toml`:\n\n```toml\n[project]\nversion = \"0.0.2\"\n```\n\n## Adding dependencies\n\nTo add zrb-ollama dependencies, you can edit the following section in `pyproject.toml`:\n\n```toml\n[project]\ndependencies = [\n \"Jinja2==3.1.2\",\n \"jsons==1.6.3\"\n]\n```\n\n## Adding script\n\nTo make zrb-package-name executable, you can edit the following section in `pyproject.toml`:\n\n```toml\n[project-scripts]\nzrb-ollama = \"zrb-ollama.__main__:hello\"\n```\n\nThis will look for `hello` callable inside of your `__main__.py` file\n",
"bugtrack_url": null,
"license": "AGPL-3.0-or-later",
"summary": "Zrb LLM plugin",
"version": "0.0.31",
"project_urls": {
"Documentation": "https://github.com/state-alchemists/zrb-ollama",
"Homepage": "https://github.com/state-alchemists/zrb-ollama",
"Repository": "https://github.com/state-alchemists/zrb-ollama"
},
"split_keywords": [
"ollama",
" openai",
" bedrock"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f99b1329c4f1e52bba30ae532e98e1e0b73107fc8f7c2ad171dc38d27632b289",
"md5": "1de251cf4cd218a4414237d608b03866",
"sha256": "d3bc5343e73c1e0c9702c0779d4112a73130b625c59d7019fb45c58235d4a996"
},
"downloads": -1,
"filename": "zrb_ollama-0.0.31-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1de251cf4cd218a4414237d608b03866",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0.0,>=3.10.0",
"size": 26788,
"upload_time": "2024-04-21T09:06:01",
"upload_time_iso_8601": "2024-04-21T09:06:01.962527Z",
"url": "https://files.pythonhosted.org/packages/f9/9b/1329c4f1e52bba30ae532e98e1e0b73107fc8f7c2ad171dc38d27632b289/zrb_ollama-0.0.31-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "617d4c7c6c29fe200c6eccbdd7de77828258f5f93b40cb46d9e95f418aa4dfa2",
"md5": "a0083e955b6d07e96baa6c2b4de5328a",
"sha256": "c1288f5f6b8a0c8dda63afb69570a15d175849cc3e5223b90b5edf1c1802c7e4"
},
"downloads": -1,
"filename": "zrb_ollama-0.0.31.tar.gz",
"has_sig": false,
"md5_digest": "a0083e955b6d07e96baa6c2b4de5328a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0.0,>=3.10.0",
"size": 22291,
"upload_time": "2024-04-21T09:06:05",
"upload_time_iso_8601": "2024-04-21T09:06:05.298587Z",
"url": "https://files.pythonhosted.org/packages/61/7d/4c7c6c29fe200c6eccbdd7de77828258f5f93b40cb46d9e95f418aa4dfa2/zrb_ollama-0.0.31.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-21 09:06:05",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "state-alchemists",
"github_project": "zrb-ollama",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "zrb-ollama"
}
JSON
