# LLM Agent Toolkit: Modular Components for AI Workflows
LLM Agent Toolkit provides minimal, modular interfaces for core components in LLM-based applications. Simplify workflows with stateless interaction, embedding encoders, memory management, tool integration, and data loaders, designed for compatibility and scalability. It prioritizes simplicity and modularity by proposing minimal wrappers designed to work across common tools, discouraging direct access to underlying technologies. Specific implementations and examples will be documented separately in a Cookbook (planned).
PyPI: 
## Attention!!!
Using this toolkit simplifies integration by providing unified and modular interfaces across platforms. Many configurations are intentionally kept at their default settings to prioritize ease of use. However, most of these components are extensible through abstract classes, allowing developers to define their own desired configurations for greater flexibility. While this approach enhances consistency and reduces complexity, advanced customization may require extending the provided abstractions.
For developers requiring full-range customization or access to the latest features, it is recommended to consider using native libraries like `ollama` and `openai` directly.
# Table of Contents
- [LLM Agent Toolkit: Modular Components for AI Workflows](#llm-agent-toolkit-modular-components-for-ai-workflows)
- [Attention!!!](#attention)
- [Table of Contents](#table-of-contents)
- [Dependecies](#dependecies)
- [Installation](#installation)
- [Fundamental Components](#fundamental-components)
- [Core:](#core)
- [Example - Ollama](#example---ollama)
- [Example - OpenAI](#example---openai)
- [Example - DeepSeek](#example---deepseek)
- [Example - Tools](#example---tools)
- [Example - Structured Output](#example---structured-output)
- [Encoder:](#encoder)
- [Memory:](#memory)
- [Tool:](#tool)
- [Loader:](#loader)
- [Chunkers:](#chunkers)
- [Planned Feature](#planned-feature)
- [License](#license)
# Dependecies
* **Ollama:** v0.5.4
# Installation
```bash
# Text Generation + Image Generation
pip install llm-agent-toolkit
# transform text to embedding through transformers's API
pip install llm-agent-toolkit["transformer"]
# transform audio to text, only works on Ubuntu
sudo apt install ffmpeg
pip install llm-agent-toolkit["transcriber"]
# entire package
sudo apt install ffmpeg
pip install llm-agent-toolkit["all"] # entire package
```
# Fundamental Components
## Core:
A stateless chat completion interface to interact with the LLM.
**Purpose**: Serves as the central execution layer that abstracts interaction with the underlying LLM model.
**Features**:
* Supports Text-to-Text and Image-to-Text.
* Enables iterative executions for multi-step workflows.
* Facilitates tool invocation as part of the workflow.
* Support models from OpenAI and Ollama.
* Support `Structured Output`.
### Example - Ollama
```python
from typing import Any
from llm_agent_toolkit import ChatCompletionConfig
from llm_agent_toolkit.core.local import Text_to_Text
CONNECTION_STRING = "http://localhost:11434"
SYSTEM_PROMPT = "You are a faithful assistant."
PROMPT = "Why is the sky blue?"
config = ChatCompletionConfig(
name="qwen2.5:7b", temperature=0.7
)
llm = Text_to_Text(
connection_string=CONNECTION_STRING,
system_prompt=SYSTEM_PROMPT,
config=config,
tools=None
)
responses: list[dict[str, Any]] = llm.run(query=PROMPT, context=None)
for response in responses:
print(response["content"])
```
### Example - OpenAI
```python
from typing import Any
from llm_agent_toolkit import ChatCompletionConfig
from llm_agent_toolkit.core.open_ai import Text_to_Text
SYSTEM_PROMPT = "You are a faithful assistant."
PROMPT = "Why is the sky blue?"
config = ChatCompletionConfig(
name="gpt-4o-mini", temperature=0.7
)
llm = Text_to_Text(
system_prompt=SYSTEM_PROMPT,
config=config,
tools=None
)
responses: list[dict[str, Any]] = llm.run(query=PROMPT, context=None)
for response in responses:
print(response["content"])
```
### Example - DeepSeek
```python
from typing import Any
from llm_agent_toolkit import ChatCompletionConfig
from llm_agent_toolkit.core.deep_seek import Text_to_Text
SYSTEM_PROMPT = "You are a faithful assistant."
PROMPT = "Why is the sky blue?"
config = ChatCompletionConfig(
name="deepseek-chat", temperature=1.0
)
llm = Text_to_Text(
system_prompt=SYSTEM_PROMPT,
config=config,
tools=None
)
responses: list[dict[str, Any]] = llm.run(query=PROMPT, context=None)
for response in responses:
print(response["content"])
```
### Example - Tools
```python
from typing import Any
from llm_agent_toolkit import ChatCompletionConfig
from llm_agent_toolkit.core.local import Text_to_Text
# This example is also compatible with llm_agent_toolkit.core.open_ai.Text_to_Text
# This example is also compatible with llm_agent_toolkit.core.deep_seek.Text_to_Text
from llm_agent_toolkit.tool import LazyTool
def adder(a: int, b: int) -> int:
"""Add a with b.
Args:
a (int): The first number.
b (int): The second number.
Returns:
int: Results
"""
return a + b
async def divider(a: int, b: int) -> float:
"""Divide a by b.
Args:
a (int): The first number.
b (int): The second number.
Returns:
float: Results
Raises:
ValueError: When b is 0
"""
if b == 0:
raise ValueError("Division by zero.")
return a / b
CONNECTION_STRING = "http://localhost:11434"
SYSTEM_PROMPT = "You are a faithful assistant."
PROMPT = "10 + 5 / 5 = ?"
add_tool = LazyTool(adder, is_coroutine_function=False)
div_tool = LazyTool(divider, is_coroutine_function=True)
llm = Text_to_Text(
connection_string=CONNECTION_STRING,
system_prompt=SYSTEM_PROMPT,
config=config,
tools=[add_tool, div_tool],
)
responses: list[dict[str, Any]] = llm.run(query=PROMPT, context=None)
for response in responses:
print(response["content"])
```
### Example - Structured Output
```python
from pydantic import BaseModel
from llm_agent_toolkit import ChatCompletionConfig, ResponseMode
from llm_agent_toolkit.core.local import Text_to_Text_SO
# Other Core that support Structured Output
### * llm_agent_toolkit.core.local.Image_to_Text_SO
### * llm_agent_toolkit.core.open_ai.OAI_StructuredOutput_Core
### * llm_agent_toolkit.core.deep_seek.Text_to_Text_SO # JSON only
# These `Core` does not support `Tool` and multi iteration execution.
# If desired, caller can call `llm.run` iteratively with progressively updated `context`.
# File example-chain.py shows steps to achieve chained execution.
# Define the Schema through pydantic BaseModel
class QnA(BaseModel):
question: str
answer: str
CONNECTION_STRING = "http://localhost:11434"
SYS_PROMPT = "You are a faithful Assistant."
PROMPT = "Write a blog post about physician-assisted suicide (euthanasia)."
CONFIG = ChatCompletionConfig(
name="llama3.2:3b", temperature=0.3, max_tokens=2048, max_output_tokens=1024
)
# Structured Output via Pydantic BaseModel
llm = Text_to_Text_SO(
connection_string=CONNECTION_STRING, system_prompt=SYS_PROMPT, config=CONFIG,
)
response_1 = llm.run(
query=PROMPT, context=None, mode=ResponseMode.SO, format=QnA,
)[0]
# Structured Output via JSON Mode
### It's essential to mention the expected JSON structure in the prompt
### and highlight it to return in JSON format.
SPROMPT = f"""
You are a helpful assistant.
Response Schema:
{
json.dumps(QnA.model_json_schema())
}
Note:
Always response in JSON format without additional comments or explanation.
"""
llm = Text_to_Text_SO(
connection_string=CONNECTION_STRING, system_prompt=SPROMPT, config=CONFIG,
)
response_2 = llm.run(
query=PROMPT, context=None, mode=ResponseMode.JSON
)
```
## Encoder:
A standardized wrapper for embedding models.
**Purpose**: Provides a minimal API to transform text into embeddings, usable with any common embedding model.
**Features**:
* Abstracts away model-specific details (e.g., dimensionality, framework differences).
* Allows flexible integration with downstream components like Memory or retrieval mechanisms.
* Support OpenAI, Ollama and Transformers.
* Support asynchronous operation.
## Memory:
Offers essential context retention capabilities.
**Purpose**: Allows efficient context management without hardcoding database or memory solutions.
**Types**:
1. *Short-term Memory*:
* Maintains recent interactions for session-based context.
2. *Vector Memory*:
* Combines embedding and storage for retrieval-augmented workflows.
* Includes optional metadata management for filtering results.
* Support Faiss and Chroma
3. *Async Vector Memory*:
* Same as Vector Memory with async support.
## Tool:
A unified interface for augmenting the LLM's functionality.
**Purpose**: Provides a lightweight abstraction for tool integration, accommodating both simple and complex tools.
**Features**:
* *Simple Tools*: Lazy wrappers for functions or basic utilities.
* *Complex Tools*: Abstract class for external APIs or multi-step operations.
## Loader:
Responsible for converting raw data into text.
**Purpose**: Handles preprocessing and content extraction from diverse formats.
**Features**:
* Covering limited type of documents, images, and audio files.
## Chunkers:
Utility to split long text into chunks.
**Features**:
* **Basic**:
* *FixedCharacterChunker*: Split text into fixed-size character chunks with optional overlapping.
* *FixedGroupChunker*: Splits text into K chunks. Supporting two levels, `word` and `character`, default is `character`.
* **Semantic**:
* *SemanticChunker*: Split text into semantically coherent chunks.
* *SimulatedAnnealingSemanticChunker*: Enhanced with Simulated Annealing optimization technique.
# Planned Feature
- A Cookbook with detailed implementation examples.
# License
This project is licensed under the GNU General Public License v3.0 License. See the [LICENSE](LICENSE) file for details.
Raw data
{
"_id": null,
"home_page": null,
"name": "llm-agent-toolkit",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "llm, agent, toolkit, large language model, memory management, tool integration, multi-modality interaction, multi-step workflow, vision, tools, structured output, chunking, chromadb, faiss, ollama, openai, deepseek",
"author": "jonah_whaler_2348",
"author_email": "jk_saga@proton.me",
"download_url": "https://files.pythonhosted.org/packages/9e/85/8609a790c5d21fbd4dceb0f1af043ae7fce5259f54c7ea2ecc3730f24d97/llm_agent_toolkit-0.0.25.tar.gz",
"platform": null,
"description": "\n\n# LLM Agent Toolkit: Modular Components for AI Workflows\nLLM Agent Toolkit provides minimal, modular interfaces for core components in LLM-based applications. Simplify workflows with stateless interaction, embedding encoders, memory management, tool integration, and data loaders, designed for compatibility and scalability. It prioritizes simplicity and modularity by proposing minimal wrappers designed to work across common tools, discouraging direct access to underlying technologies. Specific implementations and examples will be documented separately in a Cookbook (planned).\n\nPyPI: \n\n## Attention!!!\nUsing this toolkit simplifies integration by providing unified and modular interfaces across platforms. Many configurations are intentionally kept at their default settings to prioritize ease of use. However, most of these components are extensible through abstract classes, allowing developers to define their own desired configurations for greater flexibility. While this approach enhances consistency and reduces complexity, advanced customization may require extending the provided abstractions. \n\nFor developers requiring full-range customization or access to the latest features, it is recommended to consider using native libraries like `ollama` and `openai` directly.\n\n# Table of Contents\n- [LLM Agent Toolkit: Modular Components for AI Workflows](#llm-agent-toolkit-modular-components-for-ai-workflows)\n - [Attention!!!](#attention)\n- [Table of Contents](#table-of-contents)\n- [Dependecies](#dependecies)\n- [Installation](#installation)\n- [Fundamental Components](#fundamental-components)\n - [Core:](#core)\n - [Example - Ollama](#example---ollama)\n - [Example - OpenAI](#example---openai)\n - [Example - DeepSeek](#example---deepseek)\n - [Example - Tools](#example---tools)\n - [Example - Structured Output](#example---structured-output)\n - [Encoder:](#encoder)\n - [Memory:](#memory)\n - [Tool:](#tool)\n - [Loader:](#loader)\n - [Chunkers:](#chunkers)\n- [Planned Feature](#planned-feature)\n- [License](#license)\n\n# Dependecies\n\n * **Ollama:** v0.5.4\n\n# Installation\n ```bash\n # Text Generation + Image Generation\n pip install llm-agent-toolkit\n\n # transform text to embedding through transformers's API\n pip install llm-agent-toolkit[\"transformer\"] \n \n # transform audio to text, only works on Ubuntu\n sudo apt install ffmpeg\n pip install llm-agent-toolkit[\"transcriber\"]\n\n # entire package\n sudo apt install ffmpeg\n pip install llm-agent-toolkit[\"all\"] # entire package\n ```\n\n# Fundamental Components\n## Core: \n\nA stateless chat completion interface to interact with the LLM.\n\n**Purpose**: Serves as the central execution layer that abstracts interaction with the underlying LLM model.\n\n**Features**:\n* Supports Text-to-Text and Image-to-Text.\n* Enables iterative executions for multi-step workflows.\n* Facilitates tool invocation as part of the workflow.\n* Support models from OpenAI and Ollama.\n* Support `Structured Output`.\n\n### Example - Ollama\n```python\nfrom typing import Any\nfrom llm_agent_toolkit import ChatCompletionConfig\nfrom llm_agent_toolkit.core.local import Text_to_Text\n\nCONNECTION_STRING = \"http://localhost:11434\"\nSYSTEM_PROMPT = \"You are a faithful assistant.\"\nPROMPT = \"Why is the sky blue?\"\n\nconfig = ChatCompletionConfig(\n name=\"qwen2.5:7b\", temperature=0.7\n)\nllm = Text_to_Text(\n connection_string=CONNECTION_STRING,\n system_prompt=SYSTEM_PROMPT,\n config=config,\n tools=None\n)\nresponses: list[dict[str, Any]] = llm.run(query=PROMPT, context=None)\nfor response in responses:\n print(response[\"content\"])\n\n```\n\n### Example - OpenAI\n```python\nfrom typing import Any\nfrom llm_agent_toolkit import ChatCompletionConfig\nfrom llm_agent_toolkit.core.open_ai import Text_to_Text\n\nSYSTEM_PROMPT = \"You are a faithful assistant.\"\nPROMPT = \"Why is the sky blue?\"\n\nconfig = ChatCompletionConfig(\n name=\"gpt-4o-mini\", temperature=0.7\n)\nllm = Text_to_Text(\n system_prompt=SYSTEM_PROMPT,\n config=config,\n tools=None\n)\nresponses: list[dict[str, Any]] = llm.run(query=PROMPT, context=None)\nfor response in responses:\n print(response[\"content\"])\n```\n\n### Example - DeepSeek\n```python\nfrom typing import Any\nfrom llm_agent_toolkit import ChatCompletionConfig\nfrom llm_agent_toolkit.core.deep_seek import Text_to_Text\n\nSYSTEM_PROMPT = \"You are a faithful assistant.\"\nPROMPT = \"Why is the sky blue?\"\n\nconfig = ChatCompletionConfig(\n name=\"deepseek-chat\", temperature=1.0\n)\nllm = Text_to_Text(\n system_prompt=SYSTEM_PROMPT,\n config=config,\n tools=None\n)\nresponses: list[dict[str, Any]] = llm.run(query=PROMPT, context=None)\nfor response in responses:\n print(response[\"content\"])\n```\n\n### Example - Tools\n```python\nfrom typing import Any\nfrom llm_agent_toolkit import ChatCompletionConfig\nfrom llm_agent_toolkit.core.local import Text_to_Text\n# This example is also compatible with llm_agent_toolkit.core.open_ai.Text_to_Text\n# This example is also compatible with llm_agent_toolkit.core.deep_seek.Text_to_Text\nfrom llm_agent_toolkit.tool import LazyTool\n\ndef adder(a: int, b: int) -> int:\n \"\"\"Add a with b.\n\n Args:\n a (int): The first number.\n b (int): The second number.\n\n Returns:\n int: Results\n \"\"\"\n return a + b\n\n\nasync def divider(a: int, b: int) -> float:\n \"\"\"Divide a by b.\n\n Args:\n a (int): The first number.\n b (int): The second number.\n\n Returns:\n float: Results\n\n Raises:\n ValueError: When b is 0\n \"\"\"\n if b == 0:\n raise ValueError(\"Division by zero.\")\n return a / b\n\n\nCONNECTION_STRING = \"http://localhost:11434\"\nSYSTEM_PROMPT = \"You are a faithful assistant.\"\nPROMPT = \"10 + 5 / 5 = ?\"\n\nadd_tool = LazyTool(adder, is_coroutine_function=False)\ndiv_tool = LazyTool(divider, is_coroutine_function=True)\n\nllm = Text_to_Text(\n connection_string=CONNECTION_STRING,\n system_prompt=SYSTEM_PROMPT,\n config=config,\n tools=[add_tool, div_tool],\n)\n\nresponses: list[dict[str, Any]] = llm.run(query=PROMPT, context=None)\nfor response in responses:\n print(response[\"content\"])\n```\n\n### Example - Structured Output\n```python\nfrom pydantic import BaseModel\n\nfrom llm_agent_toolkit import ChatCompletionConfig, ResponseMode\nfrom llm_agent_toolkit.core.local import Text_to_Text_SO\n# Other Core that support Structured Output\n### * llm_agent_toolkit.core.local.Image_to_Text_SO\n### * llm_agent_toolkit.core.open_ai.OAI_StructuredOutput_Core\n### * llm_agent_toolkit.core.deep_seek.Text_to_Text_SO # JSON only\n# These `Core` does not support `Tool` and multi iteration execution.\n# If desired, caller can call `llm.run` iteratively with progressively updated `context`.\n# File example-chain.py shows steps to achieve chained execution.\n\n# Define the Schema through pydantic BaseModel\nclass QnA(BaseModel):\n question: str\n answer: str\n\n\nCONNECTION_STRING = \"http://localhost:11434\"\nSYS_PROMPT = \"You are a faithful Assistant.\"\nPROMPT = \"Write a blog post about physician-assisted suicide (euthanasia).\"\nCONFIG = ChatCompletionConfig(\n name=\"llama3.2:3b\", temperature=0.3, max_tokens=2048, max_output_tokens=1024\n)\n# Structured Output via Pydantic BaseModel\nllm = Text_to_Text_SO(\n connection_string=CONNECTION_STRING, system_prompt=SYS_PROMPT, config=CONFIG,\n)\nresponse_1 = llm.run(\n query=PROMPT, context=None, mode=ResponseMode.SO, format=QnA,\n)[0]\n\n# Structured Output via JSON Mode\n### It's essential to mention the expected JSON structure in the prompt \n### and highlight it to return in JSON format.\nSPROMPT = f\"\"\"\nYou are a helpful assistant.\n\nResponse Schema:\n{\n json.dumps(QnA.model_json_schema())\n}\n\nNote:\nAlways response in JSON format without additional comments or explanation.\n\"\"\"\n\nllm = Text_to_Text_SO(\n connection_string=CONNECTION_STRING, system_prompt=SPROMPT, config=CONFIG,\n)\nresponse_2 = llm.run(\n query=PROMPT, context=None, mode=ResponseMode.JSON\n)\n```\n\n## Encoder:\nA standardized wrapper for embedding models.\n\n**Purpose**: Provides a minimal API to transform text into embeddings, usable with any common embedding model.\n\n**Features**:\n* Abstracts away model-specific details (e.g., dimensionality, framework differences).\n* Allows flexible integration with downstream components like Memory or retrieval mechanisms.\n* Support OpenAI, Ollama and Transformers.\n* Support asynchronous operation.\n\n## Memory: \nOffers essential context retention capabilities.\n\n**Purpose**: Allows efficient context management without hardcoding database or memory solutions.\n\n**Types**:\n1. *Short-term Memory*:\n * Maintains recent interactions for session-based context.\n2. *Vector Memory*:\n * Combines embedding and storage for retrieval-augmented workflows.\n * Includes optional metadata management for filtering results.\n * Support Faiss and Chroma\n3. *Async Vector Memory*:\n * Same as Vector Memory with async support.\n\n## Tool:\nA unified interface for augmenting the LLM's functionality.\n\n**Purpose**: Provides a lightweight abstraction for tool integration, accommodating both simple and complex tools.\n\n**Features**:\n* *Simple Tools*: Lazy wrappers for functions or basic utilities.\n* *Complex Tools*: Abstract class for external APIs or multi-step operations.\n\n## Loader:\nResponsible for converting raw data into text.\n\n**Purpose**: Handles preprocessing and content extraction from diverse formats.\n\n**Features**:\n* Covering limited type of documents, images, and audio files.\n\n## Chunkers:\nUtility to split long text into chunks.\n\n**Features**:\n* **Basic**: \n * *FixedCharacterChunker*: Split text into fixed-size character chunks with optional overlapping.\n * *FixedGroupChunker*: Splits text into K chunks. Supporting two levels, `word` and `character`, default is `character`.\n* **Semantic**:\n * *SemanticChunker*: Split text into semantically coherent chunks.\n * *SimulatedAnnealingSemanticChunker*: Enhanced with Simulated Annealing optimization technique.\n\n# Planned Feature\n- A Cookbook with detailed implementation examples.\n\n# License\nThis project is licensed under the GNU General Public License v3.0 License. See the [LICENSE](LICENSE) file for details.\n\n\n",
"bugtrack_url": null,
"license": "GPLv3",
"summary": "LLM Agent Toolkit provides minimal, modular interfaces for core components in LLM-based applications.",
"version": "0.0.25",
"project_urls": null,
"split_keywords": [
"llm",
" agent",
" toolkit",
" large language model",
" memory management",
" tool integration",
" multi-modality interaction",
" multi-step workflow",
" vision",
" tools",
" structured output",
" chunking",
" chromadb",
" faiss",
" ollama",
" openai",
" deepseek"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c494f8bf5dffced8ce351be842838be9e9604897279cda14e54ab1fa425f5d12",
"md5": "470469796a92f17a0fed4b90fb09299a",
"sha256": "7c37cdeaeaf956bacad30ef486a114535d0eaa0cf196a796b5a178abaecd6ed1"
},
"downloads": -1,
"filename": "llm_agent_toolkit-0.0.25-py3-none-any.whl",
"has_sig": false,
"md5_digest": "470469796a92f17a0fed4b90fb09299a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 114514,
"upload_time": "2025-01-08T09:56:25",
"upload_time_iso_8601": "2025-01-08T09:56:25.840833Z",
"url": "https://files.pythonhosted.org/packages/c4/94/f8bf5dffced8ce351be842838be9e9604897279cda14e54ab1fa425f5d12/llm_agent_toolkit-0.0.25-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9e858609a790c5d21fbd4dceb0f1af043ae7fce5259f54c7ea2ecc3730f24d97",
"md5": "101848e6c97b63a538c0236708ee7625",
"sha256": "a230967fa1188187f2fdd477da11263a634facc3c13e8733cd0d3232de3488d5"
},
"downloads": -1,
"filename": "llm_agent_toolkit-0.0.25.tar.gz",
"has_sig": false,
"md5_digest": "101848e6c97b63a538c0236708ee7625",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 82078,
"upload_time": "2025-01-08T09:56:27",
"upload_time_iso_8601": "2025-01-08T09:56:27.750409Z",
"url": "https://files.pythonhosted.org/packages/9e/85/8609a790c5d21fbd4dceb0f1af043ae7fce5259f54c7ea2ecc3730f24d97/llm_agent_toolkit-0.0.25.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-08 09:56:27",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "llm-agent-toolkit"
}
JSON