# ALEA LLM Client
[](https://badge.fury.io/py/alea-llm-client)
[](https://opensource.org/licenses/MIT)
[](https://pypi.org/project/alea-llm-client/)
This is a simple, two-dependency (`httpx`, `pydantic`) LLM client for ~OpenAI APIs like:
* OpenAI
* Anthropic
* VLLM
### Supported Patterns
It provides the following patterns for all endpoints:
* `complete` and `complete_async` -> str via `ModelResponse`
* `chat` and `chat_async` -> str via `ModelResponse`
* `json` and `json_async` -> dict via `JSONModelResponse`
* `pydantic` and `pydantic_async` -> pydantic models
### Default Caching
**Result caching is enabled by default for all methods.**
To disable caching, you can either:
* set `ignore_cache=True` for each method call (`complete`, `chat`, `json`, `pydantic`)
* set `ignore_cache=True` as a kwarg at model construction
Cached objects are stored in `~/.alea/cache/{provider}/{endpoint_model_hash}/{call_hash}.json`
in compressed `.json.gz` format. You can delete these files to clear the cache.
### Authentication
Authentication is handled in the following priority order:
* an `api_key` provided at model construction
* a standard environment variable (e.g., `ANTHROPIC_API_KEY` or `OPENAI_API_KEY`)
* a key stored in `~/.alea/keys/{provider}` (e.g., openai, anthropic)
### Streaming
Given the research focus of this library, streaming generation is not supported. However,
you can directly access the `httpx` objects on `.client` and `.async_client` to stream responses
directly if you prefer.
## Installation
```bash
pip install alea-llm-client
```
## Examples
### Basic JSON Example
```python
from alea_llm_client import VLLMModel
if __name__ == "__main__":
model = VLLMModel(
endpoint="http://my.vllm.server:8000",
model="meta-llama/Meta-Llama-3.1-8B-Instruct"
)
messages = [
{
"role": "user",
"content": "Give me a JSON object with keys 'name' and 'age' for a person named Alice who is 30 years old.",
},
]
print(model.json(messages=messages, system="Respond in JSON.").data)
# Output: {'name': 'Alice', 'age': 30}
```
### Basic Completion Example with KL3M
```python
from alea_llm_client import VLLMModel
if __name__ == "__main__":
model = VLLMModel(
model="kl3m-1.7b", ignore_cache=True
)
prompt = "My name is "
print(model.complete(prompt=prompt, temperature=0.5).text)
# Output: Dr. Hermann Kamenzi, and
```
### Pydantic Example
```python
from pydantic import BaseModel
from alea_llm_client import AnthropicModel
from alea_llm_client.llms.prompts.sections import format_prompt, format_instructions
class Person(BaseModel):
name: str
age: int
if __name__ == "__main__":
model = AnthropicModel(ignore_cache=True)
instructions = [
"Provide one random record based on the SCHEMA below.",
]
prompt = format_prompt(
{
"instructions": format_instructions(instructions),
"schema": Person,
}
)
person = model.pydantic(prompt, system="Respond in JSON.", pydantic_model=Person)
print(person)
# Output: name='Olivia Chen' age=29
```
## Design
### Class Inheritance
```mermaid
classDiagram
BaseAIModel <|-- OpenAICompatibleModel
OpenAICompatibleModel <|-- AnthropicModel
OpenAICompatibleModel <|-- OpenAIModel
OpenAICompatibleModel <|-- VLLMModel
class BaseAIModel {
<<abstract>>
}
class OpenAICompatibleModel
class AnthropicModel
class OpenAIModel
class VLLMModel
```
### Example Call Flow
```mermaid
sequenceDiagram
participant Client
participant BaseAIModel
participant OpenAICompatibleModel
participant SpecificModel
participant API
Client->>BaseAIModel: json()
BaseAIModel->>BaseAIModel: _retry_wrapper()
BaseAIModel->>OpenAICompatibleModel: _json()
OpenAICompatibleModel->>OpenAICompatibleModel: format()
OpenAICompatibleModel->>OpenAICompatibleModel: _make_request()
OpenAICompatibleModel->>API: HTTP POST
API-->>OpenAICompatibleModel: Response
OpenAICompatibleModel->>OpenAICompatibleModel: _handle_json_response()
OpenAICompatibleModel-->>BaseAIModel: JSONModelResponse
BaseAIModel-->>Client: JSONModelResponse
```
## License
The ALEA LLM client is released under the MIT License. See the [LICENSE](LICENSE) file for details.
## Support
If you encounter any issues or have questions about using the ALEA LLM client library, please [open an issue](https://github.com/alea-institute/alea-llm-client/issues) on GitHub.
## Learn More
To learn more about ALEA and its software and research projects like KL3M and leeky, visit the [ALEA website](https://aleainstitute.ai/).
Raw data
{
"_id": null,
"home_page": "https://aleainstitute.ai/",
"name": "alea-llm-client",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0.0,>=3.9",
"maintainer_email": null,
"keywords": "alea, llm, client, api",
"author": "ALEA Institute",
"author_email": "hello@aleainstitute.ai",
"download_url": "https://files.pythonhosted.org/packages/e1/eb/3eb3ebe97332cf244b4445bcf15b4c43c1037752e923a387990ff29a2843/alea_llm_client-0.1.1.tar.gz",
"platform": null,
"description": "# ALEA LLM Client\n\n[](https://badge.fury.io/py/alea-llm-client)\n[](https://opensource.org/licenses/MIT)\n[](https://pypi.org/project/alea-llm-client/)\n\nThis is a simple, two-dependency (`httpx`, `pydantic`) LLM client for ~OpenAI APIs like:\n * OpenAI\n * Anthropic\n * VLLM\n\n### Supported Patterns\n\nIt provides the following patterns for all endpoints:\n * `complete` and `complete_async` -> str via `ModelResponse`\n * `chat` and `chat_async` -> str via `ModelResponse`\n * `json` and `json_async` -> dict via `JSONModelResponse`\n * `pydantic` and `pydantic_async` -> pydantic models\n\n### Default Caching\n\n**Result caching is enabled by default for all methods.**\n\nTo disable caching, you can either:\n * set `ignore_cache=True` for each method call (`complete`, `chat`, `json`, `pydantic`)\n * set `ignore_cache=True` as a kwarg at model construction\n\nCached objects are stored in `~/.alea/cache/{provider}/{endpoint_model_hash}/{call_hash}.json`\nin compressed `.json.gz` format. You can delete these files to clear the cache.\n\n### Authentication\n\nAuthentication is handled in the following priority order:\n * an `api_key` provided at model construction\n * a standard environment variable (e.g., `ANTHROPIC_API_KEY` or `OPENAI_API_KEY`)\n * a key stored in `~/.alea/keys/{provider}` (e.g., openai, anthropic)\n\n### Streaming\n\nGiven the research focus of this library, streaming generation is not supported. However,\nyou can directly access the `httpx` objects on `.client` and `.async_client` to stream responses\ndirectly if you prefer.\n\n## Installation\n\n```bash\npip install alea-llm-client\n```\n\n## Examples\n\n\n### Basic JSON Example\n\n```python\nfrom alea_llm_client import VLLMModel\n\nif __name__ == \"__main__\":\n model = VLLMModel(\n endpoint=\"http://my.vllm.server:8000\",\n model=\"meta-llama/Meta-Llama-3.1-8B-Instruct\"\n )\n\n messages = [\n {\n \"role\": \"user\",\n \"content\": \"Give me a JSON object with keys 'name' and 'age' for a person named Alice who is 30 years old.\",\n },\n ]\n\n print(model.json(messages=messages, system=\"Respond in JSON.\").data)\n\n# Output: {'name': 'Alice', 'age': 30}\n```\n\n### Basic Completion Example with KL3M\n\n```python\nfrom alea_llm_client import VLLMModel\n\nif __name__ == \"__main__\":\n model = VLLMModel(\n model=\"kl3m-1.7b\", ignore_cache=True\n )\n\n prompt = \"My name is \"\n print(model.complete(prompt=prompt, temperature=0.5).text)\n\n# Output: Dr. Hermann Kamenzi, and\n```\n\n### Pydantic Example\n```python\nfrom pydantic import BaseModel\nfrom alea_llm_client import AnthropicModel\nfrom alea_llm_client.llms.prompts.sections import format_prompt, format_instructions\n\nclass Person(BaseModel):\n name: str\n age: int\n\nif __name__ == \"__main__\":\n model = AnthropicModel(ignore_cache=True)\n\n instructions = [\n \"Provide one random record based on the SCHEMA below.\",\n ]\n prompt = format_prompt(\n {\n \"instructions\": format_instructions(instructions),\n \"schema\": Person,\n }\n )\n\n person = model.pydantic(prompt, system=\"Respond in JSON.\", pydantic_model=Person)\n print(person)\n\n# Output: name='Olivia Chen' age=29\n```\n\n\n## Design\n\n### Class Inheritance\n\n```mermaid\nclassDiagram\n BaseAIModel <|-- OpenAICompatibleModel\n OpenAICompatibleModel <|-- AnthropicModel\n OpenAICompatibleModel <|-- OpenAIModel\n OpenAICompatibleModel <|-- VLLMModel\n\n class BaseAIModel {\n <<abstract>>\n }\n class OpenAICompatibleModel\n class AnthropicModel\n class OpenAIModel\n class VLLMModel\n```\n\n### Example Call Flow\n\n```mermaid\nsequenceDiagram\n participant Client\n participant BaseAIModel\n participant OpenAICompatibleModel\n participant SpecificModel\n participant API\n\n Client->>BaseAIModel: json()\n BaseAIModel->>BaseAIModel: _retry_wrapper()\n BaseAIModel->>OpenAICompatibleModel: _json()\n OpenAICompatibleModel->>OpenAICompatibleModel: format()\n OpenAICompatibleModel->>OpenAICompatibleModel: _make_request()\n OpenAICompatibleModel->>API: HTTP POST\n API-->>OpenAICompatibleModel: Response\n OpenAICompatibleModel->>OpenAICompatibleModel: _handle_json_response()\n OpenAICompatibleModel-->>BaseAIModel: JSONModelResponse\n BaseAIModel-->>Client: JSONModelResponse\n```\n\n## License\n\nThe ALEA LLM client is released under the MIT License. See the [LICENSE](LICENSE) file for details.\n\n## Support\n\nIf you encounter any issues or have questions about using the ALEA LLM client library, please [open an issue](https://github.com/alea-institute/alea-llm-client/issues) on GitHub.\n\n## Learn More\n\nTo learn more about ALEA and its software and research projects like KL3M and leeky, visit the [ALEA website](https://aleainstitute.ai/).\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "ALEA LLM client abstraction library for Python",
"version": "0.1.1",
"project_urls": {
"Homepage": "https://aleainstitute.ai/",
"Repository": "https://github.com/alea-institute/alea-llm-client"
},
"split_keywords": [
"alea",
" llm",
" client",
" api"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8932f6bf0679867e734ce6285a69ceed2dbe9a96c6c8c0ba0a7995afa2785bbd",
"md5": "5dd01e2f93eac7d61e6dcf15d65838ff",
"sha256": "5db16d9f5cbc643a16d5ee1bc2ac8e0b3bedae9480eb5650ea9bce2b2da33d7c"
},
"downloads": -1,
"filename": "alea_llm_client-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "5dd01e2f93eac7d61e6dcf15d65838ff",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0.0,>=3.9",
"size": 25601,
"upload_time": "2024-09-11T14:36:43",
"upload_time_iso_8601": "2024-09-11T14:36:43.754656Z",
"url": "https://files.pythonhosted.org/packages/89/32/f6bf0679867e734ce6285a69ceed2dbe9a96c6c8c0ba0a7995afa2785bbd/alea_llm_client-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e1eb3eb3ebe97332cf244b4445bcf15b4c43c1037752e923a387990ff29a2843",
"md5": "387d4971350612e92fad43c91bec6e27",
"sha256": "dc3b063851424da526c23337ccd31eb9bfb1232f9512f889b4f055071cc72b3a"
},
"downloads": -1,
"filename": "alea_llm_client-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "387d4971350612e92fad43c91bec6e27",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0.0,>=3.9",
"size": 20215,
"upload_time": "2024-09-11T14:36:45",
"upload_time_iso_8601": "2024-09-11T14:36:45.468633Z",
"url": "https://files.pythonhosted.org/packages/e1/eb/3eb3ebe97332cf244b4445bcf15b4c43c1037752e923a387990ff29a2843/alea_llm_client-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-11 14:36:45",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "alea-institute",
"github_project": "alea-llm-client",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "alea-llm-client"
}
JSON
