| Name | delos-llmax JSON |
| Version |
0.11.13
JSON |
| download |
| home_page | None |
| Summary | Interface to handle multiple LLMs and AI tools. |
| upload_time | 2024-12-16 16:10:18 |
| maintainer | None |
| docs_url | None |
| author | Delos Intelligence |
| requires_python | <4.0,>=3.8.1 |
| license | None |
| keywords |
ai
llm
generative
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# llmax
Python package to manage most external and internal LLM APIs fluently.
# Installation
To install, run the following command:
```bash
python3 -m pip install delos-llmax
```
# How to use
You first have to define a list of `Deployment` as such, where you need to specify the endpoints, key and deployment_name. Then create the client:
```python
from llmax.clients import MultiAIClient
from llmax.models import Deployment, Model
deployments: dict[Model, Deployment] = {
"gpt-4o": Deployment(
model="gpt-4o",
provider="azure",
deployment_name="gpt-4o-2024-05-13",
api_key=os.getenv("LLMAX_AZURE_OPENAI_SWEDENCENTRAL_KEY", ""),
endpoint=os.getenv("LLMAX_AZURE_OPENAI_SWEDENCENTRAL_ENDPOINT", ""),
),
"whisper-1": Deployment(
model="whisper-1",
provider="azure",
deployment_name="whisper-1",
api_key=os.getenv("LLMAX_AZURE_OPENAI_SWEDENCENTRAL_KEY", ""),
endpoint=os.getenv("LLMAX_AZURE_OPENAI_SWEDENCENTRAL_ENDPOINT", ""),
api_version="2024-02-01",
),
}
client = MultiAIClient(
deployments=deployments,
)
```
Then you should define your input (that can be a text, image or audio, following the openai documentation for instance).
```python
messages = [
{"role": "user", "content": "Raconte moi une blague."},
]
```
And finally get the response:
```python
response = client.invoke_to_str(messages, model)
print(response)
```
# Specificities
When creating the client, you can also specify two functions, *increment_usage* and *get_usage*.
The first one is **Callable[[float, Model], bool]** while the second is **Callable[[], float]**.
*increment_usage* is a function that is called after a call of the llm. The float is the price and Model, the model used. It can therefore be used to update your database. *get_usage* returns whether a condition is met. For instance, it can be a function that calls your database and returns whether the user is still active.
Raw data
{
"_id": null,
"home_page": null,
"name": "delos-llmax",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8.1",
"maintainer_email": null,
"keywords": "AI, LLM, generative",
"author": "Delos Intelligence",
"author_email": "maximiliendedinechin@delosintelligence.fr",
"download_url": "https://files.pythonhosted.org/packages/a1/c5/a2650f359e4833e33e3b397d9a93b9917ca6682c71b76d57a92daaf2636f/delos_llmax-0.11.13.tar.gz",
"platform": null,
"description": "# llmax\n\nPython package to manage most external and internal LLM APIs fluently.\n\n\n# Installation\n\nTo install, run the following command:\n\n```bash\npython3 -m pip install delos-llmax\n```\n\n# How to use\n\nYou first have to define a list of `Deployment` as such, where you need to specify the endpoints, key and deployment_name. Then create the client:\n\n```python\nfrom llmax.clients import MultiAIClient\nfrom llmax.models import Deployment, Model\n\ndeployments: dict[Model, Deployment] = {\n \"gpt-4o\": Deployment(\n model=\"gpt-4o\",\n provider=\"azure\",\n deployment_name=\"gpt-4o-2024-05-13\",\n api_key=os.getenv(\"LLMAX_AZURE_OPENAI_SWEDENCENTRAL_KEY\", \"\"),\n endpoint=os.getenv(\"LLMAX_AZURE_OPENAI_SWEDENCENTRAL_ENDPOINT\", \"\"),\n ),\n \"whisper-1\": Deployment(\n model=\"whisper-1\",\n provider=\"azure\",\n deployment_name=\"whisper-1\",\n api_key=os.getenv(\"LLMAX_AZURE_OPENAI_SWEDENCENTRAL_KEY\", \"\"),\n endpoint=os.getenv(\"LLMAX_AZURE_OPENAI_SWEDENCENTRAL_ENDPOINT\", \"\"),\n api_version=\"2024-02-01\",\n ),\n }\n\nclient = MultiAIClient(\n deployments=deployments,\n )\n```\n\nThen you should define your input (that can be a text, image or audio, following the openai documentation for instance).\n\n```python\nmessages = [\n {\"role\": \"user\", \"content\": \"Raconte moi une blague.\"},\n ]\n```\n\nAnd finally get the response:\n\n```python\nresponse = client.invoke_to_str(messages, model)\nprint(response)\n```\n\n# Specificities\n\nWhen creating the client, you can also specify two functions, *increment_usage* and *get_usage*.\nThe first one is **Callable[[float, Model], bool]** while the second is **Callable[[], float]**.\n*increment_usage* is a function that is called after a call of the llm. The float is the price and Model, the model used. It can therefore be used to update your database. *get_usage* returns whether a condition is met. For instance, it can be a function that calls your database and returns whether the user is still active.\n",
"bugtrack_url": null,
"license": null,
"summary": "Interface to handle multiple LLMs and AI tools.",
"version": "0.11.13",
"project_urls": null,
"split_keywords": [
"ai",
" llm",
" generative"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1ecc4ba5337f3987f19b89f2f656fab1257a49a0021c9c6e081b961f5e0f0139",
"md5": "c7caa44cea2f8e4bad54e2b4f81dd5a6",
"sha256": "3715efbc0df2ecae461a799fa5ec0c3a5836c8bb598e68d7df35ed7210db1a0c"
},
"downloads": -1,
"filename": "delos_llmax-0.11.13-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c7caa44cea2f8e4bad54e2b4f81dd5a6",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8.1",
"size": 25905,
"upload_time": "2024-12-16T16:10:17",
"upload_time_iso_8601": "2024-12-16T16:10:17.789018Z",
"url": "https://files.pythonhosted.org/packages/1e/cc/4ba5337f3987f19b89f2f656fab1257a49a0021c9c6e081b961f5e0f0139/delos_llmax-0.11.13-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a1c5a2650f359e4833e33e3b397d9a93b9917ca6682c71b76d57a92daaf2636f",
"md5": "a4c0ac127928124e621807332c37554f",
"sha256": "49058f697a89797bd0b0238f7e670e75fdecc6b0345f318f6a3f5497ee6bdbe2"
},
"downloads": -1,
"filename": "delos_llmax-0.11.13.tar.gz",
"has_sig": false,
"md5_digest": "a4c0ac127928124e621807332c37554f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8.1",
"size": 17843,
"upload_time": "2024-12-16T16:10:18",
"upload_time_iso_8601": "2024-12-16T16:10:18.919059Z",
"url": "https://files.pythonhosted.org/packages/a1/c5/a2650f359e4833e33e3b397d9a93b9917ca6682c71b76d57a92daaf2636f/delos_llmax-0.11.13.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-16 16:10:18",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "delos-llmax"
}
