| Name | delos-llmax JSON |
| Version |
0.11.27
JSON |
| download |
| home_page | None |
| Summary | Interface to handle multiple LLMs and AI tools. |
| upload_time | 2025-02-11 16:53:51 |
| maintainer | None |
| docs_url | None |
| author | Delos Intelligence |
| requires_python | <4.0,>=3.8.1 |
| license | None |
| keywords |
ai
llm
generative
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# llmax
Python package to manage most external and internal LLM APIs fluently.
# Installation
To install, run the following command:
```bash
python3 -m pip install delos-llmax
```
# How to use
You first have to define a list of `Deployment` as such, where you need to specify the endpoints, key and deployment_name. Then create the client:
```python
from llmax.clients import MultiAIClient
from llmax.models import Deployment, Model
deployments: dict[Model, Deployment] = {
"gpt-4o": Deployment(
model="gpt-4o",
provider="azure",
deployment_name="gpt-4o-2024-05-13",
api_key=os.getenv("LLMAX_AZURE_OPENAI_SWEDENCENTRAL_KEY", ""),
endpoint=os.getenv("LLMAX_AZURE_OPENAI_SWEDENCENTRAL_ENDPOINT", ""),
),
"whisper-1": Deployment(
model="whisper-1",
provider="azure",
deployment_name="whisper-1",
api_key=os.getenv("LLMAX_AZURE_OPENAI_SWEDENCENTRAL_KEY", ""),
endpoint=os.getenv("LLMAX_AZURE_OPENAI_SWEDENCENTRAL_ENDPOINT", ""),
api_version="2024-02-01",
),
}
client = MultiAIClient(
deployments=deployments,
)
```
Then you should define your input (that can be a text, image or audio, following the openai documentation for instance).
```python
messages = [
{"role": "user", "content": "Raconte moi une blague."},
]
```
And finally get the response:
```python
response = client.invoke_to_str(messages, model)
print(response)
```
# Specificities
When creating the client, you can also specify two functions, *increment_usage* and *get_usage*.
The first one is **Callable[[float, Model], bool]** while the second is **Callable[[], float]**.
*increment_usage* is a function that is called after a call of the llm. The float is the price and Model, the model used. It can therefore be used to update your database. *get_usage* returns whether a condition is met. For instance, it can be a function that calls your database and returns whether the user is still active.
Raw data
{
"_id": null,
"home_page": null,
"name": "delos-llmax",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8.1",
"maintainer_email": null,
"keywords": "AI, LLM, generative",
"author": "Delos Intelligence",
"author_email": "maximiliendedinechin@delosintelligence.fr",
"download_url": "https://files.pythonhosted.org/packages/ff/69/1c8e79568270d65afe05b88e7b51e48e0213900329b3033dda5fbd7f8539/delos_llmax-0.11.27.tar.gz",
"platform": null,
"description": "# llmax\n\nPython package to manage most external and internal LLM APIs fluently.\n\n\n# Installation\n\nTo install, run the following command:\n\n```bash\npython3 -m pip install delos-llmax\n```\n\n# How to use\n\nYou first have to define a list of `Deployment` as such, where you need to specify the endpoints, key and deployment_name. Then create the client:\n\n```python\nfrom llmax.clients import MultiAIClient\nfrom llmax.models import Deployment, Model\n\ndeployments: dict[Model, Deployment] = {\n \"gpt-4o\": Deployment(\n model=\"gpt-4o\",\n provider=\"azure\",\n deployment_name=\"gpt-4o-2024-05-13\",\n api_key=os.getenv(\"LLMAX_AZURE_OPENAI_SWEDENCENTRAL_KEY\", \"\"),\n endpoint=os.getenv(\"LLMAX_AZURE_OPENAI_SWEDENCENTRAL_ENDPOINT\", \"\"),\n ),\n \"whisper-1\": Deployment(\n model=\"whisper-1\",\n provider=\"azure\",\n deployment_name=\"whisper-1\",\n api_key=os.getenv(\"LLMAX_AZURE_OPENAI_SWEDENCENTRAL_KEY\", \"\"),\n endpoint=os.getenv(\"LLMAX_AZURE_OPENAI_SWEDENCENTRAL_ENDPOINT\", \"\"),\n api_version=\"2024-02-01\",\n ),\n }\n\nclient = MultiAIClient(\n deployments=deployments,\n )\n```\n\nThen you should define your input (that can be a text, image or audio, following the openai documentation for instance).\n\n```python\nmessages = [\n {\"role\": \"user\", \"content\": \"Raconte moi une blague.\"},\n ]\n```\n\nAnd finally get the response:\n\n```python\nresponse = client.invoke_to_str(messages, model)\nprint(response)\n```\n\n# Specificities\n\nWhen creating the client, you can also specify two functions, *increment_usage* and *get_usage*.\nThe first one is **Callable[[float, Model], bool]** while the second is **Callable[[], float]**.\n*increment_usage* is a function that is called after a call of the llm. The float is the price and Model, the model used. It can therefore be used to update your database. *get_usage* returns whether a condition is met. For instance, it can be a function that calls your database and returns whether the user is still active.\n",
"bugtrack_url": null,
"license": null,
"summary": "Interface to handle multiple LLMs and AI tools.",
"version": "0.11.27",
"project_urls": null,
"split_keywords": [
"ai",
" llm",
" generative"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ebb35c0ef8fc9876b668267f9c647591fbfc18a23bb227532d1470787b0c2596",
"md5": "6e0a5197e38074fbcdd78c2a0d1f6dc5",
"sha256": "5441b7a0d4b18a87a7777846fc4cffefc0a6973ec6fb8d50071c535476ae2dad"
},
"downloads": -1,
"filename": "delos_llmax-0.11.27-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6e0a5197e38074fbcdd78c2a0d1f6dc5",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8.1",
"size": 26990,
"upload_time": "2025-02-11T16:53:50",
"upload_time_iso_8601": "2025-02-11T16:53:50.206238Z",
"url": "https://files.pythonhosted.org/packages/eb/b3/5c0ef8fc9876b668267f9c647591fbfc18a23bb227532d1470787b0c2596/delos_llmax-0.11.27-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ff691c8e79568270d65afe05b88e7b51e48e0213900329b3033dda5fbd7f8539",
"md5": "890fbdb28996ee874e5300fa5399c04f",
"sha256": "e4db0fd55ce34d748540cfd15b183ee01cc6452e44f386f261ca23ff7e7040fa"
},
"downloads": -1,
"filename": "delos_llmax-0.11.27.tar.gz",
"has_sig": false,
"md5_digest": "890fbdb28996ee874e5300fa5399c04f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8.1",
"size": 18025,
"upload_time": "2025-02-11T16:53:51",
"upload_time_iso_8601": "2025-02-11T16:53:51.852297Z",
"url": "https://files.pythonhosted.org/packages/ff/69/1c8e79568270d65afe05b88e7b51e48e0213900329b3033dda5fbd7f8539/delos_llmax-0.11.27.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-11 16:53:51",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "delos-llmax"
}
