| Name | delos-llmax JSON |
| Version |
0.11.3
JSON |
| download |
| home_page | None |
| Summary | Interface to handle multiple LLMs and AI tools. |
| upload_time | 2024-10-11 15:40:54 |
| maintainer | None |
| docs_url | None |
| author | Delos Intelligence |
| requires_python | <4.0,>=3.8.1 |
| license | None |
| keywords |
ai
llm
generative
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# llmax
Python package to manage most external and internal LLM APIs fluently.
# Installation
To install, run the following command:
```bash
python3 -m pip install delos-llmax
```
# How to use
You first have to define a list of `Deployment` as such, where you need to specify the endpoints, key and deployment_name. Then create the client:
```python
from llmax.clients import MultiAIClient
from llmax.models import Deployment, Model
deployments: dict[Model, Deployment] = {
"gpt-4o": Deployment(
model="gpt-4o",
provider="azure",
deployment_name="gpt-4o-2024-05-13",
api_key=os.getenv("LLMAX_AZURE_OPENAI_SWEDENCENTRAL_KEY", ""),
endpoint=os.getenv("LLMAX_AZURE_OPENAI_SWEDENCENTRAL_ENDPOINT", ""),
),
"whisper-1": Deployment(
model="whisper-1",
provider="azure",
deployment_name="whisper-1",
api_key=os.getenv("LLMAX_AZURE_OPENAI_SWEDENCENTRAL_KEY", ""),
endpoint=os.getenv("LLMAX_AZURE_OPENAI_SWEDENCENTRAL_ENDPOINT", ""),
api_version="2024-02-01",
),
}
client = MultiAIClient(
deployments=deployments,
)
```
Then you should define your input (that can be a text, image or audio, following the openai documentation for instance).
```python
messages = [
{"role": "user", "content": "Raconte moi une blague."},
]
```
And finally get the response:
```python
response = client.invoke_to_str(messages, model)
print(response)
```
# Specificities
When creating the client, you can also specify two functions, *increment_usage* and *get_usage*.
The first one is **Callable[[float, Model], bool]** while the second is **Callable[[], float]**.
*increment_usage* is a function that is called after a call of the llm. The float is the price and Model, the model used. It can therefore be used to update your database. *get_usage* returns whether a condition is met. For instance, it can be a function that calls your database and returns whether the user is still active.
Raw data
{
"_id": null,
"home_page": null,
"name": "delos-llmax",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8.1",
"maintainer_email": null,
"keywords": "AI, LLM, generative",
"author": "Delos Intelligence",
"author_email": "maximiliendedinechin@delosintelligence.fr",
"download_url": "https://files.pythonhosted.org/packages/98/c8/73429b4aeeee24c6c2f57b0ce122c0e50f3de3d9bfe91854758aae7748f8/delos_llmax-0.11.3.tar.gz",
"platform": null,
"description": "# llmax\n\nPython package to manage most external and internal LLM APIs fluently.\n\n\n# Installation\n\nTo install, run the following command:\n\n```bash\npython3 -m pip install delos-llmax\n```\n\n# How to use\n\nYou first have to define a list of `Deployment` as such, where you need to specify the endpoints, key and deployment_name. Then create the client:\n\n```python\nfrom llmax.clients import MultiAIClient\nfrom llmax.models import Deployment, Model\n\ndeployments: dict[Model, Deployment] = {\n \"gpt-4o\": Deployment(\n model=\"gpt-4o\",\n provider=\"azure\",\n deployment_name=\"gpt-4o-2024-05-13\",\n api_key=os.getenv(\"LLMAX_AZURE_OPENAI_SWEDENCENTRAL_KEY\", \"\"),\n endpoint=os.getenv(\"LLMAX_AZURE_OPENAI_SWEDENCENTRAL_ENDPOINT\", \"\"),\n ),\n \"whisper-1\": Deployment(\n model=\"whisper-1\",\n provider=\"azure\",\n deployment_name=\"whisper-1\",\n api_key=os.getenv(\"LLMAX_AZURE_OPENAI_SWEDENCENTRAL_KEY\", \"\"),\n endpoint=os.getenv(\"LLMAX_AZURE_OPENAI_SWEDENCENTRAL_ENDPOINT\", \"\"),\n api_version=\"2024-02-01\",\n ),\n }\n\nclient = MultiAIClient(\n deployments=deployments,\n )\n```\n\nThen you should define your input (that can be a text, image or audio, following the openai documentation for instance).\n\n```python\nmessages = [\n {\"role\": \"user\", \"content\": \"Raconte moi une blague.\"},\n ]\n```\n\nAnd finally get the response:\n\n```python\nresponse = client.invoke_to_str(messages, model)\nprint(response)\n```\n\n# Specificities\n\nWhen creating the client, you can also specify two functions, *increment_usage* and *get_usage*.\nThe first one is **Callable[[float, Model], bool]** while the second is **Callable[[], float]**.\n*increment_usage* is a function that is called after a call of the llm. The float is the price and Model, the model used. It can therefore be used to update your database. *get_usage* returns whether a condition is met. For instance, it can be a function that calls your database and returns whether the user is still active.\n",
"bugtrack_url": null,
"license": null,
"summary": "Interface to handle multiple LLMs and AI tools.",
"version": "0.11.3",
"project_urls": null,
"split_keywords": [
"ai",
" llm",
" generative"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ac5147c4375cdfcae0fe7eaef8a0a3e71e556b68185caebd229953d3b1750824",
"md5": "86c5523b2ae5ad6de174adb299e0fccc",
"sha256": "975502a5a5a1585296e6fd0076c8040fcd9f0b9c917ca3ed90c356e5252b60b2"
},
"downloads": -1,
"filename": "delos_llmax-0.11.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "86c5523b2ae5ad6de174adb299e0fccc",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8.1",
"size": 18091,
"upload_time": "2024-10-11T15:40:52",
"upload_time_iso_8601": "2024-10-11T15:40:52.915280Z",
"url": "https://files.pythonhosted.org/packages/ac/51/47c4375cdfcae0fe7eaef8a0a3e71e556b68185caebd229953d3b1750824/delos_llmax-0.11.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "98c873429b4aeeee24c6c2f57b0ce122c0e50f3de3d9bfe91854758aae7748f8",
"md5": "efbbd913165e7211b6cfab1812c6e481",
"sha256": "6a28ceb479d003bf532240ad47542f70885283c6d00b219f16de6e35af149c7e"
},
"downloads": -1,
"filename": "delos_llmax-0.11.3.tar.gz",
"has_sig": false,
"md5_digest": "efbbd913165e7211b6cfab1812c6e481",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8.1",
"size": 12894,
"upload_time": "2024-10-11T15:40:54",
"upload_time_iso_8601": "2024-10-11T15:40:54.540931Z",
"url": "https://files.pythonhosted.org/packages/98/c8/73429b4aeeee24c6c2f57b0ce122c0e50f3de3d9bfe91854758aae7748f8/delos_llmax-0.11.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-11 15:40:54",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "delos-llmax"
}
