# ailia LLM Python API
!! CAUTION !!
“ailia” IS NOT OPEN SOURCE SOFTWARE (OSS).
As long as user complies with the conditions stated in [License Document](https://ailia.ai/license/), user may use the Software for free of charge, but the Software is basically paid software.
## About ailia LLM
ailia LLM is a library for running LLMs on edge devices. It provides bindings for C++ and Unity.
## Install from pip
You can install the ailia LLM free evaluation package with the following command.
```
pip3 install ailia_llm
```
## Install from package
You can install the ailia LLM from Package with the following command.
```
python3 bootstrap.py
pip3 install ./
```
## Usage
```python
import ailia_llm
import os
import urllib.request
model_file_path = "gemma-2-2b-it-Q4_K_M.gguf"
if not os.path.exists(model_file_path):
urllib.request.urlretrieve(
"https://storage.googleapis.com/ailia-models/gemma/gemma-2-2b-it-Q4_K_M.gguf",
model_file_path
)
model = ailia_llm.AiliaLLM()
model.open(model_file_path)
messages = []
messages.append({"role": "system", "content": "語尾に「わん」をつけてください。"})
messages.append({"role": "user", "content": "あなたの名前は何ですか？"})
stream = model.generate(messages)
text = ""
for delta_text in stream:
text = text + delta_text
print(text)
if model.context_full():
raise Exception("Context full")
messages.append({"role": "assistant", "content": text})
```
## API specification
https://github.com/axinc-ai/ailia-sdk
Raw data
{
"_id": null,
"home_page": "https://ailia.jp/",
"name": "ailia-llm",
"maintainer": null,
"docs_url": null,
"requires_python": ">3.6",
"maintainer_email": null,
"keywords": null,
"author": "ax Inc.",
"author_email": "contact@axinc.jp",
"download_url": "https://files.pythonhosted.org/packages/9a/68/ff84830188174ceeabc76abc8f436b598fa54376db536b14979fd498a479/ailia_llm-1.2.0.0.tar.gz",
"platform": null,
"description": "# ailia LLM Python API\n\n!! CAUTION !!\n\u201cailia\u201d IS NOT OPEN SOURCE SOFTWARE (OSS).\nAs long as user complies with the conditions stated in [License Document](https://ailia.ai/license/), user may use the Software for free of charge, but the Software is basically paid software.\n\n## About ailia LLM\n\nailia LLM is a library for running LLMs on edge devices. It provides bindings for C++ and Unity.\n\n## Install from pip\n\nYou can install the ailia LLM free evaluation package with the following command.\n\n```\npip3 install ailia_llm\n```\n\n## Install from package\n\nYou can install the ailia LLM from Package with the following command.\n\n```\npython3 bootstrap.py\npip3 install ./\n```\n\n## Usage\n\n```python\nimport ailia_llm\n\nimport os\nimport urllib.request\n\nmodel_file_path = \"gemma-2-2b-it-Q4_K_M.gguf\"\nif not os.path.exists(model_file_path):\n\turllib.request.urlretrieve(\n\t\t\"https://storage.googleapis.com/ailia-models/gemma/gemma-2-2b-it-Q4_K_M.gguf\",\n\t\tmodel_file_path\n\t)\n\nmodel = ailia_llm.AiliaLLM()\nmodel.open(model_file_path)\n\nmessages = []\nmessages.append({\"role\": \"system\", \"content\": \"\u8a9e\u5c3e\u306b\u300c\u308f\u3093\u300d\u3092\u3064\u3051\u3066\u304f\u3060\u3055\u3044\u3002\"})\nmessages.append({\"role\": \"user\", \"content\": \"\u3042\u306a\u305f\u306e\u540d\u524d\u306f\u4f55\u3067\u3059\u304b\uff1f\"})\n\nstream = model.generate(messages)\n\ntext = \"\"\nfor delta_text in stream:\n\ttext = text + delta_text\nprint(text)\n\nif model.context_full():\n\traise Exception(\"Context full\")\n\nmessages.append({\"role\": \"assistant\", \"content\": text})\n```\n\n## API specification\n\nhttps://github.com/axinc-ai/ailia-sdk\n\n",
"bugtrack_url": null,
"license": "https://ailia.ai/en/license/",
"summary": "ailia LLM",
"version": "1.2.0.0",
"project_urls": {
"Homepage": "https://ailia.jp/"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "71bdd2abc5f9873f0edf48994da11c15103b736bada1497566875df8536c1e35",
"md5": "c7e14cbcc1fe99f4edb2dad9759615f9",
"sha256": "6da8d515211469356d0b293299b5eb395f25e3dd3ce72c29b895eabb7d5bbdbe"
},
"downloads": -1,
"filename": "ailia_llm-1.2.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c7e14cbcc1fe99f4edb2dad9759615f9",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">3.6",
"size": 5196497,
"upload_time": "2024-11-15T04:30:41",
"upload_time_iso_8601": "2024-11-15T04:30:41.242193Z",
"url": "https://files.pythonhosted.org/packages/71/bd/d2abc5f9873f0edf48994da11c15103b736bada1497566875df8536c1e35/ailia_llm-1.2.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9a68ff84830188174ceeabc76abc8f436b598fa54376db536b14979fd498a479",
"md5": "13849e217352f84e185d4504b20faa23",
"sha256": "f50e04454938a4dbb55e0d1f169793b1181af528938d7178f3f61ac859842c99"
},
"downloads": -1,
"filename": "ailia_llm-1.2.0.0.tar.gz",
"has_sig": false,
"md5_digest": "13849e217352f84e185d4504b20faa23",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">3.6",
"size": 5152346,
"upload_time": "2024-11-15T04:30:43",
"upload_time_iso_8601": "2024-11-15T04:30:43.646661Z",
"url": "https://files.pythonhosted.org/packages/9a/68/ff84830188174ceeabc76abc8f436b598fa54376db536b14979fd498a479/ailia_llm-1.2.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-15 04:30:43",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "ailia-llm"
}
JSON