# AI Handler
[](https://github.com/Capsize-Games/aihandler/actions/workflows/python-publish.yml)
[](https://discord.gg/PUVDDCJ7gz)
This is a simple framework for running AI models. It makes use of the huggingface API
which gives you a queue, threading, a simple API, and the ability to run Stable Diffusion and LLMs seamlessly
from your local hardware.
This is not intended to be used as a standalone application.
It can easily be extended and used to power interfaces or it can be run from the command line.
AI Handler is a work in progress. It powers two projects at the moment, but may not be ready for general use.
## Installation
This is a work in progress.
## Pre-requisites
System requirements
- Windows 10+
- Python 3.10.8
- pip 23.0.1
- CUDA toolkit 11.7
- CUDNN 8.6.0.163
- Cuda capable GPU
- 16gb+ ram
[For Windows, follow windows branch instructions](https://github.com/Capsize-Games/aihandler/tree/develop-windows)
Install
```
pip install https://github.com/w4ffl35/diffusers/archive/refs/tags/v0.15.0.ckpt_fix_0.0.1.tar.gz
pip install aihandler
```
#### Optional
These are optional instructions for installing TensorRT and Deepspeed for Windows
##### Install Tensor RT:
1. Download TensorRT-8.4.3.1.Windows10.x86_64.cuda-11.6.cudnn8.4
2. Git clone TensorRT 8.4.3.1
3. Follow their instructions to build TensorRT-8.4.3.1 python wheel
4. Install TensorRT `pip install tensorrt-*.whl`
##### Install Deepspeed:
1. Git clone Deepspeed 0.8.1
2. Follow their instructions to build Deepspeed python wheel
3. Install Deepspeed `pip install deepspeed-*.whl
---
## Environment variables
- `AIRUNNER_ENVIRONMENT` - `dev` or `prod`. Defaults to `dev`. This controls the LOG_LEVEL
- `LOG_LEVEL` - `FATAL` for production, `DEBUG` for development. Override this to force a log level
### Huggingface variables
#### Offline mode
These environment variables keep you offline until you need to download a model. This prevents unwanted online access and speeds up usage of huggingface libraries.
- `DISABLE_TELEMETRY` Keep this set to 1 at all times. Huggingface collects minimal telemetry when downloading a model from their repository but this will keep it disabled. [See more info in this github thread](https://github.com/huggingface/diffusers/pull/1833#issuecomment-1368484414)
- `HF_HUB_OFFLINE` When loading a diffusers model, huggingface libraries will attempt to download an updated cache before running the model. This prevents that check from happening (long with a boolean passed to `load_pretrained` see the runner.py file for examples)
- `TRANSFORMERS_OFFLINE` Similar to `HF_HUB_OFFLINE` but for transformers models
Raw data
{
"_id": null,
"home_page": "https://github.com/Capsize-Games/aihandler",
"name": "aihandler",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "ai,chatbot,chat,ai",
"author": "Capsize LLC",
"author_email": "contact@capsize.gg",
"download_url": "https://files.pythonhosted.org/packages/17/c9/b6a9cdfa9389b7a7989cd3bfa284c8ada70e972668f2d7910ef31addee95/aihandler-1.19.0.tar.gz",
"platform": null,
"description": "# AI Handler\n[](https://github.com/Capsize-Games/aihandler/actions/workflows/python-publish.yml)\n[](https://discord.gg/PUVDDCJ7gz)\n\n\n\n\n\n\n\nThis is a simple framework for running AI models. It makes use of the huggingface API\nwhich gives you a queue, threading, a simple API, and the ability to run Stable Diffusion and LLMs seamlessly\nfrom your local hardware.\n\nThis is not intended to be used as a standalone application.\n\nIt can easily be extended and used to power interfaces or it can be run from the command line.\n\nAI Handler is a work in progress. It powers two projects at the moment, but may not be ready for general use.\n\n## Installation\n\nThis is a work in progress.\n\n## Pre-requisites\n\nSystem requirements\n\n- Windows 10+\n- Python 3.10.8\n- pip 23.0.1\n- CUDA toolkit 11.7\n- CUDNN 8.6.0.163\n- Cuda capable GPU\n- 16gb+ ram\n\n[For Windows, follow windows branch instructions](https://github.com/Capsize-Games/aihandler/tree/develop-windows)\n\nInstall\n```\npip install https://github.com/w4ffl35/diffusers/archive/refs/tags/v0.15.0.ckpt_fix_0.0.1.tar.gz\npip install aihandler\n```\n\n#### Optional\n\nThese are optional instructions for installing TensorRT and Deepspeed for Windows\n\n##### Install Tensor RT:\n\n1. Download TensorRT-8.4.3.1.Windows10.x86_64.cuda-11.6.cudnn8.4\n2. Git clone TensorRT 8.4.3.1\n3. Follow their instructions to build TensorRT-8.4.3.1 python wheel\n4. Install TensorRT `pip install tensorrt-*.whl`\n \n##### Install Deepspeed:\n\n1. Git clone Deepspeed 0.8.1\n2. Follow their instructions to build Deepspeed python wheel\n3. Install Deepspeed `pip install deepspeed-*.whl\n\n---\n\n## Environment variables\n\n- `AIRUNNER_ENVIRONMENT` - `dev` or `prod`. Defaults to `dev`. This controls the LOG_LEVEL\n- `LOG_LEVEL` - `FATAL` for production, `DEBUG` for development. Override this to force a log level\n\n### Huggingface variables\n\n#### Offline mode\n\nThese environment variables keep you offline until you need to download a model. This prevents unwanted online access and speeds up usage of huggingface libraries.\n\n- `DISABLE_TELEMETRY` Keep this set to 1 at all times. Huggingface collects minimal telemetry when downloading a model from their repository but this will keep it disabled. [See more info in this github thread](https://github.com/huggingface/diffusers/pull/1833#issuecomment-1368484414)\n- `HF_HUB_OFFLINE` When loading a diffusers model, huggingface libraries will attempt to download an updated cache before running the model. This prevents that check from happening (long with a boolean passed to `load_pretrained` see the runner.py file for examples)\n- `TRANSFORMERS_OFFLINE` Similar to `HF_HUB_OFFLINE` but for transformers models\n",
"bugtrack_url": null,
"license": "AGPL-3.0",
"summary": "AI Handler: An engine which wraps certain huggingface models",
"version": "1.19.0",
"project_urls": {
"Homepage": "https://github.com/Capsize-Games/aihandler"
},
"split_keywords": [
"ai",
"chatbot",
"chat",
"ai"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7e2c3f79edb53ddd7906ef53a9ab26267fb3aa5bbb8993c162ad4536d6ecaf06",
"md5": "ac0e49c824d53709d1a0f5b907d24260",
"sha256": "5f2c434a9f13e51fedd4639df4bb5717180eb15a8dfb2bf2d3f59b728ff5af19"
},
"downloads": -1,
"filename": "aihandler-1.19.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ac0e49c824d53709d1a0f5b907d24260",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 61274,
"upload_time": "2023-07-14T01:36:28",
"upload_time_iso_8601": "2023-07-14T01:36:28.312575Z",
"url": "https://files.pythonhosted.org/packages/7e/2c/3f79edb53ddd7906ef53a9ab26267fb3aa5bbb8993c162ad4536d6ecaf06/aihandler-1.19.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "17c9b6a9cdfa9389b7a7989cd3bfa284c8ada70e972668f2d7910ef31addee95",
"md5": "975a991022d7be575f522c913d092683",
"sha256": "09cab787b5b54bd8d2a6f7c475c5693ba3e0f21d94a2261734ef66450ca4f829"
},
"downloads": -1,
"filename": "aihandler-1.19.0.tar.gz",
"has_sig": false,
"md5_digest": "975a991022d7be575f522c913d092683",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 54528,
"upload_time": "2023-07-14T01:36:30",
"upload_time_iso_8601": "2023-07-14T01:36:30.198309Z",
"url": "https://files.pythonhosted.org/packages/17/c9/b6a9cdfa9389b7a7989cd3bfa284c8ada70e972668f2d7910ef31addee95/aihandler-1.19.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-14 01:36:30",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Capsize-Games",
"github_project": "aihandler",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "aihandler"
}
JSON
