| Name | frequency-ai JSON |
| Version |
0.1.7
JSON |
| download |
| home_page | |
| Summary | Effiently serve LoRA tuned models |
| upload_time | 2024-01-13 22:41:13 |
| maintainer | |
| docs_url | None |
| author | Patrick Barker |
| requires_python | >=3.10,<4.0 |
| license | Apache 2.0 |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Frequency
Efficiently serve LoRA tuned models.
Frequency provides a means to hot-swap LoRA layers in ML models at the time of inference allowing for the efficient usage of large base models.
## Install
```
pip install frequency-ai
```
Install server component on Kubernetes
```
helm install frequency oci://artifact.frequency.ai/frequency-server:0.0.1
```
## Usage
Load a HuggingFace model and use adapters
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from frequency import Client
# Connect to the frequency server
client = Client("localhost:9000")
# Load an hf model onto the server
model = client.load_model(name="qwen-vl-chat", hf_repo="Qwen/Qwen-VL-Chat", type=AutoModelForCausalLM)
# Cache an adapter on the server that was trained on dog images
resp = model.cache_adapter(name="dog", hf_repo="Anima-ai/dog_lora")
# Qwen expects a specific format for describing images
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen-VL-Chat", trust_remote_code=True)
query = tokenizer.from_list_format([
{'image': 'https://hips.hearstapps.com/ghk.h-cdn.co/assets/17/30/pembroke-welsh-corgi.jpg'},
{'text': 'What is this?'},
])
# Chat with the model using the dog adapter
response, history = model.chat(query=query, adapters=["dog"])
#> Here is a picture of a Corgi
# Cache an adapter on the server that was trained on cat images
resp = model.cache_adapter(name="cat", hf_repo="Anima-ai/cat_lora")
print(resp)
query = tokenizer.from_list_format([
{'image': 'https://www.catster.com/wp-content/uploads/2023/11/Brown-tabby-cat-that-curls-up-outdoors_viper-zero_Shutterstock-800x533.jpg'},
{'text': 'What is this?'},
])
# Chat with the same model using the new cat adapter
response, history = model.chat(query=query, adapters=["cat"])
#> Here is a picture of a tabby cat
```
## Roadmap
- Tenancy
Raw data
{
"_id": null,
"home_page": "",
"name": "frequency-ai",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10,<4.0",
"maintainer_email": "",
"keywords": "",
"author": "Patrick Barker",
"author_email": "patrickbarkerco@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/27/47/d22a03f5a0e3b1db040752f1eaf7b7ad3b12457bb08d7402e3995c56fc7f/frequency_ai-0.1.7.tar.gz",
"platform": null,
"description": "# Frequency\n\nEfficiently serve LoRA tuned models.\n\nFrequency provides a means to hot-swap LoRA layers in ML models at the time of inference allowing for the efficient usage of large base models.\n\n## Install\n\n```\npip install frequency-ai\n```\n\nInstall server component on Kubernetes\n\n```\nhelm install frequency oci://artifact.frequency.ai/frequency-server:0.0.1\n```\n\n## Usage\n\nLoad a HuggingFace model and use adapters\n\n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\nfrom frequency import Client\n\n# Connect to the frequency server\nclient = Client(\"localhost:9000\")\n\n# Load an hf model onto the server\nmodel = client.load_model(name=\"qwen-vl-chat\", hf_repo=\"Qwen/Qwen-VL-Chat\", type=AutoModelForCausalLM)\n\n# Cache an adapter on the server that was trained on dog images\nresp = model.cache_adapter(name=\"dog\", hf_repo=\"Anima-ai/dog_lora\")\n\n# Qwen expects a specific format for describing images\ntokenizer = AutoTokenizer.from_pretrained(\"Qwen/Qwen-VL-Chat\", trust_remote_code=True)\nquery = tokenizer.from_list_format([\n {'image': 'https://hips.hearstapps.com/ghk.h-cdn.co/assets/17/30/pembroke-welsh-corgi.jpg'},\n {'text': 'What is this?'},\n])\n\n# Chat with the model using the dog adapter\nresponse, history = model.chat(query=query, adapters=[\"dog\"])\n#> Here is a picture of a Corgi\n\n# Cache an adapter on the server that was trained on cat images\nresp = model.cache_adapter(name=\"cat\", hf_repo=\"Anima-ai/cat_lora\")\nprint(resp)\n\nquery = tokenizer.from_list_format([\n {'image': 'https://www.catster.com/wp-content/uploads/2023/11/Brown-tabby-cat-that-curls-up-outdoors_viper-zero_Shutterstock-800x533.jpg'},\n {'text': 'What is this?'},\n])\n\n# Chat with the same model using the new cat adapter\nresponse, history = model.chat(query=query, adapters=[\"cat\"])\n#> Here is a picture of a tabby cat\n```\n\n## Roadmap\n\n- Tenancy\n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "Effiently serve LoRA tuned models",
"version": "0.1.7",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "cb536e8188d25aa039d1410dc522c00defe0208b4313f2de664d2a1843ff0c81",
"md5": "3fe7a1645fe28822ee2605f7eee7340a",
"sha256": "c5fe207f3bbcb4155c41bfca27b30b4985f533487e5924931799b46d766c46a0"
},
"downloads": -1,
"filename": "frequency_ai-0.1.7-py3-none-any.whl",
"has_sig": false,
"md5_digest": "3fe7a1645fe28822ee2605f7eee7340a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10,<4.0",
"size": 55693,
"upload_time": "2024-01-13T22:41:12",
"upload_time_iso_8601": "2024-01-13T22:41:12.075252Z",
"url": "https://files.pythonhosted.org/packages/cb/53/6e8188d25aa039d1410dc522c00defe0208b4313f2de664d2a1843ff0c81/frequency_ai-0.1.7-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "2747d22a03f5a0e3b1db040752f1eaf7b7ad3b12457bb08d7402e3995c56fc7f",
"md5": "40a1c507d1ad478bd6a5f488f92ccdd6",
"sha256": "97d33aa0f4b2fa99582368769a3838b9fcf409abc6a1385062b4942f26b3f4cb"
},
"downloads": -1,
"filename": "frequency_ai-0.1.7.tar.gz",
"has_sig": false,
"md5_digest": "40a1c507d1ad478bd6a5f488f92ccdd6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10,<4.0",
"size": 42870,
"upload_time": "2024-01-13T22:41:13",
"upload_time_iso_8601": "2024-01-13T22:41:13.652036Z",
"url": "https://files.pythonhosted.org/packages/27/47/d22a03f5a0e3b1db040752f1eaf7b7ad3b12457bb08d7402e3995c56fc7f/frequency_ai-0.1.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-13 22:41:13",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "frequency-ai"
}
