## BentoML Unsloth integrations
## Installation
```bash
pip install "bentoml[unsloth]"
```
## Examples.
See [train.py](https://github.com/bentoml/bentoml-unsloth/blob/main/train.py)
## API
To use this integration, one can use `bentoml.unsloth.build_bento`:
```python
bentoml.unsloth.build_bento(model, tokenizer)
```
If you model is continued froma fine-tuned checkpoint, then `model_name` must be passed as well:
```python
bentoml.unsloth.build_bento(model, tokenizer, model_name="llama-3-continued-from-checkpoint")
```
> [!important]
>
> Make sure to save the chat templates to tokenizer instance to make sure generations are correct based on how you setup your data pipeline.
> See [example](https://github.com/bentoml/bentoml-unsloth/blob/da52d51366ea3217a3ee644f80042b1f425e00c6/train.py#L42) and [documentation](https://huggingface.co/docs/transformers/main/en/chat_templating#advanced-adding-and-editing-chat-templates) for more information.
Raw data
{
"_id": null,
"home_page": null,
"name": "bentoml-unsloth",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "BentoML, Compound AI Systems, LLMOps, MLOps, Model Deployment, Model Inference, Model Serving",
"author": null,
"author_email": "BentoML Team <contact@bentoml.com>",
"download_url": "https://files.pythonhosted.org/packages/5c/ac/88c9e2d8ff2b7d1fe00aaaa1562f298a1680934bfa2e8a24aa9016b6a43f/bentoml_unsloth-0.1.2.tar.gz",
"platform": null,
"description": "## BentoML Unsloth integrations\n\n## Installation\n\n```bash\npip install \"bentoml[unsloth]\"\n```\n\n## Examples.\n\nSee [train.py](https://github.com/bentoml/bentoml-unsloth/blob/main/train.py)\n\n## API\n\nTo use this integration, one can use `bentoml.unsloth.build_bento`:\n\n```python\nbentoml.unsloth.build_bento(model, tokenizer)\n```\n\nIf you model is continued froma fine-tuned checkpoint, then `model_name` must be passed as well:\n\n```python\nbentoml.unsloth.build_bento(model, tokenizer, model_name=\"llama-3-continued-from-checkpoint\")\n```\n\n> [!important]\n>\n> Make sure to save the chat templates to tokenizer instance to make sure generations are correct based on how you setup your data pipeline.\n> See [example](https://github.com/bentoml/bentoml-unsloth/blob/da52d51366ea3217a3ee644f80042b1f425e00c6/train.py#L42) and [documentation](https://huggingface.co/docs/transformers/main/en/chat_templating#advanced-adding-and-editing-chat-templates) for more information.\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "BentoML: The easiest way to serve AI apps and models",
"version": "0.1.2",
"project_urls": {
"Blog": "https://bentoml.com/blog",
"Documentation": "https://docs.bentoml.com",
"GitHub": "https://github.com/bentoml/bentoml",
"Homepage": "https://bentoml.com",
"Slack": "https://l.bentoml.com/join-slack",
"Tracker": "https://github.com/bentoml/BentoML/issues",
"Twitter": "https://twitter.com/bentomlai"
},
"split_keywords": [
"bentoml",
" compound ai systems",
" llmops",
" mlops",
" model deployment",
" model inference",
" model serving"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7a639b107b9e8737a3ab698911814a9f878fe61b7dab40dd8b9622c8a9a1793a",
"md5": "e16233adb7528829b3b7e1b0598d324a",
"sha256": "2c3404cf07863fbaff579f40f9d594f0a6e73307dd76c824f34b877ce642abb7"
},
"downloads": -1,
"filename": "bentoml_unsloth-0.1.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e16233adb7528829b3b7e1b0598d324a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 6727,
"upload_time": "2024-09-24T00:35:27",
"upload_time_iso_8601": "2024-09-24T00:35:27.709204Z",
"url": "https://files.pythonhosted.org/packages/7a/63/9b107b9e8737a3ab698911814a9f878fe61b7dab40dd8b9622c8a9a1793a/bentoml_unsloth-0.1.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5cac88c9e2d8ff2b7d1fe00aaaa1562f298a1680934bfa2e8a24aa9016b6a43f",
"md5": "33678f443738719e9f0d667abc7b167c",
"sha256": "9d240000990ae6871b0366c26c46a96e017d252a716076ba004045228c556c1e"
},
"downloads": -1,
"filename": "bentoml_unsloth-0.1.2.tar.gz",
"has_sig": false,
"md5_digest": "33678f443738719e9f0d667abc7b167c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 6035,
"upload_time": "2024-09-24T00:35:28",
"upload_time_iso_8601": "2024-09-24T00:35:28.587110Z",
"url": "https://files.pythonhosted.org/packages/5c/ac/88c9e2d8ff2b7d1fe00aaaa1562f298a1680934bfa2e8a24aa9016b6a43f/bentoml_unsloth-0.1.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-24 00:35:28",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "bentoml",
"github_project": "bentoml",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "bentoml-unsloth"
}
JSON
