<div align="center">
<p>
<a align="center" href="" target="_blank">
<img
width="850"
src="https://media.roboflow.com/open-source/autodistill/autodistill-banner.png"
>
</a>
</p>
</div>
# Autodistill PaLiGemma Module
This repository contains the code supporting the PaLiGemma base model for use with [Autodistill](https://github.com/autodistill/autodistill).
[PaLiGemma](https://blog.roboflow.com/paligemma-multimodal-vision/), developed by Google, is a computer vision model trained using pairs of images and text. You can label data with PaliGemma models for use in training smaller, fine-tuned models with Autodisitll.
Read the full [Autodistill documentation](https://autodistill.github.io/autodistill/).
## Installation
To use PaLiGemma with autodistill, you need to install the following dependency:
```bash
pip3 install autodistill-paligemma
```
## Quickstart
### Auto-label with an existing model
```python
from autodistill_paligemma import PaliGemma
# define an ontology to map class names to our PaliGemma prompt
# the ontology dictionary has the format {caption: class}
# where caption is the prompt sent to the base model, and class is the label that will
# be saved for that caption in the generated annotations
# then, load the model
base_model = PaliGemma(
ontology=CaptionOntology(
{
"person": "person",
"a forklift": "forklift"
}
)
)
# label a single image
result = PaliGemma.predict("test.jpeg")
print(result)
# label a folder of images
base_model.label("./context_images", extension=".jpeg")
```
### Model fine-tuning
You can fine-tune PaliGemma models with LoRA for deployment with [Roboflow Inference](https://inference.roboflow.com).
To train a model, use this code:
```python
from autodistill_paligemma import PaLiGemmaTrainer
target_model = PaLiGemmaTrainer()
# train a model
target_model.train("./data/")
```
## License
The model weights for PaLiGemma are licensed under a custom Google license. To learn more, refer to the [Google Gemma Terms of Use](https://ai.google.dev/gemma/terms).
## 🏆 Contributing
We love your input! Please see the core Autodistill [contributing guide](https://github.com/autodistill/autodistill/blob/main/CONTRIBUTING.md) to get started. Thank you 🙏 to all our contributors!
Raw data
{
"_id": null,
"home_page": "https://github.com/autodistill/autodistill-paligemma",
"name": "autodistill-paligemma",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": null,
"author": "Roboflow",
"author_email": "support@roboflow.com",
"download_url": "https://files.pythonhosted.org/packages/12/fe/f5a7b0c40c96089716566c7d2f73075202ba67bf43458ba62576028de0bb/autodistill_paligemma-0.1.1.tar.gz",
"platform": null,
"description": "<div align=\"center\">\n <p>\n <a align=\"center\" href=\"\" target=\"_blank\">\n <img\n width=\"850\"\n src=\"https://media.roboflow.com/open-source/autodistill/autodistill-banner.png\"\n >\n </a>\n </p>\n</div>\n\n# Autodistill PaLiGemma Module\n\nThis repository contains the code supporting the PaLiGemma base model for use with [Autodistill](https://github.com/autodistill/autodistill).\n\n[PaLiGemma](https://blog.roboflow.com/paligemma-multimodal-vision/), developed by Google, is a computer vision model trained using pairs of images and text. You can label data with PaliGemma models for use in training smaller, fine-tuned models with Autodisitll.\n\nRead the full [Autodistill documentation](https://autodistill.github.io/autodistill/).\n\n## Installation\n\nTo use PaLiGemma with autodistill, you need to install the following dependency:\n\n```bash\npip3 install autodistill-paligemma\n```\n\n## Quickstart\n\n### Auto-label with an existing model\n\n```python\nfrom autodistill_paligemma import PaliGemma\n\n# define an ontology to map class names to our PaliGemma prompt\n# the ontology dictionary has the format {caption: class}\n# where caption is the prompt sent to the base model, and class is the label that will\n# be saved for that caption in the generated annotations\n# then, load the model\nbase_model = PaliGemma(\n ontology=CaptionOntology(\n {\n \"person\": \"person\",\n \"a forklift\": \"forklift\"\n }\n )\n)\n\n# label a single image\nresult = PaliGemma.predict(\"test.jpeg\")\nprint(result)\n\n# label a folder of images\nbase_model.label(\"./context_images\", extension=\".jpeg\")\n```\n\n### Model fine-tuning\n\nYou can fine-tune PaliGemma models with LoRA for deployment with [Roboflow Inference](https://inference.roboflow.com).\n\nTo train a model, use this code:\n\n```python\nfrom autodistill_paligemma import PaLiGemmaTrainer\n\ntarget_model = PaLiGemmaTrainer()\n\n# train a model\ntarget_model.train(\"./data/\")\n```\n\n## License\n\nThe model weights for PaLiGemma are licensed under a custom Google license. To learn more, refer to the [Google Gemma Terms of Use](https://ai.google.dev/gemma/terms).\n\n## \ud83c\udfc6 Contributing\n\nWe love your input! Please see the core Autodistill [contributing guide](https://github.com/autodistill/autodistill/blob/main/CONTRIBUTING.md) to get started. Thank you \ud83d\ude4f to all our contributors!\n",
"bugtrack_url": null,
"license": null,
"summary": "Auto-label data with a PaliGemma model, or ine-tune a PaLiGemma model using custom data with Autodistill.",
"version": "0.1.1",
"project_urls": {
"Homepage": "https://github.com/autodistill/autodistill-paligemma"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b8b437333cab15564caf460a34faff49cde4db5875a5306765de084c1befb56a",
"md5": "6e1598364ff9346a334ae187dbaf6ba2",
"sha256": "c3695250687bfae317789a9c572b56d0742556869ff643b019ef9f01eb77bcdc"
},
"downloads": -1,
"filename": "autodistill_paligemma-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6e1598364ff9346a334ae187dbaf6ba2",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 5019,
"upload_time": "2024-06-13T10:48:44",
"upload_time_iso_8601": "2024-06-13T10:48:44.425064Z",
"url": "https://files.pythonhosted.org/packages/b8/b4/37333cab15564caf460a34faff49cde4db5875a5306765de084c1befb56a/autodistill_paligemma-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "12fef5a7b0c40c96089716566c7d2f73075202ba67bf43458ba62576028de0bb",
"md5": "dc85b885344718a052c1e200ee54de5f",
"sha256": "b0d7edace5ebc30587c57c24658e42815dcbac3cd84fd98338315476a35b93d0"
},
"downloads": -1,
"filename": "autodistill_paligemma-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "dc85b885344718a052c1e200ee54de5f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 4896,
"upload_time": "2024-06-13T10:48:45",
"upload_time_iso_8601": "2024-06-13T10:48:45.815479Z",
"url": "https://files.pythonhosted.org/packages/12/fe/f5a7b0c40c96089716566c7d2f73075202ba67bf43458ba62576028de0bb/autodistill_paligemma-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-06-13 10:48:45",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "autodistill",
"github_project": "autodistill-paligemma",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "autodistill-paligemma"
}
JSON
