[](https://discord.gg/qUtxnK2NMf)
# Palm2 Adapter
Implementation of "PaLM2-VAdapter:" from the multi-modal model paper: "PaLM2-VAdapter: Progressively Aligned Language Model Makes a Strong Vision-language Adapter".
This model uses a perceiver resampler with a depth of 1 + a tiny palm to efficiently learn the features behind the images and then map them to the same space as the big model.
## install
`$ pip install palm2-vadapter`
## usage
```python
import torch
from palm_vadapter.main import PaLM2VAdapter
# Random text and image tensors
text = torch.randint(0, 1000, (1, 32), dtype=torch.long)
# Image tensor
img = torch.randn(1, 3, 224, 224)
# Initialize PaLM2VAdapter model
model = PaLM2VAdapter(
tiny_dim=512,
dim=512,
num_tokens=10000,
seq_length=32,
depth=6,
heads=8,
image_size=224,
patch_size=16,
)
# Forward pass through the model
out = model(text, img)
# Print the shape of the output
print(out.shape)
```
# License
MIT
## Citation
```bibtex
@misc{xiao2024palm2vadapter,
title={PaLM2-VAdapter: Progressively Aligned Language Model Makes a Strong Vision-language Adapter},
author={Junfei Xiao and Zheng Xu and Alan Yuille and Shen Yan and Boyu Wang},
year={2024},
eprint={2402.10896},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/kyegomez/PaLM2-VAdapter",
"name": "palm-vadapter",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6,<4.0",
"maintainer_email": "",
"keywords": "artificial intelligence,deep learning,optimizers,Prompt Engineering",
"author": "Kye Gomez",
"author_email": "kye@apac.ai",
"download_url": "https://files.pythonhosted.org/packages/ac/74/78b791690927a841d420b042f2f25d0eb76d48781155e5405e70e6225ea1/palm_vadapter-0.0.1.tar.gz",
"platform": null,
"description": "[](https://discord.gg/qUtxnK2NMf)\n\n# Palm2 Adapter\nImplementation of \"PaLM2-VAdapter:\" from the multi-modal model paper: \"PaLM2-VAdapter: Progressively Aligned Language Model Makes a Strong Vision-language Adapter\".\n\nThis model uses a perceiver resampler with a depth of 1 + a tiny palm to efficiently learn the features behind the images and then map them to the same space as the big model.\n\n## install\n`$ pip install palm2-vadapter`\n\n\n## usage\n```python\nimport torch\nfrom palm_vadapter.main import PaLM2VAdapter\n\n# Random text and image tensors\ntext = torch.randint(0, 1000, (1, 32), dtype=torch.long)\n\n\n# Image tensor\nimg = torch.randn(1, 3, 224, 224)\n\n# Initialize PaLM2VAdapter model\nmodel = PaLM2VAdapter(\n tiny_dim=512,\n dim=512,\n num_tokens=10000,\n seq_length=32,\n depth=6,\n heads=8,\n image_size=224,\n patch_size=16,\n)\n\n# Forward pass through the model\nout = model(text, img)\n\n# Print the shape of the output\nprint(out.shape)\n```\n\n\n# License\nMIT\n\n## Citation\n```bibtex\n@misc{xiao2024palm2vadapter,\n title={PaLM2-VAdapter: Progressively Aligned Language Model Makes a Strong Vision-language Adapter}, \n author={Junfei Xiao and Zheng Xu and Alan Yuille and Shen Yan and Boyu Wang},\n year={2024},\n eprint={2402.10896},\n archivePrefix={arXiv},\n primaryClass={cs.CV}\n}\n```",
"bugtrack_url": null,
"license": "MIT",
"summary": "Paper - Pytorch",
"version": "0.0.1",
"project_urls": {
"Documentation": "https://github.com/kyegomez/PaLM2-VAdapter",
"Homepage": "https://github.com/kyegomez/PaLM2-VAdapter",
"Repository": "https://github.com/kyegomez/PaLM2-VAdapter"
},
"split_keywords": [
"artificial intelligence",
"deep learning",
"optimizers",
"prompt engineering"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "12050a2ded7ee2e0afda301e009ec36097ade6f137ccb11b423595c94eda44ed",
"md5": "b0811b81d459f9a54b4ed8831ad5a263",
"sha256": "89668b235f180dc3269cd5703cc5616854b9dfd7ebe80a9655b3dd37bd2208c3"
},
"downloads": -1,
"filename": "palm_vadapter-0.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b0811b81d459f9a54b4ed8831ad5a263",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6,<4.0",
"size": 6995,
"upload_time": "2024-02-19T22:09:42",
"upload_time_iso_8601": "2024-02-19T22:09:42.684308Z",
"url": "https://files.pythonhosted.org/packages/12/05/0a2ded7ee2e0afda301e009ec36097ade6f137ccb11b423595c94eda44ed/palm_vadapter-0.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ac7478b791690927a841d420b042f2f25d0eb76d48781155e5405e70e6225ea1",
"md5": "6b9acc99001de047545d5a993c4027eb",
"sha256": "97e05c2289196240092faf91b0e02a69f102d212d718a884114dbb642d19ded1"
},
"downloads": -1,
"filename": "palm_vadapter-0.0.1.tar.gz",
"has_sig": false,
"md5_digest": "6b9acc99001de047545d5a993c4027eb",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6,<4.0",
"size": 7096,
"upload_time": "2024-02-19T22:09:44",
"upload_time_iso_8601": "2024-02-19T22:09:44.286482Z",
"url": "https://files.pythonhosted.org/packages/ac/74/78b791690927a841d420b042f2f25d0eb76d48781155e5405e70e6225ea1/palm_vadapter-0.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-19 22:09:44",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "kyegomez",
"github_project": "PaLM2-VAdapter",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "palm-vadapter"
}
JSON
