# Image 2 Vec with PyTorch
Medium post on building the first version from scratch: https://becominghuman.ai/extract-a-feature-vector-for-any-image-with-pytorch-9717561d1d4c
### Applications of image embeddings:
- Ranking for recommender systems
- Clustering images to different categories
- Classification tasks
- Image compression
## Available models
|Model name|Return vector length|
|----|----|
|Resnet-18|512|
|Alexnet|4096|
|Vgg-11|4096|
|Densenet|1024|
|efficientnet_b0|1280|
|efficientnet_b1|1280|
|efficientnet_b2|1408|
|efficientnet_b3|1536|
|efficientnet_b4|1792|
|efficientnet_b5|2048|
|efficientnet_b6|2304|
|efficientnet_b7|2560|
## Installation
Tested on Python 3.6 and torchvision 0.11.0 (nightly, 2021-09-25)
Requires Pytorch: http://pytorch.org/
```conda install -c pytorch-nightly torchvision```
```pip install img2vec_pytorch```
## Run test
```python -m img2vec_pytorch.test_img_to_vec```
## Using img2vec as a library
```python
from img2vec_pytorch import Img2Vec
from PIL import Image
# Initialize Img2Vec with GPU
img2vec = Img2Vec(device_preference=["cuda", "cpu"])
# Read in an image (rgb format)
img = Image.open('test.jpg')
# Get a vector from img2vec, returned as a torch FloatTensor
vec = img2vec.get_vec(img, tensor=True)
# Or submit a list
vectors = img2vec.get_vec(list_of_PIL_images)
```
##### For running the example, you will additionally need:
* Pillow: ```pip install Pillow```
* Sklearn ```pip install scikit-learn```
## Running the example
```git clone https://github.com/christiansafka/img2vec.git```
```cd img2vec/example```
```python test_img_similarity.py```
#### Expected output
```
Which filename would you like similarities for?
cat.jpg
0.72832 cat2.jpg
0.641478 catdog.jpg
0.575845 face.jpg
0.516689 face2.jpg
Which filename would you like similarities for?
face2.jpg
0.668525 face.jpg
0.516689 cat.jpg
0.50084 cat2.jpg
0.484863 catdog.jpg
```
Try adding your own photos!
#### Img2Vec Params
**cuda** = (True, False) # Run on GPU? default: False<br>
**model** = ('resnet-18', 'efficientnet_b0', etc.) # Which model to use? default: 'resnet-18'<br>
## Advanced users
----
### Read only file systems
If you use this library from the app running in read only environment (for example, docker container),
specify writable directory where app can store pre-trained models.
```bash
export TORCH_HOME=/tmp/torch
```
### Additional Parameters
**layer** = 'layer_name' or int # For advanced users, which layer of the model to extract the output from. default: 'avgpool' <br>
**layer_output_size** = int # Size of the output of your selected layer<br>
**gpu** = (0, 1, etc.) # Which GPU to use? default: 0<br>
### [Resnet-18](http://pytorch-zh.readthedocs.io/en/latest/_modules/torchvision/models/resnet.html)
Defaults: (layer = 'avgpool', layer_output_size = 512)<br>
Layer parameter must be an string representing the name of a layer below
```python
conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False)
bn1 = nn.BatchNorm2d(64)
relu = nn.ReLU(inplace=True)
maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
layer1 = self._make_layer(block, 64, layers[0])
layer2 = self._make_layer(block, 128, layers[1], stride=2)
layer3 = self._make_layer(block, 256, layers[2], stride=2)
layer4 = self._make_layer(block, 512, layers[3], stride=2)
avgpool = nn.AvgPool2d(7)
fc = nn.Linear(512 * block.expansion, num_classes)
```
### [Alexnet](http://pytorch-zh.readthedocs.io/en/latest/_modules/torchvision/models/alexnet.html)
Defaults: (layer = 2, layer_output_size = 4096)<br>
Layer parameter must be an integer representing one of the layers below
```python
alexnet.classifier = nn.Sequential(
7. nn.Dropout(), < - output_size = 9216
6. nn.Linear(256 * 6 * 6, 4096), < - output_size = 4096
5. nn.ReLU(inplace=True), < - output_size = 4096
4. nn.Dropout(), < - output_size = 4096
3. nn.Linear(4096, 4096), < - output_size = 4096
2. nn.ReLU(inplace=True), < - output_size = 4096
1. nn.Linear(4096, num_classes), < - output_size = 4096
)
```
### [Vgg](https://pytorch.org/vision/stable/_modules/torchvision/models/vgg.html)
Defaults: (layer = 2, layer_output_size = 4096)<br>
```python
vgg.classifier = nn.Sequential(
nn.Linear(512 * 7 * 7, 4096),
nn.ReLU(True),
nn.Dropout(),
nn.Linear(4096, 4096),
nn.ReLU(True),
nn.Dropout(),
nn.Linear(4096, num_classes),
)
```
### [Densenet](https://pytorch.org/vision/stable/_modules/torchvision/models/densenet.html)
Defaults: (layer = 1 from features, layer_output_size = 1024)<br>
```python
densenet.features = nn.Sequential(OrderedDict([
('conv0', nn.Conv2d(3, num_init_features, kernel_size=7, stride=2,
padding=3, bias=False)),
('norm0', nn.BatchNorm2d(num_init_features)),
('relu0', nn.ReLU(inplace=True)),
('pool0', nn.MaxPool2d(kernel_size=3, stride=2, padding=1)),
]))
```
### [EfficientNet](https://arxiv.org/abs/1905.11946)
Defaults: (layer = 1 from features, layer_output_size = 1280 for efficientnet_b0 model)<br>
## To-do
- Benchmark speed and accuracy
- Add ability to fine-tune on input data
- Export documentation to a normal place
Raw data
{
"_id": null,
"home_page": "https://github.com/psilabs-dev/img2vec",
"name": "img2vec-pytorch-2",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": "img2vec image vector classification pytorch convert",
"author": "psilabs-dev",
"author_email": "113860476+psilabs-dev@users.noreply.github.com",
"download_url": "https://files.pythonhosted.org/packages/66/0a/4523995bda67f79cf4212598a53e8d228dd23c642c252c63e863cb2c16ab/img2vec_pytorch_2-1.2.0.tar.gz",
"platform": null,
"description": "# Image 2 Vec with PyTorch\n\nMedium post on building the first version from scratch: https://becominghuman.ai/extract-a-feature-vector-for-any-image-with-pytorch-9717561d1d4c\n\n### Applications of image embeddings:\n - Ranking for recommender systems\n - Clustering images to different categories\n - Classification tasks\n - Image compression\n\n## Available models\n|Model name|Return vector length|\n|----|----|\n|Resnet-18|512|\n|Alexnet|4096|\n|Vgg-11|4096|\n|Densenet|1024|\n|efficientnet_b0|1280|\n|efficientnet_b1|1280|\n|efficientnet_b2|1408|\n|efficientnet_b3|1536|\n|efficientnet_b4|1792|\n|efficientnet_b5|2048|\n|efficientnet_b6|2304|\n|efficientnet_b7|2560|\n\n## Installation\n\nTested on Python 3.6 and torchvision 0.11.0 (nightly, 2021-09-25) \n\nRequires Pytorch: http://pytorch.org/\n\n```conda install -c pytorch-nightly torchvision```\n\n```pip install img2vec_pytorch```\n\n## Run test\n\n```python -m img2vec_pytorch.test_img_to_vec```\n\n## Using img2vec as a library\n```python\nfrom img2vec_pytorch import Img2Vec\nfrom PIL import Image\n\n# Initialize Img2Vec with GPU\nimg2vec = Img2Vec(device_preference=[\"cuda\", \"cpu\"])\n\n# Read in an image (rgb format)\nimg = Image.open('test.jpg')\n# Get a vector from img2vec, returned as a torch FloatTensor\nvec = img2vec.get_vec(img, tensor=True)\n# Or submit a list\nvectors = img2vec.get_vec(list_of_PIL_images)\n```\n\n##### For running the example, you will additionally need:\n * Pillow: ```pip install Pillow```\n * Sklearn ```pip install scikit-learn```\n\n## Running the example\n```git clone https://github.com/christiansafka/img2vec.git```\n\n```cd img2vec/example```\n\n```python test_img_similarity.py```\n\n#### Expected output\n```\nWhich filename would you like similarities for?\ncat.jpg\n0.72832 cat2.jpg\n0.641478 catdog.jpg\n0.575845 face.jpg\n0.516689 face2.jpg\n\nWhich filename would you like similarities for?\nface2.jpg\n0.668525 face.jpg\n0.516689 cat.jpg\n0.50084 cat2.jpg\n0.484863 catdog.jpg\n```\nTry adding your own photos!\n\n\n#### Img2Vec Params\n**cuda** = (True, False) # Run on GPU? default: False<br>\n**model** = ('resnet-18', 'efficientnet_b0', etc.) # Which model to use? default: 'resnet-18'<br>\n\n## Advanced users \n----\n\n### Read only file systems\n\nIf you use this library from the app running in read only environment (for example, docker container), \nspecify writable directory where app can store pre-trained models. \n\n```bash\nexport TORCH_HOME=/tmp/torch\n```\n\n### Additional Parameters\n\n**layer** = 'layer_name' or int # For advanced users, which layer of the model to extract the output from. default: 'avgpool' <br>\n**layer_output_size** = int # Size of the output of your selected layer<br>\n**gpu** = (0, 1, etc.) # Which GPU to use? default: 0<br>\n\n### [Resnet-18](http://pytorch-zh.readthedocs.io/en/latest/_modules/torchvision/models/resnet.html)\nDefaults: (layer = 'avgpool', layer_output_size = 512)<br>\nLayer parameter must be an string representing the name of a layer below\n```python\nconv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False)\nbn1 = nn.BatchNorm2d(64)\nrelu = nn.ReLU(inplace=True)\nmaxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)\nlayer1 = self._make_layer(block, 64, layers[0])\nlayer2 = self._make_layer(block, 128, layers[1], stride=2)\nlayer3 = self._make_layer(block, 256, layers[2], stride=2)\nlayer4 = self._make_layer(block, 512, layers[3], stride=2)\navgpool = nn.AvgPool2d(7)\nfc = nn.Linear(512 * block.expansion, num_classes)\n```\n### [Alexnet](http://pytorch-zh.readthedocs.io/en/latest/_modules/torchvision/models/alexnet.html)\nDefaults: (layer = 2, layer_output_size = 4096)<br>\nLayer parameter must be an integer representing one of the layers below\n```python\nalexnet.classifier = nn.Sequential(\n 7. nn.Dropout(), < - output_size = 9216\n 6. nn.Linear(256 * 6 * 6, 4096), < - output_size = 4096\n 5. nn.ReLU(inplace=True), < - output_size = 4096\n 4. nn.Dropout(),\t\t < - output_size = 4096\n 3. nn.Linear(4096, 4096),\t < - output_size = 4096\n 2. nn.ReLU(inplace=True), < - output_size = 4096\n 1. nn.Linear(4096, num_classes), < - output_size = 4096\n )\n```\n\n### [Vgg](https://pytorch.org/vision/stable/_modules/torchvision/models/vgg.html)\nDefaults: (layer = 2, layer_output_size = 4096)<br>\n```python\nvgg.classifier = nn.Sequential(\n nn.Linear(512 * 7 * 7, 4096),\n nn.ReLU(True),\n nn.Dropout(),\n nn.Linear(4096, 4096),\n nn.ReLU(True),\n nn.Dropout(),\n nn.Linear(4096, num_classes),\n )\n```\n\n### [Densenet](https://pytorch.org/vision/stable/_modules/torchvision/models/densenet.html)\nDefaults: (layer = 1 from features, layer_output_size = 1024)<br>\n```python\ndensenet.features = nn.Sequential(OrderedDict([\n\t('conv0', nn.Conv2d(3, num_init_features, kernel_size=7, stride=2,\n\t\t\t\t\t\tpadding=3, bias=False)),\n\t('norm0', nn.BatchNorm2d(num_init_features)),\n\t('relu0', nn.ReLU(inplace=True)),\n\t('pool0', nn.MaxPool2d(kernel_size=3, stride=2, padding=1)),\n]))\n```\n\n### [EfficientNet](https://arxiv.org/abs/1905.11946)\nDefaults: (layer = 1 from features, layer_output_size = 1280 for efficientnet_b0 model)<br>\n\n\n## To-do\n- Benchmark speed and accuracy\n- Add ability to fine-tune on input data\n- Export documentation to a normal place\n\n\n\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Use pre-trained models in PyTorch to extract vector embeddings for any image",
"version": "1.2.0",
"project_urls": {
"Homepage": "https://github.com/psilabs-dev/img2vec"
},
"split_keywords": [
"img2vec",
"image",
"vector",
"classification",
"pytorch",
"convert"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "85d5b2c1a7836410ee7bd9e26be68707a3a1fc33501f8a7be4eeb4bf8663fcd8",
"md5": "eb07d8f41419e46cc92b1a897b9daf73",
"sha256": "5f0f35dda51b541a99a6e9f02fa3eb66195b5ea6fa052910c45d0bd55f65126f"
},
"downloads": -1,
"filename": "img2vec_pytorch_2-1.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "eb07d8f41419e46cc92b1a897b9daf73",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 7209,
"upload_time": "2025-02-28T23:48:25",
"upload_time_iso_8601": "2025-02-28T23:48:25.368545Z",
"url": "https://files.pythonhosted.org/packages/85/d5/b2c1a7836410ee7bd9e26be68707a3a1fc33501f8a7be4eeb4bf8663fcd8/img2vec_pytorch_2-1.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "660a4523995bda67f79cf4212598a53e8d228dd23c642c252c63e863cb2c16ab",
"md5": "7e33be22f769df7a3db49cf6d47b1edf",
"sha256": "557a9bdc1233f68cf058fec0e223596438e55d7ba51944a4dc9d433a01e96f3b"
},
"downloads": -1,
"filename": "img2vec_pytorch_2-1.2.0.tar.gz",
"has_sig": false,
"md5_digest": "7e33be22f769df7a3db49cf6d47b1edf",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 6936,
"upload_time": "2025-02-28T23:48:27",
"upload_time_iso_8601": "2025-02-28T23:48:27.059655Z",
"url": "https://files.pythonhosted.org/packages/66/0a/4523995bda67f79cf4212598a53e8d228dd23c642c252c63e863cb2c16ab/img2vec_pytorch_2-1.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-28 23:48:27",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "psilabs-dev",
"github_project": "img2vec",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "img2vec-pytorch-2"
}
JSON
