# Clip2ClassDist
Clip2ClassDist is a Python script that analyzes image classes by leveraging OpenAI's CLIP model to generate image embeddings. It computes the center and radius of clusters formed by embeddings, the distribution of dot products within each class, and the average, minimum, and maximum dot products between different classes.
The script takes a list of image folders as input, with each folder representing a distinct image class. It uses the specified CLIP model to create normalized embeddings for each image in the folders. The script then analyzes the embeddings using the Faiss library for efficient similarity search and clustering.
## Installation
Clone the repository:
```bash
git clone https://github.com/ternaus/clip2classdist.git
```
Install the required dependencies:
```bash
pip install -r requirements.txt
```
## Usage
The main script compute.py can be executed with the following command:
```bash
Copy code
python clip2class_dist/compute.py --clip_model MODEL_NAME --image_folders CLASS_NAME_1 FOLDER_PATH_1 CLASS_NAME_2 FOLDER_PATH_2 [--device DEVICE]
```
Arguments:
* --clip_model: Name of the CLIP model to use (e.g., `ViT-B/32`). Where allowed models:
* `ViT-B/32`
* `RN50`
* `RN101`
* `RN50x4`
* `RN50x16`
* `ViT-B/16`
* `RN50x64`
* `ViT-L/14`
* `ViT-L/14@336px`
* --image_folders: List of pairs containing class names and corresponding folder paths containing images. Each pair should be separated by a space, and pairs should be space-separated as well (e.g., "dogs /path/to/dog_images cats /path/to/cat_images").
* --device (optional): Device to run the model on (default: "cpu"). Set to "cuda" if you have a compatible GPU.
Example usage:
```bash
python clip2class_dist/compute.py --clip_model ViT-B/32 --image_folders dogs /path/to/dog_images cats /path/to/cat_images --device cpu
```
## Output
The script will output the following information:
1. Cluster centers for each class.
2. Cluster radii for each class.
3. Dot product distributions within each class.
4. Average dot product between different classes.
5. Minimum dot product between different classes.
6. Maximum dot product between different classes.
Example output:
```bash
Cluster centers: {'dogs': array([...]), 'cats': array([...])}
Cluster radii: {'dogs': 0.2384927, 'cats': 0.27184734}
Dot product distributions: {'dogs': array([...]), 'cats': array([...])}
Average dot product between classes: 0.19983745
Minimum dot product between classes: 0.01759241
Maximum dot product between classes: 0.38273957
```
## License
This project is released under the MIT License.
Raw data
{
"_id": null,
"home_page": "https://github.com/ternaus/clip2classdist",
"name": "clip2classdist",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "Vladimir Iglovikov",
"author_email": "iglovikov@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/25/45/bd6009a55a04305bef839bc26b9a3c0bf6ef9c399bc88e12cade55e71604/clip2classdist-0.1.0.tar.gz",
"platform": null,
"description": "# Clip2ClassDist\n\nClip2ClassDist is a Python script that analyzes image classes by leveraging OpenAI's CLIP model to generate image embeddings. It computes the center and radius of clusters formed by embeddings, the distribution of dot products within each class, and the average, minimum, and maximum dot products between different classes.\n\nThe script takes a list of image folders as input, with each folder representing a distinct image class. It uses the specified CLIP model to create normalized embeddings for each image in the folders. The script then analyzes the embeddings using the Faiss library for efficient similarity search and clustering.\n\n## Installation\n\nClone the repository:\n\n```bash\ngit clone https://github.com/ternaus/clip2classdist.git\n```\n\nInstall the required dependencies:\n\n```bash\npip install -r requirements.txt\n```\n\n## Usage\n\nThe main script compute.py can be executed with the following command:\n\n```bash\nCopy code\npython clip2class_dist/compute.py --clip_model MODEL_NAME --image_folders CLASS_NAME_1 FOLDER_PATH_1 CLASS_NAME_2 FOLDER_PATH_2 [--device DEVICE]\n```\n\nArguments:\n\n* --clip_model: Name of the CLIP model to use (e.g., `ViT-B/32`). Where allowed models:\n * `ViT-B/32`\n * `RN50`\n * `RN101`\n * `RN50x4`\n * `RN50x16`\n * `ViT-B/16`\n * `RN50x64`\n * `ViT-L/14`\n * `ViT-L/14@336px`\n* --image_folders: List of pairs containing class names and corresponding folder paths containing images. Each pair should be separated by a space, and pairs should be space-separated as well (e.g., \"dogs /path/to/dog_images cats /path/to/cat_images\").\n* --device (optional): Device to run the model on (default: \"cpu\"). Set to \"cuda\" if you have a compatible GPU.\n\nExample usage:\n\n```bash\npython clip2class_dist/compute.py --clip_model ViT-B/32 --image_folders dogs /path/to/dog_images cats /path/to/cat_images --device cpu\n```\n\n## Output\n\nThe script will output the following information:\n\n1. Cluster centers for each class.\n2. Cluster radii for each class.\n3. Dot product distributions within each class.\n4. Average dot product between different classes.\n5. Minimum dot product between different classes.\n6. Maximum dot product between different classes.\n\nExample output:\n\n```bash\nCluster centers: {'dogs': array([...]), 'cats': array([...])}\nCluster radii: {'dogs': 0.2384927, 'cats': 0.27184734}\nDot product distributions: {'dogs': array([...]), 'cats': array([...])}\nAverage dot product between classes: 0.19983745\nMinimum dot product between classes: 0.01759241\nMaximum dot product between classes: 0.38273957\n```\n\n## License\n\nThis project is released under the MIT License.\n",
"bugtrack_url": null,
"license": "",
"summary": "A Python script that analyzes image classes using OpenAI CLIP model",
"version": "0.1.0",
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c906f2d878636684bb29b214a2664971bf135baca9f3127a5fa91ecbd6c6d9a2",
"md5": "d3c74592cb315b9a64b40dfd45f4735a",
"sha256": "99d0bb3ac2b0d410eb54b59c896ac824ec21d82d4e7597b3aca81287eeda4d62"
},
"downloads": -1,
"filename": "clip2classdist-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d3c74592cb315b9a64b40dfd45f4735a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 6695,
"upload_time": "2023-03-31T18:13:13",
"upload_time_iso_8601": "2023-03-31T18:13:13.898342Z",
"url": "https://files.pythonhosted.org/packages/c9/06/f2d878636684bb29b214a2664971bf135baca9f3127a5fa91ecbd6c6d9a2/clip2classdist-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "2545bd6009a55a04305bef839bc26b9a3c0bf6ef9c399bc88e12cade55e71604",
"md5": "979a1e86e344cc5db00e921b0b65f867",
"sha256": "3a409da4079ad86b001e7b071200da586c6ce1384a1025b71eec52661420b619"
},
"downloads": -1,
"filename": "clip2classdist-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "979a1e86e344cc5db00e921b0b65f867",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 6039,
"upload_time": "2023-03-31T18:13:15",
"upload_time_iso_8601": "2023-03-31T18:13:15.662553Z",
"url": "https://files.pythonhosted.org/packages/25/45/bd6009a55a04305bef839bc26b9a3c0bf6ef9c399bc88e12cade55e71604/clip2classdist-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-03-31 18:13:15",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "ternaus",
"github_project": "clip2classdist",
"lcname": "clip2classdist"
}
JSON
