| Name | ncut-pytorch JSON |
| Version |
1.8.13
JSON |
| download |
| home_page | None |
| Summary | None |
| upload_time | 2025-01-10 01:50:57 |
| maintainer | None |
| docs_url | None |
| author | Huzheng Yang |
| requires_python | None |
| license | None |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
<div style="text-align: center;">
<img src="../docs/images/ncut.svg" alt="NCUT" style="width: 80%; filter: brightness(60%) grayscale(100%);"/>
</div>
### [🌐Documentation](https://ncut-pytorch.readthedocs.io/) | [🤗HuggingFace Demo](https://huggingface.co/spaces/huzey/ncut-pytorch)
## NCUT: Nyström Normalized Cut
**Normalized Cut**, aka. spectral clustering, is a graphical method to analyze data grouping in the affinity eigenvector space. It has been widely used for unsupervised segmentation in the 2000s.
**Nyström Normalized Cut**, is a new approximation algorithm developed for large-scale graph cuts, a large-graph of million nodes can be processed in under 10s (cpu) or 2s (gpu).
---
## Installation
#### 1. Install PyTorch
<div style="text-align:">
<pre><code class="language-shell">conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
</code></pre>
</div>
#### 2. Install `ncut-pytorch`
<div style="text-align:">
<pre><code class="language-shell">pip install ncut-pytorch</code></pre>
</div>
#### Trouble Shooting
In case of `pip` install failed, please try install the build dependencies
Option A:
<div style="text-align:">
<pre><code class="language-shell">sudo apt-get update && sudo apt-get install build-essential cargo rustc -y</code></pre>
</div>
Option B:
<div style="text-align:">
<pre><code class="language-shell">conda install rust -c conda-forge</code></pre>
</div>
Option C:
<div style="text-align:">
<pre><code class="language-shell">curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh && . "$HOME/.cargo/env"</code></pre>
</div>
## Quick Start
Minimal example on how to run NCUT:
```py linenums="1"
import torch
from ncut_pytorch import NCUT, rgb_from_tsne_3d
model_features = torch.rand(20, 64, 64, 768) # (B, H, W, C)
inp = model_features.reshape(-1, 768) # flatten
eigvectors, eigvalues = NCUT(num_eig=100, device='cuda:0').fit_transform(inp)
tsne_x3d, tsne_rgb = rgb_from_tsne_3d(eigvectors, device='cuda:0')
eigvectors = eigvectors.reshape(20, 64, 64, 100) # (B, H, W, num_eig)
tsne_rgb = tsne_rgb.reshape(20, 64, 64, 3) # (B, H, W, 3)
```
#### Load Feature Extractor Model
Any backbone model works as plug-in feature extractor.
We have implemented some backbone models, here is a list of available models:
```py
from ncut_pytorch.backbone import list_models
print(list_models())
[
'SAM2(sam2_hiera_t)', 'SAM2(sam2_hiera_s)', 'SAM2(sam2_hiera_b+)', 'SAM2(sam2_hiera_l)',
'SAM(sam_vit_b)', 'SAM(sam_vit_l)', 'SAM(sam_vit_h)', 'MobileSAM(TinyViT)',
'DiNOv2reg(dinov2_vits14_reg)', 'DiNOv2reg(dinov2_vitb14_reg)', 'DiNOv2reg(dinov2_vitl14_reg)', 'DiNOv2reg(dinov2_vitg14_reg)',
'DiNOv2(dinov2_vits14)', 'DiNOv2(dinov2_vitb14)', 'DiNOv2(dinov2_vitl14)', 'DiNOv2(dinov2_vitg14)',
'DiNO(dino_vits8_896)', 'DiNO(dino_vitb8_896)', 'DiNO(dino_vits8_672)', 'DiNO(dino_vitb8_672)', 'DiNO(dino_vits8_448)', 'DiNO(dino_vitb8_448)', 'DiNO(dino_vits16_448)', 'DiNO(dino_vitb16_448)',
'Diffusion(stabilityai/stable-diffusion-2)', 'Diffusion(CompVis/stable-diffusion-v1-4)', 'Diffusion(stabilityai/stable-diffusion-3-medium-diffusers)',
'CLIP(ViT-B-16/openai)', 'CLIP(ViT-L-14/openai)', 'CLIP(ViT-H-14/openai)', 'CLIP(ViT-B-16/laion2b_s34b_b88k)',
'CLIP(convnext_base_w_320/laion_aesthetic_s13b_b82k)', 'CLIP(convnext_large_d_320/laion2b_s29b_b131k_ft_soup)', 'CLIP(convnext_xxlarge/laion2b_s34b_b82k_augreg_soup)',
'CLIP(eva02_base_patch14_448/mim_in22k_ft_in1k)', "CLIP(eva02_large_patch14_448/mim_m38m_ft_in22k_in1k)",
'MAE(vit_base)', 'MAE(vit_large)', 'MAE(vit_huge)',
'ImageNet(vit_base)'
]
```
#### Image model example:
```py linenums="1"
import torch
from ncut_pytorch import NCUT, rgb_from_tsne_3d
from ncut_pytorch.backbone import load_model, extract_features
model = load_model(model_name="SAM(sam_vit_b)")
images = torch.rand(20, 3, 1024, 1024)
model_features = extract_features(images, model, node_type='attn', layer=6)
# model_features = model(images)['attn'][6] # this also works
inp = model_features.reshape(-1, 768) # flatten
eigvectors, eigvalues = NCUT(num_eig=100, device='cuda:0').fit_transform(inp)
tsne_x3d, tsne_rgb = rgb_from_tsne_3d(eigvectors, device='cuda:0')
eigvectors = eigvectors.reshape(20, 64, 64, 100) # (B, H, W, num_eig)
tsne_rgb = tsne_rgb.reshape(20, 64, 64, 3) # (B, H, W, 3)
```
#### Text model example:
```py linenums="1"
import os
from ncut_pytorch import NCUT, rgb_from_tsne_3d
from ncut_pytorch.backbone_text import load_text_model
os.environ['HF_ACCESS_TOKEN'] = "your_huggingface_token"
llama = load_text_model("meta-llama/Meta-Llama-3.1-8B").cuda()
output_dict = llama("The quick white fox jumps over the lazy cat.")
model_features = output_dict['block'][31].squeeze(0) # 32nd block output
token_texts = output_dict['token_texts']
eigvectors, eigvalues = NCUT(num_eig=5, device='cuda:0').fit_transform(model_features)
tsne_x3d, tsne_rgb = rgb_from_tsne_3d(eigvectors, device='cuda:0')
# eigvectors.shape[0] == tsne_rgb.shape[0] == len(token_texts)
```
---
> paper in prep, Yang 2024
>
> AlignedCut: Visual Concepts Discovery on Brain-Guided Universal Feature Space, Huzheng Yang, James Gee\*, Jianbo Shi\*,2024
>
> Normalized Cuts and Image Segmentation, Jianbo Shi and Jitendra Malik, 2000
Raw data
{
"_id": null,
"home_page": null,
"name": "ncut-pytorch",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "Huzheng Yang",
"author_email": "huze.yann@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/d1/e4/5c3a7708d528e0b2a2ed926c5973d1cae5f355b2feb4f069851d574c1d7a/ncut_pytorch-1.8.13.tar.gz",
"platform": null,
"description": "<div style=\"text-align: center;\">\n <img src=\"../docs/images/ncut.svg\" alt=\"NCUT\" style=\"width: 80%; filter: brightness(60%) grayscale(100%);\"/>\n</div>\n\n### [\ud83c\udf10Documentation](https://ncut-pytorch.readthedocs.io/) | [\ud83e\udd17HuggingFace Demo](https://huggingface.co/spaces/huzey/ncut-pytorch)\n\n\n## NCUT: Nystr\u00f6m Normalized Cut\n\n**Normalized Cut**, aka. spectral clustering, is a graphical method to analyze data grouping in the affinity eigenvector space. It has been widely used for unsupervised segmentation in the 2000s.\n\n**Nystr\u00f6m Normalized Cut**, is a new approximation algorithm developed for large-scale graph cuts, a large-graph of million nodes can be processed in under 10s (cpu) or 2s (gpu). \n\n\n---\n\n## Installation\n\n#### 1. Install PyTorch\n\n<div style=\"text-align:\">\n<pre><code class=\"language-shell\">conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia\n</code></pre>\n</div>\n\n#### 2. Install `ncut-pytorch`\n\n<div style=\"text-align:\">\n <pre><code class=\"language-shell\">pip install ncut-pytorch</code></pre>\n</div>\n\n\n#### Trouble Shooting\n\nIn case of `pip` install failed, please try install the build dependencies\n\nOption A:\n<div style=\"text-align:\">\n <pre><code class=\"language-shell\">sudo apt-get update && sudo apt-get install build-essential cargo rustc -y</code></pre>\n</div>\n\nOption B:\n<div style=\"text-align:\">\n <pre><code class=\"language-shell\">conda install rust -c conda-forge</code></pre>\n</div>\n\nOption C:\n<div style=\"text-align:\">\n <pre><code class=\"language-shell\">curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh && . \"$HOME/.cargo/env\"</code></pre>\n</div>\n\n\n## Quick Start\n\n\nMinimal example on how to run NCUT:\n\n```py linenums=\"1\"\nimport torch\nfrom ncut_pytorch import NCUT, rgb_from_tsne_3d\n\nmodel_features = torch.rand(20, 64, 64, 768) # (B, H, W, C)\n\ninp = model_features.reshape(-1, 768) # flatten\neigvectors, eigvalues = NCUT(num_eig=100, device='cuda:0').fit_transform(inp)\ntsne_x3d, tsne_rgb = rgb_from_tsne_3d(eigvectors, device='cuda:0')\n\neigvectors = eigvectors.reshape(20, 64, 64, 100) # (B, H, W, num_eig)\ntsne_rgb = tsne_rgb.reshape(20, 64, 64, 3) # (B, H, W, 3)\n```\n\n#### Load Feature Extractor Model\n\nAny backbone model works as plug-in feature extractor. \nWe have implemented some backbone models, here is a list of available models:\n\n```py\nfrom ncut_pytorch.backbone import list_models\nprint(list_models())\n[\n 'SAM2(sam2_hiera_t)', 'SAM2(sam2_hiera_s)', 'SAM2(sam2_hiera_b+)', 'SAM2(sam2_hiera_l)', \n 'SAM(sam_vit_b)', 'SAM(sam_vit_l)', 'SAM(sam_vit_h)', 'MobileSAM(TinyViT)', \n 'DiNOv2reg(dinov2_vits14_reg)', 'DiNOv2reg(dinov2_vitb14_reg)', 'DiNOv2reg(dinov2_vitl14_reg)', 'DiNOv2reg(dinov2_vitg14_reg)', \n 'DiNOv2(dinov2_vits14)', 'DiNOv2(dinov2_vitb14)', 'DiNOv2(dinov2_vitl14)', 'DiNOv2(dinov2_vitg14)', \n 'DiNO(dino_vits8_896)', 'DiNO(dino_vitb8_896)', 'DiNO(dino_vits8_672)', 'DiNO(dino_vitb8_672)', 'DiNO(dino_vits8_448)', 'DiNO(dino_vitb8_448)', 'DiNO(dino_vits16_448)', 'DiNO(dino_vitb16_448)',\n 'Diffusion(stabilityai/stable-diffusion-2)', 'Diffusion(CompVis/stable-diffusion-v1-4)', 'Diffusion(stabilityai/stable-diffusion-3-medium-diffusers)',\n 'CLIP(ViT-B-16/openai)', 'CLIP(ViT-L-14/openai)', 'CLIP(ViT-H-14/openai)', 'CLIP(ViT-B-16/laion2b_s34b_b88k)', \n 'CLIP(convnext_base_w_320/laion_aesthetic_s13b_b82k)', 'CLIP(convnext_large_d_320/laion2b_s29b_b131k_ft_soup)', 'CLIP(convnext_xxlarge/laion2b_s34b_b82k_augreg_soup)', \n 'CLIP(eva02_base_patch14_448/mim_in22k_ft_in1k)', \"CLIP(eva02_large_patch14_448/mim_m38m_ft_in22k_in1k)\",\n 'MAE(vit_base)', 'MAE(vit_large)', 'MAE(vit_huge)', \n 'ImageNet(vit_base)'\n]\n```\n\n#### Image model example:\n\n```py linenums=\"1\"\nimport torch\nfrom ncut_pytorch import NCUT, rgb_from_tsne_3d\nfrom ncut_pytorch.backbone import load_model, extract_features\n\nmodel = load_model(model_name=\"SAM(sam_vit_b)\")\nimages = torch.rand(20, 3, 1024, 1024)\nmodel_features = extract_features(images, model, node_type='attn', layer=6)\n# model_features = model(images)['attn'][6] # this also works\n\ninp = model_features.reshape(-1, 768) # flatten\neigvectors, eigvalues = NCUT(num_eig=100, device='cuda:0').fit_transform(inp)\ntsne_x3d, tsne_rgb = rgb_from_tsne_3d(eigvectors, device='cuda:0')\n\neigvectors = eigvectors.reshape(20, 64, 64, 100) # (B, H, W, num_eig)\ntsne_rgb = tsne_rgb.reshape(20, 64, 64, 3) # (B, H, W, 3)\n```\n\n#### Text model example:\n\n```py linenums=\"1\"\nimport os\nfrom ncut_pytorch import NCUT, rgb_from_tsne_3d\nfrom ncut_pytorch.backbone_text import load_text_model\n\nos.environ['HF_ACCESS_TOKEN'] = \"your_huggingface_token\"\nllama = load_text_model(\"meta-llama/Meta-Llama-3.1-8B\").cuda()\noutput_dict = llama(\"The quick white fox jumps over the lazy cat.\")\n\nmodel_features = output_dict['block'][31].squeeze(0) # 32nd block output\ntoken_texts = output_dict['token_texts']\neigvectors, eigvalues = NCUT(num_eig=5, device='cuda:0').fit_transform(model_features)\ntsne_x3d, tsne_rgb = rgb_from_tsne_3d(eigvectors, device='cuda:0')\n# eigvectors.shape[0] == tsne_rgb.shape[0] == len(token_texts)\n```\n\n---\n\n> paper in prep, Yang 2024\n>\n> AlignedCut: Visual Concepts Discovery on Brain-Guided Universal Feature Space, Huzheng Yang, James Gee\\*, Jianbo Shi\\*,2024\n> \n> Normalized Cuts and Image Segmentation, Jianbo Shi and Jitendra Malik, 2000\n",
"bugtrack_url": null,
"license": null,
"summary": null,
"version": "1.8.13",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d7d7562e8a114467ab001161a644faeac6fd62832b87d00c97c5a09ec6e535fd",
"md5": "85aca36cb6667dff3bb7d7c8f08e5ad8",
"sha256": "3868edf3261a7229cd84878e717b9e6cfa6e5c90c6c6e983ed8ea9357e81b577"
},
"downloads": -1,
"filename": "ncut_pytorch-1.8.13-py3-none-any.whl",
"has_sig": false,
"md5_digest": "85aca36cb6667dff3bb7d7c8f08e5ad8",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 36712,
"upload_time": "2025-01-10T01:50:54",
"upload_time_iso_8601": "2025-01-10T01:50:54.756299Z",
"url": "https://files.pythonhosted.org/packages/d7/d7/562e8a114467ab001161a644faeac6fd62832b87d00c97c5a09ec6e535fd/ncut_pytorch-1.8.13-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d1e45c3a7708d528e0b2a2ed926c5973d1cae5f355b2feb4f069851d574c1d7a",
"md5": "8d6be46eda76ceefa901e329a88b5816",
"sha256": "c146527c2b70f813663ba4d758a46c20395b464732640f15e89440e49016cb08"
},
"downloads": -1,
"filename": "ncut_pytorch-1.8.13.tar.gz",
"has_sig": false,
"md5_digest": "8d6be46eda76ceefa901e329a88b5816",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 35299,
"upload_time": "2025-01-10T01:50:57",
"upload_time_iso_8601": "2025-01-10T01:50:57.021105Z",
"url": "https://files.pythonhosted.org/packages/d1/e4/5c3a7708d528e0b2a2ed926c5973d1cae5f355b2feb4f069851d574c1d7a/ncut_pytorch-1.8.13.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-10 01:50:57",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "ncut-pytorch"
}
