<div align="center">
# Neural Network Compression Framework (NNCF)
[Key Features](#key-features) •
[Installation](#installation-guide) •
[Documentation](#documentation) •
[Usage](#usage) •
[Tutorials and Samples](#demos-tutorials-and-samples) •
[Third-party integration](#third-party-repository-integration) •
[Model Zoo](./docs/ModelZoo.md)
[](https://github.com/openvinotoolkit/nncf/releases)
[](https://docs.openvino.ai/latest/openvino_docs_model_optimization_guide.html)
[](LICENSE)
[](https://pypi.org/project/nncf/)
</div>
Neural Network Compression Framework (NNCF) provides a suite of post-training and training-time algorithms for neural networks inference optimization in [OpenVINO™](https://docs.openvino.ai) with minimal accuracy drop.
NNCF is designed to work with models from [PyTorch](https://pytorch.org/), [TensorFlow](https://www.tensorflow.org/), [ONNX](https://onnx.ai/) and [OpenVINO™](https://docs.openvino.ai/latest/home.html).
NNCF provides [samples](#demos-tutorials-and-samples) that demonstrate the usage of compression algorithms for different use cases and models. See compression results achievable with the NNCF-powered samples at [Model Zoo page](./docs/ModelZoo.md).
The framework is organized as a Python\* package that can be built and used in a standalone mode. The framework
architecture is unified to make it easy to add different compression algorithms for both PyTorch and TensorFlow deep
learning frameworks.
## Key Features
### Post-Training Compression Algorithms
| Compression algorithm |OpenVINO|PyTorch| TensorFlow | ONNX |
|:----------------------------------------------------------------------------| :---: | :---: |:--------:|:------------------:|
| [Post-Training Quantization](./docs/compression_algorithms/post_training/Quantization.md) | Supported | Supported |Supported| Supported |
| [Weights Compression](./docs/compression_algorithms/CompressWeights.md) | Supported | Supported |Not supported| Not supported |
### Training-Time Compression Algorithms
|Compression algorithm|PyTorch|TensorFlow|
| :--- | :---: | :---: |
|[Quantization Aware Training](./docs/compression_algorithms/Quantization.md) | Supported | Supported |
|[Mixed-Precision Quantization](./docs/compression_algorithms/Quantization.md#mixed-precision-quantization) | Supported | Not supported |
|[Sparsity](./docs/compression_algorithms/Sparsity.md) | Supported | Supported |
|[Filter pruning](./docs/compression_algorithms/Pruning.md) | Supported | Supported |
|[Movement pruning](./nncf/experimental/torch/sparsity/movement/MovementSparsity.md) | Experimental | Not supported |
- Automatic, configurable model graph transformation to obtain the compressed model.
> **NOTE**: Limited support for TensorFlow models. The models created using Sequential or Keras Functional API are only supported.
- Common interface for compression methods.
- GPU-accelerated layers for faster compressed model fine-tuning.
- Distributed training support.
- Git patch for prominent third-party repository ([huggingface-transformers](https://github.com/huggingface/transformers)) demonstrating the process of integrating NNCF into custom training pipelines
- Seamless combination of pruning, sparsity and quantization algorithms. Please refer to [optimum-intel](https://github.com/huggingface/optimum-intel/tree/main/examples/openvino) for examples of
joint (movement) pruning, quantization and distillation (JPQD), end-to-end from NNCF optimization to compressed OpenVINO IR.
- Exporting PyTorch compressed models to ONNX\* checkpoints and TensorFlow compressed models to SavedModel or Frozen Graph format, ready to use with [OpenVINO™ toolkit](https://docs.openvino.ai).
- Support for [Accuracy-Aware model training](./docs/Usage.md#accuracy-aware-model-training) pipelines via the [Adaptive Compression Level Training](./docs/accuracy_aware_model_training/AdaptiveCompressionLevelTraining.md) and [Early Exit Training](./docs/accuracy_aware_model_training/EarlyExitTraining.md).
## Documentation
This documentation covers detailed information about NNCF algorithms and functions needed for the contribution to NNCF.
The latest user documentation for NNCF is available [here](https://docs.openvino.ai/latest/openvino_docs_model_optimization_guide.html).
NNCF API documentation can be found [here](https://openvinotoolkit.github.io/nncf/autoapi/nncf/).
## Usage
### Post-Training Quantization
The NNCF PTQ is the simplest way to apply 8-bit quantization. To run the algorithm you only need your model and a small (~300 samples) calibration dataset.
[OpenVINO](https://github.com/openvinotoolkit/openvino) is the preferred backend to run PTQ with, and PyTorch, TensorFlow and ONNX are also supported.
<details open><summary><b>OpenVINO</b></summary>
```python
import nncf
import openvino.runtime as ov
import torch
from torchvision import datasets, transforms
# Instantiate your uncompressed model
model = ov.Core().read_model("/model_path")
# Provide validation part of the dataset to collect statistics needed for the compression algorithm
val_dataset = datasets.ImageFolder("/path", transform=transforms.Compose([transforms.ToTensor()]))
dataset_loader = torch.utils.data.DataLoader(val_dataset, batch_size=1)
# Step 1: Initialize transformation function
def transform_fn(data_item):
images, _ = data_item
return images
# Step 2: Initialize NNCF Dataset
calibration_dataset = nncf.Dataset(dataset_loader, transform_fn)
# Step 3: Run the quantization pipeline
quantized_model = nncf.quantize(model, calibration_dataset)
```
</details>
<details><summary><b>PyTorch</b></summary>
```python
import nncf
import torch
from torchvision import datasets, models
# Instantiate your uncompressed model
model = models.mobilenet_v2()
# Provide validation part of the dataset to collect statistics needed for the compression algorithm
val_dataset = datasets.ImageFolder("/path", transform=transforms.Compose([transforms.ToTensor()]))
dataset_loader = torch.utils.data.DataLoader(val_dataset)
# Step 1: Initialize the transformation function
def transform_fn(data_item):
images, _ = data_item
return images
# Step 2: Initialize NNCF Dataset
calibration_dataset = nncf.Dataset(dataset_loader, transform_fn)
# Step 3: Run the quantization pipeline
quantized_model = nncf.quantize(model, calibration_dataset)
```
**NOTE** In case the Post-Training Quantization algorithm could not reach quality requirements you can fine-tune the quantized pytorch model. Example of the Quantization-Aware training pipeline for a pytorch model could be found [here](examples/quantization_aware_training/torch/resnet18/README.md).
</details>
<details><summary><b>TensorFlow</b></summary>
```python
import nncf
import tensorflow as tf
import tensorflow_datasets as tfds
# Instantiate your uncompressed model
model = tf.keras.applications.MobileNetV2()
# Provide validation part of the dataset to collect statistics needed for the compression algorithm
val_dataset = tfds.load("/path", split="validation",
shuffle_files=False, as_supervised=True)
# Step 1: Initialize transformation function
def transform_fn(data_item):
images, _ = data_item
return images
# Step 2: Initialize NNCF Dataset
calibration_dataset = nncf.Dataset(val_dataset, transform_fn)
# Step 3: Run the quantization pipeline
quantized_model = nncf.quantize(model, calibration_dataset)
```
</details>
<details><summary><b>ONNX</b></summary>
```python
import onnx
import nncf
import torch
from torchvision import datasets
# Instantiate your uncompressed model
onnx_model = onnx.load_model("/model_path")
# Provide validation part of the dataset to collect statistics needed for the compression algorithm
val_dataset = datasets.ImageFolder("/path", transform=transforms.Compose([transforms.ToTensor()]))
dataset_loader = torch.utils.data.DataLoader(val_dataset, batch_size=1)
# Step 1: Initialize transformation function
input_name = onnx_model.graph.input[0].name
def transform_fn(data_item):
images, _ = data_item
return {input_name: images.numpy()}
# Step 2: Initialize NNCF Dataset
calibration_dataset = nncf.Dataset(dataset_loader, transform_fn)
# Step 3: Run the quantization pipeline
quantized_model = nncf.quantize(onnx_model, calibration_dataset)
```
</details>
[//]: # (NNCF provides full [samples](#post-training-quantization-samples), which demonstrate Post-Training Quantization usage for PyTorch, TensorFlow, ONNX, OpenVINO.)
### Training-Time Compression
Below is an example of Accuracy Aware Quantization pipeline where model weights and compression parameters may be fine-tuned to achieve a higher accuracy.
<details><summary><b>PyTorch</b></summary>
```python
import torch
import nncf.torch # Important - must be imported before any other external package that depends on torch
from nncf import NNCFConfig
from nncf.torch import create_compressed_model, register_default_init_args
# Instantiate your uncompressed model
from torchvision.models.resnet import resnet50
model = resnet50()
# Load a configuration file to specify compression
nncf_config = NNCFConfig.from_json("resnet50_int8.json")
# Provide data loaders for compression algorithm initialization, if necessary
import torchvision.datasets as datasets
representative_dataset = datasets.ImageFolder("/path", transform=transforms.Compose([transforms.ToTensor()]))
init_loader = torch.utils.data.DataLoader(representative_dataset)
nncf_config = register_default_init_args(nncf_config, init_loader)
# Apply the specified compression algorithms to the model
compression_ctrl, compressed_model = create_compressed_model(model, nncf_config)
# Now use compressed_model as a usual torch.nn.Module
# to fine-tune compression parameters along with the model weights
# ... the rest of the usual PyTorch-powered training pipeline
# Export to ONNX or .pth when done fine-tuning
compression_ctrl.export_model("compressed_model.onnx")
torch.save(compressed_model.state_dict(), "compressed_model.pth")
```
**NOTE (PyTorch)**: Due to the way NNCF works within the PyTorch backend, `import nncf` must be done before any other import of `torch` in your package _or_ in third-party packages that your code utilizes, otherwise the compression may be applied incompletely.
</details>
<details><summary><b>Tensorflow</b></summary>
```python
import tensorflow as tf
from nncf import NNCFConfig
from nncf.tensorflow import create_compressed_model, register_default_init_args
# Instantiate your uncompressed model
from tensorflow.keras.applications import ResNet50
model = ResNet50()
# Load a configuration file to specify compression
nncf_config = NNCFConfig.from_json("resnet50_int8.json")
# Provide dataset for compression algorithm initialization
representative_dataset = tf.data.Dataset.list_files("/path/*.jpeg")
nncf_config = register_default_init_args(nncf_config, representative_dataset, batch_size=1)
# Apply the specified compression algorithms to the model
compression_ctrl, compressed_model = create_compressed_model(model, nncf_config)
# Now use compressed_model as a usual Keras model
# to fine-tune compression parameters along with the model weights
# ... the rest of the usual TensorFlow-powered training pipeline
# Export to Frozen Graph, TensorFlow SavedModel or .h5 when done fine-tuning
compression_ctrl.export_model("compressed_model.pb", save_format="frozen_graph")
```
</details>
For a more detailed description of NNCF usage in your training code, see [this tutorial](docs/Usage.md).
## Demos, Tutorials and Samples
For a quicker start with NNCF-powered compression, try sample notebooks and scripts presented below.
### Jupyter* Notebook Tutorials and Demos
A collection of ready-to-run Jupyter* notebooks tutorials and demos are available to explain and display NNCF compression algorithms for optimizing models for inference with the OpenVINO Toolkit.
| Notebook Tutorial Name | Compression Algorithm | Backend | Domain |
|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------:|:----------:|:-----------------------------------:|
| [BERT Quantization](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/language-quantize-bert)<br>[](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/language-quantize-bert/language-quantize-bert.ipynb) | Post-Training Quantization | OpenVINO | NLP |
| [MONAI Segmentation Model Quantization](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/ct-segmentation-quantize)<br>[](https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?filepath=notebooks%2Fct-segmentation-quantize%2Fct-scan-live-inference.ipynb) | Post-Training Quantization | OpenVINO | Segmentation |
| [PyTorch Model Quantization](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/pytorch-post-training-quantization-nncf) | Post-Training Quantization | PyTorch | Image Classification |
| [TensorFlow Model Quantization](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/tensorflow-training-openvino) | Post-Training Quantization | Tensorflow | Image Classification |
| [Quantization with Accuracy Control](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/quantizing-model-with-accuracy-control) | Post-Training Quantization with Accuracy Control | OpenVINO | Speech-to-Text,<br>Object Detection |
| [PyTorch Training-Time Compression](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/pytorch-quantization-aware-training) | Training-Time Compression | PyTorch | Image Classification |
| [TensorFlow Training-Time Compression](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/tensorflow-training-openvino) | Training-Time Compression | Tensorflow | Image Classification |
| [Joint Pruning, Quantization and Distillation for BERT](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/sparsity-optimization) | Joint Pruning, Quantization and Distillation | OpenVINO | NLP |
Below is a list of notebooks demonstrating OpenVINO conversion and inference together with NNCF compression for models from various domains.
| Demo Model | Compression Algorithm | Backend | Domain |
|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------:|:---------:|:--------------------------------------------------------------------:|
| [YOLOv8](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/yolov8-optimization)<br>[](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/yolov8-optimization/yolov8-object-detection.ipynb) | Post-Training Quantization | OpenVINO | Object Detection,<br>KeyPoint Detection,<br>Instance Segmentation |
| [YOLOv7](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/yolov7-optimization) | Post-Training Quantization | OpenVINO | Object Detection |
| [EfficientSAM](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/efficient-sam) | Post-Training Quantization | OpenVINO | Image Segmentation |
| [Segment Anything Model](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/segment-anything) | Post-Training Quantization | OpenVINO | Image Segmentation |
| [OneFormer](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/oneformer-segmentation) | Post-Training Quantization | OpenVINO | Image Segmentation |
| [InstructPix2Pix](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/instruct-pix2pix-image-editing) | Post-Training Quantization | OpenVINO | Image-to-Image |
| [CLIP](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/clip-zero-shot-image-classification) | Post-Training Quantization | OpenVINO | Image-to-Text |
| [BLIP](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/blip-visual-language-processing) | Post-Training Quantization | OpenVINO | Image-to-Text |
| [Segmind-VegaRT](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/stable-diffusion-xl/segmind-vegart.ipynb) | Post-Training Quantization | OpenVINO | Text-to-Image |
| [Latent Consistency Model](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/latent-consistency-models-image-generation) | Post-Training Quantization | OpenVINO | Text-to-Image |
| [Würstchen](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/wuerstchen-image-generation) | Post-Training Quantization | OpenVINO | Text-to-Image |
| [ControlNet QR Code Monster](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/qrcode-monster) | Post-Training Quantization | OpenVINO | Text-to-Image |
| [SDXL-turbo](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/sdxl-turbo) | Post-Training Quantization | OpenVINO | Text-to-Image,<br>Image-to-Image |
| [DeepFloyd IF](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/deepfloyd-if) | Post-Training Quantization,<br>Weight Compression | OpenVINO | Text-to-Image,<br>Image-to-Image |
| [ImageBind](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/image-bind) | Post-Training Quantization | OpenVINO | Multi-Modal Retrieval |
| [Distil-Whisper](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/distil-whisper-asr) | Post-Training Quantization | OpenVINO | Speech-to-Text |
| [Whisper](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/whisper-subtitles-generation)<br>[](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/whisper-subtitles-generation/whisper-convert.ipynb) | Post-Training Quantization | OpenVINO | Speech-to-Text |
| [MMS Speech Recognition](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/mms-massively-multilingual-speech) | Post-Training Quantization | OpenVINO | Speech-to-Text |
| [Grammar Error Correction](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/grammar-correction) | Post-Training Quantization | OpenVINO | NLP, Grammar Correction |
| [LLM Instruction Following](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-question-answering) | Weight Compression | OpenVINO | NLP, Instruction Following |
| [Dolly 2.0](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/dolly-2-instruction-following) | Weight Compression | OpenVINO | NLP, Instruction Following |
| [Stable-Zephyr-3b](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/stable-zephyr-3b-chatbot) | Weight Compression | OpenVINO | NLP, Chat Bot |
| [LLM Chat Bots](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-chatbot) | Weight Compression | OpenVINO | NLP, Chat Bot |
### Post-Training Quantization Examples
Compact scripts demonstrating quantization and corresponding inference speed boost:
| Example Name | Compression Algorithm | Backend | Domain |
|:-----------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------:|:----------:|:----------------------:|
| [OpenVINO MobileNetV2](./examples/post_training_quantization/openvino/mobilenet_v2/README.md) | Post-Training Quantization | OpenVINO | Image Classification |
| [OpenVINO YOLOv8](./examples/post_training_quantization/openvino/yolov8/README.md) | Post-Training Quantization | OpenVINO | Object Detection |
| [OpenVINO YOLOv8 QwAС](./examples/post_training_quantization/openvino/yolov8_quantize_with_accuracy_control/README.md) | Post-Training Quantization with Accuracy Control | OpenVINO | Object Detection |
| [OpenVINO Anomaly Classification](./examples/post_training_quantization/openvino/anomaly_stfpm_quantize_with_accuracy_control/README.md) | Post-Training Quantization with Accuracy Control | OpenVINO | Anomaly Classification |
| [PyTorch MobileNetV2](./examples/post_training_quantization/torch/mobilenet_v2/README.md) | Post-Training Quantization | PyTorch | Image Classification |
| [PyTorch SSD](./examples/post_training_quantization/torch/ssd300_vgg16/README.md) | Post-Training Quantization | PyTorch | Object Detection |
| [TensorFlow MobileNetV2](./examples/post_training_quantization/tensorflow/mobilenet_v2/README.md) | Post-Training Quantization | TensorFlow | Image Classification |
| [ONNX MobileNetV2](./examples/post_training_quantization/onnx/mobilenet_v2/README.md) | Post-Training Quantization | ONNX | Image Classification |
### Training-Time Compression Examples
These examples provide full pipelines including compression, training and inference for classification, detection and segmentation tasks.
| Example Name | Compression Algorithm | Backend | Domain |
|:-----------------------------------------------------------------------------------------------------------|:-------------------------:|:----------:|:---------------------:|
| [PyTorch Image Classification](./examples/torch/classification/README.md) | Training-Time Compression | PyTorch | Image Classification |
| [PyTorch Object Detection](./examples/torch/object_detection/README.md) | Training-Time Compression | PyTorch | Object Detection |
| [PyTorch Semantic Segmentation](./examples/torch/semantic_segmentation/README.md) | Training-Time Compression | PyTorch | Semantic Segmentation |
| [TensorFlow Image Classification](./examples/tensorflow/classification/README.md) | Training-Time Compression | TensorFlow | Image Classification |
| [TensorFlow Object Detection](./examples/tensorflow/object_detection/README.md) | Training-Time Compression | TensorFlow | Object Detection |
| [TensorFlow Instance Segmentation](./examples/tensorflow/segmentation/README.md) | Training-Time Compression | TensorFlow | Instance Segmentation |
## Third-party repository integration
NNCF may be straightforwardly integrated into training/evaluation pipelines of third-party repositories.
### Used by
- [OpenVINO Training Extensions](https://github.com/openvinotoolkit/training_extensions)
NNCF is integrated into OpenVINO Training Extensions as model optimization backend. So you can train, optimize and export new models based on the available model templates as well as run exported models with OpenVINO.
- [HuggingFace Optimum Intel](https://huggingface.co/docs/optimum/intel/optimization_ov)
NNCF is used as a compression backend within the renowned `transformers` repository in HuggingFace Optimum Intel.
## Installation Guide
For detailed installation instructions please refer to the [Installation](./docs/Installation.md) page.
NNCF can be installed as a regular PyPI package via pip:
```bash
pip install nncf
```
If you want to install both NNCF and the supported PyTorch version in one line, you can do this by simply running:
```bash
pip install nncf[torch]
```
Other viable options besides `[torch]` are `[tf]`, `[onnx]` and `[openvino]`.
> [!WARNING]
> The way to install the module package with the extra dependency like `pip install nncf[torch]` is deprecated and will be removed in a future release.
> Instead, it is recommended to install additional dependencies separately using the pip install command (e.g., `pip install torch`) or by explicitly specifying the dependency in your requirements file.
NNCF is also available via [conda](https://anaconda.org/conda-forge/nncf):
```bash
conda install -c conda-forge nncf
```
### System requirements
- Ubuntu\* 18.04 or later (64-bit)
- Python\* 3.8 or later
- Supported frameworks:
- PyTorch\* >=2.1, <2.3
- TensorFlow\* >=2.8.4, <=2.12.1
- ONNX\* ==1.16.0
- OpenVINO\* >=2022.3.0
This repository is tested on Python* 3.8.10, PyTorch* 2.2.1 (NVidia CUDA\* Toolkit 12.1) and TensorFlow* 2.12.1 (NVidia CUDA\* Toolkit 11.8).
## NNCF Compressed Model Zoo
List of models and compression results for them can be found at our [Model Zoo page](./docs/ModelZoo.md).
## Citing
```bi
@article{kozlov2020neural,
title = {Neural network compression framework for fast model inference},
author = {Kozlov, Alexander and Lazarevich, Ivan and Shamporov, Vasily and Lyalyushkin, Nikolay and Gorbachev, Yury},
journal = {arXiv preprint arXiv:2002.08679},
year = {2020}
}
```
## Contributing Guide
Refer to the [CONTRIBUTING.md](./CONTRIBUTING.md) file for guidelines on contributions to the NNCF repository.
## Useful links
- [Documentation](./docs)
- Example scripts (model objects available through links in respective README.md files):
- [PyTorch](./examples/torch)
- [TensorFlow](./examples/tensorflow)
- [FAQ](./docs/FAQ.md)
- [Notebooks](https://github.com/openvinotoolkit/openvino_notebooks#-model-training)
- [HuggingFace Optimum Intel](https://huggingface.co/docs/optimum/intel/optimization_ov)
- [OpenVINO Model Optimization Guide](https://docs.openvino.ai/latest/openvino_docs_model_optimization_guide.html)
## Telemetry
NNCF as part of the OpenVINO™ toolkit collects anonymous usage data for the purpose of improving OpenVINO™ tools.
You can opt-out at any time by running the following command in the Python environment where you have NNCF installed:
`opt_in_out --opt_out`
More information is available at https://docs.openvino.ai/latest/openvino_docs_telemetry_information.html.
Raw data
{
"_id": null,
"home_page": "https://github.com/openvinotoolkit/nncf",
"name": "nncf",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "compression, quantization, sparsity, mixed-precision-training, quantization-aware-training, hawq, classification, pruning, object-detection, semantic-segmentation, nas, nlp, bert, transformers, mmdetection",
"author": "Intel",
"author_email": "alexander.kozlov@intel.com",
"download_url": "https://files.pythonhosted.org/packages/44/94/75e3edf730f97aa2b42fa9f8e4a2aeda6823aabf783f6705eca1ee6a3008/nncf-2.10.0.tar.gz",
"platform": null,
"description": "<div align=\"center\">\n\n# Neural Network Compression Framework (NNCF)\n\n[Key Features](#key-features) \u2022\n[Installation](#installation-guide) \u2022\n[Documentation](#documentation) \u2022\n[Usage](#usage) \u2022\n[Tutorials and Samples](#demos-tutorials-and-samples) \u2022\n[Third-party integration](#third-party-repository-integration) \u2022\n[Model Zoo](./docs/ModelZoo.md)\n\n[](https://github.com/openvinotoolkit/nncf/releases)\n[](https://docs.openvino.ai/latest/openvino_docs_model_optimization_guide.html)\n[](LICENSE)\n[](https://pypi.org/project/nncf/)\n\n</div>\n\nNeural Network Compression Framework (NNCF) provides a suite of post-training and training-time algorithms for neural networks inference optimization in [OpenVINO™](https://docs.openvino.ai) with minimal accuracy drop.\n\nNNCF is designed to work with models from [PyTorch](https://pytorch.org/), [TensorFlow](https://www.tensorflow.org/), [ONNX](https://onnx.ai/) and [OpenVINO™](https://docs.openvino.ai/latest/home.html).\n\nNNCF provides [samples](#demos-tutorials-and-samples) that demonstrate the usage of compression algorithms for different use cases and models. See compression results achievable with the NNCF-powered samples at [Model Zoo page](./docs/ModelZoo.md).\n\nThe framework is organized as a Python\\* package that can be built and used in a standalone mode. The framework\narchitecture is unified to make it easy to add different compression algorithms for both PyTorch and TensorFlow deep\nlearning frameworks.\n\n## Key Features\n\n### Post-Training Compression Algorithms\n\n| Compression algorithm |OpenVINO|PyTorch| TensorFlow | ONNX |\n|:----------------------------------------------------------------------------| :---: | :---: |:--------:|:------------------:|\n| [Post-Training Quantization](./docs/compression_algorithms/post_training/Quantization.md) | Supported | Supported |Supported| Supported |\n| [Weights Compression](./docs/compression_algorithms/CompressWeights.md) | Supported | Supported |Not supported| Not supported |\n\n### Training-Time Compression Algorithms\n\n|Compression algorithm|PyTorch|TensorFlow|\n| :--- | :---: | :---: |\n|[Quantization Aware Training](./docs/compression_algorithms/Quantization.md) | Supported | Supported |\n|[Mixed-Precision Quantization](./docs/compression_algorithms/Quantization.md#mixed-precision-quantization) | Supported | Not supported |\n|[Sparsity](./docs/compression_algorithms/Sparsity.md) | Supported | Supported |\n|[Filter pruning](./docs/compression_algorithms/Pruning.md) | Supported | Supported |\n|[Movement pruning](./nncf/experimental/torch/sparsity/movement/MovementSparsity.md) | Experimental | Not supported |\n\n- Automatic, configurable model graph transformation to obtain the compressed model.\n > **NOTE**: Limited support for TensorFlow models. The models created using Sequential or Keras Functional API are only supported.\n- Common interface for compression methods.\n- GPU-accelerated layers for faster compressed model fine-tuning.\n- Distributed training support.\n- Git patch for prominent third-party repository ([huggingface-transformers](https://github.com/huggingface/transformers)) demonstrating the process of integrating NNCF into custom training pipelines\n- Seamless combination of pruning, sparsity and quantization algorithms. Please refer to [optimum-intel](https://github.com/huggingface/optimum-intel/tree/main/examples/openvino) for examples of\njoint (movement) pruning, quantization and distillation (JPQD), end-to-end from NNCF optimization to compressed OpenVINO IR.\n- Exporting PyTorch compressed models to ONNX\\* checkpoints and TensorFlow compressed models to SavedModel or Frozen Graph format, ready to use with [OpenVINO™ toolkit](https://docs.openvino.ai).\n- Support for [Accuracy-Aware model training](./docs/Usage.md#accuracy-aware-model-training) pipelines via the [Adaptive Compression Level Training](./docs/accuracy_aware_model_training/AdaptiveCompressionLevelTraining.md) and [Early Exit Training](./docs/accuracy_aware_model_training/EarlyExitTraining.md).\n\n## Documentation\n\nThis documentation covers detailed information about NNCF algorithms and functions needed for the contribution to NNCF.\n\nThe latest user documentation for NNCF is available [here](https://docs.openvino.ai/latest/openvino_docs_model_optimization_guide.html).\n\nNNCF API documentation can be found [here](https://openvinotoolkit.github.io/nncf/autoapi/nncf/).\n\n## Usage\n\n### Post-Training Quantization\n\nThe NNCF PTQ is the simplest way to apply 8-bit quantization. To run the algorithm you only need your model and a small (~300 samples) calibration dataset.\n\n[OpenVINO](https://github.com/openvinotoolkit/openvino) is the preferred backend to run PTQ with, and PyTorch, TensorFlow and ONNX are also supported.\n\n<details open><summary><b>OpenVINO</b></summary>\n\n```python\nimport nncf\nimport openvino.runtime as ov\nimport torch\nfrom torchvision import datasets, transforms\n\n# Instantiate your uncompressed model\nmodel = ov.Core().read_model(\"/model_path\")\n\n# Provide validation part of the dataset to collect statistics needed for the compression algorithm\nval_dataset = datasets.ImageFolder(\"/path\", transform=transforms.Compose([transforms.ToTensor()]))\ndataset_loader = torch.utils.data.DataLoader(val_dataset, batch_size=1)\n\n# Step 1: Initialize transformation function\ndef transform_fn(data_item):\n images, _ = data_item\n return images\n\n# Step 2: Initialize NNCF Dataset\ncalibration_dataset = nncf.Dataset(dataset_loader, transform_fn)\n# Step 3: Run the quantization pipeline\nquantized_model = nncf.quantize(model, calibration_dataset)\n```\n\n</details>\n\n<details><summary><b>PyTorch</b></summary>\n\n```python\nimport nncf\nimport torch\nfrom torchvision import datasets, models\n\n# Instantiate your uncompressed model\nmodel = models.mobilenet_v2()\n\n# Provide validation part of the dataset to collect statistics needed for the compression algorithm\nval_dataset = datasets.ImageFolder(\"/path\", transform=transforms.Compose([transforms.ToTensor()]))\ndataset_loader = torch.utils.data.DataLoader(val_dataset)\n\n# Step 1: Initialize the transformation function\ndef transform_fn(data_item):\n images, _ = data_item\n return images\n\n# Step 2: Initialize NNCF Dataset\ncalibration_dataset = nncf.Dataset(dataset_loader, transform_fn)\n# Step 3: Run the quantization pipeline\nquantized_model = nncf.quantize(model, calibration_dataset)\n\n```\n\n**NOTE** In case the Post-Training Quantization algorithm could not reach quality requirements you can fine-tune the quantized pytorch model. Example of the Quantization-Aware training pipeline for a pytorch model could be found [here](examples/quantization_aware_training/torch/resnet18/README.md).\n\n</details>\n\n<details><summary><b>TensorFlow</b></summary>\n\n```python\nimport nncf\nimport tensorflow as tf\nimport tensorflow_datasets as tfds\n\n# Instantiate your uncompressed model\nmodel = tf.keras.applications.MobileNetV2()\n\n# Provide validation part of the dataset to collect statistics needed for the compression algorithm\nval_dataset = tfds.load(\"/path\", split=\"validation\",\n shuffle_files=False, as_supervised=True)\n\n# Step 1: Initialize transformation function\ndef transform_fn(data_item):\n images, _ = data_item\n return images\n\n# Step 2: Initialize NNCF Dataset\ncalibration_dataset = nncf.Dataset(val_dataset, transform_fn)\n# Step 3: Run the quantization pipeline\nquantized_model = nncf.quantize(model, calibration_dataset)\n```\n\n</details>\n\n<details><summary><b>ONNX</b></summary>\n\n```python\nimport onnx\nimport nncf\nimport torch\nfrom torchvision import datasets\n\n# Instantiate your uncompressed model\nonnx_model = onnx.load_model(\"/model_path\")\n\n# Provide validation part of the dataset to collect statistics needed for the compression algorithm\nval_dataset = datasets.ImageFolder(\"/path\", transform=transforms.Compose([transforms.ToTensor()]))\ndataset_loader = torch.utils.data.DataLoader(val_dataset, batch_size=1)\n\n# Step 1: Initialize transformation function\ninput_name = onnx_model.graph.input[0].name\ndef transform_fn(data_item):\n images, _ = data_item\n return {input_name: images.numpy()}\n\n# Step 2: Initialize NNCF Dataset\ncalibration_dataset = nncf.Dataset(dataset_loader, transform_fn)\n# Step 3: Run the quantization pipeline\nquantized_model = nncf.quantize(onnx_model, calibration_dataset)\n```\n\n</details>\n\n[//]: # (NNCF provides full [samples](#post-training-quantization-samples), which demonstrate Post-Training Quantization usage for PyTorch, TensorFlow, ONNX, OpenVINO.)\n\n### Training-Time Compression\n\nBelow is an example of Accuracy Aware Quantization pipeline where model weights and compression parameters may be fine-tuned to achieve a higher accuracy.\n\n<details><summary><b>PyTorch</b></summary>\n\n```python\nimport torch\nimport nncf.torch # Important - must be imported before any other external package that depends on torch\n\nfrom nncf import NNCFConfig\nfrom nncf.torch import create_compressed_model, register_default_init_args\n\n# Instantiate your uncompressed model\nfrom torchvision.models.resnet import resnet50\nmodel = resnet50()\n\n# Load a configuration file to specify compression\nnncf_config = NNCFConfig.from_json(\"resnet50_int8.json\")\n\n# Provide data loaders for compression algorithm initialization, if necessary\nimport torchvision.datasets as datasets\nrepresentative_dataset = datasets.ImageFolder(\"/path\", transform=transforms.Compose([transforms.ToTensor()]))\ninit_loader = torch.utils.data.DataLoader(representative_dataset)\nnncf_config = register_default_init_args(nncf_config, init_loader)\n\n# Apply the specified compression algorithms to the model\ncompression_ctrl, compressed_model = create_compressed_model(model, nncf_config)\n\n# Now use compressed_model as a usual torch.nn.Module\n# to fine-tune compression parameters along with the model weights\n\n# ... the rest of the usual PyTorch-powered training pipeline\n\n# Export to ONNX or .pth when done fine-tuning\ncompression_ctrl.export_model(\"compressed_model.onnx\")\ntorch.save(compressed_model.state_dict(), \"compressed_model.pth\")\n```\n\n**NOTE (PyTorch)**: Due to the way NNCF works within the PyTorch backend, `import nncf` must be done before any other import of `torch` in your package _or_ in third-party packages that your code utilizes, otherwise the compression may be applied incompletely.\n\n</details>\n\n<details><summary><b>Tensorflow</b></summary>\n\n```python\nimport tensorflow as tf\n\nfrom nncf import NNCFConfig\nfrom nncf.tensorflow import create_compressed_model, register_default_init_args\n\n# Instantiate your uncompressed model\nfrom tensorflow.keras.applications import ResNet50\nmodel = ResNet50()\n\n# Load a configuration file to specify compression\nnncf_config = NNCFConfig.from_json(\"resnet50_int8.json\")\n\n# Provide dataset for compression algorithm initialization\nrepresentative_dataset = tf.data.Dataset.list_files(\"/path/*.jpeg\")\nnncf_config = register_default_init_args(nncf_config, representative_dataset, batch_size=1)\n\n# Apply the specified compression algorithms to the model\ncompression_ctrl, compressed_model = create_compressed_model(model, nncf_config)\n\n# Now use compressed_model as a usual Keras model\n# to fine-tune compression parameters along with the model weights\n\n# ... the rest of the usual TensorFlow-powered training pipeline\n\n# Export to Frozen Graph, TensorFlow SavedModel or .h5 when done fine-tuning\ncompression_ctrl.export_model(\"compressed_model.pb\", save_format=\"frozen_graph\")\n```\n\n</details>\n\nFor a more detailed description of NNCF usage in your training code, see [this tutorial](docs/Usage.md).\n\n## Demos, Tutorials and Samples\n\nFor a quicker start with NNCF-powered compression, try sample notebooks and scripts presented below.\n\n### Jupyter* Notebook Tutorials and Demos\n\nA collection of ready-to-run Jupyter* notebooks tutorials and demos are available to explain and display NNCF compression algorithms for optimizing models for inference with the OpenVINO Toolkit.\n\n| Notebook Tutorial Name | Compression Algorithm | Backend | Domain |\n|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------:|:----------:|:-----------------------------------:|\n| [BERT Quantization](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/language-quantize-bert)<br>[](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/language-quantize-bert/language-quantize-bert.ipynb) | Post-Training Quantization | OpenVINO | NLP |\n| [MONAI Segmentation Model Quantization](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/ct-segmentation-quantize)<br>[](https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?filepath=notebooks%2Fct-segmentation-quantize%2Fct-scan-live-inference.ipynb) | Post-Training Quantization | OpenVINO | Segmentation |\n| [PyTorch Model Quantization](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/pytorch-post-training-quantization-nncf) | Post-Training Quantization | PyTorch | Image Classification |\n| [TensorFlow Model Quantization](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/tensorflow-training-openvino) | Post-Training Quantization | Tensorflow | Image Classification |\n| [Quantization with Accuracy Control](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/quantizing-model-with-accuracy-control) | Post-Training Quantization with Accuracy Control | OpenVINO | Speech-to-Text,<br>Object Detection |\n| [PyTorch Training-Time Compression](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/pytorch-quantization-aware-training) | Training-Time Compression | PyTorch | Image Classification |\n| [TensorFlow Training-Time Compression](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/tensorflow-training-openvino) | Training-Time Compression | Tensorflow | Image Classification |\n| [Joint Pruning, Quantization and Distillation for BERT](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/sparsity-optimization) | Joint Pruning, Quantization and Distillation | OpenVINO | NLP |\n\nBelow is a list of notebooks demonstrating OpenVINO conversion and inference together with NNCF compression for models from various domains.\n\n| Demo Model | Compression Algorithm | Backend | Domain |\n|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------:|:---------:|:--------------------------------------------------------------------:|\n| [YOLOv8](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/yolov8-optimization)<br>[](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/yolov8-optimization/yolov8-object-detection.ipynb) | Post-Training Quantization | OpenVINO | Object Detection,<br>KeyPoint Detection,<br>Instance Segmentation |\n| [YOLOv7](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/yolov7-optimization) | Post-Training Quantization | OpenVINO | Object Detection |\n| [EfficientSAM](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/efficient-sam) | Post-Training Quantization | OpenVINO | Image Segmentation |\n| [Segment Anything Model](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/segment-anything) | Post-Training Quantization | OpenVINO | Image Segmentation |\n| [OneFormer](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/oneformer-segmentation) | Post-Training Quantization | OpenVINO | Image Segmentation |\n| [InstructPix2Pix](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/instruct-pix2pix-image-editing) | Post-Training Quantization | OpenVINO | Image-to-Image |\n| [CLIP](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/clip-zero-shot-image-classification) | Post-Training Quantization | OpenVINO | Image-to-Text |\n| [BLIP](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/blip-visual-language-processing) | Post-Training Quantization | OpenVINO | Image-to-Text |\n| [Segmind-VegaRT](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/stable-diffusion-xl/segmind-vegart.ipynb) | Post-Training Quantization | OpenVINO | Text-to-Image |\n| [Latent Consistency Model](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/latent-consistency-models-image-generation) | Post-Training Quantization | OpenVINO | Text-to-Image |\n| [W\u00fcrstchen](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/wuerstchen-image-generation) | Post-Training Quantization | OpenVINO | Text-to-Image |\n| [ControlNet QR Code Monster](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/qrcode-monster) | Post-Training Quantization | OpenVINO | Text-to-Image |\n| [SDXL-turbo](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/sdxl-turbo) | Post-Training Quantization | OpenVINO | Text-to-Image,<br>Image-to-Image |\n| [DeepFloyd IF](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/deepfloyd-if) | Post-Training Quantization,<br>Weight Compression | OpenVINO | Text-to-Image,<br>Image-to-Image |\n| [ImageBind](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/image-bind) | Post-Training Quantization | OpenVINO | Multi-Modal Retrieval |\n| [Distil-Whisper](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/distil-whisper-asr) | Post-Training Quantization | OpenVINO | Speech-to-Text |\n| [Whisper](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/whisper-subtitles-generation)<br>[](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/whisper-subtitles-generation/whisper-convert.ipynb) | Post-Training Quantization | OpenVINO | Speech-to-Text |\n| [MMS Speech Recognition](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/mms-massively-multilingual-speech) | Post-Training Quantization | OpenVINO | Speech-to-Text |\n| [Grammar Error Correction](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/grammar-correction) | Post-Training Quantization | OpenVINO | NLP, Grammar Correction |\n| [LLM Instruction Following](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-question-answering) | Weight Compression | OpenVINO | NLP, Instruction Following |\n| [Dolly 2.0](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/dolly-2-instruction-following) | Weight Compression | OpenVINO | NLP, Instruction Following |\n| [Stable-Zephyr-3b](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/stable-zephyr-3b-chatbot) | Weight Compression | OpenVINO | NLP, Chat Bot |\n| [LLM Chat Bots](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-chatbot) | Weight Compression | OpenVINO | NLP, Chat Bot |\n\n### Post-Training Quantization Examples\n\nCompact scripts demonstrating quantization and corresponding inference speed boost:\n\n| Example Name | Compression Algorithm | Backend | Domain |\n|:-----------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------:|:----------:|:----------------------:|\n| [OpenVINO MobileNetV2](./examples/post_training_quantization/openvino/mobilenet_v2/README.md) | Post-Training Quantization | OpenVINO | Image Classification |\n| [OpenVINO YOLOv8](./examples/post_training_quantization/openvino/yolov8/README.md) | Post-Training Quantization | OpenVINO | Object Detection |\n| [OpenVINO YOLOv8 QwA\u0421](./examples/post_training_quantization/openvino/yolov8_quantize_with_accuracy_control/README.md) | Post-Training Quantization with Accuracy Control | OpenVINO | Object Detection |\n| [OpenVINO Anomaly Classification](./examples/post_training_quantization/openvino/anomaly_stfpm_quantize_with_accuracy_control/README.md) | Post-Training Quantization with Accuracy Control | OpenVINO | Anomaly Classification |\n| [PyTorch MobileNetV2](./examples/post_training_quantization/torch/mobilenet_v2/README.md) | Post-Training Quantization | PyTorch | Image Classification |\n| [PyTorch SSD](./examples/post_training_quantization/torch/ssd300_vgg16/README.md) | Post-Training Quantization | PyTorch | Object Detection |\n| [TensorFlow MobileNetV2](./examples/post_training_quantization/tensorflow/mobilenet_v2/README.md) | Post-Training Quantization | TensorFlow | Image Classification |\n| [ONNX MobileNetV2](./examples/post_training_quantization/onnx/mobilenet_v2/README.md) | Post-Training Quantization | ONNX | Image Classification |\n\n### Training-Time Compression Examples\n\nThese examples provide full pipelines including compression, training and inference for classification, detection and segmentation tasks.\n\n| Example Name | Compression Algorithm | Backend | Domain |\n|:-----------------------------------------------------------------------------------------------------------|:-------------------------:|:----------:|:---------------------:|\n| [PyTorch Image Classification](./examples/torch/classification/README.md) | Training-Time Compression | PyTorch | Image Classification |\n| [PyTorch Object Detection](./examples/torch/object_detection/README.md) | Training-Time Compression | PyTorch | Object Detection |\n| [PyTorch Semantic Segmentation](./examples/torch/semantic_segmentation/README.md) | Training-Time Compression | PyTorch | Semantic Segmentation |\n| [TensorFlow Image Classification](./examples/tensorflow/classification/README.md) | Training-Time Compression | TensorFlow | Image Classification |\n| [TensorFlow Object Detection](./examples/tensorflow/object_detection/README.md) | Training-Time Compression | TensorFlow | Object Detection |\n| [TensorFlow Instance Segmentation](./examples/tensorflow/segmentation/README.md) | Training-Time Compression | TensorFlow | Instance Segmentation |\n\n## Third-party repository integration\n\nNNCF may be straightforwardly integrated into training/evaluation pipelines of third-party repositories.\n\n### Used by\n\n- [OpenVINO Training Extensions](https://github.com/openvinotoolkit/training_extensions)\n\n NNCF is integrated into OpenVINO Training Extensions as model optimization backend. So you can train, optimize and export new models based on the available model templates as well as run exported models with OpenVINO.\n\n- [HuggingFace Optimum Intel](https://huggingface.co/docs/optimum/intel/optimization_ov)\n\n NNCF is used as a compression backend within the renowned `transformers` repository in HuggingFace Optimum Intel.\n\n## Installation Guide\n\nFor detailed installation instructions please refer to the [Installation](./docs/Installation.md) page.\n\nNNCF can be installed as a regular PyPI package via pip:\n\n```bash\npip install nncf\n```\n\nIf you want to install both NNCF and the supported PyTorch version in one line, you can do this by simply running:\n\n```bash\npip install nncf[torch]\n```\n\nOther viable options besides `[torch]` are `[tf]`, `[onnx]` and `[openvino]`.\n\n> [!WARNING]\n> The way to install the module package with the extra dependency like `pip install nncf[torch]` is deprecated and will be removed in a future release.\n> Instead, it is recommended to install additional dependencies separately using the pip install command (e.g., `pip install torch`) or by explicitly specifying the dependency in your requirements file.\n\nNNCF is also available via [conda](https://anaconda.org/conda-forge/nncf):\n\n```bash\nconda install -c conda-forge nncf\n```\n\n### System requirements\n\n- Ubuntu\\* 18.04 or later (64-bit)\n- Python\\* 3.8 or later\n- Supported frameworks:\n - PyTorch\\* >=2.1, <2.3\n - TensorFlow\\* >=2.8.4, <=2.12.1\n - ONNX\\* ==1.16.0\n - OpenVINO\\* >=2022.3.0\n\nThis repository is tested on Python* 3.8.10, PyTorch* 2.2.1 (NVidia CUDA\\* Toolkit 12.1) and TensorFlow* 2.12.1 (NVidia CUDA\\* Toolkit 11.8).\n\n## NNCF Compressed Model Zoo\n\nList of models and compression results for them can be found at our [Model Zoo page](./docs/ModelZoo.md).\n\n## Citing\n\n```bi\n@article{kozlov2020neural,\n title = {Neural network compression framework for fast model inference},\n author = {Kozlov, Alexander and Lazarevich, Ivan and Shamporov, Vasily and Lyalyushkin, Nikolay and Gorbachev, Yury},\n journal = {arXiv preprint arXiv:2002.08679},\n year = {2020}\n}\n```\n\n## Contributing Guide\n\nRefer to the [CONTRIBUTING.md](./CONTRIBUTING.md) file for guidelines on contributions to the NNCF repository.\n\n## Useful links\n\n- [Documentation](./docs)\n- Example scripts (model objects available through links in respective README.md files):\n - [PyTorch](./examples/torch)\n - [TensorFlow](./examples/tensorflow)\n- [FAQ](./docs/FAQ.md)\n- [Notebooks](https://github.com/openvinotoolkit/openvino_notebooks#-model-training)\n- [HuggingFace Optimum Intel](https://huggingface.co/docs/optimum/intel/optimization_ov)\n- [OpenVINO Model Optimization Guide](https://docs.openvino.ai/latest/openvino_docs_model_optimization_guide.html)\n\n## Telemetry\n\nNNCF as part of the OpenVINO\u2122 toolkit collects anonymous usage data for the purpose of improving OpenVINO\u2122 tools.\nYou can opt-out at any time by running the following command in the Python environment where you have NNCF installed:\n\n`opt_in_out --opt_out`\n\nMore information is available at https://docs.openvino.ai/latest/openvino_docs_telemetry_information.html.\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Neural Networks Compression Framework",
"version": "2.10.0",
"project_urls": {
"Homepage": "https://github.com/openvinotoolkit/nncf"
},
"split_keywords": [
"compression",
" quantization",
" sparsity",
" mixed-precision-training",
" quantization-aware-training",
" hawq",
" classification",
" pruning",
" object-detection",
" semantic-segmentation",
" nas",
" nlp",
" bert",
" transformers",
" mmdetection"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d02722dd19237848a6ff498b98189ce08fee7c393a46ec42a0d675fdfe6a69b3",
"md5": "96ec64262225f158e79bfce620145c7c",
"sha256": "c621727330543908f2c024d858d5d0383a3586bf3311b20cb9ce5d2c91190634"
},
"downloads": -1,
"filename": "nncf-2.10.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "96ec64262225f158e79bfce620145c7c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 1208076,
"upload_time": "2024-04-25T12:01:51",
"upload_time_iso_8601": "2024-04-25T12:01:51.250921Z",
"url": "https://files.pythonhosted.org/packages/d0/27/22dd19237848a6ff498b98189ce08fee7c393a46ec42a0d675fdfe6a69b3/nncf-2.10.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "449475e3edf730f97aa2b42fa9f8e4a2aeda6823aabf783f6705eca1ee6a3008",
"md5": "a980e8e6147070aa2bcc4277c58878a3",
"sha256": "5187a4d23d707a888de374774b3613456771d16ac38bcba102e3f98c7a3df1bf"
},
"downloads": -1,
"filename": "nncf-2.10.0.tar.gz",
"has_sig": false,
"md5_digest": "a980e8e6147070aa2bcc4277c58878a3",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 828198,
"upload_time": "2024-04-25T12:01:53",
"upload_time_iso_8601": "2024-04-25T12:01:53.611577Z",
"url": "https://files.pythonhosted.org/packages/44/94/75e3edf730f97aa2b42fa9f8e4a2aeda6823aabf783f6705eca1ee6a3008/nncf-2.10.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-25 12:01:53",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "openvinotoolkit",
"github_project": "nncf",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "nncf"
}
JSON
