[](https://github.com/openvinotoolkit/nncf/releases)
[](https://docs.openvino.ai/nncf)
[](https://github.com/openvinotoolkit/nncf?tab=Apache-2.0-1-ov-file#readme)
[](https://pypi.org/project/nncf/)
# Neural Network Compression Framework (NNCF)
Neural Network Compression Framework (NNCF) provides a suite of post-training
and training-time algorithms for optimizing inference of neural networks in
[OpenVINO™](https://docs.openvino.ai) with a minimal accuracy drop.
NNCF is designed to work with models from [PyTorch](https://pytorch.org/),
[TensorFlow](https://www.tensorflow.org/), [ONNX](https://onnx.ai/) and
[OpenVINO™](https://docs.openvino.ai).
The framework is organized as a Python package that can be built and used
as a standalone tool. Its architecture is unified to make adding different
compression algorithms easy for both PyTorch and TensorFlow.
NNCF provides samples that demonstrate the usage of compression algorithms
for different use cases and models. See compression results achievable
with the NNCF-powered samples on the
[NNCF Model Zoo page](https://github.com/openvinotoolkit/nncf/blob/develop/docs/ModelZoo.md).
For more information about NNCF, see:
- [NNCF repository](https://github.com/openvinotoolkit/nncf)
- [User documentation](https://docs.openvino.ai/nncf)
- [NNCF API documentation](https://openvinotoolkit.github.io/nncf/autoapi/nncf/)
- [Usage examples](https://github.com/openvinotoolkit/nncf/tree/develop/docs/usage)
- [Notebook tutorials](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/README.md#model-training)
## Table of contents
- [Key Features](#key-features)
- [Installation](#installation-guide)
- [Third-party integration](#third-party-repository-integration)
- [NNCF Compressed Model Zoo](#nncf-compressed-model-zoo)
## Key Features<a id="key-features"></a>
### Post-Training Compression Algorithms
| Compression algorithm | OpenVINO | PyTorch | TensorFlow | ONNX |
| :---------------------------------------------------------------------------------------------------------------------------------------------------------- | :-------: | :-------: | :-----------: | :-----------: |
| [Post-Training Quantization](https://github.com/openvinotoolkit/nncf/blob/develop/docs/usage/post_training_compression/post_training_quantization/Usage.md) | Supported | Supported | Supported | Supported |
| [Weight Compression](https://github.com/openvinotoolkit/nncf/blob/develop/docs/usage/post_training_compression/weights_compression/Usage.md) | Supported | Supported | Not supported | Not supported |
| [Activation Sparsity](https://github.com/openvinotoolkit/nncf/blob/develop/nncf/experimental/torch/sparsify_activations/ActivationSparsity.md) | Not supported | Experimental |Not supported| Not supported |
### Training-Time Compression Algorithms
| Compression algorithm | PyTorch | TensorFlow |
| :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :----------: | :-----------: |
| [Quantization Aware Training](https://github.com/openvinotoolkit/nncf/blob/develop/docs/usage/training_time_compression/quantization_aware_training/Usage.md) | Supported | Supported |
| [Mixed-Precision Quantization](https://github.com/openvinotoolkit/nncf/blob/develop/docs/usage/training_time_compression/other_algorithms/LegacyQuantization.md#mixed-precision-quantization) | Supported | Not supported |
| [Sparsity](https://github.com/openvinotoolkit/nncf/blob/develop/docs/usage/training_time_compression/other_algorithms/Sparsity.md) | Supported | Supported |
| [Filter pruning](https://github.com/openvinotoolkit/nncf/blob/develop/docs/usage/training_time_compression/other_algorithms/Pruning.md) | Supported | Supported |
| [Movement pruning](https://github.com/openvinotoolkit/nncf/blob/develop/nncf/experimental/torch/sparsity/movement/MovementSparsity.md) | Experimental | Not supported |
- Automatic, configurable model graph transformation to obtain the compressed
model.
> **NOTE**: Limited support for TensorFlow models. Only models created using
Sequential or Keras Functional API are supported.
- Common interface for compression methods.
- GPU-accelerated layers for faster compressed model fine-tuning.
- Distributed training support.
- Git patch for prominent third-party repository
([huggingface-transformers](https://github.com/huggingface/transformers))
demonstrating the process of integrating NNCF into custom training pipelines.
- Seamless combination of pruning, sparsity, and quantization algorithms. Refer
to [optimum-intel](https://github.com/huggingface/optimum-intel/tree/main/examples/openvino)
for examples of joint (movement) pruning, quantization, and distillation
(JPQD), end-to-end from NNCF optimization to compressed OpenVINO IR.
- Exporting PyTorch compressed models to ONNX\* checkpoints and TensorFlow
compressed models to SavedModel or Frozen Graph format, ready to use with
[OpenVINO™ toolkit](https://docs.openvino.ai).
- Support for [Accuracy-Aware model training](https://github.com/openvinotoolkit/nncf/blob/develop/docs/usage/training_time_compression/other_algorithms/Usage.md#accuracy-aware-model-training)
pipelines via the [Adaptive Compression Level Training](https://github.com/openvinotoolkit/nncf/blob/develop/docs/accuracy_aware_model_training/AdaptiveCompressionLevelTraining.md)
and [Early Exit Training](https://github.com/openvinotoolkit/nncf/blob/develop/docs/accuracy_aware_model_training/EarlyExitTraining.md).
## Installation Guide<a id="installation-guide"></a>
NNCF can be installed as a regular PyPI package:
```bash
pip install nncf
```
For detailed installation instructions, refer to the
[Installation](https://github.com/openvinotoolkit/nncf/blob/develop/docs/Installation.md) guide.
### System Requirements
- Ubuntu 18.04 or later (64-bit)
- Python 3.9 or later
- Supported frameworks:
- PyTorch >=2.2, <2.5
- TensorFlow >=2.8.4, <=2.15.1
- ONNX ==1.16.0
- OpenVINO >=2022.3.0
## Third-party Repository Integration<a id="third-party-repository-integration"></a>
NNCF may be easily integrated into training/evaluation pipelines of third-party
repositories.
- [OpenVINO Training Extensions](https://github.com/openvinotoolkit/training_extensions)
NNCF is integrated into OpenVINO Training Extensions as a model optimization
backend. You can train, optimize, and export new models based on available
model templates as well as run the exported models with OpenVINO.
- [HuggingFace Optimum Intel](https://huggingface.co/docs/optimum/intel/optimization_ov)
NNCF is used as a compression backend within the renowned `transformers`
repository in HuggingFace Optimum Intel.
## NNCF Compressed Model Zoo<a id="nncf-compressed-model-zoo"></a>
A list of models and compression results for them can be found at our
[NNCF Model Zoo page](https://github.com/openvinotoolkit/nncf/blob/develop/docs/ModelZoo.md).
## Citing
```bi
@article{kozlov2020neural,
title = {Neural network compression framework for fast model inference},
author = {Kozlov, Alexander and Lazarevich, Ivan and Shamporov, Vasily and Lyalyushkin, Nikolay and Gorbachev, Yury},
journal = {arXiv preprint arXiv:2002.08679},
year = {2020}
}
```
## Telemetry
NNCF as part of the OpenVINO™ toolkit collects anonymous usage data for the
purpose of improving OpenVINO™ tools. You can opt-out at any time by running
the following command in the Python environment where you have NNCF installed:
`opt_in_out --opt_out`
More information available on [OpenVINO telemetry](https://docs.openvino.ai/nightly/about-openvino/additional-resources/telemetry.html).
Raw data
{
"_id": null,
"home_page": null,
"name": "nncf",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "bert, classification, compression, hawq, mixed-precision-training, mmdetection, nas, nlp, object-detection, pruning, quantization, quantization-aware-training, semantic-segmentation, sparsity, transformers",
"author": "Intel",
"author_email": "alexander.kozlov@intel.com",
"download_url": "https://files.pythonhosted.org/packages/47/00/3c5690dc2e5e58dd18915b5f98b19b82d739692e7065427293a510e6df7e/nncf-2.14.1.tar.gz",
"platform": null,
"description": "[](https://github.com/openvinotoolkit/nncf/releases)\n[](https://docs.openvino.ai/nncf)\n[](https://github.com/openvinotoolkit/nncf?tab=Apache-2.0-1-ov-file#readme)\n[](https://pypi.org/project/nncf/)\n\n# Neural Network Compression Framework (NNCF)\n\nNeural Network Compression Framework (NNCF) provides a suite of post-training\nand training-time algorithms for optimizing inference of neural networks in\n[OpenVINO™](https://docs.openvino.ai) with a minimal accuracy drop.\n\nNNCF is designed to work with models from [PyTorch](https://pytorch.org/),\n[TensorFlow](https://www.tensorflow.org/), [ONNX](https://onnx.ai/) and\n[OpenVINO™](https://docs.openvino.ai).\n\nThe framework is organized as a Python package that can be built and used\nas a standalone tool. Its architecture is unified to make adding different\ncompression algorithms easy for both PyTorch and TensorFlow.\n\nNNCF provides samples that demonstrate the usage of compression algorithms\nfor different use cases and models. See compression results achievable\nwith the NNCF-powered samples on the\n[NNCF Model Zoo page](https://github.com/openvinotoolkit/nncf/blob/develop/docs/ModelZoo.md).\n\nFor more information about NNCF, see:\n\n- [NNCF repository](https://github.com/openvinotoolkit/nncf)\n- [User documentation](https://docs.openvino.ai/nncf)\n- [NNCF API documentation](https://openvinotoolkit.github.io/nncf/autoapi/nncf/)\n- [Usage examples](https://github.com/openvinotoolkit/nncf/tree/develop/docs/usage)\n- [Notebook tutorials](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/README.md#model-training)\n\n## Table of contents\n\n- [Key Features](#key-features)\n- [Installation](#installation-guide)\n- [Third-party integration](#third-party-repository-integration)\n- [NNCF Compressed Model Zoo](#nncf-compressed-model-zoo)\n\n## Key Features<a id=\"key-features\"></a>\n\n### Post-Training Compression Algorithms\n\n| Compression algorithm | OpenVINO | PyTorch | TensorFlow | ONNX |\n| :---------------------------------------------------------------------------------------------------------------------------------------------------------- | :-------: | :-------: | :-----------: | :-----------: |\n| [Post-Training Quantization](https://github.com/openvinotoolkit/nncf/blob/develop/docs/usage/post_training_compression/post_training_quantization/Usage.md) | Supported | Supported | Supported | Supported |\n| [Weight Compression](https://github.com/openvinotoolkit/nncf/blob/develop/docs/usage/post_training_compression/weights_compression/Usage.md) | Supported | Supported | Not supported | Not supported |\n| [Activation Sparsity](https://github.com/openvinotoolkit/nncf/blob/develop/nncf/experimental/torch/sparsify_activations/ActivationSparsity.md) | Not supported | Experimental |Not supported| Not supported |\n\n### Training-Time Compression Algorithms\n\n| Compression algorithm | PyTorch | TensorFlow |\n| :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :----------: | :-----------: |\n| [Quantization Aware Training](https://github.com/openvinotoolkit/nncf/blob/develop/docs/usage/training_time_compression/quantization_aware_training/Usage.md) | Supported | Supported |\n| [Mixed-Precision Quantization](https://github.com/openvinotoolkit/nncf/blob/develop/docs/usage/training_time_compression/other_algorithms/LegacyQuantization.md#mixed-precision-quantization) | Supported | Not supported |\n| [Sparsity](https://github.com/openvinotoolkit/nncf/blob/develop/docs/usage/training_time_compression/other_algorithms/Sparsity.md) | Supported | Supported |\n| [Filter pruning](https://github.com/openvinotoolkit/nncf/blob/develop/docs/usage/training_time_compression/other_algorithms/Pruning.md) | Supported | Supported |\n| [Movement pruning](https://github.com/openvinotoolkit/nncf/blob/develop/nncf/experimental/torch/sparsity/movement/MovementSparsity.md) | Experimental | Not supported |\n\n- Automatic, configurable model graph transformation to obtain the compressed\n model.\n > **NOTE**: Limited support for TensorFlow models. Only models created using\n Sequential or Keras Functional API are supported.\n- Common interface for compression methods.\n- GPU-accelerated layers for faster compressed model fine-tuning.\n- Distributed training support.\n- Git patch for prominent third-party repository\n ([huggingface-transformers](https://github.com/huggingface/transformers))\n demonstrating the process of integrating NNCF into custom training pipelines.\n- Seamless combination of pruning, sparsity, and quantization algorithms. Refer\n to [optimum-intel](https://github.com/huggingface/optimum-intel/tree/main/examples/openvino)\n for examples of joint (movement) pruning, quantization, and distillation\n (JPQD), end-to-end from NNCF optimization to compressed OpenVINO IR.\n- Exporting PyTorch compressed models to ONNX\\* checkpoints and TensorFlow\n compressed models to SavedModel or Frozen Graph format, ready to use with\n [OpenVINO™ toolkit](https://docs.openvino.ai).\n- Support for [Accuracy-Aware model training](https://github.com/openvinotoolkit/nncf/blob/develop/docs/usage/training_time_compression/other_algorithms/Usage.md#accuracy-aware-model-training)\n pipelines via the [Adaptive Compression Level Training](https://github.com/openvinotoolkit/nncf/blob/develop/docs/accuracy_aware_model_training/AdaptiveCompressionLevelTraining.md)\n and [Early Exit Training](https://github.com/openvinotoolkit/nncf/blob/develop/docs/accuracy_aware_model_training/EarlyExitTraining.md).\n\n## Installation Guide<a id=\"installation-guide\"></a>\n\nNNCF can be installed as a regular PyPI package:\n\n```bash\npip install nncf\n```\n\nFor detailed installation instructions, refer to the\n[Installation](https://github.com/openvinotoolkit/nncf/blob/develop/docs/Installation.md) guide.\n\n### System Requirements\n\n- Ubuntu 18.04 or later (64-bit)\n- Python 3.9 or later\n- Supported frameworks:\n - PyTorch >=2.2, <2.5\n - TensorFlow >=2.8.4, <=2.15.1\n - ONNX ==1.16.0\n - OpenVINO >=2022.3.0\n\n## Third-party Repository Integration<a id=\"third-party-repository-integration\"></a>\n\nNNCF may be easily integrated into training/evaluation pipelines of third-party\nrepositories.\n\n- [OpenVINO Training Extensions](https://github.com/openvinotoolkit/training_extensions)\n\n NNCF is integrated into OpenVINO Training Extensions as a model optimization\n backend. You can train, optimize, and export new models based on available\n model templates as well as run the exported models with OpenVINO.\n\n- [HuggingFace Optimum Intel](https://huggingface.co/docs/optimum/intel/optimization_ov)\n\n NNCF is used as a compression backend within the renowned `transformers`\n repository in HuggingFace Optimum Intel.\n\n## NNCF Compressed Model Zoo<a id=\"nncf-compressed-model-zoo\"></a>\n\nA list of models and compression results for them can be found at our\n[NNCF Model Zoo page](https://github.com/openvinotoolkit/nncf/blob/develop/docs/ModelZoo.md).\n\n## Citing\n\n```bi\n@article{kozlov2020neural,\n title = {Neural network compression framework for fast model inference},\n author = {Kozlov, Alexander and Lazarevich, Ivan and Shamporov, Vasily and Lyalyushkin, Nikolay and Gorbachev, Yury},\n journal = {arXiv preprint arXiv:2002.08679},\n year = {2020}\n}\n```\n\n## Telemetry\n\nNNCF as part of the OpenVINO\u2122 toolkit collects anonymous usage data for the\npurpose of improving OpenVINO\u2122 tools. You can opt-out at any time by running\nthe following command in the Python environment where you have NNCF installed:\n\n`opt_in_out --opt_out`\n\nMore information available on [OpenVINO telemetry](https://docs.openvino.ai/nightly/about-openvino/additional-resources/telemetry.html).\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Neural Networks Compression Framework",
"version": "2.14.1",
"project_urls": {
"Homepage": "https://github.com/openvinotoolkit/nncf"
},
"split_keywords": [
"bert",
" classification",
" compression",
" hawq",
" mixed-precision-training",
" mmdetection",
" nas",
" nlp",
" object-detection",
" pruning",
" quantization",
" quantization-aware-training",
" semantic-segmentation",
" sparsity",
" transformers"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e0bb79294ba21276ffd856f81cf7a5cc33cf53779aeefa0ba5218d9ea4175f18",
"md5": "4254a53222614219bcb964b6defabb36",
"sha256": "b708b3366e16adfc456c08d63e49baca110820e7705294464040bd18e6047713"
},
"downloads": -1,
"filename": "nncf-2.14.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4254a53222614219bcb964b6defabb36",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 1344994,
"upload_time": "2024-12-19T12:29:23",
"upload_time_iso_8601": "2024-12-19T12:29:23.924094Z",
"url": "https://files.pythonhosted.org/packages/e0/bb/79294ba21276ffd856f81cf7a5cc33cf53779aeefa0ba5218d9ea4175f18/nncf-2.14.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "47003c5690dc2e5e58dd18915b5f98b19b82d739692e7065427293a510e6df7e",
"md5": "ff9ae220d1927f25b68ace0dbc0e4f5f",
"sha256": "b40fb3495c678d9b67f8818113e2fe3011feeb258a6338657c612fa0f3ee1f3c"
},
"downloads": -1,
"filename": "nncf-2.14.1.tar.gz",
"has_sig": false,
"md5_digest": "ff9ae220d1927f25b68ace0dbc0e4f5f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 919922,
"upload_time": "2024-12-19T12:29:25",
"upload_time_iso_8601": "2024-12-19T12:29:25.543713Z",
"url": "https://files.pythonhosted.org/packages/47/00/3c5690dc2e5e58dd18915b5f98b19b82d739692e7065427293a510e6df7e/nncf-2.14.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-19 12:29:25",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "openvinotoolkit",
"github_project": "nncf",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "nncf"
}
JSON
