mct-nightly


Namemct-nightly JSON
Version 2.0.0.20240418.439 PyPI version JSON
download
home_pageNone
SummaryA Model Compression Toolkit for neural networks
upload_time2024-04-18 00:04:49
maintainerNone
docs_urlNone
authorNone
requires_python>=3.6
licenseNone
keywords
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # Model Compression Toolkit (MCT)

Model Compression Toolkit (MCT) is an open-source project for neural network model optimization under efficient, constrained hardware.

This project provides researchers, developers, and engineers tools for optimizing and deploying state-of-the-art neural networks on efficient hardware.

Specifically, this project aims to apply quantization to compress neural networks.

<img src="docsrc/images/mct_block_diagram.svg" width="10000">

MCT is developed by researchers and engineers working at Sony Semiconductor Israel.



## Table of Contents

- [Getting Started](#getting-started)
- [Supported features](#supported-features)
- [Results](#results)
- [Troubleshooting](#trouble-shooting)
- [Contributions](#contributions)
- [License](#license)


## Getting Started

This section provides an installation and a quick starting guide.

### Installation

To install the latest stable release of MCT, run the following command:
```
pip install model-compression-toolkit
```

For installing the nightly version or installing from source, refer to the [installation guide](INSTALLATION.md).


### Quick start & tutorials 

Explore the Model Compression Toolkit (MCT) through our tutorials, 
covering compression techniques for Keras and PyTorch models. Access interactive [notebooks](tutorials/README.md) 
for hands-on learning. For example:
* [Keras MobileNetV2 post training quantization](tutorials/notebooks/keras/ptq/example_keras_imagenet.ipynb)
* [Post training quantization with PyTorch](tutorials/notebooks/pytorch/ptq/example_pytorch_quantization_mnist.ipynb)
* [Data Generation for ResNet18 with PyTorch](tutorials/notebooks/pytorch/data_generation/example_pytorch_data_generation.ipynb).

Additionally, for quick quantization of a variety of models from well-known collections,
visit the [quick-start page](tutorials/quick_start/README.md) and the
[results CSV](tutorials/quick_start/results/model_quantization_results.csv).

### Supported Versions

Currently, MCT is being tested on various Python, Pytorch and TensorFlow versions:

|             |  PyTorch 1.13                                                                                                                                                                                                               | PyTorch 2.0                                                                                                                                                                                                              | PyTorch 2.1                                                                                                                                                                                                              |
|-------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Python 3.9  | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch113.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch113.yml)   | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch20.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch20.yml)   | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch21.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch21.yml)   |
| Python 3.10 | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch112.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch112.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch113.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch113.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch20.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch20.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch21.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch21.yml) |
| Python 3.11 | | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch20.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch20.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch21.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch21.yml) |


|             | TensorFlow 2.12                                                                                                                                                                                                                        | TensorFlow 2.13                                                                                                                                                                                                                        | TensorFlow 2.14                                                                                                                                                                                                                        | TensorFlow 2.15                                                                                                                                                                                                        |
|-------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Python 3.9  | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_keras212.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_keras212.yml)   | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_keras213.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_keras213.yml)   | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_keras214.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_keras214.yml)   | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_keras215.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_keras215.yml)   |
| Python 3.10 | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras212.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras212.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras213.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras213.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras214.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras214.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras215.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras215.yml) |
| Python 3.11 | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras212.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras212.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras213.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras213.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras214.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras214.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras215.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras215.yml) |


## Supported Features
MCT offers a range of powerful features to optimize neural network models for efficient deployment. These supported features include:

### Data Generation [*](#experimental-features)
MCT provides tools for generating synthetic images based on the statistics stored in a model's batch normalization layers. These generated images are valuable for various compression tasks where image data is required, such as quantization and pruning. 
You can customize data generation configurations to suit your specific needs. [Go to the Data Generation page.](model_compression_toolkit/data_generation/README.md)

### Quantization
MCT supports different quantization methods:
* Post-training quantization (PTQ): [Keras API](https://sony.github.io/model_optimization/docs/api/api_docs/methods/keras_post_training_quantization.html), [PyTorch API](https://sony.github.io/model_optimization/docs/api/api_docs/methods/pytorch_post_training_quantization.html)
* Gradient-based post-training quantization (GPTQ): [Keras API](https://sony.github.io/model_optimization/docs/api/api_docs/methods/keras_gradient_post_training_quantization.html), [PyTorch API](https://sony.github.io/model_optimization/docs/api/api_docs/methods/pytorch_gradient_post_training_quantization.html)
* Quantization-aware training (QAT) [*](#experimental-features)


| Quantization Method                           | Complexity | Computational Cost          |
|-----------------------------------------------|------------|-----------------------------|
| PTQ                                           | Low        | Low (order of minutes)      |
| GPTQ (parameters fine-tuning using gradients) | Mild       | Mild (order of 2-3 hours)   |
| QAT                                           | High       | High (order of 12-36 hours) |


In addition, MCT supports different quantization schemes for quantizing weights and activations:

* Power-Of-Two (hardware-friendly quantization [1])
* Symmetric
* Uniform

Main features:
* <ins>Graph optimizations:</ins> Transforming the model to an equivalent (yet, more efficient) model (for example, batch-normalization layer folding to its preceding linear layer).
* <ins>Quantization parameter search:</ins> Different methods can be used to minimize the expected added quantization-noise during thresholds search (by default, we use Mean-Square-Error, but other metrics can be used such as No-Clipping, Mean-Average-Error, and more).
* <ins>Advanced quantization algorithms:</ins> To prevent a performance degradation some algorithms are applied such as: 
  * <ins>Shift negative correction:</ins> Symmetric activation quantization can hurt the model's performance when some layers output both negative and positive activations, but their range is asymmetric. For more details please visit [1].
  * <ins>Outliers filtering:</ins> Computing z-score for activation statistics to detect and remove outliers.
* <ins>Clustering:</ins> Using non-uniform quantization grid to quantize the weights and activations to match their distributions.[*](#experimental-features)
* <ins>Mixed-precision search:</ins> Assigning quantization bit-width per layer (for weights/activations), based on the layer's sensitivity to different bit-widths.
* <ins>Visualization:</ins> You can use TensorBoard to observe useful information for troubleshooting the quantized model's performance (for example, the model in different phases of the quantization, collected statistics, similarity between layers of the float and quantized model and bit-width configuration for mixed-precision quantization). For more details, please read the [visualization documentation](https://sony.github.io/model_optimization/docs/guidelines/visualization.html).   
* <ins>Target Platform Capabilities:</ins> The Target Platform Capabilities (TPC) describes the target platform (an edge device with dedicated hardware). For more details, please read the [TPC README](model_compression_toolkit/target_platform_capabilities/README.md).   

### Enhanced Post-Training Quantization (EPTQ)
As part of the GPTQ we provide an advanced optimization algorithm called EPTQ.

The specifications of the algorithm are detailed in the paper: _"**EPTQ: Enhanced Post-Training Quantization via Label-Free Hessian**"_ [4].

More details on the how to use EPTQ via MCT can be found in the [EPTQ guidelines](model_compression_toolkit/gptq/README.md).


### Structured Pruning [*](#experimental-features)
MCT introduces a structured and hardware-aware model pruning.
This pruning technique is designed to compress models for specific hardware architectures, 
taking into account the target platform's Single Instruction, Multiple Data (SIMD) capabilities. 
By pruning groups of channels (SIMD groups), our approach not only reduces model size 
and complexity, but ensures that better utilization of channels is in line with the SIMD architecture 
for a target Resource Utilization of weights memory footprint.
[Keras API](https://sony.github.io/model_optimization/docs/api/api_docs/methods/keras_pruning_experimental.html)
[Pytorch API](https://sony.github.io/model_optimization/docs/api/api_docs/methods/pytorch_pruning_experimental.html) 

#### Experimental features 

Some features are experimental and subject to future changes. 

For more details, we highly recommend visiting our project website where experimental features are mentioned as experimental.


## Results
### Keras
Graph of [MobileNetV2](https://keras.io/api/applications/mobilenet/) accuracy on ImageNet vs average bit-width of weights, using 
single-precision quantization, mixed-precision quantization, and mixed-precision quantization with GPTQ.

<img src="docsrc/images/mbv2_accuracy_graph.png">

For more results, please see [1]

### Pytorch
We quantized classification networks from the torchvision library. 
In the following table we present the ImageNet validation results for these models:

| Network Name              | Float Accuracy  | 8Bit Accuracy   | Data-Free 8Bit Accuracy |
|---------------------------|-----------------|-----------------|-------------------------|
| MobileNet V2 [3]          | 71.886          | 71.444          |71.29|
| ResNet-18 [3]             | 69.86           | 69.63           |69.53|
| SqueezeNet 1.1 [3]        | 58.128          | 57.678          ||

For more results, please refer to [quick start](https://github.com/sony/model_optimization/tree/main/tutorials/quick_start).


#### Pruning Results

Results for applying pruning to reduce the parameters of the following models by 50%:

| Model           | Dense Model Accuracy | Pruned Model Accuracy |
|-----------------|----------------------|-----------------------|
| ResNet50 [2]    | 75.1                 | 72.4                  |
| DenseNet121 [3] | 74.44                | 71.71                 |


## Trouble Shooting

If the accuracy degradation of the quantized model is too large for your application, check out the [Quantization Troubleshooting](https://github.com/sony/model_optimization/tree/main/quantization_troubleshooting.md)
for common pitfalls and some tools to improve quantization accuracy.

Check out the [FAQ](https://github.com/sony/model_optimization/tree/main/FAQ.md) for common issues.


## Contributions
MCT aims at keeping a more up-to-date fork and welcomes contributions from anyone.

*You will find more information about contributions in the [Contribution guide](CONTRIBUTING.md).


## License
[Apache License 2.0](LICENSE.md).

## References 

[1] Habi, H.V., Peretz, R., Cohen, E., Dikstein, L., Dror, O., Diamant, I., Jennings, R.H. and Netzer, A., 2021. [HPTQ: Hardware-Friendly Post Training Quantization. arXiv preprint](https://arxiv.org/abs/2109.09113).

[2] [Keras Applications](https://keras.io/api/applications/)

[3] [TORCHVISION.MODELS](https://pytorch.org/vision/stable/models.html) 

[4] Gordon, O., Habi, H. V., & Netzer, A., 2023. [EPTQ: Enhanced Post-Training Quantization via Label-Free Hessian. arXiv preprint](https://arxiv.org/abs/2309.11531)



            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "mct-nightly",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.6",
    "maintainer_email": null,
    "keywords": null,
    "author": null,
    "author_email": null,
    "download_url": "https://files.pythonhosted.org/packages/8a/fc/87cf8903a51b7a788e0e3c80c29d8ecbee45fe8feb62bbc6a3641170017a/mct-nightly-2.0.0.20240418.439.tar.gz",
    "platform": null,
    "description": "# Model Compression Toolkit (MCT)\n\nModel Compression Toolkit (MCT) is an open-source project for neural network model optimization under efficient, constrained hardware.\n\nThis project provides researchers, developers, and engineers tools for optimizing and deploying state-of-the-art neural networks on efficient hardware.\n\nSpecifically, this project aims to apply quantization to compress neural networks.\n\n<img src=\"docsrc/images/mct_block_diagram.svg\" width=\"10000\">\n\nMCT is developed by researchers and engineers working at Sony Semiconductor Israel.\n\n\n\n## Table of Contents\n\n- [Getting Started](#getting-started)\n- [Supported features](#supported-features)\n- [Results](#results)\n- [Troubleshooting](#trouble-shooting)\n- [Contributions](#contributions)\n- [License](#license)\n\n\n## Getting Started\n\nThis section provides an installation and a quick starting guide.\n\n### Installation\n\nTo install the latest stable release of MCT, run the following command:\n```\npip install model-compression-toolkit\n```\n\nFor installing the nightly version or installing from source, refer to the [installation guide](INSTALLATION.md).\n\n\n### Quick start & tutorials \n\nExplore the Model Compression Toolkit (MCT) through our tutorials, \ncovering compression techniques for Keras and PyTorch models. Access interactive [notebooks](tutorials/README.md) \nfor hands-on learning. For example:\n* [Keras MobileNetV2 post training quantization](tutorials/notebooks/keras/ptq/example_keras_imagenet.ipynb)\n* [Post training quantization with PyTorch](tutorials/notebooks/pytorch/ptq/example_pytorch_quantization_mnist.ipynb)\n* [Data Generation for ResNet18 with PyTorch](tutorials/notebooks/pytorch/data_generation/example_pytorch_data_generation.ipynb).\n\nAdditionally, for quick quantization of a variety of models from well-known collections,\nvisit the [quick-start page](tutorials/quick_start/README.md) and the\n[results CSV](tutorials/quick_start/results/model_quantization_results.csv).\n\n### Supported Versions\n\nCurrently, MCT is being tested on various Python, Pytorch and TensorFlow versions:\n\n|             |  PyTorch 1.13                                                                                                                                                                                                               | PyTorch 2.0                                                                                                                                                                                                              | PyTorch 2.1                                                                                                                                                                                                              |\n|-------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| Python 3.9  | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch113.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch113.yml)   | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch20.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch20.yml)   | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch21.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch21.yml)   |\n| Python 3.10 | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch112.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch112.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch113.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch113.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch20.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch20.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch21.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch21.yml) |\n| Python 3.11 | | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch20.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch20.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch21.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch21.yml) |\n\n\n|             | TensorFlow 2.12                                                                                                                                                                                                                        | TensorFlow 2.13                                                                                                                                                                                                                        | TensorFlow 2.14                                                                                                                                                                                                                        | TensorFlow 2.15                                                                                                                                                                                                        |\n|-------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| Python 3.9  | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_keras212.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_keras212.yml)   | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_keras213.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_keras213.yml)   | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_keras214.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_keras214.yml)   | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_keras215.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_keras215.yml)   |\n| Python 3.10 | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras212.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras212.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras213.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras213.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras214.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras214.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras215.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras215.yml) |\n| Python 3.11 | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras212.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras212.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras213.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras213.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras214.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras214.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras215.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras215.yml) |\n\n\n## Supported Features\nMCT offers a range of powerful features to optimize neural network models for efficient deployment. These supported features include:\n\n### Data Generation [*](#experimental-features)\nMCT provides tools for generating synthetic images based on the statistics stored in a model's batch normalization layers. These generated images are valuable for various compression tasks where image data is required, such as quantization and pruning. \nYou can customize data generation configurations to suit your specific needs. [Go to the Data Generation page.](model_compression_toolkit/data_generation/README.md)\n\n### Quantization\nMCT supports different quantization methods:\n* Post-training quantization (PTQ): [Keras API](https://sony.github.io/model_optimization/docs/api/api_docs/methods/keras_post_training_quantization.html), [PyTorch API](https://sony.github.io/model_optimization/docs/api/api_docs/methods/pytorch_post_training_quantization.html)\n* Gradient-based post-training quantization (GPTQ): [Keras API](https://sony.github.io/model_optimization/docs/api/api_docs/methods/keras_gradient_post_training_quantization.html), [PyTorch API](https://sony.github.io/model_optimization/docs/api/api_docs/methods/pytorch_gradient_post_training_quantization.html)\n* Quantization-aware training (QAT) [*](#experimental-features)\n\n\n| Quantization Method                           | Complexity | Computational Cost          |\n|-----------------------------------------------|------------|-----------------------------|\n| PTQ                                           | Low        | Low (order of minutes)      |\n| GPTQ (parameters fine-tuning using gradients) | Mild       | Mild (order of 2-3 hours)   |\n| QAT                                           | High       | High (order of 12-36 hours) |\n\n\nIn addition, MCT supports different quantization schemes for quantizing weights and activations:\n\n* Power-Of-Two (hardware-friendly quantization [1])\n* Symmetric\n* Uniform\n\nMain features:\n* <ins>Graph optimizations:</ins> Transforming the model to an equivalent (yet, more efficient) model (for example, batch-normalization layer folding to its preceding linear layer).\n* <ins>Quantization parameter search:</ins> Different methods can be used to minimize the expected added quantization-noise during thresholds search (by default, we use Mean-Square-Error, but other metrics can be used such as No-Clipping, Mean-Average-Error, and more).\n* <ins>Advanced quantization algorithms:</ins> To prevent a performance degradation some algorithms are applied such as: \n  * <ins>Shift negative correction:</ins> Symmetric activation quantization can hurt the model's performance when some layers output both negative and positive activations, but their range is asymmetric. For more details please visit [1].\n  * <ins>Outliers filtering:</ins> Computing z-score for activation statistics to detect and remove outliers.\n* <ins>Clustering:</ins> Using non-uniform quantization grid to quantize the weights and activations to match their distributions.[*](#experimental-features)\n* <ins>Mixed-precision search:</ins> Assigning quantization bit-width per layer (for weights/activations), based on the layer's sensitivity to different bit-widths.\n* <ins>Visualization:</ins> You can use TensorBoard to observe useful information for troubleshooting the quantized model's performance (for example, the model in different phases of the quantization, collected statistics, similarity between layers of the float and quantized model and bit-width configuration for mixed-precision quantization). For more details, please read the [visualization documentation](https://sony.github.io/model_optimization/docs/guidelines/visualization.html).   \n* <ins>Target Platform Capabilities:</ins> The Target Platform Capabilities (TPC) describes the target platform (an edge device with dedicated hardware). For more details, please read the [TPC README](model_compression_toolkit/target_platform_capabilities/README.md).   \n\n### Enhanced Post-Training Quantization (EPTQ)\nAs part of the GPTQ we provide an advanced optimization algorithm called EPTQ.\n\nThe specifications of the algorithm are detailed in the paper: _\"**EPTQ: Enhanced Post-Training Quantization via Label-Free Hessian**\"_ [4].\n\nMore details on the how to use EPTQ via MCT can be found in the [EPTQ guidelines](model_compression_toolkit/gptq/README.md).\n\n\n### Structured Pruning [*](#experimental-features)\nMCT introduces a structured and hardware-aware model pruning.\nThis pruning technique is designed to compress models for specific hardware architectures, \ntaking into account the target platform's Single Instruction, Multiple Data (SIMD) capabilities. \nBy pruning groups of channels (SIMD groups), our approach not only reduces model size \nand complexity, but ensures that better utilization of channels is in line with the SIMD architecture \nfor a target Resource Utilization of weights memory footprint.\n[Keras API](https://sony.github.io/model_optimization/docs/api/api_docs/methods/keras_pruning_experimental.html)\n[Pytorch API](https://sony.github.io/model_optimization/docs/api/api_docs/methods/pytorch_pruning_experimental.html) \n\n#### Experimental features \n\nSome features are experimental and subject to future changes. \n\nFor more details, we highly recommend visiting our project website where experimental features are mentioned as experimental.\n\n\n## Results\n### Keras\nGraph of [MobileNetV2](https://keras.io/api/applications/mobilenet/) accuracy on ImageNet vs average bit-width of weights, using \nsingle-precision quantization, mixed-precision quantization, and mixed-precision quantization with GPTQ.\n\n<img src=\"docsrc/images/mbv2_accuracy_graph.png\">\n\nFor more results, please see [1]\n\n### Pytorch\nWe quantized classification networks from the torchvision library. \nIn the following table we present the ImageNet validation results for these models:\n\n| Network Name              | Float Accuracy  | 8Bit Accuracy   | Data-Free 8Bit Accuracy |\n|---------------------------|-----------------|-----------------|-------------------------|\n| MobileNet V2 [3]          | 71.886          | 71.444          |71.29|\n| ResNet-18 [3]             | 69.86           | 69.63           |69.53|\n| SqueezeNet 1.1 [3]        | 58.128          | 57.678          ||\n\nFor more results, please refer to [quick start](https://github.com/sony/model_optimization/tree/main/tutorials/quick_start).\n\n\n#### Pruning Results\n\nResults for applying pruning to reduce the parameters of the following models by 50%:\n\n| Model           | Dense Model Accuracy | Pruned Model Accuracy |\n|-----------------|----------------------|-----------------------|\n| ResNet50 [2]    | 75.1                 | 72.4                  |\n| DenseNet121 [3] | 74.44                | 71.71                 |\n\n\n## Trouble Shooting\n\nIf the accuracy degradation of the quantized model is too large for your application, check out the [Quantization Troubleshooting](https://github.com/sony/model_optimization/tree/main/quantization_troubleshooting.md)\nfor common pitfalls and some tools to improve quantization accuracy.\n\nCheck out the [FAQ](https://github.com/sony/model_optimization/tree/main/FAQ.md) for common issues.\n\n\n## Contributions\nMCT aims at keeping a more up-to-date fork and welcomes contributions from anyone.\n\n*You will find more information about contributions in the [Contribution guide](CONTRIBUTING.md).\n\n\n## License\n[Apache License 2.0](LICENSE.md).\n\n## References \n\n[1] Habi, H.V., Peretz, R., Cohen, E., Dikstein, L., Dror, O., Diamant, I., Jennings, R.H. and Netzer, A., 2021. [HPTQ: Hardware-Friendly Post Training Quantization. arXiv preprint](https://arxiv.org/abs/2109.09113).\n\n[2] [Keras Applications](https://keras.io/api/applications/)\n\n[3] [TORCHVISION.MODELS](https://pytorch.org/vision/stable/models.html) \n\n[4] Gordon, O., Habi, H. V., & Netzer, A., 2023. [EPTQ: Enhanced Post-Training Quantization via Label-Free Hessian. arXiv preprint](https://arxiv.org/abs/2309.11531)\n\n\n",
    "bugtrack_url": null,
    "license": null,
    "summary": "A Model Compression Toolkit for neural networks",
    "version": "2.0.0.20240418.439",
    "project_urls": null,
    "split_keywords": [],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "fb3a3b97299378823a72f522589322ade16df1f4ad4d94c9b68bdc83c7e479c8",
                "md5": "b208442663aea94044242d9c3b4f9b46",
                "sha256": "9f161c2354de05faaed909613b53b3e52d6714336b21408feaac6c8e6c836d6d"
            },
            "downloads": -1,
            "filename": "mct_nightly-2.0.0.20240418.439-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "b208442663aea94044242d9c3b4f9b46",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.6",
            "size": 872980,
            "upload_time": "2024-04-18T00:04:41",
            "upload_time_iso_8601": "2024-04-18T00:04:41.515561Z",
            "url": "https://files.pythonhosted.org/packages/fb/3a/3b97299378823a72f522589322ade16df1f4ad4d94c9b68bdc83c7e479c8/mct_nightly-2.0.0.20240418.439-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "8afc87cf8903a51b7a788e0e3c80c29d8ecbee45fe8feb62bbc6a3641170017a",
                "md5": "cc57b2b30b83d43acd99a5b7f6f3c4b3",
                "sha256": "475615fedfafd01de715daafcb8a81ec10472095c59d357b829427845fe1d80e"
            },
            "downloads": -1,
            "filename": "mct-nightly-2.0.0.20240418.439.tar.gz",
            "has_sig": false,
            "md5_digest": "cc57b2b30b83d43acd99a5b7f6f3c4b3",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.6",
            "size": 455721,
            "upload_time": "2024-04-18T00:04:49",
            "upload_time_iso_8601": "2024-04-18T00:04:49.166049Z",
            "url": "https://files.pythonhosted.org/packages/8a/fc/87cf8903a51b7a788e0e3c80c29d8ecbee45fe8feb62bbc6a3641170017a/mct-nightly-2.0.0.20240418.439.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-04-18 00:04:49",
    "github": false,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "lcname": "mct-nightly"
}
        
Elapsed time: 0.26438s