# PruningDistribution
Library for pruning convolutional neural networks by varying the pruning distribution.
## Installation
PruningDistribution can be installed using pip:
```bash
pip3 install PruningDistribution
```
or if you want to run the latest version of the code, you can install from git:
```bash
git clone https://github.com/DEEP-CGPS/PruningDistribution
cd PruningDistribution
pip3 install -r requirements.txt
```
****
## Usage
### Main function
The main function "PruningDistribution" provides all the tools necessary to prune, train, and generate performance metrics by varying the pruning distribution.
#### Parameters
Parse command-line arguments for configuring and training a neural network model.
- `model_architecture (str):` Specify the architecture of the model (e.g., VGG16, AlexNet, etc.).
- `method (str):` Specify the training method (e.g., SenpisFaster, random, weight).
- `dataset (str):` Specify the dataset for training (e.g., CIFAR10, "Name of custom dataset").
- `batch_size (int):` Set the batch size for training.
- `num_epochs (int):` Specify the number of training epochs.
- `learning_rate (float):` Set the learning rate for the optimizer.
- `optimizer_val (str):` Specify the optimizer for training (e.g., SGD, Adam, etc.).
- `model_type (str):` Specify the type of the model (e.g., PRUNED or UNPRUNED).
- `device (str):` Specify the device for training (e.g., "cuda:0" for GPU).
- `model_input (torch.Tensor):` Input tensor for the model (default is a tensor of ones).
- `eval_metric (str):` Specify the evaluation metric (e.g., accuracy, f1).
- `seed (int):` Set the seed for random pruning operations.
- `list_pruning (list):` Specify the list of pruning ratios for each layer.
#### Minimal working example
```python
## 1- Definition of arguments for function usage
import sys
import torch
import torchvision
from pruningdistribution import *
import argparse
sys.argv = ['']
import argparse
import torch
parser = argparse.ArgumentParser(description='Parameters for training')
parser.add_argument('--model_architecture', type=str, default="VGG16",
help='Specify the architecture of the model (e.g., VGG16, AlexNet, etc.).')
parser.add_argument('--method', type=str, default="random",
help='Specify the training method (e.g., SenpisFaster, random, weight).')
parser.add_argument('--dataset', type=str, default="CIFAR10",
help='Specify the dataset for training (e.g., CIFAR10, "Name of custom dataset").')
parser.add_argument('--batch_size', type=int, default=8,
help='Set the batch size for training.')
parser.add_argument('--num_epochs', type=int, default=1,
help='Specify the number of training epochs.')
parser.add_argument('--learning_rate', type=float, default=1e-3,
help='Set the learning rate for the optimizer.')
parser.add_argument('--optimizer_val', type=str, default="SGD",
help='Specify the optimizer for training (e.g., SGD, Adam, etc.).')
parser.add_argument('--model_type', type=str, default="UNPRUNED",
help='Specify the type of the model (e.g., PRUNED or UNPRUNED).')
parser.add_argument('--device', type=str, default=None,
help='Specify the device for training (e.g., "cuda:0" for GPU).')
parser.add_argument('--model_input', default=torch.ones((1, 3, 224, 224)),
help='Input tensor for the model (default is a tensor of ones).')
parser.add_argument('--eval_metric', default="accuracy",
help='Specify the evaluation metric (e.g., accuracy, f1).')
parser.add_argument('--seed', type=int, default=23,
help='Set the seed for random pruning operations.')
parser.add_argument('--list_pruning', type=list,
default=[0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0],
help='Specify the list of pruning ratios for each layer.')
args = parser.parse_args()
args = parser.parse_args()
if args.device is None:
import torch
args.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
#Get Model, DATASET and TRAIN
model = get_model(10, args)
train_loader, test_loader, num_classes, trainset = get_dataset(args)
train_model(train_loader = train_loader,
test_loader = test_loader,
model = model,
num_classes = num_classes,
args = args)
#Prune trained model:
model = torch.load(f'models/{args.dataset}/{args.model_architecture}_{args.dataset}_{args.model_type}.pth')
model.to(args.device)
args.model_type = f'your_model_name_with_out_fine_tunning'
prune_model(model,num_classes,trainset, args)
```
### Submodules
pruningdistribution contains 8 modules that allow to train, prune, generate result tables, and identify model properties (e.g., parameters, number of layers):
#### modelParams:
Allows to obtain the total number of FLOPs, to generate the model summary, to obtain the number of convolutional layers and the FC.
#### train_epoch:
It is optional but it helps to train an epoch of the model, normally it is not used directly but it is used by the train_model module.
#### test_epoch:
It is optional but it helps to perform the test during the epoch of the model, normally it is not used directly but it is used by the train_model module.
#### train_model:
Given the input arguments, allows to train the desired convolutional neural network.
#### get_model:
Returns the desired model.
#### get_dataset:
Returns the desired dataset.
#### prune_model:
Prunes the model, taking into account the arguments.
#### evaluate_models:
Returns a dataframe containing the summary of the pruned model information, this to facilitate its later analysis.
## Citing
If you use this software for research or application purposes, please use the following citation:
```bibtex
@article{ ,
title = {},
journal = {SoftwareX},
volume = {},
pages = {},
year = {},
issn = {},
doi = {},
url = {},
author = {},
}
Raw data
{
"_id": null,
"home_page": "https://github.com/DEEP-CGPS/PruningDistribution",
"name": "pruningdistribution",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "python,pytorch,pruning,CNN,distribution,FLOPs",
"author": "DEEP-CGPS",
"author_email": "<est.cesar.pachon@unimilitar.edu.co>",
"download_url": "https://files.pythonhosted.org/packages/f3/b2/0573f293e02453ee0712408fde15451eead9b3c25943839c6a7ca1ddcc45/pruningdistribution-0.1.0.tar.gz",
"platform": null,
"description": "\r\n# PruningDistribution\r\n\r\n\r\n\r\nLibrary for pruning convolutional neural networks by varying the pruning distribution.\r\n\r\n\r\n\r\n## Installation\r\n\r\n\r\n\r\nPruningDistribution can be installed using pip:\r\n\r\n\r\n\r\n```bash\r\n\r\npip3 install PruningDistribution\r\n\r\n```\r\n\r\n\r\n\r\nor if you want to run the latest version of the code, you can install from git:\r\n\r\n\r\n\r\n```bash\r\n\r\ngit clone https://github.com/DEEP-CGPS/PruningDistribution\r\n\r\ncd PruningDistribution\r\n\r\npip3 install -r requirements.txt\r\n\r\n```\r\n\r\n\r\n\r\n****\r\n\r\n\r\n\r\n## Usage\r\n\r\n\r\n\r\n### Main function\r\n\r\n\r\n\r\nThe main function \"PruningDistribution\" provides all the tools necessary to prune, train, and generate performance metrics by varying the pruning distribution. \r\n\r\n\r\n\r\n#### Parameters\r\n\r\n\r\n\r\nParse command-line arguments for configuring and training a neural network model.\r\n\r\n \r\n\r\n- `model_architecture (str):` Specify the architecture of the model (e.g., VGG16, AlexNet, etc.).\r\n\r\n- `method (str):` Specify the training method (e.g., SenpisFaster, random, weight).\r\n\r\n- `dataset (str):` Specify the dataset for training (e.g., CIFAR10, \"Name of custom dataset\").\r\n\r\n- `batch_size (int):` Set the batch size for training.\r\n\r\n- `num_epochs (int):` Specify the number of training epochs.\r\n\r\n- `learning_rate (float):` Set the learning rate for the optimizer.\r\n\r\n- `optimizer_val (str):` Specify the optimizer for training (e.g., SGD, Adam, etc.).\r\n\r\n- `model_type (str):` Specify the type of the model (e.g., PRUNED or UNPRUNED).\r\n\r\n- `device (str):` Specify the device for training (e.g., \"cuda:0\" for GPU).\r\n\r\n- `model_input (torch.Tensor):` Input tensor for the model (default is a tensor of ones).\r\n\r\n- `eval_metric (str):` Specify the evaluation metric (e.g., accuracy, f1).\r\n\r\n- `seed (int):` Set the seed for random pruning operations.\r\n\r\n- `list_pruning (list):` Specify the list of pruning ratios for each layer.\r\n\r\n\r\n\r\n#### Minimal working example\r\n\r\n\r\n\r\n```python\r\n\r\n\r\n\r\n## 1- Definition of arguments for function usage\r\n\r\n\r\n\r\nimport sys\r\n\r\nimport torch\r\n\r\nimport torchvision\r\n\r\nfrom pruningdistribution import *\r\n\r\nimport argparse\r\n\r\nsys.argv = ['']\r\n\r\n\r\n\r\nimport argparse\r\n\r\nimport torch\r\n\r\n\r\n\r\nparser = argparse.ArgumentParser(description='Parameters for training')\r\n\r\n\r\n\r\nparser.add_argument('--model_architecture', type=str, default=\"VGG16\", \r\n\r\n help='Specify the architecture of the model (e.g., VGG16, AlexNet, etc.).')\r\n\r\n\r\n\r\nparser.add_argument('--method', type=str, default=\"random\", \r\n\r\n help='Specify the training method (e.g., SenpisFaster, random, weight).')\r\n\r\n\r\n\r\nparser.add_argument('--dataset', type=str, default=\"CIFAR10\", \r\n\r\n help='Specify the dataset for training (e.g., CIFAR10, \"Name of custom dataset\").')\r\n\r\n\r\n\r\nparser.add_argument('--batch_size', type=int, default=8, \r\n\r\n help='Set the batch size for training.')\r\n\r\n\r\n\r\nparser.add_argument('--num_epochs', type=int, default=1, \r\n\r\n help='Specify the number of training epochs.')\r\n\r\n\r\n\r\nparser.add_argument('--learning_rate', type=float, default=1e-3, \r\n\r\n help='Set the learning rate for the optimizer.')\r\n\r\n\r\n\r\nparser.add_argument('--optimizer_val', type=str, default=\"SGD\", \r\n\r\n help='Specify the optimizer for training (e.g., SGD, Adam, etc.).')\r\n\r\n\r\n\r\nparser.add_argument('--model_type', type=str, default=\"UNPRUNED\", \r\n\r\n help='Specify the type of the model (e.g., PRUNED or UNPRUNED).')\r\n\r\n\r\n\r\nparser.add_argument('--device', type=str, default=None, \r\n\r\n help='Specify the device for training (e.g., \"cuda:0\" for GPU).')\r\n\r\n\r\n\r\nparser.add_argument('--model_input', default=torch.ones((1, 3, 224, 224)), \r\n\r\n help='Input tensor for the model (default is a tensor of ones).')\r\n\r\n\r\n\r\nparser.add_argument('--eval_metric', default=\"accuracy\", \r\n\r\n help='Specify the evaluation metric (e.g., accuracy, f1).')\r\n\r\n\r\n\r\nparser.add_argument('--seed', type=int, default=23, \r\n\r\n help='Set the seed for random pruning operations.')\r\n\r\n\r\n\r\nparser.add_argument('--list_pruning', type=list, \r\n\r\n default=[0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0], \r\n\r\n help='Specify the list of pruning ratios for each layer.')\r\n\r\n\r\n\r\nargs = parser.parse_args()\r\n\r\n\r\n\r\n\r\n\r\nargs = parser.parse_args()\r\n\r\n\r\n\r\nif args.device is None:\r\n\r\n import torch\r\n\r\n args.device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\r\n\r\n\r\n\r\n#Get Model, DATASET and TRAIN\r\n\r\n\r\n\r\nmodel = get_model(10, args)\r\n\r\ntrain_loader, test_loader, num_classes, trainset = get_dataset(args)\r\n\r\ntrain_model(train_loader = train_loader,\r\n\r\n test_loader = test_loader,\r\n\r\n model = model,\r\n\r\n num_classes = num_classes,\r\n\r\n args = args)\r\n\r\n\r\n\r\n#Prune trained model:\r\n\r\n\r\n\r\nmodel = torch.load(f'models/{args.dataset}/{args.model_architecture}_{args.dataset}_{args.model_type}.pth')\r\n\r\nmodel.to(args.device)\r\n\r\nargs.model_type = f'your_model_name_with_out_fine_tunning'\r\n\r\nprune_model(model,num_classes,trainset, args)\r\n\r\n\r\n\r\n```\r\n\r\n\r\n\r\n### Submodules\r\n\r\n\r\n\r\npruningdistribution contains 8 modules that allow to train, prune, generate result tables, and identify model properties (e.g., parameters, number of layers):\r\n\r\n\r\n\r\n#### modelParams:\r\n\r\n\r\n\r\nAllows to obtain the total number of FLOPs, to generate the model summary, to obtain the number of convolutional layers and the FC.\r\n\r\n\r\n\r\n#### train_epoch:\r\n\r\n\r\n\r\nIt is optional but it helps to train an epoch of the model, normally it is not used directly but it is used by the train_model module.\r\n\r\n\r\n\r\n#### test_epoch:\r\n\r\n\r\n\r\nIt is optional but it helps to perform the test during the epoch of the model, normally it is not used directly but it is used by the train_model module.\r\n\r\n\r\n\r\n#### train_model:\r\n\r\n\r\n\r\nGiven the input arguments, allows to train the desired convolutional neural network.\r\n\r\n\r\n\r\n#### get_model:\r\n\r\n\r\n\r\nReturns the desired model.\r\n\r\n\r\n\r\n#### get_dataset:\r\n\r\n\r\n\r\nReturns the desired dataset.\r\n\r\n\r\n\r\n#### prune_model:\r\n\r\n\r\n\r\nPrunes the model, taking into account the arguments.\r\n\r\n\r\n\r\n#### evaluate_models:\r\n\r\n\r\n\r\nReturns a dataframe containing the summary of the pruned model information, this to facilitate its later analysis.\r\n\r\n\r\n\r\n## Citing\r\n\r\n\r\n\r\nIf you use this software for research or application purposes, please use the following citation:\r\n\r\n\r\n\r\n```bibtex\r\n\r\n@article{ ,\r\n\r\n title = {},\r\n\r\n journal = {SoftwareX},\r\n\r\n volume = {},\r\n\r\n pages = {},\r\n\r\n year = {},\r\n\r\n issn = {},\r\n\r\n doi = {},\r\n\r\n url = {},\r\n\r\n author = {},\r\n\r\n}\r\n\r\n",
"bugtrack_url": null,
"license": "",
"summary": "Pruning of CNNs with distributions",
"version": "0.1.0",
"project_urls": {
"Homepage": "https://github.com/DEEP-CGPS/PruningDistribution"
},
"split_keywords": [
"python",
"pytorch",
"pruning",
"cnn",
"distribution",
"flops"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "32b56eb3839272bd91dcad5828f4b57e170a34fffb9b775c4af12c7ee2f52328",
"md5": "373bfa0cddea8995ac95bf1cf210ac4e",
"sha256": "17a2682848ed1a56a9ca74a9c61227eaa1b2dd0f27b7ce4b0d0559074fc61842"
},
"downloads": -1,
"filename": "pruningdistribution-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "373bfa0cddea8995ac95bf1cf210ac4e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 11559,
"upload_time": "2024-01-19T02:48:59",
"upload_time_iso_8601": "2024-01-19T02:48:59.077346Z",
"url": "https://files.pythonhosted.org/packages/32/b5/6eb3839272bd91dcad5828f4b57e170a34fffb9b775c4af12c7ee2f52328/pruningdistribution-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f3b20573f293e02453ee0712408fde15451eead9b3c25943839c6a7ca1ddcc45",
"md5": "7800e3a3f61f54d84c1301336b704859",
"sha256": "dcd00a806dc0106f4b489a958a2c231f818ee365ef6913f63239d0cf7e96fd53"
},
"downloads": -1,
"filename": "pruningdistribution-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "7800e3a3f61f54d84c1301336b704859",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 11651,
"upload_time": "2024-01-19T02:49:02",
"upload_time_iso_8601": "2024-01-19T02:49:02.649685Z",
"url": "https://files.pythonhosted.org/packages/f3/b2/0573f293e02453ee0712408fde15451eead9b3c25943839c6a7ca1ddcc45/pruningdistribution-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-19 02:49:02",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "DEEP-CGPS",
"github_project": "PruningDistribution",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "pruningdistribution"
}
JSON
