# Classifier-trains
For project details, please see [README.md](https://github.com/wyhwong/classifier-trains)
## Install as a package
```bash
pip3 install classifier-trains
```
## Makefile Commands (Only when cloned from GitHub)
```bash
# Check all available commands
make help
```
## After installation
For example of pipeline configuration, please see [pipeline_config_only_train.yml](https://github.com/wyhwong/classifier-trains/blob/main/src/tests/test_data/pipeline_config_only_train.yml), [pipeline_config_only_eval.yml](https://github.com/wyhwong/classifier-trains/blob/main/src/tests/test_data/pipeline_config_only_eval.yml), [full_pipeline_config.yml](https://github.com/wyhwong/classifier-trains/blob/main/src/tests/test_data/full_pipeline_config.yml).
```bash
# Run training or evaluation
python3 -m classifier_trains run --config <path to yml config> --output_dir <output_dir>
# Run training or evaluation with profiling, which will generate a profile report
python3 -m classifier_trains profile --config <path to yml config> --output_dir <output_dir>
# Compute mean and std of dataset
python3 -m classifier_trains compute-mean-and-std --dir-path <path to dataset>
# Get output mapping of dataset
python3 -m classifier_trains get-output-mapping --dir-path <path to dataset>
```
## Expected Folder Structure for Dataset
```
Dataset Directory
├── train
│ ├── <class1>
│ │ ├── <image1>
│ │ └── ...
│ └── <class2>
│ ├── <image1>
│ └── ...
├── val
│ ├── <class1>
│ │ ├── <image1>
│ │ └── ...
│ └── <class2>
│ ├── <image1>
│ └── ...
└── eval
├── <class1>
│ ├── <image1>
│ └── ...
└── <class2>
├── <image1>
└── ...
```
## Pipeline Parameter Table
#### Pipeline Parameters
| Parameter | Description | Type | Default | Choices |
| --- | --- | --- | --- | --- |
| enable_training | Enable training | bool | False | True, False |
| enable_evaluation | Enable evaluation | bool | False | True, False |
#### Model Parameters
| Parameter | Description | Type | Default | Choices |
| --- | --- | --- | --- | --- |
| model | Model architecture to use | str | / | resnet18, resnet34, resnet50, resnet152, vgg11, vgg11_bn, vgg13, vgg13_bn, vgg16, vgg16_bn, vgg19, vgg19_bn, squeezenet1_0, squeezenet1_1, densenet121, densenet161, densenet169, densenet201, inception_v3 |
| num_classes | Number of classes | int | / | Any positive integer |
| weights | Pretrained weights | str | DEFAULT | Check PyTorch Documentation |
| checkpoint_path | Path to checkpoint | Optional[str] | None | Self defined path |
#### Dataloader Parameters
| Parameter | Description | Type | Default | Choices |
| --- | --- | --- | --- | --- |
| batch_size | Batch size | int | / | Any positive integer |
| num_workers | Number of workers | int | 0 | Any non negative integer |
#### Resize Parameters
| Parameter | Description | Type | Default | Choices |
| --- | --- | --- | --- | --- |
| width | Width of resized image | int | / | Any positive integer |
| height | Height of resized image | int | / | Any positive integer |
| interpolation | Interpolation method | str | bicubic | nearest, nearest_exact, bilinear, bicubic |
| padding | Padding | Optional[str] | None | top_left, top_right, bottom_left, bottom_right, center |
| maintain_aspect_ratio | Maintain aspect ratio | bool | False | True, False |
#### Spatial Transform Parameters
| Parameter | Description | Type | Default | Choices |
| --- | --- | --- | --- | --- |
| hflip_prob | Horizontal flip probability | float | / | Any float between 0 and 1 |
| vflip_prob | Vertical flip probability | float | / | Any float between 0 and 1 |
| max_rotate_in_degree | Maximum rotation in degree | float | 0.0 | Any non negative float |
| allow_center_crop | Allow center crop | bool | False | True, False |
| allow_random_crop | Allow random crop | bool | False | True, False |
#### Color Transform Parameters
| Parameter | Description | Type | Default | Choices |
| --- | --- | --- | --- | --- |
| allow_gray_scale | Allow gray scale | bool | False | True, False |
| allow_random_color | Allow random color | bool | False | True, False |
#### Preprocessing Parameters
| Parameter | Description | Type | Default | Choices |
| --- | --- | --- | --- | --- |
| mean | Mean of dataset | List[float] | / | Any list of floats |
| std | Standard deviation of dataset | List[float] | / | Any list of floats |
#### Optimizer Parameters
| Parameter | Description | Type | Default | Choices |
| --- | --- | --- | --- | --- |
| name | Optimizer to use | str | / | sgd, adam, rmsprop, adamw |
| lr | Learning rate | float | / | Any positive float |
| weight_decay | Weight decay | float | 0.0 | Any non negative float |
| momentum | Momentum | Optional[float] | None | Any non negative float |
| alpha | Alpha | Optional[float] | None | Any non negative float |
| betas | Betas | Optional[Tuple[float, float]] | None | Any tuple of non negative floats |
#### Scheduler Parameters
| Parameter | Description | Type | Default | Choices |
| --- | --- | --- | --- | --- |
| name | Scheduler to use | str | / | step, consine |
| lr_min | Minimum learning rate | Optional[float] | None | Any non negative float |
| step_size | Step size | Optional[int] | None | Any positive integer |
| gamma | Gamma | Optional[float] | None | Any non negative float |
#### Training Parameters
| Parameter | Description | Type | Default | Choices |
| --- | --- | --- | --- | --- |
| name | Name of the experiment | str | / | Any string |
| num_epochs | Number of epochs | int | / | Any positive integer |
| trainset_dir | Path to training dataset | str | / | Any string |
| valset_dir | Path to validation dataset | str | / | Any string |
| testset_dir | Path to testing dataset | Optional[str] | None | Any string |
| device | Device to use | str | cuda | cpu, cuda, etc. |
| max_num_hrs | Maximum number of hours to run | Optional[float] | None | Any non negative float |
| criterion | What criterion to use for best model | str | loss | loss, accuracy |
| validate_every | Validate every n epochs | int | 1 | Any positive integer |
| save_every | Save model every n epochs | int | 3 | Any positive integer |
| patience | Patience for early stopping | int | 5 | Any positive integer |
| random_seed | Random seed | int | 42 | Any positive integer |
| precision | Precision | int | 64 | 16, 32, 64 |
| export_last_as_onnx | Export last model as ONNX | bool | False | True, False |
| export_best_as_onnx | Export best model as ONNX | bool | False | True, False |
#### Evaluation Parameters
| Parameter | Description | Type | Default | Choices |
| --- | --- | --- | --- | --- |
| name | Name of the experiment | str | / | Any string |
| evalset_dir | Path to evaluation dataset | str | / | Any string |
| device | Device to use | str | cuda | cpu, cuda, etc. |
| precision | Precision | int | 64 | 16, 32, 64 |
| random_seed | Random seed | int | 42 | Any positive integer |
Raw data
{
"_id": null,
"home_page": "https://github.com/wyhwong/classifier-trains",
"name": "classifier_trains",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.11",
"maintainer_email": null,
"keywords": "pytorch, deep learning, classification, training, onnx, tensorboard, profiler",
"author": "wyhwong",
"author_email": "wyhwong@link.cuhk.edu.hk",
"download_url": "https://files.pythonhosted.org/packages/7f/c9/b0ad10e5335a3a97462bc8ae5b87186c8a67966b7a689b2c4a9948751a1c/classifier_trains-1.1.0.tar.gz",
"platform": null,
"description": "# Classifier-trains\n\nFor project details, please see [README.md](https://github.com/wyhwong/classifier-trains)\n\n## Install as a package\n```bash\npip3 install classifier-trains\n```\n\n## Makefile Commands (Only when cloned from GitHub)\n\n```bash\n# Check all available commands\nmake help\n```\n\n## After installation\n\nFor example of pipeline configuration, please see [pipeline_config_only_train.yml](https://github.com/wyhwong/classifier-trains/blob/main/src/tests/test_data/pipeline_config_only_train.yml), [pipeline_config_only_eval.yml](https://github.com/wyhwong/classifier-trains/blob/main/src/tests/test_data/pipeline_config_only_eval.yml), [full_pipeline_config.yml](https://github.com/wyhwong/classifier-trains/blob/main/src/tests/test_data/full_pipeline_config.yml).\n\n```bash\n# Run training or evaluation\npython3 -m classifier_trains run --config <path to yml config> --output_dir <output_dir>\n\n# Run training or evaluation with profiling, which will generate a profile report\npython3 -m classifier_trains profile --config <path to yml config> --output_dir <output_dir>\n\n# Compute mean and std of dataset\npython3 -m classifier_trains compute-mean-and-std --dir-path <path to dataset>\n\n# Get output mapping of dataset\npython3 -m classifier_trains get-output-mapping --dir-path <path to dataset>\n```\n\n## Expected Folder Structure for Dataset\n```\nDataset Directory\n\u251c\u2500\u2500 train\n\u2502 \u251c\u2500\u2500 <class1>\n\u2502 \u2502 \u251c\u2500\u2500 <image1>\n\u2502 \u2502 \u2514\u2500\u2500 ...\n\u2502 \u2514\u2500\u2500 <class2>\n\u2502 \u251c\u2500\u2500 <image1>\n\u2502 \u2514\u2500\u2500 ...\n\u251c\u2500\u2500 val\n\u2502 \u251c\u2500\u2500 <class1>\n\u2502 \u2502 \u251c\u2500\u2500 <image1>\n\u2502 \u2502 \u2514\u2500\u2500 ...\n\u2502 \u2514\u2500\u2500 <class2>\n\u2502 \u251c\u2500\u2500 <image1>\n\u2502 \u2514\u2500\u2500 ...\n\u2514\u2500\u2500 eval\n \u251c\u2500\u2500 <class1>\n \u2502 \u251c\u2500\u2500 <image1>\n \u2502 \u2514\u2500\u2500 ...\n \u2514\u2500\u2500 <class2>\n \u251c\u2500\u2500 <image1>\n \u2514\u2500\u2500 ...\n```\n\n## Pipeline Parameter Table\n\n#### Pipeline Parameters\n\n| Parameter | Description | Type | Default | Choices |\n| --- | --- | --- | --- | --- |\n| enable_training | Enable training | bool | False | True, False |\n| enable_evaluation | Enable evaluation | bool | False | True, False |\n\n#### Model Parameters\n\n| Parameter | Description | Type | Default | Choices |\n| --- | --- | --- | --- | --- |\n| model | Model architecture to use | str | / | resnet18, resnet34, resnet50, resnet152, vgg11, vgg11_bn, vgg13, vgg13_bn, vgg16, vgg16_bn, vgg19, vgg19_bn, squeezenet1_0, squeezenet1_1, densenet121, densenet161, densenet169, densenet201, inception_v3 |\n| num_classes | Number of classes | int | / | Any positive integer |\n| weights | Pretrained weights | str | DEFAULT | Check PyTorch Documentation |\n| checkpoint_path | Path to checkpoint | Optional[str] | None | Self defined path |\n\n#### Dataloader Parameters\n\n| Parameter | Description | Type | Default | Choices |\n| --- | --- | --- | --- | --- |\n| batch_size | Batch size | int | / | Any positive integer |\n| num_workers | Number of workers | int | 0 | Any non negative integer |\n\n#### Resize Parameters\n\n| Parameter | Description | Type | Default | Choices |\n| --- | --- | --- | --- | --- |\n| width | Width of resized image | int | / | Any positive integer |\n| height | Height of resized image | int | / | Any positive integer |\n| interpolation | Interpolation method | str | bicubic | nearest, nearest_exact, bilinear, bicubic |\n| padding | Padding | Optional[str] | None | top_left, top_right, bottom_left, bottom_right, center |\n| maintain_aspect_ratio | Maintain aspect ratio | bool | False | True, False |\n\n#### Spatial Transform Parameters\n\n| Parameter | Description | Type | Default | Choices |\n| --- | --- | --- | --- | --- |\n| hflip_prob | Horizontal flip probability | float | / | Any float between 0 and 1 |\n| vflip_prob | Vertical flip probability | float | / | Any float between 0 and 1 |\n| max_rotate_in_degree | Maximum rotation in degree | float | 0.0 | Any non negative float |\n| allow_center_crop | Allow center crop | bool | False | True, False |\n| allow_random_crop | Allow random crop | bool | False | True, False |\n\n#### Color Transform Parameters\n\n| Parameter | Description | Type | Default | Choices |\n| --- | --- | --- | --- | --- |\n| allow_gray_scale | Allow gray scale | bool | False | True, False |\n| allow_random_color | Allow random color | bool | False | True, False |\n\n#### Preprocessing Parameters\n\n| Parameter | Description | Type | Default | Choices |\n| --- | --- | --- | --- | --- |\n| mean | Mean of dataset | List[float] | / | Any list of floats |\n| std | Standard deviation of dataset | List[float] | / | Any list of floats |\n\n#### Optimizer Parameters\n\n| Parameter | Description | Type | Default | Choices |\n| --- | --- | --- | --- | --- |\n| name | Optimizer to use | str | / | sgd, adam, rmsprop, adamw |\n| lr | Learning rate | float | / | Any positive float |\n| weight_decay | Weight decay | float | 0.0 | Any non negative float |\n| momentum | Momentum | Optional[float] | None | Any non negative float |\n| alpha | Alpha | Optional[float] | None | Any non negative float |\n| betas | Betas | Optional[Tuple[float, float]] | None | Any tuple of non negative floats |\n\n#### Scheduler Parameters\n\n| Parameter | Description | Type | Default | Choices |\n| --- | --- | --- | --- | --- |\n| name | Scheduler to use | str | / | step, consine |\n| lr_min | Minimum learning rate | Optional[float] | None | Any non negative float |\n| step_size | Step size | Optional[int] | None | Any positive integer |\n| gamma | Gamma | Optional[float] | None | Any non negative float |\n\n#### Training Parameters\n\n| Parameter | Description | Type | Default | Choices |\n| --- | --- | --- | --- | --- |\n| name | Name of the experiment | str | / | Any string |\n| num_epochs | Number of epochs | int | / | Any positive integer |\n| trainset_dir | Path to training dataset | str | / | Any string |\n| valset_dir | Path to validation dataset | str | / | Any string |\n| testset_dir | Path to testing dataset | Optional[str] | None | Any string |\n| device | Device to use | str | cuda | cpu, cuda, etc. |\n| max_num_hrs | Maximum number of hours to run | Optional[float] | None | Any non negative float |\n| criterion | What criterion to use for best model | str | loss | loss, accuracy |\n| validate_every | Validate every n epochs | int | 1 | Any positive integer |\n| save_every | Save model every n epochs | int | 3 | Any positive integer |\n| patience | Patience for early stopping | int | 5 | Any positive integer |\n| random_seed | Random seed | int | 42 | Any positive integer |\n| precision | Precision | int | 64 | 16, 32, 64 |\n| export_last_as_onnx | Export last model as ONNX | bool | False | True, False |\n| export_best_as_onnx | Export best model as ONNX | bool | False | True, False |\n\n#### Evaluation Parameters\n\n| Parameter | Description | Type | Default | Choices |\n| --- | --- | --- | --- | --- |\n| name | Name of the experiment | str | / | Any string |\n| evalset_dir | Path to evaluation dataset | str | / | Any string |\n| device | Device to use | str | cuda | cpu, cuda, etc. |\n| precision | Precision | int | 64 | 16, 32, 64 |\n| random_seed | Random seed | int | 42 | Any positive integer |\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A PyTorch-based deep learning classifier training framework.",
"version": "1.1.0",
"project_urls": {
"Homepage": "https://github.com/wyhwong/classifier-trains",
"Repository": "https://github.com/wyhwong/classifier-trains"
},
"split_keywords": [
"pytorch",
" deep learning",
" classification",
" training",
" onnx",
" tensorboard",
" profiler"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "bd58a934d4a66635d54887a2e2666067b6d64ef262f0b22622353c39b59dbc92",
"md5": "a80cae1b69b44031ab5b85ac55849d13",
"sha256": "8308ec004a1d67fbcba1fd47f378679d9c12e7af32f6ca109b3232ed5d24e193"
},
"downloads": -1,
"filename": "classifier_trains-1.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "a80cae1b69b44031ab5b85ac55849d13",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.11",
"size": 24827,
"upload_time": "2024-09-18T17:00:37",
"upload_time_iso_8601": "2024-09-18T17:00:37.166569Z",
"url": "https://files.pythonhosted.org/packages/bd/58/a934d4a66635d54887a2e2666067b6d64ef262f0b22622353c39b59dbc92/classifier_trains-1.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7fc9b0ad10e5335a3a97462bc8ae5b87186c8a67966b7a689b2c4a9948751a1c",
"md5": "1c9e2d8c16a98c26ba4e2dff26d16f2b",
"sha256": "92f93dba00b76c7acb96e08dd807ad27b9719ac3bd550abc889f9c2fb81d906c"
},
"downloads": -1,
"filename": "classifier_trains-1.1.0.tar.gz",
"has_sig": false,
"md5_digest": "1c9e2d8c16a98c26ba4e2dff26d16f2b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.11",
"size": 19380,
"upload_time": "2024-09-18T17:00:38",
"upload_time_iso_8601": "2024-09-18T17:00:38.454951Z",
"url": "https://files.pythonhosted.org/packages/7f/c9/b0ad10e5335a3a97462bc8ae5b87186c8a67966b7a689b2c4a9948751a1c/classifier_trains-1.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-18 17:00:38",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "wyhwong",
"github_project": "classifier-trains",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "classifier_trains"
}
JSON
