| Name | spo4onnx JSON |
| Version |
1.0.5
JSON |
| download |
| home_page | https://github.com/PINTO0309/spo4onnx |
| Summary | Simple tool for partial optimization of ONNX. Further optimize some models that cannot be optimized with onnx-optimizer and onnxsim by several tens of percent. In particular, models containing Einsum and OneHot. |
| upload_time | 2024-05-07 06:26:24 |
| maintainer | None |
| docs_url | None |
| author | Katsuya Hyodo |
| requires_python | >=3.6 |
| license | MIT License |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# spo4onnx
Simple tool for partial optimization of ONNX.
Further optimize some models that cannot be optimized with [onnx-optimizer](https://github.com/onnx/optimizer) and [onnxsim](https://github.com/daquexian/onnx-simplifier) by several tens of percent. In particular, models containing `Einsum` and `OneHot`. In other words, the goal is to raise the optimization capacity of [onnxsim](https://github.com/daquexian/onnx-simplifier). **In the first place, the "optimization" as it is commonly called can be easily performed, since `Einsum` itself is all separated into `MatMul` if the model is built using the [opt-einsum](https://github.com/dgasmith/opt_einsum) package. What this tool aims to do, however, is to optimize ONNX without generating many `MatMul`, leaving `Einsum` as it is.** The only benefit you get with spo4onnx is that you can rejoice that you are a geek.
https://github.com/PINTO0309/simple-onnx-processing-tools
[](https://pepy.tech/project/spo4onnx)  [](https://pypi.org/project/spo4onnx/) [](https://github.com/PINTO0309/spo4onnx/actions?query=workflow%3ACodeQL)
<p align="center">
<img src="https://github.com/PINTO0309/spo4onnx/assets/33194443/0db36d5a-e3b7-4592-a176-3c02b0b8f77f" />
</p>
```
pip install -U spo4onnx \
&& pip install -U onnx \
&& pip install -U onnxruntime \
&& pip install onnxsim \
&& pip install tqdm==4.66.1 \
&& python3 -m pip install -U onnx_graphsurgeon --index-url https://pypi.ngc.nvidia.com
```
https://github.com/PINTO0309/spo4onnx/releases/download/model/wd-v1-4-moat-tagger-v2.onnx
```bash
spo4onnx -if wd-v1-4-moat-tagger-v2.onnx
```
1. Temporarily downgrade onnxsim to `0.4.30` to perform my own optimization sequence.
2. After the optimization process is complete, reinstall the original onnxsim version to restore the environment.
3. The first version modifies two OPs, `Einsum` and `OneHot`, which hinder optimization and boost the optimization operation by onnxsim to maximum performance.
4. Not all models will be effective, but the larger and more complex the structure and the larger the model, the more effective this unique optimization behavior will be.
5. Processing models with this tool that contain OPs with non-deterministic output shapes, such as `NonZero` or `NonMaxSuppression`, will break the model.
6. I have already identified models that can reduce redundant operations by up to 30%-60%.
7. An example of the most extreme optimization of my model is shown in the figure below. Example of optimization from 9,988 OP to 3,927 OP. The assumption is that this is an example of a huge ONNX with undefined Hieght and Width dimensions, set to fixed resolution and my special optimization technique applied. By making OPs such as `Tile` disappear and embedded in the model as INT64 constants, the final model file size is increased, but the model structure is greatly optimized.
https://github.com/PINTO0309/spo4onnx/releases/download/model/high_frequency_stereo_matching_kitti_iter01_1x3xHxW.onnx
Verify that the inference works properly.
```bash
sit4onnx \
-if high_frequency_stereo_matching_kitti_iter05_1x3x192x320.onnx \
-oep tensorrt
INFO: file: high_frequency_stereo_matching_kitti_iter05_1x3x192x320.onnx
INFO: providers: ['TensorrtExecutionProvider', 'CPUExecutionProvider']
INFO: input_name.1: left shape: [1, 3, 192, 320] dtype: float32
INFO: input_name.2: right shape: [1, 3, 192, 320] dtype: float32
INFO: test_loop_count: 10
INFO: total elapsed time: 185.7011318206787 ms
INFO: avg elapsed time per pred: 18.57011318206787 ms
INFO: output_name.1: output shape: [1, 1, 192, 320] dtype: float32
sit4onnx \
-if high_frequency_stereo_matching_kitti_iter05_1x3x192x320.onnx \
-oep cpu
INFO: file: high_frequency_stereo_matching_kitti_iter05_1x3x192x320.onnx
INFO: providers: ['CPUExecutionProvider']
INFO: input_name.1: left shape: [1, 3, 192, 320] dtype: float32
INFO: input_name.2: right shape: [1, 3, 192, 320] dtype: float32
INFO: test_loop_count: 10
INFO: total elapsed time: 4090.1401042938232 ms
INFO: avg elapsed time per pred: 409.0140104293823 ms
INFO: output_name.1: output shape: [1, 1, 192, 320] dtype: float32
```
## 1. CLI Usage
```
spo4onnx -h
spo4onnx \
[-h] \
-if INPUT_ONNX_FILE_PATH \
[-of OUTPUT_ONNX_FILE_PATH] \
[-ois OVERWRITE_INPUT_SHAPE [OVERWRITE_INPUT_SHAPE ...]] \
[-ot OPTIMIZATION_TIMES] \
[-tov TARGET_ONNXSIM_VERSION]
[-n]
options:
-h, --help
show this help message and exit
-if INPUT_ONNX_FILE_PATH, --input_onnx_file_path INPUT_ONNX_FILE_PATH
Input onnx file path.
-of OUTPUT_ONNX_FILE_PATH, --output_onnx_file_path OUTPUT_ONNX_FILE_PATH
Output onnx file path. If not specified,
it will overwrite the onnx specified in --input_onnx_file_path.
-ois OVERWRITE_INPUT_SHAPE [OVERWRITE_INPUT_SHAPE ...], \
--overwrite_input_shape OVERWRITE_INPUT_SHAPE [OVERWRITE_INPUT_SHAPE ...]
Overwrite the input shape.
The format is "input_1:dim0,...,dimN" "input_2:dim0,...,dimN" "input_3:dim0,...,dimN"
When there is only one input, for example, "data:1,3,224,224"
When there are multiple inputs, for example, "data1:1,3,224,224" "data2:1,3,112,112" "data3:5"
A value of 1 or more must be specified.
Numerical values other than dynamic dimensions are ignored.
-ot OPTIMIZATION_TIMES, --optimization_times OPTIMIZATION_TIMES
Number of times the optimization process is performed.
If zero is specified, the tool automatically calculates the number of optimization times.
Default: 0
-tov TARGET_ONNXSIM_VERSION, --target_onnxsim_version TARGET_ONNXSIM_VERSION
Version number of the onnxsim used for optimization.
Default: 0.4.30
-n, --non_verbose
Do not show all information logs. Only error logs are displayed.
```
## 2. In-script Usage
```python
$ python
>>> from spo4onnx import partial_optimization
>>> help(partial_optimization)
Help on function partial_optimization in module spo4onnx.onnx_partial_optimization:
partial_optimization(
input_onnx_file_path: Optional[str] = '',
onnx_graph: Optional[onnx.onnx_ml_pb2.ModelProto] = None,
output_onnx_file_path: Optional[str] = '',
overwrite_input_shape: Optional[Dict] = None,
optimization_times: Optional[int] = 0,
target_onnxsim_version: Optional[str] = '0.4.30',
non_verbose: Optional[bool] = False,
) -> onnx.onnx_ml_pb2.ModelProto
Parameters
----------
input_onnx_file_path: Optional[str]
Input onnx file path.
Either input_onnx_file_path or onnx_graph must be specified.
Default: ''
onnx_graph: Optional[onnx.ModelProto]
onnx.ModelProto.
Either input_onnx_file_path or onnx_graph must be specified.
onnx_graph If specified, ignore input_onnx_file_path and process onnx_graph.
output_onnx_file_path: Optional[str]
Output onnx file path.
If not specified, it will overwrite the onnx specified in --input_onnx_file_path.
overwrite_input_shape: Optional[Dict]
Overwrite the input shape.
The format is
{'data1': [1, 3, 224, 224], 'data2': [1, 224], 'data3': [1]}
optimization_times: Optional[int]
Number of times the optimization process is performed.
If zero is specified, the tool automatically calculates the number of optimization times.
Default: 0
non_verbose: Optional[bool]
Do not show all information logs. Only error logs are displayed.
Returns
-------
onnx_graph: onnx.ModelProto
Optimized onnx ModelProto
```
## 3. Acknowledgments
1. https://github.com/daquexian/onnx-simplifier
2. https://github.com/onnx/optimizer
3. https://github.com/dgasmith/opt_einsum
Raw data
{
"_id": null,
"home_page": "https://github.com/PINTO0309/spo4onnx",
"name": "spo4onnx",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": null,
"author": "Katsuya Hyodo",
"author_email": "rmsdh122@yahoo.co.jp",
"download_url": "https://files.pythonhosted.org/packages/20/e5/bb018f64cc98ce8e1b91f2a65492904418d117cf9b7c426e8b5d2cde898e/spo4onnx-1.0.5.tar.gz",
"platform": "linux",
"description": "# spo4onnx\nSimple tool for partial optimization of ONNX.\n\nFurther optimize some models that cannot be optimized with [onnx-optimizer](https://github.com/onnx/optimizer) and [onnxsim](https://github.com/daquexian/onnx-simplifier) by several tens of percent. In particular, models containing `Einsum` and `OneHot`. In other words, the goal is to raise the optimization capacity of [onnxsim](https://github.com/daquexian/onnx-simplifier). **In the first place, the \"optimization\" as it is commonly called can be easily performed, since `Einsum` itself is all separated into `MatMul` if the model is built using the [opt-einsum](https://github.com/dgasmith/opt_einsum) package. What this tool aims to do, however, is to optimize ONNX without generating many `MatMul`, leaving `Einsum` as it is.** The only benefit you get with spo4onnx is that you can rejoice that you are a geek.\n\nhttps://github.com/PINTO0309/simple-onnx-processing-tools\n\n[](https://pepy.tech/project/spo4onnx)  [](https://pypi.org/project/spo4onnx/) [](https://github.com/PINTO0309/spo4onnx/actions?query=workflow%3ACodeQL)\n\n<p align=\"center\">\n <img src=\"https://github.com/PINTO0309/spo4onnx/assets/33194443/0db36d5a-e3b7-4592-a176-3c02b0b8f77f\" />\n</p>\n\n```\npip install -U spo4onnx \\\n&& pip install -U onnx \\\n&& pip install -U onnxruntime \\\n&& pip install onnxsim \\\n&& pip install tqdm==4.66.1 \\\n&& python3 -m pip install -U onnx_graphsurgeon --index-url https://pypi.ngc.nvidia.com\n```\n\nhttps://github.com/PINTO0309/spo4onnx/releases/download/model/wd-v1-4-moat-tagger-v2.onnx\n\n```bash\nspo4onnx -if wd-v1-4-moat-tagger-v2.onnx\n```\n\n\n\n1. Temporarily downgrade onnxsim to `0.4.30` to perform my own optimization sequence.\n2. After the optimization process is complete, reinstall the original onnxsim version to restore the environment.\n3. The first version modifies two OPs, `Einsum` and `OneHot`, which hinder optimization and boost the optimization operation by onnxsim to maximum performance.\n4. Not all models will be effective, but the larger and more complex the structure and the larger the model, the more effective this unique optimization behavior will be.\n5. Processing models with this tool that contain OPs with non-deterministic output shapes, such as `NonZero` or `NonMaxSuppression`, will break the model.\n6. I have already identified models that can reduce redundant operations by up to 30%-60%.\n7. An example of the most extreme optimization of my model is shown in the figure below. Example of optimization from 9,988 OP to 3,927 OP. The assumption is that this is an example of a huge ONNX with undefined Hieght and Width dimensions, set to fixed resolution and my special optimization technique applied. By making OPs such as `Tile` disappear and embedded in the model as INT64 constants, the final model file size is increased, but the model structure is greatly optimized.\n\nhttps://github.com/PINTO0309/spo4onnx/releases/download/model/high_frequency_stereo_matching_kitti_iter01_1x3xHxW.onnx\n\n\n\n\n\n\n\nVerify that the inference works properly.\n```bash\nsit4onnx \\\n-if high_frequency_stereo_matching_kitti_iter05_1x3x192x320.onnx \\\n-oep tensorrt\n\nINFO: file: high_frequency_stereo_matching_kitti_iter05_1x3x192x320.onnx\nINFO: providers: ['TensorrtExecutionProvider', 'CPUExecutionProvider']\nINFO: input_name.1: left shape: [1, 3, 192, 320] dtype: float32\nINFO: input_name.2: right shape: [1, 3, 192, 320] dtype: float32\nINFO: test_loop_count: 10\nINFO: total elapsed time: 185.7011318206787 ms\nINFO: avg elapsed time per pred: 18.57011318206787 ms\nINFO: output_name.1: output shape: [1, 1, 192, 320] dtype: float32\n\nsit4onnx \\\n-if high_frequency_stereo_matching_kitti_iter05_1x3x192x320.onnx \\\n-oep cpu\n\nINFO: file: high_frequency_stereo_matching_kitti_iter05_1x3x192x320.onnx\nINFO: providers: ['CPUExecutionProvider']\nINFO: input_name.1: left shape: [1, 3, 192, 320] dtype: float32\nINFO: input_name.2: right shape: [1, 3, 192, 320] dtype: float32\nINFO: test_loop_count: 10\nINFO: total elapsed time: 4090.1401042938232 ms\nINFO: avg elapsed time per pred: 409.0140104293823 ms\nINFO: output_name.1: output shape: [1, 1, 192, 320] dtype: float32\n```\n\n## 1. CLI Usage\n```\nspo4onnx -h\n\nspo4onnx \\\n[-h] \\\n-if INPUT_ONNX_FILE_PATH \\\n[-of OUTPUT_ONNX_FILE_PATH] \\\n[-ois OVERWRITE_INPUT_SHAPE [OVERWRITE_INPUT_SHAPE ...]] \\\n[-ot OPTIMIZATION_TIMES] \\\n[-tov TARGET_ONNXSIM_VERSION]\n[-n]\n\noptions:\n -h, --help\n show this help message and exit\n\n -if INPUT_ONNX_FILE_PATH, --input_onnx_file_path INPUT_ONNX_FILE_PATH\n Input onnx file path.\n\n -of OUTPUT_ONNX_FILE_PATH, --output_onnx_file_path OUTPUT_ONNX_FILE_PATH\n Output onnx file path. If not specified,\n it will overwrite the onnx specified in --input_onnx_file_path.\n\n -ois OVERWRITE_INPUT_SHAPE [OVERWRITE_INPUT_SHAPE ...], \\\n --overwrite_input_shape OVERWRITE_INPUT_SHAPE [OVERWRITE_INPUT_SHAPE ...]\n Overwrite the input shape.\n The format is \"input_1:dim0,...,dimN\" \"input_2:dim0,...,dimN\" \"input_3:dim0,...,dimN\"\n When there is only one input, for example, \"data:1,3,224,224\"\n When there are multiple inputs, for example, \"data1:1,3,224,224\" \"data2:1,3,112,112\" \"data3:5\"\n A value of 1 or more must be specified.\n Numerical values other than dynamic dimensions are ignored.\n\n -ot OPTIMIZATION_TIMES, --optimization_times OPTIMIZATION_TIMES\n Number of times the optimization process is performed.\n If zero is specified, the tool automatically calculates the number of optimization times.\n Default: 0\n\n -tov TARGET_ONNXSIM_VERSION, --target_onnxsim_version TARGET_ONNXSIM_VERSION\n Version number of the onnxsim used for optimization.\n Default: 0.4.30\n\n -n, --non_verbose\n Do not show all information logs. Only error logs are displayed.\n```\n\n## 2. In-script Usage\n```python\n$ python\n>>> from spo4onnx import partial_optimization\n>>> help(partial_optimization)\n\nHelp on function partial_optimization in module spo4onnx.onnx_partial_optimization:\n\npartial_optimization(\n input_onnx_file_path: Optional[str] = '',\n onnx_graph: Optional[onnx.onnx_ml_pb2.ModelProto] = None,\n output_onnx_file_path: Optional[str] = '',\n overwrite_input_shape: Optional[Dict] = None,\n optimization_times: Optional[int] = 0,\n target_onnxsim_version: Optional[str] = '0.4.30',\n non_verbose: Optional[bool] = False,\n) -> onnx.onnx_ml_pb2.ModelProto\n\n Parameters\n ----------\n input_onnx_file_path: Optional[str]\n Input onnx file path.\n Either input_onnx_file_path or onnx_graph must be specified.\n Default: ''\n\n onnx_graph: Optional[onnx.ModelProto]\n onnx.ModelProto.\n Either input_onnx_file_path or onnx_graph must be specified.\n onnx_graph If specified, ignore input_onnx_file_path and process onnx_graph.\n\n output_onnx_file_path: Optional[str]\n Output onnx file path.\n If not specified, it will overwrite the onnx specified in --input_onnx_file_path.\n\n overwrite_input_shape: Optional[Dict]\n Overwrite the input shape.\n The format is\n {'data1': [1, 3, 224, 224], 'data2': [1, 224], 'data3': [1]}\n\n optimization_times: Optional[int]\n Number of times the optimization process is performed.\n If zero is specified, the tool automatically calculates the number of optimization times.\n Default: 0\n\n non_verbose: Optional[bool]\n Do not show all information logs. Only error logs are displayed.\n\n Returns\n -------\n onnx_graph: onnx.ModelProto\n Optimized onnx ModelProto\n```\n\n## 3. Acknowledgments\n1. https://github.com/daquexian/onnx-simplifier\n2. https://github.com/onnx/optimizer\n3. https://github.com/dgasmith/opt_einsum\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "Simple tool for partial optimization of ONNX. Further optimize some models that cannot be optimized with onnx-optimizer and onnxsim by several tens of percent. In particular, models containing Einsum and OneHot.",
"version": "1.0.5",
"project_urls": {
"Homepage": "https://github.com/PINTO0309/spo4onnx"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "96a02bcda978f76fb5a76c48b109ab17686267daedb1b712fb6a23d43fe9e6b6",
"md5": "b98098c7d1dda9b46a55e9ea2d76ee8d",
"sha256": "3275651f14b33fbb4fe6a3b8dd196acd6dd6ac5aa49829c129b1b487c2cb8c99"
},
"downloads": -1,
"filename": "spo4onnx-1.0.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b98098c7d1dda9b46a55e9ea2d76ee8d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 11757,
"upload_time": "2024-05-07T06:26:23",
"upload_time_iso_8601": "2024-05-07T06:26:23.471553Z",
"url": "https://files.pythonhosted.org/packages/96/a0/2bcda978f76fb5a76c48b109ab17686267daedb1b712fb6a23d43fe9e6b6/spo4onnx-1.0.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "20e5bb018f64cc98ce8e1b91f2a65492904418d117cf9b7c426e8b5d2cde898e",
"md5": "0495bd5948f2d2eb6ae4221b0204234c",
"sha256": "dd31146688045ec1dfc09de04f7718b51aa2400310958fac3ca4dda336ef1d7e"
},
"downloads": -1,
"filename": "spo4onnx-1.0.5.tar.gz",
"has_sig": false,
"md5_digest": "0495bd5948f2d2eb6ae4221b0204234c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 13107,
"upload_time": "2024-05-07T06:26:24",
"upload_time_iso_8601": "2024-05-07T06:26:24.611474Z",
"url": "https://files.pythonhosted.org/packages/20/e5/bb018f64cc98ce8e1b91f2a65492904418d117cf9b7c426e8b5d2cde898e/spo4onnx-1.0.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-07 06:26:24",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "PINTO0309",
"github_project": "spo4onnx",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "spo4onnx"
}
