| Name | onnx-perf-test JSON |
| Version |
0.7.0
JSON |
| download |
| home_page | None |
| Summary | A simple tool to measure the performance of ONNX models in Python. |
| upload_time | 2024-09-15 08:35:55 |
| maintainer | None |
| docs_url | None |
| author | Ibai Gorordo |
| requires_python | None |
| license | MIT |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# onnx-perf-test - ONNX Performance Test
A simple Python tool to measure the performance of ONNX models.
## Installation [](https://pypi.org/project/onnx-perf-test/)
```bash
pip install onnx-perf-test
```
- onnxruntime is installed with the package, for other execution providers, make sure to install the appropriate version of onnxruntime.
## Usage
```
onnx_perf_test {onnx_model} [--provider {provider}] [--num_runs {num_runs}] [--output_dir {output_dir}] [--draw] [--keep_profiling_file]
```
Arguments:
- `onnx_model`: Path to the ONNX model file.
- `--provider`: Provider to use for inferencing. Default is uses onnxruntime.get_available_providers() to get the available providers. **Options**: `TENSORRT`, `CUDA`, `CPU`...
- `--num_runs`: Number of runs to average the performance. Default is 10.
- `--output_dir`: Output directory to save the results. Does not save the results if not provided.
- `--draw`: Draw the performance graph. Requires `matplotlib` to be installed. Default is `False`.
- `--keep_profiling_file`: Keep the profiling file generated by onnxruntime. Default is `False`.
### Example
```bash
onnx_perf_test model.onnx --provider CUDA --num_runs 20 --output_dir results --draw
```
### Output
```bash
Running performance test on .\model.onnx with provider DEFAULT
Model inputs:
input.1 - Shape: [1, 3, 32, 32], Type: tensor(float)
Model outputs:
22 - Shape: [1, 10], Type: tensor(float)
Warming up the session...
Starting performance test...
Run 1: 0.53 ms
Run 2: 0.41 ms
Run 3: 0.39 ms
Run 4: 0.40 ms
Run 5: 0.39 ms
Run 6: 0.41 ms
Run 7: 0.39 ms
Run 8: 0.41 ms
Run 9: 0.42 ms
Run 10: 0.39 ms
model_run: 0.39530000000000004 ± 0.033970739042757235 ms
model_run: 0.39530000000000004 ± 0.033970739042757235 ms
SequentialExecutor::Execute: 0.3592 ± 0.02518288660534724 ms
SequentialExecutor::Execute: 0.3592 ± 0.02518288660534726 ms
FusedConv: 0.0977 ± 0.013367040726270638 ms
/conv1/Conv_kernel_time: 0.056999999999999995 ± 0.010749676997731395 ms
/conv2/Conv_kernel_time: 0.0407 ± 0.003301514803843834 ms
/conv2/Conv_fence_before: 0.0 ± 0.0 ms
/conv2/Conv_fence_after: 0.0 ± 0.0 ms
/conv1/Conv_fence_before: 0.0 ± 0.0 ms
/conv1/Conv_fence_after: 0.0 ± 0.0 ms
Gemm: 0.0967 ± 0.00279085809185793 ms
/fc1/Gemm_kernel_time: 0.038400000000000004 ± 0.0027568097504180435 ms
/fc3/Gemm_kernel_time: 0.029800000000000004 ± 0.0016193277068654818 ms
/fc2/Gemm_kernel_time: 0.028500000000000004 ± 0.0009718253158075512 ms
/fc2/Gemm_fence_before: 0.0 ± 0.0 ms
/fc3/Gemm_fence_before: 0.0 ± 0.0 ms
/fc3/Gemm_fence_after: 0.0 ± 0.0 ms
/fc2/Gemm_fence_after: 0.0 ± 0.0 ms
/fc1/Gemm_fence_after: 0.0 ± 0.0 ms
/fc1/Gemm_fence_before: 0.0 ± 0.0 ms
MaxPool: 0.049 ± 0.0024944382578492947 ms
/pool/MaxPool_kernel_time: 0.026600000000000002 ± 0.0015776212754932302 ms
/pool_1/MaxPool_kernel_time: 0.0224 ± 0.0014298407059684809 ms
/pool_1/MaxPool_fence_after: 0.0 ± 0.0 ms
/pool/MaxPool_fence_before: 0.0 ± 0.0 ms
/pool_1/MaxPool_fence_before: 0.0 ± 0.0 ms
/pool/MaxPool_fence_after: 0.0 ± 0.0 ms
Relu: 0.043199999999999995 ± 0.002699794230842212 ms
/Relu_3_kernel_time: 0.0217 ± 0.0018885620632287062 ms
/Relu_2_kernel_time: 0.0215 ± 0.001178511301977579 ms
/Relu_3_fence_before: 0.0 ± 0.0 ms
/Relu_3_fence_after: 0.0 ± 0.0 ms
/Relu_2_fence_before: 0.0 ± 0.0 ms
/Relu_2_fence_after: 0.0 ± 0.0 ms
Flatten: 0.0172 ± 0.0020439612955674524 ms
/Flatten_kernel_time: 0.0172 ± 0.0020439612955674524 ms
/Flatten_fence_after: 0.0 ± 0.0 ms
/Flatten_fence_before: 0.0 ± 0.0 ms
```
## Analyze ONNX Profiling File
Additionally, you can analyze your own `.json` profiling file generated by ONNXRuntime **using the `analyze_onnx_profiling.py` script.**
```bash
analyze_onnx_profile {onnx_profile_file} [--output_dir {output_dir}] [--draw]
```
Arguments:
- `onnx_profile_file`: Path to the ONNX profiling file.
- `--output_dir`: Output directory to save the results. Does not save the results if not provided.
- `--draw`: Draw the performance graph. Requires `matplotlib` to be installed. Default is `False`.
### Example
```bash
analyze_onnx_profile model_profile.json --output_dir results --draw
```
Raw data
{
"_id": null,
"home_page": null,
"name": "onnx-perf-test",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "Ibai Gorordo",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/64/4e/cf28243dac94a5c84587f854eef7b2c5ae56d77058c3cd3e9c93f14d243f/onnx_perf_test-0.7.0.tar.gz",
"platform": null,
"description": "# onnx-perf-test - ONNX Performance Test \r\n\r\nA simple Python tool to measure the performance of ONNX models.\r\n\r\n\r\n## Installation [](https://pypi.org/project/onnx-perf-test/)\r\n\r\n```bash\r\npip install onnx-perf-test\r\n```\r\n\r\n- onnxruntime is installed with the package, for other execution providers, make sure to install the appropriate version of onnxruntime.\r\n\r\n## Usage\r\n\r\n```\r\nonnx_perf_test {onnx_model} [--provider {provider}] [--num_runs {num_runs}] [--output_dir {output_dir}] [--draw] [--keep_profiling_file]\r\n```\r\n\r\nArguments:\r\n- `onnx_model`: Path to the ONNX model file.\r\n- `--provider`: Provider to use for inferencing. Default is uses onnxruntime.get_available_providers() to get the available providers. **Options**: `TENSORRT`, `CUDA`, `CPU`...\r\n- `--num_runs`: Number of runs to average the performance. Default is 10.\r\n- `--output_dir`: Output directory to save the results. Does not save the results if not provided.\r\n- `--draw`: Draw the performance graph. Requires `matplotlib` to be installed. Default is `False`.\r\n- `--keep_profiling_file`: Keep the profiling file generated by onnxruntime. Default is `False`.\r\n\r\n### Example\r\n\r\n```bash\r\nonnx_perf_test model.onnx --provider CUDA --num_runs 20 --output_dir results --draw\r\n```\r\n\r\n### Output\r\n```bash\r\nRunning performance test on .\\model.onnx with provider DEFAULT\r\nModel inputs:\r\n input.1 - Shape: [1, 3, 32, 32], Type: tensor(float)\r\nModel outputs:\r\n 22 - Shape: [1, 10], Type: tensor(float)\r\n\r\nWarming up the session...\r\nStarting performance test...\r\nRun 1: 0.53 ms\r\nRun 2: 0.41 ms\r\nRun 3: 0.39 ms\r\nRun 4: 0.40 ms\r\nRun 5: 0.39 ms\r\nRun 6: 0.41 ms\r\nRun 7: 0.39 ms\r\nRun 8: 0.41 ms\r\nRun 9: 0.42 ms\r\nRun 10: 0.39 ms\r\n\r\nmodel_run: 0.39530000000000004 \u00b1 0.033970739042757235 ms\r\n model_run: 0.39530000000000004 \u00b1 0.033970739042757235 ms\r\n\r\nSequentialExecutor::Execute: 0.3592 \u00b1 0.02518288660534724 ms\r\n SequentialExecutor::Execute: 0.3592 \u00b1 0.02518288660534726 ms\r\n\r\nFusedConv: 0.0977 \u00b1 0.013367040726270638 ms\r\n /conv1/Conv_kernel_time: 0.056999999999999995 \u00b1 0.010749676997731395 ms\r\n /conv2/Conv_kernel_time: 0.0407 \u00b1 0.003301514803843834 ms\r\n /conv2/Conv_fence_before: 0.0 \u00b1 0.0 ms\r\n /conv2/Conv_fence_after: 0.0 \u00b1 0.0 ms\r\n /conv1/Conv_fence_before: 0.0 \u00b1 0.0 ms\r\n /conv1/Conv_fence_after: 0.0 \u00b1 0.0 ms\r\n\r\nGemm: 0.0967 \u00b1 0.00279085809185793 ms\r\n /fc1/Gemm_kernel_time: 0.038400000000000004 \u00b1 0.0027568097504180435 ms\r\n /fc3/Gemm_kernel_time: 0.029800000000000004 \u00b1 0.0016193277068654818 ms\r\n /fc2/Gemm_kernel_time: 0.028500000000000004 \u00b1 0.0009718253158075512 ms\r\n /fc2/Gemm_fence_before: 0.0 \u00b1 0.0 ms\r\n /fc3/Gemm_fence_before: 0.0 \u00b1 0.0 ms\r\n /fc3/Gemm_fence_after: 0.0 \u00b1 0.0 ms\r\n /fc2/Gemm_fence_after: 0.0 \u00b1 0.0 ms\r\n /fc1/Gemm_fence_after: 0.0 \u00b1 0.0 ms\r\n /fc1/Gemm_fence_before: 0.0 \u00b1 0.0 ms\r\n\r\nMaxPool: 0.049 \u00b1 0.0024944382578492947 ms\r\n /pool/MaxPool_kernel_time: 0.026600000000000002 \u00b1 0.0015776212754932302 ms\r\n /pool_1/MaxPool_kernel_time: 0.0224 \u00b1 0.0014298407059684809 ms\r\n /pool_1/MaxPool_fence_after: 0.0 \u00b1 0.0 ms\r\n /pool/MaxPool_fence_before: 0.0 \u00b1 0.0 ms\r\n /pool_1/MaxPool_fence_before: 0.0 \u00b1 0.0 ms\r\n /pool/MaxPool_fence_after: 0.0 \u00b1 0.0 ms\r\n\r\nRelu: 0.043199999999999995 \u00b1 0.002699794230842212 ms\r\n /Relu_3_kernel_time: 0.0217 \u00b1 0.0018885620632287062 ms\r\n /Relu_2_kernel_time: 0.0215 \u00b1 0.001178511301977579 ms\r\n /Relu_3_fence_before: 0.0 \u00b1 0.0 ms\r\n /Relu_3_fence_after: 0.0 \u00b1 0.0 ms\r\n /Relu_2_fence_before: 0.0 \u00b1 0.0 ms\r\n /Relu_2_fence_after: 0.0 \u00b1 0.0 ms\r\n\r\nFlatten: 0.0172 \u00b1 0.0020439612955674524 ms\r\n /Flatten_kernel_time: 0.0172 \u00b1 0.0020439612955674524 ms\r\n /Flatten_fence_after: 0.0 \u00b1 0.0 ms\r\n /Flatten_fence_before: 0.0 \u00b1 0.0 ms\r\n\r\n\r\n```\r\n\r\n## Analyze ONNX Profiling File\r\nAdditionally, you can analyze your own `.json` profiling file generated by ONNXRuntime **using the `analyze_onnx_profiling.py` script.**\r\n\r\n```bash\r\nanalyze_onnx_profile {onnx_profile_file} [--output_dir {output_dir}] [--draw]\r\n```\r\n\r\nArguments:\r\n- `onnx_profile_file`: Path to the ONNX profiling file.\r\n- `--output_dir`: Output directory to save the results. Does not save the results if not provided.\r\n- `--draw`: Draw the performance graph. Requires `matplotlib` to be installed. Default is `False`.\r\n\r\n### Example\r\n\r\n```bash\r\nanalyze_onnx_profile model_profile.json --output_dir results --draw\r\n```\r\n\r\n\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A simple tool to measure the performance of ONNX models in Python.",
"version": "0.7.0",
"project_urls": {
"Homepage": "https://github.com/ibaiGorordo/onnx-perf-test"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "83bd174a4cfa8b046a36386c6cb1e5b299366d009b6b284c39d7e02ab4a068ce",
"md5": "6601817decb039bf039b43f8632bdcce",
"sha256": "dc240d25c0a83493d2f8a606077cc60d8856d3c347c875b9e87ce14badff8b79"
},
"downloads": -1,
"filename": "onnx_perf_test-0.7.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6601817decb039bf039b43f8632bdcce",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 7993,
"upload_time": "2024-09-15T08:35:53",
"upload_time_iso_8601": "2024-09-15T08:35:53.775402Z",
"url": "https://files.pythonhosted.org/packages/83/bd/174a4cfa8b046a36386c6cb1e5b299366d009b6b284c39d7e02ab4a068ce/onnx_perf_test-0.7.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "644ecf28243dac94a5c84587f854eef7b2c5ae56d77058c3cd3e9c93f14d243f",
"md5": "c12444d420141e48274d39bdce08466a",
"sha256": "d761e1a4003a95dd5183d7e85df413976729e83e4fc4dd0155ab94c2d482d44e"
},
"downloads": -1,
"filename": "onnx_perf_test-0.7.0.tar.gz",
"has_sig": false,
"md5_digest": "c12444d420141e48274d39bdce08466a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 7033,
"upload_time": "2024-09-15T08:35:55",
"upload_time_iso_8601": "2024-09-15T08:35:55.332006Z",
"url": "https://files.pythonhosted.org/packages/64/4e/cf28243dac94a5c84587f854eef7b2c5ae56d77058c3cd3e9c93f14d243f/onnx_perf_test-0.7.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-15 08:35:55",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ibaiGorordo",
"github_project": "onnx-perf-test",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "onnx-perf-test"
}
