| Name | onnx-extended JSON |
| Version |
0.3.0
JSON |
| download |
| home_page | https://github.com/sdpython/onnx-extended |
| Summary | Extends the list of supported operators in onnx reference implementation and onnxruntime, or implements faster versions in C++. |
| upload_time | 2024-09-19 22:23:54 |
| maintainer | None |
| docs_url | None |
| author | Xavier Dupré |
| requires_python | >=3.10 |
| license | Copyright (c) 2023-2024, Xavier Dupré Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
onnx
onnxruntime
cuda
openmp
cmake
cython
pybind11
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
numpy
onnx
scipy
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
.. image:: https://github.com/sdpython/onnx-extended/raw/main/_doc/_static/logo.png
:width: 120
onnx-extended: extensions for onnx and onnxruntime
==================================================
.. image:: https://dev.azure.com/xavierdupre3/onnx-extended/_apis/build/status/sdpython.onnx-extended
:target: https://dev.azure.com/xavierdupre3/onnx-extended/
.. image:: https://badge.fury.io/py/onnx-extended.svg
:target: http://badge.fury.io/py/onnx-extended
.. image:: http://img.shields.io/github/issues/sdpython/onnx-extended.png
:alt: GitHub Issues
:target: https://github.com/sdpython/onnx-extended/issues
.. image:: https://img.shields.io/badge/license-MIT-blue.svg
:alt: MIT License
:target: https://opensource.org/license/MIT/
.. image:: https://img.shields.io/github/repo-size/sdpython/onnx-extended
:target: https://github.com/sdpython/onnx-extended/
:alt: size
.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
:target: https://github.com/psf/black
**onnx-extended** extends the list of supported operators in onnx
reference implementation and `onnxruntime
<https://github.com/microsoft/onnxruntime>`_,
or implements faster versions in C++.
Documentation `onnx-extended
<https://sdpython.github.io/doc/onnx-extended/dev/>`_.
Source are available on `github/onnx-extended
<https://github.com/sdpython/onnx-extended/>`_.
Use a C++ implementation of existing operators
++++++++++++++++++++++++++++++++++++++++++++++
.. code-block:: python
import timeit
import numpy as np
from onnx import TensorProto
from onnx.helper import (
make_graph,
make_model,
make_node,
make_opsetid,
make_tensor_value_info,
)
from onnx.reference import ReferenceEvaluator
from onnxruntime import InferenceSession
from onnx_extended.ext_test_case import measure_time
from onnx_extended.reference import CReferenceEvaluator
X = make_tensor_value_info("X", TensorProto.FLOAT, [None, None, None, None])
Y = make_tensor_value_info("Y", TensorProto.FLOAT, [None, None, None, None])
B = make_tensor_value_info("B", TensorProto.FLOAT, [None, None, None, None])
W = make_tensor_value_info("W", TensorProto.FLOAT, [None, None, None, None])
node = make_node(
"Conv",
["X", "W", "B"],
["Y"],
pads=[1, 1, 1, 1],
dilations=[1, 1],
strides=[2, 2],
)
graph = make_graph([node], "g", [X, W, B], [Y])
onnx_model = make_model(graph, opset_imports=[make_opsetid("", 16)])
sH, sW = 64, 64
X = np.arange(sW * sH).reshape((1, 1, sH, sW)).astype(np.float32)
W = np.ones((1, 1, 3, 3), dtype=np.float32)
B = np.array([[[[0]]]], dtype=np.float32)
sess1 = ReferenceEvaluator(onnx_model)
sess2 = CReferenceEvaluator(onnx_model) # 100 times faster
expected = sess1.run(None, {"X": X, "W": W, "B": B})[0]
got = sess2.run(None, {"X": X, "W": W, "B": B})[0]
diff = np.abs(expected - got).max()
print(f"difference: {diff}")
f1 = lambda: sess1.run(None, {"X": X, "W": W, "B": B})[0]
f2 = lambda: sess2.run(None, {"X": X, "W": W, "B": B})[0]
print("onnx:", timeit.timeit(f1, globals=globals(), number=5))
print("onnx-extended:", timeit.timeit(f2, globals=globals(), number=5))
::
difference: 0.0
onnx: 0.024006774998269975
onnx-extended: 0.0002316169993719086
Build with CUDA, openmp, eigen, onnxruntime
+++++++++++++++++++++++++++++++++++++++++++
The package also contains some dummy examples on how to
build with C++ functions (`pybind11 <https://github.com/pybind/pybind11>`_,
`cython <https://cython.org/>`_),
with `openmp <https://www.openmp.org/>`_,
`eigen <https://eigen.tuxfamily.org/index.php>`_
with or without CUDA. It also shows how to create a custom operator
for *onnxruntime* in C++.
The version released on `pypi/onnx-extended <https://pypi.org/project/onnx-extended/>`_
only works on CPU. It needs to be manually built to enable
the code using CUDA. The build will automatically link with CUDA if it is found.
If not, some extensions might not be available.
::
python setup.py build_ext --inplace
# pip install -e .
It is possible to use a specific version of CUDA:
::
python setup.py build_ext --inplace --cuda-version=11.8
# or (not working yet)
# pip install -e . --config-settings="--cuda-version=11.8"
# pip install -e . --global-option="--cuda-version=11.8"
export USE_CUDA=11.8
pip install -e .
`NVTX <https://github.com/NVIDIA/NVTX>`_
can be enabled with the following command:
::
python setup.py build_ext --inplace --use_nvtx 1
# or (not working yet)
# pip install -e . --config-settings="--use_nvtx=1"
pip install -e . --global-option "--use_nvtx=1"
Experimental cython binding for onnxruntime
+++++++++++++++++++++++++++++++++++++++++++
The python onnxruntime package relies on pybind11 to expose
its functionalities. *onnx-extended* tries to build a cython wrapper
around the C/C++ API of onnxruntime. cython relies on python C API
and is faster than pybind11. This different may be significant when
onnxruntime is used on small graphs and tensors.
Custom kernels for onnxruntime
++++++++++++++++++++++++++++++
onnxruntime provides an API to add custom implementation
for existing or new onnx operators. An example for CPU.
::
from onnxruntime import InferenceSession, SessionOptions
from onnx_extended.ortops.optim.cpu import get_ort_ext_libs
r = get_ort_ext_libs()
opts = SessionOptions()
if r is not None:
opts.register_custom_ops_library(r[0])
sess_cus = InferenceSession(
onx_modified.SerializeToString(), opts, providers=["CPUExecutionProvider"]
)
Raw data
{
"_id": null,
"home_page": "https://github.com/sdpython/onnx-extended",
"name": "onnx-extended",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "onnx, onnxruntime, CUDA, openmp, cmake, cython, pybind11",
"author": "Xavier Dupr\u00e9",
"author_email": "Xavier Dupr\u00e9 <xavier.dupre@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/cb/7d/cd45358a907c8cd6de3db91e8881e1d5915292ffbdf3365705e0e5b48960/onnx_extended-0.3.0.tar.gz",
"platform": null,
"description": "\n.. image:: https://github.com/sdpython/onnx-extended/raw/main/_doc/_static/logo.png\n :width: 120\n\nonnx-extended: extensions for onnx and onnxruntime\n==================================================\n\n.. image:: https://dev.azure.com/xavierdupre3/onnx-extended/_apis/build/status/sdpython.onnx-extended\n :target: https://dev.azure.com/xavierdupre3/onnx-extended/\n.. image:: https://badge.fury.io/py/onnx-extended.svg\n :target: http://badge.fury.io/py/onnx-extended\n.. image:: http://img.shields.io/github/issues/sdpython/onnx-extended.png\n :alt: GitHub Issues\n :target: https://github.com/sdpython/onnx-extended/issues\n.. image:: https://img.shields.io/badge/license-MIT-blue.svg\n :alt: MIT License\n :target: https://opensource.org/license/MIT/\n.. image:: https://img.shields.io/github/repo-size/sdpython/onnx-extended\n :target: https://github.com/sdpython/onnx-extended/\n :alt: size\n.. image:: https://img.shields.io/badge/code%20style-black-000000.svg\n :target: https://github.com/psf/black\n\n**onnx-extended** extends the list of supported operators in onnx\nreference implementation and `onnxruntime\n<https://github.com/microsoft/onnxruntime>`_,\nor implements faster versions in C++.\nDocumentation `onnx-extended\n<https://sdpython.github.io/doc/onnx-extended/dev/>`_.\nSource are available on `github/onnx-extended\n<https://github.com/sdpython/onnx-extended/>`_.\n\nUse a C++ implementation of existing operators\n++++++++++++++++++++++++++++++++++++++++++++++\n\n.. code-block:: python\n\n import timeit\n import numpy as np\n from onnx import TensorProto\n from onnx.helper import (\n make_graph,\n make_model,\n make_node,\n make_opsetid,\n make_tensor_value_info,\n )\n from onnx.reference import ReferenceEvaluator\n from onnxruntime import InferenceSession\n from onnx_extended.ext_test_case import measure_time\n from onnx_extended.reference import CReferenceEvaluator\n\n\n X = make_tensor_value_info(\"X\", TensorProto.FLOAT, [None, None, None, None])\n Y = make_tensor_value_info(\"Y\", TensorProto.FLOAT, [None, None, None, None])\n B = make_tensor_value_info(\"B\", TensorProto.FLOAT, [None, None, None, None])\n W = make_tensor_value_info(\"W\", TensorProto.FLOAT, [None, None, None, None])\n node = make_node(\n \"Conv\",\n [\"X\", \"W\", \"B\"],\n [\"Y\"],\n pads=[1, 1, 1, 1],\n dilations=[1, 1],\n strides=[2, 2],\n )\n graph = make_graph([node], \"g\", [X, W, B], [Y])\n onnx_model = make_model(graph, opset_imports=[make_opsetid(\"\", 16)])\n\n sH, sW = 64, 64\n X = np.arange(sW * sH).reshape((1, 1, sH, sW)).astype(np.float32)\n W = np.ones((1, 1, 3, 3), dtype=np.float32)\n B = np.array([[[[0]]]], dtype=np.float32)\n\n sess1 = ReferenceEvaluator(onnx_model)\n sess2 = CReferenceEvaluator(onnx_model) # 100 times faster\n\n expected = sess1.run(None, {\"X\": X, \"W\": W, \"B\": B})[0]\n got = sess2.run(None, {\"X\": X, \"W\": W, \"B\": B})[0]\n diff = np.abs(expected - got).max()\n print(f\"difference: {diff}\")\n\n f1 = lambda: sess1.run(None, {\"X\": X, \"W\": W, \"B\": B})[0]\n f2 = lambda: sess2.run(None, {\"X\": X, \"W\": W, \"B\": B})[0]\n print(\"onnx:\", timeit.timeit(f1, globals=globals(), number=5))\n print(\"onnx-extended:\", timeit.timeit(f2, globals=globals(), number=5))\n\n::\n\n difference: 0.0\n onnx: 0.024006774998269975\n onnx-extended: 0.0002316169993719086\n\nBuild with CUDA, openmp, eigen, onnxruntime\n+++++++++++++++++++++++++++++++++++++++++++\n\nThe package also contains some dummy examples on how to\nbuild with C++ functions (`pybind11 <https://github.com/pybind/pybind11>`_,\n`cython <https://cython.org/>`_),\nwith `openmp <https://www.openmp.org/>`_,\n`eigen <https://eigen.tuxfamily.org/index.php>`_\nwith or without CUDA. It also shows how to create a custom operator\nfor *onnxruntime* in C++.\n\nThe version released on `pypi/onnx-extended <https://pypi.org/project/onnx-extended/>`_\nonly works on CPU. It needs to be manually built to enable\nthe code using CUDA. The build will automatically link with CUDA if it is found.\nIf not, some extensions might not be available.\n\n::\n\n python setup.py build_ext --inplace\n # pip install -e .\n\nIt is possible to use a specific version of CUDA:\n\n::\n\n python setup.py build_ext --inplace --cuda-version=11.8\n # or (not working yet)\n # pip install -e . --config-settings=\"--cuda-version=11.8\"\n # pip install -e . --global-option=\"--cuda-version=11.8\"\n export USE_CUDA=11.8\n pip install -e .\n\n`NVTX <https://github.com/NVIDIA/NVTX>`_\ncan be enabled with the following command:\n\n::\n\n python setup.py build_ext --inplace --use_nvtx 1\n # or (not working yet)\n # pip install -e . --config-settings=\"--use_nvtx=1\"\n pip install -e . --global-option \"--use_nvtx=1\"\n\nExperimental cython binding for onnxruntime\n+++++++++++++++++++++++++++++++++++++++++++\n\nThe python onnxruntime package relies on pybind11 to expose\nits functionalities. *onnx-extended* tries to build a cython wrapper\naround the C/C++ API of onnxruntime. cython relies on python C API\nand is faster than pybind11. This different may be significant when\nonnxruntime is used on small graphs and tensors.\n\nCustom kernels for onnxruntime\n++++++++++++++++++++++++++++++\n\nonnxruntime provides an API to add custom implementation\nfor existing or new onnx operators. An example for CPU.\n\n::\n\n from onnxruntime import InferenceSession, SessionOptions\n from onnx_extended.ortops.optim.cpu import get_ort_ext_libs\n\n r = get_ort_ext_libs()\n opts = SessionOptions()\n if r is not None:\n opts.register_custom_ops_library(r[0])\n\n sess_cus = InferenceSession(\n onx_modified.SerializeToString(), opts, providers=[\"CPUExecutionProvider\"]\n )\n",
"bugtrack_url": null,
"license": "\ufeffCopyright (c) 2023-2024, Xavier Dupr\u00e9 Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.",
"summary": "Extends the list of supported operators in onnx reference implementation and onnxruntime, or implements faster versions in C++.",
"version": "0.3.0",
"project_urls": {
"Changelog": "https://github.com/sdpython/onnx-extended/blob/main/CHANGELOGS.rst",
"Documentation": "https://sdpython.github.io/doc/onnx-extended/dev/",
"Homepage": "https://sdpython.github.io/doc/onnx-extended/dev/",
"Issues": "https://github.com/sdpython/onnx-extended/issues",
"Repository": "https://github.com/sdpython/onnx-extended/"
},
"split_keywords": [
"onnx",
" onnxruntime",
" cuda",
" openmp",
" cmake",
" cython",
" pybind11"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b92d5fc4e338e47513f5c23a867611e984148b5c8be52f349ba1f77381c08e13",
"md5": "e527dc0da4b8639ca736a8dca8e68411",
"sha256": "a4929cf2469bf54e414466bf012ff5b57d6526321682cecde64ce900944ae9d4"
},
"downloads": -1,
"filename": "onnx_extended-0.3.0-cp310-cp310-manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "e527dc0da4b8639ca736a8dca8e68411",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": ">=3.10",
"size": 25666320,
"upload_time": "2024-09-19T22:23:05",
"upload_time_iso_8601": "2024-09-19T22:23:05.377812Z",
"url": "https://files.pythonhosted.org/packages/b9/2d/5fc4e338e47513f5c23a867611e984148b5c8be52f349ba1f77381c08e13/onnx_extended-0.3.0-cp310-cp310-manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "58112e6738f7529e0056fe3d45cfa53ce701fae2e11f4a96793e3993e9a3d0f8",
"md5": "76cb02339547e2402c72dc16617de9a1",
"sha256": "8ddb75ef99f7fc795be4ab7b717f78dd5455df6c54605865e9dec69799632f3d"
},
"downloads": -1,
"filename": "onnx_extended-0.3.0-cp310-cp310-win_amd64.whl",
"has_sig": false,
"md5_digest": "76cb02339547e2402c72dc16617de9a1",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": ">=3.10",
"size": 66954982,
"upload_time": "2024-09-19T22:23:21",
"upload_time_iso_8601": "2024-09-19T22:23:21.616828Z",
"url": "https://files.pythonhosted.org/packages/58/11/2e6738f7529e0056fe3d45cfa53ce701fae2e11f4a96793e3993e9a3d0f8/onnx_extended-0.3.0-cp310-cp310-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d3c817f8012bb2acbeb350f257ba9b60b1c521d360c8a79c4e07210018c9c55d",
"md5": "69df873540a5f4949045820c65319c7f",
"sha256": "cc6b0d22fccc6e0657c25d8e11a40f4383ffae88854ca9d0caab043ab03fb6d8"
},
"downloads": -1,
"filename": "onnx_extended-0.3.0-cp311-cp311-manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "69df873540a5f4949045820c65319c7f",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.10",
"size": 25839227,
"upload_time": "2024-09-19T22:23:31",
"upload_time_iso_8601": "2024-09-19T22:23:31.802145Z",
"url": "https://files.pythonhosted.org/packages/d3/c8/17f8012bb2acbeb350f257ba9b60b1c521d360c8a79c4e07210018c9c55d/onnx_extended-0.3.0-cp311-cp311-manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "1301b9db129a7b9e2b36d7f82648bac2cc633835420c596f4685c591e8faefe4",
"md5": "e67d328d89374fc12fc3dfbf71e51158",
"sha256": "6c30631cb7b71919e4bd7437ef8fe13215ffbb362ed7883d93af8541f520032f"
},
"downloads": -1,
"filename": "onnx_extended-0.3.0-cp311-cp311-win_amd64.whl",
"has_sig": false,
"md5_digest": "e67d328d89374fc12fc3dfbf71e51158",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.10",
"size": 67127307,
"upload_time": "2024-09-19T22:23:47",
"upload_time_iso_8601": "2024-09-19T22:23:47.511821Z",
"url": "https://files.pythonhosted.org/packages/13/01/b9db129a7b9e2b36d7f82648bac2cc633835420c596f4685c591e8faefe4/onnx_extended-0.3.0-cp311-cp311-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "cb7dcd45358a907c8cd6de3db91e8881e1d5915292ffbdf3365705e0e5b48960",
"md5": "015587a58c5a311292051390e63b0da0",
"sha256": "add7f769b7b5d6590a807ec40a4abdb192ee4e2899222480f5182699fb34bb19"
},
"downloads": -1,
"filename": "onnx_extended-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "015587a58c5a311292051390e63b0da0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 14668385,
"upload_time": "2024-09-19T22:23:54",
"upload_time_iso_8601": "2024-09-19T22:23:54.085875Z",
"url": "https://files.pythonhosted.org/packages/cb/7d/cd45358a907c8cd6de3db91e8881e1d5915292ffbdf3365705e0e5b48960/onnx_extended-0.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-19 22:23:54",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "sdpython",
"github_project": "onnx-extended",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "numpy",
"specs": [
[
"<",
"2"
]
]
},
{
"name": "onnx",
"specs": [
[
">=",
"1.15.0"
]
]
},
{
"name": "scipy",
"specs": []
}
],
"lcname": "onnx-extended"
}
