# Polygraphy: A Deep Learning Inference Prototyping and Debugging Toolkit
## Table of Contents
- [Introduction](#introduction)
- [Installation](#installation)
- [Command-line Toolkit](#command-line-toolkit)
- [Python API](#python-api)
- [Examples](#examples)
- [How-To Guides](#how-to-guides)
- [Contributing](#contributing)
## Introduction
Polygraphy is a toolkit designed to assist in running and debugging deep learning models
in various frameworks. It includes a [Python API](./polygraphy) and
[a command-line interface (CLI)](./polygraphy/tools) built using this API.
Among other things, Polygraphy lets you:
- Run inference among multiple backends, like TensorRT and ONNX-Runtime, and compare results
(example: [API](examples/api/01_comparing_frameworks/), [CLI](examples/cli/run/01_comparing_frameworks/))
- Convert models to various formats, e.g. TensorRT engines with post-training quantization
(example: [API](examples/api/04_int8_calibration_in_tensorrt/), [CLI](examples/cli/convert/01_int8_calibration_in_tensorrt/))
- View information about various types of models
(example: [CLI](examples/cli/inspect/))
- Modify ONNX models on the command-line:
- Extract subgraphs (example: [CLI](examples/cli/surgeon/01_isolating_subgraphs/))
- Simplify and sanitize (example: [CLI](examples/cli/surgeon/02_folding_constants/))
- Isolate faulty tactics in TensorRT
(example: [CLI](examples/cli/debug/01_debugging_flaky_trt_tactics/))
## Installation
**IMPORTANT**: **Polygraphy supports only Python 3.6 and later.**
**Before following the instructions below, please ensure you are using a supported version of Python.**
### Installing Prebuilt Wheels
```bash
python -m pip install colored polygraphy --extra-index-url https://pypi.ngc.nvidia.com
```
**NOTE:** *On Linux, the command-line toolkit is usually installed to `${HOME}/.local/bin` by default.*
*Make sure to add this directory to your `PATH` environment variable.*
### Building From Source
#### Using Make Targets (Linux)
```bash
make install
```
#### Using Powershell Script (Windows)
Make sure you are allowed to execute scripts on your system then run:
```ps
.\install.ps1
```
#### Building Manually
1. Install prerequisites:
```
python -m pip install wheel
```
2. Build a wheel:
```
python setup.py bdist_wheel
```
3. Install the wheel manually from **outside** the repository:
On Linux, run:
```
python -m pip install Polygraphy/dist/polygraphy-*-py2.py3-none-any.whl
```
On Windows, using Powershell, run:
```ps
$wheel_path = gci -Name Polygraphy\dist
python -m pip install Polygraphy\dist\$wheel_path
```
**NOTE:** *It is strongly recommended to install the `colored` module for colored output*
*from Polygraphy, as this can greatly improve readability:*
```
python -m pip install colored
```
### Installing Dependencies
Polygraphy has no hard-dependencies on other Python packages. However, much of the functionality included
does require other Python packages.
#### Automatically Installing Dependencies
It's non-trivial to determine all the packages that will be required ahead of time,
since it depends on exactly what functionality is being used.
To make this easier, Polygraphy can optionally automatically install or upgrade dependencies at runtime, as they are needed.
To enable this behavior, set the `POLYGRAPHY_AUTOINSTALL_DEPS` environment variable to `1` or
`polygraphy.config.AUTOINSTALL_DEPS = True` using the Python API.
**NOTE**: *By default, dependencies will be installed using the current interpreter, and may overwrite existing*
*packages. The default installation command, which is `python -m pip install`, can be overriden by setting*
*the `POLYGRAPHY_INSTALL_CMD` environment variable, or setting `polygraphy.config.INSTALL_CMD` using the Python API.*
If you'd like Polygraphy to prompt you before automatically installing or
upgrading pacakges, set the `POLYGRAPHY_ASK_BEFORE_INSTALL` environment variable to `1`
or `polygraphy.config.ASK_BEFORE_INSTALL = True` using the Python API.
#### Installing Manually
Each `backend` directory includes a `requirements.txt` file that specifies the minimum set of packages
it depends on. This does not necessarily include all packages required for all the functionality provided
by the backend, but does serve as a good starting point.
You can install the requirements for whichever backends you're interested in with:
```bash
python -m pip install -r polygraphy/backend/<name>/requirements.txt
```
If additional packages are required, warnings or errors will be logged.
You can install the additional packages manually with:
```bash
python -m pip install <package_name>
```
## Command-line Toolkit
For details on the various tools included in the Polygraphy toolkit,
see the [CLI User Guide](./polygraphy/tools).
### Python API
For more information on the Polygraphy Python API, including a high-level overview and the
Python API reference documentation, see the [API directory](./polygraphy).
## Examples
For examples of both the CLI and Python API, see the [examples directory](./examples).
## How-To Guides
For how-to guides, see the [how-to guides directory](./how-to).
## Contributing
For information on how you can contribute to this project, see [CONTRIBUTING.md](./CONTRIBUTING.md)
Raw data
{
"_id": null,
"home_page": "https://github.com/NVIDIA/TensorRT/tree/main/tools/Polygraphy",
"name": "polygraphy",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": null,
"author": "NVIDIA",
"author_email": "svc_tensorrt@nvidia.com",
"download_url": null,
"platform": null,
"description": "# Polygraphy: A Deep Learning Inference Prototyping and Debugging Toolkit\n\n\n## Table of Contents\n\n- [Introduction](#introduction)\n- [Installation](#installation)\n- [Command-line Toolkit](#command-line-toolkit)\n- [Python API](#python-api)\n- [Examples](#examples)\n- [How-To Guides](#how-to-guides)\n- [Contributing](#contributing)\n\n\n## Introduction\n\nPolygraphy is a toolkit designed to assist in running and debugging deep learning models\nin various frameworks. It includes a [Python API](./polygraphy) and\n[a command-line interface (CLI)](./polygraphy/tools) built using this API.\n\nAmong other things, Polygraphy lets you:\n\n- Run inference among multiple backends, like TensorRT and ONNX-Runtime, and compare results\n (example: [API](examples/api/01_comparing_frameworks/), [CLI](examples/cli/run/01_comparing_frameworks/))\n- Convert models to various formats, e.g. TensorRT engines with post-training quantization\n (example: [API](examples/api/04_int8_calibration_in_tensorrt/), [CLI](examples/cli/convert/01_int8_calibration_in_tensorrt/))\n- View information about various types of models\n (example: [CLI](examples/cli/inspect/))\n- Modify ONNX models on the command-line:\n - Extract subgraphs (example: [CLI](examples/cli/surgeon/01_isolating_subgraphs/))\n - Simplify and sanitize (example: [CLI](examples/cli/surgeon/02_folding_constants/))\n- Isolate faulty tactics in TensorRT\n (example: [CLI](examples/cli/debug/01_debugging_flaky_trt_tactics/))\n\n\n## Installation\n\n**IMPORTANT**: **Polygraphy supports only Python 3.6 and later.**\n **Before following the instructions below, please ensure you are using a supported version of Python.**\n\n\n### Installing Prebuilt Wheels\n\n```bash\npython -m pip install colored polygraphy --extra-index-url https://pypi.ngc.nvidia.com\n```\n\n**NOTE:** *On Linux, the command-line toolkit is usually installed to `${HOME}/.local/bin` by default.*\n *Make sure to add this directory to your `PATH` environment variable.*\n\n\n### Building From Source\n\n#### Using Make Targets (Linux)\n\n```bash\nmake install\n```\n\n#### Using Powershell Script (Windows)\n\nMake sure you are allowed to execute scripts on your system then run:\n```ps\n.\\install.ps1\n```\n\n#### Building Manually\n\n1. Install prerequisites:\n\n```\npython -m pip install wheel\n```\n\n2. Build a wheel:\n\n```\npython setup.py bdist_wheel\n```\n\n3. Install the wheel manually from **outside** the repository:\n\n On Linux, run:\n\n ```\n python -m pip install Polygraphy/dist/polygraphy-*-py2.py3-none-any.whl\n ```\n\n On Windows, using Powershell, run:\n\n ```ps\n $wheel_path = gci -Name Polygraphy\\dist\n python -m pip install Polygraphy\\dist\\$wheel_path\n ```\n\n\n **NOTE:** *It is strongly recommended to install the `colored` module for colored output*\n *from Polygraphy, as this can greatly improve readability:*\n ```\n python -m pip install colored\n ```\n\n\n### Installing Dependencies\n\nPolygraphy has no hard-dependencies on other Python packages. However, much of the functionality included\ndoes require other Python packages.\n\n#### Automatically Installing Dependencies\n\nIt's non-trivial to determine all the packages that will be required ahead of time,\nsince it depends on exactly what functionality is being used.\n\nTo make this easier, Polygraphy can optionally automatically install or upgrade dependencies at runtime, as they are needed.\nTo enable this behavior, set the `POLYGRAPHY_AUTOINSTALL_DEPS` environment variable to `1` or\n`polygraphy.config.AUTOINSTALL_DEPS = True` using the Python API.\n\n**NOTE**: *By default, dependencies will be installed using the current interpreter, and may overwrite existing*\n *packages. The default installation command, which is `python -m pip install`, can be overriden by setting*\n *the `POLYGRAPHY_INSTALL_CMD` environment variable, or setting `polygraphy.config.INSTALL_CMD` using the Python API.*\n\nIf you'd like Polygraphy to prompt you before automatically installing or\nupgrading pacakges, set the `POLYGRAPHY_ASK_BEFORE_INSTALL` environment variable to `1`\nor `polygraphy.config.ASK_BEFORE_INSTALL = True` using the Python API.\n\n#### Installing Manually\n\nEach `backend` directory includes a `requirements.txt` file that specifies the minimum set of packages\nit depends on. This does not necessarily include all packages required for all the functionality provided\nby the backend, but does serve as a good starting point.\n\nYou can install the requirements for whichever backends you're interested in with:\n```bash\npython -m pip install -r polygraphy/backend/<name>/requirements.txt\n```\n\nIf additional packages are required, warnings or errors will be logged.\nYou can install the additional packages manually with:\n```bash\npython -m pip install <package_name>\n```\n\n\n## Command-line Toolkit\n\nFor details on the various tools included in the Polygraphy toolkit,\nsee the [CLI User Guide](./polygraphy/tools).\n\n\n### Python API\n\nFor more information on the Polygraphy Python API, including a high-level overview and the\nPython API reference documentation, see the [API directory](./polygraphy).\n\n\n## Examples\n\nFor examples of both the CLI and Python API, see the [examples directory](./examples).\n\n\n## How-To Guides\n\nFor how-to guides, see the [how-to guides directory](./how-to).\n\n\n## Contributing\n\nFor information on how you can contribute to this project, see [CONTRIBUTING.md](./CONTRIBUTING.md)\n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "Polygraphy: A Deep Learning Inference Prototyping and Debugging Toolkit",
"version": "0.49.9",
"project_urls": {
"Homepage": "https://github.com/NVIDIA/TensorRT/tree/main/tools/Polygraphy"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "4af5a2b20c677c1a856cc9e08cd0b5a5105450ed5253e369e938ddd31d91c547",
"md5": "42ace1da709703357d99c224c81a4d62",
"sha256": "62ae22825efdd3288222e5b1d2d791fe58e87844fcd848bcd1251fbce02ba956"
},
"downloads": -1,
"filename": "polygraphy-0.49.9-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "42ace1da709703357d99c224c81a4d62",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.6",
"size": 346910,
"upload_time": "2024-04-11T00:46:26",
"upload_time_iso_8601": "2024-04-11T00:46:26.430779Z",
"url": "https://files.pythonhosted.org/packages/4a/f5/a2b20c677c1a856cc9e08cd0b5a5105450ed5253e369e938ddd31d91c547/polygraphy-0.49.9-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-11 00:46:26",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "NVIDIA",
"github_project": "TensorRT",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "onnx",
"specs": []
},
{
"name": "tensorflow-gpu",
"specs": [
[
"==",
"2.9.1"
]
]
},
{
"name": "onnxruntime",
"specs": [
[
"==",
"1.8.1"
]
]
},
{
"name": "onnxruntime",
"specs": [
[
"==",
"1.12.1"
]
]
},
{
"name": "torch",
"specs": [
[
"==",
"1.10.2+cu113"
]
]
},
{
"name": "torch",
"specs": [
[
"==",
"1.10.2"
]
]
},
{
"name": "torch",
"specs": [
[
"==",
"1.11.0+cu113"
]
]
},
{
"name": "torch",
"specs": [
[
"==",
"1.11.0"
]
]
},
{
"name": "torchvision",
"specs": [
[
"==",
"0.11.3"
]
]
},
{
"name": "torchvision",
"specs": [
[
"==",
"0.12.0"
]
]
},
{
"name": "Pillow",
"specs": []
},
{
"name": "numpy",
"specs": []
},
{
"name": "pycuda",
"specs": []
},
{
"name": "pytest",
"specs": []
},
{
"name": "onnx-graphsurgeon",
"specs": []
}
],
"lcname": "polygraphy"
}
JSON
