infery


Nameinfery JSON
Version 3.9.0 PyPI version JSON
download
home_pagehttps://github.com/Deci-AI/infery-examples
SummaryDeci Run-Time Engine
upload_time2023-01-12 08:53:48
maintainer
docs_urlNone
authorDeci AI
requires_python>=3.7.5
licenseDeci Infery License
keywords deci ai inference deep learning
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls. 
            # Infery
`infery` is a runtime engine that simplifies inference, benchmark and profiling.<br>
`infery` supports all major deep learning frameworks, with a <b>unified and simple API</b>.

## Quickstart and Examples
<a href="https://github.com/Deci-AI/infery-examples">
    <img src="https://img.shields.io/badge/Public-Infery Examples-green">
</a>

### Installation Instructions
https://docs.deci.ai/docs/installing-infery-1

## Usage
![Infery Usage](https://deci.ai/wp-content/uploads/2021/06/Infery-Fig1.png)


#### 1. Load an example model
```python
>>> import infery, numpy as np
>>> model = infery.load(model_path='MyNewTask_1_0.onnx', framework_type='onnx', inference_hardware='gpu')
```

#### 2. Predict with a random numpy input

```python
>>> inputs = [np.random.random(shape).astype('float32') for shape in [[1, 243, 30, 40], [1, 172, 60, 80], [1, 102, 120, 160], [1, 64, 240, 320], [1, 153, 15, 20]]]
>>> model.predict(inputs)
[array([[[[-3.5920768, -3.5920792, -3.592102 , ..., -3.592099 ,
           -3.5920944, -3.5920882],
          [-3.592076 , -3.5919113, -3.592086 , ..., -3.5921211,
           -3.5921066, -3.5920937],
          [-3.592083 , -3.592073 , -3.5920823, ..., -3.5921297,
           -3.592109 , -3.5920937],
          ...,
          [-3.5920753, -3.5917826, -3.591444 , ..., -3.580754 ,
           -3.5816329, -3.582549 ],
          [-3.592073 , -3.5917459, -3.591257 , ..., -3.5817945,
           -3.5820704, -3.5835373],
          [-3.592073 , -3.5920737, -3.5920737, ..., -3.5853324,
           -3.5845006, -3.5856297]],
 
         [[-5.862198 , -5.8567815, -5.851764 , ..., -5.86396  ,
           -5.8639617, -5.865011 ],
          [-5.858771 , -5.8493323, -5.841462 , ..., -5.8617773,
           -5.8614554, -5.8633246],
          [-5.8560567, -5.844124 , -5.8351245, ..., -5.8598166,
           -5.859674 , -5.8624067],
          ...,
          [-5.8608136, -5.854358 , -5.8467784, ..., -5.8504496,
           -5.8563104, -5.8615303],
          [-5.86313  , -5.8587003, -5.8531966, ..., -5.8534794,
           -5.8581944, -5.8625536],
          [-5.865306 , -5.8623176, -5.8593984, ..., -5.8581495,
           -5.861572 , -5.865295 ]],
 
         [[-8.843734 , -8.840406 , -8.837172 , ..., -8.840413 ,
           -8.842026 , -8.843931 ],
          [-8.840792 , -8.836787 , -8.831037 , ..., -8.836103 ,
           -8.839954 , -8.842534 ],
          [-8.838855 , -8.833998 , -8.82706  , ..., -8.835106 ,
           -8.839087 , -8.841538 ],
          ...,
          [-8.842419 , -8.840865 , -8.838625 , ..., -8.83943  ,
           -8.843677 , -8.845087 ],
          [-8.844379 , -8.84402  , -8.843141 , ..., -8.84185  ,
           -8.844696 , -8.845202 ],
          [-8.844775 , -8.845572 , -8.845177 , ..., -8.843876 ,
           -8.8444395, -8.845926 ]]]], dtype=float32)]
```

#### 3. Benchmark the model on the current hardware
```python
>>> model.benchmark(batch_size=1)
-INFO- Benchmarking the model in batch size 1 and dimensions [(243, 30, 40), (172, 60, 80), (102, 120, 160), (64, 240, 320), (153, 15, 20)]... 
<ModelBenchmarks: {
    "batch_size": 1,
    "batch_inf_time": "6.57 ms",
    "batch_inf_time_variance": "0.02 ms",
    "model_memory_used": "1536.00 mb",
    "system_startpoint_memory_used": "1536.00 mb",
    "post_inference_memory_used": "1536.00 mb",
    "total_memory_size": "7982.00 mb",
    "throughput": "152.32 fps",
    "sample_inf_time": "6.57 ms",
    "include_io": true,
    "framework_type": "onnx",
    "framework_version": "1.10.0",
    "inference_hardware": "GPU",
    "date": "11:16:55__02-03-2022",
    "ctime": 1643879815,
    "h_to_d_mean": null,
    "d_to_h_mean": null,
    "h_to_d_variance": null,
    "d_to_h_variance": null
}>
```

## Documentation:
https://docs.deci.ai/docs/infery

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/Deci-AI/infery-examples",
    "name": "infery",
    "maintainer": "",
    "docs_url": null,
    "requires_python": ">=3.7.5",
    "maintainer_email": "",
    "keywords": "Deci,AI,Inference,Deep Learning",
    "author": "Deci AI",
    "author_email": "rnd@deci.ai",
    "download_url": "",
    "platform": null,
    "description": "# Infery\n`infery` is a runtime engine that simplifies inference, benchmark and profiling.<br>\n`infery` supports all major deep learning frameworks, with a <b>unified and simple API</b>.\n\n## Quickstart and Examples\n<a href=\"https://github.com/Deci-AI/infery-examples\">\n    <img src=\"https://img.shields.io/badge/Public-Infery Examples-green\">\n</a>\n\n### Installation Instructions\nhttps://docs.deci.ai/docs/installing-infery-1\n\n## Usage\n![Infery Usage](https://deci.ai/wp-content/uploads/2021/06/Infery-Fig1.png)\n\n\n#### 1. Load an example model\n```python\n>>> import infery, numpy as np\n>>> model = infery.load(model_path='MyNewTask_1_0.onnx', framework_type='onnx', inference_hardware='gpu')\n```\n\n#### 2. Predict with a random numpy input\n\n```python\n>>> inputs = [np.random.random(shape).astype('float32') for shape in [[1, 243, 30, 40], [1, 172, 60, 80], [1, 102, 120, 160], [1, 64, 240, 320], [1, 153, 15, 20]]]\n>>> model.predict(inputs)\n[array([[[[-3.5920768, -3.5920792, -3.592102 , ..., -3.592099 ,\n           -3.5920944, -3.5920882],\n          [-3.592076 , -3.5919113, -3.592086 , ..., -3.5921211,\n           -3.5921066, -3.5920937],\n          [-3.592083 , -3.592073 , -3.5920823, ..., -3.5921297,\n           -3.592109 , -3.5920937],\n          ...,\n          [-3.5920753, -3.5917826, -3.591444 , ..., -3.580754 ,\n           -3.5816329, -3.582549 ],\n          [-3.592073 , -3.5917459, -3.591257 , ..., -3.5817945,\n           -3.5820704, -3.5835373],\n          [-3.592073 , -3.5920737, -3.5920737, ..., -3.5853324,\n           -3.5845006, -3.5856297]],\n \n         [[-5.862198 , -5.8567815, -5.851764 , ..., -5.86396  ,\n           -5.8639617, -5.865011 ],\n          [-5.858771 , -5.8493323, -5.841462 , ..., -5.8617773,\n           -5.8614554, -5.8633246],\n          [-5.8560567, -5.844124 , -5.8351245, ..., -5.8598166,\n           -5.859674 , -5.8624067],\n          ...,\n          [-5.8608136, -5.854358 , -5.8467784, ..., -5.8504496,\n           -5.8563104, -5.8615303],\n          [-5.86313  , -5.8587003, -5.8531966, ..., -5.8534794,\n           -5.8581944, -5.8625536],\n          [-5.865306 , -5.8623176, -5.8593984, ..., -5.8581495,\n           -5.861572 , -5.865295 ]],\n \n         [[-8.843734 , -8.840406 , -8.837172 , ..., -8.840413 ,\n           -8.842026 , -8.843931 ],\n          [-8.840792 , -8.836787 , -8.831037 , ..., -8.836103 ,\n           -8.839954 , -8.842534 ],\n          [-8.838855 , -8.833998 , -8.82706  , ..., -8.835106 ,\n           -8.839087 , -8.841538 ],\n          ...,\n          [-8.842419 , -8.840865 , -8.838625 , ..., -8.83943  ,\n           -8.843677 , -8.845087 ],\n          [-8.844379 , -8.84402  , -8.843141 , ..., -8.84185  ,\n           -8.844696 , -8.845202 ],\n          [-8.844775 , -8.845572 , -8.845177 , ..., -8.843876 ,\n           -8.8444395, -8.845926 ]]]], dtype=float32)]\n```\n\n#### 3. Benchmark the model on the current hardware\n```python\n>>> model.benchmark(batch_size=1)\n-INFO- Benchmarking the model in batch size 1 and dimensions [(243, 30, 40), (172, 60, 80), (102, 120, 160), (64, 240, 320), (153, 15, 20)]... \n<ModelBenchmarks: {\n    \"batch_size\": 1,\n    \"batch_inf_time\": \"6.57 ms\",\n    \"batch_inf_time_variance\": \"0.02 ms\",\n    \"model_memory_used\": \"1536.00 mb\",\n    \"system_startpoint_memory_used\": \"1536.00 mb\",\n    \"post_inference_memory_used\": \"1536.00 mb\",\n    \"total_memory_size\": \"7982.00 mb\",\n    \"throughput\": \"152.32 fps\",\n    \"sample_inf_time\": \"6.57 ms\",\n    \"include_io\": true,\n    \"framework_type\": \"onnx\",\n    \"framework_version\": \"1.10.0\",\n    \"inference_hardware\": \"GPU\",\n    \"date\": \"11:16:55__02-03-2022\",\n    \"ctime\": 1643879815,\n    \"h_to_d_mean\": null,\n    \"d_to_h_mean\": null,\n    \"h_to_d_variance\": null,\n    \"d_to_h_variance\": null\n}>\n```\n\n## Documentation:\nhttps://docs.deci.ai/docs/infery\n\n\n\n",
    "bugtrack_url": null,
    "license": "Deci Infery License",
    "summary": "Deci Run-Time Engine",
    "version": "3.9.0",
    "split_keywords": [
        "deci",
        "ai",
        "inference",
        "deep learning"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "47242acd7101a9fe38846715a4c4fe6f76325793cbf9916a4def3d37f1680884",
                "md5": "01d9f996b0ffa68fdfccaf2675baab17",
                "sha256": "906755346c520c958579a971f87e31775c7ddabf6918c2811215edd6179a040a"
            },
            "downloads": -1,
            "filename": "infery-3.9.0-cp37-cp37m-macosx_10_15_x86_64.whl",
            "has_sig": false,
            "md5_digest": "01d9f996b0ffa68fdfccaf2675baab17",
            "packagetype": "bdist_wheel",
            "python_version": "cp37",
            "requires_python": ">=3.7.5",
            "size": 2082541,
            "upload_time": "2023-01-12T08:53:48",
            "upload_time_iso_8601": "2023-01-12T08:53:48.987658Z",
            "url": "https://files.pythonhosted.org/packages/47/24/2acd7101a9fe38846715a4c4fe6f76325793cbf9916a4def3d37f1680884/infery-3.9.0-cp37-cp37m-macosx_10_15_x86_64.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "2acaa9bccd8e1da209e2b27d1901a68b0e2894faf5ae47179bebe82a50430e90",
                "md5": "3f16e66041ddedb6f7233f20481da72c",
                "sha256": "8d170be5c5a9050513c9b8ea3b88ff8c92594141060a3697d2da3b1c4b84b640"
            },
            "downloads": -1,
            "filename": "infery-3.9.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
            "has_sig": false,
            "md5_digest": "3f16e66041ddedb6f7233f20481da72c",
            "packagetype": "bdist_wheel",
            "python_version": "cp37",
            "requires_python": ">=3.7.5",
            "size": 1748299,
            "upload_time": "2023-01-12T08:53:51",
            "upload_time_iso_8601": "2023-01-12T08:53:51.868865Z",
            "url": "https://files.pythonhosted.org/packages/2a/ca/a9bccd8e1da209e2b27d1901a68b0e2894faf5ae47179bebe82a50430e90/infery-3.9.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "18846a49550ad0d00ee3a52466c369db434192db360f9fb0ee0480e537745b7a",
                "md5": "9531d0e4465c68d031c2767361906a86",
                "sha256": "b5cbb98c6a676c9bed2940e84e3fb603d2749b415e7a433f96cc3d1b526b262a"
            },
            "downloads": -1,
            "filename": "infery-3.9.0-cp38-cp38-macosx_10_15_x86_64.whl",
            "has_sig": false,
            "md5_digest": "9531d0e4465c68d031c2767361906a86",
            "packagetype": "bdist_wheel",
            "python_version": "cp38",
            "requires_python": ">=3.7.5",
            "size": 2076477,
            "upload_time": "2023-01-12T08:53:54",
            "upload_time_iso_8601": "2023-01-12T08:53:54.752414Z",
            "url": "https://files.pythonhosted.org/packages/18/84/6a49550ad0d00ee3a52466c369db434192db360f9fb0ee0480e537745b7a/infery-3.9.0-cp38-cp38-macosx_10_15_x86_64.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "156049cd1427b9edd6c4535b594800b69ad0027ff3a81424cc2a7e2daf6737d3",
                "md5": "9f07e3a83e5e618c0058366d753524ae",
                "sha256": "adc2d38bb198e61d16f0e36e11780bbafdd4ac95a3aad232d9bc2eb6b7842203"
            },
            "downloads": -1,
            "filename": "infery-3.9.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
            "has_sig": false,
            "md5_digest": "9f07e3a83e5e618c0058366d753524ae",
            "packagetype": "bdist_wheel",
            "python_version": "cp38",
            "requires_python": ">=3.7.5",
            "size": 1850103,
            "upload_time": "2023-01-12T08:53:57",
            "upload_time_iso_8601": "2023-01-12T08:53:57.561157Z",
            "url": "https://files.pythonhosted.org/packages/15/60/49cd1427b9edd6c4535b594800b69ad0027ff3a81424cc2a7e2daf6737d3/infery-3.9.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "d90853490172135a879dc00410113d28a895135a626bb3ab2b7d49b0a5178517",
                "md5": "7ce721b39ee71233d84345009738db74",
                "sha256": "37077f4ad67f0b9d5f461818f04007c21fc1f5d1542b53d4a8ea29a5dcfa719d"
            },
            "downloads": -1,
            "filename": "infery-3.9.0-cp39-cp39-macosx_10_15_x86_64.whl",
            "has_sig": false,
            "md5_digest": "7ce721b39ee71233d84345009738db74",
            "packagetype": "bdist_wheel",
            "python_version": "cp39",
            "requires_python": ">=3.7.5",
            "size": 2101902,
            "upload_time": "2023-01-12T08:54:00",
            "upload_time_iso_8601": "2023-01-12T08:54:00.111463Z",
            "url": "https://files.pythonhosted.org/packages/d9/08/53490172135a879dc00410113d28a895135a626bb3ab2b7d49b0a5178517/infery-3.9.0-cp39-cp39-macosx_10_15_x86_64.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "6caf3e2462d82189156897dc7a96c45c6b216a2b3080df2095214a8f2986f04f",
                "md5": "4808553023ed21f70fbaa4b9ee0921cd",
                "sha256": "e9f265308f27cd272bf310681b1899c97291d4c21fc6e76b40e151746007bf9f"
            },
            "downloads": -1,
            "filename": "infery-3.9.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
            "has_sig": false,
            "md5_digest": "4808553023ed21f70fbaa4b9ee0921cd",
            "packagetype": "bdist_wheel",
            "python_version": "cp39",
            "requires_python": ">=3.7.5",
            "size": 1858625,
            "upload_time": "2023-01-12T08:54:02",
            "upload_time_iso_8601": "2023-01-12T08:54:02.829225Z",
            "url": "https://files.pythonhosted.org/packages/6c/af/3e2462d82189156897dc7a96c45c6b216a2b3080df2095214a8f2986f04f/infery-3.9.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-01-12 08:53:48",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "github_user": "Deci-AI",
    "github_project": "infery-examples",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "lcname": "infery"
}
Deci AI
Elapsed time: 0.05934s