<h3 align="center">
nndeploy: An Easy-to-Use, and High-Performance AI Deployment Framework
</h3>
## Introduction
nndeploy is an easy-to-use, and high-performance AI deployment framework. Based on the design concepts of visual workflows and multi-backend inference, developers can quickly develop SDKs for specified platforms and hardware from algorithm repositories, significantly saving development time. Furthermore, the framework has already deployed numerous AI models including LLM, AIGC generation, face swap, object detection, image segmentation, etc., ready to use out-of-the-box.
### **Simple and Easy to Use**
- **Visual Workflow**: Deploy AI algorithms through drag-and-drop operations. Visually adjust all node parameters of the AI algorithm in the frontend and quickly preview the effect after parameter tuning.
- **Custom Nodes**: Support Python/C++ custom nodes, seamlessly integrated into the visual interface without frontend code.
- **Algorithm Combination**: Flexibly combine different algorithms to quickly build innovative AI applications.
- **One-Click Deployment**: The completed workflow can be exported as a JSON configuration file with one click, supporting direct calls via Python/C++ API, achieving seamless transition from development to production environments, and fully supporting platforms like Linux, Windows, macOS, Android, iOS, etc.
### **High Performance**
- **Parallel Optimization**: Supports execution modes like serial, pipeline parallel, task parallel, etc.
- **Memory Optimization**: Optimization strategies like zero-copy, memory pool, memory reuse, etc.
- **High-Performance Optimization**: Built-in nodes optimized with C++/CUDA/Ascend C/SIMD, etc.
- **Multi-Backend Inference**: One workflow, multiple backend inference. Integrates 13 mainstream inference frameworks with zero abstraction cost, covering all platforms including cloud, desktop, mobile, edge, etc.
| Inference Framework | Application Scenario | Status |
| :----------------------------------------------------------------------------------------- | :-------------------------------- | :----- |
| [ONNXRuntime](https://github.com/microsoft/onnxruntime) | Cross-platform inference | ✅ |
| [TensorRT](https://github.com/NVIDIA/TensorRT) | NVIDIA GPU high-performance inference | ✅ |
| [OpenVINO](https://github.com/openvinotoolkit/openvino) | Intel CPU/GPU optimization | ✅ |
| [MNN](https://github.com/alibaba/MNN) | Mobile inference engine by Alibaba| ✅ |
| [TNN](https://github.com/Tencent/TNN) | Mobile inference engine by Tencent| ✅ |
| [ncnn](https://github.com/Tencent/ncnn) | Mobile inference engine by Tencent| ✅ |
| [CoreML](https://github.com/apple/coremltools) | iOS/macOS native acceleration | ✅ |
| [AscendCL](https://www.hiascend.com/zh/) | Huawei Ascend AI chip inference framework | ✅ |
| [RKNN](https://www.rock-chips.com/a/cn/downloadcenter/BriefDatasheet/index.html) | Rockchip NPU inference framework | ✅ |
| [SNPE](https://developer.qualcomm.com/software/qualcomm-neural-processing-sdk) | Qualcomm Snapdragon NPU inference framework | ✅ |
| [TVM](https://github.com/apache/tvm) | Deep learning compiler stack | ✅ |
| [PyTorch](https://pytorch.org/) | Rapid prototyping / Cloud deployment | ✅ |
| [Self-developed Inference Framework](docs/zh_cn/inference/README_INFERENCE.md) | Default inference framework | ✅ |
### **Out-of-the-Box Algorithms**
List of deployed models, with **100+ nodes** created. We will continue to deploy more high-value AI algorithms. If you have algorithms you need deployed, please let us know via [issue](https://github.com/nndeploy/nndeploy/issues).
| Application Scenario | Available Models | Remarks |
| ---------------------- | --------------------------------------------------------------------------------------------- | --------------------------------------------------- |
| **Large Language Model** |**QWen-2.5**, **QWen-3** | |
| **Image Generation** | Stable Diffusion 1.5, Stable Diffusion XL, Stable Diffusion 3, HunyuanDiT, etc. | Supports text-to-image, image-to-image, inpainting; based on **diffusers** |
| **Face Swap** | **deep-live-cam** | |
| **OCR** | **Paddle OCR** | |
| **Object Detection** | **YOLOv5, YOLOv6, YOLOv7, YOLOv8, YOLOv11, YOLOx** | |
| **Object Tracking** | FairMot | |
| **Image Segmentation** | RBMGv1.4, PPMatting, **Segment Anything** | |
| **Classification** | ResNet, MobileNet, EfficientNet, PPLcNet, GhostNet, ShuffleNet, SqueezeNet | |
| **API Service** | OPENAI, DeepSeek, Moonshot | Supports LLM and AIGC services |
> See more details in the [Deployed Model List Details](docs/zh_cn/quick_start/model_list.md)
Raw data
{
"_id": null,
"home_page": "https://github.com/nndeploy/nndeploy",
"name": "nndeploy",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "deep-learning, visual-workflow, ai-agent, easy-to-use, high-performance",
"author": "nndeploy team",
"author_email": "595961667@qq.com",
"download_url": null,
"platform": null,
"description": "\n<h3 align=\"center\">\nnndeploy: An Easy-to-Use, and High-Performance AI Deployment Framework\n</h3>\n\n## Introduction\n\nnndeploy is an easy-to-use, and high-performance AI deployment framework. Based on the design concepts of visual workflows and multi-backend inference, developers can quickly develop SDKs for specified platforms and hardware from algorithm repositories, significantly saving development time. Furthermore, the framework has already deployed numerous AI models including LLM, AIGC generation, face swap, object detection, image segmentation, etc., ready to use out-of-the-box.\n\n### **Simple and Easy to Use**\n\n- **Visual Workflow**: Deploy AI algorithms through drag-and-drop operations. Visually adjust all node parameters of the AI algorithm in the frontend and quickly preview the effect after parameter tuning.\n- **Custom Nodes**: Support Python/C++ custom nodes, seamlessly integrated into the visual interface without frontend code.\n- **Algorithm Combination**: Flexibly combine different algorithms to quickly build innovative AI applications.\n- **One-Click Deployment**: The completed workflow can be exported as a JSON configuration file with one click, supporting direct calls via Python/C++ API, achieving seamless transition from development to production environments, and fully supporting platforms like Linux, Windows, macOS, Android, iOS, etc.\n\n### **High Performance**\n\n- **Parallel Optimization**: Supports execution modes like serial, pipeline parallel, task parallel, etc.\n- **Memory Optimization**: Optimization strategies like zero-copy, memory pool, memory reuse, etc.\n- **High-Performance Optimization**: Built-in nodes optimized with C++/CUDA/Ascend C/SIMD, etc.\n- **Multi-Backend Inference**: One workflow, multiple backend inference. Integrates 13 mainstream inference frameworks with zero abstraction cost, covering all platforms including cloud, desktop, mobile, edge, etc.\n\n | Inference Framework | Application Scenario | Status |\n | :----------------------------------------------------------------------------------------- | :-------------------------------- | :----- |\n | [ONNXRuntime](https://github.com/microsoft/onnxruntime) | Cross-platform inference | \u2705 |\n | [TensorRT](https://github.com/NVIDIA/TensorRT) | NVIDIA GPU high-performance inference | \u2705 |\n | [OpenVINO](https://github.com/openvinotoolkit/openvino) | Intel CPU/GPU optimization | \u2705 |\n | [MNN](https://github.com/alibaba/MNN) | Mobile inference engine by Alibaba| \u2705 |\n | [TNN](https://github.com/Tencent/TNN) | Mobile inference engine by Tencent| \u2705 |\n | [ncnn](https://github.com/Tencent/ncnn) | Mobile inference engine by Tencent| \u2705 |\n | [CoreML](https://github.com/apple/coremltools) | iOS/macOS native acceleration | \u2705 |\n | [AscendCL](https://www.hiascend.com/zh/) | Huawei Ascend AI chip inference framework | \u2705 |\n | [RKNN](https://www.rock-chips.com/a/cn/downloadcenter/BriefDatasheet/index.html) | Rockchip NPU inference framework | \u2705 |\n | [SNPE](https://developer.qualcomm.com/software/qualcomm-neural-processing-sdk) | Qualcomm Snapdragon NPU inference framework | \u2705 |\n | [TVM](https://github.com/apache/tvm) | Deep learning compiler stack | \u2705 |\n | [PyTorch](https://pytorch.org/) | Rapid prototyping / Cloud deployment | \u2705 |\n | [Self-developed Inference Framework](docs/zh_cn/inference/README_INFERENCE.md) | Default inference framework | \u2705 |\n\n### **Out-of-the-Box Algorithms**\n\nList of deployed models, with **100+ nodes** created. We will continue to deploy more high-value AI algorithms. If you have algorithms you need deployed, please let us know via [issue](https://github.com/nndeploy/nndeploy/issues).\n\n| Application Scenario | Available Models | Remarks |\n| ---------------------- | --------------------------------------------------------------------------------------------- | --------------------------------------------------- |\n| **Large Language Model** |**QWen-2.5**, **QWen-3** | |\n| **Image Generation** | Stable Diffusion 1.5, Stable Diffusion XL, Stable Diffusion 3, HunyuanDiT, etc. | Supports text-to-image, image-to-image, inpainting; based on **diffusers** |\n| **Face Swap** | **deep-live-cam** | |\n| **OCR** | **Paddle OCR** | |\n| **Object Detection** | **YOLOv5, YOLOv6, YOLOv7, YOLOv8, YOLOv11, YOLOx** | |\n| **Object Tracking** | FairMot | |\n| **Image Segmentation** | RBMGv1.4, PPMatting, **Segment Anything** | |\n| **Classification** | ResNet, MobileNet, EfficientNet, PPLcNet, GhostNet, ShuffleNet, SqueezeNet | |\n| **API Service** | OPENAI, DeepSeek, Moonshot | Supports LLM and AIGC services |\n\n> See more details in the [Deployed Model List Details](docs/zh_cn/quick_start/model_list.md)\n",
"bugtrack_url": null,
"license": "Apache License 2.0",
"summary": "An Easy-to-Use and High-Performance Edge AI Deployment Framework",
"version": "3.0.6",
"project_urls": {
"Bug Reports": "https://github.com/nndeploy/nndeploy/issues",
"Homepage": "https://github.com/nndeploy/nndeploy",
"Source": "https://github.com/nndeploy/nndeploy"
},
"split_keywords": [
"deep-learning",
" visual-workflow",
" ai-agent",
" easy-to-use",
" high-performance"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "f7312c670585b4d4873503b7a83b8e0fa68639c6ed7305e301c17d5366385bfb",
"md5": "b1d42bc60e752091cd72f49bc0e06c88",
"sha256": "4bf6df4547814fa98713b8b40c9e0ecc77ef27028ebcf87d481de46e83784313"
},
"downloads": -1,
"filename": "nndeploy-3.0.6-cp310-cp310-manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "b1d42bc60e752091cd72f49bc0e06c88",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": ">=3.10",
"size": 64483776,
"upload_time": "2025-11-04T07:35:52",
"upload_time_iso_8601": "2025-11-04T07:35:52.347230Z",
"url": "https://files.pythonhosted.org/packages/f7/31/2c670585b4d4873503b7a83b8e0fa68639c6ed7305e301c17d5366385bfb/nndeploy-3.0.6-cp310-cp310-manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "d470e893d05a29eb044d22803731eef4b5e2cebbcb1ccffbf066a929f3fe87e1",
"md5": "94966d61757e936d3f621f6591fbda3e",
"sha256": "b429a39d369bf6805456d5ed7b13cd48603b735dff850b80e10709260b806451"
},
"downloads": -1,
"filename": "nndeploy-3.0.6-cp311-cp311-manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "94966d61757e936d3f621f6591fbda3e",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.10",
"size": 65963463,
"upload_time": "2025-11-04T07:35:56",
"upload_time_iso_8601": "2025-11-04T07:35:56.010412Z",
"url": "https://files.pythonhosted.org/packages/d4/70/e893d05a29eb044d22803731eef4b5e2cebbcb1ccffbf066a929f3fe87e1/nndeploy-3.0.6-cp311-cp311-manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "a8c18d198f3c81098af935a6147f54fb2fb232ed53b96e44adeca48111e537d5",
"md5": "7298881ac43cf86aa70b7e4356f4cbc1",
"sha256": "99073167f67677e6b7e61dddfd2b9b8dd7d0baf3f0578e5a0077bd75107e28e9"
},
"downloads": -1,
"filename": "nndeploy-3.0.6-cp312-cp312-manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "7298881ac43cf86aa70b7e4356f4cbc1",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": ">=3.10",
"size": 67453821,
"upload_time": "2025-11-04T07:35:59",
"upload_time_iso_8601": "2025-11-04T07:35:59.925218Z",
"url": "https://files.pythonhosted.org/packages/a8/c1/8d198f3c81098af935a6147f54fb2fb232ed53b96e44adeca48111e537d5/nndeploy-3.0.6-cp312-cp312-manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-11-04 07:35:52",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "nndeploy",
"github_project": "nndeploy",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "cython",
"specs": []
},
{
"name": "packaging",
"specs": []
},
{
"name": "Pillow",
"specs": []
},
{
"name": "numpy",
"specs": []
},
{
"name": "opencv-python",
"specs": [
[
">=",
"4.8.0"
]
]
},
{
"name": "modelscope",
"specs": []
},
{
"name": "multiprocess",
"specs": []
},
{
"name": "requests",
"specs": [
[
">=",
"2.31.0"
]
]
},
{
"name": "fastapi",
"specs": [
[
">=",
"0.104.0"
]
]
},
{
"name": "uvicorn",
"specs": [
[
">=",
"0.24.0"
]
]
},
{
"name": "websockets",
"specs": [
[
">=",
"11.0"
]
]
},
{
"name": "python-multipart",
"specs": [
[
">=",
"0.0.6"
]
]
},
{
"name": "pydantic",
"specs": [
[
">=",
"2.0.0"
]
]
},
{
"name": "chardet",
"specs": [
[
">=",
"5.2.0"
]
]
},
{
"name": "torch",
"specs": [
[
">=",
"2.0.0"
]
]
},
{
"name": "torchvision",
"specs": [
[
">=",
"0.15.0"
]
]
},
{
"name": "accelerate",
"specs": []
},
{
"name": "diffusers",
"specs": []
},
{
"name": "transformers",
"specs": []
},
{
"name": "onnx",
"specs": [
[
">=",
"1.16.0"
]
]
},
{
"name": "onnxruntime",
"specs": [
[
">=",
"1.18.0"
]
]
}
],
"lcname": "nndeploy"
}
JSON
