| Name | fastnn JSON |
| Version |
0.4.0
JSON |
| download |
| home_page | https://andrewchang.dev/fastnn/ |
| Summary | A python library and framework for fast neural network computations. |
| upload_time | 2023-11-01 21:08:18 |
| maintainer | |
| docs_url | None |
| author | Andrew Chang |
| requires_python | >=3.9,<3.12 |
| license | MIT |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Fast Neural Networks (FastNN)
A framework for deploying serializable and optimizable neural net models at scale in production via. the NVIDIA Triton Inference Server.
<p align="center">
<a href="https://hub.docker.com/r/aychang/fastnn">
<img src="https://img.shields.io/docker/cloud/build/aychang/fastnn"
</a>
<a href="https://badge.fury.io/py/fastnn">
<img src="https://badge.fury.io/py/fastnn.svg">
</a>
<a href="https://github.com/aychang95/fastnn/blob/master/LICENSE">
<img src="https://img.shields.io/github/license/aychang95/fastnn">
</a>
</p>
## [**FastNN Docker Release Selector (Ubuntu 18.04)**](https://andrewchang.dev/fastnn/index.html#fastnn-docker-release-selector-ubuntu-1804)
## [Documentation](https://andrewchang.dev/fastnn) - Guides, Models, API References
## Features:
- **Data Processing**
- Intuitive data processing modules for encoding human-readible data into tensors compatible with deep learning libraries
- **Model Exporting**
- FastNN torch modules and tools for exporting models via. `TorchScript` tracing and scripting to a production environment. Now includes Text Generation.
- **Model Zoo**
- Various exported models hosted in this repo via. git-lfs and AWS S3. Includes models from the HuggingFace's Transformers and
TorchVision
- **Model Deployment**
- Deploy models using Triton Inference Server on CPU/GPU-compatible server(s) with helm or docker
- **FastNN Client**
- Client wrapper for Triton Inference Server's client module for programmatic requests with python
## Pre-Requisites:
Git LFS is required if you'd like to use any of the models provided by FastNN in `./model_repository`.
Cloning this repository without Git LFS will clone a repository with LFS pointers, not the actual model.
After the repository is cloned and Git LFS is installed, set up with `git lfs install`.
Download specific models with:
```sh
git lfs pull --include="model_repository/<path-to-model-dir>" --exclude=""
```
Download ALL models with:
```sh
git lfs pull
# Or
#git lfs pull --include="*" --exclude=""
```
## Quickstart and Installation:
### *Pre-requisites:*
Requirements: Python 3.9+, PyTorch 2+, Triton Client
Optional: CUDA Compatible GPU, NVIDIA Drivers, cudnn (PyTorch pre-built wheels)
1. To install PyTorch with TorchVision, please refer to the installation instructions on their web page [here](https://pytorch.org/get-started/locally/#start-locally).
2. The tritonclient package wheels are not hosted on the public PyPI server. We need to add the address of NVIDA's private python package index to the environment. You can complete these steps and install the tritonclient package by running the following.
```sh
# If you cloned this repo, you can just uncomment and run the one line below
#sh ./scripts/install_triton_client.
pip install nvidia-pyindex
pip install tritonclient[all]
```
### **Install via. pip**
Once the above requirements are set, you can install fastnn with the command below:
```sh
pip install fastnn
```
If you are comfortable with the latest default stable releases of PyTorch you can skip step 1 in the pre-requisites and run `pip install fastnn[torch]` instead.
### **Install from source with Poetry for development**
You will need to install poetry by referring to the installation instructions on their web page [here](https://python-poetry.org/docs/#installation).
After cloning the repository, just run `poetry install` to install with the .lock file.
Activate the virtual environment with the command below:
```sh
poetry shell
```
### **Docker**
Official FastNN images are hosted on [Docker Hub](https://hub.docker.com/r/aychang/fastnn).
Select FastNN package and image versions by referencing the [documentation](https://andrewchang.dev/fastnn/index.html#fastnn-docker-release-selector-ubuntu-1804). Development and runtime environments are available.
Jupyter lab and notebook servers are accessible with notebook examples and terminal access `localhost:8888` with every image.
Run the latest FastNN image by running below:
```sh
docker run --gpus all --rm -it -p 8888:8888 aychang/fastnn:latest
```
Run images with specific configurations as can see in the example command below:
```sh
docker run --gpus all --rm -it -p 8888:8888 aychang/fastnn:0.1.0-cuda11.0-runtime-ubuntu18.04-py3.7
```
## Triton Inference Server: Local or Kubernetes Cluster
### **Local Deployment**
Requirements:
- Docker 19.03+
GPU Inference Requirements:
- NVIDIA CUDA-Compatible GPU
- [NVIDIA Container Toolkit (nvidia-docker)](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html)
Local deployment of the Triton Server uses the EXPLICIT model control mode. Local models must be explicitly specified with the `--load-model`
argument in `./scripts/start_triton_local.sh`
```sh
export MODEL_REPOSITORY=$(pwd)/model_repository
sh ./scripts/start_triton_local.sh
```
### **Helm Chart install in Kubernetes Cluster**
Requirements: kubectl 1.17+, Helm 3+, Kubernetes 1.17+
You can currently install the local FastNN helm chart with the following instuctions:
```sh
cd ./k8s
helm install fastnn .
export MODEL_REPOSITORY=$(pwd)/model_repository
```
Note: The current local helm chart installation deploys Triton using the NONE model control mode. All models available in the S3 Model Zoo will be deployed...good luck.
Deployed models and model control mode can be edited in the helm chart deployment configuration file.
# License
This project is licensed under the terms of the MIT license.
Raw data
{
"_id": null,
"home_page": "https://andrewchang.dev/fastnn/",
"name": "fastnn",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9,<3.12",
"maintainer_email": "",
"keywords": "",
"author": "Andrew Chang",
"author_email": "aychang995@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/a9/2d/7f964c3926d8f65fc8f82c1a7ee12528d9238eded2588cb19c68cfd9637e/fastnn-0.4.0.tar.gz",
"platform": null,
"description": "# Fast Neural Networks (FastNN)\n\nA framework for deploying serializable and optimizable neural net models at scale in production via. the NVIDIA Triton Inference Server.\n\n<p align=\"center\">\n <a href=\"https://hub.docker.com/r/aychang/fastnn\">\n <img src=\"https://img.shields.io/docker/cloud/build/aychang/fastnn\"\n </a>\n <a href=\"https://badge.fury.io/py/fastnn\">\n <img src=\"https://badge.fury.io/py/fastnn.svg\">\n </a>\n <a href=\"https://github.com/aychang95/fastnn/blob/master/LICENSE\">\n <img src=\"https://img.shields.io/github/license/aychang95/fastnn\">\n </a>\n</p>\n\n## [**FastNN Docker Release Selector (Ubuntu 18.04)**](https://andrewchang.dev/fastnn/index.html#fastnn-docker-release-selector-ubuntu-1804)\n\n## [Documentation](https://andrewchang.dev/fastnn) - Guides, Models, API References\n\n## Features:\n - **Data Processing**\n - Intuitive data processing modules for encoding human-readible data into tensors compatible with deep learning libraries\n - **Model Exporting**\n - FastNN torch modules and tools for exporting models via. `TorchScript` tracing and scripting to a production environment. Now includes Text Generation.\n - **Model Zoo**\n - Various exported models hosted in this repo via. git-lfs and AWS S3. Includes models from the HuggingFace's Transformers and \n TorchVision\n - **Model Deployment**\n - Deploy models using Triton Inference Server on CPU/GPU-compatible server(s) with helm or docker\n - **FastNN Client**\n - Client wrapper for Triton Inference Server's client module for programmatic requests with python\n\n\n## Pre-Requisites:\n\nGit LFS is required if you'd like to use any of the models provided by FastNN in `./model_repository`.\n\nCloning this repository without Git LFS will clone a repository with LFS pointers, not the actual model.\n\nAfter the repository is cloned and Git LFS is installed, set up with `git lfs install`.\n\nDownload specific models with:\n\n```sh\ngit lfs pull --include=\"model_repository/<path-to-model-dir>\" --exclude=\"\"\n```\n\nDownload ALL models with:\n\n```sh\ngit lfs pull\n\n# Or\n#git lfs pull --include=\"*\" --exclude=\"\"\n```\n\n\n## Quickstart and Installation:\n\n### *Pre-requisites:*\n\nRequirements: Python 3.9+, PyTorch 2+, Triton Client\n\nOptional: CUDA Compatible GPU, NVIDIA Drivers, cudnn (PyTorch pre-built wheels)\n\n1. To install PyTorch with TorchVision, please refer to the installation instructions on their web page [here](https://pytorch.org/get-started/locally/#start-locally).\n\n2. The tritonclient package wheels are not hosted on the public PyPI server. We need to add the address of NVIDA's private python package index to the environment. You can complete these steps and install the tritonclient package by running the following.\n\n```sh\n# If you cloned this repo, you can just uncomment and run the one line below\n#sh ./scripts/install_triton_client.\npip install nvidia-pyindex\npip install tritonclient[all]\n```\n\n### **Install via. pip**\n\nOnce the above requirements are set, you can install fastnn with the command below:\n\n```sh\npip install fastnn\n```\n\nIf you are comfortable with the latest default stable releases of PyTorch you can skip step 1 in the pre-requisites and run `pip install fastnn[torch]` instead.\n\n\n### **Install from source with Poetry for development**\n\nYou will need to install poetry by referring to the installation instructions on their web page [here](https://python-poetry.org/docs/#installation).\n\nAfter cloning the repository, just run `poetry install` to install with the .lock file.\n\nActivate the virtual environment with the command below:\n\n```sh\npoetry shell\n```\n\n\n### **Docker**\n\nOfficial FastNN images are hosted on [Docker Hub](https://hub.docker.com/r/aychang/fastnn).\n\nSelect FastNN package and image versions by referencing the [documentation](https://andrewchang.dev/fastnn/index.html#fastnn-docker-release-selector-ubuntu-1804). Development and runtime environments are available.\n\nJupyter lab and notebook servers are accessible with notebook examples and terminal access `localhost:8888` with every image.\n\n\nRun the latest FastNN image by running below:\n\n```sh\ndocker run --gpus all --rm -it -p 8888:8888 aychang/fastnn:latest\n```\n\nRun images with specific configurations as can see in the example command below:\n\n```sh\ndocker run --gpus all --rm -it -p 8888:8888 aychang/fastnn:0.1.0-cuda11.0-runtime-ubuntu18.04-py3.7\n\n```\n\n\n## Triton Inference Server: Local or Kubernetes Cluster\n\n\n### **Local Deployment**\n\nRequirements:\n - Docker 19.03+\n\nGPU Inference Requirements:\n\n - NVIDIA CUDA-Compatible GPU\n \n - [NVIDIA Container Toolkit (nvidia-docker)](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html)\n\nLocal deployment of the Triton Server uses the EXPLICIT model control mode. Local models must be explicitly specified with the `--load-model` \nargument in `./scripts/start_triton_local.sh`\n\n```sh\n\nexport MODEL_REPOSITORY=$(pwd)/model_repository\nsh ./scripts/start_triton_local.sh\n\n```\n\n\n### **Helm Chart install in Kubernetes Cluster**\n\nRequirements: kubectl 1.17+, Helm 3+, Kubernetes 1.17+\n\nYou can currently install the local FastNN helm chart with the following instuctions:\n\n```sh\n\ncd ./k8s\nhelm install fastnn .\nexport MODEL_REPOSITORY=$(pwd)/model_repository\n\n```\n\nNote: The current local helm chart installation deploys Triton using the NONE model control mode. All models available in the S3 Model Zoo will be deployed...good luck. \nDeployed models and model control mode can be edited in the helm chart deployment configuration file.\n\n# License\n\nThis project is licensed under the terms of the MIT license.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A python library and framework for fast neural network computations.",
"version": "0.4.0",
"project_urls": {
"Documentation": "https://andrewchang.dev/fastnn/",
"Homepage": "https://andrewchang.dev/fastnn/",
"Repository": "https://github.com/aychang95/fastnn"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "87163e4eebebbf440cf8977077f845b807f271e920e1b549dc9acf12b4c95dd5",
"md5": "d22a4e2f91889f7cfde44dea9c31eaa6",
"sha256": "d1574665559b74a19a0dec850bbdc8141c8bd49faee3c4e8da6f70e1a3d33755"
},
"downloads": -1,
"filename": "fastnn-0.4.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d22a4e2f91889f7cfde44dea9c31eaa6",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9,<3.12",
"size": 33294,
"upload_time": "2023-11-01T21:08:16",
"upload_time_iso_8601": "2023-11-01T21:08:16.669793Z",
"url": "https://files.pythonhosted.org/packages/87/16/3e4eebebbf440cf8977077f845b807f271e920e1b549dc9acf12b4c95dd5/fastnn-0.4.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a92d7f964c3926d8f65fc8f82c1a7ee12528d9238eded2588cb19c68cfd9637e",
"md5": "2c5de5794f1f56ec5ab88a10cf488fb9",
"sha256": "93b01b7dcb8ea22f131eda9cad22bdef555e78d8dde58b5fcd49df87411f7f70"
},
"downloads": -1,
"filename": "fastnn-0.4.0.tar.gz",
"has_sig": false,
"md5_digest": "2c5de5794f1f56ec5ab88a10cf488fb9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9,<3.12",
"size": 28766,
"upload_time": "2023-11-01T21:08:18",
"upload_time_iso_8601": "2023-11-01T21:08:18.183129Z",
"url": "https://files.pythonhosted.org/packages/a9/2d/7f964c3926d8f65fc8f82c1a7ee12528d9238eded2588cb19c68cfd9637e/fastnn-0.4.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-01 21:08:18",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "aychang95",
"github_project": "fastnn",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "fastnn"
}
