## fastDeploy
#### easy and performant micro-services for Python Deep Learning inference pipelines
- Deploy any python inference pipeline with minimal extra code
- Auto batching of concurrent inputs is enabled out of the box
- no changes to inference code (unlike tf-serving etc), entire pipeline is run as is
- Promethues metrics (open metrics) are exposed for monitoring
- Auto generates clean dockerfiles and kubernetes health check, scaling friendly APIs
- sequentially chained inference pipelines are supported out of the box
- can be queried from any language via easy to use rest apis
- easy to understand (simple consumer producer arch) and simple code base
#### Installation:
```bash
pip install --upgrade fastdeploy fdclient
# fdclient is optional, only needed if you want to use python client
```
#### [CLI explained](https://github.com/notAI-tech/fastDeploy/blob/master/cli.md)
#### Start fastDeploy server on a recipe:
```bash
# Invoke fastdeploy
python -m fastdeploy --help
# or
fastdeploy --help
# Start prediction "loop" for recipe "echo"
fastdeploy --loop --recipe recipes/echo
# Start rest apis for recipe "echo"
fastdeploy --rest --recipe recipes/echo
```
#### Send a request and get predictions:
- [Python client usage](https://github.com/notAI-tech/fastDeploy/blob/master/clients/python/README.md)
- [curl usage]()
- [Nodejs client usage]()
#### auto generate dockerfile and build docker image:
```bash
# Write the dockerfile for recipe "echo"
# and builds the docker image if docker is installed
# base defaults to python:3.8-slim
fastdeploy --build --recipe recipes/echo
# Run docker image
docker run -it -p8080:8080 fastdeploy_echo
```
#### Serving your model (recipe):
- [Writing your model/pipeline's recipe](https://github.com/notAI-tech/fastDeploy/blob/master/recipe.md)
### Where to use fastDeploy?
- to deploy any non ultra light weight models i.e: most DL models, >50ms inference time per example
- if the model/pipeline benefits from batch inference, fastDeploy is perfect for your use-case
- if you are going to have individual inputs (example, user's search input which needs to be vectorized or image to be classified)
- in the case of individual inputs, requests coming in at close intervals will be batched together and sent to the model as a batch
- perfect for creating internal micro services separating your model, pre and post processing from business logic
- since prediction loop and inference endpoints are separated and are connected via sqlite backed queue, can be scaled independently
### Where not to use fastDeploy?
- non cpu/gpu heavy models that are better of running parallely rather than in batch
- if your predictor calls some external API or uploads to s3 etc in a blocking way
- io heavy non batching use cases (eg: query ES or db for each input)
- for these cases better to directly do from rest api code (instead of consumer producer mechanism) so that high concurrency can be achieved
Raw data
{
"_id": null,
"home_page": "https://github.com/notAI-tech/fastDeploy",
"name": "fastdeploy",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6.0",
"maintainer_email": null,
"keywords": null,
"author": "BEDAPUDI PRANEETH",
"author_email": "praneeth@bpraneeth.com",
"download_url": "https://files.pythonhosted.org/packages/f0/87/35d833b9c938f29ab873714e4a744c83545a6a27725bfb7e2b63e13c8a9e/fastdeploy-3.1.1.tar.gz",
"platform": null,
"description": "\n## fastDeploy\n#### easy and performant micro-services for Python Deep Learning inference pipelines\n\n- Deploy any python inference pipeline with minimal extra code\n- Auto batching of concurrent inputs is enabled out of the box\n- no changes to inference code (unlike tf-serving etc), entire pipeline is run as is\n- Promethues metrics (open metrics) are exposed for monitoring\n- Auto generates clean dockerfiles and kubernetes health check, scaling friendly APIs\n- sequentially chained inference pipelines are supported out of the box\n- can be queried from any language via easy to use rest apis\n- easy to understand (simple consumer producer arch) and simple code base\n\n\n#### Installation:\n```bash\npip install --upgrade fastdeploy fdclient\n# fdclient is optional, only needed if you want to use python client\n```\n\n#### [CLI explained](https://github.com/notAI-tech/fastDeploy/blob/master/cli.md)\n\n#### Start fastDeploy server on a recipe: \n```bash\n# Invoke fastdeploy \npython -m fastdeploy --help\n# or\nfastdeploy --help\n\n# Start prediction \"loop\" for recipe \"echo\"\nfastdeploy --loop --recipe recipes/echo\n\n# Start rest apis for recipe \"echo\"\nfastdeploy --rest --recipe recipes/echo\n```\n\n#### Send a request and get predictions:\n\n- [Python client usage](https://github.com/notAI-tech/fastDeploy/blob/master/clients/python/README.md)\n\n- [curl usage]()\n\n- [Nodejs client usage]()\n\n#### auto generate dockerfile and build docker image:\n```bash\n# Write the dockerfile for recipe \"echo\"\n# and builds the docker image if docker is installed\n# base defaults to python:3.8-slim\nfastdeploy --build --recipe recipes/echo\n\n# Run docker image\ndocker run -it -p8080:8080 fastdeploy_echo\n```\n\n#### Serving your model (recipe):\n\n- [Writing your model/pipeline's recipe](https://github.com/notAI-tech/fastDeploy/blob/master/recipe.md)\n\n\n### Where to use fastDeploy?\n\n- to deploy any non ultra light weight models i.e: most DL models, >50ms inference time per example\n- if the model/pipeline benefits from batch inference, fastDeploy is perfect for your use-case\n- if you are going to have individual inputs (example, user's search input which needs to be vectorized or image to be classified)\n- in the case of individual inputs, requests coming in at close intervals will be batched together and sent to the model as a batch\n- perfect for creating internal micro services separating your model, pre and post processing from business logic\n- since prediction loop and inference endpoints are separated and are connected via sqlite backed queue, can be scaled independently\n\n\n### Where not to use fastDeploy?\n- non cpu/gpu heavy models that are better of running parallely rather than in batch\n- if your predictor calls some external API or uploads to s3 etc in a blocking way\n- io heavy non batching use cases (eg: query ES or db for each input)\n- for these cases better to directly do from rest api code (instead of consumer producer mechanism) so that high concurrency can be achieved\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Deploy DL/ ML inference pipelines with minimal extra code.",
"version": "3.1.1",
"project_urls": {
"Homepage": "https://github.com/notAI-tech/fastDeploy"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "84f1a8bf61b9dd0b58a8d0a73e42e73c4fe7e77206757dca7035018003f8bf99",
"md5": "f8ee98bafd0622a2f0e87dd82fec0f3c",
"sha256": "4c209874f6884c74d3d33d3b9f76b54ef8641b1bb8f0384eab782c3f6b36180b"
},
"downloads": -1,
"filename": "fastdeploy-3.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f8ee98bafd0622a2f0e87dd82fec0f3c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6.0",
"size": 16683,
"upload_time": "2024-11-10T13:34:08",
"upload_time_iso_8601": "2024-11-10T13:34:08.500189Z",
"url": "https://files.pythonhosted.org/packages/84/f1/a8bf61b9dd0b58a8d0a73e42e73c4fe7e77206757dca7035018003f8bf99/fastdeploy-3.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f08735d833b9c938f29ab873714e4a744c83545a6a27725bfb7e2b63e13c8a9e",
"md5": "d195699531dc2a0a0774b7a2ba4a72a5",
"sha256": "d080338d0806b8176df0a8779f6c0fbb4f1fa6379cb527eca2457d5608e7f329"
},
"downloads": -1,
"filename": "fastdeploy-3.1.1.tar.gz",
"has_sig": false,
"md5_digest": "d195699531dc2a0a0774b7a2ba4a72a5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6.0",
"size": 16886,
"upload_time": "2024-11-10T13:34:10",
"upload_time_iso_8601": "2024-11-10T13:34:10.489803Z",
"url": "https://files.pythonhosted.org/packages/f0/87/35d833b9c938f29ab873714e4a744c83545a6a27725bfb7e2b63e13c8a9e/fastdeploy-3.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-10 13:34:10",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "notAI-tech",
"github_project": "fastDeploy",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "fastdeploy"
}
JSON
