| Name | model-inference-fastapi JSON |
| Version |
0.0.1a0
JSON |
| download |
| home_page | |
| Summary | Use fastapi for model inference over http |
| upload_time | 2023-03-19 13:52:53 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.7 |
| license | MIT License Copyright (c) 2023 Robots Limited Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
api
deep learning
deploy
inference
machine learning
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# model-inference-fastapi
Running inference on ML model over http using fastapi
## Installation
```shell
pip install model-inference-fastapi
```
## Usage
1. Create a model service file `model_service.py` implementing the `ModelServiceBase` interface like so.
```python
from typing import Dict, List, Optional
from model_inference_fastapi.model_service_base import ModelServiceBase
class ModelService(ModelServiceBase):
__model = None
@staticmethod
def validate(data: Optional[List[Dict]] = None, image_files: Optional[List[bytes]] = None) -> Optional[str]:
return
@staticmethod
def predict(data: Optional[List[Dict]] = None, image_files: Optional[List[bytes]] = None) -> List[Dict]:
return [{"status": "Ready to Go!!!"}]
```
For instance if you are trying to deploy an image classification model (
e.g. [ResNet50](https://arxiv.org/abs/1512.03385)) using [torch](https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html). You will need
the following `model_service.py` example in pytorch.
```python
import json
from typing import Dict, List, Optional
import torch
from torchvision.models import resnet50
from torchvision.transforms import Resize, Compose, CenterCrop, ToTensor, Normalize
from model_inference_fastapi.model_service_base import ModelServiceBase
from model_inference_fastapi.utils import convert_file_to_pil_image
# derived from https://keras.io/api/applications/
class ModelService(ModelServiceBase):
__model = resnet50(weights="IMAGENET1K_V2")
__transform = Compose([
Resize(size=256),
CenterCrop(size=224),
ToTensor(),
Normalize([0.485, 0.456, 0.406],
[0.229, 0.224, 0.225])
])
with open("imagenet1000_clsidx_to_labels.txt") as f:
__class_names = eval(f.read())
@staticmethod
def validate(*, data: Optional[List[Dict]] = None, image_files: Optional[List[bytes]] = None):
if len(image_files) != 1:
return "There must be at least one image file in image_files"
@staticmethod
def predict(*, data: Optional[List[Dict]] = None, image_files: Optional[List[bytes]] = None) -> List[Dict]:
test_image_tensor = ModelService.__transform(convert_file_to_pil_image(image_files[0]))
test_image_tensor = test_image_tensor.view(1, 3, 224, 224)
with torch.no_grad():
ModelService.__model.eval()
# Model outputs log probabilities
out = ModelService.__model(test_image_tensor)
ps = torch.exp(out)
top_k, top_class = ps.topk(1, dim=1)
print("Output class : ", ModelService.__class_names[top_class.cpu().numpy()[0][0]])
return [{"class": ModelService.__class_names[top_class.cpu().numpy()[0][0]]}]
```
2. Run Server
```shell
uvicorn model_inference_fastapi.main:app
```
3. Visit the API testing page powered by FastAPI here http://127.0.0.1:8000/docs/.
- Navigate to the `/predict/` AKA _Prediction Route_ dropdown
- Click the **Try it out button**
- For the `image_files` _**array**_ click Add string item button
- Choose a file to upload
- Click **_execute_**
- Scroll down to see the server response
## Testing `model_service.py`
Here is a sample test file for image classification `model_service.py`
```python
# replace this import with import for you own `model_service.py`
from model_inference_fastapi.model_service import ModelService
def test_predict():
with open('/path/to/image.jpg', 'rb') as file:
prediction = ModelService.predict(image_files=[file.read()])
assert prediction[0]["class"] == "dog"
```
### Using the docker image
See example dir for more info.
1. create a `Dockerfile` like so
```Dockerfile
FROM robotstech/model-inference-fastapi
EXPOSE 8000
WORKDIR /app
COPY requirements.txt ./requirements.txt
RUN pip install --upgrade pip
RUN pip install -r requirements.txt
# move model_service.py
COPY ./model_service.py ./model_inference_fastapi/
# move data or other resources to root dir of model-inference
COPY ./imagenet1000_clsidx_to_labels.txt .
```
The key info here is to move the `model_service.py` and **_data/resource_** to the right directory
Raw data
{
"_id": null,
"home_page": "",
"name": "model-inference-fastapi",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "iamr0b0tx <tech@robotslimited.com>",
"keywords": "api,deep learning,deploy,inference,machine learning",
"author": "",
"author_email": "iamr0b0tx <tech@robotslimited.com>",
"download_url": "https://files.pythonhosted.org/packages/b4/05/72307b1e6a1a20a083006376ab48252128e3ef04ec546832011dc5d51e68/model_inference_fastapi-0.0.1a0.tar.gz",
"platform": null,
"description": "# model-inference-fastapi\n\nRunning inference on ML model over http using fastapi\n\n## Installation\n\n```shell\npip install model-inference-fastapi\n```\n\n## Usage\n\n1. Create a model service file `model_service.py` implementing the `ModelServiceBase` interface like so.\n ```python\n from typing import Dict, List, Optional\n from model_inference_fastapi.model_service_base import ModelServiceBase\n \n \n class ModelService(ModelServiceBase):\n __model = None\n \n @staticmethod\n def validate(data: Optional[List[Dict]] = None, image_files: Optional[List[bytes]] = None) -> Optional[str]:\n return\n \n @staticmethod\n def predict(data: Optional[List[Dict]] = None, image_files: Optional[List[bytes]] = None) -> List[Dict]:\n return [{\"status\": \"Ready to Go!!!\"}]\n ```\n\n For instance if you are trying to deploy an image classification model (\n e.g. [ResNet50](https://arxiv.org/abs/1512.03385)) using [torch](https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html). You will need\n the following `model_service.py` example in pytorch.\n ```python\n import json\n from typing import Dict, List, Optional\n \n import torch\n from torchvision.models import resnet50\n from torchvision.transforms import Resize, Compose, CenterCrop, ToTensor, Normalize\n \n from model_inference_fastapi.model_service_base import ModelServiceBase\n from model_inference_fastapi.utils import convert_file_to_pil_image\n \n \n # derived from https://keras.io/api/applications/\n class ModelService(ModelServiceBase):\n __model = resnet50(weights=\"IMAGENET1K_V2\")\n __transform = Compose([\n Resize(size=256),\n CenterCrop(size=224),\n ToTensor(),\n Normalize([0.485, 0.456, 0.406],\n [0.229, 0.224, 0.225])\n ])\n \n with open(\"imagenet1000_clsidx_to_labels.txt\") as f:\n __class_names = eval(f.read())\n \n @staticmethod\n def validate(*, data: Optional[List[Dict]] = None, image_files: Optional[List[bytes]] = None):\n if len(image_files) != 1:\n return \"There must be at least one image file in image_files\"\n \n @staticmethod\n def predict(*, data: Optional[List[Dict]] = None, image_files: Optional[List[bytes]] = None) -> List[Dict]:\n test_image_tensor = ModelService.__transform(convert_file_to_pil_image(image_files[0]))\n test_image_tensor = test_image_tensor.view(1, 3, 224, 224)\n \n with torch.no_grad():\n ModelService.__model.eval()\n # Model outputs log probabilities\n out = ModelService.__model(test_image_tensor)\n ps = torch.exp(out)\n top_k, top_class = ps.topk(1, dim=1)\n print(\"Output class : \", ModelService.__class_names[top_class.cpu().numpy()[0][0]])\n \n return [{\"class\": ModelService.__class_names[top_class.cpu().numpy()[0][0]]}]\n ```\n\n2. Run Server\n ```shell\n uvicorn model_inference_fastapi.main:app\n ```\n\n3. Visit the API testing page powered by FastAPI here http://127.0.0.1:8000/docs/.\n - Navigate to the `/predict/` AKA _Prediction Route_ dropdown\n - Click the **Try it out button**\n - For the `image_files` _**array**_ click Add string item button\n - Choose a file to upload\n - Click **_execute_**\n - Scroll down to see the server response\n\n## Testing `model_service.py`\n\nHere is a sample test file for image classification `model_service.py`\n\n```python\n# replace this import with import for you own `model_service.py`\nfrom model_inference_fastapi.model_service import ModelService\n\ndef test_predict():\n with open('/path/to/image.jpg', 'rb') as file:\n prediction = ModelService.predict(image_files=[file.read()])\n assert prediction[0][\"class\"] == \"dog\" \n```\n\n### Using the docker image\nSee example dir for more info.\n1. create a `Dockerfile` like so\n```Dockerfile\nFROM robotstech/model-inference-fastapi\n\nEXPOSE 8000\nWORKDIR /app\n\nCOPY requirements.txt ./requirements.txt\n\nRUN pip install --upgrade pip\nRUN pip install -r requirements.txt\n\n# move model_service.py\nCOPY ./model_service.py ./model_inference_fastapi/\n\n# move data or other resources to root dir of model-inference\nCOPY ./imagenet1000_clsidx_to_labels.txt .\n```\n\nThe key info here is to move the `model_service.py` and **_data/resource_** to the right directory\n",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2023 Robots Limited Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.",
"summary": "Use fastapi for model inference over http",
"version": "0.0.1a0",
"split_keywords": [
"api",
"deep learning",
"deploy",
"inference",
"machine learning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c46baedb446468f3939b8d06ccd18f68db0c6061b2bbc1c1a9512b5a30727960",
"md5": "1a081543421e9e085af7afdaf8ad3c0c",
"sha256": "72481c2beb7166c17290e6dfdca26c09b67237e9cc22228d2ce48dd0ab8a59eb"
},
"downloads": -1,
"filename": "model_inference_fastapi-0.0.1a0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1a081543421e9e085af7afdaf8ad3c0c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 7158,
"upload_time": "2023-03-19T13:52:51",
"upload_time_iso_8601": "2023-03-19T13:52:51.522750Z",
"url": "https://files.pythonhosted.org/packages/c4/6b/aedb446468f3939b8d06ccd18f68db0c6061b2bbc1c1a9512b5a30727960/model_inference_fastapi-0.0.1a0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "b40572307b1e6a1a20a083006376ab48252128e3ef04ec546832011dc5d51e68",
"md5": "15e59fe6c3096624933ec3023e7e4633",
"sha256": "14ae01ecb8e384d1090ff2d50531f75da88d6ed195178ed9ac22e9b2d7b458ea"
},
"downloads": -1,
"filename": "model_inference_fastapi-0.0.1a0.tar.gz",
"has_sig": false,
"md5_digest": "15e59fe6c3096624933ec3023e7e4633",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 63362,
"upload_time": "2023-03-19T13:52:53",
"upload_time_iso_8601": "2023-03-19T13:52:53.661649Z",
"url": "https://files.pythonhosted.org/packages/b4/05/72307b1e6a1a20a083006376ab48252128e3ef04ec546832011dc5d51e68/model_inference_fastapi-0.0.1a0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-03-19 13:52:53",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "model-inference-fastapi"
}
