Project Description
===================
*This version of multi-model-server has been forked from version 1.1.11 and altered for multithreading GPU inference in scenarios where preload_model=True. Using the traditional multiprocessing library resulted in shortcomings when attempting to share models where GPU memory was allocated. This fork is an attempt to solve that. All other work within this library is to the credit of the original authors. ~Blake@duke.ai*
===================
Multi Model Server (MMS) is a flexible and easy to use tool for
serving deep learning models exported from `MXNet <http://mxnet.io/>`__
or the Open Neural Network Exchange (`ONNX <http://onnx.ai/>`__).
Use the MMS Server CLI, or the pre-configured Docker images, to start a
service that sets up HTTP endpoints to handle model inference requests.
Detailed documentation and examples are provided in the `docs
folder <https://github.com/awslabs/multi-model-server/blob/master/docs/README.md>`__.
Prerequisites
-------------
* **java 8**: Required. MMS use java to serve HTTP requests. You must install java 8 (or later) and make sure java is on available in $PATH environment variable *before* installing MMS. If you have multiple java installed, you can use $JAVA_HOME environment vairable to control which java to use.
* **mxnet**: `mxnet` will not be installed by default with MMS 1.0 any more. You have to install it manually if you use MxNet.
For ubuntu:
::
sudo apt-get install openjdk-8-jre-headless
For centos
::
sudo yum install java-1.8.0-openjdk
For Mac:
::
brew tap caskroom/versions
brew update
brew cask install java8
Install MxNet:
::
pip install mxnet
MXNet offers MKL pip packages that will be much faster when running on Intel hardware.
To install mkl package for CPU:
::
pip install mxnet-mkl
or for GPU instance:
::
pip install mxnet-cu92mkl
Installation
------------
::
pip install multi-model-server
Development
-----------
We welcome new contributors of all experience levels. For information on
how to install MMS for development, refer to the `MMS
docs <https://github.com/awslabs/multi-model-server/blob/master/docs/install.md>`__.
Important links
---------------
- `Official source code
repo <https://github.com/awslabs/multi-model-server>`__
- `Download
releases <https://pypi.org/project/multi-model-server/#files>`__
- `Issue
tracker <https://github.com/awslabs/multi-model-server/issues>`__
Source code
-----------
You can check the latest source code as follows:
::
git clone https://github.com/awslabs/multi-model-server.git
Testing
-------
After installation, try out the MMS Quickstart for
- `Serving a Model <https://github.com/awslabs/multi-model-server/blob/master/README.md#serve-a-model>`__
- `Create a Model Archive <https://github.com/awslabs/multi-model-server/blob/master/README.md#model-archive>`__.
Help and Support
----------------
- `Documentation <https://github.com/awslabs/multi-model-server/blob/master/docs/README.md>`__
- `Forum <https://discuss.mxnet.io/latest>`__
Citation
--------
If you use MMS in a publication or project, please cite MMS:
https://github.com/awslabs/multi-model-server
Raw data
{
"_id": null,
"home_page": "https://github.com/deathstarenterprise/multi-model-server-gpu",
"name": "multi-model-server-gpu",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "Multi Model Server Serving Deep Learning Inference AI (Altered for multiprocessing GPU inference)",
"author": "Trinity team + Blake Donahoo",
"author_email": "noreply@amazon.com, blake@duke.ai",
"download_url": "https://files.pythonhosted.org/packages/aa/b2/9f689a4e39adf85ea157b2bb1bd78a0202bb9c12b28945feb4c690386f8f/multi_model_server_gpu-0.0.2.tar.gz",
"platform": null,
"description": "Project Description\n===================\n\n*This version of multi-model-server has been forked from version 1.1.11 and altered for multithreading GPU inference in scenarios where preload_model=True. Using the traditional multiprocessing library resulted in shortcomings when attempting to share models where GPU memory was allocated. This fork is an attempt to solve that. All other work within this library is to the credit of the original authors. ~Blake@duke.ai*\n\n===================\n\nMulti Model Server (MMS) is a flexible and easy to use tool for\nserving deep learning models exported from `MXNet <http://mxnet.io/>`__\nor the Open Neural Network Exchange (`ONNX <http://onnx.ai/>`__).\n\nUse the MMS Server CLI, or the pre-configured Docker images, to start a\nservice that sets up HTTP endpoints to handle model inference requests.\n\nDetailed documentation and examples are provided in the `docs\nfolder <https://github.com/awslabs/multi-model-server/blob/master/docs/README.md>`__.\n\nPrerequisites\n-------------\n\n* **java 8**: Required. MMS use java to serve HTTP requests. You must install java 8 (or later) and make sure java is on available in $PATH environment variable *before* installing MMS. If you have multiple java installed, you can use $JAVA_HOME environment vairable to control which java to use.\n* **mxnet**: `mxnet` will not be installed by default with MMS 1.0 any more. You have to install it manually if you use MxNet.\n\nFor ubuntu:\n::\n\n sudo apt-get install openjdk-8-jre-headless\n\n\nFor centos\n::\n\n sudo yum install java-1.8.0-openjdk\n\n\nFor Mac:\n::\n\n brew tap caskroom/versions\n brew update\n brew cask install java8\n\n\nInstall MxNet:\n::\n\n pip install mxnet\n\nMXNet offers MKL pip packages that will be much faster when running on Intel hardware.\nTo install mkl package for CPU:\n::\n\n pip install mxnet-mkl\n\nor for GPU instance:\n\n::\n\n pip install mxnet-cu92mkl\n\n\nInstallation\n------------\n\n::\n\n pip install multi-model-server\n\nDevelopment\n-----------\n\nWe welcome new contributors of all experience levels. For information on\nhow to install MMS for development, refer to the `MMS\ndocs <https://github.com/awslabs/multi-model-server/blob/master/docs/install.md>`__.\n\nImportant links\n---------------\n\n- `Official source code\n repo <https://github.com/awslabs/multi-model-server>`__\n- `Download\n releases <https://pypi.org/project/multi-model-server/#files>`__\n- `Issue\n tracker <https://github.com/awslabs/multi-model-server/issues>`__\n\nSource code\n-----------\n\nYou can check the latest source code as follows:\n\n::\n\n git clone https://github.com/awslabs/multi-model-server.git\n\nTesting\n-------\n\nAfter installation, try out the MMS Quickstart for\n\n- `Serving a Model <https://github.com/awslabs/multi-model-server/blob/master/README.md#serve-a-model>`__\n- `Create a Model Archive <https://github.com/awslabs/multi-model-server/blob/master/README.md#model-archive>`__.\n\nHelp and Support\n----------------\n\n- `Documentation <https://github.com/awslabs/multi-model-server/blob/master/docs/README.md>`__\n- `Forum <https://discuss.mxnet.io/latest>`__\n\nCitation\n--------\n\nIf you use MMS in a publication or project, please cite MMS:\nhttps://github.com/awslabs/multi-model-server\n",
"bugtrack_url": null,
"license": "Apache License Version 2.0",
"summary": "(Altered for multiprocessing GPU Inference) Multi Model Server is a tool for serving neural net models for inference",
"version": "0.0.2",
"project_urls": {
"Homepage": "https://github.com/deathstarenterprise/multi-model-server-gpu"
},
"split_keywords": [
"multi",
"model",
"server",
"serving",
"deep",
"learning",
"inference",
"ai",
"(altered",
"for",
"multiprocessing",
"gpu",
"inference)"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c152dd74513dc5758b93b9d59539336948b82ddba243d9272a352aee13c6928f",
"md5": "43461c59eb74192223240be8e687045c",
"sha256": "d167071905fe742ceb2c068d479672c15241a7eca55cca5e5beb8307e1a38922"
},
"downloads": -1,
"filename": "multi_model_server_gpu-0.0.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "43461c59eb74192223240be8e687045c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 53547,
"upload_time": "2024-09-09T19:18:01",
"upload_time_iso_8601": "2024-09-09T19:18:01.550592Z",
"url": "https://files.pythonhosted.org/packages/c1/52/dd74513dc5758b93b9d59539336948b82ddba243d9272a352aee13c6928f/multi_model_server_gpu-0.0.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "aab29f689a4e39adf85ea157b2bb1bd78a0202bb9c12b28945feb4c690386f8f",
"md5": "c6bcae09ea743a657d768873c22732e9",
"sha256": "5ec6a0d168b32389c24159c3e01094010dac23029fa77b20a01ad703057d2368"
},
"downloads": -1,
"filename": "multi_model_server_gpu-0.0.2.tar.gz",
"has_sig": false,
"md5_digest": "c6bcae09ea743a657d768873c22732e9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 38723,
"upload_time": "2024-09-09T19:18:04",
"upload_time_iso_8601": "2024-09-09T19:18:04.566666Z",
"url": "https://files.pythonhosted.org/packages/aa/b2/9f689a4e39adf85ea157b2bb1bd78a0202bb9c12b28945feb4c690386f8f/multi_model_server_gpu-0.0.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-09 19:18:04",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "deathstarenterprise",
"github_project": "multi-model-server-gpu",
"travis_ci": false,
"coveralls": true,
"github_actions": false,
"circle": true,
"lcname": "multi-model-server-gpu"
}
JSON
