| Name | inference-gpu JSON |
| Version |
0.31.1
JSON |
| download |
| home_page | https://github.com/roboflow/inference |
| Summary | With no prior knowledge of machine learning or device-specific deployment, you can deploy a computer vision model to a range of devices and environments using Roboflow Inference. |
| upload_time | 2024-12-13 18:57:25 |
| maintainer | None |
| docs_url | None |
| author | Roboflow |
| requires_python | <3.13,>=3.8 |
| license | None |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
<div align="center">
<p>
<a align="center" href="" target="https://inference.roboflow.com/">
<img
width="100%"
src="https://github.com/roboflow/inference/blob/main/banner.png?raw=true"
>
</a>
</p>
<br>
[notebooks](https://github.com/roboflow/notebooks) | [supervision](https://github.com/roboflow/supervision) | [autodistill](https://github.com/autodistill/autodistill) | [maestro](https://github.com/roboflow/multimodal-maestro)
<br>
[](https://badge.fury.io/py/inference)
[](https://pypistats.org/packages/inference)
[](https://github.com/roboflow/inference/blob/main/LICENSE.core)
[](https://huggingface.co/spaces/Roboflow/workflows)
[](https://discord.gg/GbfgXGJ8Bk)
</div>
## 👋 hello
Roboflow Inference is an open-source platform designed to simplify the deployment of computer vision models. It enables developers to perform object detection, classification, and instance segmentation and utilize foundation models like [CLIP](https://inference.roboflow.com/foundation/clip), [Segment Anything](https://inference.roboflow.com/foundation/sam), and [YOLO-World](https://inference.roboflow.com/foundation/yolo_world) through a Python-native package, a self-hosted inference server, or a fully [managed API](https://docs.roboflow.com/).
Explore our [enterprise options](https://roboflow.com/sales) for advanced features like server deployment, active learning, and commercial licenses for YOLOv5 and YOLOv8.
## 💻 install
Inference package requires [**Python>=3.8,<=3.11**](https://www.python.org/). Click [here](https://inference.roboflow.com/quickstart/docker/) to learn more about running Inference inside Docker.
```bash
pip install inference
```
<details>
<summary>👉 additional considerations</summary>
- hardware
Enhance model performance in GPU-accelerated environments by installing CUDA-compatible dependencies.
```bash
pip install inference-gpu
```
- models
The `inference` and `inference-gpu` packages install only the minimal shared dependencies. Install model-specific dependencies to ensure code compatibility and license compliance. Learn more about the [models](https://inference.roboflow.com/#extras) supported by Inference.
```bash
pip install inference[yolo-world]
```
</details>
## 🔥 quickstart
Use Inference SDK to run models locally with just a few lines of code. The image input can be a URL, a numpy array (BGR), or a PIL image.
```python
from inference import get_model
model = get_model(model_id="yolov8n-640")
results = model.infer("https://media.roboflow.com/inference/people-walking.jpg")
```
<details>
<summary>👉 roboflow models</summary>
<br>
Set up your `ROBOFLOW_API_KEY` to access thousands of fine-tuned models shared by the [Roboflow Universe](https://universe.roboflow.com/) community and your custom model. Navigate to 🔑 keys section to learn more.
```python
from inference import get_model
model = get_model(model_id="soccer-players-5fuqs/1")
results = model.infer(
image="https://media.roboflow.com/inference/soccer.jpg",
confidence=0.5,
iou_threshold=0.5
)
```
</details>
<details>
<summary>👉 foundational models</summary>
- [CLIP Embeddings](https://inference.roboflow.com/foundation/clip) - generate text and image embeddings that you can use for zero-shot classification or assessing image similarity.
```python
from inference.models import Clip
model = Clip()
embeddings_text = clip.embed_text("a football match")
embeddings_image = model.embed_image("https://media.roboflow.com/inference/soccer.jpg")
```
- [Segment Anything](https://inference.roboflow.com/foundation/sam) - segment all objects visible in the image or only those associated with selected points or boxes.
```python
from inference.models import SegmentAnything
model = SegmentAnything()
result = model.segment_image("https://media.roboflow.com/inference/soccer.jpg")
```
- [YOLO-World](https://inference.roboflow.com/foundation/yolo_world) - an almost real-time zero-shot detector that enables the detection of any objects without any training.
```python
from inference.models import YOLOWorld
model = YOLOWorld(model_id="yolo_world/l")
result = model.infer(
image="https://media.roboflow.com/inference/dog.jpeg",
text=["person", "backpack", "dog", "eye", "nose", "ear", "tongue"],
confidence=0.03
)
```
</details>
## 📟 inference server
- deploy server
The inference server is distributed via Docker. Behind the scenes, inference will download and run the image that is appropriate for your hardware. [Here](https://inference.roboflow.com/quickstart/docker/#advanced-build-a-docker-container-from-scratch), you can learn more about the supported images.
```bash
inference server start
```
- run client
Consume inference server predictions using the HTTP client available in the Inference SDK.
```python
from inference_sdk import InferenceHTTPClient
client = InferenceHTTPClient(
api_url="http://localhost:9001",
api_key=<ROBOFLOW_API_KEY>
)
with client.use_model(model_id="soccer-players-5fuqs/1"):
predictions = client.infer("https://media.roboflow.com/inference/soccer.jpg")
```
If you're using the hosted API, change the local API URL to `https://detect.roboflow.com`. Accessing the hosted inference server and/or using any of the fine-tuned models require a `ROBOFLOW_API_KEY`. For further information, visit the 🔑 keys section.
## 🎥 inference pipeline
The inference pipeline is an efficient method for processing static video files and streams. Select a model, define the video source, and set a callback action. You can choose from predefined callbacks that allow you to [display results](https://inference.roboflow.com/docs/reference/inference/core/interfaces/stream/sinks/#inference.core.interfaces.stream.sinks.render_boxes) on the screen or [save them to a file](https://inference.roboflow.com/docs/reference/inference/core/interfaces/stream/sinks/#inference.core.interfaces.stream.sinks.VideoFileSink).
```python
from inference import InferencePipeline
from inference.core.interfaces.stream.sinks import render_boxes
pipeline = InferencePipeline.init(
model_id="yolov8x-1280",
video_reference="https://media.roboflow.com/inference/people-walking.mp4",
on_prediction=render_boxes
)
pipeline.start()
pipeline.join()
```
## 🔑 keys
Inference enables the deployment of a wide range of pre-trained and foundational models without an API key. To access thousands of fine-tuned models shared by the [Roboflow Universe](https://universe.roboflow.com/) community, [configure your](https://app.roboflow.com/settings/api) API key.
```bash
export ROBOFLOW_API_KEY=<YOUR_API_KEY>
```
## 📚 documentation
Visit our [documentation](https://inference.roboflow.com) to explore comprehensive guides, detailed API references, and a wide array of tutorials designed to help you harness the full potential of the Inference package.
## ⚡️ Model-specific extras
Explore the list of [`inference` extras](https://inference.roboflow.com/#extras) to install model-specific dependencies.
## © license
See the "Self Hosting and Edge Deployment" section of the [Roboflow Licensing](https://roboflow.com/licensing) documentation for information on how Roboflow Inference is licensed.
## 🏆 contribution
We would love your input to improve Roboflow Inference! Please see our [contributing guide](https://github.com/roboflow/inference/blob/master/CONTRIBUTING.md) to get started. Thank you to all of our contributors! 🙏
<br>
<div align="center">
<div align="center">
<a href="https://youtube.com/roboflow">
<img
src="https://media.roboflow.com/notebooks/template/icons/purple/youtube.png?ik-sdk-version=javascript-1.4.3&updatedAt=1672949634652"
width="3%"
/>
</a>
<img src="https://raw.githubusercontent.com/ultralytics/assets/main/social/logo-transparent.png" width="3%"/>
<a href="https://roboflow.com">
<img
src="https://media.roboflow.com/notebooks/template/icons/purple/roboflow-app.png?ik-sdk-version=javascript-1.4.3&updatedAt=1672949746649"
width="3%"
/>
</a>
<img src="https://raw.githubusercontent.com/ultralytics/assets/main/social/logo-transparent.png" width="3%"/>
<a href="https://www.linkedin.com/company/roboflow-ai/">
<img
src="https://media.roboflow.com/notebooks/template/icons/purple/linkedin.png?ik-sdk-version=javascript-1.4.3&updatedAt=1672949633691"
width="3%"
/>
</a>
<img src="https://raw.githubusercontent.com/ultralytics/assets/main/social/logo-transparent.png" width="3%"/>
<a href="https://docs.roboflow.com">
<img
src="https://media.roboflow.com/notebooks/template/icons/purple/knowledge.png?ik-sdk-version=javascript-1.4.3&updatedAt=1672949634511"
width="3%"
/>
</a>
<img src="https://raw.githubusercontent.com/ultralytics/assets/main/social/logo-transparent.png" width="3%"/>
<a href="https://disuss.roboflow.com">
<img
src="https://media.roboflow.com/notebooks/template/icons/purple/forum.png?ik-sdk-version=javascript-1.4.3&updatedAt=1672949633584"
width="3%"
/>
<img src="https://raw.githubusercontent.com/ultralytics/assets/main/social/logo-transparent.png" width="3%"/>
<a href="https://blog.roboflow.com">
<img
src="https://media.roboflow.com/notebooks/template/icons/purple/blog.png?ik-sdk-version=javascript-1.4.3&updatedAt=1672949633605"
width="3%"
/>
</a>
</a>
</div>
</div>
Raw data
{
"_id": null,
"home_page": "https://github.com/roboflow/inference",
"name": "inference-gpu",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.13,>=3.8",
"maintainer_email": null,
"keywords": null,
"author": "Roboflow",
"author_email": "help@roboflow.com",
"download_url": null,
"platform": null,
"description": "<div align=\"center\">\n <p>\n <a align=\"center\" href=\"\" target=\"https://inference.roboflow.com/\">\n <img\n width=\"100%\"\n src=\"https://github.com/roboflow/inference/blob/main/banner.png?raw=true\"\n >\n </a>\n </p>\n\n <br>\n\n[notebooks](https://github.com/roboflow/notebooks) | [supervision](https://github.com/roboflow/supervision) | [autodistill](https://github.com/autodistill/autodistill) | [maestro](https://github.com/roboflow/multimodal-maestro)\n\n <br>\n\n[](https://badge.fury.io/py/inference)\n[](https://pypistats.org/packages/inference)\n\n[](https://github.com/roboflow/inference/blob/main/LICENSE.core)\n[](https://huggingface.co/spaces/Roboflow/workflows)\n[](https://discord.gg/GbfgXGJ8Bk)\n\n</div>\n\n## \ud83d\udc4b hello\n\nRoboflow Inference is an open-source platform designed to simplify the deployment of computer vision models. It enables developers to perform object detection, classification, and instance segmentation and utilize foundation models like [CLIP](https://inference.roboflow.com/foundation/clip), [Segment Anything](https://inference.roboflow.com/foundation/sam), and [YOLO-World](https://inference.roboflow.com/foundation/yolo_world) through a Python-native package, a self-hosted inference server, or a fully [managed API](https://docs.roboflow.com/).\n\nExplore our [enterprise options](https://roboflow.com/sales) for advanced features like server deployment, active learning, and commercial licenses for YOLOv5 and YOLOv8.\n\n## \ud83d\udcbb install\n\nInference package requires [**Python>=3.8,<=3.11**](https://www.python.org/). Click [here](https://inference.roboflow.com/quickstart/docker/) to learn more about running Inference inside Docker.\n\n```bash\npip install inference\n```\n\n<details>\n<summary>\ud83d\udc49 additional considerations</summary>\n\n\n- hardware\n\n Enhance model performance in GPU-accelerated environments by installing CUDA-compatible dependencies.\n \n ```bash\n pip install inference-gpu\n ```\n\n- models\n\n The `inference` and `inference-gpu` packages install only the minimal shared dependencies. Install model-specific dependencies to ensure code compatibility and license compliance. Learn more about the [models](https://inference.roboflow.com/#extras) supported by Inference.\n\n ```bash\n pip install inference[yolo-world]\n ```\n\n</details>\n\n## \ud83d\udd25 quickstart\n\nUse Inference SDK to run models locally with just a few lines of code. The image input can be a URL, a numpy array (BGR), or a PIL image.\n\n```python\nfrom inference import get_model\n\nmodel = get_model(model_id=\"yolov8n-640\")\n\nresults = model.infer(\"https://media.roboflow.com/inference/people-walking.jpg\")\n```\n\n<details>\n<summary>\ud83d\udc49 roboflow models</summary>\n\n<br>\n\nSet up your `ROBOFLOW_API_KEY` to access thousands of fine-tuned models shared by the [Roboflow Universe](https://universe.roboflow.com/) community and your custom model. Navigate to \ud83d\udd11 keys section to learn more.\n\n```python\nfrom inference import get_model\n\nmodel = get_model(model_id=\"soccer-players-5fuqs/1\")\n\nresults = model.infer(\n image=\"https://media.roboflow.com/inference/soccer.jpg\",\n confidence=0.5,\n iou_threshold=0.5\n)\n```\n\n</details>\n\n<details>\n<summary>\ud83d\udc49 foundational models</summary>\n\n\n- [CLIP Embeddings](https://inference.roboflow.com/foundation/clip) - generate text and image embeddings that you can use for zero-shot classification or assessing image similarity.\n\n ```python\n from inference.models import Clip\n\n model = Clip()\n\n embeddings_text = clip.embed_text(\"a football match\")\n embeddings_image = model.embed_image(\"https://media.roboflow.com/inference/soccer.jpg\")\n ```\n\n- [Segment Anything](https://inference.roboflow.com/foundation/sam) - segment all objects visible in the image or only those associated with selected points or boxes.\n\n ```python\n from inference.models import SegmentAnything\n\n model = SegmentAnything()\n\n result = model.segment_image(\"https://media.roboflow.com/inference/soccer.jpg\")\n ```\n\n- [YOLO-World](https://inference.roboflow.com/foundation/yolo_world) - an almost real-time zero-shot detector that enables the detection of any objects without any training.\n\n ```python\n from inference.models import YOLOWorld\n\n model = YOLOWorld(model_id=\"yolo_world/l\")\n \n result = model.infer(\n image=\"https://media.roboflow.com/inference/dog.jpeg\",\n text=[\"person\", \"backpack\", \"dog\", \"eye\", \"nose\", \"ear\", \"tongue\"],\n confidence=0.03\n )\n ```\n\n</details>\n\n## \ud83d\udcdf inference server\n\n- deploy server\n\n \n The inference server is distributed via Docker. Behind the scenes, inference will download and run the image that is appropriate for your hardware. [Here](https://inference.roboflow.com/quickstart/docker/#advanced-build-a-docker-container-from-scratch), you can learn more about the supported images.\n\n ```bash\n inference server start\n ```\n\n- run client\n \n Consume inference server predictions using the HTTP client available in the Inference SDK.\n\n ```python\n from inference_sdk import InferenceHTTPClient\n \n client = InferenceHTTPClient(\n api_url=\"http://localhost:9001\",\n api_key=<ROBOFLOW_API_KEY>\n )\n with client.use_model(model_id=\"soccer-players-5fuqs/1\"):\n predictions = client.infer(\"https://media.roboflow.com/inference/soccer.jpg\")\n ```\n \n If you're using the hosted API, change the local API URL to `https://detect.roboflow.com`. Accessing the hosted inference server and/or using any of the fine-tuned models require a `ROBOFLOW_API_KEY`. For further information, visit the \ud83d\udd11 keys section.\n\n## \ud83c\udfa5 inference pipeline\n\nThe inference pipeline is an efficient method for processing static video files and streams. Select a model, define the video source, and set a callback action. You can choose from predefined callbacks that allow you to [display results](https://inference.roboflow.com/docs/reference/inference/core/interfaces/stream/sinks/#inference.core.interfaces.stream.sinks.render_boxes) on the screen or [save them to a file](https://inference.roboflow.com/docs/reference/inference/core/interfaces/stream/sinks/#inference.core.interfaces.stream.sinks.VideoFileSink).\n\n```python\nfrom inference import InferencePipeline\nfrom inference.core.interfaces.stream.sinks import render_boxes\n\npipeline = InferencePipeline.init(\n model_id=\"yolov8x-1280\",\n video_reference=\"https://media.roboflow.com/inference/people-walking.mp4\",\n on_prediction=render_boxes\n)\n\npipeline.start()\npipeline.join()\n```\n\n## \ud83d\udd11 keys\n\nInference enables the deployment of a wide range of pre-trained and foundational models without an API key. To access thousands of fine-tuned models shared by the [Roboflow Universe](https://universe.roboflow.com/) community, [configure your](https://app.roboflow.com/settings/api) API key.\n\n```bash\nexport ROBOFLOW_API_KEY=<YOUR_API_KEY>\n```\n\n## \ud83d\udcda documentation\n\nVisit our [documentation](https://inference.roboflow.com) to explore comprehensive guides, detailed API references, and a wide array of tutorials designed to help you harness the full potential of the Inference package.\n\n## \u26a1\ufe0f Model-specific extras\n\nExplore the list of [`inference` extras](https://inference.roboflow.com/#extras) to install model-specific dependencies. \n\n## \u00a9 license\n\nSee the \"Self Hosting and Edge Deployment\" section of the [Roboflow Licensing](https://roboflow.com/licensing) documentation for information on how Roboflow Inference is licensed.\n\n## \ud83c\udfc6 contribution\n\nWe would love your input to improve Roboflow Inference! Please see our [contributing guide](https://github.com/roboflow/inference/blob/master/CONTRIBUTING.md) to get started. Thank you to all of our contributors! \ud83d\ude4f\n\n\n<br>\n\n<div align=\"center\">\n <div align=\"center\">\n <a href=\"https://youtube.com/roboflow\">\n <img\n src=\"https://media.roboflow.com/notebooks/template/icons/purple/youtube.png?ik-sdk-version=javascript-1.4.3&updatedAt=1672949634652\"\n width=\"3%\"\n />\n </a>\n <img src=\"https://raw.githubusercontent.com/ultralytics/assets/main/social/logo-transparent.png\" width=\"3%\"/>\n <a href=\"https://roboflow.com\">\n <img\n src=\"https://media.roboflow.com/notebooks/template/icons/purple/roboflow-app.png?ik-sdk-version=javascript-1.4.3&updatedAt=1672949746649\"\n width=\"3%\"\n />\n </a>\n <img src=\"https://raw.githubusercontent.com/ultralytics/assets/main/social/logo-transparent.png\" width=\"3%\"/>\n <a href=\"https://www.linkedin.com/company/roboflow-ai/\">\n <img\n src=\"https://media.roboflow.com/notebooks/template/icons/purple/linkedin.png?ik-sdk-version=javascript-1.4.3&updatedAt=1672949633691\"\n width=\"3%\"\n />\n </a>\n <img src=\"https://raw.githubusercontent.com/ultralytics/assets/main/social/logo-transparent.png\" width=\"3%\"/>\n <a href=\"https://docs.roboflow.com\">\n <img\n src=\"https://media.roboflow.com/notebooks/template/icons/purple/knowledge.png?ik-sdk-version=javascript-1.4.3&updatedAt=1672949634511\"\n width=\"3%\"\n />\n </a>\n <img src=\"https://raw.githubusercontent.com/ultralytics/assets/main/social/logo-transparent.png\" width=\"3%\"/>\n <a href=\"https://disuss.roboflow.com\">\n <img\n src=\"https://media.roboflow.com/notebooks/template/icons/purple/forum.png?ik-sdk-version=javascript-1.4.3&updatedAt=1672949633584\"\n width=\"3%\"\n />\n <img src=\"https://raw.githubusercontent.com/ultralytics/assets/main/social/logo-transparent.png\" width=\"3%\"/>\n <a href=\"https://blog.roboflow.com\">\n <img\n src=\"https://media.roboflow.com/notebooks/template/icons/purple/blog.png?ik-sdk-version=javascript-1.4.3&updatedAt=1672949633605\"\n width=\"3%\"\n />\n </a>\n </a>\n </div>\n</div>\n",
"bugtrack_url": null,
"license": null,
"summary": "With no prior knowledge of machine learning or device-specific deployment, you can deploy a computer vision model to a range of devices and environments using Roboflow Inference.",
"version": "0.31.1",
"project_urls": {
"Homepage": "https://github.com/roboflow/inference"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b3ea394c69bc84e66308d16031b39470c94b3267da96343a7feb2ee8f5f55239",
"md5": "538b1e4050db46f0d4417a84620b55fe",
"sha256": "d1ce86ff72036cafcb7f21f5381763a59c58b05ee654318ee601cee72c020733"
},
"downloads": -1,
"filename": "inference_gpu-0.31.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "538b1e4050db46f0d4417a84620b55fe",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.13,>=3.8",
"size": 888704,
"upload_time": "2024-12-13T18:57:25",
"upload_time_iso_8601": "2024-12-13T18:57:25.594066Z",
"url": "https://files.pythonhosted.org/packages/b3/ea/394c69bc84e66308d16031b39470c94b3267da96343a7feb2ee8f5f55239/inference_gpu-0.31.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-13 18:57:25",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "roboflow",
"github_project": "inference",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "inference-gpu"
}
