<!--
SPDX-FileCopyrightText: Copyright (c) 2024-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
# Dynamo Python Bindings
Python bindings for the Dynamo runtime system, enabling distributed computing capabilities for machine learning workloads.
## 🚀 Quick Start
1. Install `uv`: https://docs.astral.sh/uv/#getting-started
```
curl -LsSf https://astral.sh/uv/install.sh | sh
```
2. Install `protoc` protobuf compiler: https://grpc.io/docs/protoc-installation/.
For example on an Ubuntu/Debian system:
```
apt install protobuf-compiler
```
3. Setup a virtualenv
```
uv venv
source .venv/bin/activate
uv pip install maturin
```
4. Build and install dynamo wheel
```
maturin develop --uv
```
## Run Examples
### Prerequisite
See [README.md](../../../docs/runtime/README.md#prerequisites).
### Hello World Example
1. Start 3 separate shells, and activate the virtual environment in each
```
source .venv/bin/activate
```
2. In one shell (shell 1), run example server the instance-1
```
python3 ./examples/hello_world/server.py
```
3. (Optional) In another shell (shell 2), run example the server instance-2
```
python3 ./examples/hello_world/server.py
```
4. In the last shell (shell 3), run the example client:
```
python3 ./examples/hello_world/client.py
```
If you run the example client in rapid succession, and you started more than
one server instance above, you should see the requests from the client being
distributed across the server instances in each server's output. If only one
server instance is started, you should see the requests go to that server
each time.
## Performance
The performance impacts of synchronizing the Python and Rust async runtimes
is a critical consideration when optimizing the performance of a highly
concurrent and parallel distributed system.
The Python GIL is a global critical section and is ultimately the death of
parallelism. To compound that, when Rust async futures become ready,
accessing the GIL on those async event loop needs to be considered carefully.
Under high load, accessing the GIL or performing CPU intensive tasks on
on the event loop threads can starve out other async tasks for CPU resources.
However, performing a `tokio::task::spawn_blocking` is not without overheads
as well.
If bouncing many small message back-and-forth between the Python and Rust
event loops where Rust requires GIL access, this is pattern where moving the
code from Python to Rust will give you significant gains.
Raw data
{
"_id": null,
"home_page": "https://github.com/ai-dynamo/dynamo",
"name": "ai-dynamo-runtime",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "llm, genai, inference, nvidia, distributed, dynamo",
"author": "NVIDIA",
"author_email": "\"NVIDIA Inc.\" <sw-dl-dynamo@nvidia.com>",
"download_url": null,
"platform": null,
"description": "<!--\nSPDX-FileCopyrightText: Copyright (c) 2024-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.\nSPDX-License-Identifier: Apache-2.0\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttps://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n-->\n\n# Dynamo Python Bindings\n\nPython bindings for the Dynamo runtime system, enabling distributed computing capabilities for machine learning workloads.\n\n## \ud83d\ude80 Quick Start\n\n1. Install `uv`: https://docs.astral.sh/uv/#getting-started\n```\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n```\n\n2. Install `protoc` protobuf compiler: https://grpc.io/docs/protoc-installation/.\n\nFor example on an Ubuntu/Debian system:\n```\napt install protobuf-compiler\n```\n\n3. Setup a virtualenv\n\n```\nuv venv\nsource .venv/bin/activate\nuv pip install maturin\n```\n\n4. Build and install dynamo wheel\n```\nmaturin develop --uv\n```\n\n## Run Examples\n\n### Prerequisite\n\nSee [README.md](../../../docs/runtime/README.md#prerequisites).\n\n### Hello World Example\n\n1. Start 3 separate shells, and activate the virtual environment in each\n```\nsource .venv/bin/activate\n```\n\n2. In one shell (shell 1), run example server the instance-1\n```\npython3 ./examples/hello_world/server.py\n```\n\n3. (Optional) In another shell (shell 2), run example the server instance-2\n```\npython3 ./examples/hello_world/server.py\n```\n\n4. In the last shell (shell 3), run the example client:\n```\npython3 ./examples/hello_world/client.py\n```\n\nIf you run the example client in rapid succession, and you started more than\none server instance above, you should see the requests from the client being\ndistributed across the server instances in each server's output. If only one\nserver instance is started, you should see the requests go to that server\neach time.\n\n## Performance\n\nThe performance impacts of synchronizing the Python and Rust async runtimes\nis a critical consideration when optimizing the performance of a highly\nconcurrent and parallel distributed system.\n\nThe Python GIL is a global critical section and is ultimately the death of\nparallelism. To compound that, when Rust async futures become ready,\naccessing the GIL on those async event loop needs to be considered carefully.\nUnder high load, accessing the GIL or performing CPU intensive tasks on\non the event loop threads can starve out other async tasks for CPU resources.\nHowever, performing a `tokio::task::spawn_blocking` is not without overheads\nas well.\n\nIf bouncing many small message back-and-forth between the Python and Rust\nevent loops where Rust requires GIL access, this is pattern where moving the\ncode from Python to Rust will give you significant gains.\n\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Dynamo Inference Framework Runtime",
"version": "0.3.2",
"project_urls": {
"Homepage": "https://github.com/ai-dynamo/dynamo",
"Source Code": "https://github.com/ai-dynamo/dynamo.git"
},
"split_keywords": [
"llm",
" genai",
" inference",
" nvidia",
" distributed",
" dynamo"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d4d894043a5313b18b097268cc7d47ead587e56de8b03e72f101d3a62c88a3ed",
"md5": "32f2ccd22ab6de8ffa8accb2fcee2b42",
"sha256": "f473192a5d67187b985f1708f0d0ef9222faf4483139fc7d58c63067168265d8"
},
"downloads": -1,
"filename": "ai_dynamo_runtime-0.3.2-cp310-cp310-manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "32f2ccd22ab6de8ffa8accb2fcee2b42",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": ">=3.10",
"size": 12403790,
"upload_time": "2025-07-18T14:21:47",
"upload_time_iso_8601": "2025-07-18T14:21:47.975397Z",
"url": "https://files.pythonhosted.org/packages/d4/d8/94043a5313b18b097268cc7d47ead587e56de8b03e72f101d3a62c88a3ed/ai_dynamo_runtime-0.3.2-cp310-cp310-manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5e5ac16dbd45f70567cea4af212bb9c67b59641293813b7991e874b7c13f256a",
"md5": "63362cc378717611c7968135e05dbd54",
"sha256": "1c1411607c9d9c82e1baf6ce55d1f1c330b11d40f66fbea798509d0ea60d97be"
},
"downloads": -1,
"filename": "ai_dynamo_runtime-0.3.2-cp311-cp311-manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "63362cc378717611c7968135e05dbd54",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.10",
"size": 12402533,
"upload_time": "2025-07-18T14:21:28",
"upload_time_iso_8601": "2025-07-18T14:21:28.418192Z",
"url": "https://files.pythonhosted.org/packages/5e/5a/c16dbd45f70567cea4af212bb9c67b59641293813b7991e874b7c13f256a/ai_dynamo_runtime-0.3.2-cp311-cp311-manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4252389a243c52141966df40664d7af57b64f87f591b2cb1a9c822bf4c6a9a7e",
"md5": "b9c14579a404d068c497006cea9ecb9f",
"sha256": "574941cbe7a2ab33a6da5559c9e33f7199253bec52db93a96edfb06f18355207"
},
"downloads": -1,
"filename": "ai_dynamo_runtime-0.3.2-cp312-cp312-manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "b9c14579a404d068c497006cea9ecb9f",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": ">=3.10",
"size": 13644591,
"upload_time": "2025-07-18T14:22:07",
"upload_time_iso_8601": "2025-07-18T14:22:07.137702Z",
"url": "https://files.pythonhosted.org/packages/42/52/389a243c52141966df40664d7af57b64f87f591b2cb1a9c822bf4c6a9a7e/ai_dynamo_runtime-0.3.2-cp312-cp312-manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-18 14:21:47",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ai-dynamo",
"github_project": "dynamo",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "ai-dynamo-runtime"
}
JSON
