<p align="center">
<picture>
<img alt="Trainy Konduktor Logo" src="https://raw.githubusercontent.com/Trainy-ai/konduktor/main/docs/source/images/konduktor-logo-white-no-background.png" width="353" height="64" style="max-width: 100%;">
</picture>
<br/>
<br/>
</p>
Konduktor is a source available Run:AI alternative built on [Kubernetes](https://kubernetes.io). Konduktor uses existing open source tools to build a platform that makes it easy for ML Researchers to submit batch jobs and for administrative/infra teams to easily manage GPU clusters.
## How it works
Konduktor uses a combination of open source projects. Where tools exist with MIT, Apache, or another compatible open license, we want to use and even contribute to that tool. Where we see gaps in tooling, we build it and open source ourselves.
### Architecture
Konduktor can be self-hosted and run on any certified Kubernetes distribution or managed by us. Contact us at founders@trainy.ai if you are just interested in the managed version. We're focused on tooling for clusters with NVIDIA cards for now but in the future we may expand to our scope to support other accelerators.
<p align="center">
<img alt="architecture" src="https://raw.githubusercontent.com/Trainy-ai/konduktor/main/docs/source/images/architecture.png" width=80%>
</p>
For ML researchers
- [Skypilot](https://skypilot.readthedocs.io/en/latest/docs/index.html) - user friendly batch job framework, where users only need to specify the resource requirements of their job and a script to launch that makes simple to scale work across multiple nodes. Works with most ML application frameworks. [Examples](https://github.com/skypilot-org/skypilot/tree/master/examples)
```
num_nodes: 100
resources:
accelerators: H100:8
cloud: kubernetes
labels:
kueue.x-k8s.io/queue-name: user-queue
kueue.x-k8s.io/priority-class: low-priority
setup: |
pip install torch torchvision
run: |
torchrun \
--nproc_per_node 8 \
--rdzv_id=1 --rdzv_endpoint=$master_addr:1234 \
--rdzv_backend=c10d --nnodes $num_nodes \
torch_ddp_benchmark.py --distributed-backend nccl
```
For cluster administrators
- [DCGM Exporter](https://github.com/NVIDIA/dcgm-exporter), [GPU operator](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/), [Network Operator](https://github.com/Mellanox/network-operator) - For installing NVIDIA driver, container runtime, and exporting node health metrics.
- [Kueue](https://kueue.sigs.k8s.io/docs/) - centralized creation of job queues, gang-scheduling, and resource quotas and sharing across projects.
- [Prometheus](https://prometheus.io/) - For publishing metrics about node health and workload queues.
- [OpenTelemetry](https://opentelemetry.io/) - For pushing logs from each node
- [Grafana, Loki](https://grafana.com/) - Visualizations for metrics/logging solution.
## Community & Support
- [Discord](https://discord.com/invite/HQUBJSVgAP)
- founders@trainy.ai
## Contributor Guide
Format your code with
```
poetry install --with dev
bash format.sh
```
Raw data
{
"_id": null,
"home_page": null,
"name": "konduktor-nightly",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.9",
"maintainer_email": null,
"keywords": null,
"author": "Andrew Aikawa",
"author_email": "asai@berkeley.edu",
"download_url": "https://files.pythonhosted.org/packages/3e/14/c7cad37750de095be5a4a29b7f0ff1432517577806857753ed70110db206/konduktor_nightly-0.1.0.dev20241221104318.tar.gz",
"platform": null,
"description": "\n<p align=\"center\">\n <picture>\n <img alt=\"Trainy Konduktor Logo\" src=\"https://raw.githubusercontent.com/Trainy-ai/konduktor/main/docs/source/images/konduktor-logo-white-no-background.png\" width=\"353\" height=\"64\" style=\"max-width: 100%;\">\n </picture>\n <br/>\n <br/>\n</p>\n\nKonduktor is a source available Run:AI alternative built on [Kubernetes](https://kubernetes.io). Konduktor uses existing open source tools to build a platform that makes it easy for ML Researchers to submit batch jobs and for administrative/infra teams to easily manage GPU clusters.\n\n## How it works\n\nKonduktor uses a combination of open source projects. Where tools exist with MIT, Apache, or another compatible open license, we want to use and even contribute to that tool. Where we see gaps in tooling, we build it and open source ourselves.\n\n### Architecture\n\nKonduktor can be self-hosted and run on any certified Kubernetes distribution or managed by us. Contact us at founders@trainy.ai if you are just interested in the managed version. We're focused on tooling for clusters with NVIDIA cards for now but in the future we may expand to our scope to support other accelerators.\n\n<p align=\"center\">\n <img alt=\"architecture\" src=\"https://raw.githubusercontent.com/Trainy-ai/konduktor/main/docs/source/images/architecture.png\" width=80%>\n</p>\n\nFor ML researchers\n- [Skypilot](https://skypilot.readthedocs.io/en/latest/docs/index.html) - user friendly batch job framework, where users only need to specify the resource requirements of their job and a script to launch that makes simple to scale work across multiple nodes. Works with most ML application frameworks. [Examples](https://github.com/skypilot-org/skypilot/tree/master/examples)\n\n```\nnum_nodes: 100\n\nresources:\n accelerators: H100:8\n cloud: kubernetes\n labels:\n kueue.x-k8s.io/queue-name: user-queue\n kueue.x-k8s.io/priority-class: low-priority\n\nsetup: |\n pip install torch torchvision\n\nrun: |\n torchrun \\\n --nproc_per_node 8 \\\n --rdzv_id=1 --rdzv_endpoint=$master_addr:1234 \\\n --rdzv_backend=c10d --nnodes $num_nodes \\\n torch_ddp_benchmark.py --distributed-backend nccl\n```\n\nFor cluster administrators\n- [DCGM Exporter](https://github.com/NVIDIA/dcgm-exporter), [GPU operator](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/), [Network Operator](https://github.com/Mellanox/network-operator) - For installing NVIDIA driver, container runtime, and exporting node health metrics.\n- [Kueue](https://kueue.sigs.k8s.io/docs/) - centralized creation of job queues, gang-scheduling, and resource quotas and sharing across projects.\n- [Prometheus](https://prometheus.io/) - For publishing metrics about node health and workload queues.\n- [OpenTelemetry](https://opentelemetry.io/) - For pushing logs from each node\n- [Grafana, Loki](https://grafana.com/) - Visualizations for metrics/logging solution.\n\n\n## Community & Support\n- [Discord](https://discord.com/invite/HQUBJSVgAP)\n- founders@trainy.ai\n\n## Contributor Guide\n\nFormat your code with\n```\npoetry install --with dev\nbash format.sh\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "GPU Cluster Health Management",
"version": "0.1.0.dev20241221104318",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0b71636ce22af73438857422bf3139d9a2a7796dcc9695af73b232abc5b7856d",
"md5": "ea3a48335679fd92b0755d62d0d46498",
"sha256": "cc00d24a7803d664d75b3ccb12a91363e6e1fa1ac507fa19622223b92c710475"
},
"downloads": -1,
"filename": "konduktor_nightly-0.1.0.dev20241221104318-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ea3a48335679fd92b0755d62d0d46498",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.9",
"size": 106961,
"upload_time": "2024-12-21T10:43:20",
"upload_time_iso_8601": "2024-12-21T10:43:20.556492Z",
"url": "https://files.pythonhosted.org/packages/0b/71/636ce22af73438857422bf3139d9a2a7796dcc9695af73b232abc5b7856d/konduktor_nightly-0.1.0.dev20241221104318-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3e14c7cad37750de095be5a4a29b7f0ff1432517577806857753ed70110db206",
"md5": "1bde05d45dde14a659698222c10cd21a",
"sha256": "e674b48c1a2176ceee4d2d470c95b98d80fe85ba8cd567834765fd0f7503d384"
},
"downloads": -1,
"filename": "konduktor_nightly-0.1.0.dev20241221104318.tar.gz",
"has_sig": false,
"md5_digest": "1bde05d45dde14a659698222c10cd21a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.9",
"size": 96159,
"upload_time": "2024-12-21T10:43:22",
"upload_time_iso_8601": "2024-12-21T10:43:22.854473Z",
"url": "https://files.pythonhosted.org/packages/3e/14/c7cad37750de095be5a4a29b7f0ff1432517577806857753ed70110db206/konduktor_nightly-0.1.0.dev20241221104318.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-21 10:43:22",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "konduktor-nightly"
}
JSON