# Lightning ⚡ Bagua
**Deep Learning Training Acceleration with [Bagua](https://tutorials.baguasys.com/) and [Lightning AI](https://lightning.ai)**
[](https://lightning.ai/)
[](https://badge.fury.io/py/lightning-bagua)
[](https://pypi.org/project/lightning-bagua/)
[](https://pepy.tech/project/lightning-bagua)
[](https://lightning-ai.github.io/lightning-Bagua/)
[](https://github.com/Lightning-AI/lightning-bagua/actions/workflows/ci-checks.yml)
[](https://dev.azure.com/Lightning-AI/compatibility/_build/latest?definitionId=47&branchName=main)
[](https://results.pre-commit.ci/latest/github/Lightning-AI/lightning-Bagua/main)
[Bagua](https://github.com/BaguaSys/bagua) is a deep learning training acceleration framework which supports multiple advanced distributed
training algorithms including:
- [Gradient AllReduce](https://tutorials.baguasys.com/algorithms/gradient-allreduce) for centralized synchronous communication, where gradients are averaged among all workers.
- [Decentralized SGD](https://tutorials.baguasys.com/algorithms/decentralized) for decentralized synchronous communication, where each worker exchanges data with one or a few specific workers.
- [ByteGrad](https://tutorials.baguasys.com/algorithms/bytegrad) and [QAdam](https://tutorials.baguasys.com/algorithms/q-adam) for low precision communication, where data is compressed into low precision before communication.
- [Asynchronous Model Average](https://tutorials.baguasys.com/algorithms/async-model-average) for asynchronous communication, where workers are not required to be synchronized in the same iteration in a lock-step style.
By default, Bagua uses *Gradient AllReduce* algorithm, which is also the algorithm implemented in DDP, but Bagua can usually produce a higher training throughput due to its backend written in Rust.
## Installation
```bash
pip install -U lightning-bagua
```
## Usage
Simply set the strategy argument in the Trainer:
```python
from lightning import Trainer
# train on 4 GPUs (using Bagua mode)
trainer = Trainer(strategy="bagua", accelerator="gpu", devices=4)
```
See [Bagua Tutorials](https://tutorials.baguasys.com/) for more details on installation and advanced features.
Raw data
{
"_id": null,
"home_page": "https://github.com/Lightning-AI/lightning-Bagua",
"name": "lightning-bagua",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "deep learning,pytorch,AI",
"author": "Lightning-AI et al.",
"author_email": "name@lightning.ai",
"download_url": "https://files.pythonhosted.org/packages/61/c8/232a596b7809ef6f3957c3e7027fa2af848d0b12507374db06bdf1b2d4ef/lightning-bagua-0.1.0.tar.gz",
"platform": null,
"description": "# Lightning \u26a1 Bagua\n\n**Deep Learning Training Acceleration with [Bagua](https://tutorials.baguasys.com/) and [Lightning AI](https://lightning.ai)**\n\n[](https://lightning.ai/)\n[](https://badge.fury.io/py/lightning-bagua)\n[](https://pypi.org/project/lightning-bagua/)\n[](https://pepy.tech/project/lightning-bagua)\n[](https://lightning-ai.github.io/lightning-Bagua/)\n\n[](https://github.com/Lightning-AI/lightning-bagua/actions/workflows/ci-checks.yml)\n[](https://dev.azure.com/Lightning-AI/compatibility/_build/latest?definitionId=47&branchName=main)\n[](https://results.pre-commit.ci/latest/github/Lightning-AI/lightning-Bagua/main)\n\n[Bagua](https://github.com/BaguaSys/bagua) is a deep learning training acceleration framework which supports multiple advanced distributed\ntraining algorithms including:\n\n- [Gradient AllReduce](https://tutorials.baguasys.com/algorithms/gradient-allreduce) for centralized synchronous communication, where gradients are averaged among all workers.\n- [Decentralized SGD](https://tutorials.baguasys.com/algorithms/decentralized) for decentralized synchronous communication, where each worker exchanges data with one or a few specific workers.\n- [ByteGrad](https://tutorials.baguasys.com/algorithms/bytegrad) and [QAdam](https://tutorials.baguasys.com/algorithms/q-adam) for low precision communication, where data is compressed into low precision before communication.\n- [Asynchronous Model Average](https://tutorials.baguasys.com/algorithms/async-model-average) for asynchronous communication, where workers are not required to be synchronized in the same iteration in a lock-step style.\n\nBy default, Bagua uses *Gradient AllReduce* algorithm, which is also the algorithm implemented in DDP, but Bagua can usually produce a higher training throughput due to its backend written in Rust.\n\n## Installation\n\n```bash\npip install -U lightning-bagua\n```\n\n## Usage\n\nSimply set the strategy argument in the Trainer:\n\n```python\nfrom lightning import Trainer\n\n# train on 4 GPUs (using Bagua mode)\ntrainer = Trainer(strategy=\"bagua\", accelerator=\"gpu\", devices=4)\n```\n\nSee [Bagua Tutorials](https://tutorials.baguasys.com/) for more details on installation and advanced features.\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Deep Learning Training Acceleration with Bagua and Lightning AI",
"version": "0.1.0",
"split_keywords": [
"deep learning",
"pytorch",
"ai"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6f4a33799300a1fa158e4545a6639325b91a26ddef5d336fa514089915905fd9",
"md5": "627d8f8496791b2f517f3adf8c361920",
"sha256": "8cb1e0156be950c1859061c9586d606f9543c96c5e0bb8dd128c7075a4233295"
},
"downloads": -1,
"filename": "lightning_bagua-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "627d8f8496791b2f517f3adf8c361920",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 13018,
"upload_time": "2023-03-22T08:07:31",
"upload_time_iso_8601": "2023-03-22T08:07:31.311197Z",
"url": "https://files.pythonhosted.org/packages/6f/4a/33799300a1fa158e4545a6639325b91a26ddef5d336fa514089915905fd9/lightning_bagua-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "61c8232a596b7809ef6f3957c3e7027fa2af848d0b12507374db06bdf1b2d4ef",
"md5": "0408d97afcf28643e2bc0d41124d9d92",
"sha256": "3ca4f1659816b6d62cc604081fa0b4210a0b80de45cc604c9bdcb2c906cb4d4c"
},
"downloads": -1,
"filename": "lightning-bagua-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "0408d97afcf28643e2bc0d41124d9d92",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 14371,
"upload_time": "2023-03-22T08:07:32",
"upload_time_iso_8601": "2023-03-22T08:07:32.979457Z",
"url": "https://files.pythonhosted.org/packages/61/c8/232a596b7809ef6f3957c3e7027fa2af848d0b12507374db06bdf1b2d4ef/lightning-bagua-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-03-22 08:07:32",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "Lightning-AI",
"github_project": "lightning-Bagua",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "lightning-bagua"
}
JSON
