torchgpipe
==========
A GPipe_ implementation in PyTorch_.
.. _GPipe: https://arxiv.org/abs/1811.06965
.. _PyTorch: https://pytorch.org/
.. sourcecode:: python
from torchgpipe import GPipe
model = nn.Sequential(a, b, c, d)
model = GPipe(model, balance=[1, 1, 1, 1], chunks=8)
for input in data_loader:
output = model(input)
What is GPipe?
~~~~~~~~~~~~~~
GPipe is a scalable pipeline parallelism library published by Google Brain,
which allows efficient training of large, memory-consuming models. According to
the paper, GPipe can train a 25x larger model by using 8x devices (TPU), and
train a model 3.5x faster by using 4x devices.
`GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism
<https://arxiv.org/abs/1811.06965>`_
Google trained AmoebaNet-B with 557M parameters over GPipe. This model has
achieved 84.3% top-1 and 97.0% top-5 accuracy on ImageNet classification
benchmark (the state-of-the-art performance as of May 2019).
Links
~~~~~
- Source Code: https://github.com/kakaobrain/torchgpipe
- Documentation: https://torchgpipe.readthedocs.io/
- Original Paper: https://arxiv.org/abs/1811.06965
Raw data
{
"_id": null,
"home_page": "https://github.com/kakaobrain/torchgpipe",
"name": "torchgpipe",
"maintainer": "Heungsub Lee, Myungryong Jeong, Chiheon Kim",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "pytorch gpipe",
"author": "Kakao Brain",
"author_email": "",
"download_url": "",
"platform": "",
"description": "\ntorchgpipe\n==========\n\nA GPipe_ implementation in PyTorch_.\n\n.. _GPipe: https://arxiv.org/abs/1811.06965\n.. _PyTorch: https://pytorch.org/\n\n.. sourcecode:: python\n\n from torchgpipe import GPipe\n\n model = nn.Sequential(a, b, c, d)\n model = GPipe(model, balance=[1, 1, 1, 1], chunks=8)\n\n for input in data_loader:\n output = model(input)\n\nWhat is GPipe?\n~~~~~~~~~~~~~~\n\nGPipe is a scalable pipeline parallelism library published by Google Brain,\nwhich allows efficient training of large, memory-consuming models. According to\nthe paper, GPipe can train a 25x larger model by using 8x devices (TPU), and\ntrain a model 3.5x faster by using 4x devices.\n\n`GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism\n<https://arxiv.org/abs/1811.06965>`_\n\nGoogle trained AmoebaNet-B with 557M parameters over GPipe. This model has\nachieved 84.3% top-1 and 97.0% top-5 accuracy on ImageNet classification\nbenchmark (the state-of-the-art performance as of May 2019).\n\nLinks\n~~~~~\n\n- Source Code: https://github.com/kakaobrain/torchgpipe\n- Documentation: https://torchgpipe.readthedocs.io/\n- Original Paper: https://arxiv.org/abs/1811.06965\n\n\n\n",
"bugtrack_url": null,
"license": "BSD-3-Clause",
"summary": "GPipe for PyTorch",
"version": "0.0.7",
"split_keywords": [
"pytorch",
"gpipe"
],
"urls": [
{
"comment_text": "",
"digests": {
"md5": "75a817456ed1a0fe59b76a5abc6bfc45",
"sha256": "052c704f8c03b7695110e81853e4166c1fc1db3ad1e4cc0fa4eceb8a63e32627"
},
"downloads": -1,
"filename": "torchgpipe-0.0.7-py3-none-any.whl",
"has_sig": false,
"md5_digest": "75a817456ed1a0fe59b76a5abc6bfc45",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 39061,
"upload_time": "2020-09-18T13:59:56",
"upload_time_iso_8601": "2020-09-18T13:59:56.944453Z",
"url": "https://files.pythonhosted.org/packages/f4/84/97f3c3b27b666de92477dda6425dd7cb56c7bbaeb115c3a8a1ec7dbe8e05/torchgpipe-0.0.7-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2020-09-18 13:59:56",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "kakaobrain",
"github_project": "torchgpipe",
"travis_ci": true,
"coveralls": false,
"github_actions": false,
"lcname": "torchgpipe"
}
JSON
