======
MemCNN
======
.. image:: https://img.shields.io/circleci/build/github/silvandeleemput/memcnn/master.svg
:alt: CircleCI - Status master branch
:target: https://circleci.com/gh/silvandeleemput/memcnn/tree/master
.. image:: https://img.shields.io/docker/cloud/build/silvandeleemput/memcnn.svg
:alt: Docker - Status
:target: https://hub.docker.com/r/silvandeleemput/memcnn
.. image:: https://readthedocs.org/projects/memcnn/badge/?version=latest
:alt: Documentation - Status master branch
:target: https://memcnn.readthedocs.io/en/latest/?badge=latest
.. image:: https://img.shields.io/codacy/grade/95de32e0d7c54d038611da47e9f0948b/master.svg
:alt: Codacy - Branch grade
:target: https://app.codacy.com/project/silvandeleemput/memcnn/dashboardgit
.. image:: https://img.shields.io/codecov/c/gh/silvandeleemput/memcnn/master.svg
:alt: Codecov - Status master branch
:target: https://codecov.io/gh/silvandeleemput/memcnn
.. image:: https://img.shields.io/pypi/v/memcnn.svg
:alt: PyPI - Latest release
:target: https://pypi.python.org/pypi/memcnn
.. image:: https://img.shields.io/conda/vn/silvandeleemput/memcnn?label=anaconda
:alt: Conda - Latest release
:target: https://anaconda.org/silvandeleemput/memcnn
.. image:: https://img.shields.io/pypi/implementation/memcnn.svg
:alt: PyPI - Implementation
:target: https://pypi.python.org/pypi/memcnn
.. image:: https://img.shields.io/pypi/pyversions/memcnn.svg
:alt: PyPI - Python version
:target: https://pypi.python.org/pypi/memcnn
.. image:: https://img.shields.io/github/license/silvandeleemput/memcnn.svg
:alt: GitHub - Repository license
:target: https://github.com/silvandeleemput/memcnn/blob/master/LICENSE.txt
.. image:: http://joss.theoj.org/papers/10.21105/joss.01576/status.svg
:alt: JOSS - DOI
:target: https://doi.org/10.21105/joss.01576
A `PyTorch <http://pytorch.org/>`__ framework for developing memory-efficient invertible neural networks.
* Free software: `MIT license <https://github.com/silvandeleemput/memcnn/blob/master/LICENSE.txt>`__ (please cite our work if you use it)
* Documentation: https://memcnn.readthedocs.io.
* Installation: https://memcnn.readthedocs.io/en/latest/installation.html
Features
--------
* Enable memory savings during training by wrapping arbitrary invertible PyTorch functions with the `InvertibleModuleWrapper` class.
* Simple toggling of memory saving by setting the `keep_input` property of the `InvertibleModuleWrapper`.
* Turn arbitrary non-linear PyTorch functions into invertible versions using the `AdditiveCoupling` or the `AffineCoupling` classes.
* Training and evaluation code for reproducing RevNet experiments using MemCNN.
* CI tests for Python v3.7 and torch v1.0, v1.1, v1.4 and v1.7 with good code coverage.
Examples
--------
Creating an AdditiveCoupling with memory savings
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code:: python
import torch
import torch.nn as nn
import memcnn
# define a new torch Module with a sequence of operations: Relu o BatchNorm2d o Conv2d
class ExampleOperation(nn.Module):
def __init__(self, channels):
super(ExampleOperation, self).__init__()
self.seq = nn.Sequential(
nn.Conv2d(in_channels=channels, out_channels=channels,
kernel_size=(3, 3), padding=1),
nn.BatchNorm2d(num_features=channels),
nn.ReLU(inplace=True)
)
def forward(self, x):
return self.seq(x)
# generate some random input data (batch_size, num_channels, y_elements, x_elements)
X = torch.rand(2, 10, 8, 8)
# application of the operation(s) the normal way
model_normal = ExampleOperation(channels=10)
model_normal.eval()
Y = model_normal(X)
# turn the ExampleOperation invertible using an additive coupling
invertible_module = memcnn.AdditiveCoupling(
Fm=ExampleOperation(channels=10 // 2),
Gm=ExampleOperation(channels=10 // 2)
)
# test that it is actually a valid invertible module (has a valid inverse method)
assert memcnn.is_invertible_module(invertible_module, test_input_shape=X.shape)
# wrap our invertible_module using the InvertibleModuleWrapper and benefit from memory savings during training
invertible_module_wrapper = memcnn.InvertibleModuleWrapper(fn=invertible_module, keep_input=True, keep_input_inverse=True)
# by default the module is set to training, the following sets this to evaluation
# note that this is required to pass input tensors to the model with requires_grad=False (inference only)
invertible_module_wrapper.eval()
# test that the wrapped module is also a valid invertible module
assert memcnn.is_invertible_module(invertible_module_wrapper, test_input_shape=X.shape)
# compute the forward pass using the wrapper
Y2 = invertible_module_wrapper.forward(X)
# the input (X) can be approximated (X2) by applying the inverse method of the wrapper on Y2
X2 = invertible_module_wrapper.inverse(Y2)
# test that the input and approximation are similar
assert torch.allclose(X, X2, atol=1e-06)
Run PyTorch Experiments
-----------------------
After installing MemCNN run:
.. code:: bash
python -m memcnn.train [MODEL] [DATASET] [--fresh] [--no-cuda]
* Available values for ``DATASET`` are ``cifar10`` and ``cifar100``.
* Available values for ``MODEL`` are ``resnet32``, ``resnet110``, ``resnet164``, ``revnet38``, ``revnet110``, ``revnet164``
* Use the ``--fresh`` flag to remove earlier experiment results.
* Use the ``--no-cuda`` flag to train on the CPU rather than the GPU through CUDA.
Datasets are automatically downloaded if they are not available.
When using Python 3.* replace the ``python`` directive with the appropriate Python 3 directive. For example when using the MemCNN docker image use ``python3.6``.
When MemCNN was installed using `pip` or from sources you might need to setup a configuration file before running this command.
Read the corresponding section about how to do this here: https://memcnn.readthedocs.io/en/latest/installation.html
Raw data
{
"_id": null,
"home_page": "http://pypi.python.org/pypi/memcnn/",
"name": "memcnn",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "memcnn invertible PyTorch",
"author": "S.C. van de Leemput",
"author_email": "silvandeleemput@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/01/ca/469972769f97393650e1d10b6ea394fbb7475c73c22fd107640538a92bdc/memcnn-1.5.2.tar.gz",
"platform": null,
"description": "======\nMemCNN\n======\n\n.. image:: https://img.shields.io/circleci/build/github/silvandeleemput/memcnn/master.svg \n :alt: CircleCI - Status master branch\n :target: https://circleci.com/gh/silvandeleemput/memcnn/tree/master\n\n.. image:: https://img.shields.io/docker/cloud/build/silvandeleemput/memcnn.svg\n :alt: Docker - Status\n :target: https://hub.docker.com/r/silvandeleemput/memcnn\n\n.. image:: https://readthedocs.org/projects/memcnn/badge/?version=latest \n :alt: Documentation - Status master branch\n :target: https://memcnn.readthedocs.io/en/latest/?badge=latest\n\n.. image:: https://img.shields.io/codacy/grade/95de32e0d7c54d038611da47e9f0948b/master.svg\n :alt: Codacy - Branch grade\n :target: https://app.codacy.com/project/silvandeleemput/memcnn/dashboardgit\n\n.. image:: https://img.shields.io/codecov/c/gh/silvandeleemput/memcnn/master.svg \n :alt: Codecov - Status master branch\n :target: https://codecov.io/gh/silvandeleemput/memcnn\n\n.. image:: https://img.shields.io/pypi/v/memcnn.svg\n :alt: PyPI - Latest release\n :target: https://pypi.python.org/pypi/memcnn\n\n.. image:: https://img.shields.io/conda/vn/silvandeleemput/memcnn?label=anaconda\n :alt: Conda - Latest release\n :target: https://anaconda.org/silvandeleemput/memcnn\n\n.. image:: https://img.shields.io/pypi/implementation/memcnn.svg \n :alt: PyPI - Implementation\n :target: https://pypi.python.org/pypi/memcnn\n\n.. image:: https://img.shields.io/pypi/pyversions/memcnn.svg \n :alt: PyPI - Python version\n :target: https://pypi.python.org/pypi/memcnn\n\n.. image:: https://img.shields.io/github/license/silvandeleemput/memcnn.svg \n :alt: GitHub - Repository license\n :target: https://github.com/silvandeleemput/memcnn/blob/master/LICENSE.txt\n\n.. image:: http://joss.theoj.org/papers/10.21105/joss.01576/status.svg\n :alt: JOSS - DOI\n :target: https://doi.org/10.21105/joss.01576\n\nA `PyTorch <http://pytorch.org/>`__ framework for developing memory-efficient invertible neural networks.\n\n* Free software: `MIT license <https://github.com/silvandeleemput/memcnn/blob/master/LICENSE.txt>`__ (please cite our work if you use it)\n* Documentation: https://memcnn.readthedocs.io.\n* Installation: https://memcnn.readthedocs.io/en/latest/installation.html\n\nFeatures\n--------\n\n* Enable memory savings during training by wrapping arbitrary invertible PyTorch functions with the `InvertibleModuleWrapper` class.\n* Simple toggling of memory saving by setting the `keep_input` property of the `InvertibleModuleWrapper`.\n* Turn arbitrary non-linear PyTorch functions into invertible versions using the `AdditiveCoupling` or the `AffineCoupling` classes.\n* Training and evaluation code for reproducing RevNet experiments using MemCNN.\n* CI tests for Python v3.7 and torch v1.0, v1.1, v1.4 and v1.7 with good code coverage.\n\nExamples\n--------\n\nCreating an AdditiveCoupling with memory savings\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n.. code:: python\n\n import torch\n import torch.nn as nn\n import memcnn\n\n\n # define a new torch Module with a sequence of operations: Relu o BatchNorm2d o Conv2d\n class ExampleOperation(nn.Module):\n def __init__(self, channels):\n super(ExampleOperation, self).__init__()\n self.seq = nn.Sequential(\n nn.Conv2d(in_channels=channels, out_channels=channels,\n kernel_size=(3, 3), padding=1),\n nn.BatchNorm2d(num_features=channels),\n nn.ReLU(inplace=True)\n )\n\n def forward(self, x):\n return self.seq(x)\n\n\n # generate some random input data (batch_size, num_channels, y_elements, x_elements)\n X = torch.rand(2, 10, 8, 8)\n\n # application of the operation(s) the normal way\n model_normal = ExampleOperation(channels=10)\n model_normal.eval()\n\n Y = model_normal(X)\n\n # turn the ExampleOperation invertible using an additive coupling\n invertible_module = memcnn.AdditiveCoupling(\n Fm=ExampleOperation(channels=10 // 2),\n Gm=ExampleOperation(channels=10 // 2)\n )\n\n # test that it is actually a valid invertible module (has a valid inverse method)\n assert memcnn.is_invertible_module(invertible_module, test_input_shape=X.shape)\n\n # wrap our invertible_module using the InvertibleModuleWrapper and benefit from memory savings during training\n invertible_module_wrapper = memcnn.InvertibleModuleWrapper(fn=invertible_module, keep_input=True, keep_input_inverse=True)\n\n # by default the module is set to training, the following sets this to evaluation\n # note that this is required to pass input tensors to the model with requires_grad=False (inference only)\n invertible_module_wrapper.eval()\n\n # test that the wrapped module is also a valid invertible module\n assert memcnn.is_invertible_module(invertible_module_wrapper, test_input_shape=X.shape)\n\n # compute the forward pass using the wrapper\n Y2 = invertible_module_wrapper.forward(X)\n\n # the input (X) can be approximated (X2) by applying the inverse method of the wrapper on Y2\n X2 = invertible_module_wrapper.inverse(Y2)\n\n # test that the input and approximation are similar\n assert torch.allclose(X, X2, atol=1e-06)\n\nRun PyTorch Experiments\n-----------------------\n\nAfter installing MemCNN run:\n\n.. code:: bash\n\n python -m memcnn.train [MODEL] [DATASET] [--fresh] [--no-cuda]\n\n* Available values for ``DATASET`` are ``cifar10`` and ``cifar100``.\n* Available values for ``MODEL`` are ``resnet32``, ``resnet110``, ``resnet164``, ``revnet38``, ``revnet110``, ``revnet164``\n* Use the ``--fresh`` flag to remove earlier experiment results.\n* Use the ``--no-cuda`` flag to train on the CPU rather than the GPU through CUDA.\n\nDatasets are automatically downloaded if they are not available.\n\nWhen using Python 3.* replace the ``python`` directive with the appropriate Python 3 directive. For example when using the MemCNN docker image use ``python3.6``.\n\nWhen MemCNN was installed using `pip` or from sources you might need to setup a configuration file before running this command.\nRead the corresponding section about how to do this here: https://memcnn.readthedocs.io/en/latest/installation.html\n\n\n\n",
"bugtrack_url": null,
"license": "LICENSE.txt",
"summary": "A PyTorch framework for developing memory efficient deep invertible networks.",
"version": "1.5.2",
"project_urls": {
"Homepage": "http://pypi.python.org/pypi/memcnn/"
},
"split_keywords": [
"memcnn",
"invertible",
"pytorch"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5f5a02e2609a2e4e9ab475ea20d50796f6435882b4bca53add92ce2865e4724b",
"md5": "5244faf6f3094bb49892ecd95821681a",
"sha256": "6414a70e18564e3b3354cebcf86b151f778db03f3b55b2719bbf83d3ba71d9dc"
},
"downloads": -1,
"filename": "memcnn-1.5.2-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "5244faf6f3094bb49892ecd95821681a",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 50274,
"upload_time": "2023-05-10T09:29:49",
"upload_time_iso_8601": "2023-05-10T09:29:49.521255Z",
"url": "https://files.pythonhosted.org/packages/5f/5a/02e2609a2e4e9ab475ea20d50796f6435882b4bca53add92ce2865e4724b/memcnn-1.5.2-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "01ca469972769f97393650e1d10b6ea394fbb7475c73c22fd107640538a92bdc",
"md5": "9218c7318278a6faf5807e1bdba547ea",
"sha256": "5a090f58867fbe0463d81c7ef38868db80886c7d1abbc085676153d7887d824a"
},
"downloads": -1,
"filename": "memcnn-1.5.2.tar.gz",
"has_sig": false,
"md5_digest": "9218c7318278a6faf5807e1bdba547ea",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 50635,
"upload_time": "2023-05-10T09:29:51",
"upload_time_iso_8601": "2023-05-10T09:29:51.890688Z",
"url": "https://files.pythonhosted.org/packages/01/ca/469972769f97393650e1d10b6ea394fbb7475c73c22fd107640538a92bdc/memcnn-1.5.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-05-10 09:29:51",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "memcnn"
}
JSON