Torch Liberator - Deploy PyTorch Models
---------------------------------------
|GitlabCIPipeline| |GitlabCICoverage| |Pypi| |Downloads|
+----------------------+-------------------------------------------------------------+
| Main Page | https://gitlab.kitware.com/computer-vision/torch_liberator |
+----------------------+-------------------------------------------------------------+
| Github Mirror | https://github.com/Kitware/torch_liberator |
+----------------------+-------------------------------------------------------------+
| Pypi | https://pypi.org/project/torch_liberator |
+----------------------+-------------------------------------------------------------+
| Torch Hackathon 2021 | `Youtube Video`_, `Google Slides`_, and `Submission Page`_ |
+----------------------+-------------------------------------------------------------+
.. _Youtube Video: https://www.youtube.com/watch?v=GQqtn61iNsc
.. _Google Slides: https://docs.google.com/presentation/d/1w9XHkPjtLRj29dw50WP0rSHRRlEfhksP_Sf8XldTSYE
.. _Submission Page: https://devpost.com/software/torchliberator-partial-weight-loading
Torch Liberator is a Python module containing a set of tools for reading and
writing relevant parts of deep networks.
Typically, when moving a deep-network trained with torch, you have to keep
track of the entire codebase that defined the model file in addition to the
checkpoint file containing the learned weights. Torch Liberator contains tools
to extract relevant source code and bundle it with weights and serializing it
into a single-file deployment.
Note: as of torch 1.9, torch comes with a `torch.package <https://pytorch.org/docs/stable/package.html>`__
submodule which contains a method for saving model weights with model
structure. We recommend using ``torch.package`` over the single-file deployments
provided in this package. Thus the ``load_partial_state`` logic is the main
code of interest provided in this module.
Installation
------------
.. code:: bash
pip install torch_liberator
# OR with a specific branch
pip install git+https://gitlab.kitware.com/computer-vision/torch_liberator.git@main
Partial State Loading
---------------------
New in ``0.1.0`` torch liberator now exposes a public ``load_partial_state``
function, which does it best to "shove" weights from one model into another
model. There are several methods to compute associations between layer names in
one model to layer names in another, the most general being the "embedding"
method, and the slightly more structured "isomorphism" option.
Have you ever had the scenario where you use one model as a sub-model in a
bigger network? Then you had to load pretrained subnetwork state into that
bigger model?
The latest version of ``torch_liberator.load_patial_state`` can handle this by
solving a maximum-common-subtree-isomorphism problem. This computes the largest
possible mapping between the two state dictionaries that share consistent
suffixes.
.. code:: python
>>> import torchvision
>>> import torch
>>> import torch_liberator
>>> resnet50 = torchvision.models.resnet50()
>>> class CustomModel(torch.nn.Module):
>>> def __init__(self):
>>> super().__init__()
>>> self.module = resnet50
>>> self.extra = torch.nn.Linear(1, 1)
>>> # Directly load resnet50 state into a model that has it as an embedded subnetwork
>>> model = CustomModel()
>>> model_state_dict = resnet50.state_dict()
>>> # load partial state returns information about what it did
>>> info = torch_liberator.load_partial_state(model, model_state_dict, association='isomorphism', verbose=1)
>>> print(len(info['seen']['full_add']))
>>> print(len(info['self_unset']))
>>> print(len(info['other_unused']))
320
2
0
It can also handle loading common state between two models that share some
underlying structure.
.. code:: python
>>> import torchvision
>>> import torch
>>> import torch_liberator
>>> resnet50 = torchvision.models.resnet50()
>>> class CustomModel1(torch.nn.Module):
>>> def __init__(self):
>>> super().__init__()
>>> self.module = resnet50
>>> self.custom_model1_layer = torch.nn.Linear(1, 1)
>>> class CustomModel2(torch.nn.Module):
>>> def __init__(self):
>>> super().__init__()
>>> self.backbone = resnet50
>>> self.custom_model2_layer = torch.nn.Linear(1, 1)
>>> # Load as much of model1 state into model2 as possible
>>> model1 = CustomModel1()
>>> model2 = CustomModel2()
>>> model2_state_dict = model2.state_dict()
>>> # load partial state returns information about what it did
>>> info = torch_liberator.load_partial_state(model1, model2_state_dict, association='isomorphism', verbose=1)
>>> print(len(info['seen']['full_add']))
>>> print(len(info['seen']['skipped']))
>>> print(len(info['self_unset']))
>>> print(len(info['other_unused']))
320
2
2
2
.. code:: python
>>> import torchvision
>>> import torch_liberator
>>> #
>>> faster_rcnn = torchvision.models.detection.faster_rcnn.fasterrcnn_resnet50_fpn()
>>> resnet50 = torchvision.models.resnet50(pretrained=True)
>>> state_dict = resnet50.state_dict()
>>> # Load partial state return a dictionary that tells you how well it did
>>> info = torch_liberator.load_partial_state(faster_rcnn, state_dict, verbose=0, association='embedding')
>>> print(ub.map_vals(len, info['seen']))
>>> print(ub.map_vals(len, ub.dict_diff(info, ['seen'])))
{'full_add': 265, 'skipped': 55}
{'other_unused': 55, 'self_unset': 30}
>>> # Load partial state return a dictionary that tells you how well it did
>>> info = torch_liberator.load_partial_state(faster_rcnn, state_dict, verbose=0, association='isomorphism')
>>> print(ub.map_vals(len, info['seen']))
>>> print(ub.map_vals(len, ub.dict_diff(info, ['seen'])))
{'full_add': 265, 'skipped': 55}
{'other_unused': 55, 'self_unset': 30}
Also, if the sizes of the tensor don't quite fit, they will be mangled, i.e.
"shoved-in" as best as possible. See the docstring for more detail.
Stand-alone Single-File Model Deployments
-----------------------------------------
The original purpose of ``torch_liberator`` was to build standalone torch
packages that contained both the model code and the model weight. It still does
that but ``torch.package`` new in torch 1.9, might be a better solution moving
forward. See `torch.package <https://pytorch.org/docs/stable/package.html>`__
for details.
Torch Liberator builds on the
`liberator <https://gitlab.kitware.com/python/liberator>`__ library to statically
extract pytorch code that defines a model's topology and bundle that with a
pretrained weights file. This results in a single-file deployment package and
can potentially remove dependencies on the codebase used to train the model.
Torch Liberator can also read these deployment files and create an instance of
the model initialized with the correct pretrained weights.
The API is ok, but it does need improvement. However, the current version is in
a working state. There aren't any high level docs, but there are a lot of
docstrings and doctests. The example here gives a good overview of the code by
extracting the AlexNet model from torchvision.
.. code:: python
>>> import torch_liberator
>>> from torch_liberator.deployer import DeployedModel
>>> from torchvision import models
>>> print('--- DEFINE A MODEL ---')
>>> model = models.alexnet(pretrained=False) # false for test speed
>>> initkw = dict(num_classes=1000) # not all models nicely supply this
>>> model._initkw = initkw
--- DEFINE A MODEL ---
>>> print('--- DEPLOY THE MODEL ---')
>>> zip_fpath = torch_liberator.deploy(model, 'test-deploy.zip')
--- DEPLOY THE MODEL ---
[DEPLOYER] Deployed zipfpath=/tmp/tmpeqd3y_rx/test-deploy.zip
>>> print('--- LOAD THE DEPLOYED MODEL ---')
>>> loader = DeployedModel(zip_fpath)
>>> model = loader.load_model()
--- LOAD THE DEPLOYED MODEL ---
Loading data onto None from <zopen(<_io.BufferedReader name='/tmp/tmpg1kln3kw/test-deploy/deploy_snapshot.pt'> mode=rb)>
Pretrained weights are a perfect fit
The major weirdness right now, is you either have to explicitly define "initkw"
(which are the keyword arguments used to create an instance of our model) at
deploy time, or you can set it as the ``_initkw`` attribute of your model (or
if your keyword arguments all exist as member variables of the class,
torch_liberator tries to be smart and infer what initkw should be).
There is also a torch-liberator CLI that can be used to package a weight file,
a python model file, and optional json metadata.
.. code:: bash
python -m torch_liberator \
--model <path-to-the-liberated-py-file> \
--weights <path-to-the-torch-pth-weight-file> \
--info <path-to-train-info-json-file> \
--dst my_custom_deployfile.zip
.. |Pypi| image:: https://img.shields.io/pypi/v/torch_liberator.svg
:target: https://pypi.python.org/pypi/torch_liberator
.. |Downloads| image:: https://img.shields.io/pypi/dm/torch_liberator.svg
:target: https://pypistats.org/packages/torch_liberator
.. |ReadTheDocs| image:: https://readthedocs.org/projects/torch_liberator/badge/?version=latest
:target: http://torch_liberator.readthedocs.io/en/latest/
.. # See: https://ci.appveyor.com/project/jon.crall/torch_liberator/settings/badges
.. .. |Appveyor| image:: https://ci.appveyor.com/api/projects/status/py3s2d6tyfjc8lm3/branch/main?svg=true
.. :target: https://ci.appveyor.com/project/jon.crall/torch_liberator/branch/main
.. |GitlabCIPipeline| image:: https://gitlab.kitware.com/computer-vision/torch_liberator/badges/main/pipeline.svg
:target: https://gitlab.kitware.com/computer-vision/torch_liberator/-/jobs
.. |GitlabCICoverage| image:: https://gitlab.kitware.com/computer-vision/torch_liberator/badges/main/coverage.svg?job=coverage
:target: https://gitlab.kitware.com/computer-vision/torch_liberator/commits/main
Raw data
{
"_id": null,
"home_page": "https://gitlab.kitware.com/computer-vision/torch_liberator",
"name": "torch-liberator",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "",
"author": "Jon Crall",
"author_email": "jon.crall@kitware.com",
"download_url": "",
"platform": null,
"description": "Torch Liberator - Deploy PyTorch Models \n---------------------------------------\n\n|GitlabCIPipeline| |GitlabCICoverage| |Pypi| |Downloads| \n\n+----------------------+-------------------------------------------------------------+\n| Main Page | https://gitlab.kitware.com/computer-vision/torch_liberator |\n+----------------------+-------------------------------------------------------------+\n| Github Mirror | https://github.com/Kitware/torch_liberator |\n+----------------------+-------------------------------------------------------------+\n| Pypi | https://pypi.org/project/torch_liberator |\n+----------------------+-------------------------------------------------------------+\n| Torch Hackathon 2021 | `Youtube Video`_, `Google Slides`_, and `Submission Page`_ |\n+----------------------+-------------------------------------------------------------+\n\n.. _Youtube Video: https://www.youtube.com/watch?v=GQqtn61iNsc\n.. _Google Slides: https://docs.google.com/presentation/d/1w9XHkPjtLRj29dw50WP0rSHRRlEfhksP_Sf8XldTSYE\n.. _Submission Page: https://devpost.com/software/torchliberator-partial-weight-loading\n\nTorch Liberator is a Python module containing a set of tools for reading and\nwriting relevant parts of deep networks.\n\nTypically, when moving a deep-network trained with torch, you have to keep\ntrack of the entire codebase that defined the model file in addition to the\ncheckpoint file containing the learned weights. Torch Liberator contains tools\nto extract relevant source code and bundle it with weights and serializing it\ninto a single-file deployment.\n\nNote: as of torch 1.9, torch comes with a `torch.package <https://pytorch.org/docs/stable/package.html>`__ \nsubmodule which contains a method for saving model weights with model\nstructure. We recommend using ``torch.package`` over the single-file deployments\nprovided in this package. Thus the ``load_partial_state`` logic is the main\ncode of interest provided in this module.\n\n\n\nInstallation\n------------\n\n.. code:: bash\n\n pip install torch_liberator\n\n # OR with a specific branch\n\n pip install git+https://gitlab.kitware.com/computer-vision/torch_liberator.git@main\n\n\nPartial State Loading\n---------------------\n\nNew in ``0.1.0`` torch liberator now exposes a public ``load_partial_state``\nfunction, which does it best to \"shove\" weights from one model into another\nmodel. There are several methods to compute associations between layer names in\none model to layer names in another, the most general being the \"embedding\"\nmethod, and the slightly more structured \"isomorphism\" option.\n\n\nHave you ever had the scenario where you use one model as a sub-model in a\nbigger network? Then you had to load pretrained subnetwork state into that\nbigger model? \n\nThe latest version of ``torch_liberator.load_patial_state`` can handle this by\nsolving a maximum-common-subtree-isomorphism problem. This computes the largest\npossible mapping between the two state dictionaries that share consistent\nsuffixes.\n\n.. code:: python \n\n >>> import torchvision\n >>> import torch\n >>> import torch_liberator\n >>> resnet50 = torchvision.models.resnet50()\n >>> class CustomModel(torch.nn.Module):\n >>> def __init__(self):\n >>> super().__init__()\n >>> self.module = resnet50\n >>> self.extra = torch.nn.Linear(1, 1)\n >>> # Directly load resnet50 state into a model that has it as an embedded subnetwork\n >>> model = CustomModel()\n >>> model_state_dict = resnet50.state_dict()\n >>> # load partial state returns information about what it did\n >>> info = torch_liberator.load_partial_state(model, model_state_dict, association='isomorphism', verbose=1)\n >>> print(len(info['seen']['full_add']))\n >>> print(len(info['self_unset']))\n >>> print(len(info['other_unused']))\n 320\n 2\n 0\n \n\nIt can also handle loading common state between two models that share some\nunderlying structure.\n\n.. code:: python \n\n >>> import torchvision\n >>> import torch\n >>> import torch_liberator\n >>> resnet50 = torchvision.models.resnet50()\n >>> class CustomModel1(torch.nn.Module):\n >>> def __init__(self):\n >>> super().__init__()\n >>> self.module = resnet50\n >>> self.custom_model1_layer = torch.nn.Linear(1, 1)\n >>> class CustomModel2(torch.nn.Module):\n >>> def __init__(self):\n >>> super().__init__()\n >>> self.backbone = resnet50\n >>> self.custom_model2_layer = torch.nn.Linear(1, 1)\n >>> # Load as much of model1 state into model2 as possible\n >>> model1 = CustomModel1()\n >>> model2 = CustomModel2()\n >>> model2_state_dict = model2.state_dict()\n >>> # load partial state returns information about what it did\n >>> info = torch_liberator.load_partial_state(model1, model2_state_dict, association='isomorphism', verbose=1)\n >>> print(len(info['seen']['full_add']))\n >>> print(len(info['seen']['skipped']))\n >>> print(len(info['self_unset']))\n >>> print(len(info['other_unused']))\n 320\n 2\n 2\n 2\n\n\n.. code:: python \n\n\n >>> import torchvision\n >>> import torch_liberator\n >>> #\n >>> faster_rcnn = torchvision.models.detection.faster_rcnn.fasterrcnn_resnet50_fpn()\n >>> resnet50 = torchvision.models.resnet50(pretrained=True)\n >>> state_dict = resnet50.state_dict()\n >>> # Load partial state return a dictionary that tells you how well it did\n >>> info = torch_liberator.load_partial_state(faster_rcnn, state_dict, verbose=0, association='embedding')\n >>> print(ub.map_vals(len, info['seen']))\n >>> print(ub.map_vals(len, ub.dict_diff(info, ['seen'])))\n {'full_add': 265, 'skipped': 55}\n {'other_unused': 55, 'self_unset': 30}\n\n >>> # Load partial state return a dictionary that tells you how well it did\n >>> info = torch_liberator.load_partial_state(faster_rcnn, state_dict, verbose=0, association='isomorphism')\n >>> print(ub.map_vals(len, info['seen']))\n >>> print(ub.map_vals(len, ub.dict_diff(info, ['seen'])))\n {'full_add': 265, 'skipped': 55}\n {'other_unused': 55, 'self_unset': 30}\n \n \n\nAlso, if the sizes of the tensor don't quite fit, they will be mangled, i.e.\n\"shoved-in\" as best as possible. See the docstring for more detail.\n\n\nStand-alone Single-File Model Deployments\n-----------------------------------------\n\nThe original purpose of ``torch_liberator`` was to build standalone torch\npackages that contained both the model code and the model weight. It still does\nthat but ``torch.package`` new in torch 1.9, might be a better solution moving\nforward. See `torch.package <https://pytorch.org/docs/stable/package.html>`__\nfor details.\n\nTorch Liberator builds on the\n`liberator <https://gitlab.kitware.com/python/liberator>`__ library to statically\nextract pytorch code that defines a model's topology and bundle that with a\npretrained weights file. This results in a single-file deployment package and\ncan potentially remove dependencies on the codebase used to train the model.\n\nTorch Liberator can also read these deployment files and create an instance of\nthe model initialized with the correct pretrained weights.\n\nThe API is ok, but it does need improvement. However, the current version is in\na working state. There aren't any high level docs, but there are a lot of\ndocstrings and doctests. The example here gives a good overview of the code by\nextracting the AlexNet model from torchvision.\n\n\n.. code:: python \n\n >>> import torch_liberator\n >>> from torch_liberator.deployer import DeployedModel\n >>> from torchvision import models\n\n >>> print('--- DEFINE A MODEL ---')\n >>> model = models.alexnet(pretrained=False) # false for test speed\n >>> initkw = dict(num_classes=1000) # not all models nicely supply this\n >>> model._initkw = initkw\n --- DEFINE A MODEL ---\n\n >>> print('--- DEPLOY THE MODEL ---')\n >>> zip_fpath = torch_liberator.deploy(model, 'test-deploy.zip')\n --- DEPLOY THE MODEL ---\n [DEPLOYER] Deployed zipfpath=/tmp/tmpeqd3y_rx/test-deploy.zip\n \n\n >>> print('--- LOAD THE DEPLOYED MODEL ---')\n >>> loader = DeployedModel(zip_fpath)\n >>> model = loader.load_model()\n --- LOAD THE DEPLOYED MODEL ---\n Loading data onto None from <zopen(<_io.BufferedReader name='/tmp/tmpg1kln3kw/test-deploy/deploy_snapshot.pt'> mode=rb)>\n Pretrained weights are a perfect fit\n \n\nThe major weirdness right now, is you either have to explicitly define \"initkw\"\n(which are the keyword arguments used to create an instance of our model) at\ndeploy time, or you can set it as the ``_initkw`` attribute of your model (or\nif your keyword arguments all exist as member variables of the class,\ntorch_liberator tries to be smart and infer what initkw should be).\n\n\nThere is also a torch-liberator CLI that can be used to package a weight file,\na python model file, and optional json metadata.\n\n.. code:: bash\n\n python -m torch_liberator \\\n --model <path-to-the-liberated-py-file> \\\n --weights <path-to-the-torch-pth-weight-file> \\\n --info <path-to-train-info-json-file> \\\n --dst my_custom_deployfile.zip\n\n\n.. |Pypi| image:: https://img.shields.io/pypi/v/torch_liberator.svg\n :target: https://pypi.python.org/pypi/torch_liberator\n\n.. |Downloads| image:: https://img.shields.io/pypi/dm/torch_liberator.svg\n :target: https://pypistats.org/packages/torch_liberator\n\n.. |ReadTheDocs| image:: https://readthedocs.org/projects/torch_liberator/badge/?version=latest\n :target: http://torch_liberator.readthedocs.io/en/latest/\n\n.. # See: https://ci.appveyor.com/project/jon.crall/torch_liberator/settings/badges\n.. .. |Appveyor| image:: https://ci.appveyor.com/api/projects/status/py3s2d6tyfjc8lm3/branch/main?svg=true\n.. :target: https://ci.appveyor.com/project/jon.crall/torch_liberator/branch/main\n\n.. |GitlabCIPipeline| image:: https://gitlab.kitware.com/computer-vision/torch_liberator/badges/main/pipeline.svg\n :target: https://gitlab.kitware.com/computer-vision/torch_liberator/-/jobs\n\n.. |GitlabCICoverage| image:: https://gitlab.kitware.com/computer-vision/torch_liberator/badges/main/coverage.svg?job=coverage\n :target: https://gitlab.kitware.com/computer-vision/torch_liberator/commits/main\n\n",
"bugtrack_url": null,
"license": "Apache 2",
"summary": "The torch liberator module",
"version": "0.2.1",
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6633cb0d53200183b597a459e5017d5ef2d37730d679feb3088d6dde121a8d7e",
"md5": "77ced679a5764bcb2327cd6bc3de7b0b",
"sha256": "5aacbfb0327508e3d54e4a8b1cce2ee95b8fa7576da9b8493acac9ecf69e1440"
},
"downloads": -1,
"filename": "torch_liberator-0.2.1-py3-none-any.whl",
"has_sig": true,
"md5_digest": "77ced679a5764bcb2327cd6bc3de7b0b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 47390,
"upload_time": "2023-02-12T00:52:34",
"upload_time_iso_8601": "2023-02-12T00:52:34.846925Z",
"url": "https://files.pythonhosted.org/packages/66/33/cb0d53200183b597a459e5017d5ef2d37730d679feb3088d6dde121a8d7e/torch_liberator-0.2.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-02-12 00:52:34",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "torch-liberator"
}
JSON