Sets of classes and interfaces for the definition of deep learning models.
The framework is conceived to be extensible and serializable, in order to provide models that can be stored and/or sent through the network.
## Interfaces.py
Set of abstract classes that describe models and model providers.
### DeepLearningClass
The rationale behind this is that an algorithm not only depends on the model, but might require some preprocessing steps (reformatting, normalization) before the application of the model itself. The interface is also "incremental-learning-oriented" as it provides abstract methods for the calculation and integration of deltas. The class provides operator override for the calculation of deltas (D = A-B) and call override for the apply method.
An instance of DeepLearningClass should be able to simply provide the expected result when called with the appropriate input data (in form of dict). The most common data for our case will be:
- Input: `{'image': 2D image, 'resolution': Sequence with pixel sizes}`
- Output: for Classifiers: `str`, for Segmenters: `dict[str: image]` where str is the label and the image is a 2D numpy array containing the mask corresponding to the string.
### ModelProvider
This is an abstract class that is intended to encapsulate the logic of model load/transfer. It will have two subclasses, LocalModelProvider and RemoteModelProvider. In both cases, the method load_model of this class accepts a description of the requested model and will return an instance of DeepLearningClass. In principle, LocalModelProvider will be used on the server, and RemoteModelProvider on the client.
## DynamicDLModel
Concrete implementation of a DeepLearningClass which provides flexibility and serialization. This class delegates the important methods of the model implementation to top-level functions that are passed at the construction time. These functions can be easily serialized with the dill library.
Constructor:
def __init__(self, model_id, # a unique ID to avoid mixing different models
init_model_function, # inits the model. Accepts no parameters and returns the model
apply_model_function, # function that applies the model. Has the object, and image, and a sequence containing resolutions as parameters
weights_to_model_function = default_keras_weights_to_model_function, # put model weights inside the model.
model_to_weights_function = default_keras_model_to_weights_function, # get the weights from the model in a pickable format
calc_delta_function = default_keras_delta_function, # calculate the weight delta
apply_delta_function = default_keras_apply_delta_function, # apply a weight delta
incremental_learn_function = None, # function to perform an incremental learning step
weights = None): # initial weights
This allows the implementation of a very generic deep learning algorithm which includes the preprocessing steps in a way that can be serialized and defined at runtime, so if we want to change the model, we don't need to change the code of the client or server, as the implementation is self-contained within the model.
The class provides the methods `dump(file_descriptor)` and `str = dumps()` to serialize and the static methods `Load(file_descriptor)` and `Loads(str)` to deserialize.
Default functions for loading/setting keras weights and calculating deltas from keras models (which provide a get_weights(), set_weights() interface with lists of numpy arrays) are currently provided.
**Important note when defining the functions**: in order for them to be serializable, they must be completely self-contained. That is, all imports should happen inside the functions and all the external function call should be implemented as nested functions. Common algorithms (such as padorcut.py which pads or cuts an image to fit it to a specific matrix size) should be placed in the repository.
Raw data
{
"_id": null,
"home_page": "https://github.com/dafne-imaging/dafne-dl",
"name": "dafne-dl-torch-models",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": null,
"author": "Francesco Santini",
"author_email": "francesco.santini@unibas.ch",
"download_url": "https://files.pythonhosted.org/packages/5f/58/44925b72cc9b380782092778d849d5d30323bb5ef59ff032f0e275261be1/dafne_dl_torch_models-2.0a3.tar.gz",
"platform": null,
"description": "Sets of classes and interfaces for the definition of deep learning models.\nThe framework is conceived to be extensible and serializable, in order to provide models that can be stored and/or sent through the network.\n## Interfaces.py\nSet of abstract classes that describe models and model providers. \n### DeepLearningClass\nThe rationale behind this is that an algorithm not only depends on the model, but might require some preprocessing steps (reformatting, normalization) before the application of the model itself. The interface is also \"incremental-learning-oriented\" as it provides abstract methods for the calculation and integration of deltas. The class provides operator override for the calculation of deltas (D = A-B) and call override for the apply method.\nAn instance of DeepLearningClass should be able to simply provide the expected result when called with the appropriate input data (in form of dict). The most common data for our case will be:\n\n - Input: `{'image': 2D image, 'resolution': Sequence with pixel sizes}`\n - Output: for Classifiers: `str`, for Segmenters: `dict[str: image]` where str is the label and the image is a 2D numpy array containing the mask corresponding to the string.\n\n### ModelProvider\nThis is an abstract class that is intended to encapsulate the logic of model load/transfer. It will have two subclasses, LocalModelProvider and RemoteModelProvider. In both cases, the method load_model of this class accepts a description of the requested model and will return an instance of DeepLearningClass. In principle, LocalModelProvider will be used on the server, and RemoteModelProvider on the client.\n\n## DynamicDLModel\nConcrete implementation of a DeepLearningClass which provides flexibility and serialization. This class delegates the important methods of the model implementation to top-level functions that are passed at the construction time. These functions can be easily serialized with the dill library.\nConstructor:\n\n def __init__(self, model_id, # a unique ID to avoid mixing different models\n init_model_function, # inits the model. Accepts no parameters and returns the model\n apply_model_function, # function that applies the model. Has the object, and image, and a sequence containing resolutions as parameters\n weights_to_model_function = default_keras_weights_to_model_function, # put model weights inside the model.\n model_to_weights_function = default_keras_model_to_weights_function, # get the weights from the model in a pickable format\n calc_delta_function = default_keras_delta_function, # calculate the weight delta\n apply_delta_function = default_keras_apply_delta_function, # apply a weight delta\n incremental_learn_function = None, # function to perform an incremental learning step\n weights = None): # initial weights\nThis allows the implementation of a very generic deep learning algorithm which includes the preprocessing steps in a way that can be serialized and defined at runtime, so if we want to change the model, we don't need to change the code of the client or server, as the implementation is self-contained within the model.\nThe class provides the methods `dump(file_descriptor)` and `str = dumps()` to serialize and the static methods `Load(file_descriptor)` and `Loads(str)` to deserialize.\nDefault functions for loading/setting keras weights and calculating deltas from keras models (which provide a get_weights(), set_weights() interface with lists of numpy arrays) are currently provided.\n**Important note when defining the functions**: in order for them to be serializable, they must be completely self-contained. That is, all imports should happen inside the functions and all the external function call should be implemented as nested functions. Common algorithms (such as padorcut.py which pads or cuts an image to fit it to a specific matrix size) should be placed in the repository.\n",
"bugtrack_url": null,
"license": null,
"summary": "Deep Learning module for Dafne",
"version": "2.0a3",
"project_urls": {
"Bug Tracker": "https://github.com/dafne-imaging/dafne-dl/issues",
"Homepage": "https://github.com/dafne-imaging/dafne-dl"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "29d4cfae44818ecf641ce46b0f106c2c1ff6acf534b799859f8040b928fa8280",
"md5": "e19eed1c7ba28ebad853cc3ea61df3b4",
"sha256": "9d609f05a95eea1f9c53c67d285f532eae73a2da52e006dd565029960840ff73"
},
"downloads": -1,
"filename": "dafne_dl_torch_models-2.0a3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e19eed1c7ba28ebad853cc3ea61df3b4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 49779,
"upload_time": "2024-10-29T14:09:09",
"upload_time_iso_8601": "2024-10-29T14:09:09.133458Z",
"url": "https://files.pythonhosted.org/packages/29/d4/cfae44818ecf641ce46b0f106c2c1ff6acf534b799859f8040b928fa8280/dafne_dl_torch_models-2.0a3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5f5844925b72cc9b380782092778d849d5d30323bb5ef59ff032f0e275261be1",
"md5": "448fd5ff2040ce302220d7f04bb64a24",
"sha256": "ae3e11c13ae4124b98321441a889799595e04242ded6627b7f75fa1dcb89ffc5"
},
"downloads": -1,
"filename": "dafne_dl_torch_models-2.0a3.tar.gz",
"has_sig": false,
"md5_digest": "448fd5ff2040ce302220d7f04bb64a24",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 38160,
"upload_time": "2024-10-29T14:09:10",
"upload_time_iso_8601": "2024-10-29T14:09:10.457581Z",
"url": "https://files.pythonhosted.org/packages/5f/58/44925b72cc9b380782092778d849d5d30323bb5ef59ff032f0e275261be1/dafne_dl_torch_models-2.0a3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-29 14:09:10",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "dafne-imaging",
"github_project": "dafne-dl",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "dafne-dl-torch-models"
}
JSON
