[](https://github.com/carlgogo/dl4ds)
[](https://github.com/carlgogo/dl4ds)
[](https://colab.research.google.com/github/carlos-gg/dl4ds/blob/master/notebooks/DL4DS_tutorial.ipynb)
# Deep Learning for empirical DownScaling
`DL4DS` (Deep Learning for empirical DownScaling) is a Python package that implements state-of-the-art and novel deep learning algorithms for empirical downscaling of gridded Earth science data.
The general architecture of `DL4DS` is shown on the image below. A low-resolution gridded dataset can be downscaled, with the help of (an arbitrary number of) auxiliary predictor and static variables, and a high-resolution reference dataset. The mapping between the low- and high-resolution data is learned with either a supervised or a conditional generative adversarial DL model.
<img src="https://github.com/carlos-gg/dl4ds/raw/master/docs/img/fig_workflow.png" alt="drawing" width="800"/>
The training can be done from explicit pairs of high- and low-resolution samples (MOS-style, e.g., high-res observations and low-res numerical weather prediction model output) or only with a HR dataset (PerfectProg-style, e.g., high-res observations or high-res model output).
A wide variety of network architectures have been implemented in `DL4DS`. The main modelling approaches can be combined into many different architectures:
|Downscaling type |Training (loss type) |Sample type |Backbone section |Upsampling method |
|--- |--- |--- |--- |---|
|MOS (explicit pairs of HR and LR data) |Supervised (non-adversarial) |Spatial |Plain convolutional |Pre-upsampling via interpolation |
|PerfectProg (implicit pairs, only HR data) |Conditional Adversarial |Spatio-temporal |Residual |Post-upsampling via sub-pixel convolution |
| | | |Dense |Post-upsampling via resize convolution |
| | | |Unet (PIN, Spatial samples) |Post-upsampling via deconvolution |
| | | |Convnext (Spatial samples) | |
In `DL4DS`, we implement a channel attention mechanism to exploit inter-channel relationship of features by providing a weight for each channel in order to enhance those that contribute the most to the optimizaiton and learning process. Aditionally, a Localized Convolutional Block (LCB) is located in the output module of the networks in `DL4DS`. With the LCB we learn location-specific information via a locally connected layer with biases.
`DL4DS` is built on top of Tensorflow/Keras and supports distributed GPU training (data parallelism) thanks to Horovod.
# API documentation
Check out the API documentation [here](https://carlos-gg.github.io/dl4ds/).
# Installation
```
pip install dl4d
```
# Example notebooks
A first Colab notebook can be found in the notebooks folder. Click the badge at the top to open the notebook on Google Colab.
Raw data
{
"_id": null,
"home_page": "https://github.com/carlgogo/dl4ds",
"name": "dl4d",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "deep learning, downscaling, super-resolution, neural networks, Earth data, EO",
"author": "Carlos Alberto Gomez Gonzalez",
"author_email": "carlos.gomez@bsc.es",
"download_url": "https://files.pythonhosted.org/packages/a6/af/7e663a71483162e153eddb945d4f834ed6e5b1ee85cfca1e8c244246c6a1/dl4d-1.8.1.tar.gz",
"platform": null,
"description": "[](https://github.com/carlgogo/dl4ds) \r\n[](https://github.com/carlgogo/dl4ds) \r\n[](https://colab.research.google.com/github/carlos-gg/dl4ds/blob/master/notebooks/DL4DS_tutorial.ipynb)\r\n\r\n# Deep Learning for empirical DownScaling\r\n\r\n`DL4DS` (Deep Learning for empirical DownScaling) is a Python package that implements state-of-the-art and novel deep learning algorithms for empirical downscaling of gridded Earth science data. \r\n\r\nThe general architecture of `DL4DS` is shown on the image below. A low-resolution gridded dataset can be downscaled, with the help of (an arbitrary number of) auxiliary predictor and static variables, and a high-resolution reference dataset. The mapping between the low- and high-resolution data is learned with either a supervised or a conditional generative adversarial DL model.\r\n\r\n<img src=\"https://github.com/carlos-gg/dl4ds/raw/master/docs/img/fig_workflow.png\" alt=\"drawing\" width=\"800\"/>\r\n\r\nThe training can be done from explicit pairs of high- and low-resolution samples (MOS-style, e.g., high-res observations and low-res numerical weather prediction model output) or only with a HR dataset (PerfectProg-style, e.g., high-res observations or high-res model output).\r\n\r\nA wide variety of network architectures have been implemented in `DL4DS`. The main modelling approaches can be combined into many different architectures:\r\n\r\n|Downscaling type |Training (loss type) |Sample type |Backbone section |Upsampling method |\r\n|--- |--- |--- |--- |---|\r\n|MOS (explicit pairs of HR and LR data) |Supervised (non-adversarial) |Spatial |Plain convolutional |Pre-upsampling via interpolation |\r\n|PerfectProg (implicit pairs, only HR data) |Conditional Adversarial |Spatio-temporal |Residual |Post-upsampling via sub-pixel convolution |\r\n| | | |Dense |Post-upsampling via resize convolution |\r\n| | | |Unet (PIN, Spatial samples) |Post-upsampling via deconvolution |\r\n| | | |Convnext (Spatial samples) | |\r\n\r\nIn `DL4DS`, we implement a channel attention mechanism to exploit inter-channel relationship of features by providing a weight for each channel in order to enhance those that contribute the most to the optimizaiton and learning process. Aditionally, a Localized Convolutional Block (LCB) is located in the output module of the networks in `DL4DS`. With the LCB we learn location-specific information via a locally connected layer with biases. \r\n\r\n`DL4DS` is built on top of Tensorflow/Keras and supports distributed GPU training (data parallelism) thanks to Horovod. \r\n\r\n# API documentation \r\n\r\nCheck out the API documentation [here](https://carlos-gg.github.io/dl4ds/).\r\n\r\n# Installation\r\n\r\n```\r\npip install dl4d\r\n```\r\n\r\n# Example notebooks\r\n\r\nA first Colab notebook can be found in the notebooks folder. Click the badge at the top to open the notebook on Google Colab.\r\n\r\n\r\n\r\n\r\n",
"bugtrack_url": null,
"license": "Apache v2.0",
"summary": "Deep Learning for empirical DownScaling",
"version": "1.8.1",
"project_urls": {
"Homepage": "https://github.com/carlgogo/dl4ds"
},
"split_keywords": [
"deep learning",
" downscaling",
" super-resolution",
" neural networks",
" earth data",
" eo"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a6af7e663a71483162e153eddb945d4f834ed6e5b1ee85cfca1e8c244246c6a1",
"md5": "ffa2a6c968eb833833c40676853f811c",
"sha256": "2fa0346dea52035cf6223cd301465028e3b0cfc2fb132ebfbab2f435e935a9a2"
},
"downloads": -1,
"filename": "dl4d-1.8.1.tar.gz",
"has_sig": false,
"md5_digest": "ffa2a6c968eb833833c40676853f811c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 51678,
"upload_time": "2024-08-01T06:46:47",
"upload_time_iso_8601": "2024-08-01T06:46:47.396317Z",
"url": "https://files.pythonhosted.org/packages/a6/af/7e663a71483162e153eddb945d4f834ed6e5b1ee85cfca1e8c244246c6a1/dl4d-1.8.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-01 06:46:47",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "carlgogo",
"github_project": "dl4ds",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "dl4d"
}
JSON
