# XRPatcher - A ML-Oriented Generic Patcher for `xarray` Data Structures
[**Installation**](#installation)
|[**Examples**](#examples)
[](https://colab.research.google.com/github/jejjohnson/xrpatcher/blob/main/notebooks/pytorch_integration.ipynb)
* J. Emmanuel Johnson
* Quentin Febvre
---
## About
This is a lightweight package to create patch *items* from xarray data structures.
This makes it more compatible with machine learning datasets and dataloaders like PyTorch or TensorFlow.
The user simply needs to define the patch dimensions and the stride dimensions and you are good to go!
It also reconstructs (or unpatchifies) from arbitrary patches which allows for more robust inference procedures, e.g. to account for border effects from CNN models.
---
## ⏩ Examples<a id="examples"></a>
### Quick Example
```python
import xarray as xr
from xrpatcher import XRDAPatcher
# load demo dataset
data = xr.tutorial.load_dataset("eraint_uvz")
# extract demo dataarray
data = data.u[..., :240, :360]
# Instantiate the patching logic for training
patches = dict(longitude=30, latitude=30)
train_patcher = XRDAPatcher(
da=data,
patches=patches,
strides=patches, # No Overlap
check_full_scan=True # check no extra dimensions
)
# Instantiate the patching logic for testing
patches = dict(longitude=30, latitude=30)
strides = dict(longitude=5, latitude=5)
test_patcher = XRDAPatcher(
da=data,
patches=patches,
strides=strides, # Overlap
check_full_scan=True # check no extra dimensions
)
```
### Extended Example
**Example 1: Patching Crash Course** [](https://colab.research.google.com/github/jejjohnson/xrpatcher/blob/main/notebooks/patching_cc.ipynb)
> We have an extended example where we demonstrate some of the ways to do the reconstruction!
**Example 2: PyTorch Integration** [](https://colab.research.google.com/github/jejjohnson/xrpatcher/blob/main/notebooks/pytorch_integration.ipynb)
> We have an extended example where we demonstrate some nifty PyTorch Integration.
---
## 🛠️ Installation<a id="installation"></a>
### pip
We can directly install it via pip from the
```bash
pip install "git+https://github.com/jejjohnson/xrpatcher.git"
```
### Cloning
We can also clone the git repository
```bash
git clone https://github.com/jejjohnson/xrpatcher.git
cd xrpatcher
```
#### Conda Environment (RECOMMENDED)
We use conda/mamba as our package manager. To install from the provided environment files
run the following command.
```bash
mamba env create -n environment.yaml
```
#### poetry
The easiest way to get started is to simply use the poetry package which installs all necessary dev packages as well
```bash
poetry install
```
#### pip
We can also install via `pip` as well
```bash
pip install .
```
---
## Inspiration
There are a few other packages that gave us inspiration for this.
* [xbatcher](https://xbatcher.readthedocs.io/en/latest/index.html)
* [PatchExtractor (scikit-learn)](https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.image.PatchExtractor.html)
Raw data
{
"_id": null,
"home_page": "https://github.com/jejjohnson/xrpatcher",
"name": "xrpatcher",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<3.12",
"maintainer_email": "",
"keywords": "xarray,patcher,pytorch",
"author": "J. Emmanuel Johnson",
"author_email": "jemanjohnson34@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/c7/53/22f1b7ce06c6d5b13660f384ad3838634cd4e1663c842c11e37f168a7a45/xrpatcher-0.0.1.tar.gz",
"platform": null,
"description": "# XRPatcher - A ML-Oriented Generic Patcher for `xarray` Data Structures\n\n[**Installation**](#installation)\n|[**Examples**](#examples)\n\n\n\n[](https://colab.research.google.com/github/jejjohnson/xrpatcher/blob/main/notebooks/pytorch_integration.ipynb)\n\n\n* J. Emmanuel Johnson\n* Quentin Febvre\n\n---\n## About\n\nThis is a lightweight package to create patch *items* from xarray data structures.\nThis makes it more compatible with machine learning datasets and dataloaders like PyTorch or TensorFlow.\nThe user simply needs to define the patch dimensions and the stride dimensions and you are good to go!\nIt also reconstructs (or unpatchifies) from arbitrary patches which allows for more robust inference procedures, e.g. to account for border effects from CNN models.\n\n---\n## \u23e9 Examples<a id=\"examples\"></a>\n\n### Quick Example\n\n```python\nimport xarray as xr\nfrom xrpatcher import XRDAPatcher\n\n# load demo dataset\ndata = xr.tutorial.load_dataset(\"eraint_uvz\")\n\n# extract demo dataarray\ndata = data.u[..., :240, :360]\n\n# Instantiate the patching logic for training\npatches = dict(longitude=30, latitude=30)\ntrain_patcher = XRDAPatcher(\n da=data,\n patches=patches,\n strides=patches, # No Overlap\n check_full_scan=True # check no extra dimensions\n)\n\n# Instantiate the patching logic for testing\npatches = dict(longitude=30, latitude=30)\nstrides = dict(longitude=5, latitude=5)\n\ntest_patcher = XRDAPatcher(\n da=data,\n patches=patches,\n strides=strides, # Overlap\n check_full_scan=True # check no extra dimensions\n)\n```\n\n### Extended Example\n\n**Example 1: Patching Crash Course** [](https://colab.research.google.com/github/jejjohnson/xrpatcher/blob/main/notebooks/patching_cc.ipynb)\n\n> We have an extended example where we demonstrate some of the ways to do the reconstruction!\n\n**Example 2: PyTorch Integration** [](https://colab.research.google.com/github/jejjohnson/xrpatcher/blob/main/notebooks/pytorch_integration.ipynb)\n\n> We have an extended example where we demonstrate some nifty PyTorch Integration.\n\n---\n\n## \ud83d\udee0\ufe0f Installation<a id=\"installation\"></a>\n\n\n### pip\n\nWe can directly install it via pip from the\n\n```bash\npip install \"git+https://github.com/jejjohnson/xrpatcher.git\"\n```\n\n### Cloning\n\nWe can also clone the git repository\n\n```bash\ngit clone https://github.com/jejjohnson/xrpatcher.git\ncd xrpatcher\n```\n\n#### Conda Environment (RECOMMENDED)\n\nWe use conda/mamba as our package manager. To install from the provided environment files\nrun the following command.\n\n```bash\nmamba env create -n environment.yaml\n```\n\n#### poetry\n\nThe easiest way to get started is to simply use the poetry package which installs all necessary dev packages as well\n\n```bash\npoetry install\n```\n\n#### pip\n\nWe can also install via `pip` as well\n\n```bash\npip install .\n```\n\n---\n## Inspiration\n\nThere are a few other packages that gave us inspiration for this.\n\n* [xbatcher](https://xbatcher.readthedocs.io/en/latest/index.html)\n* [PatchExtractor (scikit-learn)](https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.image.PatchExtractor.html)\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A ML-oriented generic patcher for xarray data structures.",
"version": "0.0.1",
"project_urls": {
"Homepage": "https://github.com/jejjohnson/xrpatcher",
"Repository": "https://github.com/jejjohnson/xrpatcher"
},
"split_keywords": [
"xarray",
"patcher",
"pytorch"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "9fbb1772af46db80aba04acca6490163cbc7932b6f2faffd582d37c4010184ba",
"md5": "723969e530e26e64544efe85198b1241",
"sha256": "ba874b2f70320cce1f10a07ed7270814cee52987c9f1fff00bfa3335fd102019"
},
"downloads": -1,
"filename": "xrpatcher-0.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "723969e530e26e64544efe85198b1241",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<3.12",
"size": 11570,
"upload_time": "2023-06-14T09:43:28",
"upload_time_iso_8601": "2023-06-14T09:43:28.350352Z",
"url": "https://files.pythonhosted.org/packages/9f/bb/1772af46db80aba04acca6490163cbc7932b6f2faffd582d37c4010184ba/xrpatcher-0.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c75322f1b7ce06c6d5b13660f384ad3838634cd4e1663c842c11e37f168a7a45",
"md5": "96358ccb65c3c25ff7a96007fa751139",
"sha256": "d98663430ee3070512e44ae65b2bc47b335e866a0d4a3ab864129c932a2ec97f"
},
"downloads": -1,
"filename": "xrpatcher-0.0.1.tar.gz",
"has_sig": false,
"md5_digest": "96358ccb65c3c25ff7a96007fa751139",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<3.12",
"size": 11702,
"upload_time": "2023-06-14T09:43:30",
"upload_time_iso_8601": "2023-06-14T09:43:30.224771Z",
"url": "https://files.pythonhosted.org/packages/c7/53/22f1b7ce06c6d5b13660f384ad3838634cd4e1663c842c11e37f168a7a45/xrpatcher-0.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-14 09:43:30",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "jejjohnson",
"github_project": "xrpatcher",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "xrpatcher"
}
JSON
