| Name | hdf5tools JSON |
| Version |
0.4.5
JSON |
| download |
| home_page | None |
| Summary | Python hdf5 tools |
| upload_time | 2024-12-15 01:55:38 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | None |
| license | None |
| keywords |
tethys
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
hdf5-tools
==================================
This git repository contains a python package with an H5 class to load and combine one or more HDF5 data files (or xarray datasets) with optional filters. The class will then export the combined data to an HDF5 file, file object, or xr.Dataset. This class is designed to be fast and safe on memory. This means that files of any size can be combined and saved even on a PC with low memory (unlike xarray).
Installation
------------
Using pip:
.. code::
pip install hdf5tools
Or using conda/mamba from conda-forge:
.. code::
conda install -c conda-forge hdf5tools
Usage
-------
Currently, only the **Combine** class is recommended for other to use.
First, initiate the class with one or many: paths to netcdf3, netcdf4, or hdf5 files; xr.Dataset objects (opened via xr.open_dataset); or h5py.File objects.
.. code:: python
from hdf5tools import Combine
###############################
### Parameters
path1 = '/path/to/file1.nc'
path2 = '/path/to/file2.nc'
##############################
### Combine files
c1 = Combine([path1, path2])
If you want to do some kind of selection via the coordinates or only select some of the data variables/coordinates then use the **.sel** method (like in xarray). Be sure to read the docstrings for additional info about the input parameters.
.. code:: python
c2 = c1.sel({'time': slice('2001-01-01', '2020-01-01'), 'latitude': slice(0, 10)}, include_data_vars=['precipitation'])
And finally, save the combined data to a single hdf5/netcdf4 file using the **.to_hdf5** method. The only additional parameters that are important include the output which should be a path or a io.Bytes object, and the compression parameter. If you plan on using this file outside of the python environment, use gzip for compression, otherwise use lzf. The docstrings have more details.
.. code:: python
output = '/path/to/output.nc'
c2.to_hdf5(output, compression='gzip')
If you've passed xr.Dataset objects to Combine, it will be slower than passing the file as a path on disk. Only pass xr.Dataset objects to Combine if you don't want to write the intermediate file to disk before reading it into Combine.
The package also comes with a bonus function called **xr_to_hdf5**. It is a convenience function to convert a single xr.Dataset to an hdf5/netcdf4 file.
.. code:: python
from hdf5tools import xr_to_hdf5
xr_to_hdf5(xr_dataset, output, compression='gzip')
Raw data
{
"_id": null,
"home_page": null,
"name": "hdf5tools",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "tethys",
"author": null,
"author_email": "Mike Kittridge <mullenkamp1@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/0c/13/88810505a70d3ae1d36bdb1c46bebb01b912ada25c51040f4e9c46124e5d/hdf5tools-0.4.5.tar.gz",
"platform": null,
"description": "hdf5-tools\n==================================\n\nThis git repository contains a python package with an H5 class to load and combine one or more HDF5 data files (or xarray datasets) with optional filters. The class will then export the combined data to an HDF5 file, file object, or xr.Dataset. This class is designed to be fast and safe on memory. This means that files of any size can be combined and saved even on a PC with low memory (unlike xarray).\n\nInstallation\n------------\nUsing pip:\n\n.. code::\n\n pip install hdf5tools\n\n\nOr using conda/mamba from conda-forge:\n\n.. code::\n\n conda install -c conda-forge hdf5tools\n\n\nUsage\n-------\nCurrently, only the **Combine** class is recommended for other to use.\n\nFirst, initiate the class with one or many: paths to netcdf3, netcdf4, or hdf5 files; xr.Dataset objects (opened via xr.open_dataset); or h5py.File objects.\n\n.. code:: python\n\n from hdf5tools import Combine\n\n ###############################\n ### Parameters\n\n path1 = '/path/to/file1.nc'\n path2 = '/path/to/file2.nc'\n\n ##############################\n ### Combine files\n\n c1 = Combine([path1, path2])\n\n\n\nIf you want to do some kind of selection via the coordinates or only select some of the data variables/coordinates then use the **.sel** method (like in xarray). Be sure to read the docstrings for additional info about the input parameters.\n\n.. code:: python\n\n c2 = c1.sel({'time': slice('2001-01-01', '2020-01-01'), 'latitude': slice(0, 10)}, include_data_vars=['precipitation'])\n\n\nAnd finally, save the combined data to a single hdf5/netcdf4 file using the **.to_hdf5** method. The only additional parameters that are important include the output which should be a path or a io.Bytes object, and the compression parameter. If you plan on using this file outside of the python environment, use gzip for compression, otherwise use lzf. The docstrings have more details.\n\n\n.. code:: python\n\n output = '/path/to/output.nc'\n\n c2.to_hdf5(output, compression='gzip')\n\n\nIf you've passed xr.Dataset objects to Combine, it will be slower than passing the file as a path on disk. Only pass xr.Dataset objects to Combine if you don't want to write the intermediate file to disk before reading it into Combine.\n\nThe package also comes with a bonus function called **xr_to_hdf5**. It is a convenience function to convert a single xr.Dataset to an hdf5/netcdf4 file.\n\n.. code:: python\n\n from hdf5tools import xr_to_hdf5\n\n xr_to_hdf5(xr_dataset, output, compression='gzip')\n\n",
"bugtrack_url": null,
"license": null,
"summary": "Python hdf5 tools",
"version": "0.4.5",
"project_urls": {
"Homepage": "https://github.com/mullenkamp/hdf5tools"
},
"split_keywords": [
"tethys"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d9a1af39849ab1a748fc90a3e5c568aa776617dc7aa8006fe548046b06c863e3",
"md5": "d635c8896ff58d7941f585b1ea1b10b1",
"sha256": "3729c99948529e03ac1411827a52f811fd6cad8c8d2e498ec341ebfee14013fd"
},
"downloads": -1,
"filename": "hdf5tools-0.4.5-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "d635c8896ff58d7941f585b1ea1b10b1",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 33369,
"upload_time": "2024-12-15T01:55:35",
"upload_time_iso_8601": "2024-12-15T01:55:35.302593Z",
"url": "https://files.pythonhosted.org/packages/d9/a1/af39849ab1a748fc90a3e5c568aa776617dc7aa8006fe548046b06c863e3/hdf5tools-0.4.5-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "0c1388810505a70d3ae1d36bdb1c46bebb01b912ada25c51040f4e9c46124e5d",
"md5": "da15d15bb25986735c8de675719b4e4b",
"sha256": "7233ab22e7f07ddb95cd02f2b72acd61b17c05734b12e49a7dfb3b31c25f2cfe"
},
"downloads": -1,
"filename": "hdf5tools-0.4.5.tar.gz",
"has_sig": false,
"md5_digest": "da15d15bb25986735c8de675719b4e4b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 29551,
"upload_time": "2024-12-15T01:55:38",
"upload_time_iso_8601": "2024-12-15T01:55:38.218336Z",
"url": "https://files.pythonhosted.org/packages/0c/13/88810505a70d3ae1d36bdb1c46bebb01b912ada25c51040f4e9c46124e5d/hdf5tools-0.4.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-15 01:55:38",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "mullenkamp",
"github_project": "hdf5tools",
"github_not_found": true,
"lcname": "hdf5tools"
}
