| Name | intake-satpy JSON |
| Version |
0.1.0
JSON |
| download |
| home_page | |
| Summary | Intake drivers using Satpy to read and manipulate data |
| upload_time | 2022-12-21 18:20:52 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.8 |
| license | GPLv3 |
| keywords |
intake
pytroll
satpy
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Intake - Satpy Drivers
This package adds additional drivers for the
[Intake](https://intake.readthedocs.io/en/latest/) library using the
[Satpy](https://satpy.readthedocs.io/en/stable/) library to read data files.
This package also depends on
[intake-xarray](https://intake-xarray.readthedocs.io/en/latest/) to define
the Xarray container type (xarray `Dataset`) which these Satpy-based drivers
produce.
## Installation
To add this package to an existing `pip` based environment, run:
```bash
pip install intake-satpy
```
Or if you have a conda-based environment you can install it from the
conda-forge channel:
```bash
conda install -c conda-forge intake-satpy
```
## Usage
This package currently only supplies one intake driver named `satpy`.
As with any intake driver, the `satpy` driver can be used in a couple
different ways. A few examples are shown below.
### Inline Usage
Once the `intake-satpy` package is installed, you can use this driver by
calling `intake.open_satpy`. At the time of writing, it is best to provide
as much information to configure/control Satpy as you can by passing the
`scene_kwargs` and `load_kwargs`.
```
import intake
from glob import glob
data_source = intake.open_satpy(
glob("/data/satellite/abi/*.nc"),
scene_kwargs={"reader": "abi_l1b"},
load_kwargs={"wishlist": ["C01"]},
)
dataset = data_source.read_chunked()
```
The `read_chunked` method will return an xarray `Dataset` object that will
contain the products that Satpy was able to create. Data will be represented
as dask arrays underneath. The `data_source.to_dask()` method will also
produce this result. The `data_source.read()` method will return the same
xarray `Dataset` object but data will be loaded into memory as numpy arrays.
Care must be taken as the large satellite formats read by Satpy can quickly
fill up your system's memory if loaded in this way.
By default, if `wishlist` is not provided as a load keyword argument
(see above), then all available "reader" level products will be loaded. This
means those that can be read directly from the file and does not include
any Satpy "composites".
Also by default the loaded dataset is "resampled" using Satpy's "native"
resampler to the finest resolution of the loaded products. This allows
for all products to exist in a single xarray `Dataset` object. This behavior
can be customized by providing `resample_kwargs` to the source creation
(`open_satpy` call).
### Catalog Usage - Local
The `satpy` driver can also be used in a catalog definition. See the
[examples/local_abi_l1b.yaml](https://github.com/pytroll/intake-satpy/blob/main/examples/local_abi_l1b.yaml)
catalog definition file for an example. With a catalog like this you could then
do:
```python
import intake
cat = intake.open_catalog("examples/local_abi_l1b.yaml")
source = cat.abi_l1b(base_dir="/data/satellite/abi")
dataset = source.read_chunked()
```
A wishlist of products to load can be provided to the source when creating it:
```python
cat = intake.open_catalog("examples/local_abi_l1b.yaml")
source = cat.abi_l1b(base_dir="/data/satellite/abi", load_kwargs={"wishlist": ["C01"]})
dataset = source.read_chunked()
```
As with the inline usage, if `wishlist` is not provided then all reader-level
products will be loaded.
### Catalog Usage - S3
Some of Satpy's readers can also read data from remote storage like S3 buckets.
An example catalog is included in the `examples/` directory of the
`intake-satpy` repository.
Note that Satpy's performance for reading S3 files is currently very slow, but
is being worked on. It is likely not suitable for loading data outside of the
network where the S3 storage is (AWS in this example) until future updates to
Satpy and NetCDF are made.
```python
import intake
cat = intake.open_catalog("examples/aws_abi_l1b_20220101_18.yaml")
source = cat.abi_l1b()
dataset = source.read_chunked()
```
Raw data
{
"_id": null,
"home_page": "",
"name": "intake-satpy",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "intake,pytroll,satpy",
"author": "",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/c3/6d/c4cca00874621429c278baebd5b83dff07f5626e46d7923ac205bfd24dc9/intake_satpy-0.1.0.tar.gz",
"platform": null,
"description": "# Intake - Satpy Drivers\n\nThis package adds additional drivers for the\n[Intake](https://intake.readthedocs.io/en/latest/) library using the\n[Satpy](https://satpy.readthedocs.io/en/stable/) library to read data files.\nThis package also depends on\n[intake-xarray](https://intake-xarray.readthedocs.io/en/latest/) to define\nthe Xarray container type (xarray `Dataset`) which these Satpy-based drivers\nproduce.\n\n## Installation\n\nTo add this package to an existing `pip` based environment, run:\n\n```bash\npip install intake-satpy\n```\n\nOr if you have a conda-based environment you can install it from the\nconda-forge channel:\n\n```bash\nconda install -c conda-forge intake-satpy\n```\n\n## Usage\n\nThis package currently only supplies one intake driver named `satpy`.\nAs with any intake driver, the `satpy` driver can be used in a couple\ndifferent ways. A few examples are shown below.\n\n### Inline Usage\n\nOnce the `intake-satpy` package is installed, you can use this driver by\ncalling `intake.open_satpy`. At the time of writing, it is best to provide\nas much information to configure/control Satpy as you can by passing the\n`scene_kwargs` and `load_kwargs`.\n\n```\nimport intake\nfrom glob import glob\n\ndata_source = intake.open_satpy(\n glob(\"/data/satellite/abi/*.nc\"),\n scene_kwargs={\"reader\": \"abi_l1b\"},\n load_kwargs={\"wishlist\": [\"C01\"]},\n)\ndataset = data_source.read_chunked()\n```\n\nThe `read_chunked` method will return an xarray `Dataset` object that will\ncontain the products that Satpy was able to create. Data will be represented\nas dask arrays underneath. The `data_source.to_dask()` method will also\nproduce this result. The `data_source.read()` method will return the same\nxarray `Dataset` object but data will be loaded into memory as numpy arrays.\nCare must be taken as the large satellite formats read by Satpy can quickly\nfill up your system's memory if loaded in this way.\n\nBy default, if `wishlist` is not provided as a load keyword argument\n(see above), then all available \"reader\" level products will be loaded. This\nmeans those that can be read directly from the file and does not include\nany Satpy \"composites\".\n\nAlso by default the loaded dataset is \"resampled\" using Satpy's \"native\"\nresampler to the finest resolution of the loaded products. This allows\nfor all products to exist in a single xarray `Dataset` object. This behavior\ncan be customized by providing `resample_kwargs` to the source creation\n(`open_satpy` call).\n\n### Catalog Usage - Local\n\nThe `satpy` driver can also be used in a catalog definition. See the\n[examples/local_abi_l1b.yaml](https://github.com/pytroll/intake-satpy/blob/main/examples/local_abi_l1b.yaml)\ncatalog definition file for an example. With a catalog like this you could then\ndo:\n\n```python\nimport intake\n\ncat = intake.open_catalog(\"examples/local_abi_l1b.yaml\")\nsource = cat.abi_l1b(base_dir=\"/data/satellite/abi\")\ndataset = source.read_chunked()\n\n```\n\nA wishlist of products to load can be provided to the source when creating it:\n\n```python\ncat = intake.open_catalog(\"examples/local_abi_l1b.yaml\")\nsource = cat.abi_l1b(base_dir=\"/data/satellite/abi\", load_kwargs={\"wishlist\": [\"C01\"]})\ndataset = source.read_chunked()\n```\n\nAs with the inline usage, if `wishlist` is not provided then all reader-level\nproducts will be loaded.\n\n### Catalog Usage - S3\n\nSome of Satpy's readers can also read data from remote storage like S3 buckets.\nAn example catalog is included in the `examples/` directory of the\n`intake-satpy` repository.\n\nNote that Satpy's performance for reading S3 files is currently very slow, but\nis being worked on. It is likely not suitable for loading data outside of the\nnetwork where the S3 storage is (AWS in this example) until future updates to\nSatpy and NetCDF are made.\n\n```python\nimport intake\n\ncat = intake.open_catalog(\"examples/aws_abi_l1b_20220101_18.yaml\")\nsource = cat.abi_l1b()\ndataset = source.read_chunked()\n\n```\n",
"bugtrack_url": null,
"license": "GPLv3",
"summary": "Intake drivers using Satpy to read and manipulate data",
"version": "0.1.0",
"split_keywords": [
"intake",
"pytroll",
"satpy"
],
"urls": [
{
"comment_text": "",
"digests": {
"md5": "8475e9809e7526d4188a9e772c96c573",
"sha256": "6c1f37b35f9a75f789a5b661a7cfcdc41017c78f92f0b02b6de7f89c40f39f6f"
},
"downloads": -1,
"filename": "intake_satpy-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "8475e9809e7526d4188a9e772c96c573",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 20542,
"upload_time": "2022-12-21T18:20:52",
"upload_time_iso_8601": "2022-12-21T18:20:52.893812Z",
"url": "https://files.pythonhosted.org/packages/c3/6d/c4cca00874621429c278baebd5b83dff07f5626e46d7923ac205bfd24dc9/intake_satpy-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-12-21 18:20:52",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "intake-satpy"
}
