# STAC Catalogue Utilities
STAC Catalogue Utilities is a library that provides utility functions implemented in the Python 3 scripting
language that facilitate the generation of STAC files from existing files and folders.
This library was developed in the context of the [EOEPCA](https://eoepca.org/) project.
## Installation
### Install from PyPi (recommended)
```shell
pip install stac_cat_utils
```
### Install from source
```shell
git clone https://github.com/SpaceApplications/stac_cat_utils
cd stac_cat_utils
pip install .
```
## Design Notes
This Python3 library provides functionality to generate STAC (and optionally Datacube compatible) files. It can take existing files and folders as inputs. It can handle a variety of file formats. It does this by using the following 3rd party libraries, each used for their respective capabilities:
- [stactools](https://pypi.org/project/stactools/)
- [stactools-browse](https://pypi.org/project/stactools-browse/)
- [stactools-landsat](https://pypi.org/project/stactools-landsat/)
- [stactools-sentinel1](https://pypi.org/project/stactools-sentinel1/)
- [stactools-sentinel2](https://pypi.org/project/stactools-sentinel2/)
- [pystac](https://pypi.org/project/pystac/)
- [rio_stac](https://pypi.org/project/rio-stac/)
This library combines the functionality of these packages to provide a single command line utility to easily create STAC files. The library optionally provides more granular control over which files to include/exclude from the resulting STAC file using path and regex matching.
## Usage
### STAC Generator
The generation of the STAC files, for existing files and folders, is handled by the `StacCatalogGenerator` class:
```python
from stac_cat_utils.stac_generator import StacCatalogGenerator
stac_generator = StacCatalogGenerator()
```
Concrete generation of STAC files is handled by the `create` and `save` method of the `StacCatalogGenerator` generator:
1. `create`: Return an STAC STACCatalog object (pystac.Catalog augmented with additional features) for the given source path.
* `src_path`: (Required) Root path of the folder.
* `catalog_name`: (Optional) Name of the catalogue. Default: "Catalogue".
* `collection_paths`: (Optional) List of paths that must be considered as collections. Array of strings, globs and Path instances. Default: None.
* `item_paths`: (Optional) List of paths that must be considered as items. Array of strings, globs and Path instances. Default: None.
* `ignore_paths`: (Optional) List of paths to ignore. Array of strings, globs and Path instances. Default: None.
* `asset_href_prefix`: (Optional) prefix to append to all assets href. Default: '/'.
```python
from stac_cat_utils.stac_generator import StacCatalogGenerator
stac_generator = StacCatalogGenerator()
catalog = stac_generator.create('.')
```
2. `save`: Saves the generated STAC STACCatalog object to a destination path.
* `dest_path`: (Optional) Destination path where the STAC catalog is saved. Default: 'stac_<catalog_name>' .
* `asset_href_prefix`: (Optional) prefix to append to all assets href. Default: '/'.
```python
from stac_cat-utils.stac_generator import StacCatalogGenerator
stac_generator = StacCatalogGenerator()
catalog = stac_generator.create('.')
stac_generator.save()
```
### Datacube
The catalog and collection created during the generation process are augmented with methods to support the [Datacube Extension Specification
](https://github.com/stac-extensions/datacube).
The following methods are available for:
1. `STACCatalog`:
* `make_cube_compliant`: make all collection of the catalog datacube compliant if possible
```python
from stac_cat_utils.stac_generator import StacCatalogGenerator
stac_generator = StacCatalogGenerator()
catalog = stac_generator.create('.')
catalog.make_datacube_compliant()
```
2. `STACCollection`:
* `make_datacube_compliant`: make the collection datacube compliant if possible
* `add_horizontal_dimension`: add a [Horizontal Dimension](https://github.com/stac-extensions/datacube#horizontal-spatial-raster-dimension-object) to the collection
* `add_vertical_dimension`: add a [Vertical Dimension](https://github.com/stac-extensions/datacube#vertical-spatial-dimension-object) to the collection
* `add_temporal_dimension`: add a [Temporal Dimension](https://github.com/stac-extensions/datacube#temporal-dimension-object) to the collection
* `add_additional_dimension`: add a [Custom Dimension](https://github.com/stac-extensions/datacube#additional-dimension-object) to the collection
* `add_dimension_variable`: add a [Dimension Variable](https://github.com/stac-extensions/datacube#variable-object) to the collection
```python
import datetime
from stac_cat_utils.stac_generator import StacCatalogGenerator
stac_generator = StacCatalogGenerator()
catalog = stac_generator.create('.')
for collection in catalog.get_all_collections():
# Collection Dimension example
collection.make_datacube_compliant()
collection.add_horizontal_dimension('x_axis', axis='x', extent=[33, 36])
collection.add_vertical_dimension('z_axis', extent=[33, 36])
collection.add_temporal_dimension('time', extent=[datetime.datetime.now().isoformat(), (datetime.datetime.now().isoformat()])
collection.add_additional_dimension('extra', type='test', values=['ex1', 'ex2'])
collection.add_dimension_variable('a_variable', type='data', values=['test', 'test1'])
```
During the creation of a Datacube compliant STAC file, the library does the following:
Verify that, for each Collection in the Catalogue, all the Items share exactly the same properties except the time.
- All the Collection Items must have the same platform, sensor, mode, etc.
- All the Collection Items must have the same geometry and bbox
- All the Collection Items must have the same list of assets
## Examples
Python script showcasing the usage of the library are available in under the `examples` folder.
## Running the tests
This section details the procedure of running the included test cases.
### Setup
Create a virtual environment (Optional but recommended):
```bash
python3 -m venv venv
```
Activate the virtual environment:
```bash
source venv/bin/activate
```
Install the requirements:
```bash
pip install -r requirements.txt
```
Run the tests:
```bash
python -m unittest test/test_stac_generator.py
```
Raw data
{
"_id": null,
"home_page": "https://github.com/SpaceApplications/stac-cat-utils",
"name": "stac-cat-utils",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "STAC,pystac,STAC generation",
"author": "Space Applications Services",
"author_email": "sys_eos@spaceapplications.com",
"download_url": "https://files.pythonhosted.org/packages/7f/0d/b33b4dbbf1a689539b3a4ab4ccf5ec423f89222ee48981585ef588b7acc3/stac_cat_utils-0.1.0.tar.gz",
"platform": null,
"description": "# STAC Catalogue Utilities\n\nSTAC Catalogue Utilities is a library that provides utility functions implemented in the Python 3 scripting \nlanguage that facilitate the generation of STAC files from existing files and folders.\n\nThis library was developed in the context of the [EOEPCA](https://eoepca.org/) project.\n\n## Installation\n### Install from PyPi (recommended)\n\n```shell\npip install stac_cat_utils\n```\n\n### Install from source\n```shell\ngit clone https://github.com/SpaceApplications/stac_cat_utils\ncd stac_cat_utils\npip install .\n```\n\n## Design Notes\nThis Python3 library provides functionality to generate STAC (and optionally Datacube compatible) files. It can take existing files and folders as inputs. It can handle a variety of file formats. It does this by using the following 3rd party libraries, each used for their respective capabilities:\n- [stactools](https://pypi.org/project/stactools/)\n- [stactools-browse](https://pypi.org/project/stactools-browse/)\n- [stactools-landsat](https://pypi.org/project/stactools-landsat/)\n- [stactools-sentinel1](https://pypi.org/project/stactools-sentinel1/)\n- [stactools-sentinel2](https://pypi.org/project/stactools-sentinel2/)\n- [pystac](https://pypi.org/project/pystac/)\n- [rio_stac](https://pypi.org/project/rio-stac/)\n\nThis library combines the functionality of these packages to provide a single command line utility to easily create STAC files. The library optionally provides more granular control over which files to include/exclude from the resulting STAC file using path and regex matching. \n\n## Usage\n\n### STAC Generator\n\nThe generation of the STAC files, for existing files and folders, is handled by the `StacCatalogGenerator` class:\n```python\nfrom stac_cat_utils.stac_generator import StacCatalogGenerator\nstac_generator = StacCatalogGenerator()\n```\n\nConcrete generation of STAC files is handled by the `create` and `save` method of the `StacCatalogGenerator` generator:\n\n1. `create`: Return an STAC STACCatalog object (pystac.Catalog augmented with additional features) for the given source path.\n * `src_path`: (Required) Root path of the folder.\n * `catalog_name`: (Optional) Name of the catalogue. Default: \"Catalogue\".\n * `collection_paths`: (Optional) List of paths that must be considered as collections. Array of strings, globs and Path instances. Default: None.\n * `item_paths`: (Optional) List of paths that must be considered as items. Array of strings, globs and Path instances. Default: None.\n * `ignore_paths`: (Optional) List of paths to ignore. Array of strings, globs and Path instances. Default: None.\n * `asset_href_prefix`: (Optional) prefix to append to all assets href. Default: '/'.\n ```python\n from stac_cat_utils.stac_generator import StacCatalogGenerator\n stac_generator = StacCatalogGenerator()\n catalog = stac_generator.create('.')\n ```\n\n2. `save`: Saves the generated STAC STACCatalog object to a destination path.\n * `dest_path`: (Optional) Destination path where the STAC catalog is saved. Default: 'stac_<catalog_name>' .\n * `asset_href_prefix`: (Optional) prefix to append to all assets href. Default: '/'.\n ```python\n from stac_cat-utils.stac_generator import StacCatalogGenerator\n stac_generator = StacCatalogGenerator()\n catalog = stac_generator.create('.')\n stac_generator.save()\n ```\n### Datacube\nThe catalog and collection created during the generation process are augmented with methods to support the [Datacube Extension Specification\n](https://github.com/stac-extensions/datacube).\n\nThe following methods are available for:\n1. `STACCatalog`:\n * `make_cube_compliant`: make all collection of the catalog datacube compliant if possible\n ```python\n from stac_cat_utils.stac_generator import StacCatalogGenerator\n stac_generator = StacCatalogGenerator()\n catalog = stac_generator.create('.')\n catalog.make_datacube_compliant()\n ```\n\n2. `STACCollection`:\n * `make_datacube_compliant`: make the collection datacube compliant if possible\n * `add_horizontal_dimension`: add a [Horizontal Dimension](https://github.com/stac-extensions/datacube#horizontal-spatial-raster-dimension-object) to the collection\n * `add_vertical_dimension`: add a [Vertical Dimension](https://github.com/stac-extensions/datacube#vertical-spatial-dimension-object) to the collection\n * `add_temporal_dimension`: add a [Temporal Dimension](https://github.com/stac-extensions/datacube#temporal-dimension-object) to the collection\n * `add_additional_dimension`: add a [Custom Dimension](https://github.com/stac-extensions/datacube#additional-dimension-object) to the collection\n * `add_dimension_variable`: add a [Dimension Variable](https://github.com/stac-extensions/datacube#variable-object) to the collection\n ```python\n import datetime\n from stac_cat_utils.stac_generator import StacCatalogGenerator\n\n stac_generator = StacCatalogGenerator()\n catalog = stac_generator.create('.')\n \n for collection in catalog.get_all_collections():\n # Collection Dimension example\n collection.make_datacube_compliant()\n collection.add_horizontal_dimension('x_axis', axis='x', extent=[33, 36])\n collection.add_vertical_dimension('z_axis', extent=[33, 36])\n collection.add_temporal_dimension('time', extent=[datetime.datetime.now().isoformat(), (datetime.datetime.now().isoformat()])\n collection.add_additional_dimension('extra', type='test', values=['ex1', 'ex2'])\n collection.add_dimension_variable('a_variable', type='data', values=['test', 'test1'])\n ```\n\nDuring the creation of a Datacube compliant STAC file, the library does the following:\n\nVerify that, for each Collection in the Catalogue, all the Items share exactly the same properties except the time.\n - All the Collection Items must have the same platform, sensor, mode, etc.\n - All the Collection Items must have the same geometry and bbox\n - All the Collection Items must have the same list of assets \n\n## Examples\nPython script showcasing the usage of the library are available in under the `examples` folder.\n\n## Running the tests\nThis section details the procedure of running the included test cases.\n\n### Setup\nCreate a virtual environment (Optional but recommended):\n```bash\npython3 -m venv venv\n```\nActivate the virtual environment:\n```bash\nsource venv/bin/activate\n```\nInstall the requirements:\n```bash\npip install -r requirements.txt\n```\nRun the tests:\n```bash\npython -m unittest test/test_stac_generator.py\n```\n",
"bugtrack_url": null,
"license": "BSD",
"summary": "Package of utility functions facilitating generation of STAC files from existing files and folders.",
"version": "0.1.0",
"project_urls": {
"Homepage": "https://github.com/SpaceApplications/stac-cat-utils"
},
"split_keywords": [
"stac",
"pystac",
"stac generation"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7f0db33b4dbbf1a689539b3a4ab4ccf5ec423f89222ee48981585ef588b7acc3",
"md5": "993892c26f2e5c303a7843d1b5d9cda0",
"sha256": "9a2b773016d52ce0bc8a217676834a19f1f14a97f3e65c1be39eda30d51c70ba"
},
"downloads": -1,
"filename": "stac_cat_utils-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "993892c26f2e5c303a7843d1b5d9cda0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 23433,
"upload_time": "2023-07-17T16:34:56",
"upload_time_iso_8601": "2023-07-17T16:34:56.627520Z",
"url": "https://files.pythonhosted.org/packages/7f/0d/b33b4dbbf1a689539b3a4ab4ccf5ec423f89222ee48981585ef588b7acc3/stac_cat_utils-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-17 16:34:56",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "SpaceApplications",
"github_project": "stac-cat-utils",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "stac-cat-utils"
}
JSON
