| Name | stac-repository JSON |
| Version |
0.0.4
JSON |
| download |
| home_page | None |
| Summary | A (git-)versionned STAC catalog storage and management system |
| upload_time | 2025-07-18 14:33:25 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.8 |
| license | OPEN LICENCE 2.0/LICENCE OUVERTE 2.0 "Reuse" of the "Information" covered by this licence The "Grantor" grants the "Reuser" the free, non-exclusive right to "Reuse" the "Information" subject of this licence, for commercial or non-commercial purposes, worldwide and for an unlimited period, in accordance with the conditions stated below. The "Reuser" is free to reuse the "Information": - To reproduce it, copy it. - To adapt, modify, retrieve and transform it in order to create "derived information", products and services. - To share, disseminate, redistribute, publish and transmit it. - To exploit it for commercial purposes, e.g., by combining it with other information, or by including it in his/her own product or application. Subject to: - An acknowledgement of the authorship of the "Information": its source (at least, the name of the "Grantor") and the date of the most recent update of the reused "Information". Specifically, the "Reuser" may satisfy this condition by pointing, via a hypertext link, to the source of the "Information" and so supplying an actual acknowledgement of its authorship. For example: "Ministry of xxx—Original data downloaded from 'http://www.data.gouv.fr/fr/datasets/xxx/', updated on 14 February 2017". This acknowledgement of authorship does not confer any official status on the "Reuse" of the "Information", and must not suggest any sort of recognition or endorsement on the part of the "Grantor", or any other public entity, of the "Reuser" or of their "Reuse". Personal data The "Information" made available may contain "Personal data" that may be subject to "Reuse". If this is the case, the "Grantor" informs the "Reuser" about its existence. The "Information" may be freely reused, within the rights granted by this licence, subject to compliance with the legal framework relating to personal data protection. Intellectual property rights It is guaranteed to The "Reuser" that potential "Intellectual property rights" held by third parties or by the "Grantor" on "Information" do not interfere with the rights granted by this licence. When the "Grantor" holds transferable "Intellectual property rights" on the "Information", he/she assigns these to the "Reuser" on a non-exclusive basis, free of charge, worldwide, for the entire duration of the "Intellectual property rights", and the "Reuser" is free to use the "Information" for any purpose that complies with the rights and conditions defined in this licence. Liability The "Information" is made available as it is produced or received by the "Grantor", without any other express or tacit guarantee than those set out in this licence. The "Grantor" does not guarantee the absence of errors or inaccuracies in the "Information", nor a continuous supply of the "Information". He/she cannot be held responsible for any loss, prejudice or damage of any kind caused to third parties as a result of the "Reuse". The "Reuser" is solely responsible for the "Reuse" of the "Information". This "Reuse" must not mislead third parties as to the contents of the "Information", its source or its date of update. Applicable legislation This licence is governed by French law. Compatibility of this licence This licence has been designed to be compatible with any free licence that at least requires an acknowledgement of authorship, and specifically with the previous version of this licence as well as with the following licences: United Kingdom's "Open Government Licence" (OGL), Creative Commons' "Creative Commons Attribution" (CC-BY) and Open Knowledge Foundation's "Open Data Commons Attribution" (ODC-BY). Definitions Within the meaning of this licence, are to be considered as: - The "Grantor": any person granting the right to "Reuse" "Information" under the rights and conditions set out in this licence. - The "Information": - any public information contained in documents disclosed or published by any administration referred to in the first paragraph of Article L. 300-2 of the "Code des relations entre le public et l'administration (CRPA)", - any information made available by any person under the terms and conditions of this licence. - The "Reuse": the use of the "Information" for other purposes than those for which it was produced or received. - The "Reuser": any person reusing the "Information" in accordance with the conditions of this licence. - "Personal data": any information relating to an identified or identifiable natural person who may be identified directly or indirectly. Its "Reuse" is conditional on the respect of the existing legal framework. - "Derived information": any new data or information created directly from the "Information" or from a combination of the "Information" and other data or information not subject to this licence. - "Intellectual property rights": all rights identified as such under the "Code de la propriété intellectuelle" (including copyright, rights related to copyright, sui generis rights of database producers, etc.). About this licence This licence is intended to be used by administrations for the reuse of their public information. It can also be used by any individual wishing to supply "Information" under the conditions defined in this licence. France has a comprehensive legal framework aiming at the spontaneous dissemination by the administrations of their public information in order to ensure the widest possible reuse of this information. The right to "Reuse" the administrations' "Information" is governed by the "Code des relations entre le public et l'administration (CRPA)". This licence facilitates the unrestricted and free of charge reuse of public information and is one of the licences which can be used by the administration pursuant to the decree issued under article L. 323-2 of the CRPA. Under the Prime Minister's authority, the Etalab mission is mandated to open up the maximum amount of data held by State administrations and public institutions. Etalab has drawn up the Open Licence to facilitate the unrestricted and free of charge reuse of public information, as defined by article L. 321-1 of the CRPA. This licence is version 2.0 of the Open Licence. Etalab reserves the right to propose new versions of the Open Licence. Nevertheless, "Reusers" may continue to reuse information obtained under this licence should they so wish. |
| keywords |
pystac
stac
database
git
repository
storage
store
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# STAC Repository
<p align="center">
<img src="https://stacspec.org/public/images-original/STAC-01.png" style="vertical-align: middle; max-width: 400px; max-height: 100px; margin: 20px;" />
<img src="https://git-scm.com/images/logo@2x.png" alt="FastAPI" style="vertical-align: middle; max-width: 100px; max-height: 100px; margin: 20px;" />
</p>
A (git-)versionned [STAC](https://stacspec.org/en) catalog storage and management system.
Project in late stage development phase.
## Introduction and Features
`stac-repository` is a storage system and command-line interface for managing STAC catalogs. It implements advanced features necessary to build and maintain a complex STAC catalog :
- **Automated ingestions (Non-STAC)**
Automated ingestion of non-STAC products via custom `stac-processors` modules. These are designed for ease of implementation via the `StacProcessor` Protocol. To view installed processors, run `stac-repository show-processors`.
- **Automated Ingestions (STAC)**
Automated ingestion of STAC items and collections using a built-in `stac-processor`.
- **Backend Support**
Support for multiple storage backends, including built-in Git+LFS `"git"` and local filesystem `"file"`. To view installed backends, run `stac-repository show-backends`. The architecture is also designed to facilitate the development of additional backends (e.g. FTP, NoSQL databases).
- **Transactional Operations**
Transactional ingestions, updates, and deletions to ensure catalog integrity and atomicity
- **Immutable History**
Immutable history of all transactions (note: this feature is not supported by the local filesystem backend).
- **Backup and Rollback**
Backup and rollback of the catalog at any point in history.
- **Export**
Export command to extract the catalog to the local filesystem independently of the underlying storage backend.
These capabilities make stac-repository a powerful tool for robust STAC catalog management.
## Installation
`stac-repository` is available directly on [pypi](https://pypi.org/project/stac-repository/).
```bach
pip install stac-repository
```
This installs two commands : `stac-repository`, and `stac-processor` to try out a processor without ingesting.
## Usage
```bash
stac-repository --help
```
```bash
Usage: stac_repository_cli [OPTIONS] COMMAND [ARGS]...
🌍🛰️ STAC Repository
The interface to manage STAC catalogs.
╭─ Options ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --repository TEXT Repository URI - interpreted by the chosen backend. [default: None] │
│ --backend TEXT Backend. [default: file] │
│ --install-completion Install completion for the current shell. │
│ --show-completion Show completion for the current shell, to copy it or customize the installation. │
│ --help Show this message and exit. │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ version Shows stac-repository version number. │
│ show-backends Shows installed stac-repository backends. │
│ show-processors Shows installed stac-repository processors. │
│ init Initializes the repository. │
│ config Get or set the repository configuration options - interpreted by the chosen backend. │
│ ingest Ingests some products from various sources (eventually using an installed processor). │
│ prune Removes some products from the catalog. │
│ history Logs the catalog history. │
│ rollback Rollbacks the catalog to a previous commit. Support depends on the chosen backend. │
│ export Exports the catalog. If a commit ref is specified, exports the catalog as it was at that point in time. │
│ backup Backups the repository. If a commit ref is specified, backups the repository only up to this point in time. │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
```
Let's initialize a new repository (using the default `"file"` backend - implicit `--backend file`)
```bash
stac-repository --repository test_repository init
```
Since we didn't specify `--root-catalog` we are prompted for some basic information :
```
Initialize from an existing root catalog file ? (Leave blank to use the interactive initializer):
id (root): test_catalog
title: A Simple Demo Catalog
description: This is a simple demo catalog.
license (proprietary):
{
'id': 'test_catalog',
'description': 'This is a simple demo catalog.',
'stac_version': '1.0.0',
'links': [],
'title': 'A Simple Demo Catalog',
'type': 'Catalog',
'license': 'proprietary'
}
Use as root catalog ? [y/n] (n): y
```
The newly created catalog :
```
test_repository/
test_repository/catalog.json
```
Let's ingest some STAC product using the default processor (implicit `--processor stac`)
```bash
stac-repository --repository test_repository ingest ~/test_catalogs/thermavolc/
```
```
• ~/test_catalogs/thermavolc/ : Discovered products ~/test_catalogs/thermavolc/collection.json
• ~/test_catalogs/thermavolc/collection.json : Cataloged
```
Our catalog now looks like this :
```
test_repository/
test_repository/orthophotos-Summit-20221116
test_repository/orthophotos-Summit-20221116/collection.json
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Visible
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Visible/20221116_Summit_Visible_orthophoto.png
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Visible/20221116_Summit_Visible_orthophoto.tif.aux.xml
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Visible/20221116_Summit_Visible_orthophoto.tif
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Visible/orthophoto-Summit-20221116-Visible.json
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-DTM
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-DTM/20221116_Summit_DTM.tif.aux.xml
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-DTM/orthophoto-Summit-20221116-DTM.json
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-DTM/20221116_Summit_DTM.png
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-DTM/20221116_Summit_DTM.tif
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Thermal
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Thermal/20221116_Summit_Thermal_orthophoto.png
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Thermal/20221116_Summit_Thermal_orthophoto.tif
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Thermal/orthophoto-Summit-20221116-Thermal.json
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Thermal/20221116_Summit_Thermal_orthophoto.tif.aux.xml
test_repository/catalog.json
```
Finally let's delete some product
```bash
stac-repository --repository test_repository prune orthophoto-Summit-20221116-Visible
```
```
• orthophoto-Summit-20221116-Visible : Uncataloged
```
Finally our demo catalog looks like this :
```
test_repository/
test_repository/orthophotos-Summit-20221116
test_repository/orthophotos-Summit-20221116/collection.json
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-DTM
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-DTM/20221116_Summit_DTM.tif.aux.xml
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-DTM/orthophoto-Summit-20221116-DTM.json
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-DTM/20221116_Summit_DTM.png
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-DTM/20221116_Summit_DTM.tif
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Thermal
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Thermal/20221116_Summit_Thermal_orthophoto.png
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Thermal/20221116_Summit_Thermal_orthophoto.tif
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Thermal/orthophoto-Summit-20221116-Thermal.json
test_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Thermal/20221116_Summit_Thermal_orthophoto.tif.aux.xml
test_repository/catalog.json
```
## Motivation - _Why this Project ?_
**Full motivation is detailed in [`motivation.md`](https://github.com/fntb/stac-repository/blob/main/docs/motivation.md).**
Presenting scientific data products as STAC catalogs is primarily motivated by the objective of achieving data [FAIR](https://en.wikipedia.org/wiki/FAIR_data)-ness (Findable, Accessible, Interoperable, Reusable). The mature [STAC ecosystem](https://stacindex.org/ecosystem) makes building such a catalog relatively straightforward for simple cases (e.g. [pystac | Creating a Landsat 8 STAC](https://pystac.readthedocs.io/en/stable/tutorials.html#creating-a-landsat-8-stac)).
However, building and maintaining a complex STAC catalog - one subject to incremental changes over an extended period and encompassing diverse product types (e.g., satellite scenes, InSAR interferograms, InSAR time series) - introduces significant challenges. Effective maintenance necessitates capabilities for data rollback, backup, and exploring historical changes. And routine data ingestion requires automation of product conversion and ingestion, which itself requires transactional operations to ensure data integrity.
It is precisely to address these complex catalog management challenges that `stac-repository` was developed. It provides a robust storage system and CLI that integrates with and abstracts away the complexities of the underlying chosen backend (like Git+LFS). This approach allows `stac-repository` to offer transactional integrity, immutable history, backup/rollback capabilities, by treating the STAC catalog as a versioned data product, without requiring users to directly interact with the underlying backend.
While `stac-repository` greatly simplifies complex STAC catalog management, the underlying architecture introduces limitations. The Git+LFS backend, for instance, provides strong versioning capabilities but introduces a dependency on Git and Git LFS, which may require some foundational understanding for advanced operations or troubleshooting. For extremely large catalogs with millions of items or very high update frequencies, performance characteristics of the current backends will not be enough. While the local filesystem backend simplifies setup, it foregoes the immutable history provided by Git-based backends.
## The Processor Protocol
A processor is a python module implementing the processor protocol described [processor.py](https://github.com/fntb/stac-repository/blob/main/stac_repository/processor.py).
An example can be found in [`stac-processor.py`](https://github.com/fntb/stac-repository/blob/main/stac_repository/stac_processor.py)
## Source & Contributing
```bash
just --list
```
See [the Justfile](https://github.com/fntb/stac-repository/blob/main/justfile).
## History
**stac-repository** is being actively developped at the [OPGC](https://opgc.uca.fr/) an observatory for the sciences of the universe (OSU) belonging to the [CNRS](https://www.cnrs.fr/en) and the [UCA](https://www.uca.fr/) by its main author Pierre Fontbonne [@fntb](https://github.com/fntb).
## License
[OPEN LICENCE 2.0](https://github.com/fntb/stac-repository/blob/main/LICENCE.txt)
Raw data
{
"_id": null,
"home_page": null,
"name": "stac-repository",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "Pierre Fontbonne <pierre.fontbonne@uca.fr>, Yannick Gu\u00e9henneux <yannick.guehenneux@uca.fr>",
"keywords": "pystac, stac, database, git, repository, storage, store",
"author": null,
"author_email": "Pierre Fontbonne <pierre.fontbonne@uca.fr>",
"download_url": "https://files.pythonhosted.org/packages/d6/c3/7f2eb90e38059df98601f44b093b8a6d5184579754965e0298466d78cbd4/stac_repository-0.0.4.tar.gz",
"platform": null,
"description": "# STAC Repository\n\n<p align=\"center\">\n <img src=\"https://stacspec.org/public/images-original/STAC-01.png\" style=\"vertical-align: middle; max-width: 400px; max-height: 100px; margin: 20px;\" />\n <img src=\"https://git-scm.com/images/logo@2x.png\" alt=\"FastAPI\" style=\"vertical-align: middle; max-width: 100px; max-height: 100px; margin: 20px;\" />\n</p>\n\nA (git-)versionned [STAC](https://stacspec.org/en) catalog storage and management system.\n\nProject in late stage development phase.\n\n## Introduction and Features\n\n`stac-repository` is a storage system and command-line interface for managing STAC catalogs. It implements advanced features necessary to build and maintain a complex STAC catalog :\n\n- **Automated ingestions (Non-STAC)**\n\n Automated ingestion of non-STAC products via custom `stac-processors` modules. These are designed for ease of implementation via the `StacProcessor` Protocol. To view installed processors, run `stac-repository show-processors`.\n\n- **Automated Ingestions (STAC)**\n\n Automated ingestion of STAC items and collections using a built-in `stac-processor`.\n\n- **Backend Support**\n\n Support for multiple storage backends, including built-in Git+LFS `\"git\"` and local filesystem `\"file\"`. To view installed backends, run `stac-repository show-backends`. The architecture is also designed to facilitate the development of additional backends (e.g. FTP, NoSQL databases).\n\n- **Transactional Operations**\n\n Transactional ingestions, updates, and deletions to ensure catalog integrity and atomicity\n\n- **Immutable History**\n\n Immutable history of all transactions (note: this feature is not supported by the local filesystem backend).\n\n- **Backup and Rollback**\n\n Backup and rollback of the catalog at any point in history.\n\n- **Export**\n\n Export command to extract the catalog to the local filesystem independently of the underlying storage backend.\n\nThese capabilities make stac-repository a powerful tool for robust STAC catalog management.\n\n## Installation\n\n`stac-repository` is available directly on [pypi](https://pypi.org/project/stac-repository/).\n\n```bach\npip install stac-repository\n```\n\nThis installs two commands : `stac-repository`, and `stac-processor` to try out a processor without ingesting.\n\n## Usage\n\n```bash\nstac-repository --help\n```\n\n```bash\nUsage: stac_repository_cli [OPTIONS] COMMAND [ARGS]...\n\n \ud83c\udf0d\ud83d\udef0\ufe0f STAC Repository\n\n The interface to manage STAC catalogs.\n\n\u256d\u2500 Options \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256e\n\u2502 --repository TEXT Repository URI - interpreted by the chosen backend. [default: None] \u2502\n\u2502 --backend TEXT Backend. [default: file] \u2502\n\u2502 --install-completion Install completion for the current shell. \u2502\n\u2502 --show-completion Show completion for the current shell, to copy it or customize the installation. \u2502\n\u2502 --help Show this message and exit. \u2502\n\u2570\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256f\n\u256d\u2500 Commands \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256e\n\u2502 version Shows stac-repository version number. \u2502\n\u2502 show-backends Shows installed stac-repository backends. \u2502\n\u2502 show-processors Shows installed stac-repository processors. \u2502\n\u2502 init Initializes the repository. \u2502\n\u2502 config Get or set the repository configuration options - interpreted by the chosen backend. \u2502\n\u2502 ingest Ingests some products from various sources (eventually using an installed processor). \u2502\n\u2502 prune Removes some products from the catalog. \u2502\n\u2502 history Logs the catalog history. \u2502\n\u2502 rollback Rollbacks the catalog to a previous commit. Support depends on the chosen backend. \u2502\n\u2502 export Exports the catalog. If a commit ref is specified, exports the catalog as it was at that point in time. \u2502\n\u2502 backup Backups the repository. If a commit ref is specified, backups the repository only up to this point in time. \u2502\n\u2570\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256f\n```\n\nLet's initialize a new repository (using the default `\"file\"` backend - implicit `--backend file`)\n\n```bash\nstac-repository --repository test_repository init\n```\n\nSince we didn't specify `--root-catalog` we are prompted for some basic information :\n\n```\nInitialize from an existing root catalog file ? (Leave blank to use the interactive initializer):\nid (root): test_catalog\ntitle: A Simple Demo Catalog\ndescription: This is a simple demo catalog.\nlicense (proprietary):\n{\n 'id': 'test_catalog',\n 'description': 'This is a simple demo catalog.',\n 'stac_version': '1.0.0',\n 'links': [],\n 'title': 'A Simple Demo Catalog',\n 'type': 'Catalog',\n 'license': 'proprietary'\n}\nUse as root catalog ? [y/n] (n): y\n```\n\nThe newly created catalog :\n\n```\ntest_repository/\ntest_repository/catalog.json\n```\n\nLet's ingest some STAC product using the default processor (implicit `--processor stac`)\n\n```bash\nstac-repository --repository test_repository ingest ~/test_catalogs/thermavolc/\n```\n\n```\n \u2022 ~/test_catalogs/thermavolc/ : Discovered products ~/test_catalogs/thermavolc/collection.json\n \u2022 ~/test_catalogs/thermavolc/collection.json : Cataloged\n```\n\nOur catalog now looks like this :\n\n```\ntest_repository/\ntest_repository/orthophotos-Summit-20221116\ntest_repository/orthophotos-Summit-20221116/collection.json\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Visible\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Visible/20221116_Summit_Visible_orthophoto.png\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Visible/20221116_Summit_Visible_orthophoto.tif.aux.xml\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Visible/20221116_Summit_Visible_orthophoto.tif\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Visible/orthophoto-Summit-20221116-Visible.json\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-DTM\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-DTM/20221116_Summit_DTM.tif.aux.xml\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-DTM/orthophoto-Summit-20221116-DTM.json\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-DTM/20221116_Summit_DTM.png\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-DTM/20221116_Summit_DTM.tif\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Thermal\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Thermal/20221116_Summit_Thermal_orthophoto.png\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Thermal/20221116_Summit_Thermal_orthophoto.tif\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Thermal/orthophoto-Summit-20221116-Thermal.json\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Thermal/20221116_Summit_Thermal_orthophoto.tif.aux.xml\ntest_repository/catalog.json\n```\n\nFinally let's delete some product\n\n```bash\nstac-repository --repository test_repository prune orthophoto-Summit-20221116-Visible\n```\n\n```\n \u2022 orthophoto-Summit-20221116-Visible : Uncataloged\n```\n\nFinally our demo catalog looks like this :\n\n```\ntest_repository/\ntest_repository/orthophotos-Summit-20221116\ntest_repository/orthophotos-Summit-20221116/collection.json\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-DTM\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-DTM/20221116_Summit_DTM.tif.aux.xml\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-DTM/orthophoto-Summit-20221116-DTM.json\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-DTM/20221116_Summit_DTM.png\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-DTM/20221116_Summit_DTM.tif\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Thermal\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Thermal/20221116_Summit_Thermal_orthophoto.png\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Thermal/20221116_Summit_Thermal_orthophoto.tif\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Thermal/orthophoto-Summit-20221116-Thermal.json\ntest_repository/orthophotos-Summit-20221116/orthophoto-Summit-20221116-Thermal/20221116_Summit_Thermal_orthophoto.tif.aux.xml\ntest_repository/catalog.json\n```\n\n## Motivation - _Why this Project ?_\n\n**Full motivation is detailed in [`motivation.md`](https://github.com/fntb/stac-repository/blob/main/docs/motivation.md).**\n\nPresenting scientific data products as STAC catalogs is primarily motivated by the objective of achieving data [FAIR](https://en.wikipedia.org/wiki/FAIR_data)-ness (Findable, Accessible, Interoperable, Reusable). The mature [STAC ecosystem](https://stacindex.org/ecosystem) makes building such a catalog relatively straightforward for simple cases (e.g. [pystac | Creating a Landsat 8 STAC](https://pystac.readthedocs.io/en/stable/tutorials.html#creating-a-landsat-8-stac)).\n\nHowever, building and maintaining a complex STAC catalog - one subject to incremental changes over an extended period and encompassing diverse product types (e.g., satellite scenes, InSAR interferograms, InSAR time series) - introduces significant challenges. Effective maintenance necessitates capabilities for data rollback, backup, and exploring historical changes. And routine data ingestion requires automation of product conversion and ingestion, which itself requires transactional operations to ensure data integrity.\n\nIt is precisely to address these complex catalog management challenges that `stac-repository` was developed. It provides a robust storage system and CLI that integrates with and abstracts away the complexities of the underlying chosen backend (like Git+LFS). This approach allows `stac-repository` to offer transactional integrity, immutable history, backup/rollback capabilities, by treating the STAC catalog as a versioned data product, without requiring users to directly interact with the underlying backend.\n\nWhile `stac-repository` greatly simplifies complex STAC catalog management, the underlying architecture introduces limitations. The Git+LFS backend, for instance, provides strong versioning capabilities but introduces a dependency on Git and Git LFS, which may require some foundational understanding for advanced operations or troubleshooting. For extremely large catalogs with millions of items or very high update frequencies, performance characteristics of the current backends will not be enough. While the local filesystem backend simplifies setup, it foregoes the immutable history provided by Git-based backends.\n\n## The Processor Protocol\n\nA processor is a python module implementing the processor protocol described [processor.py](https://github.com/fntb/stac-repository/blob/main/stac_repository/processor.py).\n\nAn example can be found in [`stac-processor.py`](https://github.com/fntb/stac-repository/blob/main/stac_repository/stac_processor.py)\n\n## Source & Contributing\n\n```bash\njust --list\n```\n\nSee [the Justfile](https://github.com/fntb/stac-repository/blob/main/justfile).\n\n## History\n\n**stac-repository** is being actively developped at the [OPGC](https://opgc.uca.fr/) an observatory for the sciences of the universe (OSU) belonging to the [CNRS](https://www.cnrs.fr/en) and the [UCA](https://www.uca.fr/) by its main author Pierre Fontbonne [@fntb](https://github.com/fntb).\n\n## License\n\n[OPEN LICENCE 2.0](https://github.com/fntb/stac-repository/blob/main/LICENCE.txt)\n",
"bugtrack_url": null,
"license": "OPEN LICENCE 2.0/LICENCE OUVERTE 2.0 \"Reuse\" of the \"Information\" covered by this licence The \"Grantor\" grants the \"Reuser\" the free, non-exclusive right to \"Reuse\" the \"Information\" subject of this licence, for commercial or non-commercial purposes, worldwide and for an unlimited period, in accordance with the conditions stated below. The \"Reuser\" is free to reuse the \"Information\": - To reproduce it, copy it. - To adapt, modify, retrieve and transform it in order to create \"derived information\", products and services. - To share, disseminate, redistribute, publish and transmit it. - To exploit it for commercial purposes, e.g., by combining it with other information, or by including it in his/her own product or application. Subject to: - An acknowledgement of the authorship of the \"Information\": its source (at least, the name of the \"Grantor\") and the date of the most recent update of the reused \"Information\". Specifically, the \"Reuser\" may satisfy this condition by pointing, via a hypertext link, to the source of the \"Information\" and so supplying an actual acknowledgement of its authorship. For example: \"Ministry of xxx\u2014Original data downloaded from 'http://www.data.gouv.fr/fr/datasets/xxx/', updated on 14 February 2017\". This acknowledgement of authorship does not confer any official status on the \"Reuse\" of the \"Information\", and must not suggest any sort of recognition or endorsement on the part of the \"Grantor\", or any other public entity, of the \"Reuser\" or of their \"Reuse\". Personal data The \"Information\" made available may contain \"Personal data\" that may be subject to \"Reuse\". If this is the case, the \"Grantor\" informs the \"Reuser\" about its existence. The \"Information\" may be freely reused, within the rights granted by this licence, subject to compliance with the legal framework relating to personal data protection. Intellectual property rights It is guaranteed to The \"Reuser\" that potential \"Intellectual property rights\" held by third parties or by the \"Grantor\" on \"Information\" do not interfere with the rights granted by this licence. When the \"Grantor\" holds transferable \"Intellectual property rights\" on the \"Information\", he/she assigns these to the \"Reuser\" on a non-exclusive basis, free of charge, worldwide, for the entire duration of the \"Intellectual property rights\", and the \"Reuser\" is free to use the \"Information\" for any purpose that complies with the rights and conditions defined in this licence. Liability The \"Information\" is made available as it is produced or received by the \"Grantor\", without any other express or tacit guarantee than those set out in this licence. The \"Grantor\" does not guarantee the absence of errors or inaccuracies in the \"Information\", nor a continuous supply of the \"Information\". He/she cannot be held responsible for any loss, prejudice or damage of any kind caused to third parties as a result of the \"Reuse\". The \"Reuser\" is solely responsible for the \"Reuse\" of the \"Information\". This \"Reuse\" must not mislead third parties as to the contents of the \"Information\", its source or its date of update. Applicable legislation This licence is governed by French law. Compatibility of this licence This licence has been designed to be compatible with any free licence that at least requires an acknowledgement of authorship, and specifically with the previous version of this licence as well as with the following licences: United Kingdom's \"Open Government Licence\" (OGL), Creative Commons' \"Creative Commons Attribution\" (CC-BY) and Open Knowledge Foundation's \"Open Data Commons Attribution\" (ODC-BY). Definitions Within the meaning of this licence, are to be considered as: - The \"Grantor\": any person granting the right to \"Reuse\" \"Information\" under the rights and conditions set out in this licence. - The \"Information\": - any public information contained in documents disclosed or published by any administration referred to in the first paragraph of Article L. 300-2 of the \"Code des relations entre le public et l'administration (CRPA)\", - any information made available by any person under the terms and conditions of this licence. - The \"Reuse\": the use of the \"Information\" for other purposes than those for which it was produced or received. - The \"Reuser\": any person reusing the \"Information\" in accordance with the conditions of this licence. - \"Personal data\": any information relating to an identified or identifiable natural person who may be identified directly or indirectly. Its \"Reuse\" is conditional on the respect of the existing legal framework. - \"Derived information\": any new data or information created directly from the \"Information\" or from a combination of the \"Information\" and other data or information not subject to this licence. - \"Intellectual property rights\": all rights identified as such under the \"Code de la propri\u00e9t\u00e9 intellectuelle\" (including copyright, rights related to copyright, sui generis rights of database producers, etc.). About this licence This licence is intended to be used by administrations for the reuse of their public information. It can also be used by any individual wishing to supply \"Information\" under the conditions defined in this licence. France has a comprehensive legal framework aiming at the spontaneous dissemination by the administrations of their public information in order to ensure the widest possible reuse of this information. The right to \"Reuse\" the administrations' \"Information\" is governed by the \"Code des relations entre le public et l'administration (CRPA)\". This licence facilitates the unrestricted and free of charge reuse of public information and is one of the licences which can be used by the administration pursuant to the decree issued under article L. 323-2 of the CRPA. Under the Prime Minister's authority, the Etalab mission is mandated to open up the maximum amount of data held by State administrations and public institutions. Etalab has drawn up the Open Licence to facilitate the unrestricted and free of charge reuse of public information, as defined by article L. 321-1 of the CRPA. This licence is version 2.0 of the Open Licence. Etalab reserves the right to propose new versions of the Open Licence. Nevertheless, \"Reusers\" may continue to reuse information obtained under this licence should they so wish.",
"summary": "A (git-)versionned STAC catalog storage and management system",
"version": "0.0.4",
"project_urls": {
"Documentation": "https://github.com/fntb/stac-repository",
"Homepage": "https://github.com/fntb/stac-repository",
"Issues": "https://github.com/fntb/stac-repository",
"Repository": "https://github.com/fntb/stac-repository"
},
"split_keywords": [
"pystac",
" stac",
" database",
" git",
" repository",
" storage",
" store"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "7e64ad291d853f8ef2d66d6895be0fed5da2d40dccf28e4870e77c07149dd2ff",
"md5": "e77c73f5ae7ac8419af070c05e808246",
"sha256": "68e791fd16f9b2658a2cc33a1c8df509234c9c005620d79e4ea68d5192fadd1d"
},
"downloads": -1,
"filename": "stac_repository-0.0.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e77c73f5ae7ac8419af070c05e808246",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 43052,
"upload_time": "2025-07-18T14:33:23",
"upload_time_iso_8601": "2025-07-18T14:33:23.286447Z",
"url": "https://files.pythonhosted.org/packages/7e/64/ad291d853f8ef2d66d6895be0fed5da2d40dccf28e4870e77c07149dd2ff/stac_repository-0.0.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "d6c37f2eb90e38059df98601f44b093b8a6d5184579754965e0298466d78cbd4",
"md5": "83529085e5111153ac3478bd904479da",
"sha256": "91a9af7c8d7272f85ba8e9b51114c6b566931c09647ccf57bc7656b03f1b07e4"
},
"downloads": -1,
"filename": "stac_repository-0.0.4.tar.gz",
"has_sig": false,
"md5_digest": "83529085e5111153ac3478bd904479da",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 39870,
"upload_time": "2025-07-18T14:33:25",
"upload_time_iso_8601": "2025-07-18T14:33:25.107278Z",
"url": "https://files.pythonhosted.org/packages/d6/c3/7f2eb90e38059df98601f44b093b8a6d5184579754965e0298466d78cbd4/stac_repository-0.0.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-18 14:33:25",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "fntb",
"github_project": "stac-repository",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "stac-repository"
}
