# `climdex-kit`: compute, publish, analyse
* [Background](#background)
* [Content](#content)
* [Installation](#installation)
* [Usage](#usage)
* [Data Organization](#data-organization)
* [Logging](#logging)
* [Examples](#examples)
[](docs/uml/)
This project contains a Python package for the parallelized
local computation of [scenario](https://climatescenarios.org/primer/)-aware
climate indices starting from input time-series
of climate projections.
The package comes with an pre-compiled initial set of indices, mostly
relying on the [CDO](https://code.mpimet.mpg.de/projects/cdo) operators.
For the more advanced ones (SPI and SPEI), the [climate-indices](https://climate-indices.readthedocs.io)
Python package is used.
This set of indices can be easily customised or extended: please
check out [how to contribute](CONTRIBUTING.md) if you are interested.
The package is written in Python and for most of the implemented climate indices
relies on both the [CDO](https://code.mpimet.mpg.de/projects/cdo/wiki/Cdo#Documentation)
library, and the [climate-indices](https://github.com/monocongo/climate_indices) package.
## Background
A climate index is information derived from one or more climate variables
(precipitation, mean temperature, etc). The index can range from a simple
conditional counting of days against a threshold, to more complex
statistical processing.
The information is usually calculated spatially
on a pixel-by-pixel basis, with possibly some form of aggregation over time.
Hence the spatio-temporal domain of an index is generally a time-series
with the same spatial resolution, and with either the same or a coarser
temporal step.
The availability of robust and easily interpretable information about
the spatial distribution and temporal evolution of climate related-hazards,
especially climate extremes, is an increasing need not only for the
research community but also for a wide range of sectors and applications.
The European Environmental Agency has been currently supporting, in the
framework of the European Topic Centre on Climate Change impacts, vulnerability
and Adaptation (ETC-CCA), the review and selection of suitable climate-related
indices for Europe to be recommended and implemented for adaptation purposes.
## Content
The project is structured as follows:
* `src/` : source folder containing the Python package implementation
* `test/` : test suites
* `etc/` : folder with configuration files (most notably the **indices.ini**
file with the definition of the climate indices
* `notebooks/` : folder with the Jupyter notebooks
* `docs/` : documentation folder
* `requirements.txt`/`environment.yml` : package dependencies for *pip*
and *conda* environments
* `conda-lock.yml` : environment lock file
* `Makefile` : set of rules for building and installing the package
* `AUTHORS` : list of authors
* `CONTRIBUTING.md` : help for developers
* `CHANGELOG.md` : list of notable changes for each release of the project
* `README.md` : this file
The following files are required for packaging and distribution:
* `pyproject.toml` : tells what is required to build the project
* `setup.cfg`/`setup.py` : static/dynamic package metadata for **setuptools**
## Installation
The package is available as both *pip* sdist/wheel and *conda* package.
### Dependencies
The package relies on the
[CDO](https://code.mpimet.mpg.de/projects/cdo/wiki/Cdo#Installation-and-Supported-Platforms)
operators library v1.9.9. If you are using *conda* environments, you can install the package as follows:
```sh
conda install -c conda-forge cdo=1.9.9
```
In alternative, you can install the `climdex-kit` *conda* package as explained later in this file.
### Install from PyPI via pip
```sh
$ python3 -m pip install climdex-kit
```
### Install as conda package
```sh
TODO
```
### Install in development mode
For developers: refer to the instructions in the [CONTRIBUTING](CONTRIBUTING.md)
file for the setup of the development environment instead.
## Usage
The `climdex` Python package provides a set sub-commands for the specific actions to be taken
* `list`/`ls` : list all available indices
* `show`/`sh` : show the details of a specific index
* `mlut` : shows the lookup-table of the models that are used by the input ensembles
* `compute`/`co` : compute one or more indices
* `analyse`/`an` : analyses input indices ensembles to extract aggregated more readable data
There is thus a hierarchical organization of the CLI arguments.
At any level of the hierarchy, the `--help`/`-h` option can be called to print
the help message.
### general args
| <div style="width:30%">option</div> | <div style="width:60%">description</div> | allowed values |
|:------------------------------------|:-----------------------------------------|:--------------:|
| `--version` | Get the version number of the program | |
| `--idx-conf`/`-c` FILE | Alternative indices configuration file (.ini) (default is ./etc/indices.ini) | abs/rel path |
| `--log-conf`/`-L` FILE | Alternative logging configuration file (.yaml) (default: ./etc/logging.yml) | abs/rel path |
| `-d` | Enable debug mode | |
(See `$ python -m climdex -h` for a full synopsis)
### {list,ls} args
This sub-command currently does not provide any option.
Run `$ python -m climdex list` to get a summary of all available climate indices.
### {show,sh} args
| <div style="width:30%">option</div> | <div style="width:60%">description</div> | allowed values |
|:------------------------------------|:-----------------------------------------|:--------------:|
| `INDEX` | The index configuration to be visualized | see `{list,ls}` sub-command |
(See `$ python -m climdex show -h` for a full synopsis)
### {mlut} args
No args to be provided for this action: the output will be a table of the mappings from
indices to model names that could be used for more handy filtering (`--model` instead of `--regex`) or e.g. used to encode
models coordinates when loading ensembles as OGC grid coverages for rasdaman.
```
========== Climate Models LUT ====================
#00 --> EUR-11_CNRM-CERFACS-CNRM-CM5_CLMcom-CCLM4-8-17_r1i1p1_v1
#01 --> EUR-11_CNRM-CERFACS-CNRM-CM5_CLMcom-ETH-COSMO-crCLIM-v1-1_r1i1p1_v1
#02 --> EUR-11_CNRM-CERFACS-CNRM-CM5_CNRM-ALADIN63_r1i1p1_v2
#03 --> EUR-11_CNRM-CERFACS-CNRM-CM5_KNMI-RACMO22E_r1i1p1_v2
...
```
### {compute,co} args
| <div style="width:30%">option</div> | <div style="width:60%">description</div> | allowed values |
|:------------------------------------|:-----------------------------------------|:--------------:|
| `--index` / `-i` INDEX | A comma-separated list of indices to be computed | see `{list,ls}` sub-command |
| `--multiprocessing` N | The CPU parallelism to be employed | int>0 (N of CPUs) or one among {`one`, `all_but_one`, `all`} |
| `--idir` DIR | Root folder where to look for input files (expected structure: *$input_dir/variable/scenario/\*.nc*) | abs/rel path |
| `--odir`/`-o` DIR | Root folder where to store indices files | abs/rel path |
| `--scenario`/`-s` S | White-space separated list of scenarios | sub-folders of input variables |
| `--regex`/`-x` R | Filter input files with a regular expression | regex |
| `--metadata-only`/`-m` | Only re-set the output attributes (metadata) on existing indices files (compute the index file too on non-existing file instead) | |
| `--dry-run`/`-n` | Only print jobs to output without doing anything | |
| `--force`/`-f` | Force overwrite of existing output indices files (otherwise execution is stopped) | |
(See `$ python -m climdex compute -h` for a full synopsis)
### {analyse,an} args
Given the huge amount of options for the analysis sub-command, we will subdivide them
into three different categories.
Plotting utilities for analysis outputs can be found in the [`plots`](src/climdex/analyse/plots.py) module.
(See `$ python -m climdex analyse -h` for a full synopsis)
#### analysis I/O and execution
| <div style="width:30%">option</div> | <div style="width:60%">description</div> | allowed values |
|:------------------------------------|:-----------------------------------------|:--------------:|
| `--type`/`-t` | The type of analysis | {`map`,`trend`/`trn`, `uncertainty`/`unc` } |
| `--src-type` TYPE | The type of the source of the input indices | {`local`, `openeo`, `wcps`} |
| `--src` SRC | Source of the climate indices | either input folder or endpoint URL |
| `--odir`/`-o` DIR | Output folder where to store analysis files | |
| `--oformat` FMT | Format of the output analysis files | {`nc`, `png`, `json`, `geojson` } |
| `--multiprocessing` N | The CPU parallelism to be employed | int>0 (N of CPUs) or one among {`one`, `all_but_one`, `all`} |
| `--dry-run`/`-n` | Only print jobs to output without doing anything | |
| `--force`/`-f` | Force overwrite of existing output indices files (otherwise execution is stopped) | |
| `--lenient`/`-l` | Silently accept empty outputs when input filters are not overlapping with input domain (otherwise exceptions are raised) | |
### input filtering
These arguments lets the user reduce the scope of the analysis, at different scales: from
the index future scenario, to intra-ensemble model selection, and also at the "pixel" level
within a single future projection.
#### dimensions trimming/clipping
"Image"-level scoping options.
| <div style="width:30%">option</div> | <div style="width:60%">description</div> | allowed values |
|:------------------------------------|:-----------------------------------------|:--------------:|
| `--index` / `-i` INDEX[,INDEX...] | A comma-separated list of indices to be analysed | see `{list,ls}` sub-command |
| `--scenario`/`-s` S [S ...] | White-space separated list of future scenarios | |
| `--model`/`-m` M [M ...] | White-space separated list of climate models | see `mlut` sub-command |
| `--bbox` xmin,ymin,xmax,ymax | Input horizontal bounding-box | input native coordinates |
| `--wgs84-bbox` lat_min,lon_min,lat_max,lon_max | Input horizontal WGS84 bounding-box | latitude/longitude decimal degrees coordinates |
| `--clip` GEOM | Horizontal spatial clipping | WKT geometry text or file containing WKT |
| `--clip-id` I | When clip is a file, the indexing label to be used to describe each geometry in the analysis output files | |
| `--tint` [t0[,t1] ...] | White-space separated list of input time interval(s) of analysis | timestep `t` or time interval `t0,t1` |
| `--baseline` t0,t1 | The baseline time interval | |
#### pixel-based filtering
"Pixel"-level/fine-grained filtering options.
| <div style="width:30%">option</div> | <div style="width:60%">description</div> | allowed values |
|:------------------------------------|:-----------------------------------------|:--------------:|
| `--hfilter` [h],[H] | Pixel-level altitude filter (needs a `--dem`) | { `h,`, `h,H`, `,H` } |
| `--dem` FILE | The Digital Elevation Model to be used for `--hfilter` | DEM with same H-grid as input indices |
| `--significant` DIM [DIM ...] | Keep only statistically significant (Wilcoxon test) "pixels" with the given [core dimensions](https://tutorial.xarray.dev/advanced/apply_ufunc/core-dimensions.html) | |
| `--conf-level` L | The confidence level to be used in the Wilcoxon test (`--significant`) (default is 95%) | [1-99] int |
| `--perc-pos` PERC | Keep only the sections where PERC% of the values in the ensemle are >0 | [0-100] int |
| `--perc-neg` PERC | Keep only the sections where PERC% of the values in the ensemle are <0 | [0-100] int
### aggregation
Finally, these arguments let the user specify how many and which types of aggregation are
to be applied across the dimensions of the climate indices, which span the horizontal XY spatial axes,
time, the future scenarios, and models within the ensemble.
| <div style="width:30%">option</div> | <div style="width:60%">description</div> | allowed values |
|:------------------------------------|:-----------------------------------------|:--------------:|
|`--xyaggr` A [A ...] | Spatial (horizontal) aggregation | {`avg`, `med`, `min`, `max`, `qX`, `pXX`, `range`, `perc_pos`, `perc_neg`} |
| `--taggr` A [A ...] | Temporal aggregation | {`avg`, `med`, `min`, `max`, `qX`, `pXX`, `range`, `perc_pos`, `perc_neg`} |
| `--eaggr` A [A ...] | Ensemble aggregation | {`avg`, `med`, `min`, `max`, `qX`, `pXX`, `range`, `perc_pos`, `perc_neg`} |
| `--baseline-op` OP | The operator used to compare target interval (`--tint`) with baseline (`--baseline`) | {`diff`, `perc_diff`, `ratio`} |
## Data organization
The package expects a fixed organization of the input datasets and a fixed naming scheme
in order to properly extract all the metadata.
The path and name of a climate projection NetCDF starting from the $IDIR input
root directory (`--idir` in the command line) shall be as follows:
```sh
$IDIR/{var}/{scenario}/{var}_{model}_{timeres}_{yearstart}{yearend}_{scenario}.nc
```
Being:
* `{var}` : the climate variable (whose label shall also coincide with the name of the
variable in the NetCDF)
* `{scenario}` : the name of the emissions scenario
* `{model}` : the name of the climate model used to create the projection
* `timeres` : the time-step of the time-series (e.g. day, month, etc)
* `yearstart` / `yearend` : time range of the time-series (`YYYY` format)
For ancillary scenario-independent datasets (e.g. land mask),
the `{scenario}/` sub-folder can be omitted mandatory, and the name
of the dataset shall be `{var}.nc`.
Analogously, given the $ODIR output specified via `--odir/-o` CLI argument,
each index file will be stored then as follows:
```sh
$ODIR/{index}/{scenario}/{index}_{model}_{timeres}_{yearstart}{yearend}_{scenario}.nc
```
## Logging
By default the program logs to both console (with colored output to highlight warnings and errors),
and to a file called `climdex.log` in the current working directory.
The configuration of both loggers can be found in `./etc/logging.yaml`, otherwise use
the `--log-conf`/`-L` option to set an alternative configuration.
## Examples
Here some examples of command-line calls of the climdex-kit available commands:
```sh
# list all avaiable indices
$ python -m climdex list
# show the configuration details of the index [spei12]
$ python -m climdex show spei12
# show the model indexing used
$ python -m climdex mlut
# compute the frost days [fd] and 12-months SPI [spi12] index on all available climate
# projections for scenario rcp85 and by using 3 CPUs
$ python -m climdex \
compute \
--index amt,spi12 \
--multiprocessing 3 \
--scenario rcp85 \
--idir $IDIR \
--odir $ODIR
# update the metadata of all existing indexes of scenario rcp85, and compute the missing ones anew
# using all CPUs + turn on debug mode + dry run only
$ python -m climdex -d \
compute \
--index all \
--scenario rcp85 \
--multiprocessing all \
--metadata-only \
--idir $IDIR \
--odir $ODIR \
--dry-run
# re-compute the [fd] and [tn] indices for the model "EUR-11_CNRM-CERFACS-CNRM-CM5_CLMcom-CCLM4-8-17_r1i1p1_v1"
# and scenario rcp45 and keep it on a separate file for comparison with existing
# + use 1 CPU (sequential execution)
$ python -m climdex \
compute \
--index fd,tn \
--scenario rcp45 \
--multiprocessing one \
--idir $IDIR \
--odir $ODIR \
--regex "*EUR-11_CNRM-CERFACS-CNRM-CM5_CLMcom-CCLM4-8-17_r1i1p1_v1*"
# re-compute (and overwrite when existing) all indexes on rcp45 and rcp85 scenarios
# + using all available CPUs except one
$ python -m climdex \
compute \
--index all \
--scenario rcp45 rcp85 \
--multiprocessing all_but_one \
--idir $IDIR \
--odir $ODIR \
--force
# extract the 2D maps of relative difference with respect to [1981,2010] baseline of the [rx5day] and [r95ptot]
# indices over the time interval [2041,2070], averaging over time and pickling the ensemble median value,
# and for both rcp45 and rcp85 scenarios
# (input from locally available NetCDFs computed with climdex-compute)
$ python -m climdex \
analyse -t map \
--index rx5day,r95ptot \
--src-type local \
--src $IDIR \
--odir $ODIR \
--oformat nc \
--scenario rcp45 rcp85 \
--tint 2041,2070 \
--baseline 1981,2010 \
--baseline-op "perc_diff" \
--taggr avg \
--eaggr med
# extract time profile of 5th, 50th, and 95th time percentiles of [su] and [tr] indices over the whole series
# over averaged grid points below 700m of altitude and both rcp45 and rcp85 scenarios
# (input from locally available NetCDFs computed with climdex-compute)
$ python -m climdex \
analyse -t trend \
--index su,tr \
--src-type local \
--src $IDIR \
--odir $ODIR \
--oformat json \
--scenario rcp45 rcp85 \
--hfilter ,700 \
--dem $DEM_FILE\
--xyaggr avg \
--eaggr p05 med p95
# extract the five-number statistics (min, 5p, med, 95p, max) of [tx90p] and [tn90p] indices over the three
# different time intervals, with values relative to a [1981-2010] baseline
# over averaged grid points for both rcp45 and rcp85 scenarios
# (input from locally available NetCDFs computed with climdex-compute)
$ python -m climdex \
analyse -t trend \
--index su,tr \
--src-type local \
--src $IDIR \
--odir $ODIR \
--oformat json \
--scenario rcp45 rcp85 \
--tint 2011,2040 2041,2070 2071,2100 \
--baseline 1981,2010 \
--baseline-op "diff" \
--taggr avg \
--xyaggr avg \
--eaggr min p05 med p95 max
# extract the map of data range and percentage of agreement (positive direction) within the ensemble
# of relative differences with respect to [1981-2010] baseline for the time period 2081-2100
# for the [tr] index only over the grid points above the 1'000m of altitude, for the rcp45 scenario
# (input from locally available NetCDFs computed with climdex-compute)
$ python -m climdex \
analyse -t map \
--index tr \
--src-type local\
--src $IDIR \
--odir $ODIR \
--oformat nc \
--scenario rcp45 \
--tint 2081,2100 \
--baseline 1981,2010 \
--baseline-op "diff" \
--taggr avg \
--eaggr range perc_pos \
--hfilter 1000, \
--dem $DEM_FILE
# For each geometry in the regions.shp file (uniquely identified and to be serialized using "catasto_code" field), compute
# five time-series of the ensemble i) minimum, ii) 1st quartile, iii) median, iv) 3rd quartile, and
# v) maximum of the spatial averages for each timestep wihtin the 2071-2100 time period and
# serialize it as a GeoJSON Feature Collection, for both rcp45 and rcp85 climate scenarios,
# parallelizing the tasks across all available CPUs
$ python -m climdex \
analyse -t trn \
--index hwd \
--src-type local \
--src $IDIR \
--odir $ODIR \
--oformat geojson \
--scenario rcp45 rcp85 \
--tint 2071,2100 \
--xyaggr avg \
--eaggr min q1 med q3 max \
--clip regions.shp \
--clip-id catasto_code \
--multiprocessing all
```
## Credits
This project is funded by the
[FAct CLIMAX](https://www.eurac.edu/en/institutes-centers/center-for-climate-change-and-transformation/projects/fact-climax)
project at Eurac Research (Center for Climate Change and Transformation, Institute for Earth Observation).
[](https://www.eurac.edu/en/institutes-centers/center-for-climate-change-and-transformation)
Raw data
{
"_id": null,
"home_page": "https://gitlab.inf.unibz.it/earth_observation_public/cdr/climdex-kit",
"name": "climdex-kit",
"maintainer": "Piero Campalani",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "piero.campalani@eurac.edu",
"keywords": "indices, climate, scenario",
"author": "Piero Campalani",
"author_email": "piero.campalani@eurac.edu",
"download_url": "https://files.pythonhosted.org/packages/79/c3/1663583fc60e12a2602cbafacfe15482fc964db7beaaccdbe85baf8d168c/climdex_kit-1.0.0.tar.gz",
"platform": null,
"description": "\n\n\n\n\n\n# `climdex-kit`: compute, publish, analyse\n\n* [Background](#background)\n* [Content](#content)\n* [Installation](#installation)\n* [Usage](#usage)\n* [Data Organization](#data-organization)\n* [Logging](#logging)\n* [Examples](#examples)\n\n[](docs/uml/)\n\nThis project contains a Python package for the parallelized\nlocal computation of [scenario](https://climatescenarios.org/primer/)-aware\nclimate indices starting from input time-series\nof climate projections.\n\nThe package comes with an pre-compiled initial set of indices, mostly\nrelying on the [CDO](https://code.mpimet.mpg.de/projects/cdo) operators.\nFor the more advanced ones (SPI and SPEI), the [climate-indices](https://climate-indices.readthedocs.io)\nPython package is used.\nThis set of indices can be easily customised or extended: please\ncheck out [how to contribute](CONTRIBUTING.md) if you are interested.\n\nThe package is written in Python and for most of the implemented climate indices\nrelies on both the [CDO](https://code.mpimet.mpg.de/projects/cdo/wiki/Cdo#Documentation)\nlibrary, and the [climate-indices](https://github.com/monocongo/climate_indices) package.\n\n## Background\n\nA climate index is information derived from one or more climate variables\n(precipitation, mean temperature, etc). The index can range from a simple\nconditional counting of days against a threshold, to more complex\nstatistical processing.\n\nThe information is usually calculated spatially\non a pixel-by-pixel basis, with possibly some form of aggregation over time.\nHence the spatio-temporal domain of an index is generally a time-series\nwith the same spatial resolution, and with either the same or a coarser\ntemporal step.\n\nThe availability of robust and easily interpretable information about\nthe spatial distribution and temporal evolution of climate related-hazards,\nespecially climate extremes, is an increasing need not only for the\nresearch community but also for a wide range of sectors and applications.\n\nThe European Environmental Agency has been currently supporting, in the\nframework of the European Topic Centre on Climate Change impacts, vulnerability\nand Adaptation (ETC-CCA), the review and selection of suitable climate-related\nindices for Europe to be recommended and implemented for adaptation purposes.\n\n\n## Content\n\nThe project is structured as follows:\n\n* `src/` : source folder containing the Python package implementation\n* `test/` : test suites\n* `etc/` : folder with configuration files (most notably the **indices.ini**\n file with the definition of the climate indices\n* `notebooks/` : folder with the Jupyter notebooks\n* `docs/` : documentation folder\n* `requirements.txt`/`environment.yml` : package dependencies for *pip*\n and *conda* environments\n* `conda-lock.yml` : environment lock file\n* `Makefile` : set of rules for building and installing the package\n* `AUTHORS` : list of authors\n* `CONTRIBUTING.md` : help for developers\n* `CHANGELOG.md` : list of notable changes for each release of the project\n* `README.md` : this file\n\nThe following files are required for packaging and distribution:\n\n* `pyproject.toml` : tells what is required to build the project\n* `setup.cfg`/`setup.py` : static/dynamic package metadata for **setuptools**\n\n\n## Installation\n\nThe package is available as both *pip* sdist/wheel and *conda* package.\n\n### Dependencies\n\nThe package relies on the\n[CDO](https://code.mpimet.mpg.de/projects/cdo/wiki/Cdo#Installation-and-Supported-Platforms)\noperators library v1.9.9. If you are using *conda* environments, you can install the package as follows:\n\n```sh\nconda install -c conda-forge cdo=1.9.9\n```\n\nIn alternative, you can install the `climdex-kit` *conda* package as explained later in this file.\n\n### Install from PyPI via pip\n\n```sh\n$ python3 -m pip install climdex-kit\n```\n\n### Install as conda package\n\n```sh\nTODO\n```\n\n### Install in development mode\n\nFor developers: refer to the instructions in the [CONTRIBUTING](CONTRIBUTING.md)\nfile for the setup of the development environment instead.\n\n## Usage\n\nThe `climdex` Python package provides a set sub-commands for the specific actions to be taken\n\n* `list`/`ls` : list all available indices\n* `show`/`sh` : show the details of a specific index\n* `mlut` : shows the lookup-table of the models that are used by the input ensembles\n* `compute`/`co` : compute one or more indices\n* `analyse`/`an` : analyses input indices ensembles to extract aggregated more readable data\n\nThere is thus a hierarchical organization of the CLI arguments.\nAt any level of the hierarchy, the `--help`/`-h` option can be called to print\nthe help message.\n\n### general args\n\n| <div style=\"width:30%\">option</div> | <div style=\"width:60%\">description</div> | allowed values |\n|:------------------------------------|:-----------------------------------------|:--------------:|\n| `--version` | Get the version number of the program | |\n| `--idx-conf`/`-c` FILE | Alternative indices configuration file (.ini) (default is ./etc/indices.ini) | abs/rel path |\n| `--log-conf`/`-L` FILE | Alternative logging configuration file (.yaml) (default: ./etc/logging.yml) | abs/rel path |\n| `-d` | Enable debug mode | |\n\n(See `$ python -m climdex -h` for a full synopsis)\n\n\n### {list,ls} args\n\nThis sub-command currently does not provide any option.\nRun `$ python -m climdex list` to get a summary of all available climate indices.\n\n\n### {show,sh} args\n\n| <div style=\"width:30%\">option</div> | <div style=\"width:60%\">description</div> | allowed values |\n|:------------------------------------|:-----------------------------------------|:--------------:|\n| `INDEX` | The index configuration to be visualized | see `{list,ls}` sub-command | \n\n(See `$ python -m climdex show -h` for a full synopsis)\n\n### {mlut} args\n\nNo args to be provided for this action: the output will be a table of the mappings from\nindices to model names that could be used for more handy filtering (`--model` instead of `--regex`) or e.g. used to encode \nmodels coordinates when loading ensembles as OGC grid coverages for rasdaman.\n\n```\n========== Climate Models LUT ====================\n #00 --> EUR-11_CNRM-CERFACS-CNRM-CM5_CLMcom-CCLM4-8-17_r1i1p1_v1\n #01 --> EUR-11_CNRM-CERFACS-CNRM-CM5_CLMcom-ETH-COSMO-crCLIM-v1-1_r1i1p1_v1\n #02 --> EUR-11_CNRM-CERFACS-CNRM-CM5_CNRM-ALADIN63_r1i1p1_v2\n #03 --> EUR-11_CNRM-CERFACS-CNRM-CM5_KNMI-RACMO22E_r1i1p1_v2\n ...\n```\n\n### {compute,co} args\n\n| <div style=\"width:30%\">option</div> | <div style=\"width:60%\">description</div> | allowed values |\n|:------------------------------------|:-----------------------------------------|:--------------:|\n| `--index` / `-i` INDEX | A comma-separated list of indices to be computed | see `{list,ls}` sub-command | \n| `--multiprocessing` N | The CPU parallelism to be employed | int>0 (N of CPUs) or one among {`one`, `all_but_one`, `all`} |\n| `--idir` DIR | Root folder where to look for input files (expected structure: *$input_dir/variable/scenario/\\*.nc*) | abs/rel path |\n| `--odir`/`-o` DIR | Root folder where to store indices files | abs/rel path |\n| `--scenario`/`-s` S | White-space separated list of scenarios | sub-folders of input variables |\n| `--regex`/`-x` R | Filter input files with a regular expression | regex |\n| `--metadata-only`/`-m` | Only re-set the output attributes (metadata) on existing indices files (compute the index file too on non-existing file instead) | |\n| `--dry-run`/`-n` | Only print jobs to output without doing anything | |\n| `--force`/`-f` | Force overwrite of existing output indices files (otherwise execution is stopped) | |\n\n(See `$ python -m climdex compute -h` for a full synopsis)\n\n\n### {analyse,an} args\n\nGiven the huge amount of options for the analysis sub-command, we will subdivide them\ninto three different categories.\n\nPlotting utilities for analysis outputs can be found in the [`plots`](src/climdex/analyse/plots.py) module.\n\n(See `$ python -m climdex analyse -h` for a full synopsis)\n\n#### analysis I/O and execution\n\n| <div style=\"width:30%\">option</div> | <div style=\"width:60%\">description</div> | allowed values |\n|:------------------------------------|:-----------------------------------------|:--------------:|\n| `--type`/`-t` | The type of analysis | {`map`,`trend`/`trn`, `uncertainty`/`unc` } | \n| `--src-type` TYPE | The type of the source of the input indices | {`local`, `openeo`, `wcps`} |\n| `--src` SRC | Source of the climate indices | either input folder or endpoint URL |\n| `--odir`/`-o` DIR | Output folder where to store analysis files | |\n| `--oformat` FMT | Format of the output analysis files | {`nc`, `png`, `json`, `geojson` } |\n| `--multiprocessing` N | The CPU parallelism to be employed | int>0 (N of CPUs) or one among {`one`, `all_but_one`, `all`} |\n| `--dry-run`/`-n` | Only print jobs to output without doing anything | |\n| `--force`/`-f` | Force overwrite of existing output indices files (otherwise execution is stopped) | |\n| `--lenient`/`-l` | Silently accept empty outputs when input filters are not overlapping with input domain (otherwise exceptions are raised) | |\n\n### input filtering\n\nThese arguments lets the user reduce the scope of the analysis, at different scales: from\nthe index future scenario, to intra-ensemble model selection, and also at the \"pixel\" level\nwithin a single future projection.\n\n#### dimensions trimming/clipping\n\n\"Image\"-level scoping options.\n\n| <div style=\"width:30%\">option</div> | <div style=\"width:60%\">description</div> | allowed values |\n|:------------------------------------|:-----------------------------------------|:--------------:|\n| `--index` / `-i` INDEX[,INDEX...] | A comma-separated list of indices to be analysed | see `{list,ls}` sub-command |\n| `--scenario`/`-s` S [S ...] | White-space separated list of future scenarios | |\n| `--model`/`-m` M [M ...] | White-space separated list of climate models | see `mlut` sub-command |\n| `--bbox` xmin,ymin,xmax,ymax | Input horizontal bounding-box | input native coordinates |\n| `--wgs84-bbox` lat_min,lon_min,lat_max,lon_max | Input horizontal WGS84 bounding-box | latitude/longitude decimal degrees coordinates |\n| `--clip` GEOM | Horizontal spatial clipping | WKT geometry text or file containing WKT |\n| `--clip-id` I | When clip is a file, the indexing label to be used to describe each geometry in the analysis output files | |\n| `--tint` [t0[,t1] ...] | White-space separated list of input time interval(s) of analysis | timestep `t` or time interval `t0,t1` |\n| `--baseline` t0,t1 | The baseline time interval | |\n\n#### pixel-based filtering\n\n\"Pixel\"-level/fine-grained filtering options.\n\n| <div style=\"width:30%\">option</div> | <div style=\"width:60%\">description</div> | allowed values |\n|:------------------------------------|:-----------------------------------------|:--------------:|\n| `--hfilter` [h],[H] | Pixel-level altitude filter (needs a `--dem`) | { `h,`, `h,H`, `,H` } |\n| `--dem` FILE | The Digital Elevation Model to be used for `--hfilter` | DEM with same H-grid as input indices |\n| `--significant` DIM [DIM ...] | Keep only statistically significant (Wilcoxon test) \"pixels\" with the given [core dimensions](https://tutorial.xarray.dev/advanced/apply_ufunc/core-dimensions.html) | |\n| `--conf-level` L | The confidence level to be used in the Wilcoxon test (`--significant`) (default is 95%) | [1-99] int |\n| `--perc-pos` PERC | Keep only the sections where PERC% of the values in the ensemle are >0 | [0-100] int |\n| `--perc-neg` PERC | Keep only the sections where PERC% of the values in the ensemle are <0 | [0-100] int \n\n### aggregation\n\nFinally, these arguments let the user specify how many and which types of aggregation are\nto be applied across the dimensions of the climate indices, which span the horizontal XY spatial axes,\ntime, the future scenarios, and models within the ensemble.\n\n| <div style=\"width:30%\">option</div> | <div style=\"width:60%\">description</div> | allowed values |\n|:------------------------------------|:-----------------------------------------|:--------------:|\n|`--xyaggr` A [A ...] | Spatial (horizontal) aggregation | {`avg`, `med`, `min`, `max`, `qX`, `pXX`, `range`, `perc_pos`, `perc_neg`} |\n| `--taggr` A [A ...] | Temporal aggregation | {`avg`, `med`, `min`, `max`, `qX`, `pXX`, `range`, `perc_pos`, `perc_neg`} |\n| `--eaggr` A [A ...] | Ensemble aggregation | {`avg`, `med`, `min`, `max`, `qX`, `pXX`, `range`, `perc_pos`, `perc_neg`} |\n| `--baseline-op` OP | The operator used to compare target interval (`--tint`) with baseline (`--baseline`) | {`diff`, `perc_diff`, `ratio`} |\n\n\n## Data organization\n\nThe package expects a fixed organization of the input datasets and a fixed naming scheme\nin order to properly extract all the metadata.\n\nThe path and name of a climate projection NetCDF starting from the $IDIR input\nroot directory (`--idir` in the command line) shall be as follows:\n\n```sh\n$IDIR/{var}/{scenario}/{var}_{model}_{timeres}_{yearstart}{yearend}_{scenario}.nc\n```\n\nBeing:\n\n* `{var}` : the climate variable (whose label shall also coincide with the name of the\nvariable in the NetCDF)\n* `{scenario}` : the name of the emissions scenario\n* `{model}` : the name of the climate model used to create the projection\n* `timeres` : the time-step of the time-series (e.g. day, month, etc)\n* `yearstart` / `yearend` : time range of the time-series (`YYYY` format)\n\nFor ancillary scenario-independent datasets (e.g. land mask),\nthe `{scenario}/` sub-folder can be omitted mandatory, and the name\nof the dataset shall be `{var}.nc`.\n\nAnalogously, given the $ODIR output specified via `--odir/-o` CLI argument, \neach index file will be stored then as follows:\n\n```sh\n$ODIR/{index}/{scenario}/{index}_{model}_{timeres}_{yearstart}{yearend}_{scenario}.nc\n``` \n\n## Logging\n\nBy default the program logs to both console (with colored output to highlight warnings and errors),\nand to a file called `climdex.log` in the current working directory.\n\nThe configuration of both loggers can be found in `./etc/logging.yaml`, otherwise use\nthe `--log-conf`/`-L` option to set an alternative configuration.\n\n\n## Examples\n\nHere some examples of command-line calls of the climdex-kit available commands:\n\n```sh\n# list all avaiable indices\n$ python -m climdex list\n\n# show the configuration details of the index [spei12]\n$ python -m climdex show spei12\n\n# show the model indexing used\n$ python -m climdex mlut\n\n# compute the frost days [fd] and 12-months SPI [spi12] index on all available climate\n# projections for scenario rcp85 and by using 3 CPUs\n$ python -m climdex \\ \n compute \\\n --index amt,spi12 \\\n --multiprocessing 3 \\\n --scenario rcp85 \\\n --idir $IDIR \\\n --odir $ODIR\n\n# update the metadata of all existing indexes of scenario rcp85, and compute the missing ones anew\n# using all CPUs + turn on debug mode + dry run only\n$ python -m climdex -d \\ \n compute \\\n --index all \\\n --scenario rcp85 \\\n --multiprocessing all \\\n --metadata-only \\\n --idir $IDIR \\\n --odir $ODIR \\\n --dry-run\n\n# re-compute the [fd] and [tn] indices for the model \"EUR-11_CNRM-CERFACS-CNRM-CM5_CLMcom-CCLM4-8-17_r1i1p1_v1\"\n# and scenario rcp45 and keep it on a separate file for comparison with existing\n# + use 1 CPU (sequential execution)\n$ python -m climdex \\ \n compute \\\n --index fd,tn \\\n --scenario rcp45 \\\n --multiprocessing one \\\n --idir $IDIR \\\n --odir $ODIR \\\n --regex \"*EUR-11_CNRM-CERFACS-CNRM-CM5_CLMcom-CCLM4-8-17_r1i1p1_v1*\"\n \n# re-compute (and overwrite when existing) all indexes on rcp45 and rcp85 scenarios\n# + using all available CPUs except one\n$ python -m climdex \\ \n compute \\\n --index all \\\n --scenario rcp45 rcp85 \\\n --multiprocessing all_but_one \\\n --idir $IDIR \\\n --odir $ODIR \\\n --force\n\n# extract the 2D maps of relative difference with respect to [1981,2010] baseline of the [rx5day] and [r95ptot]\n# indices over the time interval [2041,2070], averaging over time and pickling the ensemble median value,\n# and for both rcp45 and rcp85 scenarios\n# (input from locally available NetCDFs computed with climdex-compute)\n$ python -m climdex \\\n analyse -t map \\\n --index rx5day,r95ptot \\\n --src-type local \\\n --src $IDIR \\\n --odir $ODIR \\\n --oformat nc \\\n --scenario rcp45 rcp85 \\\n --tint 2041,2070 \\\n --baseline 1981,2010 \\\n --baseline-op \"perc_diff\" \\\n --taggr avg \\\n --eaggr med\n\n# extract time profile of 5th, 50th, and 95th time percentiles of [su] and [tr] indices over the whole series\n# over averaged grid points below 700m of altitude and both rcp45 and rcp85 scenarios\n# (input from locally available NetCDFs computed with climdex-compute)\n$ python -m climdex \\ \n analyse -t trend \\\n --index su,tr \\\n --src-type local \\\n --src $IDIR \\\n --odir $ODIR \\\n --oformat json \\\n --scenario rcp45 rcp85 \\\n --hfilter ,700 \\\n --dem $DEM_FILE\\\n --xyaggr avg \\\n --eaggr p05 med p95\n\n# extract the five-number statistics (min, 5p, med, 95p, max) of [tx90p] and [tn90p] indices over the three \n# different time intervals, with values relative to a [1981-2010] baseline\n# over averaged grid points for both rcp45 and rcp85 scenarios\n# (input from locally available NetCDFs computed with climdex-compute)\n$ python -m climdex \\ \n analyse -t trend \\\n --index su,tr \\\n --src-type local \\\n --src $IDIR \\\n --odir $ODIR \\\n --oformat json \\\n --scenario rcp45 rcp85 \\\n --tint 2011,2040 2041,2070 2071,2100 \\\n --baseline 1981,2010 \\\n --baseline-op \"diff\" \\\n --taggr avg \\\n --xyaggr avg \\\n --eaggr min p05 med p95 max\n\n# extract the map of data range and percentage of agreement (positive direction) within the ensemble\n# of relative differences with respect to [1981-2010] baseline for the time period 2081-2100\n# for the [tr] index only over the grid points above the 1'000m of altitude, for the rcp45 scenario\n# (input from locally available NetCDFs computed with climdex-compute)\n$ python -m climdex \\\n analyse -t map \\\n --index tr \\\n --src-type local\\\n --src $IDIR \\\n --odir $ODIR \\\n --oformat nc \\\n --scenario rcp45 \\\n --tint 2081,2100 \\\n --baseline 1981,2010 \\\n --baseline-op \"diff\" \\\n --taggr avg \\\n --eaggr range perc_pos \\\n --hfilter 1000, \\\n --dem $DEM_FILE\n\n# For each geometry in the regions.shp file (uniquely identified and to be serialized using \"catasto_code\" field), compute\n# five time-series of the ensemble i) minimum, ii) 1st quartile, iii) median, iv) 3rd quartile, and\n# v) maximum of the spatial averages for each timestep wihtin the 2071-2100 time period and\n# serialize it as a GeoJSON Feature Collection, for both rcp45 and rcp85 climate scenarios,\n# parallelizing the tasks across all available CPUs\n$ python -m climdex \\\n analyse -t trn \\\n --index hwd \\\n --src-type local \\\n --src $IDIR \\\n --odir $ODIR \\\n --oformat geojson \\\n --scenario rcp45 rcp85 \\\n --tint 2071,2100 \\\n --xyaggr avg \\\n --eaggr min q1 med q3 max \\\n --clip regions.shp \\\n --clip-id catasto_code \\\n --multiprocessing all \n```\n\n## Credits\nThis project is funded by the\n [FAct CLIMAX](https://www.eurac.edu/en/institutes-centers/center-for-climate-change-and-transformation/projects/fact-climax)\nproject at Eurac Research (Center for Climate Change and Transformation, Institute for Earth Observation).\n\n[](https://www.eurac.edu/en/institutes-centers/center-for-climate-change-and-transformation)\n\n",
"bugtrack_url": null,
"license": null,
"summary": "Tool for multi-scenario calculation, publishing, and analysis of indices from 3D time-series of climate projections.",
"version": "1.0.0",
"project_urls": {
"Bug Tracker": "https://gitlab.inf.unibz.it/earth_observation_public/cdr/climdex-kit/-/issues",
"Download": "https://gitlab.inf.unibz.it/earth_observation_public/cdr/climdex-kit/-/releases/v1.0.0",
"Homepage": "https://gitlab.inf.unibz.it/earth_observation_public/cdr/climdex-kit",
"Source": "https://gitlab.inf.unibz.it/earth_observation_public/cdr/climdex-kit"
},
"split_keywords": [
"indices",
" climate",
" scenario"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "db5b9ac841ef364afb6f8a443030a28c865e6f78f3aa5467d7d6fa50644de7c8",
"md5": "7f71a4b70c3248aa1f70fbd8bbcb0a49",
"sha256": "dfcf219169a63f3b7c0f294a1869549d17834615959b4af69c07e0e4602f3893"
},
"downloads": -1,
"filename": "climdex_kit-1.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7f71a4b70c3248aa1f70fbd8bbcb0a49",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 146012,
"upload_time": "2024-10-02T17:03:47",
"upload_time_iso_8601": "2024-10-02T17:03:47.511835Z",
"url": "https://files.pythonhosted.org/packages/db/5b/9ac841ef364afb6f8a443030a28c865e6f78f3aa5467d7d6fa50644de7c8/climdex_kit-1.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "79c31663583fc60e12a2602cbafacfe15482fc964db7beaaccdbe85baf8d168c",
"md5": "b77e49046b987029862eca92b40592bd",
"sha256": "45355d00d6d24c526be001b575778b61eec4b397d9072343bc9b7c6951b6edef"
},
"downloads": -1,
"filename": "climdex_kit-1.0.0.tar.gz",
"has_sig": false,
"md5_digest": "b77e49046b987029862eca92b40592bd",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 1009335,
"upload_time": "2024-10-02T17:03:49",
"upload_time_iso_8601": "2024-10-02T17:03:49.646043Z",
"url": "https://files.pythonhosted.org/packages/79/c3/1663583fc60e12a2602cbafacfe15482fc964db7beaaccdbe85baf8d168c/climdex_kit-1.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-02 17:03:49",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "climdex-kit"
}
JSON