# ngiab-cal
A Python CLI tool to simplify hydrologic model calibration for NextGen In A Box (NGIAB) workflows.
## Table of Contents
- [What is this?](#what-is-this)
- [Installation](#installation)
- [Requirements](#requirements)
- [Basic Usage](#basic-usage)
- [Advanced Options](#advanced-options)
- [Calibration Process](#calibration-process)
- [Calibration Configuration File](#calibration-configuration-file)
- [Example: Calibrating CAMELS Basins](#example-calibrating-camels-basins)
- [How It Works](#how-it-works)
- [How is ngen-cal running?](#how-is-ngen-cal-running)
- [Development](#development)
- [License](#license)
- [Acknowledgments](#acknowledgments)
## What is this?
ngiab-cal is a utility that works with the [NGIAB folder structure](https://docs.ciroh.org/training-NGIAB-101/data-preparation.html#nextgen-run-directory-structure-ngen-run). It automates the creation of a calibration directory with all necessary configuration files to run a modified version of [ngen-cal](https://github.com/CIROH-UA/ngen-cal/tree/ngiab_cal).
The tool simplifies these key tasks:
- Creating calibration configurations
- Running the calibration process using Docker
- Copying calibrated parameters back to your model configuration
## Installation
Several installation options are available:
### Using `uvx` (recommended: for its speed,and efficient environment management)
```bash
# Run directly without installation
uvx ngiab-cal --help
# Or install as a tool
uv tool install ngiab-cal
ngiab-cal --help
```
### Using `pipx`
```bash
pipx install ngiab-cal
ngiab-cal --help
```
### Traditional pip installation
```bash
pip install ngiab-cal
python -m ngiab_cal --help
```
## Requirements
- Python 3.10+
- Docker (for running calibrations)
- Internet connection (for downloading USGS data)
## Basic Usage
```bash
# Create calibration configuration
ngiab-cal /path/to/ngiab/data/folder -g USGS_GAGE_ID
# Create and run calibration (200 iterations)
ngiab-cal /path/to/ngiab/data/folder -g USGS_GAGE_ID --run -i 200
# Force recreation of calibration configuration
ngiab-cal /path/to/ngiab/data/folder -g USGS_GAGE_ID -f
```
## Advanced Options
```
usage: ngiab-cal [-h] [-g GAGE] [-f] [--run] [-i ITERATIONS] [--debug] [-w WARMUP] [--calibration_ratio CALIBRATION_RATIO]
data_folder
Create a calibration config for ngen-cal
positional arguments:
data_folder Path to the folder you wish to calibrate
options:
-h, --help show this help message and exit
-g GAGE, --gage GAGE Gage ID to use for calibration
-f, --force Overwrite existing configuration
--run Try to automatically run the calibration, this may be unstable
-i ITERATIONS, --iterations ITERATIONS
Default:100 number of iterations to calibrate for
--debug enable debug logging
-w WARMUP, --warmup WARMUP
Default:365
Number of days at the beginning of the simulation
to exclude from calibration objective metric calculation
--calibration_ratio CALIBRATION_RATIO, --cr CALIBRATION_RATIO
Default:0.5
How to split time after warmup into calibration and validation.
1 == 100% calibration, 0 == 100% validation, 0.8 == 80% calibration 20% validation
```
## Calibration Process
The tool applies a standard hydrological modeling workflow, which involves warmup, calibration, and validation periods. The --warmup period is crucial for allowing the model to reach a stable state before its performance is evaluated against observed data. Following the warmup, the remaining period is typically divided into calibration (where model parameters are adjusted to match observations) and validation (where the model's performance with the calibrated parameters is tested on an independent dataset). The tool facilitates this split, as detailed in the diagram and options below.
```
Default calibration settings on a 5 year period
| year 1 | year 2 | year 3 | year 4 | year 5 |
|<- warmup ->|<- calibration ->| |
|<- warmup ->|<- validation ->|
```
## Example: Calibrating CAMELS Basins
Here's a script to calibrate all CAMELS basins:
```bash
#!/bin/bash
# Download a list of CAMELS gage ids
wget https://raw.githubusercontent.com/peckhams/nextgen_basin_repo/5e1317256a9365ae3a24a250358314e1e9ffc339/CAMELS/Data/camels_name.txt ./camels_name.txt
output_folder=$(cat ~/.ngiab/preprocessor)
while read line
do
gage=$(echo "$line" | cut -d ';' -f 1)
echo $gage
# subset the hydrofabric, calculate mean-average area forcings, generate model config files
uvx --from ngiab_data_preprocess cli -i gage-"$gage" -sfr --start 2007-10-01 --end 2013-09-30
# calibrate gage for 200 iterations
uvx ngiab-cal "$output_folder"/gage-"$gage" -g "$gage" --run -i 200
done < <(tail -n +2 ./camels_name.txt)
```
## Calibration Configuration File
ngiab-cal generates a configuration file at `calibration/ngen_cal_conf.yaml` that controls the calibration process. This file is created from a template with the following key sections:
### General Configuration
```yaml
general:
strategy:
type: estimation
algorithm: dds # Uses Dynamically Dimensioned Search algorithm
name: calib # Don't modify this
log: true # Enable logging
workdir: /ngen/ngen/data/calibration # Don't modify this working directory in the Docker container
yaml_file: /ngen/ngen/data/calibration/ngen_cal_conf.yaml # Don't modify this either
iterations: 100 # Number of calibration iterations (customizable with -i flag)
restart: 0 # Start from beginning (0) or resume from iteration
```
### Model Parameters
The file defines parameters for both CFE (Conceptual Functional Equivalent) and Noah-OWP-Modular (an extended, refactored version of the Noah-MP land surface model) hydrologic models:
```yaml
CFE:
- name: b # CFE parameter name
min: 2.0 # Minimum allowed value
max: 15.0 # Maximum allowed value
init: 4.05 # Initial value
- name: satpsi
min: 0.03
max: 0.955
init: 0.355
# Additional parameters...
```
For each parameter, the configuration specifies:
- `name`: Parameter identifier
- `min`: Minimum allowed value during calibration
- `max`: Maximum allowed value during calibration
- `init`: Initial value
### Evaluation Configuration
```yaml
eval_params:
objective: kge # Kling-Gupta Efficiency as objective function
evaluation_start: "..." # Start time for calibration period
evaluation_stop: "..." # End time for calibration period
valid_start_time: "..." # Start time including warmup
valid_end_time: "..." # End time of simulation
# Additional time parameters...
basinID: 01646500 # USGS gage ID
site_name: "USGS 01646500: " # Label for plots
```
These time periods follow the calibration process diagram shown earlier, with separate periods for warmup, calibration, and validation.
## How It Works
1. **Input Validation** - Checks that all required files are present
2. **USGS Data Download** - Retrieves observed streamflow for calibration
3. **Configuration Creation** - Generates necessary files for ngen-cal
4. **Docker Execution** - Runs the calibration in a containerized environment
5. **Parameter Extraction** - Copies calibrated parameters back to your configuration
## How is ngen-cal running?
The tool uses a custom [branch of CIROH-UA/ngen-cal](https://github.com/CIROH-UA/ngen-cal/tree/ngiab_cal) called `ngiab_cal`. This branch contains:
- A modified version of ngen-cal
- A Dockerfile that builds on top of the Next Gen In-A-Box container
- Installation of ngen-cal inside the container
When you provide the `--run` argument, it downloads a pre-built Docker image (`joshcu/ngiab-cal:demo`) to run the calibration.
## Limitations
ngiab-cal has several important limitations to be aware of:
1. **Limited Calibration Algorithms**: Currently only configured to use the Dynamically Dimensioned Search (DDS) algorithm. Other algorithms available in the main ngen-cal branch are not supported.
2. **Limited Model Support**: Only calibrates parameters for CFE (Conceptual Functional Equivalent) and NoahOWP-Modular hydrologic models. Other models in the NextGen framework are not currently supported.
3. **Single-Gage Calibration**: Designed for calibrating single-basin models using one USGS streamgage. Multi-gage or multi-objective calibration is not supported.
4. **Custom ngen-cal Branch**: Uses a modified version of ngen-cal from the `ngiab_cal` branch, which differs from the main branch. Features from newer releases of ngen-cal may not be available.
5. **Compatibility Concerns**: The customized ngen-cal implementation may not be compatible with future changes to either ngen-cal or the NGIAB framework.
## Development
Contributions welcome! Comprehensive development instructions coming soon
## Visualisation
During calibration, plots will be created inside `./calibration/Output/Calibration_Run/ngen_*******_worker/Plot_Iteration/`
After calibration, the tool will copy the validated parameters back into the root config folder which allows you to run ngiab with the [guide script](https://github.com/CIROH-UA/NGIAB-CloudInfra/blob/main/guide.sh) and use the teehr / tethys visualiser.
More streamlined workflow coming soon.
## License
"Software code created by U.S. Government employees is not subject to copyright
in the United States (17 U.S.C. ยง105). The United States/Department of Commerce
reserve all rights to seek and obtain copyright protection in countries other
than the United States for Software authored in its entirety by the Department
of Commerce. To this end, the Department of Commerce hereby grants to Recipient
a royalty-free, nonexclusive license to use, copy, and create derivative works
of the Software outside of the United States."
## Acknowledgments
- [CIROH](https://docs.ciroh.org/) for NextGen In A Box
- [NGEN-CAL](https://github.com/NOAA-OWP/ngen-cal) developers
Raw data
{
"_id": null,
"home_page": null,
"name": "ngiab-cal",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.11",
"maintainer_email": null,
"keywords": "NGIAB, NextGen, hydrologcal modeling, model calibration, ngen-cal, CFE, Noah-OWP, Docker",
"author": null,
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/8c/1b/21350f9692164078c67c34338ca4db2a421765cd20c33c8f4591fef8de96/ngiab_cal-0.3.2.tar.gz",
"platform": null,
"description": "# ngiab-cal\n\nA Python CLI tool to simplify hydrologic model calibration for NextGen In A Box (NGIAB) workflows.\n\n## Table of Contents\n- [What is this?](#what-is-this)\n- [Installation](#installation)\n- [Requirements](#requirements)\n- [Basic Usage](#basic-usage)\n- [Advanced Options](#advanced-options)\n- [Calibration Process](#calibration-process)\n- [Calibration Configuration File](#calibration-configuration-file)\n- [Example: Calibrating CAMELS Basins](#example-calibrating-camels-basins)\n- [How It Works](#how-it-works)\n- [How is ngen-cal running?](#how-is-ngen-cal-running)\n- [Development](#development)\n- [License](#license)\n- [Acknowledgments](#acknowledgments)\n\n## What is this?\n\nngiab-cal is a utility that works with the [NGIAB folder structure](https://docs.ciroh.org/training-NGIAB-101/data-preparation.html#nextgen-run-directory-structure-ngen-run). It automates the creation of a calibration directory with all necessary configuration files to run a modified version of [ngen-cal](https://github.com/CIROH-UA/ngen-cal/tree/ngiab_cal).\n\nThe tool simplifies these key tasks:\n- Creating calibration configurations\n- Running the calibration process using Docker\n- Copying calibrated parameters back to your model configuration\n\n## Installation\n\nSeveral installation options are available:\n\n### Using `uvx` (recommended: for its speed,and efficient environment management)\n```bash\n# Run directly without installation\nuvx ngiab-cal --help\n\n# Or install as a tool\nuv tool install ngiab-cal\nngiab-cal --help\n```\n\n### Using `pipx`\n```bash\npipx install ngiab-cal\nngiab-cal --help\n```\n\n### Traditional pip installation\n```bash\npip install ngiab-cal\npython -m ngiab_cal --help\n```\n\n## Requirements\n\n- Python 3.10+\n- Docker (for running calibrations)\n- Internet connection (for downloading USGS data)\n\n## Basic Usage\n\n```bash\n# Create calibration configuration\nngiab-cal /path/to/ngiab/data/folder -g USGS_GAGE_ID\n\n# Create and run calibration (200 iterations)\nngiab-cal /path/to/ngiab/data/folder -g USGS_GAGE_ID --run -i 200\n\n# Force recreation of calibration configuration\nngiab-cal /path/to/ngiab/data/folder -g USGS_GAGE_ID -f\n```\n\n## Advanced Options\n\n```\nusage: ngiab-cal [-h] [-g GAGE] [-f] [--run] [-i ITERATIONS] [--debug] [-w WARMUP] [--calibration_ratio CALIBRATION_RATIO]\n data_folder\n\nCreate a calibration config for ngen-cal\n\npositional arguments:\n data_folder Path to the folder you wish to calibrate\n\noptions:\n -h, --help show this help message and exit\n -g GAGE, --gage GAGE Gage ID to use for calibration\n -f, --force Overwrite existing configuration\n --run Try to automatically run the calibration, this may be unstable\n -i ITERATIONS, --iterations ITERATIONS\n Default:100 number of iterations to calibrate for\n\n --debug enable debug logging\n -w WARMUP, --warmup WARMUP\n Default:365\n Number of days at the beginning of the simulation\n to exclude from calibration objective metric calculation\n --calibration_ratio CALIBRATION_RATIO, --cr CALIBRATION_RATIO\n Default:0.5\n How to split time after warmup into calibration and validation.\n 1 == 100% calibration, 0 == 100% validation, 0.8 == 80% calibration 20% validation\n```\n\n## Calibration Process\n\nThe tool applies a standard hydrological modeling workflow, which involves warmup, calibration, and validation periods. The --warmup period is crucial for allowing the model to reach a stable state before its performance is evaluated against observed data. Following the warmup, the remaining period is typically divided into calibration (where model parameters are adjusted to match observations) and validation (where the model's performance with the calibrated parameters is tested on an independent dataset). The tool facilitates this split, as detailed in the diagram and options below. \n\n```\nDefault calibration settings on a 5 year period\n| year 1 | year 2 | year 3 | year 4 | year 5 |\n|<- warmup ->|<- calibration ->| |\n|<- warmup ->|<- validation ->|\n```\n\n## Example: Calibrating CAMELS Basins\n\nHere's a script to calibrate all CAMELS basins:\n\n```bash\n#!/bin/bash\n# Download a list of CAMELS gage ids\nwget https://raw.githubusercontent.com/peckhams/nextgen_basin_repo/5e1317256a9365ae3a24a250358314e1e9ffc339/CAMELS/Data/camels_name.txt ./camels_name.txt\noutput_folder=$(cat ~/.ngiab/preprocessor)\nwhile read line\ndo\n gage=$(echo \"$line\" | cut -d ';' -f 1)\n echo $gage\n # subset the hydrofabric, calculate mean-average area forcings, generate model config files\n uvx --from ngiab_data_preprocess cli -i gage-\"$gage\" -sfr --start 2007-10-01 --end 2013-09-30\n # calibrate gage for 200 iterations\n uvx ngiab-cal \"$output_folder\"/gage-\"$gage\" -g \"$gage\" --run -i 200\ndone < <(tail -n +2 ./camels_name.txt)\n```\n\n## Calibration Configuration File\n\nngiab-cal generates a configuration file at `calibration/ngen_cal_conf.yaml` that controls the calibration process. This file is created from a template with the following key sections:\n\n### General Configuration\n```yaml\ngeneral:\n strategy:\n type: estimation\n algorithm: dds # Uses Dynamically Dimensioned Search algorithm\n name: calib # Don't modify this\n log: true # Enable logging\n workdir: /ngen/ngen/data/calibration # Don't modify this working directory in the Docker container\n yaml_file: /ngen/ngen/data/calibration/ngen_cal_conf.yaml # Don't modify this either\n iterations: 100 # Number of calibration iterations (customizable with -i flag)\n restart: 0 # Start from beginning (0) or resume from iteration\n```\n\n### Model Parameters\nThe file defines parameters for both CFE (Conceptual Functional Equivalent) and Noah-OWP-Modular (an extended, refactored version of the Noah-MP land surface model) hydrologic models:\n\n```yaml\nCFE:\n - name: b # CFE parameter name\n min: 2.0 # Minimum allowed value\n max: 15.0 # Maximum allowed value\n init: 4.05 # Initial value\n - name: satpsi\n min: 0.03\n max: 0.955\n init: 0.355\n # Additional parameters...\n```\n\nFor each parameter, the configuration specifies:\n- `name`: Parameter identifier\n- `min`: Minimum allowed value during calibration\n- `max`: Maximum allowed value during calibration\n- `init`: Initial value\n\n### Evaluation Configuration\n```yaml\neval_params:\n objective: kge # Kling-Gupta Efficiency as objective function\n evaluation_start: \"...\" # Start time for calibration period\n evaluation_stop: \"...\" # End time for calibration period\n valid_start_time: \"...\" # Start time including warmup\n valid_end_time: \"...\" # End time of simulation\n # Additional time parameters...\n basinID: 01646500 # USGS gage ID\n site_name: \"USGS 01646500: \" # Label for plots\n```\n\nThese time periods follow the calibration process diagram shown earlier, with separate periods for warmup, calibration, and validation.\n\n## How It Works\n\n1. **Input Validation** - Checks that all required files are present\n2. **USGS Data Download** - Retrieves observed streamflow for calibration\n3. **Configuration Creation** - Generates necessary files for ngen-cal\n4. **Docker Execution** - Runs the calibration in a containerized environment\n5. **Parameter Extraction** - Copies calibrated parameters back to your configuration\n\n## How is ngen-cal running?\n\nThe tool uses a custom [branch of CIROH-UA/ngen-cal](https://github.com/CIROH-UA/ngen-cal/tree/ngiab_cal) called `ngiab_cal`. This branch contains:\n- A modified version of ngen-cal\n- A Dockerfile that builds on top of the Next Gen In-A-Box container\n- Installation of ngen-cal inside the container\n\nWhen you provide the `--run` argument, it downloads a pre-built Docker image (`joshcu/ngiab-cal:demo`) to run the calibration.\n\n## Limitations\n\nngiab-cal has several important limitations to be aware of:\n\n1. **Limited Calibration Algorithms**: Currently only configured to use the Dynamically Dimensioned Search (DDS) algorithm. Other algorithms available in the main ngen-cal branch are not supported.\n\n2. **Limited Model Support**: Only calibrates parameters for CFE (Conceptual Functional Equivalent) and NoahOWP-Modular hydrologic models. Other models in the NextGen framework are not currently supported.\n\n3. **Single-Gage Calibration**: Designed for calibrating single-basin models using one USGS streamgage. Multi-gage or multi-objective calibration is not supported.\n\n4. **Custom ngen-cal Branch**: Uses a modified version of ngen-cal from the `ngiab_cal` branch, which differs from the main branch. Features from newer releases of ngen-cal may not be available.\n\n5. **Compatibility Concerns**: The customized ngen-cal implementation may not be compatible with future changes to either ngen-cal or the NGIAB framework.\n\n\n## Development\n\nContributions welcome! Comprehensive development instructions coming soon\n\n## Visualisation\nDuring calibration, plots will be created inside `./calibration/Output/Calibration_Run/ngen_*******_worker/Plot_Iteration/`\nAfter calibration, the tool will copy the validated parameters back into the root config folder which allows you to run ngiab with the [guide script](https://github.com/CIROH-UA/NGIAB-CloudInfra/blob/main/guide.sh) and use the teehr / tethys visualiser.\nMore streamlined workflow coming soon.\n\n\n## License\n\n\"Software code created by U.S. Government employees is not subject to copyright\nin the United States (17 U.S.C. \u00a7105). The United States/Department of Commerce\nreserve all rights to seek and obtain copyright protection in countries other\nthan the United States for Software authored in its entirety by the Department\nof Commerce. To this end, the Department of Commerce hereby grants to Recipient\na royalty-free, nonexclusive license to use, copy, and create derivative works\nof the Software outside of the United States.\"\n\n## Acknowledgments\n \n- [CIROH](https://docs.ciroh.org/) for NextGen In A Box\n- [NGEN-CAL](https://github.com/NOAA-OWP/ngen-cal) developers\n",
"bugtrack_url": null,
"license": null,
"summary": "A Python CLI tool to simplify hydrologic model calibration for NextGen In A Box (NGIAB) workflows.",
"version": "0.3.2",
"project_urls": null,
"split_keywords": [
"ngiab",
" nextgen",
" hydrologcal modeling",
" model calibration",
" ngen-cal",
" cfe",
" noah-owp",
" docker"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "c3ccf2567c2a82375af6ad553dbcd8d5f62644a2831b0eeb40a5fb56b99319d0",
"md5": "756f041afaed4474e7ac3d173e8bfb21",
"sha256": "5a405e4c4e68b86fc5f91ba3523f7abc40e388a1f0811ce8711619c3b16700d4"
},
"downloads": -1,
"filename": "ngiab_cal-0.3.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "756f041afaed4474e7ac3d173e8bfb21",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11",
"size": 13615,
"upload_time": "2025-07-14T17:19:34",
"upload_time_iso_8601": "2025-07-14T17:19:34.327139Z",
"url": "https://files.pythonhosted.org/packages/c3/cc/f2567c2a82375af6ad553dbcd8d5f62644a2831b0eeb40a5fb56b99319d0/ngiab_cal-0.3.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "8c1b21350f9692164078c67c34338ca4db2a421765cd20c33c8f4591fef8de96",
"md5": "19d6bc36e60345e16a565ad0f9ea825b",
"sha256": "6e8da5f7946d3ac0681be7d71b3a4d5966dd959394dfc4d53407e19e75190f77"
},
"downloads": -1,
"filename": "ngiab_cal-0.3.2.tar.gz",
"has_sig": false,
"md5_digest": "19d6bc36e60345e16a565ad0f9ea825b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11",
"size": 60127,
"upload_time": "2025-07-14T17:19:35",
"upload_time_iso_8601": "2025-07-14T17:19:35.160297Z",
"url": "https://files.pythonhosted.org/packages/8c/1b/21350f9692164078c67c34338ca4db2a421765cd20c33c8f4591fef8de96/ngiab_cal-0.3.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-14 17:19:35",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "ngiab-cal"
}
JSON