| Name | metget JSON |
| Version |
0.6.0
JSON |
| download |
| home_page | |
| Summary | A client package for interaction with a MetGet server instance |
| upload_time | 2024-03-05 00:09:06 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.8 |
| license | MIT |
| keywords |
forecast
meteorology
stormsurge
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
MetGet is an application which allows users to query, format, and blend meteorological data from various sources
to be used in hydrodynamic modeling applications.
[](https://opensource.org/licenses/MIT)
[](https://badge.fury.io/py/metget)
[](https://anaconda.org/conda-forge/metget)
[](https://anaconda.org/conda-forge/metget)
[](https://codecov.io/gh/waterinstitute/MetGet)
[](https://pepy.tech/project/metget)
## Development Partners
MetGet is developed by [The Water Institute](https://thewaterinstitute.org) and has been funded by the
University of North Carolina at Chapel Hill [Coastal Resilience Center of Excellence](https://www.coastalresiliencecenter.org).
## MetGet Client Application
The MetGet client application is a tool that allows you to interact with the MetGet server. It may be used
as a traditional command line tool or the `MetGetBuildRest` class may be imported and run in a custom application.
The MetGet client handles the interaction with the server via RESTful API calls and downloading output
data from S3 buckets. The client application is not a requirement for using MetGet, but it is a convenient
way to interact with the server.
## MetGet Server Access
The `metget` client application interacts with an instance of a [metget-server](https://github.com/waterinstitute/metget-server),
which is a Kubernetes application that archives, builds, and provides to users meteorological data through the client application.
The client does not require that you use any specific MetGet server instance. Instead, you may choose to operate your own
`metget-server` instance or you may [contact](mailto:zcobell@thewaterinstitute.org) The Water Institute to request access to
an available instance.
### Installation
The MetGet client application can be installed using from the PyPi repository or via Anaconda using the
`conda-forge` repository. The below commands show how to install the client and its dependencies.
Using PyPi:
```bash
$ pip3 install metget
```
Using Anaconda:
```bash
$ conda install -c conda-forge metget
```
### Terminology
There are several terms used in the MetGet client application that are important to understand. The below list of terms
aims to explain these terms and why they are available as options in the client application.
1. `analysis` - The term analysis is synonymous with the term "nowcast". It is the time when the model is initialized, or the "zero" hour of a model run. Requesting that MetGet provide "analysis" data via the `--analysis` command line option will fore MetGet to return only zero hour for any forecast runs. This option is incompatible with "--multiple-forecasts" and "--initialization-skip".
2. `multiple-forecasts` - This option is used to signify that the user wants to request multiple forecast hours from the model. This allows MetGet to string together different forecasts to create quasi-hindcast data. The selection of available data is done by selecting unique times with the smallest offset from the start of their respective forecasts. This option is incompatible with `--analysis`.
3. `initialization-skip` - In some instances, it may be useful to skip the first [n] hours of a particular forecast when there is a potentially severe initialization that occurs in the atmospheric model. This option instructs MetGet to ignore the first [n] hours of a forecast.
### Environment Variables
There are three influential environment variables which can be set as a convenience to the user. These variables are:
* `METGET_API_KEY` - The API key used to authenticate with MetGet
* `METGET_ENDPOINT` - The URL of the MetGet server, i.e. `https://metget.server.org`
* `METGET_API_VERSION` - The version of the MetGet API to use. The system will default to `2`.
### Usage
The client application can request multiple types of data from the server. The below examples show how to request
data from a MetGet server instance via the command line tool.
Note: In the below examples, we assume that the above environment variables have been set. If they have not been set,
you will need to pass additional command line flags.
#### Example 1 - Request GFS data
This example requests data from the GFS for a two-day period. We will generate the data from a single self-consistent forecast
formatted as a single NetCDF file. The data will be interpolated to a 0.25 degree grid and cover a portion of the Gulf of Mexico.
```bash
$ metget build --domain gfs 0.25 -100 10 -80 30 \
--start "2023-06-01 00:00" \
--end "2023-06-03 00:00" \
--timestep 3600 \
--output metget_gfs_data.nc \
--format generic-netcdf
```
#### Example 2 - Request GFS data with multiple forecasts
This example requests data from the GFS for a thirty-day period. We will generate the data from multiple forecasts as a
quasi-hindcast. The data will be interpolated to a 0.25 degree grid and cover a portion of the Gulf of Mexico.
```bash
$ metget build --domain gfs 0.25 -100 10 -80 30 \
--start "2023-05-01 00:00" \
--end "2023-06-01 00:00" \
--output metget_gfs_data.nc \
--format generic-netcdf \
--timestep 3600 \
--multiple-forecasts
```
#### Example 3 - Request GFS data with multiple forecasts and initialization skip
This example requests data from the GFS for a thirty-day period. We will generate the data from multiple forecasts as a
quasi-hindcast. The data will be interpolated to a 0.25 degree grid and cover a portion of the Gulf of Mexico. We will
skip the first 6 hours of each forecast.
```bash
$ metget build --domain gfs 0.25 -100 10 -80 30 \
--start "2023-05-01 00:00" \
--end "2023-06-01 00:00" \
--timestep 3600 \
--format generic-netcdf \
--multiple-forecasts \
--initialization-skip 6 \
--output metget_gfs_data.nc
```
#### Example 4 - Request HWRF data overlaid on GFS data
This example requests data from the HWRF model (Hurricane Mawar) with background forcing from the GFS model. Note that at the present time,
the output format must support multi-domain data. Currently, the only format that supports this is `owi-ascii`. The
data will be interpolated to a 0.25 degree grid and cover a portion of the Gulf of Mexico.
```bash
$ metget build --domain hwrf-mawar02w 0.15 -90 20 -85 25 \
--domain gfs 0.25 -100 10 -80 30 \
--start "2023-06-01 00:00" \
--end "2023-06-03 00:00" \
--timestep 3600 \
--backfill \
--format owi-ascii \
--output metget_hwrf_data
```
#### Example 5 - Get the status of the GFS model runs
This example demonstrates the ability for the system to provide information about what data is currently available
```bash
$ metget status gfs --format pretty
Status for model: GFS (class: synoptic)
+---------------------+---------------------+------------------+
| Forecast Cycle | End Time | Status |
+---------------------+---------------------+------------------+
| 2023-07-06 18:00:00 | 2023-07-17 09:00:00 | incomplete (255) |
| 2023-07-06 12:00:00 | 2023-07-22 12:00:00 | complete |
| 2023-07-06 06:00:00 | 2023-07-22 06:00:00 | complete |
| 2023-07-06 00:00:00 | 2023-07-22 00:00:00 | complete |
| 2023-07-05 18:00:00 | 2023-07-21 18:00:00 | complete |
| 2023-07-05 12:00:00 | 2023-07-21 12:00:00 | complete |
| 2023-07-05 06:00:00 | 2023-07-21 06:00:00 | complete |
| 2023-07-05 00:00:00 | 2023-07-21 00:00:00 | complete |
| 2023-07-04 18:00:00 | 2023-07-20 18:00:00 | complete |
| 2023-07-04 12:00:00 | 2023-07-20 12:00:00 | complete |
| 2023-07-04 06:00:00 | 2023-07-20 06:00:00 | complete |
| 2023-07-04 00:00:00 | 2023-07-20 00:00:00 | complete |
+---------------------+---------------------+------------------+
```
#### Example 6: Retrieve GEOJSON track data for a specific storm
This example demonstrates the ability to retrieve track data for a specific storm. The track data is returned in GEOJSON format.
```bash
$ metget track --year 2022 --storm 9 --basin al --type forecast --advisory 3 #...Hurricane Ian 2022, Advisory 3 track data
$ metget track --year 2022 --storm 9 --basin al --type besttrack #...Hurricane Ian 2022, best track data
```
### Custom Applications
The MetGet client library (`MetGetBuildRest`) can be imported and used in custom applications. The below example shows how to use the
client application in a custom Python script. Below is a simple version of how the main client application interacts with the `metget-client` library.
You can use this as a starting point for your own custom applications.
```python
import argparse
from metget.metget_build import MetGetBuildRest
metget_server = "https://metget.server.org"
metget_api_key = "my-api-key"
metget_api_version = 2
args = argparse.Namespace
output_format = 'owi-ascii'
# ...Building the request
request_data = MetGetBuildRest.generate_request_json(
analysis=args.analysis,
multiple_forecasts=args.multiple_forecasts,
start_date=args.start,
end_date=args.end,
format=output_format,
data_type=args.variable,
backfill=args.backfill,
time_step=args.timestep,
domains=MetGetBuildRest.parse_command_line_domains(
args.domain, args.initialization_skip
),
compression=args.compression,
epsg=args.epsg,
filename=args.output,
strict=args.strict,
dry_run=args.dryrun,
save_json_request=args.save_json_request,
)
client = MetGetBuildRest(metget_server, metget_api_key, metget_api_version)
data_id, status_code = client.make_metget_request(request_data)
client.download_metget_data(
data_id,
args.check_interval,
args.max_wait,
)
```
Raw data
{
"_id": null,
"home_page": "",
"name": "metget",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "forecast,meteorology,stormsurge",
"author": "",
"author_email": "Zach Cobell <zcobell@thewaterinstitute.org>",
"download_url": "https://files.pythonhosted.org/packages/79/93/aeb52f86133591da232a3846d21bdb7a971e76e545e8648b6425f787827f/metget-0.6.0.tar.gz",
"platform": null,
"description": "\nMetGet is an application which allows users to query, format, and blend meteorological data from various sources\nto be used in hydrodynamic modeling applications.\n\n[](https://opensource.org/licenses/MIT)\n[](https://badge.fury.io/py/metget)\n[](https://anaconda.org/conda-forge/metget)\n[](https://anaconda.org/conda-forge/metget)\n\n[](https://codecov.io/gh/waterinstitute/MetGet)\n[](https://pepy.tech/project/metget)\n\n\n## Development Partners\n\nMetGet is developed by [The Water Institute](https://thewaterinstitute.org) and has been funded by the\nUniversity of North Carolina at Chapel Hill [Coastal Resilience Center of Excellence](https://www.coastalresiliencecenter.org).\n\n\n\n\n\n\n\n## MetGet Client Application\nThe MetGet client application is a tool that allows you to interact with the MetGet server. It may be used\nas a traditional command line tool or the `MetGetBuildRest` class may be imported and run in a custom application.\n\nThe MetGet client handles the interaction with the server via RESTful API calls and downloading output\ndata from S3 buckets. The client application is not a requirement for using MetGet, but it is a convenient\nway to interact with the server.\n\n## MetGet Server Access\nThe `metget` client application interacts with an instance of a [metget-server](https://github.com/waterinstitute/metget-server),\nwhich is a Kubernetes application that archives, builds, and provides to users meteorological data through the client application.\nThe client does not require that you use any specific MetGet server instance. Instead, you may choose to operate your own\n`metget-server` instance or you may [contact](mailto:zcobell@thewaterinstitute.org) The Water Institute to request access to\nan available instance.\n\n### Installation\n\nThe MetGet client application can be installed using from the PyPi repository or via Anaconda using the\n`conda-forge` repository. The below commands show how to install the client and its dependencies.\n\nUsing PyPi:\n```bash\n$ pip3 install metget\n```\n\nUsing Anaconda:\n```bash\n$ conda install -c conda-forge metget\n```\n\n### Terminology\nThere are several terms used in the MetGet client application that are important to understand. The below list of terms\naims to explain these terms and why they are available as options in the client application.\n\n1. `analysis` - The term analysis is synonymous with the term \"nowcast\". It is the time when the model is initialized, or the \"zero\" hour of a model run. Requesting that MetGet provide \"analysis\" data via the `--analysis` command line option will fore MetGet to return only zero hour for any forecast runs. This option is incompatible with \"--multiple-forecasts\" and \"--initialization-skip\".\n2. `multiple-forecasts` - This option is used to signify that the user wants to request multiple forecast hours from the model. This allows MetGet to string together different forecasts to create quasi-hindcast data. The selection of available data is done by selecting unique times with the smallest offset from the start of their respective forecasts. This option is incompatible with `--analysis`.\n3. `initialization-skip` - In some instances, it may be useful to skip the first [n] hours of a particular forecast when there is a potentially severe initialization that occurs in the atmospheric model. This option instructs MetGet to ignore the first [n] hours of a forecast.\n\n### Environment Variables\nThere are three influential environment variables which can be set as a convenience to the user. These variables are:\n* `METGET_API_KEY` - The API key used to authenticate with MetGet\n* `METGET_ENDPOINT` - The URL of the MetGet server, i.e. `https://metget.server.org`\n* `METGET_API_VERSION` - The version of the MetGet API to use. The system will default to `2`.\n\n### Usage\n\nThe client application can request multiple types of data from the server. The below examples show how to request\ndata from a MetGet server instance via the command line tool.\n\nNote: In the below examples, we assume that the above environment variables have been set. If they have not been set,\nyou will need to pass additional command line flags.\n\n#### Example 1 - Request GFS data\nThis example requests data from the GFS for a two-day period. We will generate the data from a single self-consistent forecast\nformatted as a single NetCDF file. The data will be interpolated to a 0.25 degree grid and cover a portion of the Gulf of Mexico.\n\n```bash\n$ metget build --domain gfs 0.25 -100 10 -80 30 \\\n --start \"2023-06-01 00:00\" \\\n --end \"2023-06-03 00:00\" \\\n --timestep 3600 \\\n --output metget_gfs_data.nc \\\n --format generic-netcdf\n```\n\n#### Example 2 - Request GFS data with multiple forecasts\nThis example requests data from the GFS for a thirty-day period. We will generate the data from multiple forecasts as a\nquasi-hindcast. The data will be interpolated to a 0.25 degree grid and cover a portion of the Gulf of Mexico.\n\n```bash\n$ metget build --domain gfs 0.25 -100 10 -80 30 \\\n --start \"2023-05-01 00:00\" \\\n --end \"2023-06-01 00:00\" \\\n --output metget_gfs_data.nc \\\n --format generic-netcdf \\\n --timestep 3600 \\\n --multiple-forecasts\n```\n\n#### Example 3 - Request GFS data with multiple forecasts and initialization skip\nThis example requests data from the GFS for a thirty-day period. We will generate the data from multiple forecasts as a\nquasi-hindcast. The data will be interpolated to a 0.25 degree grid and cover a portion of the Gulf of Mexico. We will\nskip the first 6 hours of each forecast.\n\n```bash\n$ metget build --domain gfs 0.25 -100 10 -80 30 \\\n --start \"2023-05-01 00:00\" \\\n --end \"2023-06-01 00:00\" \\\n --timestep 3600 \\\n --format generic-netcdf \\\n --multiple-forecasts \\\n --initialization-skip 6 \\\n --output metget_gfs_data.nc\n```\n\n#### Example 4 - Request HWRF data overlaid on GFS data\nThis example requests data from the HWRF model (Hurricane Mawar) with background forcing from the GFS model. Note that at the present time,\nthe output format must support multi-domain data. Currently, the only format that supports this is `owi-ascii`. The\ndata will be interpolated to a 0.25 degree grid and cover a portion of the Gulf of Mexico.\n\n```bash\n$ metget build --domain hwrf-mawar02w 0.15 -90 20 -85 25 \\\n --domain gfs 0.25 -100 10 -80 30 \\\n --start \"2023-06-01 00:00\" \\\n --end \"2023-06-03 00:00\" \\\n --timestep 3600 \\\n --backfill \\\n --format owi-ascii \\\n --output metget_hwrf_data\n```\n\n#### Example 5 - Get the status of the GFS model runs\nThis example demonstrates the ability for the system to provide information about what data is currently available\n\n```bash\n$ metget status gfs --format pretty\nStatus for model: GFS (class: synoptic)\n+---------------------+---------------------+------------------+\n| Forecast Cycle | End Time | Status |\n+---------------------+---------------------+------------------+\n| 2023-07-06 18:00:00 | 2023-07-17 09:00:00 | incomplete (255) |\n| 2023-07-06 12:00:00 | 2023-07-22 12:00:00 | complete |\n| 2023-07-06 06:00:00 | 2023-07-22 06:00:00 | complete |\n| 2023-07-06 00:00:00 | 2023-07-22 00:00:00 | complete |\n| 2023-07-05 18:00:00 | 2023-07-21 18:00:00 | complete |\n| 2023-07-05 12:00:00 | 2023-07-21 12:00:00 | complete |\n| 2023-07-05 06:00:00 | 2023-07-21 06:00:00 | complete |\n| 2023-07-05 00:00:00 | 2023-07-21 00:00:00 | complete |\n| 2023-07-04 18:00:00 | 2023-07-20 18:00:00 | complete |\n| 2023-07-04 12:00:00 | 2023-07-20 12:00:00 | complete |\n| 2023-07-04 06:00:00 | 2023-07-20 06:00:00 | complete |\n| 2023-07-04 00:00:00 | 2023-07-20 00:00:00 | complete |\n+---------------------+---------------------+------------------+\n```\n\n#### Example 6: Retrieve GEOJSON track data for a specific storm\nThis example demonstrates the ability to retrieve track data for a specific storm. The track data is returned in GEOJSON format.\n\n```bash\n$ metget track --year 2022 --storm 9 --basin al --type forecast --advisory 3 #...Hurricane Ian 2022, Advisory 3 track data\n$ metget track --year 2022 --storm 9 --basin al --type besttrack #...Hurricane Ian 2022, best track data\n```\n\n### Custom Applications\nThe MetGet client library (`MetGetBuildRest`) can be imported and used in custom applications. The below example shows how to use the\nclient application in a custom Python script. Below is a simple version of how the main client application interacts with the `metget-client` library.\nYou can use this as a starting point for your own custom applications.\n\n```python\nimport argparse\nfrom metget.metget_build import MetGetBuildRest\n\nmetget_server = \"https://metget.server.org\"\nmetget_api_key = \"my-api-key\"\nmetget_api_version = 2\n\nargs = argparse.Namespace\noutput_format = 'owi-ascii'\n\n# ...Building the request\nrequest_data = MetGetBuildRest.generate_request_json(\n analysis=args.analysis,\n multiple_forecasts=args.multiple_forecasts,\n start_date=args.start,\n end_date=args.end,\n format=output_format,\n data_type=args.variable,\n backfill=args.backfill,\n time_step=args.timestep,\n domains=MetGetBuildRest.parse_command_line_domains(\n args.domain, args.initialization_skip\n ),\n compression=args.compression,\n epsg=args.epsg,\n filename=args.output,\n strict=args.strict,\n dry_run=args.dryrun,\n save_json_request=args.save_json_request,\n)\n\nclient = MetGetBuildRest(metget_server, metget_api_key, metget_api_version)\ndata_id, status_code = client.make_metget_request(request_data)\nclient.download_metget_data(\n data_id,\n args.check_interval,\n args.max_wait,\n)\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A client package for interaction with a MetGet server instance",
"version": "0.6.0",
"project_urls": {
"Bug Reports": "https://github.com/waterinstitute/metget/issues",
"Homepage": "https://github.com/waterinstitute/metget",
"Source": "https://github.com/waterinstitute/metget"
},
"split_keywords": [
"forecast",
"meteorology",
"stormsurge"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "88fc042c89700ac546a309f219fd8312da83e67b6d7521802992acca3e636712",
"md5": "f79204552d3b92ea1fd03d64fa1a29b9",
"sha256": "d0eabab0fd58f4d43cf563e308ab5058bd5afc0dafda5cc28f772cc5650222e5"
},
"downloads": -1,
"filename": "metget-0.6.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f79204552d3b92ea1fd03d64fa1a29b9",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 26057,
"upload_time": "2024-03-05T00:09:04",
"upload_time_iso_8601": "2024-03-05T00:09:04.742220Z",
"url": "https://files.pythonhosted.org/packages/88/fc/042c89700ac546a309f219fd8312da83e67b6d7521802992acca3e636712/metget-0.6.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7993aeb52f86133591da232a3846d21bdb7a971e76e545e8648b6425f787827f",
"md5": "5bb7c2114ed98617a5886216192dfc35",
"sha256": "b23f8310c9751fe9c161769271480f4c9b1b4a08f72060237ac475f41f2aeb1c"
},
"downloads": -1,
"filename": "metget-0.6.0.tar.gz",
"has_sig": false,
"md5_digest": "5bb7c2114ed98617a5886216192dfc35",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 26832,
"upload_time": "2024-03-05T00:09:06",
"upload_time_iso_8601": "2024-03-05T00:09:06.562691Z",
"url": "https://files.pythonhosted.org/packages/79/93/aeb52f86133591da232a3846d21bdb7a971e76e545e8648b6425f787827f/metget-0.6.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-05 00:09:06",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "waterinstitute",
"github_project": "metget",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "metget"
}
