Forecast verification software
==============================
.. image:: https://img.shields.io/github/v/release/WFRT/verif.svg
:target: https://github.com/WFRT/verif/releases
.. image:: https://github.com/WFRT/verif/actions/workflows/python-package.yml/badge.svg
:target: https://github.com/WFRT/verif/actions/workflows/python-package.yml
Verif is a command-line tool that lets you verify the quality of weather forecasts for point
locations. It can also compare forecasts from different forecasting systems (that have different
models, post-processing methods, etc).
The program reads files with observations and forecasts in a specific format (see "Input files"
below). The input files contain information about dates, forecast lead times, and locations such
that statistics can be aggregated across different dimensions. To ensure a fair comparison among
files, Verif will discard data points where one or more forecast systems have missing forecasts.
Since Verif is a command-line tool, it can be used in scripts to automatically create
verification figures.
Verif version 1.2 has been released (see "Installation Instruction" below). We welcome suggestions
for improvements. Verif is developed by Thomas Nipen (thomasn@met.no), with contributions from many.
Resources
---------
For more information on how to use Verif, **check out the wiki** at https://github.com/WFRT/verif/wiki.
Found a bug? Please report it in the **issue tracker** at https://github.com/WFRT/verif/issues. Reach
out to the Verif community in the **discussions** at https://github.com/WFRT/verif/discussions.
Features
--------
* Deterministic metrics such as MAE, bias, correlation, RMSE (e.g. ``-m mae``)
* Threshold-based metrics such as the false alarm rate, ETS, EDI, Yule's Q (e.g. ``-m ets``)
* Probabilistic metrics such as brier score, PIT-histogram, reliability diagrams (e.g. ``-m bs``)
* Special plots like Taylor diagrams (``-m taylor``), quantile-quantile plots (``-m qq``).
* Plot scores as a function of date, lead time, station altitude/lat/longitude (e.g. ``-x date``)
* Show scores on maps (``-type map``)
* Subset the data by specifying a date range and lat/lon range (``-latrange 58,60``)
* Export to text (``-type text``)
* Options to adjust font sizes, label positions, tick marks, legends, etc (``-labfs 14``)
* Anomaly statistics relative to a baseline like climatology (``-c climfile.txt``)
* Output to png, jpeg, eps, etc and specify dimensions and resolution (``-f image.png -dpi 300``)
.. image:: other/image.jpg
:alt: Example plots
:width: 400
:align: center
Installing on Ubuntu
--------------------
**Prerequisites**
Verif requires a pip installation and Python 3. The python package "cartopy" is optional, but provides
a background map when verification scores are plotted on a map. To install Cartopy, with its GEOS and
PROJ4 dependencies, do the following:
.. code-block:: bash
sudo apt-get update
sudo apt-get install libgeos-dev libproj-dev
sudo pip3 install cartopy
**Installing using pip**
After this, the easiest is to install the lastest version of Verif using pip:
.. code-block:: bash
sudo pip install verif
Verif should then be accessible by typing ``verif`` on the command-line. If you do not have
sudo-rights, then install verif as follows:
.. code-block:: bash
pip install verif --user
This will create the executable ``~/.local/bin/verif``. Add this to your PATH environment
variable if necessary (i.e add ``export PATH=$PATH:~/.local/bin`` to ``~/.bashrc``).
To upgrade to a newer version of Verif, run the following:
.. code-block:: bash
pip install verif --upgrade
**Installing from source**
Alternatively, to install from source, download the source code of the latest version:
https://github.com/WFRT/verif/releases/. Unzip the file and navigate into the extracted folder.
Then install Verif by executing the following inside the extracted folder:
.. code-block:: bash
sudo pip install -r requirements.txt
sudo python setup.py install
This will create the executable ``/usr/local/bin/verif``. Add ``/usr/local/bin`` to your PATH environment
variable if necessary. If you do not have sudo privileges do:
.. code-block:: bash
pip install -r requirements.txt --user
python setup.py install --user
This will create the executable ``~/.local/bin/verif``. Add ``~/.local/bin`` to your PATH environment
variable.
Installing on Mac OSX
---------------------
Follow the proceedure as for Ubuntu (either installing with pip or from source). If installing from
source, then look for the line "Installing verif script to <some directory>", as this will indicate
what folder Verif is installed into. Add the folder to your PATH environment variable if necessary.
Examples
--------
To test Verif, you can download example datasets from the github
`discussion page <https://github.com/WFRT/verif/discussions>`_. For example, download the following two files from the wind speed dataset: `MEPS.nc <https://thredds.met.no/thredds/fileServer/metusers/thomasn/verif_datasets/short_range_wind/MEPS.nc>`_ (2.5 km regional model; 20MB file size) and `ECMWF.nc <https://thredds.met.no/thredds/fileServer/metusers/thomasn/verif_datasets/short_range_wind/ECMWF.nc>`_ (0.2° global model; 24MB file size). Then run the following
commands to test out the software:
.. code-block:: bash
# Shows mean absolute error as a function of lead-time
verif MEPS.nc ECMWF.nc -m mae
# Shows average observed and forecasted values as a function on time
verif MEPS.nc ECMWF.nc -m obsfcst -x time
# Shows equitable threat score as a function of threshold
verif MEPS.nc ECMWF.nc -m ets
# Shows a reliability diagram for a threshold of 13.9 m/s (gale force winds)
verif MEPS.nc ECMWF.nc -m reliability -r 13.9
# Shows Brier skill score as a function of threshold
verif MEPS.nc ECMWF.nc -m bss -x threshold
Copyright and license
---------------------
Copyright © 2013-2024 UBC Weather Forecast Research Team. Verif is licensed under the 3-clause
BSD license. See LICENSE file.
Raw data
{
"_id": null,
"home_page": "https://github.com/WFRT/verif",
"name": "verif",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "meteorology verification weather prediction",
"author": "Thomas Nipen",
"author_email": "thomas.nipen@met.no",
"download_url": "https://files.pythonhosted.org/packages/98/35/e531c9299a9d1a2e80fd4815a0719981fe0565b367f877db03c3359901f6/verif-1.3.0.tar.gz",
"platform": null,
"description": "Forecast verification software\n==============================\n.. image:: https://img.shields.io/github/v/release/WFRT/verif.svg\n :target: https://github.com/WFRT/verif/releases\n.. image:: https://github.com/WFRT/verif/actions/workflows/python-package.yml/badge.svg\n :target: https://github.com/WFRT/verif/actions/workflows/python-package.yml\n\nVerif is a command-line tool that lets you verify the quality of weather forecasts for point\nlocations. It can also compare forecasts from different forecasting systems (that have different\nmodels, post-processing methods, etc).\n\nThe program reads files with observations and forecasts in a specific format (see \"Input files\"\nbelow). The input files contain information about dates, forecast lead times, and locations such\nthat statistics can be aggregated across different dimensions. To ensure a fair comparison among\nfiles, Verif will discard data points where one or more forecast systems have missing forecasts.\nSince Verif is a command-line tool, it can be used in scripts to automatically create\nverification figures.\n\nVerif version 1.2 has been released (see \"Installation Instruction\" below). We welcome suggestions\nfor improvements. Verif is developed by Thomas Nipen (thomasn@met.no), with contributions from many.\n\nResources\n---------\n\nFor more information on how to use Verif, **check out the wiki** at https://github.com/WFRT/verif/wiki.\nFound a bug? Please report it in the **issue tracker** at https://github.com/WFRT/verif/issues. Reach\nout to the Verif community in the **discussions** at https://github.com/WFRT/verif/discussions.\n\nFeatures\n--------\n\n* Deterministic metrics such as MAE, bias, correlation, RMSE (e.g. ``-m mae``)\n* Threshold-based metrics such as the false alarm rate, ETS, EDI, Yule's Q (e.g. ``-m ets``)\n* Probabilistic metrics such as brier score, PIT-histogram, reliability diagrams (e.g. ``-m bs``)\n* Special plots like Taylor diagrams (``-m taylor``), quantile-quantile plots (``-m qq``).\n* Plot scores as a function of date, lead time, station altitude/lat/longitude (e.g. ``-x date``)\n* Show scores on maps (``-type map``)\n* Subset the data by specifying a date range and lat/lon range (``-latrange 58,60``)\n* Export to text (``-type text``)\n* Options to adjust font sizes, label positions, tick marks, legends, etc (``-labfs 14``)\n* Anomaly statistics relative to a baseline like climatology (``-c climfile.txt``)\n* Output to png, jpeg, eps, etc and specify dimensions and resolution (``-f image.png -dpi 300``)\n\n.. image:: other/image.jpg\n :alt: Example plots\n :width: 400\n :align: center\n\nInstalling on Ubuntu\n--------------------\n\n**Prerequisites**\n\nVerif requires a pip installation and Python 3. The python package \"cartopy\" is optional, but provides\na background map when verification scores are plotted on a map. To install Cartopy, with its GEOS and\nPROJ4 dependencies, do the following:\n\n.. code-block:: bash\n\n sudo apt-get update\n sudo apt-get install libgeos-dev libproj-dev\n sudo pip3 install cartopy\n\n**Installing using pip**\n\nAfter this, the easiest is to install the lastest version of Verif using pip:\n\n.. code-block:: bash\n\n sudo pip install verif\n\nVerif should then be accessible by typing ``verif`` on the command-line. If you do not have\nsudo-rights, then install verif as follows:\n\n.. code-block:: bash\n\n pip install verif --user\n\nThis will create the executable ``~/.local/bin/verif``. Add this to your PATH environment\nvariable if necessary (i.e add ``export PATH=$PATH:~/.local/bin`` to ``~/.bashrc``).\n\nTo upgrade to a newer version of Verif, run the following:\n\n.. code-block:: bash\n\n pip install verif --upgrade\n\n**Installing from source**\n\nAlternatively, to install from source, download the source code of the latest version:\nhttps://github.com/WFRT/verif/releases/. Unzip the file and navigate into the extracted folder.\n\nThen install Verif by executing the following inside the extracted folder:\n\n.. code-block:: bash\n\n sudo pip install -r requirements.txt\n sudo python setup.py install\n\nThis will create the executable ``/usr/local/bin/verif``. Add ``/usr/local/bin`` to your PATH environment\nvariable if necessary. If you do not have sudo privileges do:\n\n.. code-block:: bash\n\n pip install -r requirements.txt --user\n python setup.py install --user\n\nThis will create the executable ``~/.local/bin/verif``. Add ``~/.local/bin`` to your PATH environment\nvariable.\n\nInstalling on Mac OSX\n---------------------\n\nFollow the proceedure as for Ubuntu (either installing with pip or from source). If installing from\nsource, then look for the line \"Installing verif script to <some directory>\", as this will indicate\nwhat folder Verif is installed into. Add the folder to your PATH environment variable if necessary.\n\nExamples\n--------\nTo test Verif, you can download example datasets from the github\n`discussion page <https://github.com/WFRT/verif/discussions>`_. For example, download the following two files from the wind speed dataset: `MEPS.nc <https://thredds.met.no/thredds/fileServer/metusers/thomasn/verif_datasets/short_range_wind/MEPS.nc>`_ (2.5 km regional model; 20MB file size) and `ECMWF.nc <https://thredds.met.no/thredds/fileServer/metusers/thomasn/verif_datasets/short_range_wind/ECMWF.nc>`_ (0.2\u00b0 global model; 24MB file size). Then run the following\ncommands to test out the software:\n\n.. code-block:: bash\n\n # Shows mean absolute error as a function of lead-time\n verif MEPS.nc ECMWF.nc -m mae\n # Shows average observed and forecasted values as a function on time\n verif MEPS.nc ECMWF.nc -m obsfcst -x time\n # Shows equitable threat score as a function of threshold\n verif MEPS.nc ECMWF.nc -m ets\n # Shows a reliability diagram for a threshold of 13.9 m/s (gale force winds)\n verif MEPS.nc ECMWF.nc -m reliability -r 13.9\n # Shows Brier skill score as a function of threshold\n verif MEPS.nc ECMWF.nc -m bss -x threshold\n\nCopyright and license\n---------------------\n\nCopyright \u00a9 2013-2024 UBC Weather Forecast Research Team. Verif is licensed under the 3-clause\nBSD license. See LICENSE file.\n\n\n",
"bugtrack_url": null,
"license": "BSD-3",
"summary": "A verification program for meteorological forecasts and observations",
"version": "1.3.0",
"project_urls": {
"Homepage": "https://github.com/WFRT/verif"
},
"split_keywords": [
"meteorology",
"verification",
"weather",
"prediction"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ebfc7b1e750caba7ebcf3ec3883e4412eeace35210a43552c996c3dadf30e3ba",
"md5": "b9e19bdd649b4604d452af5dafe7b9de",
"sha256": "198903ab4f4830afbcd51807b13aa5922dfa6159ce98535b4c15d080f214c9b4"
},
"downloads": -1,
"filename": "verif-1.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b9e19bdd649b4604d452af5dafe7b9de",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 81164,
"upload_time": "2024-03-10T20:18:36",
"upload_time_iso_8601": "2024-03-10T20:18:36.884997Z",
"url": "https://files.pythonhosted.org/packages/eb/fc/7b1e750caba7ebcf3ec3883e4412eeace35210a43552c996c3dadf30e3ba/verif-1.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9835e531c9299a9d1a2e80fd4815a0719981fe0565b367f877db03c3359901f6",
"md5": "35f20096a684d6f785a1eb30832dbae0",
"sha256": "213f2b0988cd3c7ee34b783f1d21faf523a8dbc5844a4b47b283183b8ff21df8"
},
"downloads": -1,
"filename": "verif-1.3.0.tar.gz",
"has_sig": false,
"md5_digest": "35f20096a684d6f785a1eb30832dbae0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 80350,
"upload_time": "2024-03-10T20:18:38",
"upload_time_iso_8601": "2024-03-10T20:18:38.676297Z",
"url": "https://files.pythonhosted.org/packages/98/35/e531c9299a9d1a2e80fd4815a0719981fe0565b367f877db03c3359901f6/verif-1.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-10 20:18:38",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "WFRT",
"github_project": "verif",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "scipy",
"specs": []
},
{
"name": "numpy",
"specs": [
[
">=",
"1.7"
]
]
},
{
"name": "matplotlib",
"specs": []
},
{
"name": "coveralls",
"specs": []
},
{
"name": "pep8",
"specs": []
},
{
"name": "netCDF4",
"specs": []
},
{
"name": "future",
"specs": []
}
],
"tox": true,
"lcname": "verif"
}
JSON
