# Graph Weather
<!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section -->
[](#contributors-)
<!-- ALL-CONTRIBUTORS-BADGE:END -->
Implementation of the Graph Weather paper (https://arxiv.org/pdf/2202.07575.pdf) in PyTorch. Additionally, an implementation
of a modified model that assimilates raw or processed observations into analysis files.
## Installation
This library can be installed through
```bash
pip install graph-weather
```
## Example Usage
The models generate the graphs internally, so the only thing that needs to be passed to the model is the node features
in the same order as the ```lat_lons```.
```python
import torch
from graph_weather import GraphWeatherForecaster
from graph_weather.models.losses import NormalizedMSELoss
lat_lons = []
for lat in range(-90, 90, 1):
for lon in range(0, 360, 1):
lat_lons.append((lat, lon))
model = GraphWeatherForecaster(lat_lons)
# Generate 78 random features + 24 non-NWP features (i.e. landsea mask)
features = torch.randn((2, len(lat_lons), 102))
target = torch.randn((2, len(lat_lons), 78))
out = model(features)
criterion = NormalizedMSELoss(lat_lons=lat_lons, feature_variance=torch.randn((78,)))
loss = criterion(out, target)
loss.backward()
```
And for the assimilation model, which assumes each lat/lon point also has a height above ground, and each observation
is a single value + the relative time. The assimlation model also assumes the desired output grid is given to it as
well.
```python
import torch
import numpy as np
from graph_weather import GraphWeatherAssimilator
from graph_weather.models.losses import NormalizedMSELoss
obs_lat_lons = []
for lat in range(-90, 90, 7):
for lon in range(0, 180, 6):
obs_lat_lons.append((lat, lon, np.random.random(1)))
for lon in 360 * np.random.random(100):
obs_lat_lons.append((lat, lon, np.random.random(1)))
output_lat_lons = []
for lat in range(-90, 90, 5):
for lon in range(0, 360, 5):
output_lat_lons.append((lat, lon))
model = GraphWeatherAssimilator(output_lat_lons=output_lat_lons, analysis_dim=24)
features = torch.randn((1, len(obs_lat_lons), 2))
lat_lon_heights = torch.tensor(obs_lat_lons)
out = model(features, lat_lon_heights)
assert not torch.isnan(out).all()
assert out.size() == (1, len(output_lat_lons), 24)
criterion = torch.nn.MSELoss()
loss = criterion(out, torch.randn((1, len(output_lat_lons), 24)))
loss.backward()
```
## Pretrained Weights
Coming soon! We plan to train a model on GFS 0.25 degree operational forecasts, as well as MetOffice NWP forecasts.
We also plan trying out adaptive meshes, and predicting future satellite imagery as well.
## Training Data
Training data will be available through HuggingFace Datasets for the GFS forecasts. The initial set of data is available for [GFSv16 forecasts, raw observations, and FNL Analysis files from 2016 to 2022](https://huggingface.co/datasets/openclimatefix/gfs-reforecast), and for [ERA5 Reanlaysis](https://huggingface.co/datasets/openclimatefix/era5-reanalysis). MetOffice NWP forecasts we cannot
redistribute, but can be accessed through [CEDA](https://data.ceda.ac.uk/).
## Contributors ✨
Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):
<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->
<!-- prettier-ignore-start -->
<!-- markdownlint-disable -->
<table>
<tbody>
<tr>
<td align="center" valign="top" width="14.28%"><a href="https://www.jacobbieker.com"><img src="https://avatars.githubusercontent.com/u/7170359?v=4?s=100" width="100px;" alt="Jacob Bieker"/><br /><sub><b>Jacob Bieker</b></sub></a><br /><a href="https://github.com/openclimatefix/graph_weather/commits?author=jacobbieker" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="http://jack-kelly.com"><img src="https://avatars.githubusercontent.com/u/460756?v=4?s=100" width="100px;" alt="Jack Kelly"/><br /><sub><b>Jack Kelly</b></sub></a><br /><a href="#ideas-JackKelly" title="Ideas, Planning, & Feedback">🤔</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/byphilipp"><img src="https://avatars.githubusercontent.com/u/59995258?v=4?s=100" width="100px;" alt="byphilipp"/><br /><sub><b>byphilipp</b></sub></a><br /><a href="#ideas-byphilipp" title="Ideas, Planning, & Feedback">🤔</a></td>
<td align="center" valign="top" width="14.28%"><a href="http://iki.fi/markus.kaukonen"><img src="https://avatars.githubusercontent.com/u/6195764?v=4?s=100" width="100px;" alt="Markus Kaukonen"/><br /><sub><b>Markus Kaukonen</b></sub></a><br /><a href="#question-paapu88" title="Answering Questions">💬</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/MoHawastaken"><img src="https://avatars.githubusercontent.com/u/55447473?v=4?s=100" width="100px;" alt="MoHawastaken"/><br /><sub><b>MoHawastaken</b></sub></a><br /><a href="https://github.com/openclimatefix/graph_weather/issues?q=author%3AMoHawastaken" title="Bug reports">🐛</a></td>
<td align="center" valign="top" width="14.28%"><a href="http://www.ecmwf.int"><img src="https://avatars.githubusercontent.com/u/47196359?v=4?s=100" width="100px;" alt="Mihai"/><br /><sub><b>Mihai</b></sub></a><br /><a href="#question-mishooax" title="Answering Questions">💬</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/vitusbenson"><img src="https://avatars.githubusercontent.com/u/33334860?v=4?s=100" width="100px;" alt="Vitus Benson"/><br /><sub><b>Vitus Benson</b></sub></a><br /><a href="https://github.com/openclimatefix/graph_weather/issues?q=author%3Avitusbenson" title="Bug reports">🐛</a></td>
</tr>
<tr>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/dongZheX"><img src="https://avatars.githubusercontent.com/u/36361726?v=4?s=100" width="100px;" alt="dongZheX"/><br /><sub><b>dongZheX</b></sub></a><br /><a href="#question-dongZheX" title="Answering Questions">💬</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/sabbir2331"><img src="https://avatars.githubusercontent.com/u/25061297?v=4?s=100" width="100px;" alt="sabbir2331"/><br /><sub><b>sabbir2331</b></sub></a><br /><a href="#question-sabbir2331" title="Answering Questions">💬</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/rnwzd"><img src="https://avatars.githubusercontent.com/u/58804597?v=4?s=100" width="100px;" alt="Lorenzo Breschi"/><br /><sub><b>Lorenzo Breschi</b></sub></a><br /><a href="https://github.com/openclimatefix/graph_weather/commits?author=rnwzd" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/gbruno16"><img src="https://avatars.githubusercontent.com/u/72879691?v=4?s=100" width="100px;" alt="gbruno16"/><br /><sub><b>gbruno16</b></sub></a><br /><a href="https://github.com/openclimatefix/graph_weather/commits?author=gbruno16" title="Code">💻</a></td>
</tr>
</tbody>
</table>
<!-- markdownlint-restore -->
<!-- prettier-ignore-end -->
<!-- ALL-CONTRIBUTORS-LIST:END -->
This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!
Raw data
{
"_id": null,
"home_page": "https://github.com/openclimatefix/graph_weather",
"name": "graph-weather",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "Jacob Bieker",
"author_email": "jacob@bieker.tech",
"download_url": "https://files.pythonhosted.org/packages/c4/2b/c4b05e4d886a16a318903198c21b5b9a81f486655a1f9ea48c01519bc646/graph_weather-1.0.84.tar.gz",
"platform": null,
"description": "# Graph Weather\n<!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section -->\n[](#contributors-)\n<!-- ALL-CONTRIBUTORS-BADGE:END -->\nImplementation of the Graph Weather paper (https://arxiv.org/pdf/2202.07575.pdf) in PyTorch. Additionally, an implementation\nof a modified model that assimilates raw or processed observations into analysis files.\n\n\n## Installation\n\nThis library can be installed through\n\n```bash\npip install graph-weather\n```\n\n## Example Usage\n\nThe models generate the graphs internally, so the only thing that needs to be passed to the model is the node features\nin the same order as the ```lat_lons```.\n\n```python\nimport torch\nfrom graph_weather import GraphWeatherForecaster\nfrom graph_weather.models.losses import NormalizedMSELoss\n\nlat_lons = []\nfor lat in range(-90, 90, 1):\n for lon in range(0, 360, 1):\n lat_lons.append((lat, lon))\nmodel = GraphWeatherForecaster(lat_lons)\n\n# Generate 78 random features + 24 non-NWP features (i.e. landsea mask)\nfeatures = torch.randn((2, len(lat_lons), 102))\n\ntarget = torch.randn((2, len(lat_lons), 78))\nout = model(features)\n\ncriterion = NormalizedMSELoss(lat_lons=lat_lons, feature_variance=torch.randn((78,)))\nloss = criterion(out, target)\nloss.backward()\n```\n\nAnd for the assimilation model, which assumes each lat/lon point also has a height above ground, and each observation\nis a single value + the relative time. The assimlation model also assumes the desired output grid is given to it as\nwell.\n\n```python\nimport torch\nimport numpy as np\nfrom graph_weather import GraphWeatherAssimilator\nfrom graph_weather.models.losses import NormalizedMSELoss\n\nobs_lat_lons = []\nfor lat in range(-90, 90, 7):\n for lon in range(0, 180, 6):\n obs_lat_lons.append((lat, lon, np.random.random(1)))\n for lon in 360 * np.random.random(100):\n obs_lat_lons.append((lat, lon, np.random.random(1)))\n\noutput_lat_lons = []\nfor lat in range(-90, 90, 5):\n for lon in range(0, 360, 5):\n output_lat_lons.append((lat, lon))\nmodel = GraphWeatherAssimilator(output_lat_lons=output_lat_lons, analysis_dim=24)\n\nfeatures = torch.randn((1, len(obs_lat_lons), 2))\nlat_lon_heights = torch.tensor(obs_lat_lons)\nout = model(features, lat_lon_heights)\nassert not torch.isnan(out).all()\nassert out.size() == (1, len(output_lat_lons), 24)\n\ncriterion = torch.nn.MSELoss()\nloss = criterion(out, torch.randn((1, len(output_lat_lons), 24)))\nloss.backward()\n```\n\n## Pretrained Weights\nComing soon! We plan to train a model on GFS 0.25 degree operational forecasts, as well as MetOffice NWP forecasts.\nWe also plan trying out adaptive meshes, and predicting future satellite imagery as well.\n\n## Training Data\nTraining data will be available through HuggingFace Datasets for the GFS forecasts. The initial set of data is available for [GFSv16 forecasts, raw observations, and FNL Analysis files from 2016 to 2022](https://huggingface.co/datasets/openclimatefix/gfs-reforecast), and for [ERA5 Reanlaysis](https://huggingface.co/datasets/openclimatefix/era5-reanalysis). MetOffice NWP forecasts we cannot\nredistribute, but can be accessed through [CEDA](https://data.ceda.ac.uk/).\n\n## Contributors \u2728\n\nThanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):\n\n<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->\n<!-- prettier-ignore-start -->\n<!-- markdownlint-disable -->\n<table>\n <tbody>\n <tr>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://www.jacobbieker.com\"><img src=\"https://avatars.githubusercontent.com/u/7170359?v=4?s=100\" width=\"100px;\" alt=\"Jacob Bieker\"/><br /><sub><b>Jacob Bieker</b></sub></a><br /><a href=\"https://github.com/openclimatefix/graph_weather/commits?author=jacobbieker\" title=\"Code\">\ud83d\udcbb</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"http://jack-kelly.com\"><img src=\"https://avatars.githubusercontent.com/u/460756?v=4?s=100\" width=\"100px;\" alt=\"Jack Kelly\"/><br /><sub><b>Jack Kelly</b></sub></a><br /><a href=\"#ideas-JackKelly\" title=\"Ideas, Planning, & Feedback\">\ud83e\udd14</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/byphilipp\"><img src=\"https://avatars.githubusercontent.com/u/59995258?v=4?s=100\" width=\"100px;\" alt=\"byphilipp\"/><br /><sub><b>byphilipp</b></sub></a><br /><a href=\"#ideas-byphilipp\" title=\"Ideas, Planning, & Feedback\">\ud83e\udd14</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"http://iki.fi/markus.kaukonen\"><img src=\"https://avatars.githubusercontent.com/u/6195764?v=4?s=100\" width=\"100px;\" alt=\"Markus Kaukonen\"/><br /><sub><b>Markus Kaukonen</b></sub></a><br /><a href=\"#question-paapu88\" title=\"Answering Questions\">\ud83d\udcac</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/MoHawastaken\"><img src=\"https://avatars.githubusercontent.com/u/55447473?v=4?s=100\" width=\"100px;\" alt=\"MoHawastaken\"/><br /><sub><b>MoHawastaken</b></sub></a><br /><a href=\"https://github.com/openclimatefix/graph_weather/issues?q=author%3AMoHawastaken\" title=\"Bug reports\">\ud83d\udc1b</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"http://www.ecmwf.int\"><img src=\"https://avatars.githubusercontent.com/u/47196359?v=4?s=100\" width=\"100px;\" alt=\"Mihai\"/><br /><sub><b>Mihai</b></sub></a><br /><a href=\"#question-mishooax\" title=\"Answering Questions\">\ud83d\udcac</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/vitusbenson\"><img src=\"https://avatars.githubusercontent.com/u/33334860?v=4?s=100\" width=\"100px;\" alt=\"Vitus Benson\"/><br /><sub><b>Vitus Benson</b></sub></a><br /><a href=\"https://github.com/openclimatefix/graph_weather/issues?q=author%3Avitusbenson\" title=\"Bug reports\">\ud83d\udc1b</a></td>\n </tr>\n <tr>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/dongZheX\"><img src=\"https://avatars.githubusercontent.com/u/36361726?v=4?s=100\" width=\"100px;\" alt=\"dongZheX\"/><br /><sub><b>dongZheX</b></sub></a><br /><a href=\"#question-dongZheX\" title=\"Answering Questions\">\ud83d\udcac</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/sabbir2331\"><img src=\"https://avatars.githubusercontent.com/u/25061297?v=4?s=100\" width=\"100px;\" alt=\"sabbir2331\"/><br /><sub><b>sabbir2331</b></sub></a><br /><a href=\"#question-sabbir2331\" title=\"Answering Questions\">\ud83d\udcac</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/rnwzd\"><img src=\"https://avatars.githubusercontent.com/u/58804597?v=4?s=100\" width=\"100px;\" alt=\"Lorenzo Breschi\"/><br /><sub><b>Lorenzo Breschi</b></sub></a><br /><a href=\"https://github.com/openclimatefix/graph_weather/commits?author=rnwzd\" title=\"Code\">\ud83d\udcbb</a></td>\n <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://github.com/gbruno16\"><img src=\"https://avatars.githubusercontent.com/u/72879691?v=4?s=100\" width=\"100px;\" alt=\"gbruno16\"/><br /><sub><b>gbruno16</b></sub></a><br /><a href=\"https://github.com/openclimatefix/graph_weather/commits?author=gbruno16\" title=\"Code\">\ud83d\udcbb</a></td>\n </tr>\n </tbody>\n</table>\n\n<!-- markdownlint-restore -->\n<!-- prettier-ignore-end -->\n\n<!-- ALL-CONTRIBUTORS-LIST:END -->\n\nThis project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "Weather Forecasting with Graph Neural Networks",
"version": "1.0.84",
"project_urls": {
"Homepage": "https://github.com/openclimatefix/graph_weather"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c4b9fd71a7329761ce7e9a3bc1ccc7fc4ad1de1907bf6b114f21632f7388bbba",
"md5": "6fb76fc422b70278a40c757444f75180",
"sha256": "099c209743b56a4dc3220c9e865645053b4774c03a89a4c0f955de29aa6f3595"
},
"downloads": -1,
"filename": "graph_weather-1.0.84-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6fb76fc422b70278a40c757444f75180",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 146242,
"upload_time": "2024-09-23T19:00:30",
"upload_time_iso_8601": "2024-09-23T19:00:30.395281Z",
"url": "https://files.pythonhosted.org/packages/c4/b9/fd71a7329761ce7e9a3bc1ccc7fc4ad1de1907bf6b114f21632f7388bbba/graph_weather-1.0.84-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c42bc4b05e4d886a16a318903198c21b5b9a81f486655a1f9ea48c01519bc646",
"md5": "6755635349464468386bc3976539e515",
"sha256": "fe346d612d39035b3ef28ea33fcb3837a3b12346a8eff52f4c67900757952d56"
},
"downloads": -1,
"filename": "graph_weather-1.0.84.tar.gz",
"has_sig": false,
"md5_digest": "6755635349464468386bc3976539e515",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 130401,
"upload_time": "2024-09-23T19:00:32",
"upload_time_iso_8601": "2024-09-23T19:00:32.922340Z",
"url": "https://files.pythonhosted.org/packages/c4/2b/c4b05e4d886a16a318903198c21b5b9a81f486655a1f9ea48c01519bc646/graph_weather-1.0.84.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-23 19:00:32",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "openclimatefix",
"github_project": "graph_weather",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "h3",
"specs": []
},
{
"name": "huggingface-hub",
"specs": []
},
{
"name": "datasets",
"specs": []
},
{
"name": "einops",
"specs": []
},
{
"name": "torch-geometric-temporal",
"specs": []
},
{
"name": "torch_harmonics",
"specs": []
},
{
"name": "trimesh",
"specs": []
},
{
"name": "rtree",
"specs": []
},
{
"name": "xarray",
"specs": []
},
{
"name": "setuptools",
"specs": []
},
{
"name": "pydantic",
"specs": []
},
{
"name": "safetensors",
"specs": []
}
],
"lcname": "graph-weather"
}
JSON
