# NABQR
[](https://pypi.python.org/pypi/nabqr)
[](https://nabqr.readthedocs.io/en/latest/?version=latest)
- **Free software**: MIT license
- **Documentation**: [NABQR Documentation](https://nabqr.readthedocs.io)
README for nabqr package
=======================
## Table of Contents
- [Introduction](#introduction)
- [Getting Started](#getting-started)
- [Main functions](#main-functions)
- [Test file](#test-file)
- [Notes](#notes)
- [Credits](#credits)
---
## Introduction
This section provides an overview of the project. Discuss the goals, purpose, and high-level summary here.
NABQR is a method for sequential error-corrections tailored for wind power forecast in Denmark.
The method is based on the paper: *Sequential methods for Error Corrections in Wind Power Forecasts*, with the following abstract:
> Wind power is a rapidly expanding renewable energy source and is set for continued growth in the future. This leads to parts of the world relying on an inherently volatile energy source.
> Efficient operation of such systems requires reliable probabilistic forecasts of future wind power production to better manage the uncertainty that wind power bring. These forecasts provide critical insights, enabling wind power producers and system operators to maximize the economic benefits of renewable energy while minimizing its potential adverse effects on grid stability.
> This study introduces sequential methods to correct errors in power production forecasts derived from numerical weather predictions.
> We introduce Neural Adaptive Basis for (Time-Adaptive) Quantile Regression (NABQR), a novel approach that combines neural networks with Time-Adaptive Quantile Regression (TAQR) to enhance the accuracy of wind power production forecasts.
> First, NABQR corrects power production ensembles using neural networks.
> Our study identifies Long Short-Term Memory networks as the most effective architecture for this purpose.
> Second, TAQR is applied to the corrected ensembles to obtain optimal median predictions along with quantile descriptions of the forecast density.
> The method achieves substantial improvements upwards of 40% in mean absolute terms. Additionally, we explore the potential of this methodology for applications in energy trading.
> The method is available as an open-source Python package to support further research and applications in renewable energy forecasting.
- **Free software**: MIT license
- **Documentation**: [NABQR Documentation](https://nabqr.readthedocs.io)
---
## Getting Started
### Installation
`pip install nabqr`
Then see the [Test file](#test-file) section for an example of how to use the package.
## Main functions
### Pipeline
```python
import nabqr as nq
```
```python
nq.pipeline(X, y,
name = "TEST",
training_size = 0.8,
epochs = 100,
timesteps_for_lstm = [0,1,2,6,12,24,48],
**kwargs)
```
The pipeline trains a LSTM network to correct the provided ensembles.
It then runs the TAQR algorithm on the corrected ensembles to predict the observations, y, on the test set.
**Parameters:**
- **X**: `pd.DataFrame` or `np.array`, shape `(n_timesteps, n_ensembles)`
- The ensemble data to be corrected.
- **y**: `pd.Series` or `np.array`, shape `(n_timesteps,)`
- The observations to be predicted.
- **name**: `str`
- The name of the dataset.
- **training_size**: `float`
- The proportion of the data to be used for training.
- **epochs**: `int`
- The number of epochs to train the LSTM.
- **timesteps_for_lstm**: `list`
- The timesteps to use for the LSTM.
**Output:**
The pipeline saves the following outputs and also returns them:
- **Actuals Out of Sample**:
- File: `results_<today>_<data_source>_actuals_out_of_sample.npy`
- Description: Contains the actual observations that are out of the sample.
- **Corrected Ensembles**:
- File: `results_<today>_<data_source>_corrected_ensembles.csv`
- Description: A CSV file containing the corrected ensemble data.
- **TAQR Results**:
- File: `results_<today>_<data_source>_taqr_results.npy`
- Description: Contains the results from the Time-Adaptive Quantile Regression (TAQR).
- **BETA Parameters**:
- File: `results_<today>_<data_source>_BETA_output.npy`
- Description: Contains the BETA parameters from the TAQR.
Note: `<today>` is the current date in the format `YYYY-MM-DD`, and `<data_source>` is the name of the dataset.
The pipeline trains a LSTM network to correct the provided ensembles and then runs the TAQR algorithm on the corrected ensembles to predict the observations, y, on the test set.
### Time-Adaptive Quantile Regression
nabqr also include a time-adaptive quantile regression model, which can be used independently of the pipeline.
```python
import nabqr as nq
```
```python
nq.run_taqr(corrected_ensembles, actuals, quantiles, n_init, n_full, n_in_X)
```
Run TAQR on `corrected_ensembles`, `X`, based on the actual values, `y`, and the given quantiles.
**Parameters:**
- **corrected_ensembles**: `np.array`, shape `(n_timesteps, n_ensembles)`
- The corrected ensembles to run TAQR on.
- **actuals**: `np.array`, shape `(n_timesteps,)`
- The actual values to run TAQR on.
- **quantiles**: `list`
- The quantiles to run TAQR for.
- **n_init**: `int`
- The number of initial timesteps to use for warm start.
- **n_full**: `int`
- The total number of timesteps to run TAQR for.
- **n_in_X**: `int`
- The number of timesteps to include in the design matrix.
## Test file
Here we introduce the function `simulate_correlated_ar1_process`, which can be used to simulate multivariate AR data. The entire file can be run by
```python
import nabqr as nq
nq.run_nabqr_pipeline(...)
# or
from nabqr import run_nabqr_pipeline
run_nabqr_pipeline(...)
```
```python
from functions import *
from helper_functions import simulate_correlated_ar1_process, set_n_closest_to_zero
import matplotlib.pyplot as plt
import scienceplots
plt.style.use(['no-latex'])
from visualization import visualize_results
import datetime as dt
# Example usage. Inputs:
offset = np.arange(10, 500, 15)
m = len(offset)
corr_matrix = 0.8 * np.ones((m, m)) + 0.2 * np.eye(m) # Example correlation structure
data_source = "NABQR-TEST"
today = dt.datetime.today().strftime('%Y-%m-%d')
simulated_data, actuals = simulate_correlated_ar1_process(5000, 0.995, 8, m, corr_matrix, offset, smooth=5)
# Optional kwargs
quantiles_taqr = [0.01, 0.1, 0.3, 0.5, 0.7, 0.9, 0.99]
pipeline(simulated_data, actuals, data_source, training_size = 0.7, epochs = 100, timesteps_for_lstm = [0,1,2,6,12,24], quantiles_taqr = quantiles_taqr)
# Import old results
CE = pd.read_csv(f"results_{today}_{data_source}_corrected_ensembles.csv")
y_hat = np.load(f"results_{today}_{data_source}_actuals_out_of_sample.npy")
q_hat = np.load(f"results_{today}_{data_source}_taqr_results.npy")
# Call the visualization function
visualize_results(y_hat, q_hat, "NABQR-TEST example")
```
We provide an overview of the shapes for this test file:
```python
simulated_data.shape: (5000, 33)
actuals.shape: (5000,)
m: 33
len(quantiles_taqr): 7
```
## Requirements
- Python 3.10 or later
- icecream, matplotlib, numpy, pandas, properscoring, rich, SciencePlots, scikit_learn, scipy, tensorflow, tensorflow_probability, torch, typer, sphinx_rtd_theme, myst_parser, tf_keras
- R with the following packages: quantreg, readr
## Credits
This package was partially created with [Cookiecutter](https://github.com/audreyr/cookiecutter) and the [`audreyr/cookiecutter-pypackage`](https://github.com/audreyr/cookiecutter-pypackage) project template.
Raw data
{
"_id": null,
"home_page": "https://github.com/bast0320/nabqr",
"name": "nabqr",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "nabqr, energy, quantile, forecasting",
"author": "Bastian S. J\u00f8rgensen",
"author_email": "bassc@dtu.dk",
"download_url": "https://files.pythonhosted.org/packages/5e/29/d64117552fd5a6554e7b14a2233423f3807416679ea72fbcc6b371bc38fe/nabqr-0.0.28.tar.gz",
"platform": null,
"description": "# NABQR\n\n[](https://pypi.python.org/pypi/nabqr)\n[](https://nabqr.readthedocs.io/en/latest/?version=latest)\n\n- **Free software**: MIT license \n- **Documentation**: [NABQR Documentation](https://nabqr.readthedocs.io)\n\nREADME for nabqr package\n=======================\n\n## Table of Contents\n- [Introduction](#introduction)\n- [Getting Started](#getting-started)\n- [Main functions](#main-functions)\n- [Test file](#test-file)\n- [Notes](#notes)\n- [Credits](#credits)\n---\n\n## Introduction\n\nThis section provides an overview of the project. Discuss the goals, purpose, and high-level summary here.\n\n\nNABQR is a method for sequential error-corrections tailored for wind power forecast in Denmark.\n\nThe method is based on the paper: *Sequential methods for Error Corrections in Wind Power Forecasts*, with the following abstract:\n> Wind power is a rapidly expanding renewable energy source and is set for continued growth in the future. This leads to parts of the world relying on an inherently volatile energy source.\n> Efficient operation of such systems requires reliable probabilistic forecasts of future wind power production to better manage the uncertainty that wind power bring. These forecasts provide critical insights, enabling wind power producers and system operators to maximize the economic benefits of renewable energy while minimizing its potential adverse effects on grid stability.\n> This study introduces sequential methods to correct errors in power production forecasts derived from numerical weather predictions. \n> We introduce Neural Adaptive Basis for (Time-Adaptive) Quantile Regression (NABQR), a novel approach that combines neural networks with Time-Adaptive Quantile Regression (TAQR) to enhance the accuracy of wind power production forecasts. \n> First, NABQR corrects power production ensembles using neural networks.\n> Our study identifies Long Short-Term Memory networks as the most effective architecture for this purpose.\n> Second, TAQR is applied to the corrected ensembles to obtain optimal median predictions along with quantile descriptions of the forecast density. \n> The method achieves substantial improvements upwards of 40% in mean absolute terms. Additionally, we explore the potential of this methodology for applications in energy trading.\n> The method is available as an open-source Python package to support further research and applications in renewable energy forecasting.\n\n\n- **Free software**: MIT license \n- **Documentation**: [NABQR Documentation](https://nabqr.readthedocs.io)\n---\n\n## Getting Started\n\n### Installation\n`pip install nabqr`\n\nThen see the [Test file](#test-file) section for an example of how to use the package.\n\n## Main functions\n### Pipeline\n```python\nimport nabqr as nq\n```\n\n```python\nnq.pipeline(X, y, \n name = \"TEST\",\n training_size = 0.8, \n epochs = 100,\n timesteps_for_lstm = [0,1,2,6,12,24,48],\n **kwargs)\n```\n\nThe pipeline trains a LSTM network to correct the provided ensembles.\nIt then runs the TAQR algorithm on the corrected ensembles to predict the observations, y, on the test set.\n\n**Parameters:**\n\n- **X**: `pd.DataFrame` or `np.array`, shape `(n_timesteps, n_ensembles)`\n - The ensemble data to be corrected.\n- **y**: `pd.Series` or `np.array`, shape `(n_timesteps,)`\n - The observations to be predicted.\n- **name**: `str`\n - The name of the dataset.\n- **training_size**: `float`\n - The proportion of the data to be used for training.\n- **epochs**: `int`\n - The number of epochs to train the LSTM.\n- **timesteps_for_lstm**: `list`\n - The timesteps to use for the LSTM.\n\n**Output:**\nThe pipeline saves the following outputs and also returns them:\n\n- **Actuals Out of Sample**: \n - File: `results_<today>_<data_source>_actuals_out_of_sample.npy`\n - Description: Contains the actual observations that are out of the sample.\n\n- **Corrected Ensembles**: \n - File: `results_<today>_<data_source>_corrected_ensembles.csv`\n - Description: A CSV file containing the corrected ensemble data.\n\n- **TAQR Results**: \n - File: `results_<today>_<data_source>_taqr_results.npy`\n - Description: Contains the results from the Time-Adaptive Quantile Regression (TAQR).\n\n- **BETA Parameters**: \n - File: `results_<today>_<data_source>_BETA_output.npy`\n - Description: Contains the BETA parameters from the TAQR.\n\nNote: `<today>` is the current date in the format `YYYY-MM-DD`, and `<data_source>` is the name of the dataset.\n\n\nThe pipeline trains a LSTM network to correct the provided ensembles and then runs the TAQR algorithm on the corrected ensembles to predict the observations, y, on the test set.\n\n### Time-Adaptive Quantile Regression\nnabqr also include a time-adaptive quantile regression model, which can be used independently of the pipeline.\n```python\nimport nabqr as nq\n```\n```python\nnq.run_taqr(corrected_ensembles, actuals, quantiles, n_init, n_full, n_in_X)\n```\n\nRun TAQR on `corrected_ensembles`, `X`, based on the actual values, `y`, and the given quantiles.\n\n**Parameters:**\n\n- **corrected_ensembles**: `np.array`, shape `(n_timesteps, n_ensembles)`\n - The corrected ensembles to run TAQR on.\n- **actuals**: `np.array`, shape `(n_timesteps,)`\n - The actual values to run TAQR on.\n- **quantiles**: `list`\n - The quantiles to run TAQR for.\n- **n_init**: `int`\n - The number of initial timesteps to use for warm start.\n- **n_full**: `int`\n - The total number of timesteps to run TAQR for.\n- **n_in_X**: `int`\n - The number of timesteps to include in the design matrix.\n\n\n## Test file \nHere we introduce the function `simulate_correlated_ar1_process`, which can be used to simulate multivariate AR data. The entire file can be run by \n```python\nimport nabqr as nq\nnq.run_nabqr_pipeline(...)\n# or\nfrom nabqr import run_nabqr_pipeline\nrun_nabqr_pipeline(...)\n```\n\n```python\nfrom functions import *\nfrom helper_functions import simulate_correlated_ar1_process, set_n_closest_to_zero\nimport matplotlib.pyplot as plt\nimport scienceplots\nplt.style.use(['no-latex'])\nfrom visualization import visualize_results \nimport datetime as dt\n# Example usage. Inputs:\noffset = np.arange(10, 500, 15)\nm = len(offset)\ncorr_matrix = 0.8 * np.ones((m, m)) + 0.2 * np.eye(m) # Example correlation structure\ndata_source = \"NABQR-TEST\"\ntoday = dt.datetime.today().strftime('%Y-%m-%d')\n\nsimulated_data, actuals = simulate_correlated_ar1_process(5000, 0.995, 8, m, corr_matrix, offset, smooth=5)\n\n# Optional kwargs\nquantiles_taqr = [0.01, 0.1, 0.3, 0.5, 0.7, 0.9, 0.99]\n\npipeline(simulated_data, actuals, data_source, training_size = 0.7, epochs = 100, timesteps_for_lstm = [0,1,2,6,12,24], quantiles_taqr = quantiles_taqr)\n\n# Import old results\nCE = pd.read_csv(f\"results_{today}_{data_source}_corrected_ensembles.csv\")\ny_hat = np.load(f\"results_{today}_{data_source}_actuals_out_of_sample.npy\")\nq_hat = np.load(f\"results_{today}_{data_source}_taqr_results.npy\")\n\n# Call the visualization function\nvisualize_results(y_hat, q_hat, \"NABQR-TEST example\")\n```\n\nWe provide an overview of the shapes for this test file:\n```python\nsimulated_data.shape: (5000, 33)\nactuals.shape: (5000,)\nm: 33\nlen(quantiles_taqr): 7\n```\n\n## Requirements\n\n- Python 3.10 or later\n- icecream, matplotlib, numpy, pandas, properscoring, rich, SciencePlots, scikit_learn, scipy, tensorflow, tensorflow_probability, torch, typer, sphinx_rtd_theme, myst_parser, tf_keras\n- R with the following packages: quantreg, readr\n\n## Credits\n\nThis package was partially created with [Cookiecutter](https://github.com/audreyr/cookiecutter) and the [`audreyr/cookiecutter-pypackage`](https://github.com/audreyr/cookiecutter-pypackage) project template.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "NABQR is a method for sequential error-corrections tailored for wind power forecast in Denmark",
"version": "0.0.28",
"project_urls": {
"Homepage": "https://github.com/bast0320/nabqr"
},
"split_keywords": [
"nabqr",
" energy",
" quantile",
" forecasting"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6cf5ea93377d10a3b0dd101290bd6dbda0e419cd437d2179dff6ccb0a9b04fba",
"md5": "08edcd11394aa93db485a1752586faeb",
"sha256": "5d45b6ac38100329c25912f3e6c74f6313fcc74cd6ff47b9dd4914b4bbc4a0ff"
},
"downloads": -1,
"filename": "nabqr-0.0.28-py3-none-any.whl",
"has_sig": false,
"md5_digest": "08edcd11394aa93db485a1752586faeb",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 44686,
"upload_time": "2024-12-12T14:25:03",
"upload_time_iso_8601": "2024-12-12T14:25:03.856256Z",
"url": "https://files.pythonhosted.org/packages/6c/f5/ea93377d10a3b0dd101290bd6dbda0e419cd437d2179dff6ccb0a9b04fba/nabqr-0.0.28-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5e29d64117552fd5a6554e7b14a2233423f3807416679ea72fbcc6b371bc38fe",
"md5": "17e7b696548dbe2b3f896f48c49aad40",
"sha256": "df742e1cf6c9d9e7ccb534baebc8dd10a0793f6aff190b5b3c4bf5b528689600"
},
"downloads": -1,
"filename": "nabqr-0.0.28.tar.gz",
"has_sig": false,
"md5_digest": "17e7b696548dbe2b3f896f48c49aad40",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 35905,
"upload_time": "2024-12-12T14:25:05",
"upload_time_iso_8601": "2024-12-12T14:25:05.229466Z",
"url": "https://files.pythonhosted.org/packages/5e/29/d64117552fd5a6554e7b14a2233423f3807416679ea72fbcc6b371bc38fe/nabqr-0.0.28.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-12 14:25:05",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "bast0320",
"github_project": "nabqr",
"travis_ci": true,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "icecream",
"specs": [
[
">=",
"2.1.3"
]
]
},
{
"name": "matplotlib",
"specs": [
[
">=",
"3.8.4"
]
]
},
{
"name": "numpy",
"specs": [
[
">=",
"1.26.0"
],
[
"<",
"2.0.0"
]
]
},
{
"name": "pandas",
"specs": [
[
">=",
"2.2.3"
]
]
},
{
"name": "properscoring",
"specs": [
[
"==",
"0.1"
]
]
},
{
"name": "rich",
"specs": [
[
">=",
"13.9.4"
]
]
},
{
"name": "SciencePlots",
"specs": [
[
"==",
"2.1.1"
]
]
},
{
"name": "scikit_learn",
"specs": [
[
">=",
"1.4.2"
]
]
},
{
"name": "scipy",
"specs": [
[
">=",
"1.14.1"
]
]
},
{
"name": "tensorflow",
"specs": [
[
">=",
"2.16.1"
]
]
},
{
"name": "tensorflow_probability",
"specs": [
[
">=",
"0.24.0"
]
]
},
{
"name": "torch",
"specs": [
[
">=",
"2.4.1"
]
]
},
{
"name": "typer",
"specs": [
[
">=",
"0.13.1"
]
]
},
{
"name": "sphinx_rtd_theme",
"specs": [
[
">=",
"3.0.2"
]
]
},
{
"name": "myst_parser",
"specs": [
[
">=",
"4.0.0"
]
]
},
{
"name": "tf_keras",
"specs": [
[
">=",
"2.16.0"
]
]
},
{
"name": "autodocsumm",
"specs": [
[
">=",
"0.2.14"
]
]
},
{
"name": "sphinx-mdinclude",
"specs": [
[
">=",
"0.6.2"
]
]
}
],
"lcname": "nabqr"
}
JSON
