# TrueSight ✨
The TrueSight model is a hybrid forecasting tool that uses statistical forecasting models together with a Deep Neural Network (DNN) to make predictions. The TrueSight `Preprocessor` class is responsible for getting all the statistical forecasters in one place. It can handle forecasters from packages like `statsforecast`, `scikit-learn`, `pmdarima`, and others. You just need to the `ModelWrapper` Class to standardize the method calls.
All you need to do before using this package, is create a pandas dataframe with the following structure:
- unique_id: A string that uniquely identifies each time series in the dataframe
- ds: A datetime column with the date of each time step. The dates must be in the correct frequency for the date_freq parameter
- y: The values of the time series
and run the steps in the Usage section. Easy peasy! 😎
## Instalation 💻
To install the TrueSight package, just run:
```
pip install truesight
```
We also recommend installing the `statsforecast` package for the statistical forecasters:
```
pip install statsforecast
```
## Usage 🚀
Import the necessary modules
``` python
import tensorflow as tf
from truesight.preprocessing import Preprocessor
from truesight.core import TrueSight
from truesight.metrics import Evaluator, smape, mape, mse, rmse, mae
from truesight.utils import AutoTune, generate_syntetic_data
```
Load the data
``` python
num_time_steps = 60
season_length = 12
forecast_horizon = 12
df = generate_syntetic_data(num_time_steps, season_length, 100)
```
Create and run the preprocessor class. You can include as many statistical models as you need in the `models` parameter, just make sure to use the `ModelWrapper` class. However, keep in mind that more models mean longer processing time. It's important to set a fallback model in case any of the informed models fail to fit.
``` python
from statsforecast.models import SeasonalNaive, AutoETS
from sklearn.linear_model import LinearRegression
from truesight.models import AdditiveDecomposition
from truesight.utils import ModelWrapper
models = [
ModelWrapper(LinearRegression, horizon=forecast_horizon, season_length=season_length, alias="LinearRegression"),
ModelWrapper(SeasonalNaive, horizon=forecast_horizon, season_length=season_length),
ModelWrapper(AutoETS, horizon=forecast_horizon, season_length=season_length),
ModelWrapper(AdditiveDecomposition, horizon=forecast_horizon, season_length=season_length)
]
preprocessor = Preprocessor(df)
X_train, Y_train, ids_train, X_val, Y_val, ids_val, models = preprocessor.make_dataset(
forecast_horizon = 12,
season_length = 12,
date_freq = "MS",
models = models,
fallback_model = ModelWrapper(SeasonalNaive, horizon=forecast_horizon, season_length=season_length),
verbose = True
)
```
Create the model and automatical automatically find the hyperparameters
``` python
optimizer = tf.keras.optimizers.Adam
hparams, optimizer = AutoTune(optimizer=optimizer).tune(X_train, Y_train, n_trials = 20, epochs = 10, batch_size = 32, stats_models = models)
ts = TrueSight(models, forecast_horizon)
ts.set_hparams(hparams)
ts.compile(optimizer=optimizer, loss='mse')
```
Or set then manually
``` python
optimizer = tf.keras.optimizers.Adam
ts = TrueSight(models, forecast_horizon, filter_size = 128, context_size = 512, hidden_size = 1024, dropout_rate = 0.1)
ts.compile(optimizer=optimizer, loss='mse')
```
Train the model, as the model is built on the tensorflow framework, any tensorflow callback can be used
``` python
callbacks = [
tf.keras.callbacks.EarlyStopping(patience = 100, restore_best_weights = True, monitor = "val_loss"),
tf.keras.callbacks.ReduceLROnPlateau(monitor = "val_loss", factor = 0.5, patience = 25, verbose = False),
]
ts.fit(
x = X_train, y = Y_train,
validation_data = [X_val, Y_val],
batch_size = 128,
epochs = 1000,
verbose = False,
callbacks = callbacks,
)
ts.plot_training_history()
```
Evaluate the results
``` python
yhat = ts.predict(X_val, n_repeats = 100, n_quantiles = 15, verbose = False)
evaluator = Evaluator(X_val, Y_val, yhat, ids_val)
evaluator.evaluate_prediction(evaluators = [smape, mape, mse, rmse, mae], return_mean = True)
```
| metric | value |
|-------:|----------:|
|smape | 0.234369|
|mape | 0.293816|
|mse | 816.238082|
|rmse | 21.218396|
|mae | 15.885432|
``` python
evaluator.plot_exemple()
```
Raw data
{
"_id": null,
"home_page": "https://github.com/renanklehm/true-sight",
"name": "truesight",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "time series,prediction",
"author": "Renan Otvin Klehm",
"author_email": "renanotivin@hotmail.com",
"download_url": "https://files.pythonhosted.org/packages/37/97/781a5150c55ddabc9f65eba4d6ad085e66c0069a12e37d6b1f9eb9f6b98d/truesight-0.0.3a1.tar.gz",
"platform": null,
"description": "# TrueSight \u2728\n\nThe TrueSight model is a hybrid forecasting tool that uses statistical forecasting models together with a Deep Neural Network (DNN) to make predictions. The TrueSight `Preprocessor` class is responsible for getting all the statistical forecasters in one place. It can handle forecasters from packages like `statsforecast`, `scikit-learn`, `pmdarima`, and others. You just need to the `ModelWrapper` Class to standardize the method calls.\n\nAll you need to do before using this package, is create a pandas dataframe with the following structure:\n\n - unique_id: A string that uniquely identifies each time series in the dataframe\n - ds: A datetime column with the date of each time step. The dates must be in the correct frequency for the date_freq parameter\n - y: The values of the time series\n\nand run the steps in the Usage section. Easy peasy! \ud83d\ude0e\n\n## Instalation \ud83d\udcbb\n\nTo install the TrueSight package, just run:\n\n```\npip install truesight\n```\n\nWe also recommend installing the `statsforecast` package for the statistical forecasters:\n\n```\npip install statsforecast\n```\n\n## Usage \ud83d\ude80\n\nImport the necessary modules\n\n``` python\nimport tensorflow as tf\nfrom truesight.preprocessing import Preprocessor\nfrom truesight.core import TrueSight\nfrom truesight.metrics import Evaluator, smape, mape, mse, rmse, mae\nfrom truesight.utils import AutoTune, generate_syntetic_data\n```\n\nLoad the data\n\n``` python\nnum_time_steps = 60\nseason_length = 12\nforecast_horizon = 12\ndf = generate_syntetic_data(num_time_steps, season_length, 100)\n```\nCreate and run the preprocessor class. You can include as many statistical models as you need in the `models` parameter, just make sure to use the `ModelWrapper` class. However, keep in mind that more models mean longer processing time. It's important to set a fallback model in case any of the informed models fail to fit.\n\n``` python\nfrom statsforecast.models import SeasonalNaive, AutoETS\nfrom sklearn.linear_model import LinearRegression\nfrom truesight.models import AdditiveDecomposition\nfrom truesight.utils import ModelWrapper\n\nmodels = [\n ModelWrapper(LinearRegression, horizon=forecast_horizon, season_length=season_length, alias=\"LinearRegression\"), \n ModelWrapper(SeasonalNaive, horizon=forecast_horizon, season_length=season_length), \n ModelWrapper(AutoETS, horizon=forecast_horizon, season_length=season_length),\n ModelWrapper(AdditiveDecomposition, horizon=forecast_horizon, season_length=season_length)\n]\n\npreprocessor = Preprocessor(df)\nX_train, Y_train, ids_train, X_val, Y_val, ids_val, models = preprocessor.make_dataset(\n forecast_horizon = 12, \n season_length = 12,\n date_freq = \"MS\", \n models = models, \n fallback_model = ModelWrapper(SeasonalNaive, horizon=forecast_horizon, season_length=season_length),\n verbose = True\n )\n```\n\nCreate the model and automatical automatically find the hyperparameters\n\n``` python\noptimizer = tf.keras.optimizers.Adam\nhparams, optimizer = AutoTune(optimizer=optimizer).tune(X_train, Y_train, n_trials = 20, epochs = 10, batch_size = 32, stats_models = models)\nts = TrueSight(models, forecast_horizon)\nts.set_hparams(hparams)\nts.compile(optimizer=optimizer, loss='mse')\n```\n\nOr set then manually\n\n``` python\noptimizer = tf.keras.optimizers.Adam\nts = TrueSight(models, forecast_horizon, filter_size = 128, context_size = 512, hidden_size = 1024, dropout_rate = 0.1)\nts.compile(optimizer=optimizer, loss='mse')\n```\n\nTrain the model, as the model is built on the tensorflow framework, any tensorflow callback can be used\n\n``` python\ncallbacks = [\n tf.keras.callbacks.EarlyStopping(patience = 100, restore_best_weights = True, monitor = \"val_loss\"),\n tf.keras.callbacks.ReduceLROnPlateau(monitor = \"val_loss\", factor = 0.5, patience = 25, verbose = False),\n]\nts.fit(\n x = X_train, y = Y_train, \n validation_data = [X_val, Y_val], \n batch_size = 128, \n epochs = 1000, \n verbose = False, \n callbacks = callbacks,\n)\nts.plot_training_history()\n```\n\n\n\nEvaluate the results\n\n``` python\nyhat = ts.predict(X_val, n_repeats = 100, n_quantiles = 15, verbose = False)\nevaluator = Evaluator(X_val, Y_val, yhat, ids_val)\nevaluator.evaluate_prediction(evaluators = [smape, mape, mse, rmse, mae], return_mean = True)\n```\n| metric | value |\n|-------:|----------:|\n|smape | 0.234369|\n|mape | 0.293816|\n|mse | 816.238082|\n|rmse | 21.218396|\n|mae | 15.885432|\n\n\n``` python\nevaluator.plot_exemple()\n```\n\n",
"bugtrack_url": null,
"license": "",
"summary": "Truesight is a python package for time series prediction using deep learning and statistical models.",
"version": "0.0.3a1",
"project_urls": {
"Download": "https://github.com/renanklehm/true-sight/archive/refs/tags/v0.0.1-alpha.tar.gz",
"Homepage": "https://github.com/renanklehm/true-sight"
},
"split_keywords": [
"time series",
"prediction"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b2846c5c96afc13e4c9b96d5ee9275bbe6af513c7d802f7a1b971be6294d8670",
"md5": "509f7132c4c2e9fd38c7b568f60b1275",
"sha256": "9c4bda03446f47bfc9b42f60f3f396fb6756f4a7f44312e2cfca858e927629e5"
},
"downloads": -1,
"filename": "truesight-0.0.3a1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "509f7132c4c2e9fd38c7b568f60b1275",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 16845,
"upload_time": "2023-05-26T19:31:20",
"upload_time_iso_8601": "2023-05-26T19:31:20.265938Z",
"url": "https://files.pythonhosted.org/packages/b2/84/6c5c96afc13e4c9b96d5ee9275bbe6af513c7d802f7a1b971be6294d8670/truesight-0.0.3a1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3797781a5150c55ddabc9f65eba4d6ad085e66c0069a12e37d6b1f9eb9f6b98d",
"md5": "38ba997d266ca11ccb7a37fad2b98285",
"sha256": "4dcc4719ddc861f0c5c4b76f14c9a126dc106804fe5120cbf12d57a12ea72d29"
},
"downloads": -1,
"filename": "truesight-0.0.3a1.tar.gz",
"has_sig": false,
"md5_digest": "38ba997d266ca11ccb7a37fad2b98285",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 16027,
"upload_time": "2023-05-26T19:31:21",
"upload_time_iso_8601": "2023-05-26T19:31:21.836507Z",
"url": "https://files.pythonhosted.org/packages/37/97/781a5150c55ddabc9f65eba4d6ad085e66c0069a12e37d6b1f9eb9f6b98d/truesight-0.0.3a1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-05-26 19:31:21",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "renanklehm",
"github_project": "true-sight",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "truesight"
}
JSON
