[PyDLM](https://pydlm.github.io/) [](https://app.circleci.com/pipelines/github/wwrechard/pydlm)
[](https://coveralls.io/github/wwrechard/pydlm?branch=master)
=======================================================
Welcome to [pydlm](https://pydlm.github.io/), a flexible time series modeling library for python. This library is based on the Bayesian dynamic linear model (Harrison and West, 1999) and optimized for fast model fitting and inference.
Updates
-------------------------------------------
* Version 0.1.1.13 has been released on PyPI.
* Migrated all the unnecessary `print()` to the default python logging operations, such as `logging.info`, `logging.warning` and `logging.critical`.
* Users can now set the model logging level to suppress unnecessary information during model run.
* Example to only print warning information:
```python
...
my_model.setLoggingLevel('WARNING')
my_model.fit()
```
* Updated the package version manage through `pip-compile` and `requirements.txt`.
Installation
------------
You can get the package (current version 0.1.1.11) from `pypi` by
$ pip install pydlm
You can also get the latest from [github](https://github.com/wwrechard/PyDLM)
$ git clone git@github.com:wwrechard/pydlm.git pydlm
$ cd pydlm
$ pip install pip-tools
$ pip install -r requirements.txt
$ pip install -e . --no-deps
`pydlm` depends on the following modules,
* `numpy` (for core functionality)
* `matplotlib` (for plotting results)
* `Sphinx` (for generating documentation)
* `unittest` (for testing)
Google data science post example
-----------------
We use the example from the [Google data science post](http://www.unofficialgoogledatascience.com/2017/07/fitting-bayesian-structural-time-series.html) as an example to show how `pydlm` could be used to analyze the real world data. The code and data is placed under `examples/unemployment_insurance/...`. The dataset contains weekly counts of initial claims for unemployment during 2004 - 2012 and is available from the R package `bsts` (which is a popular R package for time series modeling). The raw data is shown below (left)
<p align="center">
<img src="/doc/source/img/unemployment_2.png" width=48%/>
<img src="/doc/source/img/unemployment_1.png" width=48%/>
</p>
We see strong annual pattern and some local trend from the data.
<h4> A simple model </h4>
Following the Google's post, we first build a simple model with only local linear trend and seasonality component.
```python
from pydlm import dlm, trend, seasonality
# A linear trend
linear_trend = trend(degree=1, discount=0.95, name='linear_trend', w=10)
# A seasonality
seasonal52 = seasonality(period=52, discount=0.99, name='seasonal52', w=10)
# Build a simple dlm
simple_dlm = dlm(time_series) + linear_trend + seasonal52
```
In the actual code, the time series data is scored in the variable `time_series`. `degree=1` indicates the trend is linear (2 stands for quadratic) and `period=52` means the seasonality has a periodicy of 52. Since the seasonality is generally more stable, we set its discount factor to 0.99. For local linear trend, we use 0.95 to allow for some flexibility. `w=10` is the prior guess on the variance of each component, the larger number the more uncertain. For actual meaning of these parameters, please refer to the [user manual](https://pydlm.github.io/#dynamic-linear-models-user-manual). After the model built, we can fit the model and plot the result (shown above, right figure)
```python
# Fit the model
simple_dlm.fit()
# Plot the fitted results
simple_dlm.turnOff('data points')
simple_dlm.plot()
```
The blue curve is the forward filtering result, the green curve is the one-day ahead prediction and the red curve is the backward smoothed result. The light-colored ribbon around the curve is the confidence interval (you might need to zoom-in to see it). The one-day ahead prediction shows this simple model captures the time series somewhat good but loses accuracy around the peak crisis at Week 280 (which is between year 2008 - 2009). The one-day-ahead mean squared prediction error is **0.173** which can be obtained by calling
```python
simple_dlm.getMSE()
```
We can decompose the time series into each of its components
```python
# Plot each component (attribute the time series to each component)
simple_dlm.turnOff('predict plot')
simple_dlm.turnOff('filtered plot')
simple_dlm.plot('linear_trend')
simple_dlm.plot('seasonal52')
```
<p align="center">
<img src="/doc/source/img/unemployment_3-1.png" width=49%/>
<img src="/doc/source/img/unemployment_4-1.png" width=49%/>
</p>
Most of the time series shape is attributed to the local linear trend and the strong seasonality pattern is easily seen. To further verify the performance, we use this simple model for long-term forecasting. In particular, we use the previous **351 week**'s data to forecast the next **200 weeks** and the previous **251 week**'s data to forecast the next **200 weeks**. We lay the predicted results on top of the real data
```python
# Plot the prediction give the first 351 weeks and forcast the next 200 weeks.
simple_dlm.plotPredictN(date=350, N=200)
# Plot the prediction give the first 251 weeks and forcast the next 200 weeks.
simple_dlm.plotPredictN(date=250, N=200)
```
<p align="center">
<img src="/doc/source/img/unemployment_5.png" width=49%/>
<img src="/doc/source/img/unemployment_6.png" width=49%/>
</p>
From the figure we see that after the crisis peak around 2008 - 2009 (Week 280), the simple model can accurately forecast the next 200 weeks (left figure) given the first 351 weeks. However, the model fails to capture the change near the peak if the forecasting start before Week 280 (right figure).
<h4> Dynamic linear regression </h4>
Now we build a more sophiscated model with extra variables in the data file. The extra variables are stored in the variable `features` in the actual code. To build the dynamic linear regression model, we simply add a new component
```python
# Build a dynamic regression model
from pydlm import dynamic
regressor10 = dynamic(features=features, discount=1.0, name='regressor10', w=10)
drm = dlm(time_series) + linear_trend + seasonal52 + regressor10
drm.fit()
drm.getMSE()
# Plot the fitted results
drm.turnOff('data points')
drm.plot()
```
`dynamic` is the component for modeling dynamically changing predictors, which accepts `features` as its argument. The above code plots the fitted result (top left).
<p align="center">
<img src="/doc/source/img/unemployment_7.png" width=48%/>
<img src="/doc/source/img/unemployment_8-1.png" width=48%/>
<img src="/doc/source/img/unemployment_9-1.png" width=48%/>
<img src="/doc/source/img/unemployment_10-1.png" width=48%/>
</p>
The one-day ahead prediction looks much better than the simple model, particularly around the crisis peak. The mean prediction error is **0.099** which is a 100% improvement over the simple model. Similarly, we also decompose the time series into the three components
```python
drm.turnOff('predict plot')
drm.turnOff('filtered plot')
drm.plot('linear_trend')
drm.plot('seasonal52')
drm.plot('regressor10')
```
This time, the shape of the time series is mostly attributed to the regressor and the linear trend looks more linear. If we do long-term forecasting again, i.e., use the previous **301 week**'s data to forecast the next **150 weeks** and the previous **251 week**'s data to forecast the next **200 weeks**
```python
drm.plotPredictN(date=300, N=150)
drm.plotPredictN(date=250, N=200)
```
<p align="center">
<img src="/doc/source/img/unemployment_11.png" width=48%/>
<img src="/doc/source/img/unemployment_12.png" width=48%/>
</p>
The results look much better compared to the simple model
Documentation
-------------
Detailed documentation is provided in [PyDLM](https://pydlm.github.io/) with special attention to the [User manual](https://pydlm.github.io/#dynamic-linear-models-user-manual).
Raw data
{
"_id": null,
"home_page": "https://github.com/wwrechard/pydlm",
"name": "pydlm",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "dlm bayes bayesian kalman filter smoothing dynamic model",
"author": "Xiangyu Wang",
"author_email": "wwrechard@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/84/b2/fb3621fdf7d11b62c3acd1d0e57c6d5232177dc7c948b83401c2ca3490f0/pydlm-0.1.1.13.tar.gz",
"platform": null,
"description": "[PyDLM](https://pydlm.github.io/) [](https://app.circleci.com/pipelines/github/wwrechard/pydlm)\n [](https://coveralls.io/github/wwrechard/pydlm?branch=master)\n=======================================================\n\n\nWelcome to [pydlm](https://pydlm.github.io/), a flexible time series modeling library for python. This library is based on the Bayesian dynamic linear model (Harrison and West, 1999) and optimized for fast model fitting and inference.\n\nUpdates\n-------------------------------------------\n* Version 0.1.1.13 has been released on PyPI.\n * Migrated all the unnecessary `print()` to the default python logging operations, such as `logging.info`, `logging.warning` and `logging.critical`.\n * Users can now set the model logging level to suppress unnecessary information during model run.\n * Example to only print warning information: \n\n ```python\n ...\n\t my_model.setLoggingLevel('WARNING')\n\t my_model.fit()\n ```\n * Updated the package version manage through `pip-compile` and `requirements.txt`.\n\nInstallation\n------------\nYou can get the package (current version 0.1.1.11) from `pypi` by\n\n $ pip install pydlm\n\nYou can also get the latest from [github](https://github.com/wwrechard/PyDLM)\n\n $ git clone git@github.com:wwrechard/pydlm.git pydlm\n $ cd pydlm\n $ pip install pip-tools\n $ pip install -r requirements.txt\n $ pip install -e . --no-deps\n\n`pydlm` depends on the following modules,\n\n* `numpy` (for core functionality)\n* `matplotlib` (for plotting results)\n* `Sphinx` (for generating documentation)\n* `unittest` (for testing)\n\nGoogle data science post example\n-----------------\nWe use the example from the [Google data science post](http://www.unofficialgoogledatascience.com/2017/07/fitting-bayesian-structural-time-series.html) as an example to show how `pydlm` could be used to analyze the real world data. The code and data is placed under `examples/unemployment_insurance/...`. The dataset contains weekly counts of initial claims for unemployment during 2004 - 2012 and is available from the R package `bsts` (which is a popular R package for time series modeling). The raw data is shown below (left)\n<p align=\"center\">\n<img src=\"/doc/source/img/unemployment_2.png\" width=48%/>\n<img src=\"/doc/source/img/unemployment_1.png\" width=48%/>\n</p>\nWe see strong annual pattern and some local trend from the data.\n<h4> A simple model </h4>\nFollowing the Google's post, we first build a simple model with only local linear trend and seasonality component.\n\n```python\nfrom pydlm import dlm, trend, seasonality\n# A linear trend\nlinear_trend = trend(degree=1, discount=0.95, name='linear_trend', w=10)\n# A seasonality\nseasonal52 = seasonality(period=52, discount=0.99, name='seasonal52', w=10)\n# Build a simple dlm\nsimple_dlm = dlm(time_series) + linear_trend + seasonal52\n```\n\nIn the actual code, the time series data is scored in the variable `time_series`. `degree=1` indicates the trend is linear (2 stands for quadratic) and `period=52` means the seasonality has a periodicy of 52. Since the seasonality is generally more stable, we set its discount factor to 0.99. For local linear trend, we use 0.95 to allow for some flexibility. `w=10` is the prior guess on the variance of each component, the larger number the more uncertain. For actual meaning of these parameters, please refer to the [user manual](https://pydlm.github.io/#dynamic-linear-models-user-manual). After the model built, we can fit the model and plot the result (shown above, right figure)\n\n```python\n# Fit the model\nsimple_dlm.fit()\n# Plot the fitted results\nsimple_dlm.turnOff('data points')\nsimple_dlm.plot()\n```\n\nThe blue curve is the forward filtering result, the green curve is the one-day ahead prediction and the red curve is the backward smoothed result. The light-colored ribbon around the curve is the confidence interval (you might need to zoom-in to see it). The one-day ahead prediction shows this simple model captures the time series somewhat good but loses accuracy around the peak crisis at Week 280 (which is between year 2008 - 2009). The one-day-ahead mean squared prediction error is **0.173** which can be obtained by calling\n\n```python\nsimple_dlm.getMSE()\n```\n\nWe can decompose the time series into each of its components\n\n```python\n# Plot each component (attribute the time series to each component)\nsimple_dlm.turnOff('predict plot')\nsimple_dlm.turnOff('filtered plot')\nsimple_dlm.plot('linear_trend')\nsimple_dlm.plot('seasonal52')\n```\n\n<p align=\"center\">\n<img src=\"/doc/source/img/unemployment_3-1.png\" width=49%/>\n<img src=\"/doc/source/img/unemployment_4-1.png\" width=49%/>\n</p>\n\nMost of the time series shape is attributed to the local linear trend and the strong seasonality pattern is easily seen. To further verify the performance, we use this simple model for long-term forecasting. In particular, we use the previous **351 week**'s data to forecast the next **200 weeks** and the previous **251 week**'s data to forecast the next **200 weeks**. We lay the predicted results on top of the real data\n\n```python\n# Plot the prediction give the first 351 weeks and forcast the next 200 weeks.\nsimple_dlm.plotPredictN(date=350, N=200)\n# Plot the prediction give the first 251 weeks and forcast the next 200 weeks.\nsimple_dlm.plotPredictN(date=250, N=200)\n```\n\n<p align=\"center\">\n<img src=\"/doc/source/img/unemployment_5.png\" width=49%/>\n<img src=\"/doc/source/img/unemployment_6.png\" width=49%/>\n</p>\n\nFrom the figure we see that after the crisis peak around 2008 - 2009 (Week 280), the simple model can accurately forecast the next 200 weeks (left figure) given the first 351 weeks. However, the model fails to capture the change near the peak if the forecasting start before Week 280 (right figure).\n\n<h4> Dynamic linear regression </h4>\nNow we build a more sophiscated model with extra variables in the data file. The extra variables are stored in the variable `features` in the actual code. To build the dynamic linear regression model, we simply add a new component\n\n```python\n# Build a dynamic regression model\nfrom pydlm import dynamic\nregressor10 = dynamic(features=features, discount=1.0, name='regressor10', w=10)\ndrm = dlm(time_series) + linear_trend + seasonal52 + regressor10\ndrm.fit()\ndrm.getMSE()\n\n# Plot the fitted results\ndrm.turnOff('data points')\ndrm.plot()\n```\n\n`dynamic` is the component for modeling dynamically changing predictors, which accepts `features` as its argument. The above code plots the fitted result (top left).\n\n<p align=\"center\">\n<img src=\"/doc/source/img/unemployment_7.png\" width=48%/>\n<img src=\"/doc/source/img/unemployment_8-1.png\" width=48%/>\n<img src=\"/doc/source/img/unemployment_9-1.png\" width=48%/>\n<img src=\"/doc/source/img/unemployment_10-1.png\" width=48%/>\n</p>\n\nThe one-day ahead prediction looks much better than the simple model, particularly around the crisis peak. The mean prediction error is **0.099** which is a 100% improvement over the simple model. Similarly, we also decompose the time series into the three components\n\n```python\ndrm.turnOff('predict plot')\ndrm.turnOff('filtered plot')\ndrm.plot('linear_trend')\ndrm.plot('seasonal52')\ndrm.plot('regressor10')\n```\n\nThis time, the shape of the time series is mostly attributed to the regressor and the linear trend looks more linear. If we do long-term forecasting again, i.e., use the previous **301 week**'s data to forecast the next **150 weeks** and the previous **251 week**'s data to forecast the next **200 weeks**\n\n```python\ndrm.plotPredictN(date=300, N=150)\ndrm.plotPredictN(date=250, N=200)\n```\n\n<p align=\"center\">\n<img src=\"/doc/source/img/unemployment_11.png\" width=48%/>\n<img src=\"/doc/source/img/unemployment_12.png\" width=48%/>\n</p>\n\nThe results look much better compared to the simple model\n\nDocumentation\n-------------\nDetailed documentation is provided in [PyDLM](https://pydlm.github.io/) with special attention to the [User manual](https://pydlm.github.io/#dynamic-linear-models-user-manual).\n",
"bugtrack_url": null,
"license": "BSD",
"summary": "A python library for the Bayesian dynamic linear model for time series modeling",
"version": "0.1.1.13",
"project_urls": {
"Homepage": "https://github.com/wwrechard/pydlm"
},
"split_keywords": [
"dlm",
"bayes",
"bayesian",
"kalman",
"filter",
"smoothing",
"dynamic",
"model"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e560b0ccdf6b0344b013e95541093bb279ef3d4e1175ed308f9e2c95a520fa53",
"md5": "1bdbba19e57f20c6556a9721ff81190d",
"sha256": "3ab47c3c05e01538ea2bf6717a4a5c5c73cc80a8538a35a61820c67c6dc01854"
},
"downloads": -1,
"filename": "pydlm-0.1.1.13-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "1bdbba19e57f20c6556a9721ff81190d",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 64270,
"upload_time": "2024-08-13T04:20:45",
"upload_time_iso_8601": "2024-08-13T04:20:45.486854Z",
"url": "https://files.pythonhosted.org/packages/e5/60/b0ccdf6b0344b013e95541093bb279ef3d4e1175ed308f9e2c95a520fa53/pydlm-0.1.1.13-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "84b2fb3621fdf7d11b62c3acd1d0e57c6d5232177dc7c948b83401c2ca3490f0",
"md5": "231318951e5ff21ffe7ba0be6f40ff3a",
"sha256": "312071a2b086090203dde04a60c34f9a3f562e7578dd8ac594a076049718bb99"
},
"downloads": -1,
"filename": "pydlm-0.1.1.13.tar.gz",
"has_sig": false,
"md5_digest": "231318951e5ff21ffe7ba0be6f40ff3a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 2192756,
"upload_time": "2024-08-13T04:20:47",
"upload_time_iso_8601": "2024-08-13T04:20:47.364823Z",
"url": "https://files.pythonhosted.org/packages/84/b2/fb3621fdf7d11b62c3acd1d0e57c6d5232177dc7c948b83401c2ca3490f0/pydlm-0.1.1.13.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-13 04:20:47",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "wwrechard",
"github_project": "pydlm",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"circle": true,
"requirements": [
{
"name": "contourpy",
"specs": [
[
"==",
"1.0.7"
]
]
},
{
"name": "cycler",
"specs": [
[
"==",
"0.11.0"
]
]
},
{
"name": "fonttools",
"specs": [
[
"==",
"4.39.3"
]
]
},
{
"name": "kiwisolver",
"specs": [
[
"==",
"1.4.4"
]
]
},
{
"name": "matplotlib",
"specs": [
[
"==",
"3.7.1"
]
]
},
{
"name": "numpy",
"specs": [
[
"==",
"1.24.3"
]
]
},
{
"name": "packaging",
"specs": [
[
"==",
"23.1"
]
]
},
{
"name": "pillow",
"specs": [
[
"==",
"10.3.0"
]
]
},
{
"name": "pyparsing",
"specs": [
[
"==",
"3.0.9"
]
]
},
{
"name": "python-dateutil",
"specs": [
[
"==",
"2.8.2"
]
]
},
{
"name": "six",
"specs": [
[
"==",
"1.16.0"
]
]
}
],
"lcname": "pydlm"
}
JSON
