# Model Radar 🎯
A framework for aspect-based evaluation of time series forecasting models based on Nixtla's ecosystem.
[](https://pypi.org/project/modelradar/)
[](https://github.com/vcerqueira/modelradar)
[](https://pepy.tech/project/modelradar)
## Overview
Model Radar introduces a novel aspect-based forecasting evaluation approach that goes beyond traditional aggregate metrics. Our framework enables:
- Fine-grained performance analysis across different forecasting aspects
- Better understanding of model behavior in varying conditions
- More informed model selection based on specific use case requirements
## 🚀 Getting Started
Check the `notebooks` folder for usage examples and tutorials.
Check **[ModelRadar-Experiments](https://github.com/vcerqueira/experiments-modelradar)** repository for a thorough
application of ModelRadar.
## Installation
You can install modelradar using pip:
```bash
pip install modelradar
```
### [Optional] Installation from source
To install modelradar from source, clone the repository and run the following command:
```bash
git clone https://github.com/vcerqueira/modelradar
pip install -e modelradar
```
### Prerequisites
Required dependencies:
```
utilsforecast==0.2.11
numpy==1.26.4
plotnine==0.14.5
```
⚠️ I've noticed some issues when running with more recent versions of numpy and utilsforecast.
Try to use the versions above.
### Examples
Besides the examples in the `notebooks` folder, here's some outputs you can get from *modelradar*:
- Spider chart with overview on several dimensions:
- Parallel coordinates chart with overview on several dimensions:
- Barplot chart controlling for a given variable (in this case, anomaly status):
- Grouped bar plot showing win/draw/loss ratios wrt different models:
<img src="assets/examples/win_ratios.png" width="70%" alt="radar2">
## 📑 References
> Cerqueira, V., Roque, L., & Soares, C. "Forecasting with Deep Learning: Beyond Average of Average of Average Performance." Discovery Science: 27th International Conference, DS 2024, Pisa, Italy, 2024, Proceedings 27. Springer International Publishing, 2024.
Check DS24 folder to reproduce the experiments published on this paper.
The main repository and package contains an updated framework.
### **⚠️ WARNING**
> modelradar is in the early stages of development.
> The codebase may undergo significant changes.
> If you encounter any issues, please report
> them in [GitHub Issues](https://github.com/vcerqueira/modelradar/issues)
Raw data
{
"_id": null,
"home_page": null,
"name": "modelradar",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "Data Science, Evaluation, Forecasting, Machine Learning, Time Series",
"author": null,
"author_email": "Vitor Cerqueira <cerqueira.vitormanuel@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/fb/f5/e27a310309644496564ec8f9e50f91c51bb1bb4a418bcf20382dace3adf2/modelradar-0.2.1.tar.gz",
"platform": null,
"description": "# Model Radar \ud83c\udfaf\n\nA framework for aspect-based evaluation of time series forecasting models based on Nixtla's ecosystem.\n\n[](https://pypi.org/project/modelradar/)\n[](https://github.com/vcerqueira/modelradar)\n[](https://pepy.tech/project/modelradar)\n\n## Overview\n\nModel Radar introduces a novel aspect-based forecasting evaluation approach that goes beyond traditional aggregate metrics. Our framework enables:\n- Fine-grained performance analysis across different forecasting aspects\n- Better understanding of model behavior in varying conditions\n- More informed model selection based on specific use case requirements\n\n## \ud83d\ude80 Getting Started\n\nCheck the `notebooks` folder for usage examples and tutorials.\n\nCheck **[ModelRadar-Experiments](https://github.com/vcerqueira/experiments-modelradar)** repository for a thorough\napplication of ModelRadar.\n\n## Installation\n\nYou can install modelradar using pip:\n\n```bash\npip install modelradar\n```\n\n### [Optional] Installation from source\n\nTo install modelradar from source, clone the repository and run the following command:\n\n```bash\ngit clone https://github.com/vcerqueira/modelradar\npip install -e modelradar\n```\n\n### Prerequisites\n\nRequired dependencies:\n```\nutilsforecast==0.2.11\nnumpy==1.26.4\nplotnine==0.14.5\n```\n\n\u26a0\ufe0f I've noticed some issues when running with more recent versions of numpy and utilsforecast. \nTry to use the versions above.\n\n### Examples\n\nBesides the examples in the `notebooks` folder, here's some outputs you can get from *modelradar*:\n\n- Spider chart with overview on several dimensions:\n\n\n\n- Parallel coordinates chart with overview on several dimensions:\n\n\n\n- Barplot chart controlling for a given variable (in this case, anomaly status):\n\n\n\n- Grouped bar plot showing win/draw/loss ratios wrt different models:\n\n<img src=\"assets/examples/win_ratios.png\" width=\"70%\" alt=\"radar2\">\n\n\n## \ud83d\udcd1 References\n\n> Cerqueira, V., Roque, L., & Soares, C. \"Forecasting with Deep Learning: Beyond Average of Average of Average Performance.\" Discovery Science: 27th International Conference, DS 2024, Pisa, Italy, 2024, Proceedings 27. Springer International Publishing, 2024.\n\nCheck DS24 folder to reproduce the experiments published on this paper.\nThe main repository and package contains an updated framework.\n\n### **\u26a0\ufe0f WARNING**\n\n> modelradar is in the early stages of development. \n> The codebase may undergo significant changes. \n> If you encounter any issues, please report\n> them in [GitHub Issues](https://github.com/vcerqueira/modelradar/issues)",
"bugtrack_url": null,
"license": null,
"summary": "Aspect-based Forecasting Accuracy",
"version": "0.2.1",
"project_urls": {
"Bug Tracker": "https://github.com/vcerqueira/modelradar/issues",
"Homepage": "https://github.com/vcerqueira/modelradar"
},
"split_keywords": [
"data science",
" evaluation",
" forecasting",
" machine learning",
" time series"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6a526ffce2ba6981c298c506d444915ac64c2fbd40dc88b17b083bb621648bec",
"md5": "343badab26f2dc32c14fa100dc6acb66",
"sha256": "79c67a339fb14a5a1866cd2eb3065446dfb1e52e3afe4970c10309f1fc874c21"
},
"downloads": -1,
"filename": "modelradar-0.2.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "343badab26f2dc32c14fa100dc6acb66",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 19249,
"upload_time": "2025-07-15T20:52:21",
"upload_time_iso_8601": "2025-07-15T20:52:21.411290Z",
"url": "https://files.pythonhosted.org/packages/6a/52/6ffce2ba6981c298c506d444915ac64c2fbd40dc88b17b083bb621648bec/modelradar-0.2.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "fbf5e27a310309644496564ec8f9e50f91c51bb1bb4a418bcf20382dace3adf2",
"md5": "5c4c8c4f7372806d71f4d51da3d21814",
"sha256": "cc0106c28b1b471d22922e5f7647e368a2de038d7f547204eb12a21418262793"
},
"downloads": -1,
"filename": "modelradar-0.2.1.tar.gz",
"has_sig": false,
"md5_digest": "5c4c8c4f7372806d71f4d51da3d21814",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 6723582,
"upload_time": "2025-07-15T20:52:23",
"upload_time_iso_8601": "2025-07-15T20:52:23.793782Z",
"url": "https://files.pythonhosted.org/packages/fb/f5/e27a310309644496564ec8f9e50f91c51bb1bb4a418bcf20382dace3adf2/modelradar-0.2.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-15 20:52:23",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "vcerqueira",
"github_project": "modelradar",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "modelradar"
}
JSON
