# Model Confidence Set
The `model-confidence-set` package provides a Python implementation of the Model Confidence Set (MCS) procedure [(Hansen, Lunde, and Nason, 2011)](https://www.jstor.org/stable/41057463), a statistical method for comparing and selecting models based on their performance. It allows users to identify a set of models that are statistically indistinguishable from the best model, given a statistical confidence level.
This package
- supports both stationary and block bootstrap methods.
- implements two methods for p-value computation: *relative* and *sequential*.
- optionally displays progress during computation.
## Installation
To install `model-confidence-set`, simply use pip:
```bash
pip install model-confidence-set
```
## Usage
To use the Model Confidence Set in your Python code, follow the example below:
```python
import numpy as np
import pandas as pd
from model_confidence_set import ModelConfidenceSet
# Example losses matrix where rows are observations and columns are models
losses = np.random.rand(100, 5) # 100 observations for 5 models
# Initialize the MCS procedure (5'000 bootstrap iterations, 5% confidence level)
mcs = ModelConfidenceSet(losses, n_boot=5000, alpha=0.05, show_progress=True)
# Compute the MCS
mcs.compute()
# Retrieve the results as a pandas DataFrame (use as_dataframe=False for a dict)
results = mcs.results()
print(results)
```
## Parameters
- `losses`: A 2D `numpy.ndarray` or `pandas.DataFrame` containing loss values of models. Rows correspond to observations, and columns correspond to different models.
- `n_boot`: Number of bootstrap replications for computing p-values. Default is `5000`.
- `alpha`: Significance level for determining model confidence set. Default is `0.05`.
- `block_len`: The length of blocks for the block bootstrap. If `None`, it defaults to the number of observations.
- `bootstrap_variant`: Specifies the bootstrap variant to use. Options are `'stationary'` or `'block'`. Default is `'stationary'`.
- `method`: The method used for p-value calculation. Options are `'R'` for *relative* or `'SQ'` for *sequential*. Default is `'R'`.
- `show_progress`: Whether to show a progress bar during bootstrap computations. Default is `False`.
## Acknowledgments
This package draws inspiration from
+ the [Matlab implementation by Kevin Sheppard](https://www.kevinsheppard.com/code/matlab/mfe-toolbox/)
+ the [Python implementation by Michael Gong](https://michael-gong.com/blogs/model-confidence-set/).
Raw data
{
"_id": null,
"home_page": "https://github.com/JLDC/model-confidence-set",
"name": "model-confidence-set",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "model confidence set,model evaluation,statistical model comparison,model performance analysis,model selection,predictive accuracy,econometrics,financial econometrics",
"author": "Jonathan Chassot",
"author_email": "jonathan.chassot@unisg.ch",
"download_url": "https://files.pythonhosted.org/packages/cf/0e/839d012dde2c931393053e4ba6a0ad9f917a5386d6815ca112c5f25cea0a/model-confidence-set-0.1.1.tar.gz",
"platform": null,
"description": "# Model Confidence Set\n\nThe `model-confidence-set` package provides a Python implementation of the Model Confidence Set (MCS) procedure [(Hansen, Lunde, and Nason, 2011)](https://www.jstor.org/stable/41057463), a statistical method for comparing and selecting models based on their performance. It allows users to identify a set of models that are statistically indistinguishable from the best model, given a statistical confidence level.\n\nThis package\n- supports both stationary and block bootstrap methods.\n- implements two methods for p-value computation: *relative* and *sequential*.\n- optionally displays progress during computation.\n\n## Installation\n\nTo install `model-confidence-set`, simply use pip:\n\n```bash\npip install model-confidence-set\n```\n\n## Usage\nTo use the Model Confidence Set in your Python code, follow the example below:\n\n```python\nimport numpy as np\nimport pandas as pd\nfrom model_confidence_set import ModelConfidenceSet\n\n# Example losses matrix where rows are observations and columns are models\nlosses = np.random.rand(100, 5) # 100 observations for 5 models\n\n# Initialize the MCS procedure (5'000 bootstrap iterations, 5% confidence level)\nmcs = ModelConfidenceSet(losses, n_boot=5000, alpha=0.05, show_progress=True)\n\n# Compute the MCS\nmcs.compute()\n\n# Retrieve the results as a pandas DataFrame (use as_dataframe=False for a dict)\nresults = mcs.results()\nprint(results)\n```\n\n## Parameters\n- `losses`: A 2D `numpy.ndarray` or `pandas.DataFrame` containing loss values of models. Rows correspond to observations, and columns correspond to different models.\n- `n_boot`: Number of bootstrap replications for computing p-values. Default is `5000`.\n- `alpha`: Significance level for determining model confidence set. Default is `0.05`.\n- `block_len`: The length of blocks for the block bootstrap. If `None`, it defaults to the number of observations.\n- `bootstrap_variant`: Specifies the bootstrap variant to use. Options are `'stationary'` or `'block'`. Default is `'stationary'`.\n- `method`: The method used for p-value calculation. Options are `'R'` for *relative* or `'SQ'` for *sequential*. Default is `'R'`.\n- `show_progress`: Whether to show a progress bar during bootstrap computations. Default is `False`.\n\n## Acknowledgments\nThis package draws inspiration from \n+ the [Matlab implementation by Kevin Sheppard](https://www.kevinsheppard.com/code/matlab/mfe-toolbox/) \n+ the [Python implementation by Michael Gong](https://michael-gong.com/blogs/model-confidence-set/).\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "model-confidence-set provides a Python implementation of the Model Confidence Set (MCS) procedure (Hansen, Lunde, and Nason, 2011), a statistical method for comparing and selecting models based on their performance.",
"version": "0.1.1",
"project_urls": {
"Homepage": "https://github.com/JLDC/model-confidence-set"
},
"split_keywords": [
"model confidence set",
"model evaluation",
"statistical model comparison",
"model performance analysis",
"model selection",
"predictive accuracy",
"econometrics",
"financial econometrics"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a8ac196de9a41276bf1cbe7644f673657324930f9d87ccb401252a0425e7e6f6",
"md5": "3af4c28bb623cdb0fef3851190be4b5b",
"sha256": "677f9e05d9dacd6eba9165db3d92dca6119be71e478ad920232809e7952b5d11"
},
"downloads": -1,
"filename": "model_confidence_set-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "3af4c28bb623cdb0fef3851190be4b5b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 7076,
"upload_time": "2024-03-12T10:11:56",
"upload_time_iso_8601": "2024-03-12T10:11:56.085924Z",
"url": "https://files.pythonhosted.org/packages/a8/ac/196de9a41276bf1cbe7644f673657324930f9d87ccb401252a0425e7e6f6/model_confidence_set-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "cf0e839d012dde2c931393053e4ba6a0ad9f917a5386d6815ca112c5f25cea0a",
"md5": "1abf233114f5a98b66ed9493a57a6c6e",
"sha256": "2b5df8d7a83cf5d315ee63c6047ddc284017eaba245c44db1c01d56ede90b087"
},
"downloads": -1,
"filename": "model-confidence-set-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "1abf233114f5a98b66ed9493a57a6c6e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 6226,
"upload_time": "2024-03-12T10:11:58",
"upload_time_iso_8601": "2024-03-12T10:11:58.185768Z",
"url": "https://files.pythonhosted.org/packages/cf/0e/839d012dde2c931393053e4ba6a0ad9f917a5386d6815ca112c5f25cea0a/model-confidence-set-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-12 10:11:58",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "JLDC",
"github_project": "model-confidence-set",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "model-confidence-set"
}
JSON
