<p align="center">
<img src="https://raw.githubusercontent.com/bancaditalia/black-it/main/docs/logo/logo_1024.png" width="500">
<sup><a href="#footnote-1">*</a></sup>
</p>
<a href="https://pypi.org/project/black-it">
<img alt="PyPI - Python Version" src="https://img.shields.io/pypi/pyversions/black-it" />
</a>
<a href="https://github.com/bancaditalia/black-it/blob/main/LICENSE">
<img alt="GitHub" src="https://img.shields.io/github/license/bancaditalia/black-it">
</a>
<a style="border-width:0" href="https://doi.org/10.21105/joss.04622">
<img src="https://joss.theoj.org/papers/10.21105/joss.04622/status.svg" alt="DOI badge" >
</a>
<a href="https://codecov.io/gh/bancaditalia/black-it">
<img src="https://codecov.io/gh/bancaditalia/black-it/branch/main/graph/badge.svg" />
</a>
# Black-box abm calibration kit
Black-it is an easy-to-use toolbox designed to help you calibrate the parameters
in your agent-based models and simulations (ABMs), using state-of-the-art
techniques to sample the parameter search space, with no need to reinvent the
wheel.
Models from economics, epidemiology, biology, logistics, and more can be dealt
with. The software can be used as-is - if your main interest is the ABM model
itself. However, in case your research thing is to, e.g., devise new sampling
strategies for ginormous search spaces and highly non-linear model, then you can
deploy and test your new ideas on a solid, reusable, modular foundation, in a
matter of days, with no need to reimplement all the plumbings from scratch.
## Installation
This project requires Python v3.8 or later.
To install the latest version of the package from [PyPI](https://pypi.org/project/black-it/):
```
pip install black-it
```
Or, directly from GitHub:
```
pip install git+https://github.com/bancaditalia/black-it.git#egg=black-it
```
If you'd like to contribute to the package, please read the [CONTRIBUTING.md](./CONTRIBUTING.md) guide.
#### Installation on Apple Silicon machines
Due to a dependency on the `tables` package, installing *Black-it* on Apple Silicon machines can sometimes require specifying the path to the HDF5 executable as explained [here](https://stackoverflow.com/questions/73029883/could-not-find-hdf5-installation-for-pytables-on-m1-mac).
## Quick Example
The GitHub repo of Black-it contains a series ready-to-run calibration examples.
To experiment with them, simply clone the repo and enter the `examples` folder
```
git clone https://github.com/bancaditalia/black-it.git
cd black-it/examples
```
You'll find several scripts and notebooks. The following is the script named `main.py`, note that copying and pasting
the lines below will not work in general as the script needs to be inside the "examples" folder in order to run correctly.
```python
import models.simple_models as md
from black_it.calibrator import Calibrator
from black_it.loss_functions.msm import MethodOfMomentsLoss
from black_it.samplers.best_batch import BestBatchSampler
from black_it.samplers.halton import HaltonSampler
from black_it.samplers.random_forest import RandomForestSampler
true_params = [0.20, 0.20, 0.75]
bounds = [
[0.10, 0.10, 0.10], # LOWER bounds
[1.00, 1.00, 1.00], # UPPER bounds
]
bounds_step = [0.01, 0.01, 0.01] # Step size in range between bounds
batch_size = 8
halton_sampler = HaltonSampler(batch_size=batch_size)
random_forest_sampler = RandomForestSampler(batch_size=batch_size)
best_batch_sampler = BestBatchSampler(batch_size=batch_size)
# define a model to be calibrated
model = md.MarkovC_KP
# generate a synthetic dataset to test the calibrator
N = 2000
seed = 1
real_data = model(true_params, N, seed)
# define a loss
loss = MethodOfMomentsLoss()
# define the calibration seed
calibration_seed = 1
# initialize a Calibrator object
cal = Calibrator(
samplers=[halton_sampler, random_forest_sampler, best_batch_sampler],
real_data=real_data,
model=model,
parameters_bounds=bounds,
parameters_precision=bounds_step,
ensemble_size=3,
loss_function=loss,
random_state=calibration_seed,
)
# calibrate the model
params, losses = cal.calibrate(n_batches=15)
print(f"True parameters: {true_params}")
print(f"Best parameters found: {params[0]}")
```
When the calibration terminates (~half a minute), towards the end of the output
you should see the following messages:
```
True parameters: [0.2, 0.2, 0.75]
Best parameters found: [0.19 0.21 0.68]
```
## Docs
Black-it calibration is initiated via the [Calibrator](https://bancaditalia.github.io/black-it/calibrator/) which,
when called, performs three main steps.
First, a [Sampler](https://bancaditalia.github.io/black-it/samplers/) is summoned to suggest a set of promising
parameter configurations to explore.
Second, the [model](https://bancaditalia.github.io/black-it/simulator_interface/) to be calibrated is simulated for
all the selected parameters.
Third, a specific [loss function](https://bancaditalia.github.io/black-it/losses/), measuring the goodness of fitness
of the simulation data with respect to the real data, is evaluated.
These steps are performed in a loop, and this allows the samplers to progress towards better parameter values
by exploiting the knowledge of previously computed loss functions.
A more detailed explanation of how Black-it works is available
[here](https://bancaditalia.github.io/black-it/description/), while the full documentation -complete with examples
and tutorials- is available [here](https://bancaditalia.github.io/black-it/).
## Citing *Black-it*
A description of the package is available [here](https://joss.theoj.org/papers/10.21105/joss.04622).
Please consider citing it if you found this package useful for your research
```bib
@article{black_it,
title = {Black-it: A Ready-to-Use and Easy-to-Extend Calibration Kit for Agent-based Models},
journal = {Journal of Open Source Software},
publisher = {The Open Journal},
year = {2022},
volume = {7},
number = {79},
pages = {4622},
doi = {10.21105/joss.04622},
url = {https://doi.org/10.21105/joss.04622},
author = {Marco Benedetti and
Gennaro Catapano and
Francesco {De Sclavis} and
Marco Favorito and
Aldo Glielmo and
Davide Magnanimi and
Antonio Muci}
}
```
## Disclaimer
This package is an outcome of a research project. All errors are those of the authors. All views expressed are personal views, not those of Bank of Italy.
---
<p id="footnote-1">
* Credits to <a href="https://www.bankit.art/people/sara-corbo">Sara Corbo</a> for the logo.
</p>
Raw data
{
"_id": null,
"home_page": "https://github.com/bancaditalia/black-it",
"name": "black-it",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.13,>=3.9",
"maintainer_email": null,
"keywords": "agent-based models, black-box calibration",
"author": "Applied Research Team",
"author_email": "appliedresearchteam@bancaditalia.it",
"download_url": "https://files.pythonhosted.org/packages/57/4c/98817054753578f4c39c13b54819848e530f57c40614e5046261636552c0/black_it-0.3.1.tar.gz",
"platform": null,
"description": "\n<p align=\"center\">\n<img src=\"https://raw.githubusercontent.com/bancaditalia/black-it/main/docs/logo/logo_1024.png\" width=\"500\">\n<sup><a href=\"#footnote-1\">*</a></sup>\n</p>\n\n<a href=\"https://pypi.org/project/black-it\">\n <img alt=\"PyPI - Python Version\" src=\"https://img.shields.io/pypi/pyversions/black-it\" />\n</a>\n\n<a href=\"https://github.com/bancaditalia/black-it/blob/main/LICENSE\">\n <img alt=\"GitHub\" src=\"https://img.shields.io/github/license/bancaditalia/black-it\">\n</a>\n\n<a style=\"border-width:0\" href=\"https://doi.org/10.21105/joss.04622\">\n <img src=\"https://joss.theoj.org/papers/10.21105/joss.04622/status.svg\" alt=\"DOI badge\" >\n</a>\n\n<a href=\"https://codecov.io/gh/bancaditalia/black-it\">\n <img src=\"https://codecov.io/gh/bancaditalia/black-it/branch/main/graph/badge.svg\" />\n</a>\n\n# Black-box abm calibration kit\n\nBlack-it is an easy-to-use toolbox designed to help you calibrate the parameters\nin your agent-based models and simulations (ABMs), using state-of-the-art\ntechniques to sample the parameter search space, with no need to reinvent the\nwheel.\n\nModels from economics, epidemiology, biology, logistics, and more can be dealt\nwith. The software can be used as-is - if your main interest is the ABM model\nitself. However, in case your research thing is to, e.g., devise new sampling\nstrategies for ginormous search spaces and highly non-linear model, then you can\ndeploy and test your new ideas on a solid, reusable, modular foundation, in a\nmatter of days, with no need to reimplement all the plumbings from scratch.\n\n## Installation\n\nThis project requires Python v3.8 or later.\n\nTo install the latest version of the package from [PyPI](https://pypi.org/project/black-it/):\n```\npip install black-it\n```\n\nOr, directly from GitHub:\n\n```\npip install git+https://github.com/bancaditalia/black-it.git#egg=black-it\n```\n\nIf you'd like to contribute to the package, please read the [CONTRIBUTING.md](./CONTRIBUTING.md) guide.\n\n#### Installation on Apple Silicon machines\n\nDue to a dependency on the `tables` package, installing *Black-it* on Apple Silicon machines can sometimes require specifying the path to the HDF5 executable as explained [here](https://stackoverflow.com/questions/73029883/could-not-find-hdf5-installation-for-pytables-on-m1-mac).\n\n## Quick Example\n\nThe GitHub repo of Black-it contains a series ready-to-run calibration examples.\n\nTo experiment with them, simply clone the repo and enter the `examples` folder\n\n```\ngit clone https://github.com/bancaditalia/black-it.git\ncd black-it/examples\n```\n\nYou'll find several scripts and notebooks. The following is the script named `main.py`, note that copying and pasting \nthe lines below will not work in general as the script needs to be inside the \"examples\" folder in order to run correctly. \n\n```python\nimport models.simple_models as md\n\nfrom black_it.calibrator import Calibrator\nfrom black_it.loss_functions.msm import MethodOfMomentsLoss\nfrom black_it.samplers.best_batch import BestBatchSampler\nfrom black_it.samplers.halton import HaltonSampler\nfrom black_it.samplers.random_forest import RandomForestSampler\n\ntrue_params = [0.20, 0.20, 0.75]\nbounds = [\n [0.10, 0.10, 0.10], # LOWER bounds\n [1.00, 1.00, 1.00], # UPPER bounds\n]\nbounds_step = [0.01, 0.01, 0.01] # Step size in range between bounds\n\nbatch_size = 8\nhalton_sampler = HaltonSampler(batch_size=batch_size)\nrandom_forest_sampler = RandomForestSampler(batch_size=batch_size)\nbest_batch_sampler = BestBatchSampler(batch_size=batch_size)\n\n# define a model to be calibrated\nmodel = md.MarkovC_KP\n\n# generate a synthetic dataset to test the calibrator\nN = 2000\nseed = 1\nreal_data = model(true_params, N, seed)\n\n# define a loss\nloss = MethodOfMomentsLoss()\n\n# define the calibration seed\ncalibration_seed = 1\n\n# initialize a Calibrator object\ncal = Calibrator(\n samplers=[halton_sampler, random_forest_sampler, best_batch_sampler],\n real_data=real_data,\n model=model,\n parameters_bounds=bounds,\n parameters_precision=bounds_step,\n ensemble_size=3,\n loss_function=loss,\n random_state=calibration_seed,\n)\n\n# calibrate the model\nparams, losses = cal.calibrate(n_batches=15)\n\nprint(f\"True parameters: {true_params}\")\nprint(f\"Best parameters found: {params[0]}\")\n```\n\nWhen the calibration terminates (~half a minute), towards the end of the output\nyou should see the following messages:\n```\nTrue parameters: [0.2, 0.2, 0.75]\nBest parameters found: [0.19 0.21 0.68]\n```\n\n## Docs\n\nBlack-it calibration is initiated via the [Calibrator](https://bancaditalia.github.io/black-it/calibrator/) which,\nwhen called, performs three main steps.\n\nFirst, a [Sampler](https://bancaditalia.github.io/black-it/samplers/) is summoned to suggest a set of promising \nparameter configurations to explore.\n\nSecond, the [model](https://bancaditalia.github.io/black-it/simulator_interface/) to be calibrated is simulated for \nall the selected parameters.\n\nThird, a specific [loss function](https://bancaditalia.github.io/black-it/losses/), measuring the goodness of fitness \nof the simulation data with respect to the real data, is evaluated.\n\nThese steps are performed in a loop, and this allows the samplers to progress towards better parameter values \nby exploiting the knowledge of previously computed loss functions.\n\nA more detailed explanation of how Black-it works is available \n[here](https://bancaditalia.github.io/black-it/description/), while the full documentation -complete with examples \nand tutorials- is available [here](https://bancaditalia.github.io/black-it/). \n\n## Citing *Black-it*\n\nA description of the package is available [here](https://joss.theoj.org/papers/10.21105/joss.04622).\n\nPlease consider citing it if you found this package useful for your research\n\n```bib\n@article{black_it, \n title = {Black-it: A Ready-to-Use and Easy-to-Extend Calibration Kit for Agent-based Models}, \n journal = {Journal of Open Source Software},\n publisher = {The Open Journal}, \n year = {2022}, \n volume = {7}, \n number = {79}, \n pages = {4622}, \n doi = {10.21105/joss.04622}, \n url = {https://doi.org/10.21105/joss.04622}, \n author = {Marco Benedetti and \n Gennaro Catapano and \n Francesco {De Sclavis} and \n Marco Favorito and \n Aldo Glielmo and \n Davide Magnanimi and \n Antonio Muci} \n}\n```\n\n## Disclaimer\n\nThis package is an outcome of a research project. All errors are those of the authors. All views expressed are personal views, not those of Bank of Italy.\n\n---\n\n<p id=\"footnote-1\">\n* Credits to <a href=\"https://www.bankit.art/people/sara-corbo\">Sara Corbo</a> for the logo.\n</p>\n",
"bugtrack_url": null,
"license": "AGPL-3.0-or-later",
"summary": "black-it: Black-box abm calibration kit",
"version": "0.3.1",
"project_urls": {
"Bug Tracker": "https://github.com/bancaditalia/black-it/issues",
"Documentation": "https://bancaditalia.github.io/black-it",
"Homepage": "https://github.com/bancaditalia/black-it",
"Pull Requests": "https://github.com/bancaditalia/black-it/pulls",
"Repository": "https://github.com/bancaditalia/black-it.git"
},
"split_keywords": [
"agent-based models",
" black-box calibration"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1e48dea3749d0ac784697ebc5801c54f5f913e58c506294ae7a38fd1b7789c12",
"md5": "754ced5d1662452ec61df6085484c4a2",
"sha256": "2e3badf49ea91fe3219bc0df69acef9db1ff2e8ec3abb09c5e5f6b30778e73e5"
},
"downloads": -1,
"filename": "black_it-0.3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "754ced5d1662452ec61df6085484c4a2",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.13,>=3.9",
"size": 96521,
"upload_time": "2024-09-11T13:57:59",
"upload_time_iso_8601": "2024-09-11T13:57:59.724605Z",
"url": "https://files.pythonhosted.org/packages/1e/48/dea3749d0ac784697ebc5801c54f5f913e58c506294ae7a38fd1b7789c12/black_it-0.3.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "574c98817054753578f4c39c13b54819848e530f57c40614e5046261636552c0",
"md5": "2468112352229d21a655ccab7a9dfb81",
"sha256": "6bdeaab34fc1035bcad467c9b4b8451182e0220c65cc050a3f6d7f663eb71e4d"
},
"downloads": -1,
"filename": "black_it-0.3.1.tar.gz",
"has_sig": false,
"md5_digest": "2468112352229d21a655ccab7a9dfb81",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.13,>=3.9",
"size": 61899,
"upload_time": "2024-09-11T13:58:01",
"upload_time_iso_8601": "2024-09-11T13:58:01.123874Z",
"url": "https://files.pythonhosted.org/packages/57/4c/98817054753578f4c39c13b54819848e530f57c40614e5046261636552c0/black_it-0.3.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-11 13:58:01",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "bancaditalia",
"github_project": "black-it",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"tox": true,
"lcname": "black-it"
}
JSON
