[](https://github.com/automl/DACBench/actions/workflows/pytest.yml)
[](https://github.com/automl/DACBench/actions/workflows/docs.yml)
DACBench is a benchmark library for Dynamic Algorithm Configuration.
Its focus is on reproducibility and comparability of different DAC methods as well as easy analysis of the optimization process.
You can try out the basics of DACBench in Colab [here](https://colab.research.google.com/drive/1XQhJM2ErvDMu7eSSWFFEedGdw8GRChIE?usp=sharing) without any installation.
Our [examples](https://github.com/automl/DACBench/tree/main/examples) in the repository should give you an impression of what you can do with DACBench and
our [documentation](https://dacbench.readthedocs.io/) should answer any questions you might have.
You can find baseline data of static and random policies for a given version of DACBench on our [project site](https://www.tnt.uni-hannover.de/en/datasets/dacbench/).
## Installation
We recommend installing DACBench in a virtual environment:
```
conda create -n dacbench python=3.10
conda activate dacbench
git clone https://github.com/automl/DACBench.git
cd DACBench
git submodule update --init --recursive
pip install .
```
This command installs the base version of DACBench including the three small surrogate benchmarks and the option to install the FastDownward benchmark.
For any other benchmark, you may use a singularity container as provided by us (see next section) or install it as an additional dependency. As an example,
to install the SGDBenchmark, run:
```
pip install -e .[sgd]
```
To use FastDownward, you first need to build the solver itself. We recommend using
cmake version 3.10.2. The command is:
```
./dacbench/envs/rl-plan/fast-downward/build.py
```
You can also install all dependencies like so:
```
pip install -e .[all,dev,example,docs]
```
## Containerized Benchmarks
DACBench can run containerized versions of Benchmarks using Singularity containers to isolate their dependencies and make reproducible Singularity images.
### Building a Container
For writing your own recipe to build a Container, you can refer to `dacbench/container/singularity_recipes/recipe_template`
Install [Singularity](https://docs.sylabs.io/guides/3.6/user-guide/) and run the following to build the (in this case) cma container
```bash
cd dacbench/container/singularity_recipes
sudo singularity build cma cma.def
```
## Citing DACBench
If you use DACBench in your research or application, please cite us:
```bibtex
@inproceedings{eimer-ijcai21,
author = {T. Eimer and A. Biedenkapp and M. Reimer and S. Adriaensen and F. Hutter and M. Lindauer},
title = {DACBench: A Benchmark Library for Dynamic Algorithm Configuration},
booktitle = {Proceedings of the Thirtieth International Joint Conference on
Artificial Intelligence ({IJCAI}'21)},
year = {2021},
month = aug,
publisher = {ijcai.org},
```
Raw data
{
"_id": null,
"home_page": "https://github.com/automl/{name}",
"name": "DACBench",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": "",
"keywords": "DAC,Dynamic Algorithm Configuration,HPO",
"author": "Theresa Eimer <{author_email}>",
"author_email": "t.eimer@ai.uni-hannover.de",
"download_url": "https://files.pythonhosted.org/packages/d2/b9/51527130805f66c9dc0576d9dfb389f2a5416e01b512c75dcfbd817769ca/DACBench-0.2.0.tar.gz",
"platform": null,
"description": "[](https://github.com/automl/DACBench/actions/workflows/pytest.yml)\n[](https://github.com/automl/DACBench/actions/workflows/docs.yml)\n\nDACBench is a benchmark library for Dynamic Algorithm Configuration.\nIts focus is on reproducibility and comparability of different DAC methods as well as easy analysis of the optimization process.\n\nYou can try out the basics of DACBench in Colab [here](https://colab.research.google.com/drive/1XQhJM2ErvDMu7eSSWFFEedGdw8GRChIE?usp=sharing) without any installation.\nOur [examples](https://github.com/automl/DACBench/tree/main/examples) in the repository should give you an impression of what you can do with DACBench and \nour [documentation](https://dacbench.readthedocs.io/) should answer any questions you might have. \n\nYou can find baseline data of static and random policies for a given version of DACBench on our [project site](https://www.tnt.uni-hannover.de/en/datasets/dacbench/).\n\n\n## Installation\nWe recommend installing DACBench in a virtual environment:\n\n```\nconda create -n dacbench python=3.10\nconda activate dacbench\ngit clone https://github.com/automl/DACBench.git\ncd DACBench\ngit submodule update --init --recursive\npip install .\n```\nThis command installs the base version of DACBench including the three small surrogate benchmarks and the option to install the FastDownward benchmark.\nFor any other benchmark, you may use a singularity container as provided by us (see next section) or install it as an additional dependency. As an example, \nto install the SGDBenchmark, run:\n\n```\npip install -e .[sgd]\n```\n\nTo use FastDownward, you first need to build the solver itself. We recommend using\ncmake version 3.10.2. The command is:\n```\n./dacbench/envs/rl-plan/fast-downward/build.py\n```\n\nYou can also install all dependencies like so:\n```\npip install -e .[all,dev,example,docs]\n```\n\n## Containerized Benchmarks\n\nDACBench can run containerized versions of Benchmarks using Singularity containers to isolate their dependencies and make reproducible Singularity images. \n\n\n### Building a Container\n\nFor writing your own recipe to build a Container, you can refer to `dacbench/container/singularity_recipes/recipe_template` \n\nInstall [Singularity](https://docs.sylabs.io/guides/3.6/user-guide/) and run the following to build the (in this case) cma container\n\n```bash\ncd dacbench/container/singularity_recipes\nsudo singularity build cma cma.def\n```\n\n## Citing DACBench\nIf you use DACBench in your research or application, please cite us:\n\n```bibtex\n@inproceedings{eimer-ijcai21,\n author = {T. Eimer and A. Biedenkapp and M. Reimer and S. Adriaensen and F. Hutter and M. Lindauer},\n title = {DACBench: A Benchmark Library for Dynamic Algorithm Configuration},\n booktitle = {Proceedings of the Thirtieth International Joint Conference on\n Artificial Intelligence ({IJCAI}'21)},\n year = {2021},\n month = aug,\n publisher = {ijcai.org},\n```\n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "",
"version": "0.2.0",
"split_keywords": [
"dac",
"dynamic algorithm configuration",
"hpo"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d2b951527130805f66c9dc0576d9dfb389f2a5416e01b512c75dcfbd817769ca",
"md5": "5d1b2a8f499e0f29a36966e697347182",
"sha256": "c65d7ac157037cf1cc26499b532164e98cb38e825809bd710a9df1b0da785150"
},
"downloads": -1,
"filename": "DACBench-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "5d1b2a8f499e0f29a36966e697347182",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 2913027,
"upload_time": "2023-04-11T11:28:50",
"upload_time_iso_8601": "2023-04-11T11:28:50.963224Z",
"url": "https://files.pythonhosted.org/packages/d2/b9/51527130805f66c9dc0576d9dfb389f2a5416e01b512c75dcfbd817769ca/DACBench-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-04-11 11:28:50",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "automl",
"github_project": "{name}",
"lcname": "dacbench"
}
JSON
