# PEtab test suite
The PEtab test suite is a collection of test models for the
[PEtab](https://github.com/petab-dev/petab) parameter estimation data format.
It is intended to be used to verify and quantify PEtab-support by developers
of tools for model simulation and parameter estimation.
## Download and install the test suite
The PEtab test suite can be downloaded from GitHub via
git clone https://github.com/petab-dev/petab_test_suite
The test suite comes with all necessary files pregenerated.
In the [petabtests](petabtests) subdirectory, it contains a python module for
generating the tests and evaluating results. This can be installed via
cd petab_test_suite
pip3 install -e .
## Use the test suite
The [cases](cases) subdirectory contains a collection of enumerated tests.
Each test contains a descriptive `wxyz.md` file, and a script file `wxyz.py`
file that can be used to generate all problem and solution files for the test.
The necessary files are in the same case specific folder, starting with an
underscore.
In each case folder, there is a file `_wxyz.yaml` containing the parameter
estimation problem description, and a file `_wxyz_solution.yaml` containing
information on the expected results: chi2 value, log-likelihood, simulation
table reference, and tolerances.
After download, all necessary files are already pregenerated.
Further, installation of the petabtests module installs a routine to recreate
the problem and solution files for all test problems. It can be invoked via
cd petab_test_suite
petabtests_create
## Evaluate results
To evaluate how a tool performs on a given test problem, three metrics are
employed: Simulations, chi2 value and log-likelihood. A tool can be said to
cover a test problem if any of those values matches the ground truth values
up to some tolerance.
The python module provides convenience functions for evaluation in
[petabtests/evaluate.py](petabtests/evaluate.py).
-> [Overview of passed test cases for different tools supporting PEtab](https://github.com/PEtab-dev/PEtab#petab-features-supported-in-different-tools)
Raw data
{
"_id": null,
"home_page": "https://github.com/PEtab-dev/petab_test_suite",
"name": "petabtests",
"maintainer": "PEtab-dev",
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "yannik.schaelte@gmail.com",
"keywords": "PEtab testsuite",
"author": "PEtab-dev",
"author_email": "yannik.schaelte@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/70/47/dc025a727b6f82a4b61c1ac9c0f6a988829b2db6468b54a56ab07c1cd46d/petabtests-0.0.1.tar.gz",
"platform": null,
"description": "# PEtab test suite\n\nThe PEtab test suite is a collection of test models for the\n[PEtab](https://github.com/petab-dev/petab) parameter estimation data format.\nIt is intended to be used to verify and quantify PEtab-support by developers\nof tools for model simulation and parameter estimation.\n\n## Download and install the test suite\n\nThe PEtab test suite can be downloaded from GitHub via\n\n git clone https://github.com/petab-dev/petab_test_suite\n\nThe test suite comes with all necessary files pregenerated. \nIn the [petabtests](petabtests) subdirectory, it contains a python module for\ngenerating the tests and evaluating results. This can be installed via\n\n cd petab_test_suite\n pip3 install -e .\n\n## Use the test suite\n\nThe [cases](cases) subdirectory contains a collection of enumerated tests.\nEach test contains a descriptive `wxyz.md` file, and a script file `wxyz.py`\nfile that can be used to generate all problem and solution files for the test.\nThe necessary files are in the same case specific folder, starting with an\nunderscore. \nIn each case folder, there is a file `_wxyz.yaml` containing the parameter\nestimation problem description, and a file `_wxyz_solution.yaml` containing\ninformation on the expected results: chi2 value, log-likelihood, simulation\ntable reference, and tolerances.\n\nAfter download, all necessary files are already pregenerated.\n\nFurther, installation of the petabtests module installs a routine to recreate\nthe problem and solution files for all test problems. It can be invoked via\n\n cd petab_test_suite\n petabtests_create\n\n## Evaluate results\n\nTo evaluate how a tool performs on a given test problem, three metrics are\nemployed: Simulations, chi2 value and log-likelihood. A tool can be said to\ncover a test problem if any of those values matches the ground truth values\nup to some tolerance.\n\nThe python module provides convenience functions for evaluation in\n[petabtests/evaluate.py](petabtests/evaluate.py).\n\n-> [Overview of passed test cases for different tools supporting PEtab](https://github.com/PEtab-dev/PEtab#petab-features-supported-in-different-tools)\n",
"bugtrack_url": null,
"license": "",
"summary": "PEtab testsuite library",
"version": "0.0.1",
"project_urls": {
"Homepage": "https://github.com/PEtab-dev/petab_test_suite"
},
"split_keywords": [
"petab",
"testsuite"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7047dc025a727b6f82a4b61c1ac9c0f6a988829b2db6468b54a56ab07c1cd46d",
"md5": "488b703262c5224b76a710d5fb0e667a",
"sha256": "0dd3095ac49ff22b3ecb82ceb87124ced12d284f52497e2e9cc5dad1004602cc"
},
"downloads": -1,
"filename": "petabtests-0.0.1.tar.gz",
"has_sig": false,
"md5_digest": "488b703262c5224b76a710d5fb0e667a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 53026,
"upload_time": "2023-06-20T10:10:30",
"upload_time_iso_8601": "2023-06-20T10:10:30.516777Z",
"url": "https://files.pythonhosted.org/packages/70/47/dc025a727b6f82a4b61c1ac9c0f6a988829b2db6468b54a56ab07c1cd46d/petabtests-0.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-20 10:10:30",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "PEtab-dev",
"github_project": "petab_test_suite",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "petabtests"
}
JSON
