# DNN-TIP: Common Test Input Prioritizers Library
[](https://github.com/testingautomated-usi/dnn-tip/actions/workflows/test.yml)
[](https://github.com/psf/black)
[](https://pycqa.github.io/isort/)
[](https://img.shields.io/pypi/pyversions/dnn-tip)
[](https://pypi.org/project/dnn-tip/)
[](https://github.com/testingautomated-usi/dnn-tip/blob/develop/LICENSE)
[](https://zenodo.org/badge/latestdoi/478142616)
## Implemented Approaches
* __Surprise Adequacies__
* Distance-based Surprise Adequacy (DSA)
* Likelihood-based Surprise Adequacy (LSA)
* MultiModal-Likelihood-based Surprise Adequacy (MLSA)
* Mahalanobis-based Surprise Adequacy (MDSA)
* _abstract_ MultiModal Surprise Adequacy
* __Surprise Coverage__
* Neuron-Activation Coverage (NAC)
* K-Multisection Neuron Coverage (KMNC)
* Neuron Boundary Coverage (NBC)
* Strong Neuron Activation Coverage (SNAC)
* Top-k Neuron Coverage (TKNC)
* __Utilities__
* APFD calculation
* Coverage-Added and Coverage-Total Prioritization Methods (CAM and CTM)
If you are looking for the uncertainty metrics we also tested (including DeepGini),
head over to the sister repository [uncertainty-wizard](https://github.com/testingautomated-usi/uncertainty-wizard).
If you want to reproduce our exact experiments, there's a reproduction package and docker stuff available at [testingautomated-usi/simple-tip](https://github.com/testingautomated-usi/simple-tip).
## Installation
It's as easy as `pip install dnn-tip`.
## Documentation
Find the documentation at [https://testingautomated-usi.github.io/dnn-tip/](https://testingautomated-usi.github.io/dnn-tip/).
## Citation
Here's the reference to the paper as part of which this library was release:
```
@inproceedings{10.1145/3533767.3534375,
author = {Weiss, Michael and Tonella, Paolo},
title = {Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study)},
year = {2022},
isbn = {9781450393799},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3533767.3534375},
doi = {10.1145/3533767.3534375},
booktitle = {Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis},
pages = {139–150},
numpages = {12},
keywords = {neural networks, Test prioritization, uncertainty quantification},
location = {Virtual, South Korea},
series = {ISSTA 2022}
}
Raw data
{
"_id": null,
"home_page": "https://github.com/testingautomated-usi/dnn-tip",
"name": "dnn-tip",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "Michael Weiss",
"author_email": "michael.weiss@usi.ch",
"download_url": "https://files.pythonhosted.org/packages/d4/f3/714e5017632ca779fd548a89b54e181e8d2a5457270954b4b6a3398bbc08/dnn-tip-0.1.1.tar.gz",
"platform": null,
"description": "# DNN-TIP: Common Test Input Prioritizers Library \n\n[](https://github.com/testingautomated-usi/dnn-tip/actions/workflows/test.yml)\n[](https://github.com/psf/black)\n\n[](https://pycqa.github.io/isort/)\n[](https://img.shields.io/pypi/pyversions/dnn-tip)\n[](https://pypi.org/project/dnn-tip/)\n[](https://github.com/testingautomated-usi/dnn-tip/blob/develop/LICENSE)\n[](https://zenodo.org/badge/latestdoi/478142616)\n\n\n## Implemented Approaches\n* __Surprise Adequacies__\n * Distance-based Surprise Adequacy (DSA)\n * Likelihood-based Surprise Adequacy (LSA)\n * MultiModal-Likelihood-based Surprise Adequacy (MLSA)\n * Mahalanobis-based Surprise Adequacy (MDSA)\n * _abstract_ MultiModal Surprise Adequacy\n* __Surprise Coverage__\n * Neuron-Activation Coverage (NAC)\n * K-Multisection Neuron Coverage (KMNC)\n * Neuron Boundary Coverage (NBC)\n * Strong Neuron Activation Coverage (SNAC)\n * Top-k Neuron Coverage (TKNC)\n* __Utilities__\n * APFD calculation\n * Coverage-Added and Coverage-Total Prioritization Methods (CAM and CTM)\n\nIf you are looking for the uncertainty metrics we also tested (including DeepGini),\nhead over to the sister repository [uncertainty-wizard](https://github.com/testingautomated-usi/uncertainty-wizard).\n\nIf you want to reproduce our exact experiments, there's a reproduction package and docker stuff available at [testingautomated-usi/simple-tip](https://github.com/testingautomated-usi/simple-tip).\n\n\n\n\n## Installation\nIt's as easy as `pip install dnn-tip`.\n\n\n\n\n## Documentation\n\nFind the documentation at [https://testingautomated-usi.github.io/dnn-tip/](https://testingautomated-usi.github.io/dnn-tip/).\n\n\n## Citation\n\nHere's the reference to the paper as part of which this library was release:\n\n```\n@inproceedings{10.1145/3533767.3534375,\nauthor = {Weiss, Michael and Tonella, Paolo},\ntitle = {Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study)},\nyear = {2022},\nisbn = {9781450393799},\npublisher = {Association for Computing Machinery},\naddress = {New York, NY, USA},\nurl = {https://doi.org/10.1145/3533767.3534375},\ndoi = {10.1145/3533767.3534375},\nbooktitle = {Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis},\npages = {139\u2013150},\nnumpages = {12},\nkeywords = {neural networks, Test prioritization, uncertainty quantification},\nlocation = {Virtual, South Korea},\nseries = {ISSTA 2022}\n}\n\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A collection of DNN test input prioritizers,in particular neuron coverage and surprise adequacy.",
"version": "0.1.1",
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "af9e56e68c88b4059581179cdd0d56f2652dc381374ee754e9c9a2303698b668",
"md5": "ebf97405bd2fc8ea205aeb1d5fbd9130",
"sha256": "544763e798e6db1289296f673214bbb57e6878c0f9489041df10cc8771df917a"
},
"downloads": -1,
"filename": "dnn_tip-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ebf97405bd2fc8ea205aeb1d5fbd9130",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 15621,
"upload_time": "2023-02-03T09:31:21",
"upload_time_iso_8601": "2023-02-03T09:31:21.212752Z",
"url": "https://files.pythonhosted.org/packages/af/9e/56e68c88b4059581179cdd0d56f2652dc381374ee754e9c9a2303698b668/dnn_tip-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d4f3714e5017632ca779fd548a89b54e181e8d2a5457270954b4b6a3398bbc08",
"md5": "84dfdc4fdbdbe88e243d93689d7f4398",
"sha256": "c89eb227329b3b264b3eccbb1812ca12058245862a28948ff928edb683505281"
},
"downloads": -1,
"filename": "dnn-tip-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "84dfdc4fdbdbe88e243d93689d7f4398",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 15494,
"upload_time": "2023-02-03T09:31:23",
"upload_time_iso_8601": "2023-02-03T09:31:23.002195Z",
"url": "https://files.pythonhosted.org/packages/d4/f3/714e5017632ca779fd548a89b54e181e8d2a5457270954b4b6a3398bbc08/dnn-tip-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-02-03 09:31:23",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "testingautomated-usi",
"github_project": "dnn-tip",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "dnn-tip"
}
JSON
