| Name | bteval JSON |
| Version |
0.2.0
JSON |
| download |
| home_page | None |
| Summary | BTEval is a Python library for measuring the robustness of natural language understanding models to speech recognition errors. |
| upload_time | 2024-02-17 23:53:20 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.10 |
| license | None |
| keywords |
asr
backtranscription
nlu
tts
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# BTEval
[](https://pypi.org/project/bteval)
[](https://pypi.org/project/bteval)
-----
BTEval is a Python library for measuring the robustness of natural language understanding models to speech recognition errors.
It implements the family of `R*` robustness measures defined in [Back Transcription as a Method for Evaluating Robustness of Natural Language Understanding Models to Speech Recognition Errors](https://aclanthology.org/2023.emnlp-main.724) (Kubis et al., EMNLP 2023).
## Installation
```console
pip install bteval
```
## Usage
```python
from bteval import r1_score
y_true = ["Inform", "Request", "Inform"]
y_before = ["Inform", "Request", "Request"]
y_after = ["Inform", "Confirm", "Confirm"]
r1_score(y_true, y_before, y_after)
```
## Citing
If you use bteval for your research, please cite the following paper:
Marek Kubis, Paweł Skórzewski, Marcin Sowański, and Tomasz Zietkiewicz. 2023. [Back Transcription as a Method for Evaluating Robustness of Natural Language Understanding Models to Speech Recognition Errors](https://aclanthology.org/2023.emnlp-main.724). In *Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*, pages 11824–11835, Singapore. Association for Computational Linguistics.
```bibtex
@InProceedings{kubis-etal-2023-back,
title = "{Back Transcription as a Method for Evaluating Robustness of Natural Language Understanding Models to Speech Recognition Errors}",
author = "Kubis, Marek and Skórzewski, Paweł and Sowański, Marcin and Ziętkiewicz, Tomasz",
booktitle = "Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing",
month = dec,
year = "2023",
address = "Singapore",
publisher = "Association for Computational Linguistics",
URL = "https://aclanthology.org/2023.emnlp-main.724",
doi = "10.18653/v1/2023.emnlp-main.724",
pages = "11824--11835",
}
```
Raw data
{
"_id": null,
"home_page": null,
"name": "bteval",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "asr,backtranscription,nlu,tts",
"author": null,
"author_email": "Marek Kubis <marek.kubis@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/7d/1e/2b3ca4440eb592d59a645af8b022a3fa5899f10b2a755f51975e6d7d281d/bteval-0.2.0.tar.gz",
"platform": null,
"description": "# BTEval\n\n[](https://pypi.org/project/bteval)\n[](https://pypi.org/project/bteval)\n\n-----\n\nBTEval is a Python library for measuring the robustness of natural language understanding models to speech recognition errors.\nIt implements the family of `R*` robustness measures defined in [Back Transcription as a Method for Evaluating Robustness of Natural Language Understanding Models to Speech Recognition Errors](https://aclanthology.org/2023.emnlp-main.724) (Kubis et al., EMNLP 2023).\n\n## Installation\n\n```console\npip install bteval\n```\n\n## Usage\n\n```python\nfrom bteval import r1_score\n\ny_true = [\"Inform\", \"Request\", \"Inform\"]\ny_before = [\"Inform\", \"Request\", \"Request\"]\ny_after = [\"Inform\", \"Confirm\", \"Confirm\"]\n\nr1_score(y_true, y_before, y_after)\n```\n\n## Citing\n\nIf you use bteval for your research, please cite the following paper:\n\nMarek Kubis, Pawe\u0142 Sk\u00f3rzewski, Marcin Sowa\u0144ski, and Tomasz Zietkiewicz. 2023. [Back Transcription as a Method for Evaluating Robustness of Natural Language Understanding Models to Speech Recognition Errors](https://aclanthology.org/2023.emnlp-main.724). In *Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*, pages 11824\u201311835, Singapore. Association for Computational Linguistics.\n\n```bibtex\n@InProceedings{kubis-etal-2023-back,\n title = \"{Back Transcription as a Method for Evaluating Robustness of Natural Language Understanding Models to Speech Recognition Errors}\",\n author = \"Kubis, Marek and Sk\u00f3rzewski, Pawe\u0142 and Sowa\u0144ski, Marcin and Zi\u0119tkiewicz, Tomasz\",\n booktitle = \"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing\",\n month = dec,\n year = \"2023\",\n address = \"Singapore\",\n publisher = \"Association for Computational Linguistics\",\n URL = \"https://aclanthology.org/2023.emnlp-main.724\",\n doi = \"10.18653/v1/2023.emnlp-main.724\",\n pages = \"11824--11835\",\n}\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "BTEval is a Python library for measuring the robustness of natural language understanding models to speech recognition errors.",
"version": "0.2.0",
"project_urls": {
"Documentation": "https://github.com/marekkubis/bteval#readme",
"Homepage": "https://github.com/marekkubis/bteval",
"Issues": "https://github.com/marekkubis/bteval/issues",
"Source": "https://github.com/marekkubis/bteval"
},
"split_keywords": [
"asr",
"backtranscription",
"nlu",
"tts"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "31cbca9e25f7202393a4ba74b7ae1b2809139342d45261ad9f0f844b0d46c37e",
"md5": "79190c7fc9d5150fde99387a2b1345cc",
"sha256": "11312224fbf13b7187b941a83135a812c7bff0acdb2660ae5942b4847d303bab"
},
"downloads": -1,
"filename": "bteval-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "79190c7fc9d5150fde99387a2b1345cc",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 5493,
"upload_time": "2024-02-17T23:53:22",
"upload_time_iso_8601": "2024-02-17T23:53:22.076558Z",
"url": "https://files.pythonhosted.org/packages/31/cb/ca9e25f7202393a4ba74b7ae1b2809139342d45261ad9f0f844b0d46c37e/bteval-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "7d1e2b3ca4440eb592d59a645af8b022a3fa5899f10b2a755f51975e6d7d281d",
"md5": "0179b7d40af00da4525cf53515722463",
"sha256": "2dc917a0cec90a133581bef25fba93dbc08fea2b46d2250a6fdc968d733f8ea7"
},
"downloads": -1,
"filename": "bteval-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "0179b7d40af00da4525cf53515722463",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 6462,
"upload_time": "2024-02-17T23:53:20",
"upload_time_iso_8601": "2024-02-17T23:53:20.489145Z",
"url": "https://files.pythonhosted.org/packages/7d/1e/2b3ca4440eb592d59a645af8b022a3fa5899f10b2a755f51975e6d7d281d/bteval-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-17 23:53:20",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "marekkubis",
"github_project": "bteval#readme",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "bteval"
}
