<img src="figures/NheLPer.png" width="50%" align="right"/>
# NheLPer
**NheLPer** is Python package designed to ease *behavioral testing* of Natural Language Processing models to identify
possible capability failures.
## 1. About the project
Behavioral tests are intended to test a model against some input data while treating as a black box. The aim is to
observe the model's reaction against some perturbations that might occur once the model is productionized. For a more
detailed explanation on behavioral testing of NLP models I encourage you to read the insightful
paper: [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://arxiv.org/abs/2005.04118)
**NLPtest** provides helper objects for three different aspects:
- easily generate text samples
- test some specific behaviors of your model
- aggregate the tests outcomes of your model
## 2. Getting started
### 2.1. Installation
You can directly install **NheLPer** using [pypi](https://pypi.org/project/nhelper/):
```
pip3 install nhelper
```
### 2.2. Usage
To help you get the hang of the library we provide three different Notebooks to the user, accessible from
the `examples/` folder:
1. `Samples_generation.ipynb`: shows you how to easily generate texts using the `Generator` object.
2. `Please_Behave.ipynb`: getting familiar with the `Behavior` object.
3. `End2End_tests.ipynb`: how to run tests and get an overview of your model behavior.
# References
Below, you can find resources that were used for the creation of **NLPtest** as well as relevant resources about
behavioral testing.
* [MadeWithML](https://madewithml.com/courses/mlops/testing/#behavioral-testing)
* [CheckList](https://github.com/marcotcr/checklist)
* [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://arxiv.org/abs/2005.04118)
Raw data
{
"_id": null,
"home_page": "https://github.com/JulesBelveze/nhelper",
"name": "nhelper",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4.0",
"maintainer_email": "",
"keywords": "nlp,behavioral testing,test",
"author": "JulesBelveze",
"author_email": "jules.belveze@hotmail.fr",
"download_url": "https://files.pythonhosted.org/packages/51/31/ab96d210e5eac0e54aae164310eb70a4fea800ce57bc95e3675e513a1f5c/NheLPer-0.1.0.tar.gz",
"platform": null,
"description": "<img src=\"figures/NheLPer.png\" width=\"50%\" align=\"right\"/>\n\n# NheLPer\n\n**NheLPer** is Python package designed to ease *behavioral testing* of Natural Language Processing models to identify\npossible capability failures.\n\n## 1. About the project\n\nBehavioral tests are intended to test a model against some input data while treating as a black box. The aim is to\nobserve the model's reaction against some perturbations that might occur once the model is productionized. For a more\ndetailed explanation on behavioral testing of NLP models I encourage you to read the insightful\npaper: [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://arxiv.org/abs/2005.04118)\n\n**NLPtest** provides helper objects for three different aspects:\n\n- easily generate text samples\n- test some specific behaviors of your model\n- aggregate the tests outcomes of your model\n\n## 2. Getting started\n\n### 2.1. Installation\n\nYou can directly install **NheLPer** using [pypi](https://pypi.org/project/nhelper/):\n\n```\npip3 install nhelper\n```\n\n### 2.2. Usage\n\nTo help you get the hang of the library we provide three different Notebooks to the user, accessible from\nthe `examples/` folder:\n\n1. `Samples_generation.ipynb`: shows you how to easily generate texts using the `Generator` object.\n2. `Please_Behave.ipynb`: getting familiar with the `Behavior` object.\n3. `End2End_tests.ipynb`: how to run tests and get an overview of your model behavior.\n\n# References\n\nBelow, you can find resources that were used for the creation of **NLPtest** as well as relevant resources about\nbehavioral testing.\n\n* [MadeWithML](https://madewithml.com/courses/mlops/testing/#behavioral-testing)\n* [CheckList](https://github.com/marcotcr/checklist)\n* [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://arxiv.org/abs/2005.04118)",
"bugtrack_url": null,
"license": "LICENSE",
"summary": "\ud83e\uddea Behavioral tests for NLP models \ud83e\uddea",
"version": "0.1.0",
"split_keywords": [
"nlp",
"behavioral testing",
"test"
],
"urls": [
{
"comment_text": "",
"digests": {
"md5": "c449208f3b0f26ce7040cdbaf9a3c21d",
"sha256": "ef6e39c0b9bb574c105ea889dfb1255babe2df6a8be765474e4be17dc4fb1e39"
},
"downloads": -1,
"filename": "NheLPer-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c449208f3b0f26ce7040cdbaf9a3c21d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4.0",
"size": 12745,
"upload_time": "2022-12-23T12:53:47",
"upload_time_iso_8601": "2022-12-23T12:53:47.943154Z",
"url": "https://files.pythonhosted.org/packages/1d/08/b3585cf5c8ef07ad7e484465a462d51ce28ec3640fb517a8211bd129a73f/NheLPer-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"md5": "d2015ae63f4f218afe8205b788df05f3",
"sha256": "92ae854cc78d6d55cc7b4c3d6fa154a844114674f4f8953b43c93d2ddd953bed"
},
"downloads": -1,
"filename": "NheLPer-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "d2015ae63f4f218afe8205b788df05f3",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4.0",
"size": 49081,
"upload_time": "2022-12-23T12:53:49",
"upload_time_iso_8601": "2022-12-23T12:53:49.609011Z",
"url": "https://files.pythonhosted.org/packages/51/31/ab96d210e5eac0e54aae164310eb70a4fea800ce57bc95e3675e513a1f5c/NheLPer-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-12-23 12:53:49",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "JulesBelveze",
"github_project": "nhelper",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "nhelper"
}
JSON
