nhelper


Namenhelper JSON
Version 0.1.0 PyPI version JSON
download
home_pagehttps://github.com/JulesBelveze/nhelper
Summary🧪 Behavioral tests for NLP models 🧪
upload_time2022-12-23 12:53:49
maintainer
docs_urlNone
authorJulesBelveze
requires_python>=3.8,<4.0
licenseLICENSE
keywords nlp behavioral testing test
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            <img src="figures/NheLPer.png" width="50%" align="right"/>

# NheLPer

**NheLPer** is Python package designed to ease *behavioral testing* of Natural Language Processing models to identify
possible capability failures.

## 1. About the project

Behavioral tests are intended to test a model against some input data while treating as a black box. The aim is to
observe the model's reaction against some perturbations that might occur once the model is productionized. For a more
detailed explanation on behavioral testing of NLP models I encourage you to read the insightful
paper: [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://arxiv.org/abs/2005.04118)

**NLPtest** provides helper objects for three different aspects:

- easily generate text samples
- test some specific behaviors of your model
- aggregate the tests outcomes of your model

## 2. Getting started

### 2.1. Installation

You can directly install **NheLPer** using [pypi](https://pypi.org/project/nhelper/):

```
pip3 install nhelper
```

### 2.2. Usage

To help you get the hang of the library we provide three different Notebooks to the user, accessible from
the `examples/` folder:

1. `Samples_generation.ipynb`: shows you how to easily generate texts using the `Generator` object.
2. `Please_Behave.ipynb`: getting familiar with the `Behavior` object.
3. `End2End_tests.ipynb`: how to run tests and get an overview of your model behavior.

# References

Below, you can find resources that were used for the creation of **NLPtest** as well as relevant resources about
behavioral testing.

* [MadeWithML](https://madewithml.com/courses/mlops/testing/#behavioral-testing)
* [CheckList](https://github.com/marcotcr/checklist)
* [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://arxiv.org/abs/2005.04118)
            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/JulesBelveze/nhelper",
    "name": "nhelper",
    "maintainer": "",
    "docs_url": null,
    "requires_python": ">=3.8,<4.0",
    "maintainer_email": "",
    "keywords": "nlp,behavioral testing,test",
    "author": "JulesBelveze",
    "author_email": "jules.belveze@hotmail.fr",
    "download_url": "https://files.pythonhosted.org/packages/51/31/ab96d210e5eac0e54aae164310eb70a4fea800ce57bc95e3675e513a1f5c/NheLPer-0.1.0.tar.gz",
    "platform": null,
    "description": "<img src=\"figures/NheLPer.png\" width=\"50%\" align=\"right\"/>\n\n# NheLPer\n\n**NheLPer** is Python package designed to ease *behavioral testing* of Natural Language Processing models to identify\npossible capability failures.\n\n## 1. About the project\n\nBehavioral tests are intended to test a model against some input data while treating as a black box. The aim is to\nobserve the model's reaction against some perturbations that might occur once the model is productionized. For a more\ndetailed explanation on behavioral testing of NLP models I encourage you to read the insightful\npaper: [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://arxiv.org/abs/2005.04118)\n\n**NLPtest** provides helper objects for three different aspects:\n\n- easily generate text samples\n- test some specific behaviors of your model\n- aggregate the tests outcomes of your model\n\n## 2. Getting started\n\n### 2.1. Installation\n\nYou can directly install **NheLPer** using [pypi](https://pypi.org/project/nhelper/):\n\n```\npip3 install nhelper\n```\n\n### 2.2. Usage\n\nTo help you get the hang of the library we provide three different Notebooks to the user, accessible from\nthe `examples/` folder:\n\n1. `Samples_generation.ipynb`: shows you how to easily generate texts using the `Generator` object.\n2. `Please_Behave.ipynb`: getting familiar with the `Behavior` object.\n3. `End2End_tests.ipynb`: how to run tests and get an overview of your model behavior.\n\n# References\n\nBelow, you can find resources that were used for the creation of **NLPtest** as well as relevant resources about\nbehavioral testing.\n\n* [MadeWithML](https://madewithml.com/courses/mlops/testing/#behavioral-testing)\n* [CheckList](https://github.com/marcotcr/checklist)\n* [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://arxiv.org/abs/2005.04118)",
    "bugtrack_url": null,
    "license": "LICENSE",
    "summary": "\ud83e\uddea Behavioral tests for NLP models \ud83e\uddea",
    "version": "0.1.0",
    "split_keywords": [
        "nlp",
        "behavioral testing",
        "test"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "c449208f3b0f26ce7040cdbaf9a3c21d",
                "sha256": "ef6e39c0b9bb574c105ea889dfb1255babe2df6a8be765474e4be17dc4fb1e39"
            },
            "downloads": -1,
            "filename": "NheLPer-0.1.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "c449208f3b0f26ce7040cdbaf9a3c21d",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.8,<4.0",
            "size": 12745,
            "upload_time": "2022-12-23T12:53:47",
            "upload_time_iso_8601": "2022-12-23T12:53:47.943154Z",
            "url": "https://files.pythonhosted.org/packages/1d/08/b3585cf5c8ef07ad7e484465a462d51ce28ec3640fb517a8211bd129a73f/NheLPer-0.1.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "d2015ae63f4f218afe8205b788df05f3",
                "sha256": "92ae854cc78d6d55cc7b4c3d6fa154a844114674f4f8953b43c93d2ddd953bed"
            },
            "downloads": -1,
            "filename": "NheLPer-0.1.0.tar.gz",
            "has_sig": false,
            "md5_digest": "d2015ae63f4f218afe8205b788df05f3",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.8,<4.0",
            "size": 49081,
            "upload_time": "2022-12-23T12:53:49",
            "upload_time_iso_8601": "2022-12-23T12:53:49.609011Z",
            "url": "https://files.pythonhosted.org/packages/51/31/ab96d210e5eac0e54aae164310eb70a4fea800ce57bc95e3675e513a1f5c/NheLPer-0.1.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2022-12-23 12:53:49",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "github_user": "JulesBelveze",
    "github_project": "nhelper",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "lcname": "nhelper"
}
        
Elapsed time: 0.06764s