# NLP Test: Deliver Safe & Effective Language Models
<p align="center">
<a href="https://github.com/JohnSnowLabs/nlptest/actions" alt="build">
<img src="https://github.com/JohnSnowLabs/nlptest/workflows/build/badge.svg" /></a>
<a href="https://github.com/JohnSnowLabs/nlptest/releases" alt="Current Release Version">
<img src="https://img.shields.io/github/v/release/JohnSnowLabs/nlptest.svg?style=flat-square&logo=github" /></a>
<a href="https://github.com/JohnSnowLabs/nlptest/blob/master/LICENSE" alt="License">
<img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" /></a>
<a href="https://pypi.org/project/nlptest/" alt="PyPi downloads">
<img src="https://static.pepy.tech/personalized-badge/nlptest?period=total&units=international_system&left_color=grey&right_color=green&left_text=Downloads" /></a>
<a href="https://anaconda.org/conda-forge/nlptest" alt="Conda Version">
<img src="https://img.shields.io/conda/vn/conda-forge/nlptest.svg?style=flat-square&color=blue&logo=conda-forge" /></a>
</p>
<p align="center">
<a href="#project's-website">Project's Website</a> •
<a href="#key-features">Key Features</a> •
<a href="#how-to-use">How To Use</a> •
<a href="#community-support">Community Support</a> •
<a href="#contributing">Contributing</a> •
<a href="#mission">Mission</a> •
<a href="#license">License</a>
</p>
## Project's Website
Take a look at our official page for user documentation and examples: [nlptest.org](http://nlptest.org/)
## Key Features
- Generate and execute more than 50 distinct types of tests only with 1 line of code
- Test all aspects of model quality: robustness, bias, representation, fairness and accuracy.
- Automatically augment training data based on test results (for select models)
- Support for popular NLP frameworks for NER and Text-Classifcation: Spark NLP, Hugging Face & Transformers.
- Support for testing LLMS ( OpenAI, Cohere, AI21, Hugging Face Inference API and Azure-OpenAI LLMs) for question answering and summarization task.
## How To Use
```python
# Install nlptest
!pip install nlptest
# Import and create a Harness object
from nlptest import Harness
h = Harness(task='ner', model='dslim/bert-base-NER', hub='huggingface')
# Generate test cases, run them and view a report
h.generate().run().report()
```
> **Note**
> For more extended examples of usage and documentation, head over to [nlptest.org](https://www.nlptest.org)
## Community Support
- [Slack](https://www.johnsnowlabs.com/slack-redirect/) For live discussion with the NLP Test community, join the `#nlptest` channel
- [GitHub](https://github.com/JohnSnowLabs/nlptest/tree/main) For bug reports, feature requests, and contributions
- [Discussions](https://github.com/JohnSnowLabs/nlptest/discussions) To engage with other community members, share ideas, and show off how you use NLP Test!
## Mission
While there is a lot of talk about the need to train AI models that are safe, robust, and fair - few tools have been made available to data scientists to meet these goals. As a result, the front line of NLP models in production systems reflects a sorry state of affairs.
We propose here an early stage open-source community project that aims to fill this gap, and would love for you to join us on this mission. We aim to build on the foundation laid by previous research such as [Ribeiro et al. (2020)](https://arxiv.org/abs/2005.04118), [Song et al. (2020)](https://arxiv.org/abs/2004.00053), [Parrish et al. (2021)](https://arxiv.org/abs/2110.08193), [van Aken et al. (2021)](https://arxiv.org/abs/2111.15512) and many others.
[John Snow Labs](www.johnsnowlabs.com) has a full development team allocated to the project and is committed to improving the library for years, as we do with other open-source libraries. Expect frequent releases with new test types, tasks, languages, and platforms to be added regularly. We look forward to working together to make safe, reliable, and responsible NLP an everyday reality.
## Contributing
We welcome all sorts of contributions:
- Ideas
- Feedback
- Documentation
- Bug reports
- Development and testing
Feel free to clone the repo and submit pull-requests! You can also contribute by simply opening an issue or discussion in this repo.
## Contributors
We would like to acknowledge all contributors of this open-source community project.
<a href="https://github.com/johnsnowlabs/nlptest/graphs/contributors">
<img src="https://contrib.rocks/image?repo=johnsnowlabs/nlptest" />
</a>
## License
NLP Test is released under the [Apache License 2.0](https://github.com/JohnSnowLabs/nlptest/blob/main/LICENSE), which guarantees commercial use, modification, distribution, patent use, private use and sets limitations on trademark use, liability and warranty.
Raw data
{
"_id": null,
"home_page": "https://www.nlptest.org",
"name": "nlptest",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7, <3.12",
"maintainer_email": "",
"keywords": "NLP testing robustness bias fairness representation accuracy",
"author": "John Snow Labs",
"author_email": "support@johnsnowlabs.com",
"download_url": "https://files.pythonhosted.org/packages/87/60/a076c56666dc4b4151c8b0bd2e92e197d1c9478eaf4927370df545e9863f/nlptest-1.5.0.tar.gz",
"platform": null,
"description": "# NLP Test: Deliver Safe & Effective Language Models\r\n\r\n<p align=\"center\">\r\n <a href=\"https://github.com/JohnSnowLabs/nlptest/actions\" alt=\"build\">\r\n <img src=\"https://github.com/JohnSnowLabs/nlptest/workflows/build/badge.svg\" /></a>\r\n <a href=\"https://github.com/JohnSnowLabs/nlptest/releases\" alt=\"Current Release Version\">\r\n <img src=\"https://img.shields.io/github/v/release/JohnSnowLabs/nlptest.svg?style=flat-square&logo=github\" /></a>\r\n <a href=\"https://github.com/JohnSnowLabs/nlptest/blob/master/LICENSE\" alt=\"License\">\r\n <img src=\"https://img.shields.io/badge/License-Apache%202.0-blue.svg\" /></a>\r\n <a href=\"https://pypi.org/project/nlptest/\" alt=\"PyPi downloads\">\r\n <img src=\"https://static.pepy.tech/personalized-badge/nlptest?period=total&units=international_system&left_color=grey&right_color=green&left_text=Downloads\" /></a>\r\n <a href=\"https://anaconda.org/conda-forge/nlptest\" alt=\"Conda Version\">\r\n <img src=\"https://img.shields.io/conda/vn/conda-forge/nlptest.svg?style=flat-square&color=blue&logo=conda-forge\" /></a>\r\n</p>\r\n\r\n\r\n<p align=\"center\">\r\n <a href=\"#project's-website\">Project's Website</a> \u2022\r\n <a href=\"#key-features\">Key Features</a> \u2022\r\n <a href=\"#how-to-use\">How To Use</a> \u2022\r\n <a href=\"#community-support\">Community Support</a> \u2022\r\n <a href=\"#contributing\">Contributing</a> \u2022\r\n <a href=\"#mission\">Mission</a> \u2022\r\n <a href=\"#license\">License</a>\r\n</p>\r\n\r\n\r\n\r\n## Project's Website\r\n\r\nTake a look at our official page for user documentation and examples: [nlptest.org](http://nlptest.org/) \r\n\r\n## Key Features\r\n\r\n- Generate and execute more than 50 distinct types of tests only with 1 line of code\r\n- Test all aspects of model quality: robustness, bias, representation, fairness and accuracy.\u200b\r\n- Automatically augment training data based on test results (for select models)\u200b\r\n- Support for popular NLP frameworks for NER and Text-Classifcation: Spark NLP, Hugging Face & Transformers.\r\n- Support for testing LLMS ( OpenAI, Cohere, AI21, Hugging Face Inference API and Azure-OpenAI LLMs) for question answering and summarization task. \r\n\r\n## How To Use\r\n\r\n```python\r\n# Install nlptest\r\n!pip install nlptest\r\n\r\n# Import and create a Harness object\r\nfrom nlptest import Harness\r\nh = Harness(task='ner', model='dslim/bert-base-NER', hub='huggingface')\r\n\r\n# Generate test cases, run them and view a report\r\nh.generate().run().report()\r\n```\r\n\r\n> **Note**\r\n> For more extended examples of usage and documentation, head over to [nlptest.org](https://www.nlptest.org)\r\n\r\n## Community Support\r\n\r\n- [Slack](https://www.johnsnowlabs.com/slack-redirect/) For live discussion with the NLP Test community, join the `#nlptest` channel\r\n- [GitHub](https://github.com/JohnSnowLabs/nlptest/tree/main) For bug reports, feature requests, and contributions\r\n- [Discussions](https://github.com/JohnSnowLabs/nlptest/discussions) To engage with other community members, share ideas, and show off how you use NLP Test!\r\n\r\n## Mission\r\n\r\nWhile there is a lot of talk about the need to train AI models that are safe, robust, and fair - few tools have been made available to data scientists to meet these goals. As a result, the front line of NLP models in production systems reflects a sorry state of affairs. \r\n\r\nWe propose here an early stage open-source community project that aims to fill this gap, and would love for you to join us on this mission. We aim to build on the foundation laid by previous research such as [Ribeiro et al. (2020)](https://arxiv.org/abs/2005.04118), [Song et al. (2020)](https://arxiv.org/abs/2004.00053), [Parrish et al. (2021)](https://arxiv.org/abs/2110.08193), [van Aken et al. (2021)](https://arxiv.org/abs/2111.15512) and many others. \r\n\r\n[John Snow Labs](www.johnsnowlabs.com) has a full development team allocated to the project and is committed to improving the library for years, as we do with other open-source libraries. Expect frequent releases with new test types, tasks, languages, and platforms to be added regularly. We look forward to working together to make safe, reliable, and responsible NLP an everyday reality. \r\n\r\n## Contributing\r\n\r\nWe welcome all sorts of contributions:\r\n\r\n- Ideas\r\n- Feedback\r\n- Documentation\r\n- Bug reports\r\n- Development and testing\r\n\r\nFeel free to clone the repo and submit pull-requests! You can also contribute by simply opening an issue or discussion in this repo.\r\n\r\n## Contributors\r\n\r\nWe would like to acknowledge all contributors of this open-source community project. \r\n\r\n<a href=\"https://github.com/johnsnowlabs/nlptest/graphs/contributors\">\r\n <img src=\"https://contrib.rocks/image?repo=johnsnowlabs/nlptest\" />\r\n</a>\r\n\r\n## License\r\n\r\nNLP Test is released under the [Apache License 2.0](https://github.com/JohnSnowLabs/nlptest/blob/main/LICENSE), which guarantees commercial use, modification, distribution, patent use, private use and sets limitations on trademark use, liability and warranty.\r\n\r\n",
"bugtrack_url": null,
"license": "",
"summary": "John Snow Labs provides a library for delivering safe & effective NLP models.",
"version": "1.5.0",
"project_urls": {
"Bug Reports": "https://github.com/JohnSnowLabs/nlptest/issues",
"Homepage": "https://www.nlptest.org",
"Source": "https://github.com/JohnSnowLabs/nlptest/"
},
"split_keywords": [
"nlp",
"testing",
"robustness",
"bias",
"fairness",
"representation",
"accuracy"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a4e19cedcec0e256fcedfac3c42ce0ad8e2a1cef61ad782c0e14bb890f962b85",
"md5": "37101b7e9945338dca9ace6c6967ecea",
"sha256": "9af341d89f0561479f3b5229e1b207c2a252c8149ddc68908b6cebd8d4890111"
},
"downloads": -1,
"filename": "nlptest-1.5.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "37101b7e9945338dca9ace6c6967ecea",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7, <3.12",
"size": 59660219,
"upload_time": "2023-06-16T12:52:50",
"upload_time_iso_8601": "2023-06-16T12:52:50.019773Z",
"url": "https://files.pythonhosted.org/packages/a4/e1/9cedcec0e256fcedfac3c42ce0ad8e2a1cef61ad782c0e14bb890f962b85/nlptest-1.5.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8760a076c56666dc4b4151c8b0bd2e92e197d1c9478eaf4927370df545e9863f",
"md5": "097736cef15c944d96a1b883d3e52bdb",
"sha256": "556ac72ef3e8a1c1851bbfe8d22e4312f01e7607d6d3a44775279766744ff9e2"
},
"downloads": -1,
"filename": "nlptest-1.5.0.tar.gz",
"has_sig": false,
"md5_digest": "097736cef15c944d96a1b883d3e52bdb",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7, <3.12",
"size": 59500838,
"upload_time": "2023-06-16T12:53:33",
"upload_time_iso_8601": "2023-06-16T12:53:33.251308Z",
"url": "https://files.pythonhosted.org/packages/87/60/a076c56666dc4b4151c8b0bd2e92e197d1c9478eaf4927370df545e9863f/nlptest-1.5.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-16 12:53:33",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "JohnSnowLabs",
"github_project": "nlptest",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "nlptest"
}
JSON
