| Name | dqc-toolkit JSON |
| Version |
0.1.0
JSON |
| download |
| home_page | None |
| Summary | Data Quality Check for Machine Learning |
| upload_time | 2024-04-28 06:01:23 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.9 |
| license | MIT License Copyright (c) 2024 sumanthprabhu Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
nlp
data curation
machine learning
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
    
DQC Toolkit is a Python library and framework designed with the goal to facilitate improvement of Machine Learning models by identifying and mitigating label errors in training dataset. Currently, DQC toolkit offers `CrossValCurate` for curation of text classification datasets (binary / multi-class) using cross validation based selection.
## Installation
Installation of DQC-toolkit can be done as shown below
```python
pip install dqc-toolkit
```
## Quick Start
Assuming your text classification data is stored as a pandas dataframe `data`, with each sample represented by the `text` column and its corresponding noisy label represented by the `label` column, here is how you use `CrossValCurate` -
```python linenums="1"
from dqc import CrossValCurate
cvc = CrossValCurate()
data_curated = cvc.fit_transform(data[['text', 'label']])
```
The result stored in `data_curated` which is a pandas dataframe similar to `data` with the following columns -
```python
>>> data_curated.columns
['text', 'label', 'label_correctness_score', 'is_label_correct', 'predicted_label', 'prediction_probability']
```
* `'label_correctness_score'` represents a normalized score quantifying the correctness of `'label'`.
* `'is_label_correct'` is a boolean flag indicating whether the given `'label'` is correct (`True`) or incorrect (`False`).
* `'predicted_label'` and `'prediction_probability'` represent the curation model's prediction and the corresponding probability score.
For more details regarding different hyperparameters available in `CrossValCurate`, please refer to the [API documentation](https://sumanthprabhu.github.io/DQC-Toolkit/).
Raw data
{
"_id": null,
"home_page": null,
"name": "dqc-toolkit",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "nlp, data curation, machine learning",
"author": null,
"author_email": "Sumanth S Prabhu <sumanthprabhu.104@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/20/d8/c293f7e25aed1847a63eb7b4206d7e43d120baa0bebbebefa987e1aea261/dqc_toolkit-0.1.0.tar.gz",
"platform": null,
"description": "    \n\n\n\n\nDQC Toolkit is a Python library and framework designed with the goal to facilitate improvement of Machine Learning models by identifying and mitigating label errors in training dataset. Currently, DQC toolkit offers `CrossValCurate` for curation of text classification datasets (binary / multi-class) using cross validation based selection.\n\n## Installation\n\nInstallation of DQC-toolkit can be done as shown below\n```python\npip install dqc-toolkit\n```\n\n## Quick Start\n\n Assuming your text classification data is stored as a pandas dataframe `data`, with each sample represented by the `text` column and its corresponding noisy label represented by the `label` column, here is how you use `CrossValCurate` - \n\n\n```python linenums=\"1\"\n\nfrom dqc import CrossValCurate\n\ncvc = CrossValCurate()\ndata_curated = cvc.fit_transform(data[['text', 'label']])\n```\nThe result stored in `data_curated` which is a pandas dataframe similar to `data` with the following columns -\n```python\n>>> data_curated.columns\n['text', 'label', 'label_correctness_score', 'is_label_correct', 'predicted_label', 'prediction_probability']\n```\n\n* `'label_correctness_score'` represents a normalized score quantifying the correctness of `'label'`. \n* `'is_label_correct'` is a boolean flag indicating whether the given `'label'` is correct (`True`) or incorrect (`False`). \n* `'predicted_label'` and `'prediction_probability'` represent the curation model's prediction and the corresponding probability score. \n \nFor more details regarding different hyperparameters available in `CrossValCurate`, please refer to the [API documentation](https://sumanthprabhu.github.io/DQC-Toolkit/).\n",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2024 sumanthprabhu Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ",
"summary": "Data Quality Check for Machine Learning",
"version": "0.1.0",
"project_urls": {
"Homepage": "https://github.com/sumanthprabhu/DQC-Toolkit",
"Source": "https://github.com/sumanthprabhu/DQC-Toolkit",
"Tracker": "https://github.com/sumanthprabhu/DQC-Toolkit/issues"
},
"split_keywords": [
"nlp",
" data curation",
" machine learning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "fe6298734a5930411d130c75f68faa57b464dd3fb2bf8f780eaa3f4ddb65f75e",
"md5": "74e549ad6034bc9a0cf9236ca13ffecd",
"sha256": "460407b6737dd58430418470ecacb7b319a00a1db03a135ec50febad15bdfd5d"
},
"downloads": -1,
"filename": "dqc_toolkit-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "74e549ad6034bc9a0cf9236ca13ffecd",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 17980,
"upload_time": "2024-04-28T06:01:21",
"upload_time_iso_8601": "2024-04-28T06:01:21.496350Z",
"url": "https://files.pythonhosted.org/packages/fe/62/98734a5930411d130c75f68faa57b464dd3fb2bf8f780eaa3f4ddb65f75e/dqc_toolkit-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "20d8c293f7e25aed1847a63eb7b4206d7e43d120baa0bebbebefa987e1aea261",
"md5": "fdf441328da136c7e81ef04298b88c2a",
"sha256": "0c577bfb0b5ff9e7f0c6897cb13f3317153e0bad40d907a6138587c15ce04f51"
},
"downloads": -1,
"filename": "dqc_toolkit-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "fdf441328da136c7e81ef04298b88c2a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 16100,
"upload_time": "2024-04-28T06:01:23",
"upload_time_iso_8601": "2024-04-28T06:01:23.513705Z",
"url": "https://files.pythonhosted.org/packages/20/d8/c293f7e25aed1847a63eb7b4206d7e43d120baa0bebbebefa987e1aea261/dqc_toolkit-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-28 06:01:23",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "sumanthprabhu",
"github_project": "DQC-Toolkit",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "dqc-toolkit"
}
