| Name | dp-ai-data-harmonization JSON |
| Version |
1.0.1
JSON |
| download |
| home_page | |
| Summary | Create data quality rules and apply them to datasets. |
| upload_time | 2023-10-04 09:47:03 |
| maintainer | |
| docs_url | None |
| author | Himanshu |
| requires_python | >=3.8 |
| license | |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Data Harmonization Package
This package provides functionality for file merging and data harmonization using various algorithms and AI models.
It allows you to merge CSV files and harmonize data based on a sample-based approach using GPT-based models.
## Installation
You can install the package from PyPI using pip:
pip install data-harmonization-ai
## Usage
### DataHarmonizer Class
The `DataHarmonizer` class provides the capability to merge CSV files based on different options.
It supports the following merge options:
- ChatGPT
- GPT4
- Fuzzy Wuzzy
- Rapidfuzz
- Jaro Winkler
- JW Layered with ChatGPT
- JW Layered with GPT4
- FW Layered with GPT4
- Recursive Data Harmonization
Example usage:
from utility import DataHarmonizer
# Create an instance of DataHarmonizer
key='openai-key'
harmonizer = DataHarmonizer(key,'file1.csv', 'file2.csv', 'ChatGPT')
# Merge the files based on the specified option
result = harmonizer.merge_files()
print(result)
### DataHarmonizationWithSuggestion Class
# The DataHarmonizationWithSuggestion class allows you to harmonize data using a sample-based approach.
It takes a sample file and two data files as input.
Example usage:
from utility import DataHarmonizationWithSuggestion
# Create an instance of DataHarmonizationWithSuggestion
key = 'openai-key'
harmonizer = DataHarmonizationWithSuggestion(key, "sample_harmonized_data.csv", "file1.csv", "file2.csv")
# Harmonize the data based on the sample
result = harmonizer.harmonize_data()
print(result)
Raw data
{
"_id": null,
"home_page": "",
"name": "dp-ai-data-harmonization",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "",
"author": "Himanshu",
"author_email": "himanshu.tomar@decisionpoint.in",
"download_url": "https://files.pythonhosted.org/packages/11/74/b37c751ef7f3c192fe532506c494f7e99b2615e01921357e93b6a92557c6/dp-ai-data-harmonization-1.0.1.tar.gz",
"platform": null,
"description": "# Data Harmonization Package\n\nThis package provides functionality for file merging and data harmonization using various algorithms and AI models.\nIt allows you to merge CSV files and harmonize data based on a sample-based approach using GPT-based models.\n\n## Installation\n\nYou can install the package from PyPI using pip:\n\n\tpip install data-harmonization-ai\n\n\n## Usage\n\n### DataHarmonizer Class\n\nThe `DataHarmonizer` class provides the capability to merge CSV files based on different options.\nIt supports the following merge options:\n\n- ChatGPT\n- GPT4\n- Fuzzy Wuzzy\n- Rapidfuzz\n- Jaro Winkler\n- JW Layered with ChatGPT\n- JW Layered with GPT4\n- FW Layered with GPT4\n- Recursive Data Harmonization\n\nExample usage:\n\nfrom utility import DataHarmonizer\n\n# Create an instance of DataHarmonizer\nkey='openai-key'\nharmonizer = DataHarmonizer(key,'file1.csv', 'file2.csv', 'ChatGPT')\n\n# Merge the files based on the specified option\nresult = harmonizer.merge_files()\n\nprint(result)\n\n### DataHarmonizationWithSuggestion Class\n\n# The DataHarmonizationWithSuggestion class allows you to harmonize data using a sample-based approach.\n It takes a sample file and two data files as input.\n\nExample usage:\nfrom utility import DataHarmonizationWithSuggestion\n\n# Create an instance of DataHarmonizationWithSuggestion\n\nkey = 'openai-key'\nharmonizer = DataHarmonizationWithSuggestion(key, \"sample_harmonized_data.csv\", \"file1.csv\", \"file2.csv\")\n\n# Harmonize the data based on the sample\n\tresult = harmonizer.harmonize_data()\n\nprint(result)\n",
"bugtrack_url": null,
"license": "",
"summary": "Create data quality rules and apply them to datasets.",
"version": "1.0.1",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "63b5f92f9b47c8c127f9ab1f525d4aabbe827c4c75fae19b9ebc6ee28ac9d35b",
"md5": "fcc59c1997182c9bfbe6e28db7611935",
"sha256": "bae10325bd0e168a4c2ff8f4c7cee84be0543c539769340104f46511927e8ac5"
},
"downloads": -1,
"filename": "dp_ai_data_harmonization-1.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "fcc59c1997182c9bfbe6e28db7611935",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 30730,
"upload_time": "2023-10-04T09:47:01",
"upload_time_iso_8601": "2023-10-04T09:47:01.828376Z",
"url": "https://files.pythonhosted.org/packages/63/b5/f92f9b47c8c127f9ab1f525d4aabbe827c4c75fae19b9ebc6ee28ac9d35b/dp_ai_data_harmonization-1.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "1174b37c751ef7f3c192fe532506c494f7e99b2615e01921357e93b6a92557c6",
"md5": "4c9e807340e4171c9ca473ff90d97cc7",
"sha256": "cf55738abab900e923c227a583aeaed33567eb12829025f10cbd01fb99f157bf"
},
"downloads": -1,
"filename": "dp-ai-data-harmonization-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "4c9e807340e4171c9ca473ff90d97cc7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 14264,
"upload_time": "2023-10-04T09:47:03",
"upload_time_iso_8601": "2023-10-04T09:47:03.984770Z",
"url": "https://files.pythonhosted.org/packages/11/74/b37c751ef7f3c192fe532506c494f7e99b2615e01921357e93b6a92557c6/dp-ai-data-harmonization-1.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-10-04 09:47:03",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "dp-ai-data-harmonization"
}
