# Syndat
  
Syndat is a software package that provides basic functionalities for the evaluation and visualizsation of synthetic data. Quality scores can be computed on 3 base metrics (Discrimation, Correlation and Distribution) and data may be visualized to inspect correlation structures or statistical distribution plots.
# Installation
Install via pip:
```bash
pip install syndat
```
# Usage
## Quality metrics
Compute data quality metrics by comparing real and synthetic data in terms of their separation complexity,
distribution similarity or pairwise feature correlations:
```python
import pandas as pd
import syndat
real = pd.read_csv("real.csv")
synthetic = pd.read_csv("synthetic.csv")
# How similar are the statistical distributions of real and synthetic features
distribution_similarity_score = syndat.scores.distribution(real, synthetic)
# How hard is it for a classifier to discriminate real and synthetic data
discrimination_score = syndat.scores.discrimination(real, synthetic)
# How well are pairwise feature correlations preserved
correlation_score = syndat.scores.correlation(real, synthetic)
```
Scores are defined in a range of 0-100, with a higher score corresponding to better data fidelity.
## Visualization
Visualize real vs. synthetic data distributions, summary statistics and discriminating features:
```python
import pandas as pd
import syndat
real = pd.read_csv("real.csv")
synthetic = pd.read_csv("synthetic.csv")
# plot *all* feature distribution and store image files
syndat.visualization.plot_distributions(real, synthetic, store_destination="results/plots")
syndat.visualization.plot_correlations(real, synthetic, store_destination="results/plots")
# plot and display specific feature distribution plot
syndat.visualization.plot_numerical_feature("feature_xy", real, synthetic)
syndat.visualization.plot_numerical_feature("feature_xy", real, synthetic)
# plot a shap plot of differentiating feature for real and synthetic data
syndat.visualization.plot_shap_discrimination(real, synthetic)
```
## Postprocessing
Postprocess synthetic data to improve data fidelity:
```python
import pandas as pd
import syndat
real = pd.read_csv("real.csv")
synthetic = pd.read_csv("synthetic.csv")
# postprocess synthetic data
synthetic_post = syndat.postprocessing.assert_minmax(real, synthetic)
synthetic_post = syndat.postprocessing.normalize_float_precision(real, synthetic)
```
Raw data
{
"_id": null,
"home_page": "https://github.com/SCAI-BIO/syndat",
"name": "syndat",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "synthetic-data, data-quality, data-visualization",
"author": "Tim Adams",
"author_email": "tim.adams@scai.fraunhofer.de",
"download_url": "https://files.pythonhosted.org/packages/f1/bb/0eb243ff9774f27794b27faf7d4dbfd92578234e579b4839cd157cb25fc5/syndat-0.10.2.tar.gz",
"platform": null,
"description": "\n# Syndat\n  \n\nSyndat is a software package that provides basic functionalities for the evaluation and visualizsation of synthetic data. Quality scores can be computed on 3 base metrics (Discrimation, Correlation and Distribution) and data may be visualized to inspect correlation structures or statistical distribution plots.\n\n# Installation\n\nInstall via pip:\n\n```bash\npip install syndat\n```\n\n# Usage\n\n## Quality metrics\n\nCompute data quality metrics by comparing real and synthetic data in terms of their separation complexity, \ndistribution similarity or pairwise feature correlations:\n\n```python\nimport pandas as pd\nimport syndat\n\nreal = pd.read_csv(\"real.csv\")\nsynthetic = pd.read_csv(\"synthetic.csv\")\n\n# How similar are the statistical distributions of real and synthetic features \ndistribution_similarity_score = syndat.scores.distribution(real, synthetic)\n\n# How hard is it for a classifier to discriminate real and synthetic data\ndiscrimination_score = syndat.scores.discrimination(real, synthetic)\n\n# How well are pairwise feature correlations preserved\ncorrelation_score = syndat.scores.correlation(real, synthetic)\n```\n\nScores are defined in a range of 0-100, with a higher score corresponding to better data fidelity.\n\n## Visualization\n\nVisualize real vs. synthetic data distributions, summary statistics and discriminating features:\n\n```python\nimport pandas as pd\nimport syndat\n\nreal = pd.read_csv(\"real.csv\")\nsynthetic = pd.read_csv(\"synthetic.csv\")\n\n# plot *all* feature distribution and store image files\nsyndat.visualization.plot_distributions(real, synthetic, store_destination=\"results/plots\")\nsyndat.visualization.plot_correlations(real, synthetic, store_destination=\"results/plots\")\n\n# plot and display specific feature distribution plot\nsyndat.visualization.plot_numerical_feature(\"feature_xy\", real, synthetic)\nsyndat.visualization.plot_numerical_feature(\"feature_xy\", real, synthetic)\n\n# plot a shap plot of differentiating feature for real and synthetic data\nsyndat.visualization.plot_shap_discrimination(real, synthetic)\n```\n\n\n## Postprocessing\n\nPostprocess synthetic data to improve data fidelity:\n\n```python\nimport pandas as pd\nimport syndat\n\nreal = pd.read_csv(\"real.csv\")\nsynthetic = pd.read_csv(\"synthetic.csv\")\n\n# postprocess synthetic data\nsynthetic_post = syndat.postprocessing.assert_minmax(real, synthetic)\nsynthetic_post = syndat.postprocessing.normalize_float_precision(real, synthetic)\n```\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A library for evaluation & visualization of synthetic data.",
"version": "0.10.2",
"project_urls": {
"Documentation": "https://github.com/SCAI-BIO/syndat#readme",
"Homepage": "https://github.com/SCAI-BIO/syndat",
"Source": "https://github.com/SCAI-BIO/syndat",
"Tracker": "https://github.com/SCAI-BIO/syndat/issues"
},
"split_keywords": [
"synthetic-data",
" data-quality",
" data-visualization"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c8c3682a8b4c1abd78c3b16f7a1199bf10ba64395b15f255c878313408298e64",
"md5": "e7c9a1dfecceaeefcfa5f50a6722a93b",
"sha256": "7941e23678454df05f8f76440e0e8310ab2b24a64a0fc3b952c25619e684701f"
},
"downloads": -1,
"filename": "syndat-0.10.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e7c9a1dfecceaeefcfa5f50a6722a93b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 11944,
"upload_time": "2024-11-11T14:28:57",
"upload_time_iso_8601": "2024-11-11T14:28:57.286370Z",
"url": "https://files.pythonhosted.org/packages/c8/c3/682a8b4c1abd78c3b16f7a1199bf10ba64395b15f255c878313408298e64/syndat-0.10.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f1bb0eb243ff9774f27794b27faf7d4dbfd92578234e579b4839cd157cb25fc5",
"md5": "073f72c99907a5a28ecbff889d01578e",
"sha256": "77dd867196ba9fc118d542843fd7016144ad9000c648f1e7d2f03968f23e2dc2"
},
"downloads": -1,
"filename": "syndat-0.10.2.tar.gz",
"has_sig": false,
"md5_digest": "073f72c99907a5a28ecbff889d01578e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 14050,
"upload_time": "2024-11-11T14:28:58",
"upload_time_iso_8601": "2024-11-11T14:28:58.744134Z",
"url": "https://files.pythonhosted.org/packages/f1/bb/0eb243ff9774f27794b27faf7d4dbfd92578234e579b4839cd157cb25fc5/syndat-0.10.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-11 14:28:58",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "SCAI-BIO",
"github_project": "syndat",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "syndat"
}
JSON
