DataSae


NameDataSae JSON
Version 0.5.3 PyPI version JSON
download
home_pagehttps://github.com/jabardigitalservice/DataSae
SummaryData Quality Framework provides by Jabar Digital Service
upload_time2024-07-15 13:21:27
maintainerdataengineering@digitalservice.id, jds.dataengineer@gmail.com
docs_urlNone
authorpipinfitriadi@gmail.com, agistyaanugrah@gmail.com, riskaamalia.mail@gmail.com, rizky.prilian@gmail.com, misdan.wijaya07@gmail.com, fauzanmrabbani@gmail.com, hanieffatchudin@gmail.com, adibintangprada@gmail.com, firmanramadan97@gmail.com, raihanjauhari@gmail.com, pujinhidayah@gmail.com
requires_python>=3.8
licenseAGPL-3.0-only
keywords data quality framework data sae good data data bagus validation jds jabar digital service data engineer
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            <!--
Copyright (C) Free Software Foundation, Inc. All rights reserved.
Licensed under the AGPL-3.0-only License. See LICENSE in the project root
for license information.
-->

# DataSae

[![Docs](https://img.shields.io/badge/Docs-blue)](https://jabardigitalservice.github.io/DataSae/)
[![License](https://img.shields.io/github/license/jabardigitalservice/DataSae?logoColor=black&label=License&labelColor=black&color=brightgreen)](https://github.com/jabardigitalservice/DataSae/blob/main/LICENSE)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/DataSae?logo=python&label=Python&labelColor=black)](https://pypi.org/project/DataSae/)
[![PyPI - Version](https://img.shields.io/pypi/v/DataSae?logo=pypi&label=PyPI&labelColor=black)](https://pypi.org/project/DataSae/)
[![GitHub Action](https://img.shields.io/github/actions/workflow/status/jabardigitalservice/DataSae/python_docker.yaml?logo=GitHub&label=CI/CD&labelColor=black)](https://github.com/jabardigitalservice/DataSae/actions/workflows/python_docker.yaml)
[![Coverage](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/jabardigitalservice/DataSae/python-coverage-comment-action-data/endpoint.json&labelColor=black)](https://htmlpreview.github.io/?https://github.com/jabardigitalservice/DataSae/blob/python-coverage-comment-action-data/htmlcov/index.html)

Data Quality Framework provides by Jabar Digital Service

- [Configuration Files](#configuration-files)
- [Checker for Data Quality](#checker-for-data-quality)
  - [Command Line Interface (CLI)](#command-line-interface-cli)
  - [Python Code](#python-code)
- [Converter from Any Data Source to Pandas's DataFrame](#converter-from-any-data-source-to-pandass-dataframe)
  - [Local Computer](#local-computer)
  - [Google Spreadsheet](#google-spreadsheet)
  - [S3](#s3)
  - [SQL](#sql)
    - [MariaDB or MySQL](#mariadb-or-mysql)
    - [PostgreSQL](#postgresql)

## Configuration Files

[https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/config.json#L1-L183](https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/config.json#L1-L183)

[https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/config.yaml#L1-L120](https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/config.yaml#L1-L120)

## Checker for Data Quality

> [!NOTE]  
> You can use [DataSae Column's Function Based on Data Type](functions.md) for adding column checker function data quality in the config file.

```sh
pip install 'DataSae[converter,gsheet,s3,sql]'
```

### Command Line Interface (CLI)

```sh
datasae --help
 
 Usage: datasae [OPTIONS] FILE_PATH
 
 Checker command.
 Creates checker result based on the configuration provided in the checker section of the data source's configuration file.
╭─ Arguments ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ *    file_path      TEXT  The source path of the .json or .yaml file [default: None] [required]                                    │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --config-name                       TEXT  If the config name is not set, it will create all of the checker results [default: None] │
│ --yaml-display    --json-display          [default: yaml-display]                                                                  │
│ --save-to-file-path                 TEXT  [default: None]                                                                          │
│ --help                                    Show this message and exit.                                                              │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
```

Example commands:

```sh
datasae DataSae/tests/data/config.yaml # Check all data qualities on configuration
datasae DataSae/tests/data/config.yaml --config-name test_local # Check data quality by config name
```

> [!TIP]
> Actually, we have example for DataSae implementation in Apache Airflow, but for now it is for private use only. Internal developer can see it at this [git repository](https://gitlab.com/jdsteam/core-data-platform/data-products/example-datasae-airflow).

Example results:
[https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/checker.json#L1-L432](https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/checker.json#L1-L432)

### Python Code

```py
from datasae.converter import Config

# From JSON
config = Config('DataSae/tests/data/config.json')

# From YAML
config = Config('DataSae/tests/data/config.yaml')

# Check all data qualities on configuration
config.checker  # dict result

# Check data quality by config name
config('test_local').checker  # list of dict result
config('test_gsheet').checker  # list of dict result
config('test_s3').checker  # list of dict result
config('test_mariadb_or_mysql').checker  # list of dict result
config('test_postgresql').checker  # list of dict result
```

## Converter from Any Data Source to Pandas's DataFrame

> [!NOTE]  
> Currently support to convert from CSV, JSON, Parquet, Excel, Google Spreadsheet, and SQL.

```sh
pip install 'DataSae[converter]'
```

### Local Computer

```py
from datasae.converter import Config

# From JSON
config = Config('DataSae/tests/data/config.json')

# From YAML
config = Config('DataSae/tests/data/config.yaml')

# Local computer file to DataFrame
local = config('test_local')

df = local('path/file_name.csv', sep=',')
df = local('path/file_name.json')
df = local('path/file_name.parquet')
df = local('path/file_name.xlsx', sheet_name='Sheet1')

df = local('path/file_name.csv')  # Default: sep = ','
df = local('path/file_name.json')
df = local('path/file_name.parquet')
df = local('path/file_name.xlsx')  # Default: sheet_name = 'Sheet1'
```

### Google Spreadsheet

[https://github.com/jabardigitalservice/DataSae/blob/4308324d066c6627936773ab2d5b990adaa60100/tests/data/creds.json#L1-L12](https://github.com/jabardigitalservice/DataSae/blob/4308324d066c6627936773ab2d5b990adaa60100/tests/data/creds.json#L1-L12)

```sh
pip install 'DataSae[converter,gsheet]'
```

```py
from datasae.converter import Config

# From JSON
config = Config('DataSae/tests/data/config.json')

# From YAML
config = Config('DataSae/tests/data/config.yaml')

# Google Spreadsheet to DataFrame
gsheet = config('test_gsheet')
df = gsheet('Sheet1')
df = gsheet('Sheet1', 'gsheet_id')
```

### S3

```sh
pip install 'DataSae[converter,s3]'
```

```py
from datasae.converter import Config

# From JSON
config = Config('DataSae/tests/data/config.json')

# From YAML
config = Config('DataSae/tests/data/config.yaml')

# S3 object to DataFrame
s3 = config('test_s3')

df = s3('path/file_name.csv', sep=',')
df = s3('path/file_name.json')
df = s3('path/file_name.parquet')
df = s3('path/file_name.xlsx', sheet_name='Sheet1')

df = s3('path/file_name.csv', 'bucket_name')  # Default: sep = ','
df = s3('path/file_name.json', 'bucket_name')
df = s3('path/file_name.parquet', 'bucket_name')
df = s3('path/file_name.xlsx', 'bucket_name')  # Default: sheet_name = 'Sheet1'
```

### SQL

```sh
pip install 'DataSae[converter,sql]'
```

> [!IMPORTANT]
> For MacOS users, if [pip install failed](https://stackoverflow.com/questions/67876857/mysqlclient-wont-install-via-pip-on-macbook-pro-m1-with-latest-version-of-big-s) at `mysqlclient`, please run this and retry to install again after that.
>
> ```sh
> brew install mysql
> ```

#### MariaDB or MySQL

```py
from datasae.converter import Config

# From JSON
config = Config('DataSae/tests/data/config.json')

# From YAML
config = Config('DataSae/tests/data/config.yaml')

# MariaDB or MySQL to DataFrame
mariadb_or_mysql = config('test_mariadb_or_mysql')
df = mariadb_or_mysql('select 1 column_name from schema_name.table_name;')
df = mariadb_or_mysql('path/file_name.sql')
```

#### PostgreSQL

```py
from datasae.converter import Config

# From JSON
config = Config('DataSae/tests/data/config.json')

# From YAML
config = Config('DataSae/tests/data/config.yaml')

# PostgreSQL to DataFrame
postgresql = config('test_postgresql')
df = postgresql('select 1 column_name from schema_name.table_name;')
df = postgresql('path/file_name.sql')
```

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/jabardigitalservice/DataSae",
    "name": "DataSae",
    "maintainer": "dataengineering@digitalservice.id, jds.dataengineer@gmail.com",
    "docs_url": null,
    "requires_python": ">=3.8",
    "maintainer_email": null,
    "keywords": "data quality framework, data sae, good data, data bagus, validation, jds, jabar digital service, data engineer",
    "author": "pipinfitriadi@gmail.com, agistyaanugrah@gmail.com, riskaamalia.mail@gmail.com, rizky.prilian@gmail.com, misdan.wijaya07@gmail.com, fauzanmrabbani@gmail.com, hanieffatchudin@gmail.com, adibintangprada@gmail.com, firmanramadan97@gmail.com, raihanjauhari@gmail.com, pujinhidayah@gmail.com",
    "author_email": null,
    "download_url": "https://files.pythonhosted.org/packages/8c/d4/be10d028f9f7e1c411a92656a03f345383728cc06401cf90160e3936ffbc/datasae-0.5.3.tar.gz",
    "platform": null,
    "description": "<!--\nCopyright (C) Free Software Foundation, Inc. All rights reserved.\nLicensed under the AGPL-3.0-only License. See LICENSE in the project root\nfor license information.\n-->\n\n# DataSae\n\n[![Docs](https://img.shields.io/badge/Docs-blue)](https://jabardigitalservice.github.io/DataSae/)\n[![License](https://img.shields.io/github/license/jabardigitalservice/DataSae?logoColor=black&label=License&labelColor=black&color=brightgreen)](https://github.com/jabardigitalservice/DataSae/blob/main/LICENSE)\n[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/DataSae?logo=python&label=Python&labelColor=black)](https://pypi.org/project/DataSae/)\n[![PyPI - Version](https://img.shields.io/pypi/v/DataSae?logo=pypi&label=PyPI&labelColor=black)](https://pypi.org/project/DataSae/)\n[![GitHub Action](https://img.shields.io/github/actions/workflow/status/jabardigitalservice/DataSae/python_docker.yaml?logo=GitHub&label=CI/CD&labelColor=black)](https://github.com/jabardigitalservice/DataSae/actions/workflows/python_docker.yaml)\n[![Coverage](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/jabardigitalservice/DataSae/python-coverage-comment-action-data/endpoint.json&labelColor=black)](https://htmlpreview.github.io/?https://github.com/jabardigitalservice/DataSae/blob/python-coverage-comment-action-data/htmlcov/index.html)\n\nData Quality Framework provides by Jabar Digital Service\n\n- [Configuration Files](#configuration-files)\n- [Checker for Data Quality](#checker-for-data-quality)\n  - [Command Line Interface (CLI)](#command-line-interface-cli)\n  - [Python Code](#python-code)\n- [Converter from Any Data Source to Pandas's DataFrame](#converter-from-any-data-source-to-pandass-dataframe)\n  - [Local Computer](#local-computer)\n  - [Google Spreadsheet](#google-spreadsheet)\n  - [S3](#s3)\n  - [SQL](#sql)\n    - [MariaDB or MySQL](#mariadb-or-mysql)\n    - [PostgreSQL](#postgresql)\n\n## Configuration Files\n\n[https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/config.json#L1-L183](https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/config.json#L1-L183)\n\n[https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/config.yaml#L1-L120](https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/config.yaml#L1-L120)\n\n## Checker for Data Quality\n\n> [!NOTE]  \n> You can use [DataSae Column's Function Based on Data Type](functions.md) for adding column checker function data quality in the config file.\n\n```sh\npip install 'DataSae[converter,gsheet,s3,sql]'\n```\n\n### Command Line Interface (CLI)\n\n```sh\ndatasae --help\n \n Usage: datasae [OPTIONS] FILE_PATH\n \n Checker command.\n Creates checker result based on the configuration provided in the checker section of the data source's configuration file.\n\u256d\u2500 Arguments \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256e\n\u2502 *    file_path      TEXT  The source path of the .json or .yaml file [default: None] [required]                                    \u2502\n\u2570\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256f\n\u256d\u2500 Options \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256e\n\u2502 --config-name                       TEXT  If the config name is not set, it will create all of the checker results [default: None] \u2502\n\u2502 --yaml-display    --json-display          [default: yaml-display]                                                                  \u2502\n\u2502 --save-to-file-path                 TEXT  [default: None]                                                                          \u2502\n\u2502 --help                                    Show this message and exit.                                                              \u2502\n\u2570\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256f\n```\n\nExample commands:\n\n```sh\ndatasae DataSae/tests/data/config.yaml # Check all data qualities on configuration\ndatasae DataSae/tests/data/config.yaml --config-name test_local # Check data quality by config name\n```\n\n> [!TIP]\n> Actually, we have example for DataSae implementation in Apache Airflow, but for now it is for private use only. Internal developer can see it at this [git repository](https://gitlab.com/jdsteam/core-data-platform/data-products/example-datasae-airflow).\n\nExample results:\n[https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/checker.json#L1-L432](https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/checker.json#L1-L432)\n\n### Python Code\n\n```py\nfrom datasae.converter import Config\n\n# From JSON\nconfig = Config('DataSae/tests/data/config.json')\n\n# From YAML\nconfig = Config('DataSae/tests/data/config.yaml')\n\n# Check all data qualities on configuration\nconfig.checker  # dict result\n\n# Check data quality by config name\nconfig('test_local').checker  # list of dict result\nconfig('test_gsheet').checker  # list of dict result\nconfig('test_s3').checker  # list of dict result\nconfig('test_mariadb_or_mysql').checker  # list of dict result\nconfig('test_postgresql').checker  # list of dict result\n```\n\n## Converter from Any Data Source to Pandas's DataFrame\n\n> [!NOTE]  \n> Currently support to convert from CSV, JSON, Parquet, Excel, Google Spreadsheet, and SQL.\n\n```sh\npip install 'DataSae[converter]'\n```\n\n### Local Computer\n\n```py\nfrom datasae.converter import Config\n\n# From JSON\nconfig = Config('DataSae/tests/data/config.json')\n\n# From YAML\nconfig = Config('DataSae/tests/data/config.yaml')\n\n# Local computer file to DataFrame\nlocal = config('test_local')\n\ndf = local('path/file_name.csv', sep=',')\ndf = local('path/file_name.json')\ndf = local('path/file_name.parquet')\ndf = local('path/file_name.xlsx', sheet_name='Sheet1')\n\ndf = local('path/file_name.csv')  # Default: sep = ','\ndf = local('path/file_name.json')\ndf = local('path/file_name.parquet')\ndf = local('path/file_name.xlsx')  # Default: sheet_name = 'Sheet1'\n```\n\n### Google Spreadsheet\n\n[https://github.com/jabardigitalservice/DataSae/blob/4308324d066c6627936773ab2d5b990adaa60100/tests/data/creds.json#L1-L12](https://github.com/jabardigitalservice/DataSae/blob/4308324d066c6627936773ab2d5b990adaa60100/tests/data/creds.json#L1-L12)\n\n```sh\npip install 'DataSae[converter,gsheet]'\n```\n\n```py\nfrom datasae.converter import Config\n\n# From JSON\nconfig = Config('DataSae/tests/data/config.json')\n\n# From YAML\nconfig = Config('DataSae/tests/data/config.yaml')\n\n# Google Spreadsheet to DataFrame\ngsheet = config('test_gsheet')\ndf = gsheet('Sheet1')\ndf = gsheet('Sheet1', 'gsheet_id')\n```\n\n### S3\n\n```sh\npip install 'DataSae[converter,s3]'\n```\n\n```py\nfrom datasae.converter import Config\n\n# From JSON\nconfig = Config('DataSae/tests/data/config.json')\n\n# From YAML\nconfig = Config('DataSae/tests/data/config.yaml')\n\n# S3 object to DataFrame\ns3 = config('test_s3')\n\ndf = s3('path/file_name.csv', sep=',')\ndf = s3('path/file_name.json')\ndf = s3('path/file_name.parquet')\ndf = s3('path/file_name.xlsx', sheet_name='Sheet1')\n\ndf = s3('path/file_name.csv', 'bucket_name')  # Default: sep = ','\ndf = s3('path/file_name.json', 'bucket_name')\ndf = s3('path/file_name.parquet', 'bucket_name')\ndf = s3('path/file_name.xlsx', 'bucket_name')  # Default: sheet_name = 'Sheet1'\n```\n\n### SQL\n\n```sh\npip install 'DataSae[converter,sql]'\n```\n\n> [!IMPORTANT]\n> For MacOS users, if [pip install failed](https://stackoverflow.com/questions/67876857/mysqlclient-wont-install-via-pip-on-macbook-pro-m1-with-latest-version-of-big-s) at `mysqlclient`, please run this and retry to install again after that.\n>\n> ```sh\n> brew install mysql\n> ```\n\n#### MariaDB or MySQL\n\n```py\nfrom datasae.converter import Config\n\n# From JSON\nconfig = Config('DataSae/tests/data/config.json')\n\n# From YAML\nconfig = Config('DataSae/tests/data/config.yaml')\n\n# MariaDB or MySQL to DataFrame\nmariadb_or_mysql = config('test_mariadb_or_mysql')\ndf = mariadb_or_mysql('select 1 column_name from schema_name.table_name;')\ndf = mariadb_or_mysql('path/file_name.sql')\n```\n\n#### PostgreSQL\n\n```py\nfrom datasae.converter import Config\n\n# From JSON\nconfig = Config('DataSae/tests/data/config.json')\n\n# From YAML\nconfig = Config('DataSae/tests/data/config.yaml')\n\n# PostgreSQL to DataFrame\npostgresql = config('test_postgresql')\ndf = postgresql('select 1 column_name from schema_name.table_name;')\ndf = postgresql('path/file_name.sql')\n```\n",
    "bugtrack_url": null,
    "license": "AGPL-3.0-only",
    "summary": "Data Quality Framework provides by Jabar Digital Service",
    "version": "0.5.3",
    "project_urls": {
        "Docs": "https://jabardigitalservice.github.io/DataSae/",
        "Homepage": "https://github.com/jabardigitalservice/DataSae",
        "Repository": "https://github.com/jabardigitalservice/DataSae"
    },
    "split_keywords": [
        "data quality framework",
        " data sae",
        " good data",
        " data bagus",
        " validation",
        " jds",
        " jabar digital service",
        " data engineer"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "53e7bacecb5206c5911a42558b25503dd151b021f59948ddcfd5ea9251e92deb",
                "md5": "ebadcd326475baaa8d13110306a3e7c4",
                "sha256": "11b70a1dac77a8d50d8811804744ece2ac977c6a4bb1d1764327627a083ae721"
            },
            "downloads": -1,
            "filename": "DataSae-0.5.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "ebadcd326475baaa8d13110306a3e7c4",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.8",
            "size": 37034,
            "upload_time": "2024-07-15T13:21:25",
            "upload_time_iso_8601": "2024-07-15T13:21:25.559595Z",
            "url": "https://files.pythonhosted.org/packages/53/e7/bacecb5206c5911a42558b25503dd151b021f59948ddcfd5ea9251e92deb/DataSae-0.5.3-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "8cd4be10d028f9f7e1c411a92656a03f345383728cc06401cf90160e3936ffbc",
                "md5": "b0cff10d287dab8a87d91a0b25f53be5",
                "sha256": "903c40bed20e1888e6df658cec3d9c1d554abf97ebde1713ec049882c9229be6"
            },
            "downloads": -1,
            "filename": "datasae-0.5.3.tar.gz",
            "has_sig": false,
            "md5_digest": "b0cff10d287dab8a87d91a0b25f53be5",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.8",
            "size": 36566,
            "upload_time": "2024-07-15T13:21:27",
            "upload_time_iso_8601": "2024-07-15T13:21:27.461599Z",
            "url": "https://files.pythonhosted.org/packages/8c/d4/be10d028f9f7e1c411a92656a03f345383728cc06401cf90160e3936ffbc/datasae-0.5.3.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-07-15 13:21:27",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "jabardigitalservice",
    "github_project": "DataSae",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "datasae"
}
        
Elapsed time: 3.78199s