Name | DataSae JSON |
Version |
0.5.3
JSON |
| download |
home_page | https://github.com/jabardigitalservice/DataSae |
Summary | Data Quality Framework provides by Jabar Digital Service |
upload_time | 2024-07-15 13:21:27 |
maintainer | dataengineering@digitalservice.id, jds.dataengineer@gmail.com |
docs_url | None |
author | pipinfitriadi@gmail.com, agistyaanugrah@gmail.com, riskaamalia.mail@gmail.com, rizky.prilian@gmail.com, misdan.wijaya07@gmail.com, fauzanmrabbani@gmail.com, hanieffatchudin@gmail.com, adibintangprada@gmail.com, firmanramadan97@gmail.com, raihanjauhari@gmail.com, pujinhidayah@gmail.com |
requires_python | >=3.8 |
license | AGPL-3.0-only |
keywords |
data quality framework
data sae
good data
data bagus
validation
jds
jabar digital service
data engineer
|
VCS |
|
bugtrack_url |
|
requirements |
No requirements were recorded.
|
Travis-CI |
No Travis.
|
coveralls test coverage |
No coveralls.
|
<!--
Copyright (C) Free Software Foundation, Inc. All rights reserved.
Licensed under the AGPL-3.0-only License. See LICENSE in the project root
for license information.
-->
# DataSae
[![Docs](https://img.shields.io/badge/Docs-blue)](https://jabardigitalservice.github.io/DataSae/)
[![License](https://img.shields.io/github/license/jabardigitalservice/DataSae?logoColor=black&label=License&labelColor=black&color=brightgreen)](https://github.com/jabardigitalservice/DataSae/blob/main/LICENSE)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/DataSae?logo=python&label=Python&labelColor=black)](https://pypi.org/project/DataSae/)
[![PyPI - Version](https://img.shields.io/pypi/v/DataSae?logo=pypi&label=PyPI&labelColor=black)](https://pypi.org/project/DataSae/)
[![GitHub Action](https://img.shields.io/github/actions/workflow/status/jabardigitalservice/DataSae/python_docker.yaml?logo=GitHub&label=CI/CD&labelColor=black)](https://github.com/jabardigitalservice/DataSae/actions/workflows/python_docker.yaml)
[![Coverage](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/jabardigitalservice/DataSae/python-coverage-comment-action-data/endpoint.json&labelColor=black)](https://htmlpreview.github.io/?https://github.com/jabardigitalservice/DataSae/blob/python-coverage-comment-action-data/htmlcov/index.html)
Data Quality Framework provides by Jabar Digital Service
- [Configuration Files](#configuration-files)
- [Checker for Data Quality](#checker-for-data-quality)
- [Command Line Interface (CLI)](#command-line-interface-cli)
- [Python Code](#python-code)
- [Converter from Any Data Source to Pandas's DataFrame](#converter-from-any-data-source-to-pandass-dataframe)
- [Local Computer](#local-computer)
- [Google Spreadsheet](#google-spreadsheet)
- [S3](#s3)
- [SQL](#sql)
- [MariaDB or MySQL](#mariadb-or-mysql)
- [PostgreSQL](#postgresql)
## Configuration Files
[https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/config.json#L1-L183](https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/config.json#L1-L183)
[https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/config.yaml#L1-L120](https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/config.yaml#L1-L120)
## Checker for Data Quality
> [!NOTE]
> You can use [DataSae Column's Function Based on Data Type](functions.md) for adding column checker function data quality in the config file.
```sh
pip install 'DataSae[converter,gsheet,s3,sql]'
```
### Command Line Interface (CLI)
```sh
datasae --help
Usage: datasae [OPTIONS] FILE_PATH
Checker command.
Creates checker result based on the configuration provided in the checker section of the data source's configuration file.
╭─ Arguments ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ * file_path TEXT The source path of the .json or .yaml file [default: None] [required] │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --config-name TEXT If the config name is not set, it will create all of the checker results [default: None] │
│ --yaml-display --json-display [default: yaml-display] │
│ --save-to-file-path TEXT [default: None] │
│ --help Show this message and exit. │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
```
Example commands:
```sh
datasae DataSae/tests/data/config.yaml # Check all data qualities on configuration
datasae DataSae/tests/data/config.yaml --config-name test_local # Check data quality by config name
```
> [!TIP]
> Actually, we have example for DataSae implementation in Apache Airflow, but for now it is for private use only. Internal developer can see it at this [git repository](https://gitlab.com/jdsteam/core-data-platform/data-products/example-datasae-airflow).
Example results:
[https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/checker.json#L1-L432](https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/checker.json#L1-L432)
### Python Code
```py
from datasae.converter import Config
# From JSON
config = Config('DataSae/tests/data/config.json')
# From YAML
config = Config('DataSae/tests/data/config.yaml')
# Check all data qualities on configuration
config.checker # dict result
# Check data quality by config name
config('test_local').checker # list of dict result
config('test_gsheet').checker # list of dict result
config('test_s3').checker # list of dict result
config('test_mariadb_or_mysql').checker # list of dict result
config('test_postgresql').checker # list of dict result
```
## Converter from Any Data Source to Pandas's DataFrame
> [!NOTE]
> Currently support to convert from CSV, JSON, Parquet, Excel, Google Spreadsheet, and SQL.
```sh
pip install 'DataSae[converter]'
```
### Local Computer
```py
from datasae.converter import Config
# From JSON
config = Config('DataSae/tests/data/config.json')
# From YAML
config = Config('DataSae/tests/data/config.yaml')
# Local computer file to DataFrame
local = config('test_local')
df = local('path/file_name.csv', sep=',')
df = local('path/file_name.json')
df = local('path/file_name.parquet')
df = local('path/file_name.xlsx', sheet_name='Sheet1')
df = local('path/file_name.csv') # Default: sep = ','
df = local('path/file_name.json')
df = local('path/file_name.parquet')
df = local('path/file_name.xlsx') # Default: sheet_name = 'Sheet1'
```
### Google Spreadsheet
[https://github.com/jabardigitalservice/DataSae/blob/4308324d066c6627936773ab2d5b990adaa60100/tests/data/creds.json#L1-L12](https://github.com/jabardigitalservice/DataSae/blob/4308324d066c6627936773ab2d5b990adaa60100/tests/data/creds.json#L1-L12)
```sh
pip install 'DataSae[converter,gsheet]'
```
```py
from datasae.converter import Config
# From JSON
config = Config('DataSae/tests/data/config.json')
# From YAML
config = Config('DataSae/tests/data/config.yaml')
# Google Spreadsheet to DataFrame
gsheet = config('test_gsheet')
df = gsheet('Sheet1')
df = gsheet('Sheet1', 'gsheet_id')
```
### S3
```sh
pip install 'DataSae[converter,s3]'
```
```py
from datasae.converter import Config
# From JSON
config = Config('DataSae/tests/data/config.json')
# From YAML
config = Config('DataSae/tests/data/config.yaml')
# S3 object to DataFrame
s3 = config('test_s3')
df = s3('path/file_name.csv', sep=',')
df = s3('path/file_name.json')
df = s3('path/file_name.parquet')
df = s3('path/file_name.xlsx', sheet_name='Sheet1')
df = s3('path/file_name.csv', 'bucket_name') # Default: sep = ','
df = s3('path/file_name.json', 'bucket_name')
df = s3('path/file_name.parquet', 'bucket_name')
df = s3('path/file_name.xlsx', 'bucket_name') # Default: sheet_name = 'Sheet1'
```
### SQL
```sh
pip install 'DataSae[converter,sql]'
```
> [!IMPORTANT]
> For MacOS users, if [pip install failed](https://stackoverflow.com/questions/67876857/mysqlclient-wont-install-via-pip-on-macbook-pro-m1-with-latest-version-of-big-s) at `mysqlclient`, please run this and retry to install again after that.
>
> ```sh
> brew install mysql
> ```
#### MariaDB or MySQL
```py
from datasae.converter import Config
# From JSON
config = Config('DataSae/tests/data/config.json')
# From YAML
config = Config('DataSae/tests/data/config.yaml')
# MariaDB or MySQL to DataFrame
mariadb_or_mysql = config('test_mariadb_or_mysql')
df = mariadb_or_mysql('select 1 column_name from schema_name.table_name;')
df = mariadb_or_mysql('path/file_name.sql')
```
#### PostgreSQL
```py
from datasae.converter import Config
# From JSON
config = Config('DataSae/tests/data/config.json')
# From YAML
config = Config('DataSae/tests/data/config.yaml')
# PostgreSQL to DataFrame
postgresql = config('test_postgresql')
df = postgresql('select 1 column_name from schema_name.table_name;')
df = postgresql('path/file_name.sql')
```
Raw data
{
"_id": null,
"home_page": "https://github.com/jabardigitalservice/DataSae",
"name": "DataSae",
"maintainer": "dataengineering@digitalservice.id, jds.dataengineer@gmail.com",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "data quality framework, data sae, good data, data bagus, validation, jds, jabar digital service, data engineer",
"author": "pipinfitriadi@gmail.com, agistyaanugrah@gmail.com, riskaamalia.mail@gmail.com, rizky.prilian@gmail.com, misdan.wijaya07@gmail.com, fauzanmrabbani@gmail.com, hanieffatchudin@gmail.com, adibintangprada@gmail.com, firmanramadan97@gmail.com, raihanjauhari@gmail.com, pujinhidayah@gmail.com",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/8c/d4/be10d028f9f7e1c411a92656a03f345383728cc06401cf90160e3936ffbc/datasae-0.5.3.tar.gz",
"platform": null,
"description": "<!--\nCopyright (C) Free Software Foundation, Inc. All rights reserved.\nLicensed under the AGPL-3.0-only License. See LICENSE in the project root\nfor license information.\n-->\n\n# DataSae\n\n[![Docs](https://img.shields.io/badge/Docs-blue)](https://jabardigitalservice.github.io/DataSae/)\n[![License](https://img.shields.io/github/license/jabardigitalservice/DataSae?logoColor=black&label=License&labelColor=black&color=brightgreen)](https://github.com/jabardigitalservice/DataSae/blob/main/LICENSE)\n[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/DataSae?logo=python&label=Python&labelColor=black)](https://pypi.org/project/DataSae/)\n[![PyPI - Version](https://img.shields.io/pypi/v/DataSae?logo=pypi&label=PyPI&labelColor=black)](https://pypi.org/project/DataSae/)\n[![GitHub Action](https://img.shields.io/github/actions/workflow/status/jabardigitalservice/DataSae/python_docker.yaml?logo=GitHub&label=CI/CD&labelColor=black)](https://github.com/jabardigitalservice/DataSae/actions/workflows/python_docker.yaml)\n[![Coverage](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/jabardigitalservice/DataSae/python-coverage-comment-action-data/endpoint.json&labelColor=black)](https://htmlpreview.github.io/?https://github.com/jabardigitalservice/DataSae/blob/python-coverage-comment-action-data/htmlcov/index.html)\n\nData Quality Framework provides by Jabar Digital Service\n\n- [Configuration Files](#configuration-files)\n- [Checker for Data Quality](#checker-for-data-quality)\n - [Command Line Interface (CLI)](#command-line-interface-cli)\n - [Python Code](#python-code)\n- [Converter from Any Data Source to Pandas's DataFrame](#converter-from-any-data-source-to-pandass-dataframe)\n - [Local Computer](#local-computer)\n - [Google Spreadsheet](#google-spreadsheet)\n - [S3](#s3)\n - [SQL](#sql)\n - [MariaDB or MySQL](#mariadb-or-mysql)\n - [PostgreSQL](#postgresql)\n\n## Configuration Files\n\n[https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/config.json#L1-L183](https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/config.json#L1-L183)\n\n[https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/config.yaml#L1-L120](https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/config.yaml#L1-L120)\n\n## Checker for Data Quality\n\n> [!NOTE] \n> You can use [DataSae Column's Function Based on Data Type](functions.md) for adding column checker function data quality in the config file.\n\n```sh\npip install 'DataSae[converter,gsheet,s3,sql]'\n```\n\n### Command Line Interface (CLI)\n\n```sh\ndatasae --help\n \n Usage: datasae [OPTIONS] FILE_PATH\n \n Checker command.\n Creates checker result based on the configuration provided in the checker section of the data source's configuration file.\n\u256d\u2500 Arguments \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256e\n\u2502 * file_path TEXT The source path of the .json or .yaml file [default: None] [required] \u2502\n\u2570\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256f\n\u256d\u2500 Options \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256e\n\u2502 --config-name TEXT If the config name is not set, it will create all of the checker results [default: None] \u2502\n\u2502 --yaml-display --json-display [default: yaml-display] \u2502\n\u2502 --save-to-file-path TEXT [default: None] \u2502\n\u2502 --help Show this message and exit. \u2502\n\u2570\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256f\n```\n\nExample commands:\n\n```sh\ndatasae DataSae/tests/data/config.yaml # Check all data qualities on configuration\ndatasae DataSae/tests/data/config.yaml --config-name test_local # Check data quality by config name\n```\n\n> [!TIP]\n> Actually, we have example for DataSae implementation in Apache Airflow, but for now it is for private use only. Internal developer can see it at this [git repository](https://gitlab.com/jdsteam/core-data-platform/data-products/example-datasae-airflow).\n\nExample results:\n[https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/checker.json#L1-L432](https://github.com/jabardigitalservice/DataSae/blob/46ef80072b98ca949084b4e1ae50bcf23d07d646/tests/data/checker.json#L1-L432)\n\n### Python Code\n\n```py\nfrom datasae.converter import Config\n\n# From JSON\nconfig = Config('DataSae/tests/data/config.json')\n\n# From YAML\nconfig = Config('DataSae/tests/data/config.yaml')\n\n# Check all data qualities on configuration\nconfig.checker # dict result\n\n# Check data quality by config name\nconfig('test_local').checker # list of dict result\nconfig('test_gsheet').checker # list of dict result\nconfig('test_s3').checker # list of dict result\nconfig('test_mariadb_or_mysql').checker # list of dict result\nconfig('test_postgresql').checker # list of dict result\n```\n\n## Converter from Any Data Source to Pandas's DataFrame\n\n> [!NOTE] \n> Currently support to convert from CSV, JSON, Parquet, Excel, Google Spreadsheet, and SQL.\n\n```sh\npip install 'DataSae[converter]'\n```\n\n### Local Computer\n\n```py\nfrom datasae.converter import Config\n\n# From JSON\nconfig = Config('DataSae/tests/data/config.json')\n\n# From YAML\nconfig = Config('DataSae/tests/data/config.yaml')\n\n# Local computer file to DataFrame\nlocal = config('test_local')\n\ndf = local('path/file_name.csv', sep=',')\ndf = local('path/file_name.json')\ndf = local('path/file_name.parquet')\ndf = local('path/file_name.xlsx', sheet_name='Sheet1')\n\ndf = local('path/file_name.csv') # Default: sep = ','\ndf = local('path/file_name.json')\ndf = local('path/file_name.parquet')\ndf = local('path/file_name.xlsx') # Default: sheet_name = 'Sheet1'\n```\n\n### Google Spreadsheet\n\n[https://github.com/jabardigitalservice/DataSae/blob/4308324d066c6627936773ab2d5b990adaa60100/tests/data/creds.json#L1-L12](https://github.com/jabardigitalservice/DataSae/blob/4308324d066c6627936773ab2d5b990adaa60100/tests/data/creds.json#L1-L12)\n\n```sh\npip install 'DataSae[converter,gsheet]'\n```\n\n```py\nfrom datasae.converter import Config\n\n# From JSON\nconfig = Config('DataSae/tests/data/config.json')\n\n# From YAML\nconfig = Config('DataSae/tests/data/config.yaml')\n\n# Google Spreadsheet to DataFrame\ngsheet = config('test_gsheet')\ndf = gsheet('Sheet1')\ndf = gsheet('Sheet1', 'gsheet_id')\n```\n\n### S3\n\n```sh\npip install 'DataSae[converter,s3]'\n```\n\n```py\nfrom datasae.converter import Config\n\n# From JSON\nconfig = Config('DataSae/tests/data/config.json')\n\n# From YAML\nconfig = Config('DataSae/tests/data/config.yaml')\n\n# S3 object to DataFrame\ns3 = config('test_s3')\n\ndf = s3('path/file_name.csv', sep=',')\ndf = s3('path/file_name.json')\ndf = s3('path/file_name.parquet')\ndf = s3('path/file_name.xlsx', sheet_name='Sheet1')\n\ndf = s3('path/file_name.csv', 'bucket_name') # Default: sep = ','\ndf = s3('path/file_name.json', 'bucket_name')\ndf = s3('path/file_name.parquet', 'bucket_name')\ndf = s3('path/file_name.xlsx', 'bucket_name') # Default: sheet_name = 'Sheet1'\n```\n\n### SQL\n\n```sh\npip install 'DataSae[converter,sql]'\n```\n\n> [!IMPORTANT]\n> For MacOS users, if [pip install failed](https://stackoverflow.com/questions/67876857/mysqlclient-wont-install-via-pip-on-macbook-pro-m1-with-latest-version-of-big-s) at `mysqlclient`, please run this and retry to install again after that.\n>\n> ```sh\n> brew install mysql\n> ```\n\n#### MariaDB or MySQL\n\n```py\nfrom datasae.converter import Config\n\n# From JSON\nconfig = Config('DataSae/tests/data/config.json')\n\n# From YAML\nconfig = Config('DataSae/tests/data/config.yaml')\n\n# MariaDB or MySQL to DataFrame\nmariadb_or_mysql = config('test_mariadb_or_mysql')\ndf = mariadb_or_mysql('select 1 column_name from schema_name.table_name;')\ndf = mariadb_or_mysql('path/file_name.sql')\n```\n\n#### PostgreSQL\n\n```py\nfrom datasae.converter import Config\n\n# From JSON\nconfig = Config('DataSae/tests/data/config.json')\n\n# From YAML\nconfig = Config('DataSae/tests/data/config.yaml')\n\n# PostgreSQL to DataFrame\npostgresql = config('test_postgresql')\ndf = postgresql('select 1 column_name from schema_name.table_name;')\ndf = postgresql('path/file_name.sql')\n```\n",
"bugtrack_url": null,
"license": "AGPL-3.0-only",
"summary": "Data Quality Framework provides by Jabar Digital Service",
"version": "0.5.3",
"project_urls": {
"Docs": "https://jabardigitalservice.github.io/DataSae/",
"Homepage": "https://github.com/jabardigitalservice/DataSae",
"Repository": "https://github.com/jabardigitalservice/DataSae"
},
"split_keywords": [
"data quality framework",
" data sae",
" good data",
" data bagus",
" validation",
" jds",
" jabar digital service",
" data engineer"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "53e7bacecb5206c5911a42558b25503dd151b021f59948ddcfd5ea9251e92deb",
"md5": "ebadcd326475baaa8d13110306a3e7c4",
"sha256": "11b70a1dac77a8d50d8811804744ece2ac977c6a4bb1d1764327627a083ae721"
},
"downloads": -1,
"filename": "DataSae-0.5.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ebadcd326475baaa8d13110306a3e7c4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 37034,
"upload_time": "2024-07-15T13:21:25",
"upload_time_iso_8601": "2024-07-15T13:21:25.559595Z",
"url": "https://files.pythonhosted.org/packages/53/e7/bacecb5206c5911a42558b25503dd151b021f59948ddcfd5ea9251e92deb/DataSae-0.5.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8cd4be10d028f9f7e1c411a92656a03f345383728cc06401cf90160e3936ffbc",
"md5": "b0cff10d287dab8a87d91a0b25f53be5",
"sha256": "903c40bed20e1888e6df658cec3d9c1d554abf97ebde1713ec049882c9229be6"
},
"downloads": -1,
"filename": "datasae-0.5.3.tar.gz",
"has_sig": false,
"md5_digest": "b0cff10d287dab8a87d91a0b25f53be5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 36566,
"upload_time": "2024-07-15T13:21:27",
"upload_time_iso_8601": "2024-07-15T13:21:27.461599Z",
"url": "https://files.pythonhosted.org/packages/8c/d4/be10d028f9f7e1c411a92656a03f345383728cc06401cf90160e3936ffbc/datasae-0.5.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-15 13:21:27",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "jabardigitalservice",
"github_project": "DataSae",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "datasae"
}