# spark_gaps_date_rorc_tools
[](https://opensource.org/licenses/Apache-2.0)
[](pyup)
[](pyup)
[](codecov)
spark_gaps_date_rorc_tools is a Python library that implements get gaps dates
## Installation
The code is packaged for PyPI, so that the installation consists in running:
```sh
pip install spark-gaps-date-rorc-tools
```
## Usage
wrapper take gaps dates
```sh
config.yaml
===========
conf:
t_psan_test:
table_path: "/data/master/psan/data/t_psan_test/"
supplies : [
"/data/master/psan/data/t_ksag_test/",
"/data/master/psan/data/t_psan_test/"
]
t_kctk_cust_rating_atrb:
table_path: ""
supplies : []
example1: file.py
=================
from spark_gaps_date_rorc_tools import show_gaps_date
df_pivot = show_gaps_date(spark=spark,
config_path_name="config.yaml",
table_rorc=["t_psan_xxx"]
hdfs_uri="hdfs://pedaaslive.scmx2p100.isi",
filter_date_initial="202101",
filter_date_final="202112")
Spark Style Dataframe: file.py
==============================
df_pivot.show2(limit=10)
Pandas Style Dataframe: file.py
==============================
df_pivot2 = df_pivot.toPandas()
df_pivot2.show2()
```
## License
[Apache License 2.0](https://www.dropbox.com/s/8t6xtgk06o3ij61/LICENSE?dl=0).
## New features v1.0
## BugFix
- choco install visualcpp-build-tools
## Reference
- Jonathan Quiza [github](https://github.com/jonaqp).
- Jonathan Quiza [RumiMLSpark](http://rumi-ml.herokuapp.com/).
- Jonathan Quiza [linkedin](https://www.linkedin.com/in/jonaqp/).
Raw data
{
"_id": null,
"home_page": "https://github.com/jonaqp/spark_gaps_date_rorc_tools/",
"name": "spark-gaps-date-rorc-tools",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "spark, gaps, date",
"author": "Jonathan Quiza",
"author_email": "jony327@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/01/e7/a32fa02eac393d51e71747d009a2ebb93f297d985668048db41399e75faf/spark_gaps_date_rorc_tools-0.2.3.tar.gz",
"platform": null,
"description": "# spark_gaps_date_rorc_tools\r\n\r\n\r\n[](https://opensource.org/licenses/Apache-2.0)\r\n[](pyup)\r\n[](pyup)\r\n[](codecov)\r\n\r\n\r\n\r\n\r\nspark_gaps_date_rorc_tools is a Python library that implements get gaps dates\r\n## Installation\r\n\r\nThe code is packaged for PyPI, so that the installation consists in running:\r\n```sh\r\npip install spark-gaps-date-rorc-tools \r\n```\r\n\r\n\r\n## Usage\r\n\r\nwrapper take gaps dates\r\n\r\n```sh\r\nconfig.yaml\r\n===========\r\n conf:\r\n t_psan_test:\r\n table_path: \"/data/master/psan/data/t_psan_test/\"\r\n supplies : [\r\n \"/data/master/psan/data/t_ksag_test/\",\r\n \"/data/master/psan/data/t_psan_test/\"\r\n ]\r\n t_kctk_cust_rating_atrb:\r\n table_path: \"\"\r\n supplies : []\r\n\r\n\r\n\r\nexample1: file.py\r\n=================\r\nfrom spark_gaps_date_rorc_tools import show_gaps_date\r\ndf_pivot = show_gaps_date(spark=spark,\r\n config_path_name=\"config.yaml\",\r\n table_rorc=[\"t_psan_xxx\"]\r\n hdfs_uri=\"hdfs://pedaaslive.scmx2p100.isi\",\r\n filter_date_initial=\"202101\",\r\n filter_date_final=\"202112\")\r\n\r\nSpark Style Dataframe: file.py\r\n============================== \r\ndf_pivot.show2(limit=10)\r\n\r\n\r\n\r\nPandas Style Dataframe: file.py\r\n============================== \r\ndf_pivot2 = df_pivot.toPandas() \r\ndf_pivot2.show2()\r\n\r\n```\r\n\r\n## License\r\n\r\n[Apache License 2.0](https://www.dropbox.com/s/8t6xtgk06o3ij61/LICENSE?dl=0).\r\n\r\n\r\n## New features v1.0\r\n\r\n \r\n## BugFix\r\n- choco install visualcpp-build-tools\r\n\r\n\r\n\r\n## Reference\r\n\r\n - Jonathan Quiza [github](https://github.com/jonaqp).\r\n - Jonathan Quiza [RumiMLSpark](http://rumi-ml.herokuapp.com/).\r\n - Jonathan Quiza [linkedin](https://www.linkedin.com/in/jonaqp/).\r\n\r\n\r\n",
"bugtrack_url": null,
"license": null,
"summary": "spark_gaps_date_rorc_tools",
"version": "0.2.3",
"project_urls": {
"Download": "https://github.com/jonaqp/spark_gaps_date_rorct_tools/archive/main.zip",
"Homepage": "https://github.com/jonaqp/spark_gaps_date_rorc_tools/"
},
"split_keywords": [
"spark",
" gaps",
" date"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d23c6e0d75fddcace29c2d6bab60bb1ae25471dc25ebba30c4a41301b6b45196",
"md5": "26437cfe1f7fb86e3d98e311ce26693a",
"sha256": "03d6561045a8b364e3142053748fe68178f162ecc70f967183014b4a4ce17169"
},
"downloads": -1,
"filename": "spark_gaps_date_rorc_tools-0.2.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "26437cfe1f7fb86e3d98e311ce26693a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 10267,
"upload_time": "2024-09-12T00:31:56",
"upload_time_iso_8601": "2024-09-12T00:31:56.531546Z",
"url": "https://files.pythonhosted.org/packages/d2/3c/6e0d75fddcace29c2d6bab60bb1ae25471dc25ebba30c4a41301b6b45196/spark_gaps_date_rorc_tools-0.2.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "01e7a32fa02eac393d51e71747d009a2ebb93f297d985668048db41399e75faf",
"md5": "04e854b808db446d2dd06e6f84b3b364",
"sha256": "c266f0d3cc582f3a8a7e2af1643cacc3366a114a9fd88df8435b862922c30791"
},
"downloads": -1,
"filename": "spark_gaps_date_rorc_tools-0.2.3.tar.gz",
"has_sig": false,
"md5_digest": "04e854b808db446d2dd06e6f84b3b364",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 8928,
"upload_time": "2024-09-12T00:31:57",
"upload_time_iso_8601": "2024-09-12T00:31:57.906848Z",
"url": "https://files.pythonhosted.org/packages/01/e7/a32fa02eac393d51e71747d009a2ebb93f297d985668048db41399e75faf/spark_gaps_date_rorc_tools-0.2.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-12 00:31:57",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "jonaqp",
"github_project": "spark_gaps_date_rorc_tools",
"github_not_found": true,
"lcname": "spark-gaps-date-rorc-tools"
}
JSON
