| Name | extralo JSON |
| Version |
0.17.4
JSON |
| download |
| home_page | None |
| Summary | ETL for Python |
| upload_time | 2024-11-11 19:33:06 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.9 |
| license | None |
| keywords |
etl
data
data-engineering
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# ETL using python
Python package for extracting data from a source, transforming it and loading it to a destination, with validation in between.
The provided ETL pipeline provides useful functionality on top of the usual operations:
- **Extract**: Extract data from multiples sources, in parallel (using threads).
- **Validate**: Validate the extracted data, to make sure it matches what will be required by the transform step, using pandera schemas. This provide early fail if there is any unexpected change in the sources.
- **Transform**: Define the logic for transformation of the data, making it reusable, and allowing multiple data frames as input and multiple data frames as output.
- **Validate again**: Validate the transformed data, to make sure it matches your expectation, and what the destination will require.
- **Load**: Load multiple data, each to one or more destination, and load diferent data to diferent destinations in parallel (using threads).
## Installation
The package is available at PyPI, so you can install it using pip:
```bash
pip install extralo
```
Raw data
{
"_id": null,
"home_page": null,
"name": "extralo",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "Vitor Capdeville <vgcapdeville@hotmail.com>",
"keywords": "etl, data, data-engineering",
"author": null,
"author_email": "Vitor Capdeville <vgcapdeville@hotmail.com>",
"download_url": "https://files.pythonhosted.org/packages/ac/63/8e7b86426b2a0cc9a21901bc3c513850fcfe78ff5ed9bc22f26b0554166b/extralo-0.17.4.tar.gz",
"platform": null,
"description": "# ETL using python\n\nPython package for extracting data from a source, transforming it and loading it to a destination, with validation in between.\n\nThe provided ETL pipeline provides useful functionality on top of the usual operations:\n\n- **Extract**: Extract data from multiples sources, in parallel (using threads).\n- **Validate**: Validate the extracted data, to make sure it matches what will be required by the transform step, using pandera schemas. This provide early fail if there is any unexpected change in the sources.\n- **Transform**: Define the logic for transformation of the data, making it reusable, and allowing multiple data frames as input and multiple data frames as output.\n- **Validate again**: Validate the transformed data, to make sure it matches your expectation, and what the destination will require.\n- **Load**: Load multiple data, each to one or more destination, and load diferent data to diferent destinations in parallel (using threads).\n\n## Installation\n\nThe package is available at PyPI, so you can install it using pip:\n\n```bash\npip install extralo\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "ETL for Python",
"version": "0.17.4",
"project_urls": null,
"split_keywords": [
"etl",
" data",
" data-engineering"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5e62107b26909accd1649f1cf89d80857da13060747ff581b0e3c002b5856da2",
"md5": "ad8964372cda62da795525bf8517b6a4",
"sha256": "5b9416d5b2c68dc8d35973e338f85baa50e1bf06b979e124e6e355b354121940"
},
"downloads": -1,
"filename": "extralo-0.17.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ad8964372cda62da795525bf8517b6a4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 13852,
"upload_time": "2024-11-11T19:33:05",
"upload_time_iso_8601": "2024-11-11T19:33:05.218727Z",
"url": "https://files.pythonhosted.org/packages/5e/62/107b26909accd1649f1cf89d80857da13060747ff581b0e3c002b5856da2/extralo-0.17.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ac638e7b86426b2a0cc9a21901bc3c513850fcfe78ff5ed9bc22f26b0554166b",
"md5": "26d128ad71bbb8c6f11b2a0159a23c69",
"sha256": "6cd360bbb6252ef1cadadeabeb194c5a85c3a343c728d794fc91af4b2fc750c5"
},
"downloads": -1,
"filename": "extralo-0.17.4.tar.gz",
"has_sig": false,
"md5_digest": "26d128ad71bbb8c6f11b2a0159a23c69",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 12081,
"upload_time": "2024-11-11T19:33:06",
"upload_time_iso_8601": "2024-11-11T19:33:06.460121Z",
"url": "https://files.pythonhosted.org/packages/ac/63/8e7b86426b2a0cc9a21901bc3c513850fcfe78ff5ed9bc22f26b0554166b/extralo-0.17.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-11 19:33:06",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "extralo"
}
