| Name | dbt-duckdb-kedro-datasets JSON |
| Version |
0.1.1
JSON |
| download |
| home_page | None |
| Summary | Combine duckdb-dbt and Kedro Dataset to easily read Kedro Dataset configs (yaml), enabling conversion of Kedro projects to dbt. |
| upload_time | 2024-05-16 09:39:25 |
| maintainer | None |
| docs_url | None |
| author | Conrad |
| requires_python | <4.0,>=3.11 |
| license | None |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
Combine [duckdb-dbt](https://github.com/duckdb/dbt-duckdb/tree/master) and [Kedro](https://docs.kedro.org/en/stable/) [Datases](https://docs.kedro.org/projects/kedro-datasets/en/kedro-datasets-3.0.0/) to enable:
- extension of dbt to ingest wide array of data, and;
- conversion of Kedro projects to dbt by easily reading your Kedro data catalog configs (yaml files)
## Demo
You can add your existing Kedro definitions to your dbt sources like so:
`pip install dbt_duckdb_kedro_datasets`
```
version: 2
sources:
- name: my_source # can call this anything
schema: main
meta:
plugin: dbt_duckdb_kedro_datasets # this library
tables:
- name: my_table # can call this anything
description: "A dbt_duckdb_kedro_datasets test"
meta:
type: pandas.CSVDataset
filepath: ./data/1_raw/bikes.csv # file to ingest
load_args:
sep: ','
```
Now we can access this CSV in dbt
```
select *
from {{ source('my_source', 'my_table') }}
```
For a more complete example look at [this](example/example_dbt)
Raw data
{
"_id": null,
"home_page": null,
"name": "dbt-duckdb-kedro-datasets",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.11",
"maintainer_email": null,
"keywords": null,
"author": "Conrad",
"author_email": "conradbez1@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/23/ef/33a064806071ab5b0e8c5e24fcbfa7d12c3c3725a10322b45c0d7644f249/dbt_duckdb_kedro_datasets-0.1.1.tar.gz",
"platform": null,
"description": "Combine [duckdb-dbt](https://github.com/duckdb/dbt-duckdb/tree/master) and [Kedro](https://docs.kedro.org/en/stable/) [Datases](https://docs.kedro.org/projects/kedro-datasets/en/kedro-datasets-3.0.0/) to enable:\n\n- extension of dbt to ingest wide array of data, and;\n- conversion of Kedro projects to dbt by easily reading your Kedro data catalog configs (yaml files)\n\n## Demo\n\nYou can add your existing Kedro definitions to your dbt sources like so:\n\n`pip install dbt_duckdb_kedro_datasets`\n\n```\nversion: 2\n\nsources:\n - name: my_source # can call this anything\n schema: main\n meta:\n plugin: dbt_duckdb_kedro_datasets # this library\n tables:\n - name: my_table # can call this anything\n description: \"A dbt_duckdb_kedro_datasets test\"\n meta:\n type: pandas.CSVDataset\n filepath: ./data/1_raw/bikes.csv # file to ingest\n load_args:\n sep: ','\n```\n\nNow we can access this CSV in dbt\n\n```\nselect *\nfrom {{ source('my_source', 'my_table') }}\n```\n\nFor a more complete example look at [this](example/example_dbt)\n\n",
"bugtrack_url": null,
"license": null,
"summary": "Combine duckdb-dbt and Kedro Dataset to easily read Kedro Dataset configs (yaml), enabling conversion of Kedro projects to dbt.",
"version": "0.1.1",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6f2ac3af71e864b220e85c631dee44b27974809299e4b7a33dc8bb8e8923fd3a",
"md5": "2ab1768657ced352ac5f88818b393eb2",
"sha256": "6d69c37c27d60f277e6ef9dd563af23e7b8c84a2f3aa4a656e4b6b37626aa61b"
},
"downloads": -1,
"filename": "dbt_duckdb_kedro_datasets-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2ab1768657ced352ac5f88818b393eb2",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.11",
"size": 2645,
"upload_time": "2024-05-16T09:39:24",
"upload_time_iso_8601": "2024-05-16T09:39:24.020396Z",
"url": "https://files.pythonhosted.org/packages/6f/2a/c3af71e864b220e85c631dee44b27974809299e4b7a33dc8bb8e8923fd3a/dbt_duckdb_kedro_datasets-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "23ef33a064806071ab5b0e8c5e24fcbfa7d12c3c3725a10322b45c0d7644f249",
"md5": "03a513cfa81e7de7e091ae23ab2c7cf8",
"sha256": "654dafc94feca8402e29506dbaae1db6a15727369ce7a4f248dded3602010eae"
},
"downloads": -1,
"filename": "dbt_duckdb_kedro_datasets-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "03a513cfa81e7de7e091ae23ab2c7cf8",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.11",
"size": 2047,
"upload_time": "2024-05-16T09:39:25",
"upload_time_iso_8601": "2024-05-16T09:39:25.766386Z",
"url": "https://files.pythonhosted.org/packages/23/ef/33a064806071ab5b0e8c5e24fcbfa7d12c3c3725a10322b45c0d7644f249/dbt_duckdb_kedro_datasets-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-16 09:39:25",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "dbt-duckdb-kedro-datasets"
}
