| Name | pytrilogyt JSON |
| Version |
0.0.9
JSON |
| download |
| home_page | None |
| Summary | Enhanced Trilogy for common ETL needs. |
| upload_time | 2024-10-22 12:58:50 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | None |
| license | None |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
## Simple Declarative Data Pipelines
Combine the simplicity of Trilogy with the modern data stack such as DBT.
> [!TIP]
> Pitch: don't worry about optimizing your ETL staging tables ever again - write your final tables and let TrilogyT handle the rest.
Compile your models to ETL scripts to run on demand. Rebuild, run, and test easily.
Translates 'Persist' statements in Trilogy to scheduled ETL jobs.
Currently supported backends:
- Native (optimize a PreQL model)
- DBT
> [!WARNING]
> This is an experimental library. The API is subject to change.
## Flags
--optimize=X - Any CTE used at least X times in calculating final model outputs will be materialized for reuse.
## Install
`pip install pytrilogyt`
## How to Run
preqlt <preql_path> <output_path> <backend> --run
## DBT
For dbt, the output_path should be the root of the dbt project, where the dbt_project.yml file exists.
```bash
trilogyt dbt/models/core/ ./dbt bigquery --run
```
Each source preql file will be built into a separate DBT sub folder with one model per persist statement.
```bash
17:12:37 Running with dbt=1.7.4
17:12:38 Registered adapter: bigquery=1.7.2
17:12:38 Found 4 models, 4 tests, 0 sources, 0 exposures, 0 metrics, 447 macros, 0 groups, 0 semantic models
17:12:38
17:12:40 Concurrency: 4 threads (target='dev')
17:12:40
17:12:41 1 of 4 START sql view model dbt_test.customers ................................. [RUN]
17:12:41 2 of 4 START sql table model dbt_test.customers_preql_preqlt_gen_model ......... [RUN]
17:12:41 3 of 4 START sql table model dbt_test.my_first_dbt_model ....................... [RUN]
17:12:42 1 of 4 OK created sql view model dbt_test.customers ............................ [CREATE VIEW (0 processed) in 1.09s]
17:12:43 3 of 4 OK created sql table model dbt_test.my_first_dbt_model .................. [CREATE TABLE (2.0 rows, 0 processed) in 2.78s]
17:12:43 4 of 4 START sql view model dbt_test.my_second_dbt_model ....................... [RUN]
17:12:44 2 of 4 OK created sql table model dbt_test.customers_preql_preqlt_gen_model .... [CREATE TABLE (100.0 rows, 4.3 KiB processed) in 3.55s]
17:12:44 4 of 4 OK created sql view model dbt_test.my_second_dbt_model .................. [CREATE VIEW (0 processed) in 1.10s]
17:12:44
17:12:44 Finished running 2 view models, 2 table models in 0 hours 0 minutes and 6.37 seconds (6.37s).
17:12:45
17:12:45 Completed successfully
17:12:45
17:12:45 Done. PASS=4 WARN=0 ERROR=0 SKIP=0 TOTAL=4
customers: success
my_first_dbt_model: success
customers_preql_preqlt_gen_model: success
my_second_dbt_model: success
```
### From IO
```console
Write-Output """constant x <-5; persist into static as static select x;""" | trilogyt <output_path> bigquery
```
Raw data
{
"_id": null,
"home_page": null,
"name": "pytrilogyt",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": null,
"author_email": "preql-community@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/c4/ef/68c8c432ff178b31c46840dee524d5fa6ba5f0b954dca30461352707d5d7/pytrilogyt-0.0.9.tar.gz",
"platform": null,
"description": "## Simple Declarative Data Pipelines\n\nCombine the simplicity of Trilogy with the modern data stack such as DBT.\n\n> [!TIP]\n> Pitch: don't worry about optimizing your ETL staging tables ever again - write your final tables and let TrilogyT handle the rest. \n\n\nCompile your models to ETL scripts to run on demand. Rebuild, run, and test easily.\n\nTranslates 'Persist' statements in Trilogy to scheduled ETL jobs. \n\nCurrently supported backends:\n- Native (optimize a PreQL model)\n- DBT\n\n> [!WARNING]\n> This is an experimental library. The API is subject to change.\n\n\n## Flags\n\n--optimize=X - Any CTE used at least X times in calculating final model outputs will be materialized for reuse.\n\n\n## Install\n\n`pip install pytrilogyt`\n\n## How to Run\n\npreqlt <preql_path> <output_path> <backend> --run\n\n\n## DBT\n\nFor dbt, the output_path should be the root of the dbt project, where the dbt_project.yml file exists.\n\n\n```bash\ntrilogyt dbt/models/core/ ./dbt bigquery --run\n```\n\nEach source preql file will be built into a separate DBT sub folder with one model per persist statement.\n\n```bash\n17:12:37 Running with dbt=1.7.4\n17:12:38 Registered adapter: bigquery=1.7.2\n17:12:38 Found 4 models, 4 tests, 0 sources, 0 exposures, 0 metrics, 447 macros, 0 groups, 0 semantic models\n17:12:38\n17:12:40 Concurrency: 4 threads (target='dev')\n17:12:40\n17:12:41 1 of 4 START sql view model dbt_test.customers ................................. [RUN]\n17:12:41 2 of 4 START sql table model dbt_test.customers_preql_preqlt_gen_model ......... [RUN]\n17:12:41 3 of 4 START sql table model dbt_test.my_first_dbt_model ....................... [RUN]\n17:12:42 1 of 4 OK created sql view model dbt_test.customers ............................ [CREATE VIEW (0 processed) in 1.09s]\n17:12:43 3 of 4 OK created sql table model dbt_test.my_first_dbt_model .................. [CREATE TABLE (2.0 rows, 0 processed) in 2.78s]\n17:12:43 4 of 4 START sql view model dbt_test.my_second_dbt_model ....................... [RUN]\n17:12:44 2 of 4 OK created sql table model dbt_test.customers_preql_preqlt_gen_model .... [CREATE TABLE (100.0 rows, 4.3 KiB processed) in 3.55s]\n17:12:44 4 of 4 OK created sql view model dbt_test.my_second_dbt_model .................. [CREATE VIEW (0 processed) in 1.10s]\n17:12:44\n17:12:44 Finished running 2 view models, 2 table models in 0 hours 0 minutes and 6.37 seconds (6.37s).\n17:12:45 \n17:12:45 Completed successfully\n17:12:45\n17:12:45 Done. PASS=4 WARN=0 ERROR=0 SKIP=0 TOTAL=4\ncustomers: success\nmy_first_dbt_model: success\ncustomers_preql_preqlt_gen_model: success\nmy_second_dbt_model: success\n```\n\n\n### From IO\n\n```console\nWrite-Output \"\"\"constant x <-5; persist into static as static select x;\"\"\" | trilogyt <output_path> bigquery\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "Enhanced Trilogy for common ETL needs.",
"version": "0.0.9",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5ca84f16a174d5da043f051c2c03188bbd9de9f8668ac7672190b5d39722c9fa",
"md5": "b3cf52eec2724c83ba99899e73059925",
"sha256": "22654317527a5af54fc2ecda355302f66c2c202620609ecfd7fb6bbc3d4d288c"
},
"downloads": -1,
"filename": "pytrilogyt-0.0.9-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b3cf52eec2724c83ba99899e73059925",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 17900,
"upload_time": "2024-10-22T12:58:49",
"upload_time_iso_8601": "2024-10-22T12:58:49.838401Z",
"url": "https://files.pythonhosted.org/packages/5c/a8/4f16a174d5da043f051c2c03188bbd9de9f8668ac7672190b5d39722c9fa/pytrilogyt-0.0.9-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c4ef68c8c432ff178b31c46840dee524d5fa6ba5f0b954dca30461352707d5d7",
"md5": "5c2416ce04dd3a7a01ebdb0ea4af6f64",
"sha256": "096b69ae9707627ed74753aa5450564c0ae6dae616c3e4f028c4bc48e73f8c25"
},
"downloads": -1,
"filename": "pytrilogyt-0.0.9.tar.gz",
"has_sig": false,
"md5_digest": "5c2416ce04dd3a7a01ebdb0ea4af6f64",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 13522,
"upload_time": "2024-10-22T12:58:50",
"upload_time_iso_8601": "2024-10-22T12:58:50.816160Z",
"url": "https://files.pythonhosted.org/packages/c4/ef/68c8c432ff178b31c46840dee524d5fa6ba5f0b954dca30461352707d5d7/pytrilogyt-0.0.9.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-22 12:58:50",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "pytrilogyt"
}
