| Name | gooddata-dbt JSON |
| Version |
1.31.0
JSON |
| download |
| home_page | None |
| Summary | dbt plugin for GoodData |
| upload_time | 2024-11-14 15:17:59 |
| maintainer | None |
| docs_url | None |
| author | GoodData |
| requires_python | >=3.9.0 |
| license | MIT |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# gooddata-dbt
GoodData plugin for dbt. Reads dbt models and profiles, generates GoodData semantic model.
## Install
```shell
pip install gooddata-dbt
# Or add the corresponding line to requirements.txt
# Or install specific version
pip install gooddata-dbt==1.0.0
```
You can also install optional dependencies:
```shell
# To allow sending comments to GitHub pull requests
pip install PyGithub
# To allow automatic translation of GoodData metadata
pip install deep-translator
```
## Configuration, parametrization
Create `gooddata.yaml` file to configure so-called data products and environments.
Check [gooddata_example.yml](gooddata_example.yml) file for more details.
Parametrization of each execution can be done using environment variables / tool arguments.
Use main --help and --help for each use case to learn more.
Alternatively, you can configure everything with environment variables.
You can directly set env variables in a shell session, or store them to .env file(s).
We provide the following example:
- [.env.dev](.env.dev)
- [.env.custom.dev](.env.custom.dev) is loaded from the above file and contains sensitive variables.
Add `.env.custom.*` to .gitignore!
Then load .env files:
```bash
source .env.local
```
## Use cases
```shell
gooddata-dbt --help
```
The plugin provides the following use cases:
- provision_workspaces
- Provisions workspaces to GoodData based on gooddata.yaml file
- register_data_sources
- Registers data source in GoodData for each relevant dbt profile
- deploy_ldm
- Reads dbt models and profiles
- Scans data source (connection props from dbt profiles) through GoodData to get column data types (optional in dbt)
- Generates GoodData LDM(Logical Data Model) from dbt models. Can utilize custom gooddata-specific metadata, more below
- upload_notification
- Invalidates caches for data source
- deploy_analytics
- Reads content of `gooddata_layout` folder and deploys analytics model to GoodData
- store_analytics
- Reads analytics model from GoodData instance and stores it to disk to `gooddata_layout` folder
- test_visualizations
- Lists all visualizations execution from GoodData instance, and executes each report to validate it
- dbt_cloud
- Runs dbt cloud job through their API. Alternative to running dbt-core locally.
- If running in CI pipeline, it can also notify about performance degradations in a form of GitHub/Gitlab comment.
- dbt_cloud_stats
- Esp. for testing purposes. It's triggered from dbt_cloud as well. It collects stats and reports perf degradations.
- Check [.env.dev](.env.dev)/[.env.custom.dev](.env.custom.dev) files for how to set related env variables.
## Custom metadata in dbt models (optional)
If you want to generate optimal LDM from dbt models, sometimes you need to specify semantic metadata in dbt models.
In general, all GoodData metadata must be put to dbt models under `meta` key, except descriptions.
### Titles, descriptions
dbt supports only `description` field. For now, gooddata-dbt generates GoodData title/description from dbt description.
Can be specified for both tables and columns.
### Model ID
Per table, you can specify `model_id`. When deploying models/analytics, you can include any subset of model_ids.
```yaml
models:
- name: xxx
meta:
gooddata:
model_id: my_id
```
### GoodData entities
By default, gooddata-dbt generates GoodData entities based on the following rules:
- data type = NUMERIC (decimal number) - fact
- data_type = DATE/TIMESTAMP/TIMESTAMPTZ - date dimension
- other data types = attributes
To override the default, specify custom GoodData meta this way:
```yaml
columns:
- name: xxxx
meta:
gooddata:
ldm_type: fact/attribute/label/date/reference/primary_key
referenced_table: <table name, target of reference (FK), if ldm_type=reference>
label_type: TEXT/HYPERLINK/GEO_LATITUDE/GEO_LONGITUDE
attribute_column: <column name of attribute of label, if ldm_type=label>
sort_column: "<any column in the same table, may not be exposed as LDM object>"
sort_direction: "DESC"
# Only for labels, this label will be displayed by default in reports
default_view: true
```
Raw data
{
"_id": null,
"home_page": null,
"name": "gooddata-dbt",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9.0",
"maintainer_email": null,
"keywords": null,
"author": "GoodData",
"author_email": "support@gooddata.com",
"download_url": "https://files.pythonhosted.org/packages/bb/c5/34c2b0ded31f2332baf787982311364e5122e1f4d2b20e8c1215111444c4/gooddata_dbt-1.31.0.tar.gz",
"platform": null,
"description": "# gooddata-dbt\nGoodData plugin for dbt. Reads dbt models and profiles, generates GoodData semantic model.\n\n## Install\n\n```shell\npip install gooddata-dbt\n# Or add the corresponding line to requirements.txt\n# Or install specific version\npip install gooddata-dbt==1.0.0\n```\n\nYou can also install optional dependencies:\n```shell\n# To allow sending comments to GitHub pull requests\npip install PyGithub\n# To allow automatic translation of GoodData metadata\npip install deep-translator\n```\n\n## Configuration, parametrization\nCreate `gooddata.yaml` file to configure so-called data products and environments.\nCheck [gooddata_example.yml](gooddata_example.yml) file for more details.\n\nParametrization of each execution can be done using environment variables / tool arguments.\nUse main --help and --help for each use case to learn more.\n\nAlternatively, you can configure everything with environment variables.\nYou can directly set env variables in a shell session, or store them to .env file(s).\nWe provide the following example:\n- [.env.dev](.env.dev)\n- [.env.custom.dev](.env.custom.dev) is loaded from the above file and contains sensitive variables.\n Add `.env.custom.*` to .gitignore!\n\nThen load .env files:\n```bash\nsource .env.local\n```\n\n## Use cases\n```shell\ngooddata-dbt --help\n```\nThe plugin provides the following use cases:\n- provision_workspaces\n - Provisions workspaces to GoodData based on gooddata.yaml file\n- register_data_sources\n - Registers data source in GoodData for each relevant dbt profile\n- deploy_ldm\n - Reads dbt models and profiles\n - Scans data source (connection props from dbt profiles) through GoodData to get column data types (optional in dbt)\n - Generates GoodData LDM(Logical Data Model) from dbt models. Can utilize custom gooddata-specific metadata, more below\n- upload_notification\n - Invalidates caches for data source\n- deploy_analytics\n - Reads content of `gooddata_layout` folder and deploys analytics model to GoodData\n- store_analytics\n - Reads analytics model from GoodData instance and stores it to disk to `gooddata_layout` folder\n- test_visualizations\n - Lists all visualizations execution from GoodData instance, and executes each report to validate it\n- dbt_cloud\n - Runs dbt cloud job through their API. Alternative to running dbt-core locally.\n - If running in CI pipeline, it can also notify about performance degradations in a form of GitHub/Gitlab comment.\n- dbt_cloud_stats\n - Esp. for testing purposes. It's triggered from dbt_cloud as well. It collects stats and reports perf degradations.\n - Check [.env.dev](.env.dev)/[.env.custom.dev](.env.custom.dev) files for how to set related env variables.\n\n## Custom metadata in dbt models (optional)\nIf you want to generate optimal LDM from dbt models, sometimes you need to specify semantic metadata in dbt models.\n\nIn general, all GoodData metadata must be put to dbt models under `meta` key, except descriptions.\n\n### Titles, descriptions\ndbt supports only `description` field. For now, gooddata-dbt generates GoodData title/description from dbt description.\n\nCan be specified for both tables and columns.\n\n### Model ID\nPer table, you can specify `model_id`. When deploying models/analytics, you can include any subset of model_ids.\n```yaml\nmodels:\n - name: xxx\n meta:\n gooddata:\n model_id: my_id\n```\n\n### GoodData entities\nBy default, gooddata-dbt generates GoodData entities based on the following rules:\n- data type = NUMERIC (decimal number) - fact\n- data_type = DATE/TIMESTAMP/TIMESTAMPTZ - date dimension\n- other data types = attributes\n\nTo override the default, specify custom GoodData meta this way:\n```yaml\ncolumns:\n - name: xxxx\n meta:\n gooddata:\n ldm_type: fact/attribute/label/date/reference/primary_key\n referenced_table: <table name, target of reference (FK), if ldm_type=reference>\n label_type: TEXT/HYPERLINK/GEO_LATITUDE/GEO_LONGITUDE\n attribute_column: <column name of attribute of label, if ldm_type=label>\n sort_column: \"<any column in the same table, may not be exposed as LDM object>\"\n sort_direction: \"DESC\"\n # Only for labels, this label will be displayed by default in reports\n default_view: true\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "dbt plugin for GoodData",
"version": "1.31.0",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3e6902d92f5103b57fece71d5df9314970ede667fc83273a1dc56342febfca55",
"md5": "395c4dec4520bbc57d780477dec7bb6f",
"sha256": "a9a6f4b025b95d9972f75141c0edd8e3e2d830c1aa6676690f1baf78e72916cb"
},
"downloads": -1,
"filename": "gooddata_dbt-1.31.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "395c4dec4520bbc57d780477dec7bb6f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9.0",
"size": 50748,
"upload_time": "2024-11-14T15:17:57",
"upload_time_iso_8601": "2024-11-14T15:17:57.177307Z",
"url": "https://files.pythonhosted.org/packages/3e/69/02d92f5103b57fece71d5df9314970ede667fc83273a1dc56342febfca55/gooddata_dbt-1.31.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "bbc534c2b0ded31f2332baf787982311364e5122e1f4d2b20e8c1215111444c4",
"md5": "cdf55080d7108756aab0efd18b3fc57b",
"sha256": "db07bdcba321cd592620bc5d90ab7dbaadc69e391182d89a727013a2f8392b40"
},
"downloads": -1,
"filename": "gooddata_dbt-1.31.0.tar.gz",
"has_sig": false,
"md5_digest": "cdf55080d7108756aab0efd18b3fc57b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9.0",
"size": 47575,
"upload_time": "2024-11-14T15:17:59",
"upload_time_iso_8601": "2024-11-14T15:17:59.481950Z",
"url": "https://files.pythonhosted.org/packages/bb/c5/34c2b0ded31f2332baf787982311364e5122e1f4d2b20e8c1215111444c4/gooddata_dbt-1.31.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-14 15:17:59",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "gooddata-dbt"
}
