| Name | openlineage-dbt JSON |
| Version |
1.11.3
JSON |
| download |
| home_page | None |
| Summary | OpenLineage integration with dbt |
| upload_time | 2024-04-04 20:18:05 |
| maintainer | None |
| docs_url | None |
| author | OpenLineage |
| requires_python | >=3.8 |
| license | None |
| keywords |
openlineage
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# OpenLineage dbt integration
Wrapper script for automatic metadata collection from dbt
## Features
**Metadata**
* Model run lifecycle
* Model inputs / outputs
## Requirements
- [Python >= 3.8](https://www.python.org/downloads)
- [dbt >= 0.20](https://www.getdbt.com/)
Right now, `openlineage-dbt` only supports `bigquery`, `snowflake`, `spark` and `redshift` dbt adapters.
## Installation
```bash
$ pip3 install openlineage-dbt
```
To install from source, run:
```bash
$ pip install .
```
## Configuration
### `HTTP` Backend Environment Variables
`openlineage-dbt` uses the OpenLineage client to push data to the OpenLineage backend.
The OpenLineage client depends on environment variables:
* `OPENLINEAGE_URL` - point to service which will consume OpenLineage events
* `OPENLINEAGE_API_KEY` - set if consumer of OpenLineage events requires `Bearer` authentication key
* `OPENLINEAGE_NAMESPACE` - set if you are using something other than the `default` namespace for job namespace.
### Logging
In addition to conventional logging approaches, the OpenLineage dbt wrapper script provides an alternative way of configuring its logging behavior. By setting the `OPENLINEAGE_DBT_LOGGING` environment variable, you can establish the logging level for the `openlineage.dbt` and its child modules.
You can also set log level of `dbtol` which is deprecated.
## Usage
To begin collecting dbt metadata with OpenLineage, replace `dbt run` with `dbt-ol run`.
Additional table and column level metadata will be available if `catalog.json`, a result of running `dbt docs generate`, will be found in the target directory.
If you need to send events without running the job you can use the command `dbt-ol send-events`, it will send the metadata of your last run without running the job.
----
SPDX-License-Identifier: Apache-2.0\
Copyright 2018-2023 contributors to the OpenLineage project
Raw data
{
"_id": null,
"home_page": null,
"name": "openlineage-dbt",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "openlineage",
"author": "OpenLineage",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/68/40/07577a0bb51c3a5d7a469a7168507e981321ed1d79e0daff32a3f52a0711/openlineage-dbt-1.11.3.tar.gz",
"platform": null,
"description": "# OpenLineage dbt integration\n\nWrapper script for automatic metadata collection from dbt\n\n## Features\n\n**Metadata**\n\n* Model run lifecycle\n* Model inputs / outputs\n\n## Requirements\n\n- [Python >= 3.8](https://www.python.org/downloads)\n- [dbt >= 0.20](https://www.getdbt.com/)\n\nRight now, `openlineage-dbt` only supports `bigquery`, `snowflake`, `spark` and `redshift` dbt adapters.\n\n## Installation\n\n```bash\n$ pip3 install openlineage-dbt\n```\n\nTo install from source, run:\n\n```bash\n$ pip install .\n```\n\n## Configuration\n\n\n### `HTTP` Backend Environment Variables\n\n`openlineage-dbt` uses the OpenLineage client to push data to the OpenLineage backend.\n\nThe OpenLineage client depends on environment variables:\n\n* `OPENLINEAGE_URL` - point to service which will consume OpenLineage events\n* `OPENLINEAGE_API_KEY` - set if consumer of OpenLineage events requires `Bearer` authentication key\n* `OPENLINEAGE_NAMESPACE` - set if you are using something other than the `default` namespace for job namespace.\n\n\n### Logging\n\nIn addition to conventional logging approaches, the OpenLineage dbt wrapper script provides an alternative way of configuring its logging behavior. By setting the `OPENLINEAGE_DBT_LOGGING` environment variable, you can establish the logging level for the `openlineage.dbt` and its child modules.\n\nYou can also set log level of `dbtol` which is deprecated.\n\n## Usage\n\nTo begin collecting dbt metadata with OpenLineage, replace `dbt run` with `dbt-ol run`.\n\nAdditional table and column level metadata will be available if `catalog.json`, a result of running `dbt docs generate`, will be found in the target directory.\n\nIf you need to send events without running the job you can use the command `dbt-ol send-events`, it will send the metadata of your last run without running the job.\n\n----\nSPDX-License-Identifier: Apache-2.0\\\nCopyright 2018-2023 contributors to the OpenLineage project\n",
"bugtrack_url": null,
"license": null,
"summary": "OpenLineage integration with dbt",
"version": "1.11.3",
"project_urls": null,
"split_keywords": [
"openlineage"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d286e0dff76d5551688eb1e36000d844cf10652a60e6302f054e5e97069b9075",
"md5": "9a699f0814fe5b5e9c8f429820c9aaa7",
"sha256": "4f10a9ec86d00f6afe6eaca103fe91e292be163f0f8e49721c54a0562b21c9ff"
},
"downloads": -1,
"filename": "openlineage_dbt-1.11.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9a699f0814fe5b5e9c8f429820c9aaa7",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 4606,
"upload_time": "2024-04-04T20:17:52",
"upload_time_iso_8601": "2024-04-04T20:17:52.341816Z",
"url": "https://files.pythonhosted.org/packages/d2/86/e0dff76d5551688eb1e36000d844cf10652a60e6302f054e5e97069b9075/openlineage_dbt-1.11.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "684007577a0bb51c3a5d7a469a7168507e981321ed1d79e0daff32a3f52a0711",
"md5": "55c9f93eb3cd2f054eb0da84bd7f8b8d",
"sha256": "5c17d88ba33875400557b8be3ecbcab66b01ea0a7125905c07b6ef38f0584662"
},
"downloads": -1,
"filename": "openlineage-dbt-1.11.3.tar.gz",
"has_sig": false,
"md5_digest": "55c9f93eb3cd2f054eb0da84bd7f8b8d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 4824,
"upload_time": "2024-04-04T20:18:05",
"upload_time_iso_8601": "2024-04-04T20:18:05.556377Z",
"url": "https://files.pythonhosted.org/packages/68/40/07577a0bb51c3a5d7a469a7168507e981321ed1d79e0daff32a3f52a0711/openlineage-dbt-1.11.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-04 20:18:05",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "openlineage-dbt"
}
