# dbt-hive
The `dbt-hive` adapter allows you to use [dbt](https://www.getdbt.com/) along with [Apache Hive](https://hive.apache.org/) and [Cloudera Data Platform](https://cloudera.com)
## Getting started
- [Install dbt](https://docs.getdbt.com/docs/installation)
- Read the [introduction](https://docs.getdbt.com/docs/introduction/) and [viewpoint](https://docs.getdbt.com/docs/about/viewpoint/)
### Credits
The initial adapter code was developed by bachng2017 who agreed to transfer the ownership and continue active development.
This code base is now being actively developed and maintained by Cloudera.
### Requirements
Current version of dbt-hive use dbt-core 1.7.*. We are actively working on supporting the next version of dbt-core 1.8
Python >= 3.8
dbt-core ~= 1.7.*
impyla >= 0.18
### Install
```
pip3 install --user dbt-hive
```
### Sample profile
```
demo_project:
target: dev
outputs:
dev:
type: hive
auth_type: LDAP
user: [username]
password: [password]
schema: [schema]
host: [hive-meta-store-host]
port: 443
http_path: [http-path]
thread: 1
```
## Supported features
| Name | Supported | Iceberg |
|------|-----------|---------|
|Materialization: View | Yes | N/A |
|Materialization: Table| Yes | Yes |
|Materialization: Table with Partitions | Yes | Yes |
|Materialization: Incremental - Append | Yes | Yes|
|Materialization: Incremental - Append with Partitions | Yes | Yes|
|Materialization: Incremental - Insert+Overwrite with Partitions | Yes | No |
|Materialization: Incremental - Merge | Yes | Yes |
|Materialization: Incremental - Merge with Partitions | No | Yes* |
|Materialization: Ephemeral | No | No |
|Seeds | Yes | Yes |
|Tests | Yes | Yes |
|Snapshots | No | No |
|Documentation | Yes | No |
|Authentication: LDAP | Yes | Yes |
|Authentication: Kerberos | Yes | Yes |
### Incremental
Incremental models are explained in [dbt documentation](https://docs.getdbt.com/docs/build/incremental-models). This section covered the details about the incremental strategy supported by the dbt-hive.
| Strategy | ACID Table | Iceberg Table |
|------|------|---------|
| Incremental Full-Refresh | Yes | Yes |
| Incremental Append | Yes | Yes |
| Incremental Append with Partitions | Yes | Yes |
| Incremental Insert Overwrite | Not recommended without Partitions* | Not recommended without Partitions* |
| Incremental Insert Overwrite with Partitions | Yes | No |
| Incremental Merge | Yes | Yes* (only v2) |
| Incremental Merge with Partitions | No* | Yes* (only v2) |
**Note***:
1. Incremental Insert overwrite without the partition columns results into completely overwriting the full table and may result in the data-loss. Hence it is not recommended to used. This can happen for Hive ACID, Iceberg v1 & v2 tables.
1. Incremental Merge for iceberg v1 table is not supported because Iceberg v1 tables are not transactional.
1. Incremental Merge with partition columns is not supported because Hive ACID tables doesn't support updating values of partition columns.
Support for [On-Schema Change](https://docs.getdbt.com/docs/build/incremental-models#what-if-the-columns-of-my-incremental-model-change) strategy in dbt-hive:
| Strategy | ACID Table | Iceberg Table |
|------|------|---------|
| ignore (default) | Supported | Supported |
| fail | Supported | Supported |
| append_new_columns | Adds new columns | Adds new columns |
| sync_all_columns | Adds new columns and updates datatypes but doesn't remove existing columns | Adds new columns, updates datatypes and removes existing columns |
### Tests Coverage
#### Functional Tests
| Name | Base | Iceberg |
|------|------|---------|
|Materialization: View | Yes | N/A |
|Materialization: Table| Yes | Yes |
|Materialization: Table with Partitions | Yes | Yes |
|Materialization: Incremental - Append | Yes | Yes|
|Materialization: Incremental - Append with Partitions | Yes | Yes|
|Materialization: Incremental - Insert+Overwrite with Partitions | Yes | No |
|Materialization: Incremental - Merge | No | No |
|Materialization: Ephemeral | No | No |
|Seeds | Yes | Yes |
|Tests | Yes | Yes |
|Snapshots | No | No |
|Documentation | Yes | No |
|Authentication: LDAP | Yes | Yes |
|Authentication: Kerberos | Yes | Yes |
**Note**: Kerberos is only qualified on Unix platform.
Raw data
{
"_id": null,
"home_page": "https://github.com/cloudera/dbt-hive",
"name": "dbt-hive",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "Cloudera",
"author_email": "innovation-feedback@cloudera.com",
"download_url": "https://files.pythonhosted.org/packages/0e/7c/714020ee12122df3103735b082ff75a3a3a98b7f670e338bf04f8355dc44/dbt_hive-1.7.0.tar.gz",
"platform": null,
"description": "# dbt-hive\n\nThe `dbt-hive` adapter allows you to use [dbt](https://www.getdbt.com/) along with [Apache Hive](https://hive.apache.org/) and [Cloudera Data Platform](https://cloudera.com)\n\n\n## Getting started\n\n- [Install dbt](https://docs.getdbt.com/docs/installation)\n- Read the [introduction](https://docs.getdbt.com/docs/introduction/) and [viewpoint](https://docs.getdbt.com/docs/about/viewpoint/)\n\n### Credits\n\nThe initial adapter code was developed by bachng2017 who agreed to transfer the ownership and continue active development.\nThis code base is now being actively developed and maintained by Cloudera.\n\n### Requirements\n\nCurrent version of dbt-hive use dbt-core 1.7.*. We are actively working on supporting the next version of dbt-core 1.8\n\nPython >= 3.8\ndbt-core ~= 1.7.*\nimpyla >= 0.18\n\n### Install\n```\npip3 install --user dbt-hive\n```\n\n### Sample profile\n```\ndemo_project:\n target: dev\n outputs:\n dev:\n type: hive\n auth_type: LDAP\n user: [username]\n password: [password]\n schema: [schema]\n host: [hive-meta-store-host]\n port: 443\n http_path: [http-path]\n thread: 1\n```\n\n## Supported features\n| Name | Supported | Iceberg |\n|------|-----------|---------|\n|Materialization: View | Yes | N/A |\n|Materialization: Table| Yes | Yes |\n|Materialization: Table with Partitions | Yes | Yes |\n|Materialization: Incremental - Append | Yes | Yes|\n|Materialization: Incremental - Append with Partitions | Yes | Yes|\n|Materialization: Incremental - Insert+Overwrite with Partitions | Yes | No |\n|Materialization: Incremental - Merge | Yes | Yes |\n|Materialization: Incremental - Merge with Partitions | No | Yes* |\n|Materialization: Ephemeral | No | No |\n|Seeds | Yes | Yes |\n|Tests | Yes | Yes |\n|Snapshots | No | No |\n|Documentation | Yes | No |\n|Authentication: LDAP | Yes | Yes |\n|Authentication: Kerberos | Yes | Yes |\n\n### Incremental\n\nIncremental models are explained in [dbt documentation](https://docs.getdbt.com/docs/build/incremental-models). This section covered the details about the incremental strategy supported by the dbt-hive.\n\n| Strategy | ACID Table | Iceberg Table |\n|------|------|---------|\n| Incremental Full-Refresh | Yes | Yes |\n| Incremental Append | Yes | Yes |\n| Incremental Append with Partitions | Yes | Yes |\n| Incremental Insert Overwrite | Not recommended without Partitions* | Not recommended without Partitions* |\n| Incremental Insert Overwrite with Partitions | Yes | No |\n| Incremental Merge | Yes | Yes* (only v2) |\n| Incremental Merge with Partitions | No* | Yes* (only v2) |\n\n**Note***:\n1. Incremental Insert overwrite without the partition columns results into completely overwriting the full table and may result in the data-loss. Hence it is not recommended to used. This can happen for Hive ACID, Iceberg v1 & v2 tables.\n1. Incremental Merge for iceberg v1 table is not supported because Iceberg v1 tables are not transactional.\n1. Incremental Merge with partition columns is not supported because Hive ACID tables doesn't support updating values of partition columns.\n\n\nSupport for [On-Schema Change](https://docs.getdbt.com/docs/build/incremental-models#what-if-the-columns-of-my-incremental-model-change) strategy in dbt-hive:\n\n| Strategy | ACID Table | Iceberg Table |\n|------|------|---------|\n| ignore (default) | Supported | Supported |\n| fail | Supported | Supported |\n| append_new_columns | Adds new columns | Adds new columns |\n| sync_all_columns | Adds new columns and updates datatypes but doesn't remove existing columns | Adds new columns, updates datatypes and removes existing columns | \n\n### Tests Coverage\n\n#### Functional Tests\n| Name | Base | Iceberg |\n|------|------|---------|\n|Materialization: View | Yes | N/A |\n|Materialization: Table| Yes | Yes |\n|Materialization: Table with Partitions | Yes | Yes |\n|Materialization: Incremental - Append | Yes | Yes|\n|Materialization: Incremental - Append with Partitions | Yes | Yes|\n|Materialization: Incremental - Insert+Overwrite with Partitions | Yes | No |\n|Materialization: Incremental - Merge | No | No |\n|Materialization: Ephemeral | No | No |\n|Seeds | Yes | Yes |\n|Tests | Yes | Yes |\n|Snapshots | No | No |\n|Documentation | Yes | No |\n|Authentication: LDAP | Yes | Yes |\n|Authentication: Kerberos | Yes | Yes |\n\n**Note**: Kerberos is only qualified on Unix platform.\n",
"bugtrack_url": null,
"license": null,
"summary": "The Hive adapter plugin for dbt",
"version": "1.7.0",
"project_urls": {
"Homepage": "https://github.com/cloudera/dbt-hive"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "696d61431e0c2b5e2bf589982773e0440b192207a9271f1b40620c3701a432c7",
"md5": "d40f42c341165721901fedf8151818dc",
"sha256": "0bae22ae239f1230241037d784e7b0b235cf9db07e0805eb132aca44549656f0"
},
"downloads": -1,
"filename": "dbt_hive-1.7.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d40f42c341165721901fedf8151818dc",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 54186,
"upload_time": "2024-05-09T19:13:30",
"upload_time_iso_8601": "2024-05-09T19:13:30.861300Z",
"url": "https://files.pythonhosted.org/packages/69/6d/61431e0c2b5e2bf589982773e0440b192207a9271f1b40620c3701a432c7/dbt_hive-1.7.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "0e7c714020ee12122df3103735b082ff75a3a3a98b7f670e338bf04f8355dc44",
"md5": "970ae0059cdb439cbff50aefcc9e5f16",
"sha256": "1170e18c9b1c6fd2c7289977830aebc4b5a60bf32533c2b110256d43d6c694e4"
},
"downloads": -1,
"filename": "dbt_hive-1.7.0.tar.gz",
"has_sig": false,
"md5_digest": "970ae0059cdb439cbff50aefcc9e5f16",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 33902,
"upload_time": "2024-05-09T19:13:34",
"upload_time_iso_8601": "2024-05-09T19:13:34.664617Z",
"url": "https://files.pythonhosted.org/packages/0e/7c/714020ee12122df3103735b082ff75a3a3a98b7f670e338bf04f8355dc44/dbt_hive-1.7.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-09 19:13:34",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cloudera",
"github_project": "dbt-hive",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"tox": true,
"lcname": "dbt-hive"
}
JSON
