# diqu
[](https://diqu.iflambda.com/latest/)
[](https://pypi.org/project/diqu/)
[](https://opensource.org/license/apache-2-0/)
[](https://www.python.org)
[](https://codecov.io/gh/infinitelambda/diqu)
Automate and streamline the alerting/ notification process for dbt test results using this versatile CLI companion tool. Receive detailed alerts & test metadata seamlessly on various platforms, promoting improved collaboration on dbt project issues 🐞🚀.
## Who is this for
This tool is designed for individuals or teams seeking to automate the management of their dbt project issues (test warnings, errors... etc) outside the dbt environment.
## Features
- Automated alerts and notifications based on recorded dbt test results.
- Built-in support for dq-tools and custom query input.
- Auto-labels `deprecated` tests for quick & easy identification.
- Creates and updates Jira tickets with the latest tests' metadata.
- Sends succinct and informative messages to a dedicated Slack channel.
> **dbt alert rant 🟢 🟡 🔴 ⚫** </br></br>
> _"Run errors are red,_</br>
> _Test warnings are yellow,_</br>
> _Where's my bug alert,_</br>
> _And joint bugfix workflow?"_</br></br>
> _-- by `diqu` OG Contributors_ </br>
<img src="./docs/assets/img/diqu_concept.jpeg" alt="diqu Concept"> <!-- markdownlint-disable no-inline-html -->
### Supported Modules
- Sources (DWH connections)
- Snowflake
- CSV file
- Package (parsing dbt test results)
- [](https://hub.getdbt.com/infinitelambda/dq_tools)
- Custom query
- Alert Modules (alert/ notification)
- Jira
- Slack
## Installation
```bash
pip install diqu [--upgrade]
```
📓 _NOTE_: The required Data Warehouse (DWH) module should already be installed if you are using `diqu` in a working dbt project. If not, please perform additional steps to install these DWH modules.
For example, if you're using Snowflake:
```bash
pip install "snowflake-connector-python[pandas]"
pip install "snowflake-connector-python[secure-local-storage]"
```
## Usage
<details> <!-- markdownlint-disable no-inline-html -->
<summary>preflight</summary>
```bash
# define the query params
export ISSUE_DEPRECATED_WINDOW_IN_DAYS=your_issue_deprecation_time_in_day, default to "3"
export ISSUE_UPDATE_WINDOW_IN_DAYS=your_issue_historical_data_update_window_in_days, default to "14"
# build dq-tools log table
dbt run -s dq_tools
```
</details>
```bash
diqu alert --to slack --to jira
```
```log
04:33:17 diqu: INFO - Run with diqu==1.0.0 🏃
04:33:19 diqu: INFO - Using dbt project at: /path/to/dbt/project
04:33:19 diqu: INFO - Using dbt profiles.yml at: ~/.dbt
04:33:19 diqu: INFO - Using snowflake connection
04:33:19 diqu: INFO - Looking for the query in: ./dq_tools__get_test_results.sql
04:33:23 diqu: INFO - Alerting to: SLACK
04:33:23 diqu: INFO - ✅ Done > Slack
04:33:23 diqu: INFO - Alerting to: JIRA
04:33:23 diqu: INFO - ✅ Done > JIRA
```
## Alert Modules Configurations
### Slack
- Use the environment variables to configure the Slack Channel:
```bash
export SLACK_TOKEN=your_token
export SLACK_CHANNEL=your_channel_name
diqu alert --to slack
```
### Jira Board
- Use the environment variables to configure the JIRA Board:
```bash
export JIRA_SERVER=your_jira_server e.g. https://your_value.atlassian.net/
export JIRA_AUTH_USER=your_service_account e.g. dqt_user@your_value.com
export JIRA_AUTH_PASSWORD=your_service_token e.g. ATATTxxxxx
export JIRA_PROJECT_ID=your_project_id e.g. 106413
export JIRA_ISSUE_TYPE=your_issue_type, default to "Bug"
export JIRA_OPEN_ISSUES_FILTER_BY_SUMMARY=your_issue_filter_on_title, default to "dq-tools"
diqu alert --to jira
```
📖 For more details, please visit [the documentation site](https://diqu.iflambda.com/latest/).
## Quick Demo
[](https://www.loom.com/share/8d970dfe333c450f8f6d3859458cac99?sid=b5542b91-ca6d-42a9-b80f-55f85674ec21)
## How to Contribute
This Auto Alert (`diqu`) tool is an open-source software. Whether you are a seasoned open-source contributor or a first-time committer, we welcome and encourage you to contribute code, documentation, ideas, or problem statements to this project.
👉 See [CONTRIBUTING guideline](https://diqu.iflambda.com/latest/nav/dev/contributing.html) for more details or check out [CONTRIBUTING.md](./CONTRIBUTING.md)
🌟 And then, kudos to **our beloved Contributors**:
<a href="https://github.com/infinitelambda/diqu/graphs/contributors">
<img src="https://contrib.rocks/image?repo=infinitelambda/diqu" />
</a>
## About Infinite Lambda
Infinite Lambda is a cloud and data consultancy. We build strategies, help organizations implement them, and pass on the expertise to look after the infrastructure.
We are an Elite Snowflake Partner, a Platinum dbt Partner, and two-times Fivetran Innovation Partner of the Year for EMEA.
Naturally, we love exploring innovative solutions and sharing knowledge, so go ahead and:
🔧 Take a look around our [Git](https://github.com/infinitelambda)
✏️ Browse our [tech blog](https://infinitelambda.com/category/tech-blog/)
We are also chatty, so:
👀 Follow us on [LinkedIn](https://www.linkedin.com/company/infinite-lambda/)
👋🏼 Or just [get in touch](https://infinitelambda.com/contacts/)
[<img src="https://raw.githubusercontent.com/infinitelambda/cdn/1.0.0/general/images/GitHub-About-Section-1080x1080.png" alt="About IL" width="500">](https://infinitelambda.com/)
Raw data
{
"_id": null,
"home_page": "https://infinitelambda.com",
"name": "diqu",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9,<3.13",
"maintainer_email": "",
"keywords": "packaging,poetry,data-quality,alert,notification,collaboration,agility,flake8,markdown,lint",
"author": "Infinite Lambda",
"author_email": "contact@infinitelambda.com",
"download_url": "https://files.pythonhosted.org/packages/f4/a6/8ad9f750045885836786d5d54c9c592df921c387d03dfa912192c822303d/diqu-0.1.2.tar.gz",
"platform": null,
"description": "# diqu\n\n[](https://diqu.iflambda.com/latest/)\n[](https://pypi.org/project/diqu/)\n[](https://opensource.org/license/apache-2-0/)\n\n[](https://www.python.org)\n[](https://codecov.io/gh/infinitelambda/diqu)\n\nAutomate and streamline the alerting/ notification process for dbt test results using this versatile CLI companion tool. Receive detailed alerts & test metadata seamlessly on various platforms, promoting improved collaboration on dbt project issues \ud83d\udc1e\ud83d\ude80.\n\n## Who is this for\n\nThis tool is designed for individuals or teams seeking to automate the management of their dbt project issues (test warnings, errors... etc) outside the dbt environment.\n\n## Features\n\n- Automated alerts and notifications based on recorded dbt test results.\n- Built-in support for dq-tools and custom query input.\n- Auto-labels `deprecated` tests for quick & easy identification.\n- Creates and updates Jira tickets with the latest tests' metadata.\n- Sends succinct and informative messages to a dedicated Slack channel.\n\n> **dbt alert rant \ud83d\udfe2 \ud83d\udfe1 \ud83d\udd34 \u26ab** </br></br>\n> _\"Run errors are red,_</br>\n> _Test warnings are yellow,_</br>\n> _Where's my bug alert,_</br>\n> _And joint bugfix workflow?\"_</br></br>\n> _-- by `diqu` OG Contributors_ </br>\n\n<img src=\"./docs/assets/img/diqu_concept.jpeg\" alt=\"diqu Concept\"> <!-- markdownlint-disable no-inline-html -->\n\n### Supported Modules\n\n- Sources (DWH connections)\n - Snowflake\n - CSV file\n- Package (parsing dbt test results)\n - [](https://hub.getdbt.com/infinitelambda/dq_tools)\n - Custom query\n- Alert Modules (alert/ notification)\n - Jira\n - Slack\n\n## Installation\n\n```bash\npip install diqu [--upgrade]\n```\n\n\ud83d\udcd3 _NOTE_: The required Data Warehouse (DWH) module should already be installed if you are using `diqu` in a working dbt project. If not, please perform additional steps to install these DWH modules.\nFor example, if you're using Snowflake:\n\n```bash\npip install \"snowflake-connector-python[pandas]\"\npip install \"snowflake-connector-python[secure-local-storage]\"\n```\n\n## Usage\n\n<details> <!-- markdownlint-disable no-inline-html -->\n <summary>preflight</summary>\n\n ```bash\n # define the query params\n export ISSUE_DEPRECATED_WINDOW_IN_DAYS=your_issue_deprecation_time_in_day, default to \"3\"\n export ISSUE_UPDATE_WINDOW_IN_DAYS=your_issue_historical_data_update_window_in_days, default to \"14\"\n\n # build dq-tools log table\n dbt run -s dq_tools\n ```\n\n</details>\n\n ```bash\n diqu alert --to slack --to jira\n ```\n\n```log\n04:33:17 diqu: INFO - Run with diqu==1.0.0 \ud83c\udfc3\n04:33:19 diqu: INFO - Using dbt project at: /path/to/dbt/project\n04:33:19 diqu: INFO - Using dbt profiles.yml at: ~/.dbt\n04:33:19 diqu: INFO - Using snowflake connection\n04:33:19 diqu: INFO - Looking for the query in: ./dq_tools__get_test_results.sql\n04:33:23 diqu: INFO - Alerting to: SLACK\n04:33:23 diqu: INFO - \u2705 Done > Slack\n04:33:23 diqu: INFO - Alerting to: JIRA\n04:33:23 diqu: INFO - \u2705 Done > JIRA\n```\n\n## Alert Modules Configurations\n\n### Slack\n\n- Use the environment variables to configure the Slack Channel:\n\n ```bash\n export SLACK_TOKEN=your_token\n export SLACK_CHANNEL=your_channel_name\n \n diqu alert --to slack\n ```\n\n### Jira Board\n\n- Use the environment variables to configure the JIRA Board:\n\n ```bash\n export JIRA_SERVER=your_jira_server e.g. https://your_value.atlassian.net/\n export JIRA_AUTH_USER=your_service_account e.g. dqt_user@your_value.com\n export JIRA_AUTH_PASSWORD=your_service_token e.g. ATATTxxxxx\n export JIRA_PROJECT_ID=your_project_id e.g. 106413\n export JIRA_ISSUE_TYPE=your_issue_type, default to \"Bug\"\n export JIRA_OPEN_ISSUES_FILTER_BY_SUMMARY=your_issue_filter_on_title, default to \"dq-tools\"\n\n diqu alert --to jira\n ```\n\n\ud83d\udcd6 For more details, please visit [the documentation site](https://diqu.iflambda.com/latest/).\n\n## Quick Demo\n\n[](https://www.loom.com/share/8d970dfe333c450f8f6d3859458cac99?sid=b5542b91-ca6d-42a9-b80f-55f85674ec21)\n\n## How to Contribute\n\nThis Auto Alert (`diqu`) tool is an open-source software. Whether you are a seasoned open-source contributor or a first-time committer, we welcome and encourage you to contribute code, documentation, ideas, or problem statements to this project.\n\n\ud83d\udc49 See [CONTRIBUTING guideline](https://diqu.iflambda.com/latest/nav/dev/contributing.html) for more details or check out [CONTRIBUTING.md](./CONTRIBUTING.md)\n\n\ud83c\udf1f And then, kudos to **our beloved Contributors**:\n\n<a href=\"https://github.com/infinitelambda/diqu/graphs/contributors\">\n <img src=\"https://contrib.rocks/image?repo=infinitelambda/diqu\" />\n</a>\n\n## About Infinite Lambda\n\nInfinite Lambda is a cloud and data consultancy. We build strategies, help organizations implement them, and pass on the expertise to look after the infrastructure.\n\nWe are an Elite Snowflake Partner, a Platinum dbt Partner, and two-times Fivetran Innovation Partner of the Year for EMEA.\n\nNaturally, we love exploring innovative solutions and sharing knowledge, so go ahead and:\n\n\ud83d\udd27 Take a look around our [Git](https://github.com/infinitelambda)\n\n\u270f\ufe0f Browse our [tech blog](https://infinitelambda.com/category/tech-blog/)\n\nWe are also chatty, so:\n\n\ud83d\udc40 Follow us on [LinkedIn](https://www.linkedin.com/company/infinite-lambda/)\n\n\ud83d\udc4b\ud83c\udffc Or just [get in touch](https://infinitelambda.com/contacts/)\n\n[<img src=\"https://raw.githubusercontent.com/infinitelambda/cdn/1.0.0/general/images/GitHub-About-Section-1080x1080.png\" alt=\"About IL\" width=\"500\">](https://infinitelambda.com/)\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Data Quality CLI for the Auto-Alerts",
"version": "0.1.2",
"project_urls": {
"Documentation": "https://diqu.iflambda.com/latest/",
"Homepage": "https://infinitelambda.com",
"Repository": "https://github.com/infinitelambda/diqu"
},
"split_keywords": [
"packaging",
"poetry",
"data-quality",
"alert",
"notification",
"collaboration",
"agility",
"flake8",
"markdown",
"lint"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0a95a1b7eada467792ab63c1f5c278fb168fec10cfcd2191231e07b01c61a75e",
"md5": "7c5440316336b3283d7117de23c9152f",
"sha256": "640c013784eff4fde9b547387ea88b0e76e2c19884840133b4befd1917765f93"
},
"downloads": -1,
"filename": "diqu-0.1.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7c5440316336b3283d7117de23c9152f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9,<3.13",
"size": 30350,
"upload_time": "2024-03-04T07:49:29",
"upload_time_iso_8601": "2024-03-04T07:49:29.534381Z",
"url": "https://files.pythonhosted.org/packages/0a/95/a1b7eada467792ab63c1f5c278fb168fec10cfcd2191231e07b01c61a75e/diqu-0.1.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f4a68ad9f750045885836786d5d54c9c592df921c387d03dfa912192c822303d",
"md5": "dd19ba6223618b370174497ac4efae2d",
"sha256": "8e067fb980a5c6bd1dc855de57c9fb2e7fee9ef4259fff24ed5ceef3e0101ec9"
},
"downloads": -1,
"filename": "diqu-0.1.2.tar.gz",
"has_sig": false,
"md5_digest": "dd19ba6223618b370174497ac4efae2d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9,<3.13",
"size": 22810,
"upload_time": "2024-03-04T07:49:33",
"upload_time_iso_8601": "2024-03-04T07:49:33.026983Z",
"url": "https://files.pythonhosted.org/packages/f4/a6/8ad9f750045885836786d5d54c9c592df921c387d03dfa912192c822303d/diqu-0.1.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-04 07:49:33",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "infinitelambda",
"github_project": "diqu",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "diqu"
}
JSON
