# diqu
[](https://diqu.iflambda.com/latest/)
[](https://pypi.org/project/diqu/)
[](https://opensource.org/license/apache-2-0/)
[](https://www.python.org)
[](https://codecov.io/gh/infinitelambda/diqu)
Automate and streamline the alerting/ notification process for dbt test results using this versatile CLI companion tool. Receive detailed alerts & test metadata seamlessly on various platforms, promoting improved collaboration on dbt project issues 🐞🚀.
## Who is this for
This tool is designed for individuals or teams seeking to automate the management of their dbt project issues (test warnings, errors... etc) outside the dbt environment.
## Features
- Automated alerts and notifications based on recorded dbt test results.
- Built-in support for dq-tools and custom query input.
- Auto-labels `deprecated` tests for quick & easy identification.
- Creates and updates Jira tickets with the latest tests' metadata.
- Sends succinct and informative messages to a dedicated Slack channel.
> **dbt alert rant 🟢 🟡 🔴 ⚫** </br></br>
> _"Run errors are red,_</br>
> _Test warnings are yellow,_</br>
> _Where's my bug alert,_</br>
> _And joint bugfix workflow?"_</br></br>
> _-- by `diqu` OG Contributors_ </br>
<img src="./docs/assets/img/diqu_concept.jpeg" alt="diqu Concept"> <!-- markdownlint-disable no-inline-html -->
### Supported Modules
- Sources (DWH connections)
- Snowflake
- CSV file
- Package (parsing dbt test results)
- [](https://hub.getdbt.com/infinitelambda/dq_tools)
- Custom query
- Alert Modules (alert/ notification)
- Jira
- Slack
## Installation
```bash
pip install diqu [--upgrade]
```
📓 _NOTE_: The required Data Warehouse (DWH) module should already be installed if you are using `diqu` in a working dbt project. If not, please perform additional steps to install these DWH modules.
For example, if you're using Snowflake:
```bash
pip install "snowflake-connector-python[pandas]"
pip install "snowflake-connector-python[secure-local-storage]"
```
## Usage
<details> <!-- markdownlint-disable no-inline-html -->
<summary>preflight</summary>
```bash
# define the query params
export ISSUE_DEPRECATED_WINDOW_IN_DAYS=your_issue_deprecation_time_in_day, default to "3"
export ISSUE_UPDATE_WINDOW_IN_DAYS=your_issue_historical_data_update_window_in_days, default to "14"
# build dq-tools log table
dbt run -s dq_tools
```
</details>
```bash
diqu alert --to slack --to jira
```
```log
04:33:17 diqu: INFO - Run with diqu==1.0.0 🏃
04:33:19 diqu: INFO - Using dbt project at: /path/to/dbt/project
04:33:19 diqu: INFO - Using dbt profiles.yml at: ~/.dbt
04:33:19 diqu: INFO - Using snowflake connection
04:33:19 diqu: INFO - Looking for the query in: ./dq_tools__get_test_results.sql
04:33:23 diqu: INFO - Alerting to: SLACK
04:33:23 diqu: INFO - ✅ Done > Slack
04:33:23 diqu: INFO - Alerting to: JIRA
04:33:23 diqu: INFO - ✅ Done > JIRA
```
## Alert Modules Configurations
### Slack
- Use the environment variables to configure the Slack Channel:
```bash
export SLACK_TOKEN=your_token
export SLACK_CHANNEL=your_channel_name
diqu alert --to slack
```
### Jira Board
- Use the environment variables to configure the JIRA Board:
```bash
export JIRA_SERVER=your_jira_server e.g. https://your_value.atlassian.net/
export JIRA_AUTH_USER=your_service_account e.g. dqt_user@your_value.com
export JIRA_AUTH_PASSWORD=your_service_token e.g. ATATTxxxxx
export JIRA_PROJECT_ID=your_project_id e.g. 106413
export JIRA_ISSUE_TYPE=your_issue_type, default to "Bug"
export JIRA_OPEN_ISSUES_FILTER_BY_SUMMARY=your_issue_filter_on_title, default to "dq-tools"
diqu alert --to jira
```
📖 For more details, please visit [the documentation site](https://diqu.iflambda.com/latest/).
## Quick Demo
[](https://www.loom.com/share/8d970dfe333c450f8f6d3859458cac99?sid=b5542b91-ca6d-42a9-b80f-55f85674ec21)
## How to Contribute
This Auto Alert (`diqu`) tool is an open-source software. Whether you are a seasoned open-source contributor or a first-time committer, we welcome and encourage you to contribute code, documentation, ideas, or problem statements to this project.
👉 See [CONTRIBUTING guideline](https://diqu.iflambda.com/latest/nav/dev/contributing.html) for more details or check out [CONTRIBUTING.md](./CONTRIBUTING.md)
🌟 And then, kudos to **our beloved Contributors**:
<a href="https://github.com/infinitelambda/diqu/graphs/contributors">
<img src="https://contrib.rocks/image?repo=infinitelambda/diqu" />
</a>
## About Infinite Lambda
Infinite Lambda is a cloud and data consultancy. We build strategies, help organizations implement them, and pass on the expertise to look after the infrastructure.
We are an Elite Snowflake Partner, a Platinum dbt Partner, and two-times Fivetran Innovation Partner of the Year for EMEA.
Naturally, we love exploring innovative solutions and sharing knowledge, so go ahead and:
🔧 Take a look around our [Git](https://github.com/infinitelambda)
✏️ Browse our [tech blog](https://infinitelambda.com/category/tech-blog/)
We are also chatty, so:
👀 Follow us on [LinkedIn](https://www.linkedin.com/company/infinite-lambda/)
👋🏼 Or just [get in touch](https://infinitelambda.com/contacts/)
[<img src="https://raw.githubusercontent.com/infinitelambda/cdn/1.0.0/general/images/GitHub-About-Section-1080x1080.png" alt="About IL" width="500">](https://infinitelambda.com/)
Raw data
{
"_id": null,
"home_page": "https://infinitelambda.com",
"name": "diqu",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.13,>=3.9",
"maintainer_email": null,
"keywords": "packaging, poetry, data-quality, alert, notification, collaboration, agility, flake8, markdown, lint",
"author": "Infinite Lambda",
"author_email": "contact@infinitelambda.com",
"download_url": "https://files.pythonhosted.org/packages/83/10/8554c9aedd981e2adde2787ab51020461a2351b39a285f29aa0b75db3f3e/diqu-0.2.0.tar.gz",
"platform": null,
"description": "# diqu\n\n[](https://diqu.iflambda.com/latest/)\n[](https://pypi.org/project/diqu/)\n[](https://opensource.org/license/apache-2-0/)\n\n[](https://www.python.org)\n[](https://codecov.io/gh/infinitelambda/diqu)\n\nAutomate and streamline the alerting/ notification process for dbt test results using this versatile CLI companion tool. Receive detailed alerts & test metadata seamlessly on various platforms, promoting improved collaboration on dbt project issues \ud83d\udc1e\ud83d\ude80.\n\n## Who is this for\n\nThis tool is designed for individuals or teams seeking to automate the management of their dbt project issues (test warnings, errors... etc) outside the dbt environment.\n\n## Features\n\n- Automated alerts and notifications based on recorded dbt test results.\n- Built-in support for dq-tools and custom query input.\n- Auto-labels `deprecated` tests for quick & easy identification.\n- Creates and updates Jira tickets with the latest tests' metadata.\n- Sends succinct and informative messages to a dedicated Slack channel.\n\n> **dbt alert rant \ud83d\udfe2 \ud83d\udfe1 \ud83d\udd34 \u26ab** </br></br>\n> _\"Run errors are red,_</br>\n> _Test warnings are yellow,_</br>\n> _Where's my bug alert,_</br>\n> _And joint bugfix workflow?\"_</br></br>\n> _-- by `diqu` OG Contributors_ </br>\n\n<img src=\"./docs/assets/img/diqu_concept.jpeg\" alt=\"diqu Concept\"> <!-- markdownlint-disable no-inline-html -->\n\n### Supported Modules\n\n- Sources (DWH connections)\n - Snowflake\n - CSV file\n- Package (parsing dbt test results)\n - [](https://hub.getdbt.com/infinitelambda/dq_tools)\n - Custom query\n- Alert Modules (alert/ notification)\n - Jira\n - Slack\n\n## Installation\n\n```bash\npip install diqu [--upgrade]\n```\n\n\ud83d\udcd3 _NOTE_: The required Data Warehouse (DWH) module should already be installed if you are using `diqu` in a working dbt project. If not, please perform additional steps to install these DWH modules.\nFor example, if you're using Snowflake:\n\n```bash\npip install \"snowflake-connector-python[pandas]\"\npip install \"snowflake-connector-python[secure-local-storage]\"\n```\n\n## Usage\n\n<details> <!-- markdownlint-disable no-inline-html -->\n <summary>preflight</summary>\n\n ```bash\n # define the query params\n export ISSUE_DEPRECATED_WINDOW_IN_DAYS=your_issue_deprecation_time_in_day, default to \"3\"\n export ISSUE_UPDATE_WINDOW_IN_DAYS=your_issue_historical_data_update_window_in_days, default to \"14\"\n\n # build dq-tools log table\n dbt run -s dq_tools\n ```\n\n</details>\n\n ```bash\n diqu alert --to slack --to jira\n ```\n\n```log\n04:33:17 diqu: INFO - Run with diqu==1.0.0 \ud83c\udfc3\n04:33:19 diqu: INFO - Using dbt project at: /path/to/dbt/project\n04:33:19 diqu: INFO - Using dbt profiles.yml at: ~/.dbt\n04:33:19 diqu: INFO - Using snowflake connection\n04:33:19 diqu: INFO - Looking for the query in: ./dq_tools__get_test_results.sql\n04:33:23 diqu: INFO - Alerting to: SLACK\n04:33:23 diqu: INFO - \u2705 Done > Slack\n04:33:23 diqu: INFO - Alerting to: JIRA\n04:33:23 diqu: INFO - \u2705 Done > JIRA\n```\n\n## Alert Modules Configurations\n\n### Slack\n\n- Use the environment variables to configure the Slack Channel:\n\n ```bash\n export SLACK_TOKEN=your_token\n export SLACK_CHANNEL=your_channel_name\n \n diqu alert --to slack\n ```\n\n### Jira Board\n\n- Use the environment variables to configure the JIRA Board:\n\n ```bash\n export JIRA_SERVER=your_jira_server e.g. https://your_value.atlassian.net/\n export JIRA_AUTH_USER=your_service_account e.g. dqt_user@your_value.com\n export JIRA_AUTH_PASSWORD=your_service_token e.g. ATATTxxxxx\n export JIRA_PROJECT_ID=your_project_id e.g. 106413\n export JIRA_ISSUE_TYPE=your_issue_type, default to \"Bug\"\n export JIRA_OPEN_ISSUES_FILTER_BY_SUMMARY=your_issue_filter_on_title, default to \"dq-tools\"\n\n diqu alert --to jira\n ```\n\n\ud83d\udcd6 For more details, please visit [the documentation site](https://diqu.iflambda.com/latest/).\n\n## Quick Demo\n\n[](https://www.loom.com/share/8d970dfe333c450f8f6d3859458cac99?sid=b5542b91-ca6d-42a9-b80f-55f85674ec21)\n\n## How to Contribute\n\nThis Auto Alert (`diqu`) tool is an open-source software. Whether you are a seasoned open-source contributor or a first-time committer, we welcome and encourage you to contribute code, documentation, ideas, or problem statements to this project.\n\n\ud83d\udc49 See [CONTRIBUTING guideline](https://diqu.iflambda.com/latest/nav/dev/contributing.html) for more details or check out [CONTRIBUTING.md](./CONTRIBUTING.md)\n\n\ud83c\udf1f And then, kudos to **our beloved Contributors**:\n\n<a href=\"https://github.com/infinitelambda/diqu/graphs/contributors\">\n <img src=\"https://contrib.rocks/image?repo=infinitelambda/diqu\" />\n</a>\n\n## About Infinite Lambda\n\nInfinite Lambda is a cloud and data consultancy. We build strategies, help organizations implement them, and pass on the expertise to look after the infrastructure.\n\nWe are an Elite Snowflake Partner, a Platinum dbt Partner, and two-times Fivetran Innovation Partner of the Year for EMEA.\n\nNaturally, we love exploring innovative solutions and sharing knowledge, so go ahead and:\n\n\ud83d\udd27 Take a look around our [Git](https://github.com/infinitelambda)\n\n\u270f\ufe0f Browse our [tech blog](https://infinitelambda.com/category/tech-blog/)\n\nWe are also chatty, so:\n\n\ud83d\udc40 Follow us on [LinkedIn](https://www.linkedin.com/company/infinite-lambda/)\n\n\ud83d\udc4b\ud83c\udffc Or just [get in touch](https://infinitelambda.com/contacts/)\n\n[<img src=\"https://raw.githubusercontent.com/infinitelambda/cdn/1.0.0/general/images/GitHub-About-Section-1080x1080.png\" alt=\"About IL\" width=\"500\">](https://infinitelambda.com/)\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Data Quality CLI for the Auto-Alerts",
"version": "0.2.0",
"project_urls": {
"Documentation": "https://diqu.iflambda.com/latest/",
"Homepage": "https://infinitelambda.com",
"Repository": "https://github.com/infinitelambda/diqu"
},
"split_keywords": [
"packaging",
" poetry",
" data-quality",
" alert",
" notification",
" collaboration",
" agility",
" flake8",
" markdown",
" lint"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3c909a1b78bb94f4b98fd99b60c987777ec89ae4078582cc0cb7722357fcdced",
"md5": "7012c848da009488a1297274fcc54677",
"sha256": "1b6b12df8cf1a1e500b5f56ddb3c0a876d8c1f505132d370cc821f89fff24b0b"
},
"downloads": -1,
"filename": "diqu-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7012c848da009488a1297274fcc54677",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.13,>=3.9",
"size": 30557,
"upload_time": "2024-07-08T03:56:03",
"upload_time_iso_8601": "2024-07-08T03:56:03.234138Z",
"url": "https://files.pythonhosted.org/packages/3c/90/9a1b78bb94f4b98fd99b60c987777ec89ae4078582cc0cb7722357fcdced/diqu-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "83108554c9aedd981e2adde2787ab51020461a2351b39a285f29aa0b75db3f3e",
"md5": "055c3169c8c34b1d88cff8a8bbc59fda",
"sha256": "3f0689dbb1738c537c37710c5b596ff9afdebce9594426d20d6b7b7f2415802d"
},
"downloads": -1,
"filename": "diqu-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "055c3169c8c34b1d88cff8a8bbc59fda",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.13,>=3.9",
"size": 22984,
"upload_time": "2024-07-08T03:56:04",
"upload_time_iso_8601": "2024-07-08T03:56:04.543598Z",
"url": "https://files.pythonhosted.org/packages/83/10/8554c9aedd981e2adde2787ab51020461a2351b39a285f29aa0b75db3f3e/diqu-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-08 03:56:04",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "infinitelambda",
"github_project": "diqu",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "diqu"
}
JSON
