# dbt-cloud-jobs
[](https://img.shields.io/pypi/v/dbt-cloud-jobs)
[](https://opensource.org/licenses/MIT)
Version control your dbt Cloud jobs with YML.
# Installation
```bash
pip install dbt-cloud-jobs
```
# Quickstart
1. Create an API token in dbt Cloud:
- [Generate a service account token](https://docs.getdbt.com/docs/dbt-cloud-apis/service-tokens#generate-service-account-tokens), make sure to grant the `Jobs Admin` permission set.
- If you do not have access to create a service account token you can create a [User API token](https://docs.getdbt.com/docs/dbt-cloud-apis/user-tokens). Note that the service account method is preferred.
1. Set an environment variable with the value of the token:
```bash
export DBT_API_TOKEN="<VALUE_FROM_PREVIOUS_STEP>"
```
1. Set an environment variable for the region where your dbt Cloud account is hosted. The value must be one of "AU", "Europe" or "US" (see docs [here](https://docs.getdbt.com/dbt-cloud/api-v2#/)):
```bash
export DBT_CLOUD_REGION="<REGION>"
```
1. Import your existing dbt Cloud jobs:
```bash
dbt_cloud_jobs --import --account-id 123456 --file dbt_cloud_jobs.yml
```
1. Edit the definition of your jobs in `dbt_cloud_jobs.yml`.
1. Sync the updated definitions to dbt Cloud:
```bash
dbt_cloud_jobs --sync --file dbt_cloud_jobs.yml
```
# Recommended usage in CI/CD
## CI
In CI `dbt-cloud-jobs` should be used to verify that the provided YML file is valid. For example:
```yaml
- name: Install dbt_cloud_jobs
run: pip install dbt-cloud-jobs
- name: Validate `dbt_cloud_jobs.yml`
run: dbt_cloud_jobs --validate --file dbt_cloud_jobs.yml
```
## CD
In CD `dbt-cloud-jobs` should be used to sync the provided YML file to dbt Cloud. For example:
```yaml
- name: Install dbt_cloud_jobs
run: pip install dbt-cloud-jobs
- name: Sync `dbt_cloud_jobs.yml`
run: dbt_cloud_jobs --sync --file dbt_cloud_jobs.yml
```
# Limitations/Warnings
* Service account tokens are created at the account level. This means that if you have multiple dbt Cloud accounts you will need to create different `dbt_cloud_jobs.yml` files for each account. If you try to use `dbt-cloud-jobs` with a file that contains multiple `account_id` values, an error will be raised.
* ⚠️ `dbt_cloud_jobs` expects to "own" all the jobs in the projects where it is used. This means that if you are running `dbt_cloud_jobs --sync --file dbt_cloud_jobs.yml` for the first time in a project, any pre-existing jobs not present in your `dbt_cloud_jobs.yml` file will be deleted.
# Development
To setup your development environment, fork this repository and run:
```bash
poetry shell
poetry install
```
Set the following environment variables:
```bash
export DBT_ACCOUNT_ID=<DBT_ACCOUNT_ID>
export DBT_CLOUD_REGION="<DBT_CLOUD_REGION>"
export DBT_ENVIRONMENT_ID=<DBT_ENVIRONMENT_ID>
export DBT_PROJECT_ID=<DBT_PROJECT_ID>
export DBT_API_TOKEN="<DBT_API_TOKEN>"
```
It is highly recommended that a dedicated dbt Cloud environment be used for development.
All tests can be run via:
```bash
make test
```
# Release
Trigger the `Publish to PyPi` workflow, inputting the version to publish to PyPi. This workflow will:
- Publish the version to [PyPi](https://pypi.org/project/dbt-cloud-jobs/).
- Tag the HEAD commit of the `main` branches (tags visible [here](https://github.com/pgoslatara/dbt-cloud-jobs/tags)).
- Create a release (releases visible [here](https://github.com/pgoslatara/dbt-cloud-jobs/releases)).
Raw data
{
"_id": null,
"home_page": "https://github.com/pgoslatara/dbt-cloud-jobs",
"name": "dbt-cloud-jobs",
"maintainer": "Padraic Slattery",
"docs_url": null,
"requires_python": ">=3.8,<3.13",
"maintainer_email": "pgoslatara@gmail.com",
"keywords": "python,cli,dbt,CI/CD",
"author": "Padraic Slattery",
"author_email": "pgoslatara@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/4e/6c/930d655c0c1b4869c7d5cf520a9824e6921cec356142396de4b7abc61be1/dbt_cloud_jobs-0.0.4.tar.gz",
"platform": null,
"description": "# dbt-cloud-jobs\n\n[](https://img.shields.io/pypi/v/dbt-cloud-jobs)\n\n\n\n[](https://opensource.org/licenses/MIT)\n\nVersion control your dbt Cloud jobs with YML.\n\n# Installation\n\n```bash\npip install dbt-cloud-jobs\n```\n\n# Quickstart\n\n1. Create an API token in dbt Cloud:\n\n - [Generate a service account token](https://docs.getdbt.com/docs/dbt-cloud-apis/service-tokens#generate-service-account-tokens), make sure to grant the `Jobs Admin` permission set.\n - If you do not have access to create a service account token you can create a [User API token](https://docs.getdbt.com/docs/dbt-cloud-apis/user-tokens). Note that the service account method is preferred.\n\n1. Set an environment variable with the value of the token:\n\n ```bash\n export DBT_API_TOKEN=\"<VALUE_FROM_PREVIOUS_STEP>\"\n ```\n\n1. Set an environment variable for the region where your dbt Cloud account is hosted. The value must be one of \"AU\", \"Europe\" or \"US\" (see docs [here](https://docs.getdbt.com/dbt-cloud/api-v2#/)):\n\n ```bash\n export DBT_CLOUD_REGION=\"<REGION>\"\n ```\n\n1. Import your existing dbt Cloud jobs:\n\n ```bash\n dbt_cloud_jobs --import --account-id 123456 --file dbt_cloud_jobs.yml\n ```\n\n1. Edit the definition of your jobs in `dbt_cloud_jobs.yml`.\n\n1. Sync the updated definitions to dbt Cloud:\n\n ```bash\n dbt_cloud_jobs --sync --file dbt_cloud_jobs.yml\n ```\n\n# Recommended usage in CI/CD\n\n## CI\n\nIn CI `dbt-cloud-jobs` should be used to verify that the provided YML file is valid. For example:\n\n```yaml\n - name: Install dbt_cloud_jobs\n run: pip install dbt-cloud-jobs\n\n - name: Validate `dbt_cloud_jobs.yml`\n run: dbt_cloud_jobs --validate --file dbt_cloud_jobs.yml\n```\n\n## CD\n\nIn CD `dbt-cloud-jobs` should be used to sync the provided YML file to dbt Cloud. For example:\n\n```yaml\n - name: Install dbt_cloud_jobs\n run: pip install dbt-cloud-jobs\n\n - name: Sync `dbt_cloud_jobs.yml`\n run: dbt_cloud_jobs --sync --file dbt_cloud_jobs.yml\n```\n\n# Limitations/Warnings\n\n* Service account tokens are created at the account level. This means that if you have multiple dbt Cloud accounts you will need to create different `dbt_cloud_jobs.yml` files for each account. If you try to use `dbt-cloud-jobs` with a file that contains multiple `account_id` values, an error will be raised.\n\n* \u26a0\ufe0f `dbt_cloud_jobs` expects to \"own\" all the jobs in the projects where it is used. This means that if you are running `dbt_cloud_jobs --sync --file dbt_cloud_jobs.yml` for the first time in a project, any pre-existing jobs not present in your `dbt_cloud_jobs.yml` file will be deleted.\n\n# Development\n\nTo setup your development environment, fork this repository and run:\n\n```bash\npoetry shell\npoetry install\n```\n\nSet the following environment variables:\n```bash\nexport DBT_ACCOUNT_ID=<DBT_ACCOUNT_ID>\nexport DBT_CLOUD_REGION=\"<DBT_CLOUD_REGION>\"\nexport DBT_ENVIRONMENT_ID=<DBT_ENVIRONMENT_ID>\nexport DBT_PROJECT_ID=<DBT_PROJECT_ID>\nexport DBT_API_TOKEN=\"<DBT_API_TOKEN>\"\n```\nIt is highly recommended that a dedicated dbt Cloud environment be used for development.\n\nAll tests can be run via:\n```bash\nmake test\n```\n\n# Release\n\nTrigger the `Publish to PyPi` workflow, inputting the version to publish to PyPi. This workflow will:\n\n- Publish the version to [PyPi](https://pypi.org/project/dbt-cloud-jobs/).\n- Tag the HEAD commit of the `main` branches (tags visible [here](https://github.com/pgoslatara/dbt-cloud-jobs/tags)).\n- Create a release (releases visible [here](https://github.com/pgoslatara/dbt-cloud-jobs/releases)).\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Version control your dbt Cloud jobs with YML.",
"version": "0.0.4",
"project_urls": {
"Homepage": "https://github.com/pgoslatara/dbt-cloud-jobs",
"Repository": "https://github.com/pgoslatara/dbt-cloud-jobs"
},
"split_keywords": [
"python",
"cli",
"dbt",
"ci/cd"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a6240ef7abaddf9de0409811350531150ee32affff8b432b24995cab280f5331",
"md5": "fe9ccb41752c0af2d581f826028cede3",
"sha256": "2e7bf03e3ae03f98f26d04bc9946a4c6a9bdf5f2dd2af400d4337f260ee199d2"
},
"downloads": -1,
"filename": "dbt_cloud_jobs-0.0.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "fe9ccb41752c0af2d581f826028cede3",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<3.13",
"size": 13025,
"upload_time": "2024-03-05T09:17:13",
"upload_time_iso_8601": "2024-03-05T09:17:13.489214Z",
"url": "https://files.pythonhosted.org/packages/a6/24/0ef7abaddf9de0409811350531150ee32affff8b432b24995cab280f5331/dbt_cloud_jobs-0.0.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4e6c930d655c0c1b4869c7d5cf520a9824e6921cec356142396de4b7abc61be1",
"md5": "f4750b96c4120b3d80fa26d1afbd0c11",
"sha256": "701117d415667a09d1aa929c079a10854a314a4b6562a59091c8e513379c3b83"
},
"downloads": -1,
"filename": "dbt_cloud_jobs-0.0.4.tar.gz",
"has_sig": false,
"md5_digest": "f4750b96c4120b3d80fa26d1afbd0c11",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<3.13",
"size": 11373,
"upload_time": "2024-03-05T09:17:15",
"upload_time_iso_8601": "2024-03-05T09:17:15.241300Z",
"url": "https://files.pythonhosted.org/packages/4e/6c/930d655c0c1b4869c7d5cf520a9824e6921cec356142396de4b7abc61be1/dbt_cloud_jobs-0.0.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-05 09:17:15",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "pgoslatara",
"github_project": "dbt-cloud-jobs",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "dbt-cloud-jobs"
}
JSON
