| Name | dbt-bq-sourcegen JSON |
| Version |
0.1.1
JSON |
| download |
| home_page | None |
| Summary | Create or update dbt source YAML from BigQuery |
| upload_time | 2025-07-11 04:37:32 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.11 |
| license | None |
| keywords |
bigquery
dbt
source
yaml
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# dbt-bq-sourcegen
[](https://opensource.org/licenses/MIT)
[](https://www.python.org/downloads/)
Create or update dbt source YAML from BigQuery.
## Overview
`dbt-bq-sourcegen` automatically generates dbt source YAML files from BigQuery schemas, preserving existing configurations during updates.
It supports table filtering with wildcard patterns and maintains YAML formatting/comments.
This tool streamlines the process of keeping dbt source definitions in sync with your BigQuery datasets.
## Installation
```bash
pip install git+https://github.com/K-Oxon/dbt-bq-sourcegen.git
```
## Usage
### Apply (create or update automatically)
```bash
dbt-bq-sourcegen apply \
--project-id your-project \
--schema your_dataset \
--output models/staging/your_dataset/src_your_dataset.yml
```
### With table filtering
```bash
# Only include tables matching pattern
dbt-bq-sourcegen apply \
--project-id your-project \
--schema your_dataset \
--table-pattern "stg_*" \
--output models/staging/your_dataset/src_your_dataset.yml
# Exclude specific tables
dbt-bq-sourcegen apply \
--project-id your-project \
--schema your_dataset \
--exclude "temp" \
--output models/staging/your_dataset/src_your_dataset.yml
```
## Options
- `--project-id`: Google Cloud project ID (required)
- `--schema`, `--dataset`: BigQuery schema/dataset name (required)
- `--output`: Output YAML file path (required)
- `--table-pattern`: Table name pattern (e.g., 'stg_*')
- `--exclude`: Exclude tables containing this string
## Features
- Automatically generates dbt source YAML from BigQuery schema
- Updates existing source YAML files while preserving custom configurations
- Supports table filtering with wildcard patterns
- Preserves YAML formatting and comments
- Pure Python implementation with clean separation of concerns
## Development
```bash
# Install development dependencies
uv sync
# Run tests
uv run pytest
# Format code
uv run ruff format src/
```
Raw data
{
"_id": null,
"home_page": null,
"name": "dbt-bq-sourcegen",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.11",
"maintainer_email": null,
"keywords": "bigquery, dbt, source, yaml",
"author": null,
"author_email": "K-Oxon <ko1011qfp@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/7a/5d/2bb822f634eb70e16cfe1f0734a7ec6c6f1d99de8f5179a636da776b2266/dbt_bq_sourcegen-0.1.1.tar.gz",
"platform": null,
"description": "# dbt-bq-sourcegen\n\n[](https://opensource.org/licenses/MIT)\n[](https://www.python.org/downloads/)\n\nCreate or update dbt source YAML from BigQuery.\n\n## Overview\n\n`dbt-bq-sourcegen` automatically generates dbt source YAML files from BigQuery schemas, preserving existing configurations during updates.\nIt supports table filtering with wildcard patterns and maintains YAML formatting/comments.\nThis tool streamlines the process of keeping dbt source definitions in sync with your BigQuery datasets.\n\n## Installation\n\n```bash\npip install git+https://github.com/K-Oxon/dbt-bq-sourcegen.git\n```\n\n## Usage\n\n### Apply (create or update automatically)\n\n```bash\ndbt-bq-sourcegen apply \\\n --project-id your-project \\\n --schema your_dataset \\\n --output models/staging/your_dataset/src_your_dataset.yml\n```\n\n### With table filtering\n\n```bash\n# Only include tables matching pattern\ndbt-bq-sourcegen apply \\\n --project-id your-project \\\n --schema your_dataset \\\n --table-pattern \"stg_*\" \\\n --output models/staging/your_dataset/src_your_dataset.yml\n\n# Exclude specific tables\ndbt-bq-sourcegen apply \\\n --project-id your-project \\\n --schema your_dataset \\\n --exclude \"temp\" \\\n --output models/staging/your_dataset/src_your_dataset.yml\n```\n\n## Options\n\n- `--project-id`: Google Cloud project ID (required)\n- `--schema`, `--dataset`: BigQuery schema/dataset name (required)\n- `--output`: Output YAML file path (required)\n- `--table-pattern`: Table name pattern (e.g., 'stg_*')\n- `--exclude`: Exclude tables containing this string\n\n## Features\n\n- Automatically generates dbt source YAML from BigQuery schema\n- Updates existing source YAML files while preserving custom configurations\n- Supports table filtering with wildcard patterns\n- Preserves YAML formatting and comments\n- Pure Python implementation with clean separation of concerns\n\n## Development\n\n```bash\n# Install development dependencies\nuv sync\n\n# Run tests\nuv run pytest\n\n# Format code\nuv run ruff format src/\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "Create or update dbt source YAML from BigQuery",
"version": "0.1.1",
"project_urls": {
"Homepage": "https://github.com/K-Oxon/dbt-bq-sourcegen",
"Issues": "https://github.com/K-Oxon/dbt-bq-sourcegen/issues",
"Repository": "https://github.com/K-Oxon/dbt-bq-sourcegen"
},
"split_keywords": [
"bigquery",
" dbt",
" source",
" yaml"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "eb237e798d800b20a448dab49625958d339a773dadafea408e875ea6d5e36995",
"md5": "284290a66e39950b2a05721789564ba3",
"sha256": "17cfd394565dd7457892e2307a354c10ea349d1421bb4750474f9733c7459354"
},
"downloads": -1,
"filename": "dbt_bq_sourcegen-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "284290a66e39950b2a05721789564ba3",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11",
"size": 14573,
"upload_time": "2025-07-11T04:37:31",
"upload_time_iso_8601": "2025-07-11T04:37:31.241520Z",
"url": "https://files.pythonhosted.org/packages/eb/23/7e798d800b20a448dab49625958d339a773dadafea408e875ea6d5e36995/dbt_bq_sourcegen-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "7a5d2bb822f634eb70e16cfe1f0734a7ec6c6f1d99de8f5179a636da776b2266",
"md5": "b62331d4b38948472c49d05a5c911a82",
"sha256": "a38c29fd6fe60f05f811d789e5c8b878f4be77c5b06b434125da3cb440b86653"
},
"downloads": -1,
"filename": "dbt_bq_sourcegen-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "b62331d4b38948472c49d05a5c911a82",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11",
"size": 60665,
"upload_time": "2025-07-11T04:37:32",
"upload_time_iso_8601": "2025-07-11T04:37:32.677797Z",
"url": "https://files.pythonhosted.org/packages/7a/5d/2bb822f634eb70e16cfe1f0734a7ec6c6f1d99de8f5179a636da776b2266/dbt_bq_sourcegen-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-11 04:37:32",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "K-Oxon",
"github_project": "dbt-bq-sourcegen",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "dbt-bq-sourcegen"
}
