Concordia
<p align="center">
<em>Bring harmony to your data stack.</em>
</p>
<p align="center">
<a href="#what-is-concordia">What is it?</a> •
<a href="#how-it-works">How it Works</a> •
<a href="#key-features">Features</a> •
<a href="#getting-started">Getting Started</a> •
<a href="#usage">Usage</a> •
<a href="#configuration">Configuration</a>
</p>
What is Concordia?
Concordia is a command-line interface (CLI) tool that automates the creation and maintenance of Looker views, ensuring they are always in sync with your BigQuery data warehouse. It establishes your BigQuery/Dataform schema as the single source of truth and propagates its structure and documentation directly into your Looker project.
If you've ever had to:
- Manually create a new LookML view for every new table in your warehouse.
- Update a LookML dimension because a column name changed in BigQuery.
- Copy and paste column descriptions from a dbt/dataform model into a Looker view.
- Notice that the documentation in Looker is out of date with the real-world table.
...then Concordia is the tool for you.
## How it Works
Concordia operates on a simple, unidirectional data flow. It reads the metadata (column names, data types, descriptions) directly from your BigQuery tables and uses that information to generate clean, consistent, and documented LookML view files.
`Dataform/BigQuery (Source of Truth) -> Concordia -> Looker .view Files`
This ensures that your semantic layer in Looker is a perfect reflection of your transformation layer in the data warehouse, eliminating drift and manual effort.
## Key Features
Automated View Generation: Create a complete, well-structured LookML view from a BigQuery table with a single command.
- Documentation Sync: Automatically pulls column descriptions from BigQuery and populates the description tag in your LookML dimensions.
- Convention over Configuration: Uses smart naming conventions (e.g., for primary and foreign keys) to generate better LookML.
- Intelligent Defaults: Automatically adds a count measure, hides key fields, and creates a set for drill fields.
- Simple Configuration: A single concordia.yml file manages all project settings.
- Secure Authentication: Leverages existing Dataform credentials files or Google Application Default Credentials (ADC) so you don't have to manage new secrets.
## Getting Started
1. Installation
(Placeholder for installation instructions, e.g., pip install concordia-cli)
2. Initialization
Navigate to the root of your analytics repository and run: `concordia init`. This will create a concordia.yml file in your project. This is where you will configure the tool.
## Configuration
All configuration is handled in the concordia.yml file.
```yaml
# concordia.yml - Example Configuration
# BigQuery Connection Details
connection:
# (Recommended) Point to your Dataform credentials file.
dataform_credentials_file: './.df-credentials.json'
# If the file above is not found, Concordia falls back to Google ADC.
# The GCP project ID and location to target.
project_id: 'my-gcp-project'
location: 'europe-west2'
# The datasets to scan for tables.
datasets:
- 'marts'
- 'finance'
# Looker project configuration
looker:
project_path: './looker_project/' # Path to your local Looker git repo
views_path: 'views/base/base.view.lkml' # Path for generated base view
connection: 'bigquery-prod' # The name of your Looker connection
# Rules for how models and fields are generated
model_rules:
# Define how to identify PKs and FKs from column names
naming_conventions:
pk_suffix: '_pk'
fk_suffix: '_fk'
# Define default behaviors for generated views
defaults:
measures: [count]
hide_fields_by_suffix: ['_pk', '_fk']
# Map BigQuery data types to LookML
type_mapping:
- bq_type: 'TIMESTAMP'
lookml_type: 'dimension_group'
lookml_params: { type: 'time', timeframes: '[raw, time, date, week, month]' }
- bq_type: 'INTEGER'
lookml_type: 'dimension'
lookml_params: { type: 'number' }
# ... and so on
# BigQuery Connection Details
connection:
# (Recommended) Point to your Dataform credentials file.
dataform_credentials_file: './.df-credentials.json'
# If the file above is not found, Concordia falls back to Google ADC.
# The GCP project ID and location to target.
project_id: 'my-gcp-project'
location: 'europe-west2'
# The datasets to scan for tables.
datasets: - 'marts' - 'finance'
# Looker project configuration
looker:
project_path: './looker_project/' # Path to your local Looker git repo
views_path: 'views/generated_views.view.lkml' # File path where generated views will be written
explores_path: 'views/generated_explores.view.lkml' # File path where generated explores will be written
connection: 'bigquery-prod' # The name of your Looker connection
# Rules for how models and fields are generated
model_rules:
# Define how to identify PKs and FKs from column names
naming_conventions:
pk_suffix: '\_pk'
fk_suffix: '\_fk'
# Define default behaviors for generated views
defaults:
measures: [count]
hide_fields_by_suffix: ['_pk', '_fk']
# Map BigQuery data types to LookML
type_mapping: - bq_type: 'TIMESTAMP'
lookml_type: 'dimension_group'
lookml_params: { type: 'time', timeframes: '[raw, time, date, week, month]' } - bq_type: 'INTEGER'
lookml_type: 'dimension'
lookml_params: { type: 'number' } # ... and so on
```
## Usage
//TODO
Raw data
{
"_id": null,
"home_page": null,
"name": "concordia-cli",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.11",
"maintainer_email": null,
"keywords": "looker, lookml, bigquery, dataform, cli, dbt",
"author": "Cobry",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/90/d9/c22e978e63dbb27d37f33d45a77a7cfc36203eb831b19dced5482843decb/concordia_cli-0.2.0.tar.gz",
"platform": null,
"description": "Concordia\n\n\n\n<p align=\"center\">\n<em>Bring harmony to your data stack.</em>\n</p>\n\n<p align=\"center\">\n<a href=\"#what-is-concordia\">What is it?</a> \u2022\n<a href=\"#how-it-works\">How it Works</a> \u2022\n<a href=\"#key-features\">Features</a> \u2022\n<a href=\"#getting-started\">Getting Started</a> \u2022\n<a href=\"#usage\">Usage</a> \u2022\n<a href=\"#configuration\">Configuration</a>\n</p>\n\nWhat is Concordia?\nConcordia is a command-line interface (CLI) tool that automates the creation and maintenance of Looker views, ensuring they are always in sync with your BigQuery data warehouse. It establishes your BigQuery/Dataform schema as the single source of truth and propagates its structure and documentation directly into your Looker project.\n\nIf you've ever had to:\n\n- Manually create a new LookML view for every new table in your warehouse.\n- Update a LookML dimension because a column name changed in BigQuery.\n- Copy and paste column descriptions from a dbt/dataform model into a Looker view.\n- Notice that the documentation in Looker is out of date with the real-world table.\n\n...then Concordia is the tool for you.\n\n## How it Works\n\nConcordia operates on a simple, unidirectional data flow. It reads the metadata (column names, data types, descriptions) directly from your BigQuery tables and uses that information to generate clean, consistent, and documented LookML view files.\n\n`Dataform/BigQuery (Source of Truth) -> Concordia -> Looker .view Files`\n\nThis ensures that your semantic layer in Looker is a perfect reflection of your transformation layer in the data warehouse, eliminating drift and manual effort.\n\n## Key Features\n\nAutomated View Generation: Create a complete, well-structured LookML view from a BigQuery table with a single command.\n\n- Documentation Sync: Automatically pulls column descriptions from BigQuery and populates the description tag in your LookML dimensions.\n- Convention over Configuration: Uses smart naming conventions (e.g., for primary and foreign keys) to generate better LookML.\n- Intelligent Defaults: Automatically adds a count measure, hides key fields, and creates a set for drill fields.\n- Simple Configuration: A single concordia.yml file manages all project settings.\n- Secure Authentication: Leverages existing Dataform credentials files or Google Application Default Credentials (ADC) so you don't have to manage new secrets.\n\n## Getting Started\n\n1. Installation\n (Placeholder for installation instructions, e.g., pip install concordia-cli)\n2. Initialization\n Navigate to the root of your analytics repository and run: `concordia init`. This will create a concordia.yml file in your project. This is where you will configure the tool.\n\n## Configuration\n\nAll configuration is handled in the concordia.yml file.\n\n```yaml\n# concordia.yml - Example Configuration\n\n# BigQuery Connection Details\nconnection:\n # (Recommended) Point to your Dataform credentials file.\n dataform_credentials_file: './.df-credentials.json'\n\n # If the file above is not found, Concordia falls back to Google ADC.\n\n # The GCP project ID and location to target.\n project_id: 'my-gcp-project'\n location: 'europe-west2'\n\n # The datasets to scan for tables.\n datasets:\n - 'marts'\n - 'finance'\n\n# Looker project configuration\nlooker:\n project_path: './looker_project/' # Path to your local Looker git repo\n views_path: 'views/base/base.view.lkml' # Path for generated base view\n connection: 'bigquery-prod' # The name of your Looker connection\n\n# Rules for how models and fields are generated\nmodel_rules:\n # Define how to identify PKs and FKs from column names\n naming_conventions:\n pk_suffix: '_pk'\n fk_suffix: '_fk'\n\n # Define default behaviors for generated views\n defaults:\n measures: [count]\n hide_fields_by_suffix: ['_pk', '_fk']\n\n # Map BigQuery data types to LookML\n type_mapping:\n - bq_type: 'TIMESTAMP'\n lookml_type: 'dimension_group'\n lookml_params: { type: 'time', timeframes: '[raw, time, date, week, month]' }\n - bq_type: 'INTEGER'\n lookml_type: 'dimension'\n lookml_params: { type: 'number' }\n # ... and so on\n\n\n# BigQuery Connection Details\n\nconnection:\n\n# (Recommended) Point to your Dataform credentials file.\n\ndataform_credentials_file: './.df-credentials.json'\n\n# If the file above is not found, Concordia falls back to Google ADC.\n\n# The GCP project ID and location to target.\n\nproject_id: 'my-gcp-project'\nlocation: 'europe-west2'\n\n# The datasets to scan for tables.\n\ndatasets: - 'marts' - 'finance'\n\n# Looker project configuration\n\nlooker:\nproject_path: './looker_project/' # Path to your local Looker git repo\nviews_path: 'views/generated_views.view.lkml' # File path where generated views will be written\nexplores_path: 'views/generated_explores.view.lkml' # File path where generated explores will be written\nconnection: 'bigquery-prod' # The name of your Looker connection\n\n# Rules for how models and fields are generated\n\nmodel_rules:\n\n# Define how to identify PKs and FKs from column names\n\nnaming_conventions:\npk_suffix: '\\_pk'\nfk_suffix: '\\_fk'\n\n# Define default behaviors for generated views\n\ndefaults:\nmeasures: [count]\nhide_fields_by_suffix: ['_pk', '_fk']\n\n# Map BigQuery data types to LookML\n\ntype_mapping: - bq_type: 'TIMESTAMP'\nlookml_type: 'dimension_group'\nlookml_params: { type: 'time', timeframes: '[raw, time, date, week, month]' } - bq_type: 'INTEGER'\nlookml_type: 'dimension'\nlookml_params: { type: 'number' } # ... and so on\n```\n\n## Usage\n\n//TODO\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Concordia CLI - Generate LookML from BigQuery using Looker & Dataform",
"version": "0.2.0",
"project_urls": {
"Bug Tracker": "https://github.com/CobryDev/concordia-cli/issues",
"Homepage": "https://github.com/CobryDev/concordia-cli",
"Repository": "https://github.com/CobryDev/concordia-cli"
},
"split_keywords": [
"looker",
" lookml",
" bigquery",
" dataform",
" cli",
" dbt"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "c389fad8d624831c77382e3392bd0fdd29d55d446bbfd013dea1ab4217d7fb45",
"md5": "97e0f88e895d0c8655f11aaad734da26",
"sha256": "99c275261b2b17c687dfb19f23615cb6bc5e42ead745e0a9759cd68d6af0afce"
},
"downloads": -1,
"filename": "concordia_cli-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "97e0f88e895d0c8655f11aaad734da26",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11",
"size": 47024,
"upload_time": "2025-08-22T12:21:04",
"upload_time_iso_8601": "2025-08-22T12:21:04.233382Z",
"url": "https://files.pythonhosted.org/packages/c3/89/fad8d624831c77382e3392bd0fdd29d55d446bbfd013dea1ab4217d7fb45/concordia_cli-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "90d9c22e978e63dbb27d37f33d45a77a7cfc36203eb831b19dced5482843decb",
"md5": "7dae283c6f7da772725ea582c4a84cb6",
"sha256": "97cedbb84c31cb1acd5456130820f7160a60d46f804725516a08627030fc2b5c"
},
"downloads": -1,
"filename": "concordia_cli-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "7dae283c6f7da772725ea582c4a84cb6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11",
"size": 40760,
"upload_time": "2025-08-22T12:21:05",
"upload_time_iso_8601": "2025-08-22T12:21:05.519546Z",
"url": "https://files.pythonhosted.org/packages/90/d9/c22e978e63dbb27d37f33d45a77a7cfc36203eb831b19dced5482843decb/concordia_cli-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-22 12:21:05",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "CobryDev",
"github_project": "concordia-cli",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "click",
"specs": []
},
{
"name": "pyyaml",
"specs": []
},
{
"name": "google-cloud-bigquery",
"specs": [
[
">=",
"3.0.0"
]
]
},
{
"name": "google-auth",
"specs": [
[
">=",
"2.0.0"
]
]
},
{
"name": "jinja2",
"specs": [
[
">=",
"3.0.0"
]
]
},
{
"name": "lkml",
"specs": [
[
">=",
"1.3.0"
]
]
},
{
"name": "pandas",
"specs": [
[
">=",
"1.5.0"
]
]
},
{
"name": "pandas-gbq",
"specs": [
[
">=",
"0.19.0"
]
]
},
{
"name": "pydantic",
"specs": [
[
">=",
"2.0.0"
]
]
},
{
"name": "ruamel.yaml",
"specs": [
[
">=",
"0.17.0"
]
]
},
{
"name": "types-PyYAML",
"specs": []
},
{
"name": "pytest",
"specs": [
[
">=",
"7.0.0"
]
]
},
{
"name": "pytest-cov",
"specs": [
[
">=",
"4.0.0"
]
]
},
{
"name": "pytest-mock",
"specs": [
[
">=",
"3.10.0"
]
]
},
{
"name": "pluggy",
"specs": [
[
">=",
"1.5.0"
]
]
},
{
"name": "iniconfig",
"specs": [
[
">=",
"1.0.0"
]
]
},
{
"name": "packaging",
"specs": [
[
">=",
"20.0"
]
]
},
{
"name": "pygments",
"specs": [
[
">=",
"2.7.2"
]
]
},
{
"name": "mypy",
"specs": []
}
],
"lcname": "concordia-cli"
}
JSON
