# Dataform Templater Plugin for SQLFluff
This plugin integrates [SQLFluff](https://www.sqlfluff.com/) with Dataform projects, allowing SQLFluff to lint and format your Dataform SQLX files after templating.
## Getting Started
1. **Install the plugin**:
```bash
pip install sqlfluff-templater-dataform-full
```
2. **Configure SQLFluff**:
Add the following to your `.sqlfluff` configuration file:
```ini
[sqlfluff]
dialect = bigquery
templater = dataform-full
sql_file_exts = .sql,.sqlx
```
3. **Usage**:
Run SQLFluff as usual:
```bash
sqlfluff lint your_dataform_project/
```
## How it works
This templater operates by using the Dataform CLI to compile `.sqlx` files. It performs the following steps:
1. **Identify Blocks**: The plugin first identifies different types of blocks in your `.sqlx` file: Dataform templated SQL (`${...}`), JavaScript blocks (`js {...}`), and configuration blocks (`config {...}`).
2. **Insert Markers**: For Dataform templated SQL blocks (`${...}`), the content is temporarily wrapped with unique, invisible markers within an Immediately Invoked Function Expression (IIFE). JavaScript and config blocks are passed through largely unchanged.
3. **Compile with Dataform CLI**: A temporary Dataform project is created, relevant files are copied, and the `dataform compile` command is executed on the transformed `.sqlx` file. **The Dataform CLI must be installed and accessible in your system's PATH for this plugin to function.**
4. **Map Slices**: After compilation, the plugin parses the compiled output. It uses the inserted markers to accurately map the compiled SQL back to its original positions in the `.sqlx` source file. This allows SQLFluff to report linting and formatting issues at the correct locations.
## Configuration
You can configure the templater by adding the following options to your `.sqlfluff` file under the `[sqlfluff:templater:dataform-full]` section.
- **`project_dir`**: Specifies the path to your Dataform project root. If not provided, the templater will search for a Dataform project in the current working directory.
- **`dataform_executable`**: Sets a custom path to the Dataform executable. This is useful if the executable is not in your system's `PATH`. This setting takes precedence over the `DATAFORM_EXECUTABLE` environment variable.
- **`parsing_method`**: This templater offers two different methods for parsing `.sqlx` files before compilation. "Parsing" here means identifying the different blocks like `config {...}`, `js {...}`, and `${...}`.
- **`regex` (Default)**: A fast parser that uses regular expressions. Recommended for most use cases.
- **`char`**: A character-by-character parser that is more resilient to complex or unusual nesting of blocks.
### Example Configuration
```ini
[sqlfluff:templater:dataform-full]
# Path to your Dataform project
project_dir = path/to/your/dataform/project
# Custom path to the Dataform executable
dataform_executable = /path/to/your/dataform_cli
# To use the character-based parser instead of the default regex parser
# parsing_method = char
```
## Development
This plugin follows the standard SQLFluff plugin development guide.
The core logic resides in `src/sqlfluff_templater_dataform/templater.py`, specifically the `process` method.
## Known Issues
- The templater relies on balanced curly braces (`{}`) for identifying `js {}` and `config {}` blocks. Unbalanced braces within these blocks (e.g., unmatched `{` or `}`) will cause parsing errors during templating and lead to incorrect linting. Ensure all such blocks have a matching number of opening and closing braces.
Raw data
{
"_id": null,
"home_page": null,
"name": "sqlfluff-templater-dataform-full",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "sqlfluff, sql, linter, formatter, bigquery, dataform, templater",
"author": null,
"author_email": "Taras Kopets <tkopets@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/fc/4b/55bef648f4a112bdaaaf082e8c6d3df36fb6043726bf5073253b9d785eee/sqlfluff_templater_dataform_full-0.1.3.tar.gz",
"platform": null,
"description": "# Dataform Templater Plugin for SQLFluff\n\nThis plugin integrates [SQLFluff](https://www.sqlfluff.com/) with Dataform projects, allowing SQLFluff to lint and format your Dataform SQLX files after templating.\n\n## Getting Started\n\n1. **Install the plugin**:\n\n ```bash\n pip install sqlfluff-templater-dataform-full\n ```\n\n2. **Configure SQLFluff**:\n Add the following to your `.sqlfluff` configuration file:\n\n ```ini\n [sqlfluff]\n dialect = bigquery\n templater = dataform-full\n sql_file_exts = .sql,.sqlx\n ```\n\n3. **Usage**:\n Run SQLFluff as usual:\n ```bash\n sqlfluff lint your_dataform_project/\n ```\n\n## How it works\n\nThis templater operates by using the Dataform CLI to compile `.sqlx` files. It performs the following steps:\n\n1. **Identify Blocks**: The plugin first identifies different types of blocks in your `.sqlx` file: Dataform templated SQL (`${...}`), JavaScript blocks (`js {...}`), and configuration blocks (`config {...}`).\n2. **Insert Markers**: For Dataform templated SQL blocks (`${...}`), the content is temporarily wrapped with unique, invisible markers within an Immediately Invoked Function Expression (IIFE). JavaScript and config blocks are passed through largely unchanged.\n3. **Compile with Dataform CLI**: A temporary Dataform project is created, relevant files are copied, and the `dataform compile` command is executed on the transformed `.sqlx` file. **The Dataform CLI must be installed and accessible in your system's PATH for this plugin to function.**\n4. **Map Slices**: After compilation, the plugin parses the compiled output. It uses the inserted markers to accurately map the compiled SQL back to its original positions in the `.sqlx` source file. This allows SQLFluff to report linting and formatting issues at the correct locations.\n\n## Configuration\n\nYou can configure the templater by adding the following options to your `.sqlfluff` file under the `[sqlfluff:templater:dataform-full]` section.\n\n- **`project_dir`**: Specifies the path to your Dataform project root. If not provided, the templater will search for a Dataform project in the current working directory.\n\n- **`dataform_executable`**: Sets a custom path to the Dataform executable. This is useful if the executable is not in your system's `PATH`. This setting takes precedence over the `DATAFORM_EXECUTABLE` environment variable.\n\n- **`parsing_method`**: This templater offers two different methods for parsing `.sqlx` files before compilation. \"Parsing\" here means identifying the different blocks like `config {...}`, `js {...}`, and `${...}`.\n - **`regex` (Default)**: A fast parser that uses regular expressions. Recommended for most use cases.\n - **`char`**: A character-by-character parser that is more resilient to complex or unusual nesting of blocks.\n\n### Example Configuration\n\n```ini\n[sqlfluff:templater:dataform-full]\n# Path to your Dataform project\nproject_dir = path/to/your/dataform/project\n\n# Custom path to the Dataform executable\ndataform_executable = /path/to/your/dataform_cli\n\n# To use the character-based parser instead of the default regex parser\n# parsing_method = char\n```\n\n## Development\n\nThis plugin follows the standard SQLFluff plugin development guide.\nThe core logic resides in `src/sqlfluff_templater_dataform/templater.py`, specifically the `process` method.\n\n## Known Issues\n\n- The templater relies on balanced curly braces (`{}`) for identifying `js {}` and `config {}` blocks. Unbalanced braces within these blocks (e.g., unmatched `{` or `}`) will cause parsing errors during templating and lead to incorrect linting. Ensure all such blocks have a matching number of opening and closing braces.\n",
"bugtrack_url": null,
"license": null,
"summary": "Lint your Dataform project SQL",
"version": "0.1.3",
"project_urls": null,
"split_keywords": [
"sqlfluff",
" sql",
" linter",
" formatter",
" bigquery",
" dataform",
" templater"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "1da8bd03b178e35105f13d753af7eadc7daf7ed613c2b59580857353aef90978",
"md5": "5e118197095e7190fc3572ecd7f896da",
"sha256": "a59f2ffbcd10a004b96d91ffba4f60f4dcfbb737ecdd8501cbba4ce64eae2d4d"
},
"downloads": -1,
"filename": "sqlfluff_templater_dataform_full-0.1.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "5e118197095e7190fc3572ecd7f896da",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 11763,
"upload_time": "2025-07-28T01:36:34",
"upload_time_iso_8601": "2025-07-28T01:36:34.306866Z",
"url": "https://files.pythonhosted.org/packages/1d/a8/bd03b178e35105f13d753af7eadc7daf7ed613c2b59580857353aef90978/sqlfluff_templater_dataform_full-0.1.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "fc4b55bef648f4a112bdaaaf082e8c6d3df36fb6043726bf5073253b9d785eee",
"md5": "e85458bcfe3d140d4895779d385a9cb1",
"sha256": "b52a61b81c98dd43409abe342a6a354b90e386f52ed9ecdd4c1cf6c7eef52165"
},
"downloads": -1,
"filename": "sqlfluff_templater_dataform_full-0.1.3.tar.gz",
"has_sig": false,
"md5_digest": "e85458bcfe3d140d4895779d385a9cb1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 13076,
"upload_time": "2025-07-28T01:36:35",
"upload_time_iso_8601": "2025-07-28T01:36:35.411661Z",
"url": "https://files.pythonhosted.org/packages/fc/4b/55bef648f4a112bdaaaf082e8c6d3df36fb6043726bf5073253b9d785eee/sqlfluff_templater_dataform_full-0.1.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-28 01:36:35",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "sqlfluff-templater-dataform-full"
}
JSON