| Name | vdsnow JSON |
| Version |
0.5.3
JSON |
| download |
| home_page | None |
| Summary | A CLI for Snowflake project scaffolding, execution, and validation. |
| upload_time | 2025-07-12 11:43:19 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.10 |
| license | None |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# ❄️ vdsnow: A Modern CLI for Snowflake Development
**vdsnow** is a powerful command-line interface designed to streamline and professionalize the entire lifecycle of your Snowflake projects. It provides tools for project scaffolding, intelligent SQL execution, state management, and dependency resolution, turning complex deployments into simple, repeatable commands. vdsnow is designed by leveraging the snowflake cli (snowcli), working not only as a wrapper of it but also as a helper tool to improve your CI/CD workflows.
Think of it as a lightweight, state-aware orchestrator for your Snowflake SQL code, inspired by best practices from tools like dbt and Terraform.
A comprehensive guide to getting started with vdsnow can be found [here](https://github.com/duartevitor-alt/vdsnow-example).
---
## Core Features
- **Project Scaffolding**: Initialize a clean, standardized folder structure for your databases and schemas in seconds.
- **Intelligent Execution**: Run SQL files with automatic context detection based on folder structure.
- **Folder Validates**: Ensure that your folder structure is valid ie there are no files listed to be deployed that they don't exist in your snowflake_structure folder.
- **State-Aware Deployments**: Only deploy what's changed. `vdsnow` tracks the state of your project and can run differential deployments, saving time and reducing risk.
- **Dependency Management (DAG)**: Explicitly define dependencies between your SQL files. `vdsnow` builds a Directed Acyclic Graph (DAG) to ensure objects are deployed in the correct order, every time.
- **CI/CD Ready**: Designed for automation with headless connection support via environment variables.
- **Safe Planning**: Use the `plan` command to see what changes will be made before you apply them.
---
## Installation
`vdsnow` is managed as a Python project. In this README installation is handled via `uv`.
```bash
uv add vdsnow
```
---
## Configuration / Connections
Configuration is handled through two primary files at the root of your project. To init your connection settings, run:
```bash
uv run vdsnow connection init
```
Then, you will be prompted to enter your Snowflake informations. After that, you can edit either the `config.toml` file or the `.env` file if needed.
### 1. `config.toml`
This file defines your connection configurations. You can define multiple connections (e.g., `local`, `headless`).
*Example `config.toml`:*
```toml
default_connection_name = "local"
[connections.local]
account = ""
user = ""
role = ""
warehouse = ""
database = ""
schema = ""
authenticator = "SNOWFLAKE_JWT"
private_key_path = ""
```
### 2. `.env` File
This file stores secrets and environment-specific variables. **It should be added to your `.gitignore` file.** `vdsnow` automatically loads this file at runtime.
*Example `.env`:*
```bash
export SNOWFLAKE_CONNECTIONS_LOCAL_ACCOUNT="my_snowflake_account_identified"
...
```
---
## Project Structure
`vdsnow` creates and manages two key directories:
- `snowflake_structure/`: This is where your raw SQL files live. The folder hierarchy (`<database>/<schema>/...`) is where you organize your DDL scripts (tables, views, procedures, etc.).
- `setup/`: This directory contains auto-generated "manifest" files (`setup.sql`). These files orchestrate the execution of the raw SQL in `snowflake_structure/` by using `!source` commands. You should generally not edit these files by hand; use `vdsnow setup refresh-scripts` to update them.
---
## Command Reference
### `uv run vdsnow setup`
Commands for managing the project's folder and script structure.
- `init`: Interactively creates the initial `snowflake_structure/` and `setup/` directories (should be your first command).
- `add-database`: Adds a new database folder to the structure.
- `add-schema`: Adds a new schema folder to a specified database.
- `refresh-scripts`: **Important!** Scans your `snowflake_structure/` and regenerates all `setup.sql` manifest files to reflect the current state of your project.
- `create-ci`: Creates the necessary SQL scripts in `setup/ci/` for CI/CD workflows.
### `vdsnow sql`
Commands for planning, executing, and managing the state of your SQL deployments.
- `plan`: Compiles and displays the execution plan without running it.
- `-f, --file`: The root `.sql` file to plan (defaults to `setup/setup.sql`).
- `--local`: Shows the plan for the default sandbox context.
- `--differ`: Shows a plan containing only files that have changed since the last run.
- `execute`: Executes the SQL deployment plan - `./compiled` folder is created and aloows you to see what is running.
- `-f, --file`: The root `.sql` file to execute.
- `--local`: Runs the deployment in the default sandbox context.
- `--differ`: Executes only the files that have changed since the last run. Updates `vdstate.json` on success.
- `-q, --query`: Executes only the specified query.
- `refresh-state`: Re-scans the project and updates `vdstate.json` without executing any SQL.
- `-f, --file`: The root `.sql` file to build the state from (defaults to `setup/setup.sql`).
- `--local`: Builds the state based on the local context.
- `get-state-from-remote`: Retrieves the state from a remote Snowflake account. This command will pull the state from the remote account and overwrite `vdstate.json` in case it exists.
- `--db`: The name of the database to retrieve the state from.
- `--schema`: The name of the schema to retrieve the state from.
### `vdsnow check`
Commands for validating your project and environment.
- `version`: Checks the installed `snowcli` version.
- `folder-structure`: Validates the project structure for discrepancies. It raises an error if a file is listed to be deployed (inside `./setup`) but it's not found in `./snowflake_structure`. It raises a warning if a file is defined in `./snowflake_structure` but it's not found in `./setup`.
### `vdsnow connection`
Commands for managing Snowflake connections.
- `init`: Interactively creates or updates a connection configuration in `config.toml`.
- `test`: Tests the configured Snowflake connection.
---
## Advanced Features
### State Management (`vdstate.json`)
`vdsnow` creates a `vdstate.json` file after the first successful `execute` run. This file is a snapshot of your project, containing the path, command, dependencies, and SQL content of every file.
The `--differ` flag uses this file to determine what has changed, enabling intelligent, partial deployments.
Using `--no-local` or `--local` helps you to compile the plan and see how it's deployed locally or remotely. You can even execute commands in the destination environment (remote), but ideally you should not have create access to the destination environment.
### Dependency Management (DAG)
You can define dependencies between your SQL files to ensure they are created in the correct order. `vdsnow` uses this information to build a Directed Acyclic Graph (DAG) and generate a valid execution plan.
To declare a dependency, add a specially formatted comment to your SQL file. The path is **relative to the file you are in**.
*Example: `snowflake_structure/analytics/views/my_view.sql`*
```sql
-- This view depends on a table in the 'base' schema.
-- The path is relative to this file's location.
-- vdsnow_depends_on: ../base/tables/my_table.sql
CREATE OR REPLACE VIEW analytics.my_view AS
SELECT * FROM base.my_table;
```
---
## Example Workflow
1. **Initialize your project:**
```bash
uv run vdsnow setup init
```
2. **Initialize your connection:**
```bash
uv run vdsnow connection setup
```
3. **Add your SQL files** to the `snowflake_structure/` directory.
4. **Declare dependencies** in your SQL files using `-- vdsnow_depends_on: ...`.
5. **Refresh the setup scripts** to wire everything together:
```bash
uv run vdsnow setup refresh-scripts
```
6. **Plan your deployment** to see what will happen:
```bash
uv run vdsnow sql plan
```
7. **Execute the local deployment:**
```bash
uv run vdsnow sql execute -f ./setup/setup.sql
```
8. **Make a change** to one of your SQL files.
9. **Plan and execute the change** using `--differ`:
```bash
uv run vdsnow sql plan --differ
uv run vdsnow sql execute -f ./setup/setup.sql --differ
```
10. **Add a new schema**
```bash
uv run vdsnow setup add-schema db_raw_new --schema schema_new
```
11. **Define new objects in the new schema**
12. **Validate your folder structure**
```bash
uv run vdsnow check folder-structure
```
13. **Create CI folder once ready**
```bash
uv run vdsnow setup create-ci
```
Raw data
{
"_id": null,
"home_page": null,
"name": "vdsnow",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": null,
"author": null,
"author_email": "Vitor Duarte <vitordduarte95@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/f3/44/f942200ace5b082f06c967cd7262e300dd4c84556078d971dc4b15e53214/vdsnow-0.5.3.tar.gz",
"platform": null,
"description": "# \u2744\ufe0f vdsnow: A Modern CLI for Snowflake Development\n\n**vdsnow** is a powerful command-line interface designed to streamline and professionalize the entire lifecycle of your Snowflake projects. It provides tools for project scaffolding, intelligent SQL execution, state management, and dependency resolution, turning complex deployments into simple, repeatable commands. vdsnow is designed by leveraging the snowflake cli (snowcli), working not only as a wrapper of it but also as a helper tool to improve your CI/CD workflows.\n\nThink of it as a lightweight, state-aware orchestrator for your Snowflake SQL code, inspired by best practices from tools like dbt and Terraform.\n\nA comprehensive guide to getting started with vdsnow can be found [here](https://github.com/duartevitor-alt/vdsnow-example).\n\n---\n\n## Core Features\n\n- **Project Scaffolding**: Initialize a clean, standardized folder structure for your databases and schemas in seconds.\n- **Intelligent Execution**: Run SQL files with automatic context detection based on folder structure.\n- **Folder Validates**: Ensure that your folder structure is valid ie there are no files listed to be deployed that they don't exist in your snowflake_structure folder.\n- **State-Aware Deployments**: Only deploy what's changed. `vdsnow` tracks the state of your project and can run differential deployments, saving time and reducing risk.\n- **Dependency Management (DAG)**: Explicitly define dependencies between your SQL files. `vdsnow` builds a Directed Acyclic Graph (DAG) to ensure objects are deployed in the correct order, every time.\n- **CI/CD Ready**: Designed for automation with headless connection support via environment variables.\n- **Safe Planning**: Use the `plan` command to see what changes will be made before you apply them.\n\n---\n\n## Installation\n\n`vdsnow` is managed as a Python project. In this README installation is handled via `uv`.\n\n```bash\nuv add vdsnow\n```\n\n---\n\n## Configuration / Connections\n\nConfiguration is handled through two primary files at the root of your project. To init your connection settings, run:\n\n```bash\nuv run vdsnow connection init\n```\n\nThen, you will be prompted to enter your Snowflake informations. After that, you can edit either the `config.toml` file or the `.env` file if needed.\n\n### 1. `config.toml`\n\nThis file defines your connection configurations. You can define multiple connections (e.g., `local`, `headless`).\n\n*Example `config.toml`:*\n```toml\ndefault_connection_name = \"local\"\n\n[connections.local]\naccount = \"\"\nuser = \"\"\nrole = \"\"\nwarehouse = \"\"\ndatabase = \"\"\nschema = \"\"\nauthenticator = \"SNOWFLAKE_JWT\"\nprivate_key_path = \"\"\n\n```\n\n### 2. `.env` File\n\nThis file stores secrets and environment-specific variables. **It should be added to your `.gitignore` file.** `vdsnow` automatically loads this file at runtime.\n\n*Example `.env`:*\n```bash\nexport SNOWFLAKE_CONNECTIONS_LOCAL_ACCOUNT=\"my_snowflake_account_identified\"\n...\n```\n\n---\n\n## Project Structure\n\n`vdsnow` creates and manages two key directories:\n\n- `snowflake_structure/`: This is where your raw SQL files live. The folder hierarchy (`<database>/<schema>/...`) is where you organize your DDL scripts (tables, views, procedures, etc.).\n- `setup/`: This directory contains auto-generated \"manifest\" files (`setup.sql`). These files orchestrate the execution of the raw SQL in `snowflake_structure/` by using `!source` commands. You should generally not edit these files by hand; use `vdsnow setup refresh-scripts` to update them.\n\n---\n\n## Command Reference\n\n### `uv run vdsnow setup`\n\nCommands for managing the project's folder and script structure.\n\n- `init`: Interactively creates the initial `snowflake_structure/` and `setup/` directories (should be your first command).\n- `add-database`: Adds a new database folder to the structure.\n- `add-schema`: Adds a new schema folder to a specified database.\n- `refresh-scripts`: **Important!** Scans your `snowflake_structure/` and regenerates all `setup.sql` manifest files to reflect the current state of your project.\n- `create-ci`: Creates the necessary SQL scripts in `setup/ci/` for CI/CD workflows.\n\n### `vdsnow sql`\n\nCommands for planning, executing, and managing the state of your SQL deployments.\n\n- `plan`: Compiles and displays the execution plan without running it.\n - `-f, --file`: The root `.sql` file to plan (defaults to `setup/setup.sql`).\n - `--local`: Shows the plan for the default sandbox context.\n - `--differ`: Shows a plan containing only files that have changed since the last run.\n\n- `execute`: Executes the SQL deployment plan - `./compiled` folder is created and aloows you to see what is running.\n - `-f, --file`: The root `.sql` file to execute.\n - `--local`: Runs the deployment in the default sandbox context.\n - `--differ`: Executes only the files that have changed since the last run. Updates `vdstate.json` on success.\n - `-q, --query`: Executes only the specified query.\n\n- `refresh-state`: Re-scans the project and updates `vdstate.json` without executing any SQL.\n - `-f, --file`: The root `.sql` file to build the state from (defaults to `setup/setup.sql`).\n - `--local`: Builds the state based on the local context.\n\n- `get-state-from-remote`: Retrieves the state from a remote Snowflake account. This command will pull the state from the remote account and overwrite `vdstate.json` in case it exists.\n - `--db`: The name of the database to retrieve the state from.\n - `--schema`: The name of the schema to retrieve the state from.\n\n### `vdsnow check`\n\nCommands for validating your project and environment.\n\n- `version`: Checks the installed `snowcli` version.\n- `folder-structure`: Validates the project structure for discrepancies. It raises an error if a file is listed to be deployed (inside `./setup`) but it's not found in `./snowflake_structure`. It raises a warning if a file is defined in `./snowflake_structure` but it's not found in `./setup`.\n\n### `vdsnow connection`\n\nCommands for managing Snowflake connections.\n\n- `init`: Interactively creates or updates a connection configuration in `config.toml`.\n- `test`: Tests the configured Snowflake connection.\n\n---\n\n## Advanced Features\n\n### State Management (`vdstate.json`)\n\n`vdsnow` creates a `vdstate.json` file after the first successful `execute` run. This file is a snapshot of your project, containing the path, command, dependencies, and SQL content of every file.\n\nThe `--differ` flag uses this file to determine what has changed, enabling intelligent, partial deployments.\n\nUsing `--no-local` or `--local` helps you to compile the plan and see how it's deployed locally or remotely. You can even execute commands in the destination environment (remote), but ideally you should not have create access to the destination environment.\n\n### Dependency Management (DAG)\n\nYou can define dependencies between your SQL files to ensure they are created in the correct order. `vdsnow` uses this information to build a Directed Acyclic Graph (DAG) and generate a valid execution plan.\n\nTo declare a dependency, add a specially formatted comment to your SQL file. The path is **relative to the file you are in**.\n\n*Example: `snowflake_structure/analytics/views/my_view.sql`*\n```sql\n-- This view depends on a table in the 'base' schema.\n-- The path is relative to this file's location.\n-- vdsnow_depends_on: ../base/tables/my_table.sql\n\nCREATE OR REPLACE VIEW analytics.my_view AS\nSELECT * FROM base.my_table;\n```\n\n---\n\n## Example Workflow\n\n1. **Initialize your project:**\n ```bash\n uv run vdsnow setup init\n ```\n\n2. **Initialize your connection:**\n ```bash\n uv run vdsnow connection setup\n ```\n\n3. **Add your SQL files** to the `snowflake_structure/` directory.\n\n4. **Declare dependencies** in your SQL files using `-- vdsnow_depends_on: ...`.\n\n5. **Refresh the setup scripts** to wire everything together:\n ```bash\n uv run vdsnow setup refresh-scripts\n ```\n\n6. **Plan your deployment** to see what will happen:\n ```bash\n uv run vdsnow sql plan\n ```\n\n7. **Execute the local deployment:**\n ```bash\n uv run vdsnow sql execute -f ./setup/setup.sql\n ```\n\n8. **Make a change** to one of your SQL files.\n\n9. **Plan and execute the change** using `--differ`:\n ```bash\n uv run vdsnow sql plan --differ\n uv run vdsnow sql execute -f ./setup/setup.sql --differ\n ```\n\n10. **Add a new schema**\n ```bash\n uv run vdsnow setup add-schema db_raw_new --schema schema_new\n ```\n\n11. **Define new objects in the new schema**\n\n12. **Validate your folder structure**\n ```bash\n uv run vdsnow check folder-structure\n ```\n\n13. **Create CI folder once ready**\n ```bash\n uv run vdsnow setup create-ci\n ```\n",
"bugtrack_url": null,
"license": null,
"summary": "A CLI for Snowflake project scaffolding, execution, and validation.",
"version": "0.5.3",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "06d6d05790bb289a73388c74f2a169a6b73e284c98a7b577fd229a71dc86d537",
"md5": "8555922d1df67789224561b6a0e1f5c5",
"sha256": "4483e9de5117491b2eb57b65018bed31e60586f4c69aacf5d160830130b6127b"
},
"downloads": -1,
"filename": "vdsnow-0.5.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8555922d1df67789224561b6a0e1f5c5",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 23711,
"upload_time": "2025-07-12T11:43:18",
"upload_time_iso_8601": "2025-07-12T11:43:18.503024Z",
"url": "https://files.pythonhosted.org/packages/06/d6/d05790bb289a73388c74f2a169a6b73e284c98a7b577fd229a71dc86d537/vdsnow-0.5.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "f344f942200ace5b082f06c967cd7262e300dd4c84556078d971dc4b15e53214",
"md5": "67a219ce6980c106ccd21ed5f0683fba",
"sha256": "dd8a64376020fed8936c568c56c5952a2a83c82dc47f6b6e005805a10c00ba34"
},
"downloads": -1,
"filename": "vdsnow-0.5.3.tar.gz",
"has_sig": false,
"md5_digest": "67a219ce6980c106ccd21ed5f0683fba",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 68183,
"upload_time": "2025-07-12T11:43:19",
"upload_time_iso_8601": "2025-07-12T11:43:19.891897Z",
"url": "https://files.pythonhosted.org/packages/f3/44/f942200ace5b082f06c967cd7262e300dd4c84556078d971dc4b15e53214/vdsnow-0.5.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-12 11:43:19",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "vdsnow"
}
