| Name | snowcli-tools JSON |
| Version |
1.1.0
JSON |
| download |
| home_page | None |
| Summary | Snowflake CLI Tools: generate a Data Catalog and Dependency Graph on top of the official Snowflake CLI; includes parallel query helpers |
| upload_time | 2025-09-12 22:11:39 |
| maintainer | None |
| docs_url | None |
| author | Evan Kim |
| requires_python | >=3.12 |
| license | None |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# SNOWCLI-TOOLS
SNOWCLI-TOOLS is an ergonomic enhancement on top of the official Snowflake CLI (`snow`).
This project leverages your existing `snow` CLI profiles to add powerful, concurrent data tooling:
- **Automated Data Catalogue**: Generate a comprehensive JSON/JSONL catalogue of your Snowflake objects.
- **Dependency Graph Generation**: Generate object dependencies to understand data lineage.
- **Parallel Query Execution**: Run multiple queries concurrently for faster bulk workloads.
## Prerequisites
- Python 3.12+
- UV (recommended): https://docs.astral.sh/uv/
- The official [Snowflake CLI (`snow`)](https://docs.snowflake.com/en/user-guide/snowcli) (installed via UV below)
## Installation
Install from PyPI (recommended):
```bash
# Install the package
uv pip install snowcli-tools
# Check the CLI entry point
snowflake-cli --help
# Or run ad‑hoc without installing to your environment
uvx --from snowcli-tools snowflake-cli --version
```
PyPI project page: https://pypi.org/project/snowcli-tools/
```bash
# Clone the repository
git clone https://github.com/Evan-Kim2028/snowflake-cli-tools-py.git
cd snowflake-cli-tools-py
# Install project deps and the Snowflake CLI via UV
uv sync
uv add snowflake-cli
```
## Quick Start
```bash
# 1) Install deps + Snowflake CLI
uv sync
uv add snowflake-cli
# 2) Create or select a Snowflake CLI connection (one-time)
uv run snowflake-cli setup-connection
# 3) Smoke test
uv run snowflake-cli query "SELECT CURRENT_VERSION()"
# 4) Build a catalog (default output: ./data_catalogue)
uv run snowflake-cli catalog
# 5) Generate a dependency graph
# By default, outputs to ./dependencies (dependencies.json / dependencies.dot)
uv run snowflake-cli depgraph --account -f dot
# Or restrict to a database and emit JSON to the default directory
uv run snowflake-cli depgraph --database MY_DB -f json
# To choose a different directory or filename
uv run snowflake-cli depgraph --account -f json -o ./my_deps
uv run snowflake-cli depgraph --account -f json -o ./my_deps/graph.json
```
## Setup
This tool uses your `snow` CLI connection profiles.
Use the official `snow` CLI to create a profile with your preferred
authentication method. Two common examples:
Key‑pair (recommended for headless/automation):
```bash
snow connection add \
--connection-name my-keypair \
--account <account> \
--user <user> \
--authenticator SNOWFLAKE_JWT \
--private-key /path/to/rsa_key.p8 \
--warehouse <warehouse> \
--database <database> \
--schema <schema> \
--role <role> \
--default \
--no-interactive
```
SSO via browser (Okta/External Browser):
```bash
snow connection add \
--connection-name my-sso \
--account <account> \
--user <user> \
--authenticator externalbrowser \
--warehouse <warehouse> \
--database <database> \
--schema <schema> \
--role <role> \
--default
```
Profile selection precedence:
- CLI flag `--profile/-p`
- `SNOWFLAKE_PROFILE` env var
- Default connection in your `snow` config
Optional helper in this repo:
```bash
# Convenience only: creates a key‑pair profile via `snow connection add`
uv run snowflake-cli setup-connection
```
This helper is optional; you can always manage profiles directly with `snow`.
## Usage
All commands are run through the `snowflake-cli` entry point.
### Query Execution
Execute single queries with flexible output formats.
```bash
# Simple query with table output
uv run snowflake-cli query "SELECT * FROM my_table LIMIT 10"
# Execute and get JSON output
uv run snowflake-cli query "SELECT * FROM my_table LIMIT 10" --format json
# Preview a table's structure and content
uv run snowflake-cli preview my_table
# Execute a query from a .sql file
uv run snowflake-cli query "$(cat my_query.sql)"
```
### Data Cataloguing
Generate a data catalogue by introspecting database metadata (works with any Snowflake account). Outputs JSON by default; JSONL is available for ingestion-friendly workflows. DDL is optional and fetched concurrently when enabled.
```bash
# Build a catalog for the current database (default output: ./data_catalogue)
uv run snowflake-cli catalog
# Build for a specific database
uv run snowflake-cli catalog --database MY_DB --output-dir ./data_catalogue_db
# Build for the entire account
uv run snowflake-cli catalog --account --output-dir ./data_catalogue_all
# Include DDL (concurrent by default; opt-in)
uv run snowflake-cli catalog --database MY_DB --output-dir ./data_catalogue_ddled --include-ddl
# JSONL output
uv run snowflake-cli catalog --database MY_DB --output-dir ./data_catalogue_jsonl --format jsonl
```
Files created (per format):
- schemata.(json|jsonl)
- tables.(json|jsonl)
- columns.(json|jsonl)
- views.(json|jsonl)
- materialized_views.(json|jsonl)
- routines.(json|jsonl)
- functions.(json|jsonl)
- procedures.(json|jsonl)
- tasks.(json|jsonl)
- dynamic_tables.(json|jsonl)
- catalog_summary.json (counts)
### Dependency Graph
Create a dependency graph of Snowflake objects using either
`SNOWFLAKE.ACCOUNT_USAGE.OBJECT_DEPENDENCIES` (preferred) or a fallback to
`INFORMATION_SCHEMA.VIEW_TABLE_USAGE`.
Examples:
```bash
# Account-wide (requires privileges), Graphviz DOT
uv run snowflake-cli depgraph --account -f dot
# Restrict to a database, JSON output
uv run snowflake-cli depgraph --database PIPELINE_V2_GROOT_DB -f json
# Save to a custom directory or file
uv run snowflake-cli depgraph --account -f json -o ./my_deps
uv run snowflake-cli depgraph --account -f dot -o ./my_deps/graph.dot
```
Notes:
- ACCOUNT_USAGE has latency and requires appropriate roles; if not accessible,
the CLI falls back to view→table dependencies from INFORMATION_SCHEMA.
- Output formats: `json` (nodes/edges) and `dot` (render with Graphviz).
- Default output directory is `./dependencies` when `-o/--output` is not provided.
### Parallel Queries
Execute multiple queries concurrently based on a template.
**Example 1: Templated Queries**
```bash
# Query multiple object types in parallel
uv run snowflake-cli parallel "type_a" "type_b" \
--query-template "SELECT * FROM objects WHERE type = '{object}'" \
--output-dir ./results
```
**Example 2: Executing from a File**
You can also execute a list of queries from a file using shell commands:
```bash
# queries.txt contains one query per line
# SELECT * FROM my_table;
# SELECT COUNT(*) FROM another_table;
cat queries.txt | xargs -I {} uv run snowflake-cli query "{}"
```
## CLI Commands
| Command | Description |
| ------------------ | -------------------------------------------------------- |
| `test` | Test the current Snowflake CLI connection. |
| `query` | Execute a single SQL query (table/JSON/CSV output). |
| `parallel` | Execute multiple queries in parallel (spawns `snow`). |
| `preview` | Preview table contents. |
| `catalog` | Build a JSON/JSONL data catalog (use `--include-ddl` to add DDL). |
| `depgraph` | Generate a dependency graph (DOT/JSON output). |
| `config` | Show the current tool configuration. |
| `setup-connection` | Helper to create a persistent `snow` CLI connection. |
| `init-config` | Create a local configuration file for this tool. |
### Catalog design notes (portable by default)
- Uses SHOW commands where possible (schemas, materialized views, dynamic tables, tasks, functions, procedures) for broad visibility with minimal privileges.
- Complements SHOW with INFORMATION_SCHEMA (tables, columns, views) for standardized column-level details.
- Works with any Snowflake account because it only uses standard Snowflake metadata interfaces.
- Optional DDL capture uses GET_DDL per object and fetches concurrently for performance.
### Best practices
- Configure and test your Snowflake CLI connection first (key‑pair, Okta, OAuth are supported by `snow`).
- Run with a role that has USAGE on the target databases/schemas to maximize visibility.
- Prefer `--format jsonl` for ingestion and downstream processing; JSONL is line‑delimited and append‑friendly.
- When enabling `--include-ddl`, increase concurrency with `--max-ddl-concurrency` for large estates.
- Start with a database‑scoped run, then expand to `--account` if needed and permitted.
### Transparency and security
- This project never handles your secrets or opens browsers; it delegates all auth to your `snow` CLI.
- Use profiles appropriate for your environment (key‑pair for automation, SSO for interactive use).
## Development
```bash
# Install with development dependencies
uv sync --dev
# Run tests
uv run pytest
# Format code
uv run black src/
```
## License
This project is licensed under the MIT License.
Raw data
{
"_id": null,
"home_page": null,
"name": "snowcli-tools",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.12",
"maintainer_email": null,
"keywords": null,
"author": "Evan Kim",
"author_email": "Evan Kim <ekcopersonal@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/c4/6c/59ec65866d7b7b09b71fb465b43b4a89fbd6b75b1aaa0759213d0a471b7a/snowcli_tools-1.1.0.tar.gz",
"platform": null,
"description": "# SNOWCLI-TOOLS\n\nSNOWCLI-TOOLS is an ergonomic enhancement on top of the official Snowflake CLI (`snow`).\nThis project leverages your existing `snow` CLI profiles to add powerful, concurrent data tooling:\n\n- **Automated Data Catalogue**: Generate a comprehensive JSON/JSONL catalogue of your Snowflake objects.\n- **Dependency Graph Generation**: Generate object dependencies to understand data lineage.\n- **Parallel Query Execution**: Run multiple queries concurrently for faster bulk workloads.\n\n## Prerequisites\n\n- Python 3.12+\n- UV (recommended): https://docs.astral.sh/uv/\n- The official [Snowflake CLI (`snow`)](https://docs.snowflake.com/en/user-guide/snowcli) (installed via UV below)\n\n## Installation\n\nInstall from PyPI (recommended):\n\n```bash\n# Install the package\nuv pip install snowcli-tools\n\n# Check the CLI entry point\nsnowflake-cli --help\n\n# Or run ad\u2011hoc without installing to your environment\nuvx --from snowcli-tools snowflake-cli --version\n```\n\nPyPI project page: https://pypi.org/project/snowcli-tools/\n\n```bash\n# Clone the repository\ngit clone https://github.com/Evan-Kim2028/snowflake-cli-tools-py.git\ncd snowflake-cli-tools-py\n\n# Install project deps and the Snowflake CLI via UV\nuv sync\nuv add snowflake-cli\n```\n\n\n## Quick Start\n\n```bash\n# 1) Install deps + Snowflake CLI\nuv sync\nuv add snowflake-cli\n\n# 2) Create or select a Snowflake CLI connection (one-time)\nuv run snowflake-cli setup-connection\n\n# 3) Smoke test\nuv run snowflake-cli query \"SELECT CURRENT_VERSION()\"\n\n# 4) Build a catalog (default output: ./data_catalogue)\nuv run snowflake-cli catalog\n\n# 5) Generate a dependency graph\n# By default, outputs to ./dependencies (dependencies.json / dependencies.dot)\nuv run snowflake-cli depgraph --account -f dot\n\n# Or restrict to a database and emit JSON to the default directory\nuv run snowflake-cli depgraph --database MY_DB -f json\n\n# To choose a different directory or filename\nuv run snowflake-cli depgraph --account -f json -o ./my_deps\nuv run snowflake-cli depgraph --account -f json -o ./my_deps/graph.json\n```\n\n## Setup\n\nThis tool uses your `snow` CLI connection profiles.\n\nUse the official `snow` CLI to create a profile with your preferred\nauthentication method. Two common examples:\n\nKey\u2011pair (recommended for headless/automation):\n\n```bash\nsnow connection add \\\n --connection-name my-keypair \\\n --account <account> \\\n --user <user> \\\n --authenticator SNOWFLAKE_JWT \\\n --private-key /path/to/rsa_key.p8 \\\n --warehouse <warehouse> \\\n --database <database> \\\n --schema <schema> \\\n --role <role> \\\n --default \\\n --no-interactive\n```\n\nSSO via browser (Okta/External Browser):\n\n```bash\nsnow connection add \\\n --connection-name my-sso \\\n --account <account> \\\n --user <user> \\\n --authenticator externalbrowser \\\n --warehouse <warehouse> \\\n --database <database> \\\n --schema <schema> \\\n --role <role> \\\n --default\n```\n\nProfile selection precedence:\n\n- CLI flag `--profile/-p`\n- `SNOWFLAKE_PROFILE` env var\n- Default connection in your `snow` config\n\nOptional helper in this repo:\n\n```bash\n# Convenience only: creates a key\u2011pair profile via `snow connection add`\nuv run snowflake-cli setup-connection\n```\n\nThis helper is optional; you can always manage profiles directly with `snow`.\n\n## Usage\n\nAll commands are run through the `snowflake-cli` entry point.\n\n### Query Execution\n\nExecute single queries with flexible output formats.\n\n```bash\n# Simple query with table output\nuv run snowflake-cli query \"SELECT * FROM my_table LIMIT 10\"\n\n# Execute and get JSON output\nuv run snowflake-cli query \"SELECT * FROM my_table LIMIT 10\" --format json\n\n# Preview a table's structure and content\nuv run snowflake-cli preview my_table\n\n# Execute a query from a .sql file\nuv run snowflake-cli query \"$(cat my_query.sql)\"\n```\n\n### Data Cataloguing\n\nGenerate a data catalogue by introspecting database metadata (works with any Snowflake account). Outputs JSON by default; JSONL is available for ingestion-friendly workflows. DDL is optional and fetched concurrently when enabled.\n\n```bash\n# Build a catalog for the current database (default output: ./data_catalogue)\nuv run snowflake-cli catalog\n\n# Build for a specific database\nuv run snowflake-cli catalog --database MY_DB --output-dir ./data_catalogue_db\n\n# Build for the entire account\nuv run snowflake-cli catalog --account --output-dir ./data_catalogue_all\n\n# Include DDL (concurrent by default; opt-in)\nuv run snowflake-cli catalog --database MY_DB --output-dir ./data_catalogue_ddled --include-ddl\n\n# JSONL output\nuv run snowflake-cli catalog --database MY_DB --output-dir ./data_catalogue_jsonl --format jsonl\n```\n\nFiles created (per format):\n- schemata.(json|jsonl)\n- tables.(json|jsonl)\n- columns.(json|jsonl)\n- views.(json|jsonl)\n- materialized_views.(json|jsonl)\n- routines.(json|jsonl)\n- functions.(json|jsonl)\n- procedures.(json|jsonl)\n- tasks.(json|jsonl)\n- dynamic_tables.(json|jsonl)\n- catalog_summary.json (counts)\n\n### Dependency Graph\n\nCreate a dependency graph of Snowflake objects using either\n`SNOWFLAKE.ACCOUNT_USAGE.OBJECT_DEPENDENCIES` (preferred) or a fallback to\n`INFORMATION_SCHEMA.VIEW_TABLE_USAGE`.\n\nExamples:\n\n```bash\n# Account-wide (requires privileges), Graphviz DOT\nuv run snowflake-cli depgraph --account -f dot\n\n# Restrict to a database, JSON output\nuv run snowflake-cli depgraph --database PIPELINE_V2_GROOT_DB -f json\n\n# Save to a custom directory or file\nuv run snowflake-cli depgraph --account -f json -o ./my_deps\nuv run snowflake-cli depgraph --account -f dot -o ./my_deps/graph.dot\n```\n\nNotes:\n- ACCOUNT_USAGE has latency and requires appropriate roles; if not accessible,\n the CLI falls back to view\u2192table dependencies from INFORMATION_SCHEMA.\n- Output formats: `json` (nodes/edges) and `dot` (render with Graphviz).\n - Default output directory is `./dependencies` when `-o/--output` is not provided.\n\n### Parallel Queries\n\nExecute multiple queries concurrently based on a template.\n\n**Example 1: Templated Queries**\n```bash\n# Query multiple object types in parallel\nuv run snowflake-cli parallel \"type_a\" \"type_b\" \\\n --query-template \"SELECT * FROM objects WHERE type = '{object}'\" \\\n --output-dir ./results\n```\n\n**Example 2: Executing from a File**\n\nYou can also execute a list of queries from a file using shell commands:\n```bash\n# queries.txt contains one query per line\n# SELECT * FROM my_table;\n# SELECT COUNT(*) FROM another_table;\n\ncat queries.txt | xargs -I {} uv run snowflake-cli query \"{}\"\n```\n\n## CLI Commands\n\n| Command | Description |\n| ------------------ | -------------------------------------------------------- |\n| `test` | Test the current Snowflake CLI connection. |\n| `query` | Execute a single SQL query (table/JSON/CSV output). |\n| `parallel` | Execute multiple queries in parallel (spawns `snow`). |\n| `preview` | Preview table contents. |\n| `catalog` | Build a JSON/JSONL data catalog (use `--include-ddl` to add DDL). |\n| `depgraph` | Generate a dependency graph (DOT/JSON output). |\n| `config` | Show the current tool configuration. |\n| `setup-connection` | Helper to create a persistent `snow` CLI connection. |\n| `init-config` | Create a local configuration file for this tool. |\n\n### Catalog design notes (portable by default)\n- Uses SHOW commands where possible (schemas, materialized views, dynamic tables, tasks, functions, procedures) for broad visibility with minimal privileges.\n- Complements SHOW with INFORMATION_SCHEMA (tables, columns, views) for standardized column-level details.\n- Works with any Snowflake account because it only uses standard Snowflake metadata interfaces.\n- Optional DDL capture uses GET_DDL per object and fetches concurrently for performance.\n\n### Best practices\n- Configure and test your Snowflake CLI connection first (key\u2011pair, Okta, OAuth are supported by `snow`).\n- Run with a role that has USAGE on the target databases/schemas to maximize visibility.\n- Prefer `--format jsonl` for ingestion and downstream processing; JSONL is line\u2011delimited and append\u2011friendly.\n- When enabling `--include-ddl`, increase concurrency with `--max-ddl-concurrency` for large estates.\n- Start with a database\u2011scoped run, then expand to `--account` if needed and permitted.\n\n### Transparency and security\n- This project never handles your secrets or opens browsers; it delegates all auth to your `snow` CLI.\n- Use profiles appropriate for your environment (key\u2011pair for automation, SSO for interactive use).\n\n## Development\n\n```bash\n# Install with development dependencies\nuv sync --dev\n\n# Run tests\nuv run pytest\n\n# Format code\nuv run black src/\n```\n\n## License\n\nThis project is licensed under the MIT License.\n",
"bugtrack_url": null,
"license": null,
"summary": "Snowflake CLI Tools: generate a Data Catalog and Dependency Graph on top of the official Snowflake CLI; includes parallel query helpers",
"version": "1.1.0",
"project_urls": {
"Documentation": "https://github.com/Evan-Kim2028/snowcli-tools#readme",
"Homepage": "https://github.com/Evan-Kim2028/snowcli-tools",
"Repository": "https://github.com/Evan-Kim2028/snowcli-tools"
},
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "53c9a4412be3de9f82aedfe7f83049bb469e1a3d919277701870342a2ae835de",
"md5": "78a5f908bbab0c0529f491dc9915e223",
"sha256": "bda171c318142b871ac98b7aa3b34ca6c76ec5040339b3e8111f11321fd38b22"
},
"downloads": -1,
"filename": "snowcli_tools-1.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "78a5f908bbab0c0529f491dc9915e223",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.12",
"size": 23748,
"upload_time": "2025-09-12T22:11:38",
"upload_time_iso_8601": "2025-09-12T22:11:38.252539Z",
"url": "https://files.pythonhosted.org/packages/53/c9/a4412be3de9f82aedfe7f83049bb469e1a3d919277701870342a2ae835de/snowcli_tools-1.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "c46c59ec65866d7b7b09b71fb465b43b4a89fbd6b75b1aaa0759213d0a471b7a",
"md5": "aef18fd836122fa4fa73100c5e20c6c4",
"sha256": "85acf5a8f6211f4d71e1b8ff592d04ab83c8fcda87e102b5790e93fb7b755487"
},
"downloads": -1,
"filename": "snowcli_tools-1.1.0.tar.gz",
"has_sig": false,
"md5_digest": "aef18fd836122fa4fa73100c5e20c6c4",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.12",
"size": 19587,
"upload_time": "2025-09-12T22:11:39",
"upload_time_iso_8601": "2025-09-12T22:11:39.530457Z",
"url": "https://files.pythonhosted.org/packages/c4/6c/59ec65866d7b7b09b71fb465b43b4a89fbd6b75b1aaa0759213d0a471b7a/snowcli_tools-1.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-09-12 22:11:39",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Evan-Kim2028",
"github_project": "snowcli-tools#readme",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "snowcli-tools"
}
