# Snowflake CLI Nextflow Plugin
A Snowflake CLI plugin that enables running [Nextflow](https://www.nextflow.io/) workflows directly in [Snowpark Container Services](https://docs.snowflake.com/en/developer-guide/snowpark-container-services/overview).
## Overview
This plugin extends the Snowflake CLI with Nextflow workflow capabilities, allowing you to:
- **Run Nextflow workflows at scale** using Snowflake's compute resources
- **Stream live execution logs** through secure WebSocket connections
- **Manage workflow execution** directly from your command line
- **Leverage Snowflake's security and governance** for bioinformatics and data pipelines
### Key Features
- 🚀 **Seamless Integration**: Run Nextflow workflows directly in Snowpark Container Services
- 📊 **Real-time Monitoring**: Stream live logs and execution status via WebSocket
- 🔒 **Enterprise Security**: Built-in Snowflake authentication and authorization
- 🎯 **Simple CLI Interface**: Familiar command-line experience for workflow management
- 📦 **Flexible Configuration**: Support for Nextflow profiles and custom configurations
## Getting Started
### Prerequisites
- All prerequisites of running snowflake-cli mentioned [here](https://docs.snowflake.com/en/developer-guide/snowflake-cli/installation/installation#requirements)
- Access to a Snowflake account with Snowpark Container Services enabled
- A Snowflake compute pool configured for container workloads
- Nextflow binary correctly installed in your PATH
### Installation
1. **Install Snowflake CLI**
```bash
pip install snowflake-cli
```
2. **Install the Nextflow plugin**
```bash
pip install snowflake-cli-nextflow
```
3. **Configure Snowflake connection**
```bash
snow connection add
```
Follow the prompts to configure your Snowflake connection with appropriate credentials.
4. **Enable the Nextflow plugin**
```bash
snow plugin enable nextflow
```
The `nextflow` command should now be available in the CLI.
### Quick Start
1. **Prepare your Nextflow project**
Create a `nextflow.config` file in your project directory:
```groovy
// nextflow.config
snowflake {
computePool = 'YOUR_COMPUTE_POOL'
workDirStage = 'WORKDIR_STAGE'
stageMounts = 'INPUT:/mnt/input,OUTPUT:/mnt/output'
}
```
2. **Upload nf-snowflake image into snowflake image repository**
```bash
# Pull image from github container registry
docker pull ghcr.io/snowflake-labs/nf-snowflake:latest
# retag the image to snowflake registry
docker tag ghcr.io/snowflake-labs/nf-snowflake:latest <snowflake_registry>
# config snow nextflow to use the corresponding image
snow nextflow config set -key nf_snowflake_image -value <snowflake_registry_image>
```
3. **Run your workflow**
```bash
snow nextflow run /path/to/your/nextflow-project -profile snowflake
```
4. **Monitor execution**
The plugin will automatically:
- Upload your project to Snowflake
- Submit the workflow to Snowpark Container Services
- Stream live logs to your terminal
## Configuration
### Snowflake-specific Configuration
Your `nextflow.config` must include the following Snowflake-specific settings:
```groovy
plugins { id 'nf-snowflake@0.6.3' }
snowflake {
computePool = 'YOUR_COMPUTE_POOL'
workDirStage = 'WORKDIR_STAGE'
stageMounts = 'INPUT:/mnt/input,OUTPUT:/mnt/output'
}
```
### Configuration Parameters
- **`computePool`** (Required): The name of your Snowflake compute pool where the Nextflow containers will run. This compute pool must be configured with appropriate resources and permissions for container workloads.
- **`workDirStage`** (Required): The Snowflake stage name where the plugin will upload your workflow files and store execution artifacts. This stage serves as the working directory for your Nextflow execution.
- **`stageMounts`** (Optional): A comma-separated list of stage mounts in the format `STAGE_NAME:/mount/path`. Each mount makes a Snowflake stage available inside the container at the specified path. Use this to provide input data and collect output results.
### Example with Multiple Stage Mounts
```groovy
snowflake {
computePool = 'NEXTFLOW_POOL'
workDirStage = 'NF_WORKSPACE'
stageMounts = 'RAW_DATA:/mnt/input,RESULTS:/mnt/output,REFERENCES:/mnt/ref'
}
```
## Usage Examples
### Basic Workflow Execution
```bash
# Run with default profile
snow nextflow run ./my-workflow
# Run with specific profile
snow nextflow run ./my-workflow -profile snowflake
# Check available commands
snow nextflow --help
```
## License
This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.
## Acknowledgments
- [Nextflow](https://www.nextflow.io/) - The workflow management system
- [Snowflake](https://www.snowflake.com/) - The cloud data platform
- [Snowpark Container Services](https://docs.snowflake.com/en/developer-guide/snowpark-container-services/overview) - Container orchestration platform
Raw data
{
"_id": null,
"home_page": null,
"name": "snowflake-cli-nextflow",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "snowflake, cli, plugin, nextflow, workflow, bioinformatics, snowpark, container-services",
"author": null,
"author_email": "Haowei Yu <haowei.yu@snowflake.com>",
"download_url": "https://files.pythonhosted.org/packages/25/9f/23d1f1ebc95962a46c2c51a96935dd4a1bd7a45fc1042e341da4fde452e2/snowflake_cli_nextflow-0.0.1.tar.gz",
"platform": null,
"description": "# Snowflake CLI Nextflow Plugin\n\nA Snowflake CLI plugin that enables running [Nextflow](https://www.nextflow.io/) workflows directly in [Snowpark Container Services](https://docs.snowflake.com/en/developer-guide/snowpark-container-services/overview).\n\n## Overview\n\nThis plugin extends the Snowflake CLI with Nextflow workflow capabilities, allowing you to:\n\n- **Run Nextflow workflows at scale** using Snowflake's compute resources\n- **Stream live execution logs** through secure WebSocket connections\n- **Manage workflow execution** directly from your command line\n- **Leverage Snowflake's security and governance** for bioinformatics and data pipelines\n\n### Key Features\n\n- \ud83d\ude80 **Seamless Integration**: Run Nextflow workflows directly in Snowpark Container Services\n- \ud83d\udcca **Real-time Monitoring**: Stream live logs and execution status via WebSocket\n- \ud83d\udd12 **Enterprise Security**: Built-in Snowflake authentication and authorization\n- \ud83c\udfaf **Simple CLI Interface**: Familiar command-line experience for workflow management\n- \ud83d\udce6 **Flexible Configuration**: Support for Nextflow profiles and custom configurations\n\n## Getting Started\n\n### Prerequisites\n\n- All prerequisites of running snowflake-cli mentioned [here](https://docs.snowflake.com/en/developer-guide/snowflake-cli/installation/installation#requirements)\n- Access to a Snowflake account with Snowpark Container Services enabled\n- A Snowflake compute pool configured for container workloads\n- Nextflow binary correctly installed in your PATH\n\n### Installation\n\n1. **Install Snowflake CLI**\n ```bash\n pip install snowflake-cli\n ```\n\n2. **Install the Nextflow plugin**\n ```bash\n pip install snowflake-cli-nextflow\n ```\n\n3. **Configure Snowflake connection**\n ```bash\n snow connection add\n ```\n Follow the prompts to configure your Snowflake connection with appropriate credentials.\n\n4. **Enable the Nextflow plugin**\n ```bash\n snow plugin enable nextflow\n ```\n The `nextflow` command should now be available in the CLI.\n\n### Quick Start\n\n1. **Prepare your Nextflow project**\n \n Create a `nextflow.config` file in your project directory:\n ```groovy\n // nextflow.config\n snowflake {\n computePool = 'YOUR_COMPUTE_POOL'\n workDirStage = 'WORKDIR_STAGE'\n stageMounts = 'INPUT:/mnt/input,OUTPUT:/mnt/output'\n }\n ```\n\n2. **Upload nf-snowflake image into snowflake image repository**\n \n ```bash\n\n # Pull image from github container registry\n docker pull ghcr.io/snowflake-labs/nf-snowflake:latest\n \n # retag the image to snowflake registry\n docker tag ghcr.io/snowflake-labs/nf-snowflake:latest <snowflake_registry>\n\n # config snow nextflow to use the corresponding image\n snow nextflow config set -key nf_snowflake_image -value <snowflake_registry_image>\n ```\n\n3. **Run your workflow**\n ```bash\n snow nextflow run /path/to/your/nextflow-project -profile snowflake\n ```\n\n4. **Monitor execution**\n \n The plugin will automatically:\n - Upload your project to Snowflake\n - Submit the workflow to Snowpark Container Services\n - Stream live logs to your terminal\n\n## Configuration\n\n### Snowflake-specific Configuration\n\nYour `nextflow.config` must include the following Snowflake-specific settings:\n\n```groovy\nplugins { id 'nf-snowflake@0.6.3' }\n\nsnowflake {\n computePool = 'YOUR_COMPUTE_POOL'\n workDirStage = 'WORKDIR_STAGE'\n stageMounts = 'INPUT:/mnt/input,OUTPUT:/mnt/output'\n}\n```\n\n### Configuration Parameters\n\n- **`computePool`** (Required): The name of your Snowflake compute pool where the Nextflow containers will run. This compute pool must be configured with appropriate resources and permissions for container workloads.\n\n- **`workDirStage`** (Required): The Snowflake stage name where the plugin will upload your workflow files and store execution artifacts. This stage serves as the working directory for your Nextflow execution.\n\n- **`stageMounts`** (Optional): A comma-separated list of stage mounts in the format `STAGE_NAME:/mount/path`. Each mount makes a Snowflake stage available inside the container at the specified path. Use this to provide input data and collect output results.\n\n### Example with Multiple Stage Mounts\n\n```groovy\nsnowflake {\n computePool = 'NEXTFLOW_POOL'\n workDirStage = 'NF_WORKSPACE'\n stageMounts = 'RAW_DATA:/mnt/input,RESULTS:/mnt/output,REFERENCES:/mnt/ref'\n}\n```\n\n## Usage Examples\n\n### Basic Workflow Execution\n\n```bash\n# Run with default profile\nsnow nextflow run ./my-workflow\n\n# Run with specific profile\nsnow nextflow run ./my-workflow -profile snowflake\n\n# Check available commands\nsnow nextflow --help\n```\n\n## License\n\nThis project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.\n\n## Acknowledgments\n\n- [Nextflow](https://www.nextflow.io/) - The workflow management system\n- [Snowflake](https://www.snowflake.com/) - The cloud data platform\n- [Snowpark Container Services](https://docs.snowflake.com/en/developer-guide/snowpark-container-services/overview) - Container orchestration platform\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "A Snowflake CLI plugin that enables running Nextflow workflows directly in Snowpark Container Services",
"version": "0.0.1",
"project_urls": {
"Documentation": "https://github.com/snowflake-cli-nextflow-plugin#readme",
"Homepage": "https://github.com/snowflake-cli-nextflow-plugin",
"Issues": "https://github.com/snowflake-cli-nextflow-plugin/issues",
"Repository": "https://github.com/snowflake-cli-nextflow-plugin"
},
"split_keywords": [
"snowflake",
" cli",
" plugin",
" nextflow",
" workflow",
" bioinformatics",
" snowpark",
" container-services"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "2db8d5c05d62df2b7b7b36afd09c8a099add569c3ecbebd7528cdffc904164c6",
"md5": "60792c51b3d77c01d239b1c68ccf67ea",
"sha256": "4bc06fb3ad322b1b4217ad1b31416cf913c88650e2f21e954e3e239f6246113f"
},
"downloads": -1,
"filename": "snowflake_cli_nextflow-0.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "60792c51b3d77c01d239b1c68ccf67ea",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 18309,
"upload_time": "2025-07-14T04:25:46",
"upload_time_iso_8601": "2025-07-14T04:25:46.499204Z",
"url": "https://files.pythonhosted.org/packages/2d/b8/d5c05d62df2b7b7b36afd09c8a099add569c3ecbebd7528cdffc904164c6/snowflake_cli_nextflow-0.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "259f23d1f1ebc95962a46c2c51a96935dd4a1bd7a45fc1042e341da4fde452e2",
"md5": "9b33fcaa8fdaaafd568d7a2578b98ee9",
"sha256": "ca4e72e8246cd124e534680c03de1e9fa39974e750f5c04401a7cb3b3d2e96b5"
},
"downloads": -1,
"filename": "snowflake_cli_nextflow-0.0.1.tar.gz",
"has_sig": false,
"md5_digest": "9b33fcaa8fdaaafd568d7a2578b98ee9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 14962,
"upload_time": "2025-07-14T04:25:47",
"upload_time_iso_8601": "2025-07-14T04:25:47.819725Z",
"url": "https://files.pythonhosted.org/packages/25/9f/23d1f1ebc95962a46c2c51a96935dd4a1bd7a45fc1042e341da4fde452e2/snowflake_cli_nextflow-0.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-14 04:25:47",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "snowflake-cli-nextflow-plugin",
"github_project": "issues",
"github_not_found": true,
"lcname": "snowflake-cli-nextflow"
}
JSON
