# Snowflake CLI Nextflow Plugin
A Snowflake CLI plugin that enables running [Nextflow](https://www.nextflow.io/) workflows directly in [Snowpark Container Services](https://docs.snowflake.com/en/developer-guide/snowpark-container-services/overview).
## Overview
This plugin extends the Snowflake CLI with Nextflow workflow capabilities, allowing you to:
- **Run Nextflow workflows at scale** using Snowflake's compute resources
- **Stream live execution logs** through secure WebSocket connections
- **Manage workflow execution** directly from your command line
- **Leverage Snowflake's security and governance** for bioinformatics and data pipelines
### Key Features
- 🚀 **Seamless Integration**: Run Nextflow workflows directly in Snowpark Container Services
- 📊 **Real-time Monitoring**: Stream live logs and execution status via WebSocket
- 🔒 **Enterprise Security**: Built-in Snowflake authentication and authorization
- 🎯 **Simple CLI Interface**: Familiar command-line experience for workflow management
- 📦 **Flexible Configuration**: Support for Nextflow profiles and custom configurations
## Getting Started
### Prerequisites
- All prerequisites of running snowflake-cli mentioned [here](https://docs.snowflake.com/en/developer-guide/snowflake-cli/installation/installation#requirements)
- Access to a Snowflake account with Snowpark Container Services enabled
- A Snowflake compute pool configured for container workloads
- Nextflow binary correctly installed in your PATH
### Installation
1. **Install Snowflake CLI**
```bash
pip install snowflake-cli
```
2. **Install the Nextflow plugin**
```bash
pip install snowflake-cli-nextflow
```
3. **Configure Snowflake connection**
```bash
snow connection add
```
Follow the prompts to configure your Snowflake connection with appropriate credentials.
4. **Enable the Nextflow plugin**
```bash
snow plugin enable nextflow
```
The `nextflow` command should now be available in the CLI.
### Quick Start
1. **Prepare your Nextflow project**
Create a `nextflow.config` file in your project directory:
```groovy
// nextflow.config
snowflake {
computePool = 'YOUR_COMPUTE_POOL'
workDirStage = 'WORKDIR_STAGE'
stageMounts = 'INPUT:/mnt/input,OUTPUT:/mnt/output'
}
```
2. **Upload nf-snowflake image into snowflake image repository**
```bash
snow nextflow image push --source ghcr.io/snowflake-labs/nf-snowflake:0.7.1 --target nf_repo --update-config
```
3. **Run your workflow**
```bash
snow nextflow run /path/to/your/nextflow-project -profile snowflake
```
4. **Monitor execution**
The plugin will automatically:
- Upload your project to Snowflake
- Submit the workflow to Snowpark Container Services
- Stream live logs to your terminal
## Configuration
### Snowflake-specific Configuration
Your `nextflow.config` must include the following Snowflake-specific settings:
```groovy
plugins { id 'nf-snowflake@0.7.1' }
snowflake {
computePool = 'YOUR_COMPUTE_POOL'
workDirStage = 'WORKDIR_STAGE'
stageMounts = 'INPUT:/mnt/input,OUTPUT:/mnt/output'
}
```
### Configuration Parameters
- **`computePool`** (Required): The name of your Snowflake compute pool where the Nextflow containers will run. This compute pool must be configured with appropriate resources and permissions for container workloads.
- **`workDirStage`** (Required): The Snowflake stage name where the plugin will upload your workflow files and store execution artifacts. This stage serves as the working directory for your Nextflow execution.
- **`stageMounts`** (Optional): A comma-separated list of stage mounts in the format `STAGE_NAME:/mount/path`. Each mount makes a Snowflake stage available inside the container at the specified path. Use this to provide input data and collect output results.
### Example with Multiple Stage Mounts
```groovy
snowflake {
computePool = 'NEXTFLOW_POOL'
workDirStage = 'NF_WORKSPACE'
stageMounts = 'RAW_DATA:/mnt/input,RESULTS:/mnt/output,REFERENCES:/mnt/ref'
}
```
## Usage Examples
### Basic Workflow Execution
```bash
# Run with default profile
snow nextflow run ./my-workflow
# Run with specific profile
snow nextflow run ./my-workflow -profile snowflake
# Check available commands
snow nextflow --help
```
## License
This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.
## Acknowledgments
- [Nextflow](https://www.nextflow.io/) - The workflow management system
- [Snowflake](https://www.snowflake.com/) - The cloud data platform
- [Snowpark Container Services](https://docs.snowflake.com/en/developer-guide/snowpark-container-services/overview) - Container orchestration platform
Raw data
{
"_id": null,
"home_page": null,
"name": "snowflake-cli-nextflow",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "snowflake, cli, plugin, nextflow, workflow, bioinformatics, snowpark, container-services",
"author": null,
"author_email": "Haowei Yu <haowei.yu@snowflake.com>",
"download_url": "https://files.pythonhosted.org/packages/7b/e9/7707530aa98306cda97cc35e372673371104d5e8df1fb07362db89864378/snowflake_cli_nextflow-0.1.0.tar.gz",
"platform": null,
"description": "# Snowflake CLI Nextflow Plugin\n\nA Snowflake CLI plugin that enables running [Nextflow](https://www.nextflow.io/) workflows directly in [Snowpark Container Services](https://docs.snowflake.com/en/developer-guide/snowpark-container-services/overview).\n\n## Overview\n\nThis plugin extends the Snowflake CLI with Nextflow workflow capabilities, allowing you to:\n\n- **Run Nextflow workflows at scale** using Snowflake's compute resources\n- **Stream live execution logs** through secure WebSocket connections\n- **Manage workflow execution** directly from your command line\n- **Leverage Snowflake's security and governance** for bioinformatics and data pipelines\n\n### Key Features\n\n- \ud83d\ude80 **Seamless Integration**: Run Nextflow workflows directly in Snowpark Container Services\n- \ud83d\udcca **Real-time Monitoring**: Stream live logs and execution status via WebSocket\n- \ud83d\udd12 **Enterprise Security**: Built-in Snowflake authentication and authorization\n- \ud83c\udfaf **Simple CLI Interface**: Familiar command-line experience for workflow management\n- \ud83d\udce6 **Flexible Configuration**: Support for Nextflow profiles and custom configurations\n\n## Getting Started\n\n### Prerequisites\n\n- All prerequisites of running snowflake-cli mentioned [here](https://docs.snowflake.com/en/developer-guide/snowflake-cli/installation/installation#requirements)\n- Access to a Snowflake account with Snowpark Container Services enabled\n- A Snowflake compute pool configured for container workloads\n- Nextflow binary correctly installed in your PATH\n\n### Installation\n\n1. **Install Snowflake CLI**\n ```bash\n pip install snowflake-cli\n ```\n\n2. **Install the Nextflow plugin**\n ```bash\n pip install snowflake-cli-nextflow\n ```\n\n3. **Configure Snowflake connection**\n ```bash\n snow connection add\n ```\n Follow the prompts to configure your Snowflake connection with appropriate credentials.\n\n4. **Enable the Nextflow plugin**\n ```bash\n snow plugin enable nextflow\n ```\n The `nextflow` command should now be available in the CLI.\n\n### Quick Start\n\n1. **Prepare your Nextflow project**\n \n Create a `nextflow.config` file in your project directory:\n ```groovy\n // nextflow.config\n snowflake {\n computePool = 'YOUR_COMPUTE_POOL'\n workDirStage = 'WORKDIR_STAGE'\n stageMounts = 'INPUT:/mnt/input,OUTPUT:/mnt/output'\n }\n ```\n\n2. **Upload nf-snowflake image into snowflake image repository**\n \n ```bash\n snow nextflow image push --source ghcr.io/snowflake-labs/nf-snowflake:0.7.1 --target nf_repo --update-config\n ```\n\n3. **Run your workflow**\n ```bash\n snow nextflow run /path/to/your/nextflow-project -profile snowflake\n ```\n\n4. **Monitor execution**\n \n The plugin will automatically:\n - Upload your project to Snowflake\n - Submit the workflow to Snowpark Container Services\n - Stream live logs to your terminal\n\n## Configuration\n\n### Snowflake-specific Configuration\n\nYour `nextflow.config` must include the following Snowflake-specific settings:\n\n```groovy\nplugins { id 'nf-snowflake@0.7.1' }\n\nsnowflake {\n computePool = 'YOUR_COMPUTE_POOL'\n workDirStage = 'WORKDIR_STAGE'\n stageMounts = 'INPUT:/mnt/input,OUTPUT:/mnt/output'\n}\n```\n\n### Configuration Parameters\n\n- **`computePool`** (Required): The name of your Snowflake compute pool where the Nextflow containers will run. This compute pool must be configured with appropriate resources and permissions for container workloads.\n\n- **`workDirStage`** (Required): The Snowflake stage name where the plugin will upload your workflow files and store execution artifacts. This stage serves as the working directory for your Nextflow execution.\n\n- **`stageMounts`** (Optional): A comma-separated list of stage mounts in the format `STAGE_NAME:/mount/path`. Each mount makes a Snowflake stage available inside the container at the specified path. Use this to provide input data and collect output results.\n\n### Example with Multiple Stage Mounts\n\n```groovy\nsnowflake {\n computePool = 'NEXTFLOW_POOL'\n workDirStage = 'NF_WORKSPACE'\n stageMounts = 'RAW_DATA:/mnt/input,RESULTS:/mnt/output,REFERENCES:/mnt/ref'\n}\n```\n\n## Usage Examples\n\n### Basic Workflow Execution\n\n```bash\n# Run with default profile\nsnow nextflow run ./my-workflow\n\n# Run with specific profile\nsnow nextflow run ./my-workflow -profile snowflake\n\n# Check available commands\nsnow nextflow --help\n```\n\n## License\n\nThis project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.\n\n## Acknowledgments\n\n- [Nextflow](https://www.nextflow.io/) - The workflow management system\n- [Snowflake](https://www.snowflake.com/) - The cloud data platform\n- [Snowpark Container Services](https://docs.snowflake.com/en/developer-guide/snowpark-container-services/overview) - Container orchestration platform\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "A Snowflake CLI plugin that enables running Nextflow workflows directly in Snowpark Container Services",
"version": "0.1.0",
"project_urls": {
"Documentation": "https://github.com/Snowflake-Labs/snowflake-cli-nextflow",
"Homepage": "https://github.com/Snowflake-Labs/snowflake-cli-nextflow",
"Issues": "https://github.com/Snowflake-Labs/snowflake-cli-nextflow/issues",
"Repository": "https://github.com/Snowflake-Labs/snowflake-cli-nextflow"
},
"split_keywords": [
"snowflake",
" cli",
" plugin",
" nextflow",
" workflow",
" bioinformatics",
" snowpark",
" container-services"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "e6a3dcaaa9e1d0f821cd8bdc3257e87200187d41776420a456ee349ae78bad1b",
"md5": "52fe954be0b139fd3da57c5efa523490",
"sha256": "6ae1c3f5817ac34f8f8ac582a9a39752656089bd01036cd35a2fcea8c1c587ce"
},
"downloads": -1,
"filename": "snowflake_cli_nextflow-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "52fe954be0b139fd3da57c5efa523490",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 22079,
"upload_time": "2025-08-08T17:23:33",
"upload_time_iso_8601": "2025-08-08T17:23:33.323980Z",
"url": "https://files.pythonhosted.org/packages/e6/a3/dcaaa9e1d0f821cd8bdc3257e87200187d41776420a456ee349ae78bad1b/snowflake_cli_nextflow-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "7be97707530aa98306cda97cc35e372673371104d5e8df1fb07362db89864378",
"md5": "674b09b584aafff19a417f6d105aa9e2",
"sha256": "a7e58fc855bd9b1980fc1f876c51e9842077d47cc5c0f9bcae22ebe23f900b04"
},
"downloads": -1,
"filename": "snowflake_cli_nextflow-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "674b09b584aafff19a417f6d105aa9e2",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 20426,
"upload_time": "2025-08-08T17:23:34",
"upload_time_iso_8601": "2025-08-08T17:23:34.796947Z",
"url": "https://files.pythonhosted.org/packages/7b/e9/7707530aa98306cda97cc35e372673371104d5e8df1fb07362db89864378/snowflake_cli_nextflow-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-08 17:23:34",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Snowflake-Labs",
"github_project": "snowflake-cli-nextflow",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "snowflake-cli-nextflow"
}
JSON
