# Vigil CLI
Observable, collaborative, reproducible science platform.
## ✨ What's New in v0.2.0
**Major update with genomics support, interactive setup, and cloud execution!**
### 🎯 Interactive Project Setup (P3)
```bash
vigil new genomics-starter my-project --interactive
```
- Smart prompts with email validation
- Automatic placeholder replacement
- **97% faster** (30+ min → <1 min)
### 🧬 Genomics Template (P0)
- Complete variant analysis workflow (filter → annotate → metrics)
- 40 synthetic variants across 10 cancer genes
- Ti/Tv ratio, depth stats, quality metrics
- Parquet format (5-10x compression)
### ☁️ Cloud & HPC Support (P1 + P3)
- **HPC**: SLURM, PBS, SGE profiles (400+ line guide)
- **Cloud**: AWS Batch, Google Cloud Life Sciences, Azure Batch, Kubernetes (600+ line guide)
- Cost optimization: 70-80% savings with spot instances
### 🗄️ Database Integration (P1)
- Ensembl, ClinVar, gnomAD clients
- Local caching for offline work
- Rate limiting and error handling
### 📊 Genomics Metrics Library (P2)
- Ti/Tv ratio calculation
- Het/Hom ratio
- Depth and quality statistics
- Scientifically validated
### 🔍 Version Management (P0)
- `vigil doctor` now detects version mismatches
- Automatic upgrade recommendations
**Upgrade now:** `uv tool upgrade vigil-core` or `pipx upgrade vigil-core`
**Full details:** See [CHANGELOG.md](../../CHANGELOG.md)
---
## Installation
### Core Installation
```bash
# Install vigil-core (cryptographic kernel)
pip install vigil-core
# Or install vigil-client (includes vigil-core + platform integration)
pip install vigil-client
```
### Package Structure
- **`vigil-core`**: Cryptographic kernel for reproducible science (like `git`)
- Local operations: `run`, `promote`, `verify`, `anchor`
- Offline-first, deterministic execution
- Receipt generation and cryptographic verification
- **`vigil-client`**: Platform bridge and collaboration layer (like `gh`)
- Platform integration: `client login`, `client push`, `client pull`
- Authentication and synchronization
- Artifact management and provenance linking
### Unified CLI
Both packages provide a unified `vigil` command:
```bash
# Core commands (available with vigil-core)
vigil run
vigil promote
vigil verify
# Platform commands (available with vigil-client)
vigil client login
vigil client push
vigil client pull
```
## Hybrid Architecture
`vigil-core` provides both a **CLI tool** and an **importable library**:
### CLI Usage (Command Line)
```bash
# Create a new project
vigil new imaging-starter my-project
cd my-project
# Run pipeline
vigil run --cores 4
# Generate receipt
vigil promote
# Verify receipt
vigil verify receipt.json
```
### Library Usage (Python API)
```python
from vigil.core import run_pipeline, generate_receipt, ReceiptManager
# Run pipeline programmatically
result = run_pipeline("python train.py", dry_run=False)
# Generate receipt
artifacts = [{"uri": "model.pkl", "checksum": "abc123", "kind": "model"}]
receipt = generate_receipt(artifacts, pipeline_cmd="python train.py")
# Manage receipts
manager = ReceiptManager()
manager.save_receipt(receipt)
```
### Core API Modules
- **`vigil.core.receipts`** - Receipt generation and verification
- **`vigil.core.pipeline`** - Pipeline execution and management
- **`vigil.core.signer`** - Cryptographic signing (Ed25519)
- **`vigil.core.policy`** - Policy enforcement and evaluation
- **`vigil.core.env`** - Environment capture and analysis
### Entry Points
- **CLI**: `vigil` command (installed globally)
- **Module**: `python -m vigil` (runs CLI)
- **Library**: `from vigil.core import ...` (programmatic access)
### MCP Server Installation
To use the MCP server (`vigil mcp serve`), install with the `mcp` extras:
```bash
# Install with MCP server support
pip install "vigil-core[mcp]"
# Or with vigil-client
pip install "vigil-client[mcp]"
```
The `mcp` extras include:
- `mcp>=1.1.0` - MCP protocol server
- `polars>=0.20.0` - Fast data frame library for data previews
## Quick Start
### Local Development (vigil-core)
```bash
# Create a new project from a template (with interactive setup)
vigil new genomics-starter my-project --interactive
cd my-project
# Install dependencies
uv sync
# Sync workspace spec
vigil spec --sync
# Preview the pipeline (dry-run)
vigil dev
# Execute the pipeline
vigil run --cores 4
# Promote artifacts to receipts
vigil promote
# Verify receipts
vigil verify app/code/receipts/*.json
```
### Platform Integration (vigil-client)
```bash
# Install vigil-client for platform integration
pip install vigil-client
# Authenticate with the platform
vigil client login
# Upload receipts and artifacts
vigil client push
# Search for artifacts
vigil client artifacts-search "genomics"
# Download an artifact
vigil client pull artifact-id
# Create provenance links
vigil client link artifact-1 artifact-2 --relation INPUT_OF
```
## Commands
### Core Commands (vigil-core)
#### Project Management
- `vigil new <template> [path]` - Create a new project from a template
- `vigil new <template> [path] --interactive` - Interactive setup with smart prompts (NEW in v0.2.0)
- `vigil new --list` - List available templates (now includes genomics-starter)
#### Development
- `vigil dev` - Dry-run the pipeline and preview the DAG
- `vigil build` - Execute the pipeline without promotion
- `vigil run` - Execute targets and optionally promote
- `vigil conformance` - Check outputs against golden baselines (project-specific, not all templates include this)
#### Receipt Management
- `vigil promote` - Generate Vigil receipts from artifacts
- `vigil anchor` - Create Merkle anchors for receipts
- `vigil url` - Print the vigil:// URL for the project
- `vigil verify` - Verify receipt checksums and attestations
#### Health & Maintenance
- `vigil doctor` - Run repository health checks (includes version mismatch detection)
- `vigil spec --sync` - Sync workspace.spec.json with vigil.yaml
- `vigil spec --dry-run` - Preview workspace spec changes
- `vigil version` - Show installed version (alias for `vigil --version`)
#### Documentation & Collaboration
- `vigil card init` - Create experiment or dataset cards
- `vigil card lint` - Validate card format and required fields
- `vigil notes new` - Create timestamped lab notebook entries
- `vigil notes index` - Regenerate lab notebook index
#### Workbench & AI
- `vigil ui bootstrap` - Generate Workbench configuration
- `vigil mcp serve` - Start the MCP server for assistants
- `vigil ai propose` - Generate auto-target suggestions
- `vigil ai apply` - Execute auto-target proposal
### Platform Commands (vigil-client)
#### Authentication
- `vigil client login` - Authenticate with Vigil platform
- `vigil client logout` - Log out and clear stored credentials
- `vigil client whoami` - Show current user information
#### Synchronization
- `vigil client push` - Upload receipts and artifacts to the platform
- `vigil client pull` - Download an artifact from the platform
- `vigil client link` - Create a provenance link between artifacts
#### Artifact Management
- `vigil client artifacts` - List artifacts in the platform
- `vigil client artifacts-get <id>` - Get detailed information about a specific artifact
- `vigil client artifacts-search <query>` - Search for artifacts by name or description
#### Configuration
- `vigil client config` - Show current configuration
- `vigil client config-set-project <id>` - Set default project ID
- `vigil client config-set-remote <url>` - Set remote platform URL
**Version Management**: The `vigil doctor` command now checks for version mismatches between local and global installations, helping prevent issues from outdated CLI versions.
## Templates
Available starter templates:
- **imaging-starter**: Full-featured imaging pipeline with sciencecast timeline
- **genomics-starter**: Comprehensive genomics workflow with variant filtering, annotation, and QC metrics (includes large file handling strategies)
- **minimal-starter**: Smallest viable template for getting started
### Genomics Template Features
The `genomics-starter` template demonstrates:
- Variant quality filtering (PHRED scores, depth, allele frequency)
- Functional annotation with gene and pathway information
- Comprehensive QC metrics (Ti/Tv ratio, depth distribution, pathogenicity)
- Large file handling strategies for NGS data (BAM, VCF, CRAM)
- Data handles with offline fallbacks for disconnected development
- Parquet format for efficient variant storage
## Documentation
- [Getting Started](../../docs/getting-started.md)
- [CLI Reference](../../docs/cli-reference.md)
- [Creating Custom Templates](../../docs/creating-starters.md)
## License
Apache-2.0
Raw data
{
"_id": null,
"home_page": null,
"name": "vigil-core",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.11",
"maintainer_email": null,
"keywords": "cryptography, pipeline, provenance, receipts, reproducible, science",
"author": "Science Abundance",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/23/c6/1a92136642a7773689307869a7afbff45903d5c57850b0d4428cdadd22d5/vigil_core-0.3.10.tar.gz",
"platform": null,
"description": "# Vigil CLI\n\nObservable, collaborative, reproducible science platform.\n\n## \u2728 What's New in v0.2.0\n\n**Major update with genomics support, interactive setup, and cloud execution!**\n\n### \ud83c\udfaf Interactive Project Setup (P3)\n```bash\nvigil new genomics-starter my-project --interactive\n```\n- Smart prompts with email validation\n- Automatic placeholder replacement\n- **97% faster** (30+ min \u2192 <1 min)\n\n### \ud83e\uddec Genomics Template (P0)\n- Complete variant analysis workflow (filter \u2192 annotate \u2192 metrics)\n- 40 synthetic variants across 10 cancer genes\n- Ti/Tv ratio, depth stats, quality metrics\n- Parquet format (5-10x compression)\n\n### \u2601\ufe0f Cloud & HPC Support (P1 + P3)\n- **HPC**: SLURM, PBS, SGE profiles (400+ line guide)\n- **Cloud**: AWS Batch, Google Cloud Life Sciences, Azure Batch, Kubernetes (600+ line guide)\n- Cost optimization: 70-80% savings with spot instances\n\n### \ud83d\uddc4\ufe0f Database Integration (P1)\n- Ensembl, ClinVar, gnomAD clients\n- Local caching for offline work\n- Rate limiting and error handling\n\n### \ud83d\udcca Genomics Metrics Library (P2)\n- Ti/Tv ratio calculation\n- Het/Hom ratio\n- Depth and quality statistics\n- Scientifically validated\n\n### \ud83d\udd0d Version Management (P0)\n- `vigil doctor` now detects version mismatches\n- Automatic upgrade recommendations\n\n**Upgrade now:** `uv tool upgrade vigil-core` or `pipx upgrade vigil-core`\n\n**Full details:** See [CHANGELOG.md](../../CHANGELOG.md)\n\n---\n\n## Installation\n\n### Core Installation\n\n```bash\n# Install vigil-core (cryptographic kernel)\npip install vigil-core\n\n# Or install vigil-client (includes vigil-core + platform integration)\npip install vigil-client\n```\n\n### Package Structure\n\n- **`vigil-core`**: Cryptographic kernel for reproducible science (like `git`)\n - Local operations: `run`, `promote`, `verify`, `anchor`\n - Offline-first, deterministic execution\n - Receipt generation and cryptographic verification\n\n- **`vigil-client`**: Platform bridge and collaboration layer (like `gh`)\n - Platform integration: `client login`, `client push`, `client pull`\n - Authentication and synchronization\n - Artifact management and provenance linking\n\n### Unified CLI\n\nBoth packages provide a unified `vigil` command:\n\n```bash\n# Core commands (available with vigil-core)\nvigil run\nvigil promote\nvigil verify\n\n# Platform commands (available with vigil-client)\nvigil client login\nvigil client push\nvigil client pull\n```\n\n## Hybrid Architecture\n\n`vigil-core` provides both a **CLI tool** and an **importable library**:\n\n### CLI Usage (Command Line)\n\n```bash\n# Create a new project\nvigil new imaging-starter my-project\ncd my-project\n\n# Run pipeline\nvigil run --cores 4\n\n# Generate receipt\nvigil promote\n\n# Verify receipt\nvigil verify receipt.json\n```\n\n### Library Usage (Python API)\n\n```python\nfrom vigil.core import run_pipeline, generate_receipt, ReceiptManager\n\n# Run pipeline programmatically\nresult = run_pipeline(\"python train.py\", dry_run=False)\n\n# Generate receipt\nartifacts = [{\"uri\": \"model.pkl\", \"checksum\": \"abc123\", \"kind\": \"model\"}]\nreceipt = generate_receipt(artifacts, pipeline_cmd=\"python train.py\")\n\n# Manage receipts\nmanager = ReceiptManager()\nmanager.save_receipt(receipt)\n```\n\n### Core API Modules\n\n- **`vigil.core.receipts`** - Receipt generation and verification\n- **`vigil.core.pipeline`** - Pipeline execution and management\n- **`vigil.core.signer`** - Cryptographic signing (Ed25519)\n- **`vigil.core.policy`** - Policy enforcement and evaluation\n- **`vigil.core.env`** - Environment capture and analysis\n\n### Entry Points\n\n- **CLI**: `vigil` command (installed globally)\n- **Module**: `python -m vigil` (runs CLI)\n- **Library**: `from vigil.core import ...` (programmatic access)\n\n### MCP Server Installation\n\nTo use the MCP server (`vigil mcp serve`), install with the `mcp` extras:\n\n```bash\n# Install with MCP server support\npip install \"vigil-core[mcp]\"\n\n# Or with vigil-client\npip install \"vigil-client[mcp]\"\n```\n\nThe `mcp` extras include:\n- `mcp>=1.1.0` - MCP protocol server\n- `polars>=0.20.0` - Fast data frame library for data previews\n\n## Quick Start\n\n### Local Development (vigil-core)\n\n```bash\n# Create a new project from a template (with interactive setup)\nvigil new genomics-starter my-project --interactive\ncd my-project\n\n# Install dependencies\nuv sync\n\n# Sync workspace spec\nvigil spec --sync\n\n# Preview the pipeline (dry-run)\nvigil dev\n\n# Execute the pipeline\nvigil run --cores 4\n\n# Promote artifacts to receipts\nvigil promote\n\n# Verify receipts\nvigil verify app/code/receipts/*.json\n```\n\n### Platform Integration (vigil-client)\n\n```bash\n# Install vigil-client for platform integration\npip install vigil-client\n\n# Authenticate with the platform\nvigil client login\n\n# Upload receipts and artifacts\nvigil client push\n\n# Search for artifacts\nvigil client artifacts-search \"genomics\"\n\n# Download an artifact\nvigil client pull artifact-id\n\n# Create provenance links\nvigil client link artifact-1 artifact-2 --relation INPUT_OF\n```\n\n## Commands\n\n### Core Commands (vigil-core)\n\n#### Project Management\n- `vigil new <template> [path]` - Create a new project from a template\n- `vigil new <template> [path] --interactive` - Interactive setup with smart prompts (NEW in v0.2.0)\n- `vigil new --list` - List available templates (now includes genomics-starter)\n\n#### Development\n- `vigil dev` - Dry-run the pipeline and preview the DAG\n- `vigil build` - Execute the pipeline without promotion\n- `vigil run` - Execute targets and optionally promote\n- `vigil conformance` - Check outputs against golden baselines (project-specific, not all templates include this)\n\n#### Receipt Management\n- `vigil promote` - Generate Vigil receipts from artifacts\n- `vigil anchor` - Create Merkle anchors for receipts\n- `vigil url` - Print the vigil:// URL for the project\n- `vigil verify` - Verify receipt checksums and attestations\n\n#### Health & Maintenance\n- `vigil doctor` - Run repository health checks (includes version mismatch detection)\n- `vigil spec --sync` - Sync workspace.spec.json with vigil.yaml\n- `vigil spec --dry-run` - Preview workspace spec changes\n- `vigil version` - Show installed version (alias for `vigil --version`)\n\n#### Documentation & Collaboration\n- `vigil card init` - Create experiment or dataset cards\n- `vigil card lint` - Validate card format and required fields\n- `vigil notes new` - Create timestamped lab notebook entries\n- `vigil notes index` - Regenerate lab notebook index\n\n#### Workbench & AI\n- `vigil ui bootstrap` - Generate Workbench configuration\n- `vigil mcp serve` - Start the MCP server for assistants\n- `vigil ai propose` - Generate auto-target suggestions\n- `vigil ai apply` - Execute auto-target proposal\n\n### Platform Commands (vigil-client)\n\n#### Authentication\n- `vigil client login` - Authenticate with Vigil platform\n- `vigil client logout` - Log out and clear stored credentials\n- `vigil client whoami` - Show current user information\n\n#### Synchronization\n- `vigil client push` - Upload receipts and artifacts to the platform\n- `vigil client pull` - Download an artifact from the platform\n- `vigil client link` - Create a provenance link between artifacts\n\n#### Artifact Management\n- `vigil client artifacts` - List artifacts in the platform\n- `vigil client artifacts-get <id>` - Get detailed information about a specific artifact\n- `vigil client artifacts-search <query>` - Search for artifacts by name or description\n\n#### Configuration\n- `vigil client config` - Show current configuration\n- `vigil client config-set-project <id>` - Set default project ID\n- `vigil client config-set-remote <url>` - Set remote platform URL\n\n**Version Management**: The `vigil doctor` command now checks for version mismatches between local and global installations, helping prevent issues from outdated CLI versions.\n\n## Templates\n\nAvailable starter templates:\n\n- **imaging-starter**: Full-featured imaging pipeline with sciencecast timeline\n- **genomics-starter**: Comprehensive genomics workflow with variant filtering, annotation, and QC metrics (includes large file handling strategies)\n- **minimal-starter**: Smallest viable template for getting started\n\n### Genomics Template Features\n\nThe `genomics-starter` template demonstrates:\n- Variant quality filtering (PHRED scores, depth, allele frequency)\n- Functional annotation with gene and pathway information\n- Comprehensive QC metrics (Ti/Tv ratio, depth distribution, pathogenicity)\n- Large file handling strategies for NGS data (BAM, VCF, CRAM)\n- Data handles with offline fallbacks for disconnected development\n- Parquet format for efficient variant storage\n\n## Documentation\n\n- [Getting Started](../../docs/getting-started.md)\n- [CLI Reference](../../docs/cli-reference.md)\n- [Creating Custom Templates](../../docs/creating-starters.md)\n\n## License\n\nApache-2.0\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Vigil Core: Cryptographic kernel for reproducible science",
"version": "0.3.10",
"project_urls": null,
"split_keywords": [
"cryptography",
" pipeline",
" provenance",
" receipts",
" reproducible",
" science"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "43048e6c53ba5071ac2bfc2887f1292e453044fdd28f6c39449f26b6f8ce2de1",
"md5": "560041b5cbf2f58356bd17c8ae7f53e6",
"sha256": "598b630ee047e12993fa5f263e3582d6a54e3e2e416a6b63a68762a91d29740d"
},
"downloads": -1,
"filename": "vigil_core-0.3.10-py3-none-any.whl",
"has_sig": false,
"md5_digest": "560041b5cbf2f58356bd17c8ae7f53e6",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11",
"size": 271484,
"upload_time": "2025-10-22T00:42:47",
"upload_time_iso_8601": "2025-10-22T00:42:47.302214Z",
"url": "https://files.pythonhosted.org/packages/43/04/8e6c53ba5071ac2bfc2887f1292e453044fdd28f6c39449f26b6f8ce2de1/vigil_core-0.3.10-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "23c61a92136642a7773689307869a7afbff45903d5c57850b0d4428cdadd22d5",
"md5": "52cd9b92a22360b2f9e9d7112c6794b4",
"sha256": "9ba42c45e631915c256fa49f39cc4430ba9a784bf4bd88cf36d53b2a2be2aac2"
},
"downloads": -1,
"filename": "vigil_core-0.3.10.tar.gz",
"has_sig": false,
"md5_digest": "52cd9b92a22360b2f9e9d7112c6794b4",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11",
"size": 254215,
"upload_time": "2025-10-22T00:42:48",
"upload_time_iso_8601": "2025-10-22T00:42:48.659570Z",
"url": "https://files.pythonhosted.org/packages/23/c6/1a92136642a7773689307869a7afbff45903d5c57850b0d4428cdadd22d5/vigil_core-0.3.10.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-22 00:42:48",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "vigil-core"
}
JSON