snowflake-data-validation


Namesnowflake-data-validation JSON
Version 0.0.7 PyPI version JSON
download
home_pageNone
SummarySnowflake Data Validation
upload_time2025-07-11 17:09:52
maintainerNone
docs_urlNone
authorNone
requires_python<3.13,>=3.9
licenseApache License, Version 2.0
keywords snowflake analytics cloud data database db validation
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # Snowflake Data Validation

[![License Apache-2.0](https://img.shields.io/:license-Apache%202-brightgreen.svg)](http://www.apache.org/licenses/LICENSE-2.0.txt)
[![Python](https://img.shields.io/badge/python-3.9%2B-blue)](https://www.python.org/downloads/)

**Snowflake Data Validation** is a command-line tool and Python library for validating data migrations and ensuring data quality between source and target databases, with a focus on Snowflake and SQL Server.

---

##### This package is in Private Preview.

---

## 🚀 Features

- **Multi-level validation**: schema, statistical metrics, and data integrity.
- **Database connectors**: support for SQL Server and Snowflake.
- **User-friendly CLI**: commands for automation and orchestration.
- **Flexible configuration**: YAML-based validation workflows.
- **Detailed reporting**: comprehensive reports and progress tracking.
- **Extensible**: architecture ready for more database engines.

---

## 📦 Installation

```bash
pip install snowflake-data-validation
```

For SQL Server support:

```bash
pip install "snowflake-data-validation[sqlserver]"
```

For development and testing:

```bash
pip install "snowflake-data-validation[all]"
```

---

## ⚡ Quick Start

Run a validation from SQL Server to Snowflake:

```bash
snowflake-data-validation sqlserver run-validation --data-validation-config-file ./config/conf.yaml
```

Or using the short alias:

```bash
sdv sqlserver run-validation --data-validation-config-file ./config/conf.yaml
```

---

## 🛠️ Configuration

Create a YAML file to define your validation workflow:

```yaml
source_platform: SqlServer
target_platform: Snowflake
output_directory_path: /path/to/output
parallelization: false

source_connection:
  mode: credentials
  host: "server"
  port: 1433
  username: "user"
  password: "password"
  database: "db"

target_connection:
  mode: name
  name: "SnowflakeConnection"

validation_configuration:
  schema_validation: true
  metrics_validation: true
  row_validation: false

comparison_configuration:
  tolerance: 0.01

tables:
  - fully_qualified_name: database.schema.table1
    use_column_selection_as_exclude_list: false
    column_selection_list:
      - column1
      - column2
```

See the documentation for more advanced configuration examples.

---

## 🏗️ Architecture

- **CLI**: `main_cli.py`, `sqlserver_cli.py`, `snowflake_cli.py`
- **Connectors**: `connector/`
- **Extractors**: `extractor/`
- **Validation**: `validation/`
- **Configuration**: `configuration/`
- **Orchestrator**: `comparison_orchestrator.py`

Project structure:
```
snowflake-data-validation/
├── src/snowflake/snowflake_data_validation/
│   ├── main_cli.py
│   ├── sqlserver/
│   ├── snowflake/
│   ├── connector/
│   ├── extractor/
│   ├── validation/
│   ├── configuration/
│   ├── utils/
│   └── comparison_orchestrator.py
├── docs/
├── tests/
└── config_files/
```

---

## 📊 Reports

- Schema validation results
- Statistical comparison metrics
- Detailed error logs and recommendations

---

## 🤝 Contributing

We welcome contributions! See our [Contributing Guide](../../CONTRIBUTING.md) for details on how to collaborate, set up your development environment, and submit PRs.

---

## 📄 License

This project is licensed under the Apache License 2.0. See the [LICENSE](../../LICENSE) file for details.

---

## 🆘 Support

- **Documentation**: [Full documentation](https://github.com/snowflakedb/migrations-data-validation)
- **Issues**: [GitHub Issues](https://github.com/snowflakedb/migrations-data-validation/issues)

---

**Developed with ❄️ by Snowflake**

            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "snowflake-data-validation",
    "maintainer": null,
    "docs_url": null,
    "requires_python": "<3.13,>=3.9",
    "maintainer_email": null,
    "keywords": "Snowflake, analytics, cloud, data, database, db, validation",
    "author": null,
    "author_email": "\"Snowflake, Inc.\" <snowflake-python-libraries-dl@snowflake.com>",
    "download_url": "https://files.pythonhosted.org/packages/be/00/6f2ebaa337b8d9d18612ec02d79baf716bb532cf23b236841f36133a1da3/snowflake_data_validation-0.0.7.tar.gz",
    "platform": null,
    "description": "# Snowflake Data Validation\n\n[![License Apache-2.0](https://img.shields.io/:license-Apache%202-brightgreen.svg)](http://www.apache.org/licenses/LICENSE-2.0.txt)\n[![Python](https://img.shields.io/badge/python-3.9%2B-blue)](https://www.python.org/downloads/)\n\n**Snowflake Data Validation** is a command-line tool and Python library for validating data migrations and ensuring data quality between source and target databases, with a focus on Snowflake and SQL Server.\n\n---\n\n##### This package is in Private Preview.\n\n---\n\n## \ud83d\ude80 Features\n\n- **Multi-level validation**: schema, statistical metrics, and data integrity.\n- **Database connectors**: support for SQL Server and Snowflake.\n- **User-friendly CLI**: commands for automation and orchestration.\n- **Flexible configuration**: YAML-based validation workflows.\n- **Detailed reporting**: comprehensive reports and progress tracking.\n- **Extensible**: architecture ready for more database engines.\n\n---\n\n## \ud83d\udce6 Installation\n\n```bash\npip install snowflake-data-validation\n```\n\nFor SQL Server support:\n\n```bash\npip install \"snowflake-data-validation[sqlserver]\"\n```\n\nFor development and testing:\n\n```bash\npip install \"snowflake-data-validation[all]\"\n```\n\n---\n\n## \u26a1 Quick Start\n\nRun a validation from SQL Server to Snowflake:\n\n```bash\nsnowflake-data-validation sqlserver run-validation --data-validation-config-file ./config/conf.yaml\n```\n\nOr using the short alias:\n\n```bash\nsdv sqlserver run-validation --data-validation-config-file ./config/conf.yaml\n```\n\n---\n\n## \ud83d\udee0\ufe0f Configuration\n\nCreate a YAML file to define your validation workflow:\n\n```yaml\nsource_platform: SqlServer\ntarget_platform: Snowflake\noutput_directory_path: /path/to/output\nparallelization: false\n\nsource_connection:\n  mode: credentials\n  host: \"server\"\n  port: 1433\n  username: \"user\"\n  password: \"password\"\n  database: \"db\"\n\ntarget_connection:\n  mode: name\n  name: \"SnowflakeConnection\"\n\nvalidation_configuration:\n  schema_validation: true\n  metrics_validation: true\n  row_validation: false\n\ncomparison_configuration:\n  tolerance: 0.01\n\ntables:\n  - fully_qualified_name: database.schema.table1\n    use_column_selection_as_exclude_list: false\n    column_selection_list:\n      - column1\n      - column2\n```\n\nSee the documentation for more advanced configuration examples.\n\n---\n\n## \ud83c\udfd7\ufe0f Architecture\n\n- **CLI**: `main_cli.py`, `sqlserver_cli.py`, `snowflake_cli.py`\n- **Connectors**: `connector/`\n- **Extractors**: `extractor/`\n- **Validation**: `validation/`\n- **Configuration**: `configuration/`\n- **Orchestrator**: `comparison_orchestrator.py`\n\nProject structure:\n```\nsnowflake-data-validation/\n\u251c\u2500\u2500 src/snowflake/snowflake_data_validation/\n\u2502   \u251c\u2500\u2500 main_cli.py\n\u2502   \u251c\u2500\u2500 sqlserver/\n\u2502   \u251c\u2500\u2500 snowflake/\n\u2502   \u251c\u2500\u2500 connector/\n\u2502   \u251c\u2500\u2500 extractor/\n\u2502   \u251c\u2500\u2500 validation/\n\u2502   \u251c\u2500\u2500 configuration/\n\u2502   \u251c\u2500\u2500 utils/\n\u2502   \u2514\u2500\u2500 comparison_orchestrator.py\n\u251c\u2500\u2500 docs/\n\u251c\u2500\u2500 tests/\n\u2514\u2500\u2500 config_files/\n```\n\n---\n\n## \ud83d\udcca Reports\n\n- Schema validation results\n- Statistical comparison metrics\n- Detailed error logs and recommendations\n\n---\n\n## \ud83e\udd1d Contributing\n\nWe welcome contributions! See our [Contributing Guide](../../CONTRIBUTING.md) for details on how to collaborate, set up your development environment, and submit PRs.\n\n---\n\n## \ud83d\udcc4 License\n\nThis project is licensed under the Apache License 2.0. See the [LICENSE](../../LICENSE) file for details.\n\n---\n\n## \ud83c\udd98 Support\n\n- **Documentation**: [Full documentation](https://github.com/snowflakedb/migrations-data-validation)\n- **Issues**: [GitHub Issues](https://github.com/snowflakedb/migrations-data-validation/issues)\n\n---\n\n**Developed with \u2744\ufe0f by Snowflake**\n",
    "bugtrack_url": null,
    "license": "Apache License, Version 2.0",
    "summary": "Snowflake Data Validation",
    "version": "0.0.7",
    "project_urls": {
        "Bug Tracker": "https://github.com/snowflakedb/migrations-data-validation/issues",
        "Source code": "https://github.com/snowflakedb/migrations-data-validation/"
    },
    "split_keywords": [
        "snowflake",
        " analytics",
        " cloud",
        " data",
        " database",
        " db",
        " validation"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "3268957b03025b67320e91ea05921946747cac241428a69597aa2a7dfaad0b13",
                "md5": "776b392bdec029b8490ff9340b18c709",
                "sha256": "ec227c401ebf62893083f293ab7e9dde8a80bf4705f90ebe6ac8d42d7b49a766"
            },
            "downloads": -1,
            "filename": "snowflake_data_validation-0.0.7-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "776b392bdec029b8490ff9340b18c709",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": "<3.13,>=3.9",
            "size": 186587,
            "upload_time": "2025-07-11T17:09:50",
            "upload_time_iso_8601": "2025-07-11T17:09:50.908096Z",
            "url": "https://files.pythonhosted.org/packages/32/68/957b03025b67320e91ea05921946747cac241428a69597aa2a7dfaad0b13/snowflake_data_validation-0.0.7-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "be006f2ebaa337b8d9d18612ec02d79baf716bb532cf23b236841f36133a1da3",
                "md5": "93b9aac8111940e3eb3a8b88d92ddd0e",
                "sha256": "48a4ae12ef13041ff138cefef70ec1a3e7eafab115be0061f994b17ee5005d62"
            },
            "downloads": -1,
            "filename": "snowflake_data_validation-0.0.7.tar.gz",
            "has_sig": false,
            "md5_digest": "93b9aac8111940e3eb3a8b88d92ddd0e",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": "<3.13,>=3.9",
            "size": 154928,
            "upload_time": "2025-07-11T17:09:52",
            "upload_time_iso_8601": "2025-07-11T17:09:52.402576Z",
            "url": "https://files.pythonhosted.org/packages/be/00/6f2ebaa337b8d9d18612ec02d79baf716bb532cf23b236841f36133a1da3/snowflake_data_validation-0.0.7.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-07-11 17:09:52",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "snowflakedb",
    "github_project": "migrations-data-validation",
    "github_not_found": true,
    "lcname": "snowflake-data-validation"
}
        
Elapsed time: 2.20485s