# Snowflake Streaming Ingest Python SDK
A high-performance Python SDK for streaming data ingestion into Snowflake, built with Rust for optimal performance and reliability.
## Overview
The Snowflake Streaming Ingest Python SDK provides a Python interface for real-time data streaming into Snowflake tables. It leverages a Rust core for high performance while providing a familiar Python API for easy integration.
## Supported Platforms
### Operating Systems
- **Linux**: x86_64, aarch64
- **macOS**: arm64 (Apple Silicon)
- **Windows**: x86_64
### Python Versions
- **Python 3.9** (minimum required)
- **Python 3.10**
- **Python 3.11**
- **Python 3.12**
- **Python 3.13**
### Minimum System Requirements
#### Linux
- **glibc**: 2.26 or higher
- **Architecture**: x86_64, aarch64
#### macOS
- **Version**: macOS 11.0 (Big Sur) or higher
- **Architecture**: Apple Silicon (arm64)
#### Windows
- **Version**: Windows 10 or higher
- **Architecture**: x86_64
## Installation
### From PyPI (Recommended)
```bash
pip install snowpipe-streaming
```
## Quick Start
```python
from snowflake.ingest.streaming import StreamingIngestClient
# Create a client
client = StreamingIngestClient(
client_name="my_client",
db_name="my_database",
schema_name="my_schema",
pipe_name="my_pipe",
properties={
"account": "your_account",
"user": "your_user",
"private_key": "your_private_key",
"url": "https://your_account.snowflakecomputing.com"
}
)
# Open a channel
channel, status = client.open_channel("my_channel")
# Insert data
row_data = {
"id": 1,
"name": "John Doe",
"timestamp": "2024-01-01T00:00:00Z"
}
channel.append_row(row_data)
# Close resources
channel.close()
client.close()
```
## Features
- **High Performance**: Rust-based core for optimal throughput and low latency
- **Memory Efficient**: Minimal memory footprint with efficient data handling
- **Type Safety**: Strong typing support with Python type hints
- **Automatic Retries**: Built-in retry logic for transient failures
- **Backpressure Handling**: Intelligent backpressure management
- **Comprehensive Logging**: Detailed logging for debugging and monitoring
- **Cross-Platform**: Native support for Linux, macOS, and Windows
## Architecture
The SDK uses a hybrid Rust-Python architecture:
- **Rust Core**: High-performance data processing, networking, and Snowflake communication
- **Python Bindings**: Pythonic API using PyO3 for seamless integration
- **Stable ABI**: Uses Python's stable ABI (abi3) for forward compatibility
## Dependencies
### Core Dependencies
- **msgspec**: Fast JSON serialization/deserialization
- **Native Extensions**: Rust-based binary modules (automatically included)
### Build Dependencies (for development)
- **maturin**: Rust-Python build tool
- **pytest**: Testing framework
- **Rust toolchain**: Required for building from source
## Support
- **Documentation**: [Snowflake Documentation](https://docs.snowflake.com/)
- **Issues**: [GitHub Issues](https://github.com/snowflakedb/snowflake-ingest-sdk/issues)
- **Community**: [Snowflake Community](https://community.snowflake.com/)
## License
This project is licensed under the Apache License 2.0. See [LICENSE-2.0.txt](LICENSE-2.0.txt) for details.
## Contributing
1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Run tests: `python -m pytest tests/`
5. Submit a pull request
For more details, see the main repository's contribution guidelines.
Raw data
{
"_id": null,
"home_page": null,
"name": "snowpipe-streaming",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "snowflake, streaming, ingest, data",
"author": null,
"author_email": "\"Snowflake Inc.\" <support@snowflake.com>",
"download_url": null,
"platform": null,
"description": "# Snowflake Streaming Ingest Python SDK\n\nA high-performance Python SDK for streaming data ingestion into Snowflake, built with Rust for optimal performance and reliability.\n\n## Overview\n\nThe Snowflake Streaming Ingest Python SDK provides a Python interface for real-time data streaming into Snowflake tables. It leverages a Rust core for high performance while providing a familiar Python API for easy integration.\n\n## Supported Platforms\n\n### Operating Systems\n- **Linux**: x86_64, aarch64\n- **macOS**: arm64 (Apple Silicon)\n- **Windows**: x86_64\n\n### Python Versions\n- **Python 3.9** (minimum required)\n- **Python 3.10**\n- **Python 3.11**\n- **Python 3.12**\n- **Python 3.13**\n\n### Minimum System Requirements\n\n#### Linux\n- **glibc**: 2.26 or higher\n- **Architecture**: x86_64, aarch64\n\n#### macOS\n- **Version**: macOS 11.0 (Big Sur) or higher\n- **Architecture**: Apple Silicon (arm64)\n\n#### Windows\n- **Version**: Windows 10 or higher\n- **Architecture**: x86_64\n\n## Installation\n\n### From PyPI (Recommended)\n```bash\npip install snowpipe-streaming\n```\n\n## Quick Start\n\n```python\nfrom snowflake.ingest.streaming import StreamingIngestClient\n\n# Create a client\nclient = StreamingIngestClient(\n client_name=\"my_client\",\n db_name=\"my_database\",\n schema_name=\"my_schema\",\n pipe_name=\"my_pipe\",\n properties={\n \"account\": \"your_account\",\n \"user\": \"your_user\",\n \"private_key\": \"your_private_key\",\n \"url\": \"https://your_account.snowflakecomputing.com\"\n }\n)\n\n# Open a channel\nchannel, status = client.open_channel(\"my_channel\")\n\n# Insert data\nrow_data = {\n \"id\": 1,\n \"name\": \"John Doe\",\n \"timestamp\": \"2024-01-01T00:00:00Z\"\n}\nchannel.append_row(row_data)\n\n# Close resources\nchannel.close()\nclient.close()\n```\n\n## Features\n\n- **High Performance**: Rust-based core for optimal throughput and low latency\n- **Memory Efficient**: Minimal memory footprint with efficient data handling\n- **Type Safety**: Strong typing support with Python type hints\n- **Automatic Retries**: Built-in retry logic for transient failures\n- **Backpressure Handling**: Intelligent backpressure management\n- **Comprehensive Logging**: Detailed logging for debugging and monitoring\n- **Cross-Platform**: Native support for Linux, macOS, and Windows\n\n## Architecture\n\nThe SDK uses a hybrid Rust-Python architecture:\n- **Rust Core**: High-performance data processing, networking, and Snowflake communication\n- **Python Bindings**: Pythonic API using PyO3 for seamless integration\n- **Stable ABI**: Uses Python's stable ABI (abi3) for forward compatibility\n\n\n## Dependencies\n\n### Core Dependencies\n- **msgspec**: Fast JSON serialization/deserialization\n- **Native Extensions**: Rust-based binary modules (automatically included)\n\n### Build Dependencies (for development)\n- **maturin**: Rust-Python build tool\n- **pytest**: Testing framework\n- **Rust toolchain**: Required for building from source\n\n## Support\n\n- **Documentation**: [Snowflake Documentation](https://docs.snowflake.com/)\n- **Issues**: [GitHub Issues](https://github.com/snowflakedb/snowflake-ingest-sdk/issues)\n- **Community**: [Snowflake Community](https://community.snowflake.com/)\n\n## License\n\nThis project is licensed under the Apache License 2.0. See [LICENSE-2.0.txt](LICENSE-2.0.txt) for details.\n\n## Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Make your changes\n4. Run tests: `python -m pytest tests/`\n5. Submit a pull request\n\nFor more details, see the main repository's contribution guidelines.\n\n",
"bugtrack_url": null,
"license": "MIT or Apache-2.0",
"summary": "Snowflake Streaming Ingest SDK",
"version": "0.6.0",
"project_urls": {
"Documentation": "https://docs.snowflake.com/",
"Homepage": "https://github.com/snowflakedb/snowflake-ingest-sdk",
"Repository": "https://github.com/snowflakedb/snowflake-ingest-sdk"
},
"split_keywords": [
"snowflake",
" streaming",
" ingest",
" data"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "1972a0c9e8374c2cc8ba6ff396d7d43a0e1dc7b588a3870106c81055e83ccbd2",
"md5": "e8b74d011eb95fc961462cecbe2a369d",
"sha256": "374b353779769760c98f4b362120ba317d52154b2a8b1e20b85d29bc3ba069d8"
},
"downloads": -1,
"filename": "snowpipe_streaming-0.6.0-cp39-abi3-macosx_11_0_arm64.whl",
"has_sig": false,
"md5_digest": "e8b74d011eb95fc961462cecbe2a369d",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": ">=3.9",
"size": 7145835,
"upload_time": "2025-08-29T20:46:03",
"upload_time_iso_8601": "2025-08-29T20:46:03.761839Z",
"url": "https://files.pythonhosted.org/packages/19/72/a0c9e8374c2cc8ba6ff396d7d43a0e1dc7b588a3870106c81055e83ccbd2/snowpipe_streaming-0.6.0-cp39-abi3-macosx_11_0_arm64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "533da179fade18080c08b2baaca686136298b6ec2dbb31bdf6b8e9d623f34f72",
"md5": "b2119695bc8f18d168d486967e64adf0",
"sha256": "a2b1caebcbbe0ddc334e538cceb436f524e7673be1790804155ff43902b12234"
},
"downloads": -1,
"filename": "snowpipe_streaming-0.6.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl",
"has_sig": false,
"md5_digest": "b2119695bc8f18d168d486967e64adf0",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": ">=3.9",
"size": 8084053,
"upload_time": "2025-08-29T20:46:05",
"upload_time_iso_8601": "2025-08-29T20:46:05.613880Z",
"url": "https://files.pythonhosted.org/packages/53/3d/a179fade18080c08b2baaca686136298b6ec2dbb31bdf6b8e9d623f34f72/snowpipe_streaming-0.6.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "dc21b7adf0c8786747a8a069d2656b64fed53314142378527fa2d7a76fd84607",
"md5": "7fa0cdc8ddcfece1b11ce30dd7df4e5c",
"sha256": "bd3c12d0e7084c961a188c9789baf473b6e6bc5977e95885fb3412090490d319"
},
"downloads": -1,
"filename": "snowpipe_streaming-0.6.0-cp39-abi3-manylinux_2_24_x86_64.whl",
"has_sig": false,
"md5_digest": "7fa0cdc8ddcfece1b11ce30dd7df4e5c",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": ">=3.9",
"size": 8217283,
"upload_time": "2025-08-29T20:46:07",
"upload_time_iso_8601": "2025-08-29T20:46:07.462542Z",
"url": "https://files.pythonhosted.org/packages/dc/21/b7adf0c8786747a8a069d2656b64fed53314142378527fa2d7a76fd84607/snowpipe_streaming-0.6.0-cp39-abi3-manylinux_2_24_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "f1f3230f26b95e582f469af0f420e7caf926afd74d4782863eabbf321469d7e9",
"md5": "79e23f38cfb1332f2cf16d7a3053fc34",
"sha256": "822d8e926e973c31b994f2513cf9c0114ef6271fb25d28ba228716f5d2e80992"
},
"downloads": -1,
"filename": "snowpipe_streaming-0.6.0-cp39-abi3-win_amd64.whl",
"has_sig": false,
"md5_digest": "79e23f38cfb1332f2cf16d7a3053fc34",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": ">=3.9",
"size": 6373996,
"upload_time": "2025-08-29T20:46:09",
"upload_time_iso_8601": "2025-08-29T20:46:09.331475Z",
"url": "https://files.pythonhosted.org/packages/f1/f3/230f26b95e582f469af0f420e7caf926afd74d4782863eabbf321469d7e9/snowpipe_streaming-0.6.0-cp39-abi3-win_amd64.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-29 20:46:03",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "snowflakedb",
"github_project": "snowflake-ingest-sdk",
"github_not_found": true,
"lcname": "snowpipe-streaming"
}
JSON
