# Snowflake Streaming Ingest Python SDK
A high-performance Python SDK for streaming data ingestion into Snowflake, built with Rust for optimal performance and reliability.
## Overview
The Snowflake Streaming Ingest Python SDK provides a Python interface for real-time data streaming into Snowflake tables. It leverages a Rust core for high performance while providing a familiar Python API for easy integration.
## Supported Platforms
### Operating Systems
- **Linux**: x86_64, aarch64
- **macOS**: x86_64 (Intel), arm64 (Apple Silicon)
- **Windows**: x86_64
### Python Versions
- **Python 3.9** (minimum required)
- **Python 3.10**
- **Python 3.11**
- **Python 3.12**
### Minimum System Requirements
#### Linux
- **glibc**: 2.17 or higher (CentOS 7+, Ubuntu 14.04+, Debian 8+)
- **Architecture**: x86_64, aarch64
#### macOS
- **Version**: macOS 10.15 (Catalina) or higher
- **Architecture**: Apple Silicon (arm64)
#### Windows
- **Version**: Windows 10 or higher
- **Architecture**: x86_64
## Installation
### From PyPI (Recommended)
```bash
pip install snowpipe-streaming
```
## Quick Start
```python
from snowflake.ingest.streaming import StreamingIngestClient
# Create a client
client = StreamingIngestClient(
client_name="my_client",
db_name="my_database",
schema_name="my_schema",
pipe_name="my_pipe",
properties={
"account": "your_account",
"user": "your_user",
"private_key": "your_private_key",
"url": "https://your_account.snowflakecomputing.com"
}
)
# Open a channel
channel, status = client.open_channel("my_channel")
# Insert data
row_data = {
"id": 1,
"name": "John Doe",
"timestamp": "2024-01-01T00:00:00Z"
}
channel.append_row(row_data)
# Close resources
channel.close()
client.close()
```
## Features
- **High Performance**: Rust-based core for optimal throughput and low latency
- **Memory Efficient**: Minimal memory footprint with efficient data handling
- **Type Safety**: Strong typing support with Python type hints
- **Automatic Retries**: Built-in retry logic for transient failures
- **Backpressure Handling**: Intelligent backpressure management
- **Comprehensive Logging**: Detailed logging for debugging and monitoring
- **Cross-Platform**: Native support for Linux, macOS, and Windows
## Architecture
The SDK uses a hybrid Rust-Python architecture:
- **Rust Core**: High-performance data processing, networking, and Snowflake communication
- **Python Bindings**: Pythonic API using PyO3 for seamless integration
- **Stable ABI**: Uses Python's stable ABI (abi3) for forward compatibility
## Dependencies
### Core Dependencies
- **msgspec**: Fast JSON serialization/deserialization
- **Native Extensions**: Rust-based binary modules (automatically included)
### Build Dependencies (for development)
- **maturin**: Rust-Python build tool
- **pytest**: Testing framework
- **Rust toolchain**: Required for building from source
## Support
- **Documentation**: [Snowflake Documentation](https://docs.snowflake.com/)
- **Issues**: [GitHub Issues](https://github.com/snowflakedb/snowflake-ingest-sdk/issues)
- **Community**: [Snowflake Community](https://community.snowflake.com/)
## License
This project is licensed under the Apache License 2.0. See [LICENSE-2.0.txt](LICENSE-2.0.txt) for details.
## Contributing
1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Run tests: `python -m pytest tests/`
5. Submit a pull request
For more details, see the main repository's contribution guidelines.
Raw data
{
"_id": null,
"home_page": null,
"name": "snowpipe-streaming",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "snowflake, streaming, ingest, data",
"author": null,
"author_email": "\"Snowflake Inc.\" <support@snowflake.com>",
"download_url": null,
"platform": null,
"description": "# Snowflake Streaming Ingest Python SDK\n\nA high-performance Python SDK for streaming data ingestion into Snowflake, built with Rust for optimal performance and reliability.\n\n## Overview\n\nThe Snowflake Streaming Ingest Python SDK provides a Python interface for real-time data streaming into Snowflake tables. It leverages a Rust core for high performance while providing a familiar Python API for easy integration.\n\n## Supported Platforms\n\n### Operating Systems\n- **Linux**: x86_64, aarch64\n- **macOS**: x86_64 (Intel), arm64 (Apple Silicon)\n- **Windows**: x86_64\n\n### Python Versions\n- **Python 3.9** (minimum required)\n- **Python 3.10**\n- **Python 3.11**\n- **Python 3.12**\n\n### Minimum System Requirements\n\n#### Linux\n- **glibc**: 2.17 or higher (CentOS 7+, Ubuntu 14.04+, Debian 8+)\n- **Architecture**: x86_64, aarch64\n\n#### macOS\n- **Version**: macOS 10.15 (Catalina) or higher\n- **Architecture**: Apple Silicon (arm64)\n\n#### Windows\n- **Version**: Windows 10 or higher\n- **Architecture**: x86_64\n\n## Installation\n\n### From PyPI (Recommended)\n```bash\npip install snowpipe-streaming\n```\n\n## Quick Start\n\n```python\nfrom snowflake.ingest.streaming import StreamingIngestClient\n\n# Create a client\nclient = StreamingIngestClient(\n client_name=\"my_client\",\n db_name=\"my_database\",\n schema_name=\"my_schema\",\n pipe_name=\"my_pipe\",\n properties={\n \"account\": \"your_account\",\n \"user\": \"your_user\",\n \"private_key\": \"your_private_key\",\n \"url\": \"https://your_account.snowflakecomputing.com\"\n }\n)\n\n# Open a channel\nchannel, status = client.open_channel(\"my_channel\")\n\n# Insert data\nrow_data = {\n \"id\": 1,\n \"name\": \"John Doe\",\n \"timestamp\": \"2024-01-01T00:00:00Z\"\n}\nchannel.append_row(row_data)\n\n# Close resources\nchannel.close()\nclient.close()\n```\n\n## Features\n\n- **High Performance**: Rust-based core for optimal throughput and low latency\n- **Memory Efficient**: Minimal memory footprint with efficient data handling\n- **Type Safety**: Strong typing support with Python type hints\n- **Automatic Retries**: Built-in retry logic for transient failures\n- **Backpressure Handling**: Intelligent backpressure management\n- **Comprehensive Logging**: Detailed logging for debugging and monitoring\n- **Cross-Platform**: Native support for Linux, macOS, and Windows\n\n## Architecture\n\nThe SDK uses a hybrid Rust-Python architecture:\n- **Rust Core**: High-performance data processing, networking, and Snowflake communication\n- **Python Bindings**: Pythonic API using PyO3 for seamless integration\n- **Stable ABI**: Uses Python's stable ABI (abi3) for forward compatibility\n\n\n## Dependencies\n\n### Core Dependencies\n- **msgspec**: Fast JSON serialization/deserialization\n- **Native Extensions**: Rust-based binary modules (automatically included)\n\n### Build Dependencies (for development)\n- **maturin**: Rust-Python build tool\n- **pytest**: Testing framework\n- **Rust toolchain**: Required for building from source\n\n## Support\n\n- **Documentation**: [Snowflake Documentation](https://docs.snowflake.com/)\n- **Issues**: [GitHub Issues](https://github.com/snowflakedb/snowflake-ingest-sdk/issues)\n- **Community**: [Snowflake Community](https://community.snowflake.com/)\n\n## License\n\nThis project is licensed under the Apache License 2.0. See [LICENSE-2.0.txt](LICENSE-2.0.txt) for details.\n\n## Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Make your changes\n4. Run tests: `python -m pytest tests/`\n5. Submit a pull request\n\nFor more details, see the main repository's contribution guidelines.\n\n",
"bugtrack_url": null,
"license": "MIT or Apache-2.0",
"summary": "Snowflake Streaming Ingest SDK",
"version": "0.5.0",
"project_urls": {
"Documentation": "https://docs.snowflake.com/",
"Homepage": "https://github.com/snowflakedb/snowflake-ingest-sdk",
"Repository": "https://github.com/snowflakedb/snowflake-ingest-sdk"
},
"split_keywords": [
"snowflake",
" streaming",
" ingest",
" data"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "983f2dc884bc26baaf173ec56ff85e47324c03faa5d84a1a601ea299bfb3bf5a",
"md5": "e97756faa3a91fbc5e41c375cdb49e8b",
"sha256": "3bb1a012f3148d57ee0615acfff14811ba69694223a3f9d2901bc462792d77cc"
},
"downloads": -1,
"filename": "snowpipe_streaming-0.5.0-cp39-abi3-macosx_11_0_arm64.whl",
"has_sig": false,
"md5_digest": "e97756faa3a91fbc5e41c375cdb49e8b",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": ">=3.9",
"size": 7301907,
"upload_time": "2025-08-07T22:23:09",
"upload_time_iso_8601": "2025-08-07T22:23:09.775855Z",
"url": "https://files.pythonhosted.org/packages/98/3f/2dc884bc26baaf173ec56ff85e47324c03faa5d84a1a601ea299bfb3bf5a/snowpipe_streaming-0.5.0-cp39-abi3-macosx_11_0_arm64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "86fc1bd81912d72265ed13a96d2a6107ead11abf846871f00e0b8ee0731273a8",
"md5": "04931ab5ffec09f13d2174f44a6c557c",
"sha256": "acc5c51244f13b851966563bd18187a2e50a409b469b06391a1d2940a1669b33"
},
"downloads": -1,
"filename": "snowpipe_streaming-0.5.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl",
"has_sig": false,
"md5_digest": "04931ab5ffec09f13d2174f44a6c557c",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": ">=3.9",
"size": 10208773,
"upload_time": "2025-08-07T22:23:11",
"upload_time_iso_8601": "2025-08-07T22:23:11.984231Z",
"url": "https://files.pythonhosted.org/packages/86/fc/1bd81912d72265ed13a96d2a6107ead11abf846871f00e0b8ee0731273a8/snowpipe_streaming-0.5.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "9dd5b7e82f533b6fe4cf862eb87f5a191366fb28f75e616d2a1796a3df4dee7b",
"md5": "ddba0868924a86572c894adfca624594",
"sha256": "b662401e5693a8a47888e6e552eeaf7ab212c20aaf2a45a236e769fc2cd61ac9"
},
"downloads": -1,
"filename": "snowpipe_streaming-0.5.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
"has_sig": false,
"md5_digest": "ddba0868924a86572c894adfca624594",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": ">=3.9",
"size": 10596210,
"upload_time": "2025-08-07T22:23:13",
"upload_time_iso_8601": "2025-08-07T22:23:13.888933Z",
"url": "https://files.pythonhosted.org/packages/9d/d5/b7e82f533b6fe4cf862eb87f5a191366fb28f75e616d2a1796a3df4dee7b/snowpipe_streaming-0.5.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "46f58feff1ca924d4c2f18430847bc28064d5b5da1f84bcb90a02d1490b0f776",
"md5": "bd0c33988ea472bdda4190b59b95e7d1",
"sha256": "1e62be46e25d1def957957b1aeab71be59b0233c70ba37f126549b1d58ac6bb7"
},
"downloads": -1,
"filename": "snowpipe_streaming-0.5.0-cp39-abi3-win_amd64.whl",
"has_sig": false,
"md5_digest": "bd0c33988ea472bdda4190b59b95e7d1",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": ">=3.9",
"size": 6514251,
"upload_time": "2025-08-07T22:23:16",
"upload_time_iso_8601": "2025-08-07T22:23:16.253295Z",
"url": "https://files.pythonhosted.org/packages/46/f5/8feff1ca924d4c2f18430847bc28064d5b5da1f84bcb90a02d1490b0f776/snowpipe_streaming-0.5.0-cp39-abi3-win_amd64.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-07 22:23:09",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "snowflakedb",
"github_project": "snowflake-ingest-sdk",
"github_not_found": true,
"lcname": "snowpipe-streaming"
}
JSON
