# SingleStore Bytewax Connector
A high-performance connector for integrating SingleStore with Bytewax dataflows.
## Features
- Seamless integration with SingleStore's CDC (Change Data Capture) functionality
- Support for multiple event types (Insert, Update, Delete, etc.)
- Stateful processing with resumable offsets
- Compatible with Bytewax's dataflow paradigm
## Installation
Install the connector using pip:
```
pip install singlestore-bytewax-connector
```
## Quick Start
Here's a simple example of how to use the SingleStore Bytewax Connector:
```python
from bytewax.dataflow import Dataflow
import bytewax.operators as op
from bytewax.connectors.stdio import StdOutSink
from singlestore_source import SingleStoreSource
flow = Dataflow("singlestore-cdc")
source = SingleStoreSource(
host="127.0.0.1",
port=3306,
user="root",
password="password",
database="test",
table="test",
event_types=["Insert", "Update", "Delete"],
)
stream = op.input("input", flow, source)
op.output("output", stream, StdOutSink())
```
This example sets up a dataflow that captures CDC events from a SingleStore table and outputs them to stdout.
## Configuration
The `SingleStoreSource` class accepts the following parameters:
- `host`: SingleStore server hostname
- `port`: SingleStore server port
- `user`: Database username
- `password`: Database password
- `database`: Name of the database
- `table`: Name of the table to observe
- `event_types`: List of event types to capture (default includes all types)
## Development
To set up the development environment:
```
pip install -e .[dev]
```
Run tests using:
```
python -m tox
```
## License
This project is licensed under the Apache License 2.0. See the [LICENSE](LICENSE) file for details.
## Acknowledgements
This project is built on top of [Bytewax](https://bytewax.io/) and integrates with [SingleStore](https://www.singlestore.com/). We're grateful for their excellent technologies that make this connector possible.
Raw data
{
"_id": null,
"home_page": null,
"name": "singlestore-bytewax-connector",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "Tom K\u00fchl <hello@tomkuehl.io>",
"keywords": "bytewax, streaming, connector, singlestore",
"author": null,
"author_email": "Tom K\u00fchl <hello@tomkuehl.io>",
"download_url": "https://files.pythonhosted.org/packages/ce/ac/23e0c47bede76af0dddea300306599b02d293e09c3d446a1b0b8b99d08ab/singlestore_bytewax_connector-0.0.1.tar.gz",
"platform": null,
"description": "# SingleStore Bytewax Connector\n\nA high-performance connector for integrating SingleStore with Bytewax dataflows.\n\n## Features\n\n- Seamless integration with SingleStore's CDC (Change Data Capture) functionality\n- Support for multiple event types (Insert, Update, Delete, etc.)\n- Stateful processing with resumable offsets\n- Compatible with Bytewax's dataflow paradigm\n\n## Installation\n\nInstall the connector using pip:\n\n```\npip install singlestore-bytewax-connector\n```\n\n## Quick Start\n\nHere's a simple example of how to use the SingleStore Bytewax Connector:\n\n```python\nfrom bytewax.dataflow import Dataflow\nimport bytewax.operators as op\nfrom bytewax.connectors.stdio import StdOutSink\nfrom singlestore_source import SingleStoreSource\n\nflow = Dataflow(\"singlestore-cdc\")\n\nsource = SingleStoreSource(\n host=\"127.0.0.1\",\n port=3306,\n user=\"root\",\n password=\"password\",\n database=\"test\",\n table=\"test\",\n event_types=[\"Insert\", \"Update\", \"Delete\"],\n)\n\nstream = op.input(\"input\", flow, source)\nop.output(\"output\", stream, StdOutSink())\n```\n\nThis example sets up a dataflow that captures CDC events from a SingleStore table and outputs them to stdout.\n\n## Configuration\n\nThe `SingleStoreSource` class accepts the following parameters:\n\n- `host`: SingleStore server hostname\n- `port`: SingleStore server port\n- `user`: Database username\n- `password`: Database password\n- `database`: Name of the database\n- `table`: Name of the table to observe\n- `event_types`: List of event types to capture (default includes all types)\n\n## Development\n\nTo set up the development environment:\n\n```\npip install -e .[dev]\n```\n\nRun tests using:\n\n```\npython -m tox\n```\n\n## License\n\nThis project is licensed under the Apache License 2.0. See the [LICENSE](LICENSE) file for details.\n\n## Acknowledgements\n\nThis project is built on top of [Bytewax](https://bytewax.io/) and integrates with [SingleStore](https://www.singlestore.com/). We're grateful for their excellent technologies that make this connector possible.\n",
"bugtrack_url": null,
"license": null,
"summary": "A Bytewax connector for SingleStore",
"version": "0.0.1",
"project_urls": {
"Bug Reports": "https://github.com/tomkuehl/singlestore-bytewax-connector/issues",
"Homepage": "https://github.com/tomkuehl/singlestore-bytewax-connector",
"Source": "https://github.com/tomkuehl/singlestore-bytewax-connector/"
},
"split_keywords": [
"bytewax",
" streaming",
" connector",
" singlestore"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "4d4fe998ce30ccec3a6f79dee277cdb204d20034e707721b5ce754a3ac2c5470",
"md5": "87efd75d339d3cd790d7596bcc36f681",
"sha256": "2da8f0e66ed6020104a6021bdef5e962d08cb8dc553ffc808816d0148f99420c"
},
"downloads": -1,
"filename": "singlestore_bytewax_connector-0.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "87efd75d339d3cd790d7596bcc36f681",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 9099,
"upload_time": "2024-07-09T12:21:22",
"upload_time_iso_8601": "2024-07-09T12:21:22.651509Z",
"url": "https://files.pythonhosted.org/packages/4d/4f/e998ce30ccec3a6f79dee277cdb204d20034e707721b5ce754a3ac2c5470/singlestore_bytewax_connector-0.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ceac23e0c47bede76af0dddea300306599b02d293e09c3d446a1b0b8b99d08ab",
"md5": "c6506d95bb0e643ae0a5a362e50b5328",
"sha256": "43ede56bb382543a29b9fdc9cb47857863e6f480c566f509c5db99f076796b2e"
},
"downloads": -1,
"filename": "singlestore_bytewax_connector-0.0.1.tar.gz",
"has_sig": false,
"md5_digest": "c6506d95bb0e643ae0a5a362e50b5328",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 8756,
"upload_time": "2024-07-09T12:21:24",
"upload_time_iso_8601": "2024-07-09T12:21:24.083402Z",
"url": "https://files.pythonhosted.org/packages/ce/ac/23e0c47bede76af0dddea300306599b02d293e09c3d446a1b0b8b99d08ab/singlestore_bytewax_connector-0.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-09 12:21:24",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "tomkuehl",
"github_project": "singlestore-bytewax-connector",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "singlestore-bytewax-connector"
}
JSON
