[](https://github.com/Menziess/slipstream-async/actions/workflows/triggered-tests.yml)
[](https://slipstream.readthedocs.io/en/latest/?badge=latest)
[](https://pypi.org/project/slipstream-async/)
# Slipstream
<img src="https://raw.githubusercontent.com/menziess/slipstream/master/docs/source/_static/logo.png" width="25%" height="25%" align="right" />
Slipstream provides a data-flow model to simplify development of stateful streaming applications.
```sh
pip install slipstream-async
```
```py
from asyncio import run
from slipstream import handle, stream
async def messages():
for emoji in '🏆📞🐟👌':
yield emoji
@handle(messages(), sink=[print])
def handle_message(msg):
yield f'Hello {msg}!'
if __name__ == '__main__':
run(stream())
```
```sh
Hello 🏆!
Hello 📞!
Hello 🐟!
Hello 👌!
```
## Usage
Slipstream components interoperate with basic python building blocks:
- `Any`-thing can be passed around as data
- Any `Callable` may be used as a sink
- `AsyncIterables` act as sources
- Parallelize through `handle`
<img src="https://raw.githubusercontent.com/menziess/slipstream/master/docs/source/_static/demo.gif" />
A many-to-many relation is established by passing multiple sources / sinks.
## Quickstart
Install Slipstream along with `aiokafka` (latest):
```sh
pip install slipstream-async[kafka]
```
Spin up a local Kafka broker with [docker-compose.yml](docker-compose.yml), using `localhost:29091` to connect:
```sh
docker compose up broker -d
```
Copy-paste [this snippet](https://slipstream.readthedocs.io/en/stable/getting_started.html#kafka).
## Features
- [`slipstream.handle`](https://slipstream.readthedocs.io/en/stable/slipstream.html#slipstream.handle): bind streams (iterables) and sinks (callables) to user defined handler functions
- [`slipstream.stream`](https://slipstream.readthedocs.io/en/stable/slipstream.html#slipstream.stream): start streaming
- [`slipstream.Topic`](https://slipstream.readthedocs.io/en/stable/slipstream.html#slipstream.core.Topic): consume from (iterable), and produce to (callable) kafka using [**aiokafka**](https://aiokafka.readthedocs.io/en/stable/index.html)
- [`slipstream.Cache`](https://slipstream.readthedocs.io/en/stable/slipstream.html#slipstream.Cache): store data to disk using [**rocksdict**](https://rocksdict.github.io/RocksDict/rocksdict.html)
- [`slipstream.Conf`](https://slipstream.readthedocs.io/en/stable/slipstream.html#slipstream.Conf): set global kafka configuration (can be overridden per topic)
- [`slipstream.codecs.JsonCodec`](https://slipstream.readthedocs.io/en/stable/autoapi/slipstream/codecs/index.html#slipstream.codecs.JsonCodec): serialize and deserialize json messages
- [`slipstream.checkpointing.Checkpoint`](https://slipstream.readthedocs.io/en/stable/autoapi/slipstream/checkpointing/index.html#slipstream.checkpointing.Checkpoint): recover from stream downtimes
Raw data
{
"_id": null,
"home_page": null,
"name": "slipstream-async",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "python, streaming, kafka, stream-processing, data-engineering, dataflow, data-processing, streaming-data, slipstream, stateful-streaming",
"author": null,
"author_email": "Menziess <stefan_schenk@hotmail.com>",
"download_url": "https://files.pythonhosted.org/packages/9c/05/1363e591c45590602888f8cd645b72a99ee136e1e2fc6d8a5542c99b650a/slipstream_async-1.0.4.tar.gz",
"platform": null,
"description": "[](https://github.com/Menziess/slipstream-async/actions/workflows/triggered-tests.yml)\n[](https://slipstream.readthedocs.io/en/latest/?badge=latest)\n[](https://pypi.org/project/slipstream-async/)\n\n# Slipstream\n\n<img src=\"https://raw.githubusercontent.com/menziess/slipstream/master/docs/source/_static/logo.png\" width=\"25%\" height=\"25%\" align=\"right\" />\n\nSlipstream provides a data-flow model to simplify development of stateful streaming applications.\n\n```sh\npip install slipstream-async\n```\n\n```py\nfrom asyncio import run\n\nfrom slipstream import handle, stream\n\n\nasync def messages():\n for emoji in '\ud83c\udfc6\ud83d\udcde\ud83d\udc1f\ud83d\udc4c':\n yield emoji\n\n\n@handle(messages(), sink=[print])\ndef handle_message(msg):\n yield f'Hello {msg}!'\n\n\nif __name__ == '__main__':\n run(stream())\n```\n\n```sh\nHello \ud83c\udfc6!\nHello \ud83d\udcde!\nHello \ud83d\udc1f!\nHello \ud83d\udc4c!\n```\n\n## Usage\n\nSlipstream components interoperate with basic python building blocks:\n\n- `Any`-thing can be passed around as data\n- Any `Callable` may be used as a sink\n- `AsyncIterables` act as sources\n- Parallelize through `handle`\n\n<img src=\"https://raw.githubusercontent.com/menziess/slipstream/master/docs/source/_static/demo.gif\" />\n\nA many-to-many relation is established by passing multiple sources / sinks.\n\n## Quickstart\n\nInstall Slipstream along with `aiokafka` (latest):\n\n```sh\npip install slipstream-async[kafka]\n```\n\nSpin up a local Kafka broker with [docker-compose.yml](docker-compose.yml), using `localhost:29091` to connect:\n\n```sh\ndocker compose up broker -d\n```\n\nCopy-paste [this snippet](https://slipstream.readthedocs.io/en/stable/getting_started.html#kafka).\n\n## Features\n\n- [`slipstream.handle`](https://slipstream.readthedocs.io/en/stable/slipstream.html#slipstream.handle): bind streams (iterables) and sinks (callables) to user defined handler functions\n- [`slipstream.stream`](https://slipstream.readthedocs.io/en/stable/slipstream.html#slipstream.stream): start streaming\n- [`slipstream.Topic`](https://slipstream.readthedocs.io/en/stable/slipstream.html#slipstream.core.Topic): consume from (iterable), and produce to (callable) kafka using [**aiokafka**](https://aiokafka.readthedocs.io/en/stable/index.html)\n- [`slipstream.Cache`](https://slipstream.readthedocs.io/en/stable/slipstream.html#slipstream.Cache): store data to disk using [**rocksdict**](https://rocksdict.github.io/RocksDict/rocksdict.html)\n- [`slipstream.Conf`](https://slipstream.readthedocs.io/en/stable/slipstream.html#slipstream.Conf): set global kafka configuration (can be overridden per topic)\n- [`slipstream.codecs.JsonCodec`](https://slipstream.readthedocs.io/en/stable/autoapi/slipstream/codecs/index.html#slipstream.codecs.JsonCodec): serialize and deserialize json messages\n- [`slipstream.checkpointing.Checkpoint`](https://slipstream.readthedocs.io/en/stable/autoapi/slipstream/checkpointing/index.html#slipstream.checkpointing.Checkpoint): recover from stream downtimes\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Streamline your stream processing.",
"version": "1.0.4",
"project_urls": {
"documentation": "https://slipstream.readthedocs.io",
"repository": "https://github.com/Menziess/slipstream"
},
"split_keywords": [
"python",
" streaming",
" kafka",
" stream-processing",
" data-engineering",
" dataflow",
" data-processing",
" streaming-data",
" slipstream",
" stateful-streaming"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "e9375a773a7e48e6e583f133b6a4bdb417a59a2278ff5037aa2fe8b12bd79cfd",
"md5": "10ff9c0e40927abd38c460b17ba9ba1e",
"sha256": "f6753301b402903141c765a6375a433114463ee9512b466f5c7dd9ee30b91ca2"
},
"downloads": -1,
"filename": "slipstream_async-1.0.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "10ff9c0e40927abd38c460b17ba9ba1e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 24528,
"upload_time": "2025-08-01T15:35:35",
"upload_time_iso_8601": "2025-08-01T15:35:35.425364Z",
"url": "https://files.pythonhosted.org/packages/e9/37/5a773a7e48e6e583f133b6a4bdb417a59a2278ff5037aa2fe8b12bd79cfd/slipstream_async-1.0.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "9c051363e591c45590602888f8cd645b72a99ee136e1e2fc6d8a5542c99b650a",
"md5": "b92d0cf04c6fcdedad73415d17e63d35",
"sha256": "2899892195fc2b9e1284a332a6d471417dd0c1fdc7d745913056279b6f8ad13c"
},
"downloads": -1,
"filename": "slipstream_async-1.0.4.tar.gz",
"has_sig": false,
"md5_digest": "b92d0cf04c6fcdedad73415d17e63d35",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 32596,
"upload_time": "2025-08-01T15:35:36",
"upload_time_iso_8601": "2025-08-01T15:35:36.700881Z",
"url": "https://files.pythonhosted.org/packages/9c/05/1363e591c45590602888f8cd645b72a99ee136e1e2fc6d8a5542c99b650a/slipstream_async-1.0.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-01 15:35:36",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Menziess",
"github_project": "slipstream",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "slipstream-async"
}
JSON
