| Name | slipstream-async JSON |
| Version |
0.0.4a0
JSON |
| download |
| home_page | None |
| Summary | Streamline your stream processing. |
| upload_time | 2024-11-18 22:07:10 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.10 |
| license | MIT |
| keywords |
kafka
pubsub
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
[](https://github.com/Menziess/slipstream-async/actions/workflows/triggered-tests.yml)
[](https://slipstream.readthedocs.io/en/latest/?badge=latest)
[](https://pypi.org/project/slipstream-async/)
# Slipstream
<img src="https://raw.githubusercontent.com/menziess/slipstream/master/res/logo.png" width="25%" height="25%" align="right" />
Slipstream provides a data-flow model to simplify development of stateful streaming applications.
```sh
pip install slipstream-async
```
```py
from asyncio import run
from slipstream import handle, stream
async def messages():
for emoji in '🏆📞🐟👌':
yield emoji
@handle(messages(), sink=[print])
def handle_message(msg):
yield f'Hello {msg}!'
if __name__ == '__main__':
run(stream())
```
```sh
Hello 🏆!
Hello 📞!
Hello 🐟!
Hello 👌!
```
## Usage
Async `iterables` are sources, (async) `callables` are sinks.
Decorate handler functions using `handle`, then run `stream` to start processing:
<img src="https://raw.githubusercontent.com/menziess/slipstream/master/res/demo.gif" />
Multiple sources and sinks can be provided to establish many-to-many relations between them.
The 4 emoji's were printed using the callable `print`.
## Quickstart
Install `aiokafka` (latest) along with slipstream:
```sh
pip install slipstream-async[kafka]
```
Spin up a local Kafka broker with [docker-compose.yml](docker-compose.yml), using `localhost:29091` to connect:
```sh
docker compose up broker -d
```
Follow the docs and set up a Kafka connection: [slipstream.readthedocs.io](https://slipstream.readthedocs.io).
## Features
- [`slipstream.handle`](slipstream/__init__.py): bind streams (iterables) and sinks (callables) to user defined handler functions
- [`slipstream.stream`](slipstream/__init__.py): start streaming
- [`slipstream.Topic`](slipstream/core.py): consume from (iterable), and produce to (callable) kafka using [**aiokafka**](https://aiokafka.readthedocs.io/en/stable/index.html)
- [`slipstream.Cache`](slipstream/caching.py): store data to disk using [**rocksdict**](https://congyuwang.github.io/RocksDict/rocksdict.html)
- [`slipstream.Conf`](slipstream/core.py): set global kafka configuration (can be overridden per topic)
- [`slipstream.codecs.JsonCodec`](slipstream/codecs.py): serialize and deserialize json messages
Raw data
{
"_id": null,
"home_page": null,
"name": "slipstream-async",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "kafka, pubsub",
"author": null,
"author_email": "Menziess <stefan_schenk@hotmail.com>",
"download_url": "https://files.pythonhosted.org/packages/82/99/acb5f3a550aebfda9274ad13554b50149d2704a6c8c614f371a6d1862cac/slipstream_async-0.0.4a0.tar.gz",
"platform": null,
"description": "[](https://github.com/Menziess/slipstream-async/actions/workflows/triggered-tests.yml)\n[](https://slipstream.readthedocs.io/en/latest/?badge=latest)\n[](https://pypi.org/project/slipstream-async/)\n\n# Slipstream\n\n<img src=\"https://raw.githubusercontent.com/menziess/slipstream/master/res/logo.png\" width=\"25%\" height=\"25%\" align=\"right\" />\n\nSlipstream provides a data-flow model to simplify development of stateful streaming applications.\n\n```sh\npip install slipstream-async\n```\n\n```py\nfrom asyncio import run\n\nfrom slipstream import handle, stream\n\n\nasync def messages():\n for emoji in '\ud83c\udfc6\ud83d\udcde\ud83d\udc1f\ud83d\udc4c':\n yield emoji\n\n\n@handle(messages(), sink=[print])\ndef handle_message(msg):\n yield f'Hello {msg}!'\n\n\nif __name__ == '__main__':\n run(stream())\n```\n\n```sh\nHello \ud83c\udfc6!\nHello \ud83d\udcde!\nHello \ud83d\udc1f!\nHello \ud83d\udc4c!\n```\n\n## Usage\n\nAsync `iterables` are sources, (async) `callables` are sinks.\n\nDecorate handler functions using `handle`, then run `stream` to start processing:\n\n<img src=\"https://raw.githubusercontent.com/menziess/slipstream/master/res/demo.gif\" />\n\nMultiple sources and sinks can be provided to establish many-to-many relations between them.\nThe 4 emoji's were printed using the callable `print`.\n\n## Quickstart\n\nInstall `aiokafka` (latest) along with slipstream:\n\n```sh\npip install slipstream-async[kafka]\n```\n\nSpin up a local Kafka broker with [docker-compose.yml](docker-compose.yml), using `localhost:29091` to connect:\n\n```sh\ndocker compose up broker -d\n```\n\nFollow the docs and set up a Kafka connection: [slipstream.readthedocs.io](https://slipstream.readthedocs.io).\n\n## Features\n\n- [`slipstream.handle`](slipstream/__init__.py): bind streams (iterables) and sinks (callables) to user defined handler functions\n- [`slipstream.stream`](slipstream/__init__.py): start streaming\n- [`slipstream.Topic`](slipstream/core.py): consume from (iterable), and produce to (callable) kafka using [**aiokafka**](https://aiokafka.readthedocs.io/en/stable/index.html)\n- [`slipstream.Cache`](slipstream/caching.py): store data to disk using [**rocksdict**](https://congyuwang.github.io/RocksDict/rocksdict.html)\n- [`slipstream.Conf`](slipstream/core.py): set global kafka configuration (can be overridden per topic)\n- [`slipstream.codecs.JsonCodec`](slipstream/codecs.py): serialize and deserialize json messages\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Streamline your stream processing.",
"version": "0.0.4a0",
"project_urls": {
"documentation": "https://slipstream.readthedocs.io",
"repository": "https://github.com/Menziess/slipstream"
},
"split_keywords": [
"kafka",
" pubsub"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "145f1adb3e915ea7ac0e1c31a9c87bb6f0f146b8ba4537dfebd645ae5b1c6968",
"md5": "b7d5b416b495664adac46501727c3b00",
"sha256": "79b54d38b60304b51a297169b765af8db91bded1e6d645ae923b9c849f5df8cd"
},
"downloads": -1,
"filename": "slipstream_async-0.0.4a0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b7d5b416b495664adac46501727c3b00",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 12039,
"upload_time": "2024-11-18T22:07:09",
"upload_time_iso_8601": "2024-11-18T22:07:09.026502Z",
"url": "https://files.pythonhosted.org/packages/14/5f/1adb3e915ea7ac0e1c31a9c87bb6f0f146b8ba4537dfebd645ae5b1c6968/slipstream_async-0.0.4a0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8299acb5f3a550aebfda9274ad13554b50149d2704a6c8c614f371a6d1862cac",
"md5": "9d4a7f1006d5cd15a439e28c9431b317",
"sha256": "36c79f557150f63f750ef18e419e4b82aa7e4f85b7d49f4d19ad1f861270d30d"
},
"downloads": -1,
"filename": "slipstream_async-0.0.4a0.tar.gz",
"has_sig": false,
"md5_digest": "9d4a7f1006d5cd15a439e28c9431b317",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 13953,
"upload_time": "2024-11-18T22:07:10",
"upload_time_iso_8601": "2024-11-18T22:07:10.605506Z",
"url": "https://files.pythonhosted.org/packages/82/99/acb5f3a550aebfda9274ad13554b50149d2704a6c8c614f371a6d1862cac/slipstream_async-0.0.4a0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-18 22:07:10",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Menziess",
"github_project": "slipstream",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "slipstream-async"
}
