aiosafeconsumer
===============
.. image:: https://github.com/lostclus/aiosafeconsumer/actions/workflows/tests.yml/badge.svg
:target: https://github.com/lostclus/aiosafeconsumer/actions
.. image:: https://img.shields.io/pypi/v/aiosafeconsumer.svg
:target: https://pypi.org/project/aiosafeconsumer/
:alt: Current version on PyPi
.. image:: https://img.shields.io/pypi/pyversions/aiosafeconsumer
:alt: PyPI - Python Version
aiosafeconsumer is a library that provides abstractions and some implementations
to consume data somewhere and process it.
Features:
* Based on AsyncIO
* Type annotated
Abstractions:
* `DataSource` - waits for data and returns batch of records using Python generator
* `DataProcessor` - accepts batch of records and precess it
* `DataTransformer` - accepts batch of records and transform it and calls
another processor to precess it. Extends `DataProcessor`
* `Worker` - abstract worker
* `ConsumerWorker` - connects `DataSource` and `DataProcessor`
* `DataWriter` - base abstraction to perform data synchronization
Current implementations:
* `KafkaSource` - read data from Kafka
* `RedisWriter` - synchronize data in Redis
* `WorkerPool` - controller to setup and run workers in parallel. Can handle worker failures and restarts workers when it fails or exits.
Recommend producer library: https://github.com/lostclus/django-kafka-streamer
Example application: https://github.com/lostclus/WeatherApp
Raw data
{
"_id": null,
"home_page": null,
"name": "aiosafeconsumer",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "asyncio, consumer, microservices",
"author": null,
"author_email": "Kostiantyn Korikov <lostclus@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/56/ad/2d45bf1a3207a9095cea32e0eaf56a558975f8b39f7261b0febd73b46633/aiosafeconsumer-0.0.3.tar.gz",
"platform": null,
"description": "aiosafeconsumer\n===============\n\n.. image:: https://github.com/lostclus/aiosafeconsumer/actions/workflows/tests.yml/badge.svg\n :target: https://github.com/lostclus/aiosafeconsumer/actions\n\n.. image:: https://img.shields.io/pypi/v/aiosafeconsumer.svg\n :target: https://pypi.org/project/aiosafeconsumer/\n :alt: Current version on PyPi\n\n.. image:: https://img.shields.io/pypi/pyversions/aiosafeconsumer\n :alt: PyPI - Python Version\n\naiosafeconsumer is a library that provides abstractions and some implementations\nto consume data somewhere and process it.\n\nFeatures:\n\n* Based on AsyncIO\n* Type annotated\n\nAbstractions:\n\n* `DataSource` - waits for data and returns batch of records using Python generator\n* `DataProcessor` - accepts batch of records and precess it\n* `DataTransformer` - accepts batch of records and transform it and calls\n another processor to precess it. Extends `DataProcessor`\n* `Worker` - abstract worker\n* `ConsumerWorker` - connects `DataSource` and `DataProcessor`\n* `DataWriter` - base abstraction to perform data synchronization\n\nCurrent implementations:\n\n* `KafkaSource` - read data from Kafka\n* `RedisWriter` - synchronize data in Redis\n* `WorkerPool` - controller to setup and run workers in parallel. Can handle worker failures and restarts workers when it fails or exits.\n\nRecommend producer library: https://github.com/lostclus/django-kafka-streamer\n\nExample application: https://github.com/lostclus/WeatherApp\n",
"bugtrack_url": null,
"license": null,
"summary": "Safely consume and process data.",
"version": "0.0.3",
"project_urls": {
"Repository": "http://github.com/lostclus/aiosafeconsumer"
},
"split_keywords": [
"asyncio",
" consumer",
" microservices"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "085de8e18e063473b412439c39630f18f6e8d76b94dcea877104b422ff8082c0",
"md5": "3c9a7efc7d83b0e6ed3ac5ffc0d5f599",
"sha256": "dd515b1fa73b9d5c63a114b3eb5ca958b84f230ee6e1ff8b3a402a85b457ed63"
},
"downloads": -1,
"filename": "aiosafeconsumer-0.0.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "3c9a7efc7d83b0e6ed3ac5ffc0d5f599",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 17833,
"upload_time": "2024-10-28T09:23:13",
"upload_time_iso_8601": "2024-10-28T09:23:13.630237Z",
"url": "https://files.pythonhosted.org/packages/08/5d/e8e18e063473b412439c39630f18f6e8d76b94dcea877104b422ff8082c0/aiosafeconsumer-0.0.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "56ad2d45bf1a3207a9095cea32e0eaf56a558975f8b39f7261b0febd73b46633",
"md5": "51cc7e552fbb353e8b621d97dcda413f",
"sha256": "7599790a0efb69b6648d30abd0728a1fb5817fcfe98fdfbe410f797ba4160880"
},
"downloads": -1,
"filename": "aiosafeconsumer-0.0.3.tar.gz",
"has_sig": false,
"md5_digest": "51cc7e552fbb353e8b621d97dcda413f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 27304,
"upload_time": "2024-10-28T09:23:15",
"upload_time_iso_8601": "2024-10-28T09:23:15.717929Z",
"url": "https://files.pythonhosted.org/packages/56/ad/2d45bf1a3207a9095cea32e0eaf56a558975f8b39f7261b0febd73b46633/aiosafeconsumer-0.0.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-28 09:23:15",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "lostclus",
"github_project": "aiosafeconsumer",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "aiosafeconsumer"
}
JSON
