# Arroyo
<p align="center">
<!-- do not specify height so that it scales proportionally on mobile -->
<img src=docs/source/_static/arroyo-banner.png width=583 />
</p>
`Arroyo` is a library to build streaming applications that consume from and produce to Kafka.
Arroyo consists of three components:
* Consumer and producer backends
- The Kafka backend is a wrapper around the librdkafka client, and attempts to simplify rebalancing and offset management even further
- There is also an in memory and a file based consumer and producer implementation that can be used for testing
* A strategy interface
- Arroyo includes a number of pre-built strategies such as `RunTask`, `Filter`, `Reduce`, `CommitOffsets` and more.
- Users can write their own strategies, though in most cases this should not be needed as the library aims to provide generic, reusable strategies that cover most stream processing use cases
- Strategies can be chained together to form complex message processing pipelines.
* A streaming engine which manages the relationship between the consumer and strategies
- The `StreamProcessor` controls progress by the consumer and schedules work for execution by the strategies.
All documentation is in the `docs` directory. It is hosted at https://getsentry.github.io/arroyo/ and can be built locally by running `make docs`
Raw data
{
"_id": null,
"home_page": "https://github.com/getsentry/arroyo",
"name": "sentry-arroyo",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "Sentry",
"author_email": "oss@sentry.io",
"download_url": "https://files.pythonhosted.org/packages/76/5c/ad6f1a5b7f5cc25c57d1f80fc3de9956ed8265b0da60640eb95ad9fbb862/sentry-arroyo-2.17.3.tar.gz",
"platform": null,
"description": "# Arroyo\n\n<p align=\"center\">\n <!-- do not specify height so that it scales proportionally on mobile -->\n <img src=docs/source/_static/arroyo-banner.png width=583 />\n</p>\n\n`Arroyo` is a library to build streaming applications that consume from and produce to Kafka.\n\nArroyo consists of three components:\n\n* Consumer and producer backends\n - The Kafka backend is a wrapper around the librdkafka client, and attempts to simplify rebalancing and offset management even further\n - There is also an in memory and a file based consumer and producer implementation that can be used for testing\n* A strategy interface\n - Arroyo includes a number of pre-built strategies such as `RunTask`, `Filter`, `Reduce`, `CommitOffsets` and more.\n - Users can write their own strategies, though in most cases this should not be needed as the library aims to provide generic, reusable strategies that cover most stream processing use cases\n - Strategies can be chained together to form complex message processing pipelines.\n* A streaming engine which manages the relationship between the consumer and strategies\n - The `StreamProcessor` controls progress by the consumer and schedules work for execution by the strategies.\n\nAll documentation is in the `docs` directory. It is hosted at https://getsentry.github.io/arroyo/ and can be built locally by running `make docs`\n\n\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Arroyo is a Python library for working with streaming data.",
"version": "2.17.3",
"project_urls": {
"Homepage": "https://github.com/getsentry/arroyo"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "957f93c87061f8a5876a257208ad0494bb0b5d7fbae7f83d6ad75466504f90f5",
"md5": "653dacff1a6af58825d4d37a2bcb9686",
"sha256": "0f3318f7a35b356e386c669855bbaa8dcce6ff2399460d30233310b0acc880bd"
},
"downloads": -1,
"filename": "sentry_arroyo-2.17.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "653dacff1a6af58825d4d37a2bcb9686",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 103114,
"upload_time": "2024-04-09T17:38:56",
"upload_time_iso_8601": "2024-04-09T17:38:56.178349Z",
"url": "https://files.pythonhosted.org/packages/95/7f/93c87061f8a5876a257208ad0494bb0b5d7fbae7f83d6ad75466504f90f5/sentry_arroyo-2.17.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "765cad6f1a5b7f5cc25c57d1f80fc3de9956ed8265b0da60640eb95ad9fbb862",
"md5": "11d4112b0a2375c9913aec241e5874c7",
"sha256": "6a25f482590dd648c52a31c3cfe76871157f2c699d41eac245a9f2cf7c44ef41"
},
"downloads": -1,
"filename": "sentry-arroyo-2.17.3.tar.gz",
"has_sig": false,
"md5_digest": "11d4112b0a2375c9913aec241e5874c7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 76362,
"upload_time": "2024-04-09T17:38:58",
"upload_time_iso_8601": "2024-04-09T17:38:58.005053Z",
"url": "https://files.pythonhosted.org/packages/76/5c/ad6f1a5b7f5cc25c57d1f80fc3de9956ed8265b0da60640eb95ad9fbb862/sentry-arroyo-2.17.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-09 17:38:58",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "getsentry",
"github_project": "arroyo",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "confluent-kafka",
"specs": [
[
">=",
"2.3.0"
]
]
}
],
"lcname": "sentry-arroyo"
}
JSON
