Celery Batches
==============
.. image:: https://img.shields.io/pypi/v/celery-batches.svg
:target: https://pypi.org/project/celery-batches/
.. image:: https://github.com/clokep/celery-batches/actions/workflows/main.yml/badge.svg
:target: https://github.com/clokep/celery-batches/actions/workflows/main.yml
.. image:: https://readthedocs.org/projects/celery-batches/badge/?version=latest
:target: https://celery-batches.readthedocs.io/en/latest/?badge=latest
:alt: Documentation Status
Celery Batches provides a ``Task`` class that allows processing of multiple
Celery task calls together as a list. The buffer of tasks calls is flushed on a
timer and based on the number of queued tasks.
Some potential use-cases for batching of task calls include:
* De-duplicating tasks.
* Accumulating / only handling the latest task with similar arguments.
* Bulk inserting / updating of data.
* Tasks with expensive setup that can run across a range of arguments.
What do I need?
===============
celery-batches version runs on,
- Python (3.8, 3.9, 3.10, 3.11, 3.12)
- PyPy3 (7.6)
And is tested with Celery ~= 5.0.
If you're running an older version of Python, you need to be running
an older version of celery-batches:
- Python 2.7: celery-batches 0.3.
- Python 3.4: celery-batches 0.2.
- Python 3.5: celery-batches 0.3.
- Python 3.6: celery-batches 0.5.
- Python 3.7: celery-batches 0.7.
If you're running an older version of Celery, you need to be running
an older version of celery-batches:
- Celery < 4.0: Use `celery.contrib.batches` instead.
- Celery 4.0 - 4.3: celery-batches 0.3.
- Celery 4.4: celery-batches 0.7.
History
=======
Celery Batches was distributed as part of Celery (as ``celery.contrib.batches``)
until Celery 4.0. This project updates the Batches code to maintain compatiblity
with newer versions of Celery and other fixes. See the Changelog for details.
Additionally, this repository includes the full history of the code from
``celery.contrib.batches``, but rewritten to the ``celery_batches/__init__.py``
file.
Raw data
{
"_id": null,
"home_page": "https://github.com/clokep/celery-batches",
"name": "celery-batches",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "task, job, queue, distributed, messaging, actor",
"author": "Patrick Cloke",
"author_email": "clokep@patrick.cloke.us",
"download_url": "https://files.pythonhosted.org/packages/5b/c0/b89d650c1e29b41b5d117e4b2d35c60155542fed1fae44ea4ecb6171aca0/celery_batches-0.9.tar.gz",
"platform": null,
"description": "Celery Batches\n==============\n\n.. image:: https://img.shields.io/pypi/v/celery-batches.svg\n :target: https://pypi.org/project/celery-batches/\n\n.. image:: https://github.com/clokep/celery-batches/actions/workflows/main.yml/badge.svg\n :target: https://github.com/clokep/celery-batches/actions/workflows/main.yml\n\n.. image:: https://readthedocs.org/projects/celery-batches/badge/?version=latest\n :target: https://celery-batches.readthedocs.io/en/latest/?badge=latest\n :alt: Documentation Status\n\nCelery Batches provides a ``Task`` class that allows processing of multiple\nCelery task calls together as a list. The buffer of tasks calls is flushed on a\ntimer and based on the number of queued tasks.\n\nSome potential use-cases for batching of task calls include:\n\n* De-duplicating tasks.\n* Accumulating / only handling the latest task with similar arguments.\n* Bulk inserting / updating of data.\n* Tasks with expensive setup that can run across a range of arguments.\n\nWhat do I need?\n===============\n\ncelery-batches version runs on,\n\n- Python (3.8, 3.9, 3.10, 3.11, 3.12)\n- PyPy3 (7.6)\n\nAnd is tested with Celery ~= 5.0.\n\nIf you're running an older version of Python, you need to be running\nan older version of celery-batches:\n\n- Python 2.7: celery-batches 0.3.\n- Python 3.4: celery-batches 0.2.\n- Python 3.5: celery-batches 0.3.\n- Python 3.6: celery-batches 0.5.\n- Python 3.7: celery-batches 0.7.\n\nIf you're running an older version of Celery, you need to be running\nan older version of celery-batches:\n\n- Celery < 4.0: Use `celery.contrib.batches` instead.\n- Celery 4.0 - 4.3: celery-batches 0.3.\n- Celery 4.4: celery-batches 0.7.\n\nHistory\n=======\n\nCelery Batches was distributed as part of Celery (as ``celery.contrib.batches``)\nuntil Celery 4.0. This project updates the Batches code to maintain compatiblity\nwith newer versions of Celery and other fixes. See the Changelog for details.\n\nAdditionally, this repository includes the full history of the code from\n``celery.contrib.batches``, but rewritten to the ``celery_batches/__init__.py``\nfile.\n",
"bugtrack_url": null,
"license": "BSD",
"summary": "Experimental task class that buffers messages and processes them as a list.",
"version": "0.9",
"project_urls": {
"Documentation": "https://celery-batches.readthedocs.io",
"Funding": "https://github.com/sponsors/clokep",
"Homepage": "https://github.com/clokep/celery-batches",
"Release notes": "https://github.com/clokep/celery-batches/blob/main/CHANGELOG.rst",
"Source": "https://github.com/clokep/celery-batches",
"Tracker": "https://github.com/clokep/celery-batches/issues"
},
"split_keywords": [
"task",
" job",
" queue",
" distributed",
" messaging",
" actor"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3e49127369856386c0fd7ddd3577afa5c5dfca85a53ce2be3d3a5698705f082c",
"md5": "270a013fb4fad6e8348b047177f837d3",
"sha256": "532ae4f32026b1eac0810dace13389a904611259bbefab816699f77551efbf5b"
},
"downloads": -1,
"filename": "celery_batches-0.9-py3-none-any.whl",
"has_sig": false,
"md5_digest": "270a013fb4fad6e8348b047177f837d3",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 8597,
"upload_time": "2024-06-03T20:34:26",
"upload_time_iso_8601": "2024-06-03T20:34:26.902595Z",
"url": "https://files.pythonhosted.org/packages/3e/49/127369856386c0fd7ddd3577afa5c5dfca85a53ce2be3d3a5698705f082c/celery_batches-0.9-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5bc0b89d650c1e29b41b5d117e4b2d35c60155542fed1fae44ea4ecb6171aca0",
"md5": "46cb4dfffe33495e6756355aaa2d8309",
"sha256": "4dd63588314d64491aae9384bd54df4fd00b11250ea39c9bdbb79dca6aee9797"
},
"downloads": -1,
"filename": "celery_batches-0.9.tar.gz",
"has_sig": false,
"md5_digest": "46cb4dfffe33495e6756355aaa2d8309",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 8448,
"upload_time": "2024-06-03T20:34:28",
"upload_time_iso_8601": "2024-06-03T20:34:28.442536Z",
"url": "https://files.pythonhosted.org/packages/5b/c0/b89d650c1e29b41b5d117e4b2d35c60155542fed1fae44ea4ecb6171aca0/celery_batches-0.9.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-06-03 20:34:28",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "clokep",
"github_project": "celery-batches",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"tox": true,
"lcname": "celery-batches"
}
JSON
