.. image:: http://media.charlesleifer.com/blog/photos/huey2-logo.png
*a lightweight alternative*.
huey is:
* a task queue
* written in python
* clean and simple API
* redis, sqlite, file-system, or in-memory storage
* `example code <https://github.com/coleifer/huey/tree/master/examples/>`_.
* `read the documentation <https://huey.readthedocs.io/>`_.
huey supports:
* multi-process, multi-thread or greenlet task execution models
* schedule tasks to execute at a given time, or after a given delay
* schedule recurring tasks, like a crontab
* automatically retry tasks that fail
* task prioritization
* task result storage
* task expiration
* task locking
* task pipelines and chains
.. image:: http://i.imgur.com/2EpRs.jpg
At a glance
-----------
.. code-block:: python
from huey import RedisHuey, crontab
huey = RedisHuey('my-app', host='redis.myapp.com')
@huey.task()
def add_numbers(a, b):
return a + b
@huey.task(retries=2, retry_delay=60)
def flaky_task(url):
# This task might fail, in which case it will be retried up to 2 times
# with a delay of 60s between retries.
return this_might_fail(url)
@huey.periodic_task(crontab(minute='0', hour='3'))
def nightly_backup():
sync_all_data()
Calling a ``task``-decorated function will enqueue the function call for
execution by the consumer. A special result handle is returned immediately,
which can be used to fetch the result once the task is finished:
.. code-block:: pycon
>>> from demo import add_numbers
>>> res = add_numbers(1, 2)
>>> res
<Result: task 6b6f36fc-da0d-4069-b46c-c0d4ccff1df6>
>>> res()
3
Tasks can be scheduled to run in the future:
.. code-block:: pycon
>>> res = add_numbers.schedule((2, 3), delay=10) # Will be run in ~10s.
>>> res(blocking=True) # Will block until task finishes, in ~10s.
5
For much more, check out the `guide <https://huey.readthedocs.io/en/latest/guide.html>`_
or take a look at the `example code <https://github.com/coleifer/huey/tree/master/examples/>`_.
Running the consumer
^^^^^^^^^^^^^^^^^^^^
Run the consumer with four worker processes:
.. code-block:: console
$ huey_consumer.py my_app.huey -k process -w 4
To run the consumer with a single worker thread (default):
.. code-block:: console
$ huey_consumer.py my_app.huey
If your work-loads are mostly IO-bound, you can run the consumer with threads
or greenlets instead. Because greenlets are so lightweight, you can run quite a
few of them efficiently:
.. code-block:: console
$ huey_consumer.py my_app.huey -k greenlet -w 32
Storage
-------
Huey's design and feature-set were informed by the capabilities of the
`Redis <https://redis.io>`_ database. Redis is a fantastic fit for a
lightweight task queueing library like Huey: it's self-contained, versatile,
and can be a multi-purpose solution for other web-application tasks like
caching, event publishing, analytics, rate-limiting, and more.
Although Huey was designed with Redis in mind, the storage system implements a
simple API and many other tools could be used instead of Redis if that's your
preference.
Huey comes with builtin support for Redis, Sqlite and in-memory storage.
Documentation
----------------
`See Huey documentation <https://huey.readthedocs.io/>`_.
Project page
---------------
`See source code and issue tracker on Github <https://github.com/coleifer/huey/>`_.
Huey is named in honor of my cat:
.. image:: http://m.charlesleifer.com/t/800x-/blog/photos/p1473037658.76.jpg?key=mD9_qMaKBAuGPi95KzXYqg
Raw data
{
"_id": null,
"home_page": "http://github.com/coleifer/huey/",
"name": "huey",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "Charles Leifer",
"author_email": "coleifer@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/9c/fe/2e063984cdd512aa71e9c9c2a9200b58a830c532d25ca2c6cbc8e44bf7b7/huey-2.5.2.tar.gz",
"platform": null,
"description": ".. image:: http://media.charlesleifer.com/blog/photos/huey2-logo.png\n\n*a lightweight alternative*.\n\nhuey is:\n\n* a task queue\n* written in python\n* clean and simple API\n* redis, sqlite, file-system, or in-memory storage\n* `example code <https://github.com/coleifer/huey/tree/master/examples/>`_.\n* `read the documentation <https://huey.readthedocs.io/>`_.\n\nhuey supports:\n\n* multi-process, multi-thread or greenlet task execution models\n* schedule tasks to execute at a given time, or after a given delay\n* schedule recurring tasks, like a crontab\n* automatically retry tasks that fail\n* task prioritization\n* task result storage\n* task expiration\n* task locking\n* task pipelines and chains\n\n.. image:: http://i.imgur.com/2EpRs.jpg\n\nAt a glance\n-----------\n\n.. code-block:: python\n\n from huey import RedisHuey, crontab\n\n huey = RedisHuey('my-app', host='redis.myapp.com')\n\n @huey.task()\n def add_numbers(a, b):\n return a + b\n\n @huey.task(retries=2, retry_delay=60)\n def flaky_task(url):\n # This task might fail, in which case it will be retried up to 2 times\n # with a delay of 60s between retries.\n return this_might_fail(url)\n\n @huey.periodic_task(crontab(minute='0', hour='3'))\n def nightly_backup():\n sync_all_data()\n\nCalling a ``task``-decorated function will enqueue the function call for\nexecution by the consumer. A special result handle is returned immediately,\nwhich can be used to fetch the result once the task is finished:\n\n.. code-block:: pycon\n\n >>> from demo import add_numbers\n >>> res = add_numbers(1, 2)\n >>> res\n <Result: task 6b6f36fc-da0d-4069-b46c-c0d4ccff1df6>\n\n >>> res()\n 3\n\nTasks can be scheduled to run in the future:\n\n.. code-block:: pycon\n\n >>> res = add_numbers.schedule((2, 3), delay=10) # Will be run in ~10s.\n >>> res(blocking=True) # Will block until task finishes, in ~10s.\n 5\n\nFor much more, check out the `guide <https://huey.readthedocs.io/en/latest/guide.html>`_\nor take a look at the `example code <https://github.com/coleifer/huey/tree/master/examples/>`_.\n\nRunning the consumer\n^^^^^^^^^^^^^^^^^^^^\n\nRun the consumer with four worker processes:\n\n.. code-block:: console\n\n $ huey_consumer.py my_app.huey -k process -w 4\n\nTo run the consumer with a single worker thread (default):\n\n.. code-block:: console\n\n $ huey_consumer.py my_app.huey\n\nIf your work-loads are mostly IO-bound, you can run the consumer with threads\nor greenlets instead. Because greenlets are so lightweight, you can run quite a\nfew of them efficiently:\n\n.. code-block:: console\n\n $ huey_consumer.py my_app.huey -k greenlet -w 32\n\nStorage\n-------\n\nHuey's design and feature-set were informed by the capabilities of the\n`Redis <https://redis.io>`_ database. Redis is a fantastic fit for a\nlightweight task queueing library like Huey: it's self-contained, versatile,\nand can be a multi-purpose solution for other web-application tasks like\ncaching, event publishing, analytics, rate-limiting, and more.\n\nAlthough Huey was designed with Redis in mind, the storage system implements a\nsimple API and many other tools could be used instead of Redis if that's your\npreference.\n\nHuey comes with builtin support for Redis, Sqlite and in-memory storage.\n\nDocumentation\n----------------\n\n`See Huey documentation <https://huey.readthedocs.io/>`_.\n\nProject page\n---------------\n\n`See source code and issue tracker on Github <https://github.com/coleifer/huey/>`_.\n\nHuey is named in honor of my cat:\n\n.. image:: http://m.charlesleifer.com/t/800x-/blog/photos/p1473037658.76.jpg?key=mD9_qMaKBAuGPi95KzXYqg\n\n",
"bugtrack_url": null,
"license": null,
"summary": "huey, a little task queue",
"version": "2.5.2",
"project_urls": {
"Homepage": "http://github.com/coleifer/huey/"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "9cfe2e063984cdd512aa71e9c9c2a9200b58a830c532d25ca2c6cbc8e44bf7b7",
"md5": "c5e364ade0f02369bbe550dffc03e9bd",
"sha256": "df33db474c05414ed40ee2110e9df692369871734da22d74ffb035a4bd74047f"
},
"downloads": -1,
"filename": "huey-2.5.2.tar.gz",
"has_sig": false,
"md5_digest": "c5e364ade0f02369bbe550dffc03e9bd",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 889357,
"upload_time": "2024-09-25T18:03:33",
"upload_time_iso_8601": "2024-09-25T18:03:33.092408Z",
"url": "https://files.pythonhosted.org/packages/9c/fe/2e063984cdd512aa71e9c9c2a9200b58a830c532d25ca2c6cbc8e44bf7b7/huey-2.5.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-25 18:03:33",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "coleifer",
"github_project": "huey",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "huey"
}
JSON
