.. teaser-begin
==========
``aqueue``
==========
``aqueue`` is an async task queue with live progress display.
You put items (tasks) in, and they get processed, possibly creating more items which
get processed, and so on, until all items are completed. A typical use case would be to scrape a
website.
Meanwhile, a nice visualization of the queue's goings-on is displayed in the terminal.
.. image:: https://raw.githubusercontent.com/t-mart/aqueue/master/docs/_static/demo.gif
:alt: Demonstration of aqueue
.. note::
``aqueue``, or any asynchronous framework, is only going to be helpful if you're performing
**I/O-bound** work.
Installation
============
``aqueue`` is a Python package `hosted on PyPI <https://pypi.org/project/aqueue/>`_. The recommended
installation method is `pip <https://pip.pypa.io/en/stable/>`_-installing into a virtual
environment:
.. code-block:: shell
pip install aqueue
Getting Started
===============
There's two things you need to do to use aqueue:
1. Implement your `Item <https://t-mart.github.io/aqueue/#items>`_ subclasses.
2. `Start your queue <https://t-mart.github.io/aqueue/#starting-your-queue>`_ with one of those
items.
.. teaser-end
Example
=======
If you had a hierarchy of items like this...
.. image:: docs/_static/simple-diagram.png
:alt: Simple item hierarchy with one root item and many children items stemming from it.
Then, you might process it with ``aqueue`` like this...
.. code-block:: python
import aqueue
class RootItem(aqueue.Item):
async def process(self) -> aqueue.ProcessRetVal:
# display what we're doing in the worker status panel
self.set_worker_desc("Processing RootItem")
# make an HTTP request, parse it, etc
...
# when you discover more items you want to process, enqueue them by yield-ing
# them:
for _ in range(3):
yield ChildItem()
async def after_children_processed(self) -> None:
# run this method when this Item and all other Items it enqueued are done
print("All done!")
class ChildItem(aqueue.Item):
# track the enqueueing and completion of these items in the overall panel
track_overall: bool = True
async def process(self) -> aqueue.ProcessRetVal:
self.set_worker_desc("Processing ChildItem")
# this child item has no further children to enqueue, so it doesn't yield
# anything
if __name__ == "__main__":
aqueue.run_queue(
initial_items=[RootItem()],
num_workers=2,
)
.. -project-information-
Project Information
===================
- **License**: `MIT <https://choosealicense.com/licenses/mit/>`_
- **PyPI**: https://pypi.org/project/aqueue/
- **Source Code**: https://github.com/t-mart/aqueue
- **Documentation**: https://t-mart.github.io/aqueue/
- **Supported Python Versions**: 3.10 and later
Raw data
{
"_id": null,
"home_page": "https://t-mart.github.io/aqueue/",
"name": "aqueue",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.12,<4.0",
"maintainer_email": "",
"keywords": "queue,async,task",
"author": "Tim Martin",
"author_email": "tim@timmart.in",
"download_url": "https://files.pythonhosted.org/packages/43/ca/88aa93bc89a8e2d41e4f351799f9e88efb9648564d4531900548eec638b9/aqueue-0.9.2.tar.gz",
"platform": null,
"description": ".. teaser-begin\n\n==========\n``aqueue``\n==========\n\n``aqueue`` is an async task queue with live progress display.\n\nYou put items (tasks) in, and they get processed, possibly creating more items which\nget processed, and so on, until all items are completed. A typical use case would be to scrape a\nwebsite.\n\nMeanwhile, a nice visualization of the queue's goings-on is displayed in the terminal.\n\n.. image:: https://raw.githubusercontent.com/t-mart/aqueue/master/docs/_static/demo.gif\n :alt: Demonstration of aqueue\n\n.. note::\n\n ``aqueue``, or any asynchronous framework, is only going to be helpful if you're performing\n **I/O-bound** work.\n\n\nInstallation\n============\n\n``aqueue`` is a Python package `hosted on PyPI <https://pypi.org/project/aqueue/>`_. The recommended\ninstallation method is `pip <https://pip.pypa.io/en/stable/>`_-installing into a virtual\nenvironment:\n\n.. code-block:: shell\n\n pip install aqueue\n\nGetting Started\n===============\n\nThere's two things you need to do to use aqueue:\n\n1. Implement your `Item <https://t-mart.github.io/aqueue/#items>`_ subclasses.\n2. `Start your queue <https://t-mart.github.io/aqueue/#starting-your-queue>`_ with one of those\n items.\n\n.. teaser-end\n\nExample\n=======\n\nIf you had a hierarchy of items like this...\n\n.. image:: docs/_static/simple-diagram.png\n :alt: Simple item hierarchy with one root item and many children items stemming from it.\n\nThen, you might process it with ``aqueue`` like this...\n\n.. code-block:: python\n\n import aqueue\n\n\n class RootItem(aqueue.Item):\n async def process(self) -> aqueue.ProcessRetVal:\n # display what we're doing in the worker status panel\n self.set_worker_desc(\"Processing RootItem\")\n\n # make an HTTP request, parse it, etc\n ...\n\n # when you discover more items you want to process, enqueue them by yield-ing\n # them:\n for _ in range(3):\n yield ChildItem()\n\n async def after_children_processed(self) -> None:\n # run this method when this Item and all other Items it enqueued are done\n print(\"All done!\")\n\n\n class ChildItem(aqueue.Item):\n\n # track the enqueueing and completion of these items in the overall panel\n track_overall: bool = True\n\n async def process(self) -> aqueue.ProcessRetVal:\n self.set_worker_desc(\"Processing ChildItem\")\n # this child item has no further children to enqueue, so it doesn't yield\n # anything\n\n\n if __name__ == \"__main__\":\n aqueue.run_queue(\n initial_items=[RootItem()],\n num_workers=2,\n )\n\n.. -project-information-\n\nProject Information\n===================\n\n- **License**: `MIT <https://choosealicense.com/licenses/mit/>`_\n- **PyPI**: https://pypi.org/project/aqueue/\n- **Source Code**: https://github.com/t-mart/aqueue\n- **Documentation**: https://t-mart.github.io/aqueue/\n- **Supported Python Versions**: 3.10 and later\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "An async task queue with live progress display",
"version": "0.9.2",
"project_urls": {
"Documentation": "https://t-mart.github.io/aqueue/",
"Homepage": "https://t-mart.github.io/aqueue/",
"Repository": "https://github.com/t-mart/aqueue"
},
"split_keywords": [
"queue",
"async",
"task"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d1b3dc26a517ff7f3f0654b5bb37709ea703c892121aec24d9f6313bd2cb9ada",
"md5": "8f558100f84ddf5edcd0500a04cab53c",
"sha256": "129965603f914afe2639127d5ce3c281fc58c6f511d69f1b6c9d798913ef3a38"
},
"downloads": -1,
"filename": "aqueue-0.9.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8f558100f84ddf5edcd0500a04cab53c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.12,<4.0",
"size": 11844,
"upload_time": "2024-03-05T19:44:29",
"upload_time_iso_8601": "2024-03-05T19:44:29.929954Z",
"url": "https://files.pythonhosted.org/packages/d1/b3/dc26a517ff7f3f0654b5bb37709ea703c892121aec24d9f6313bd2cb9ada/aqueue-0.9.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "43ca88aa93bc89a8e2d41e4f351799f9e88efb9648564d4531900548eec638b9",
"md5": "48a14a2e8d5beb13af69040345885070",
"sha256": "96bead7a0576df06d5ecca4437cf2d5425a7a1d55312739397d02b671db5fca2"
},
"downloads": -1,
"filename": "aqueue-0.9.2.tar.gz",
"has_sig": false,
"md5_digest": "48a14a2e8d5beb13af69040345885070",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.12,<4.0",
"size": 11523,
"upload_time": "2024-03-05T19:44:31",
"upload_time_iso_8601": "2024-03-05T19:44:31.486607Z",
"url": "https://files.pythonhosted.org/packages/43/ca/88aa93bc89a8e2d41e4f351799f9e88efb9648564d4531900548eec638b9/aqueue-0.9.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-05 19:44:31",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "t-mart",
"github_project": "aqueue",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "aqueue"
}
JSON
