| Name | queue-processor JSON |
| Version |
1.1.2
JSON |
| download |
| home_page | |
| Summary | Python3 thread based async queue manager |
| upload_time | 2023-12-26 03:06:23 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.6 |
| license | |
| keywords |
queue
thread
async
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# queue_processor Python3 Queue
Homepage: https://www.learningtopi.com/python-modules-applications/queue_processor/
This package is a simple threading based queue manager. A queue can be created
with a specific maximum length as well as default command and final callback functions.
When an item is added to the queue, the queue_processor will create a thread to execute
the command function that was either provided when the item was added, or the default
function for the queue. Arguments will be passed to the function and the return
value will be captured. If a funal callback function is provided, the return value
as well as the status will be returned along with a copy of the arguments.
The purpose of this class is to provide a modular and reusable queue that can be
used in many different projects for any async tasks that need to be performed in
sequence. The queue_processor operates as a FIFO queue and can provide a delay
between tasks (i.e. for an IR transmitter where a pause is needed).
## Use Example
from queue_processor import QueueManager, STATUS_OK, STATUS_TIMEOUT, STATUS_EXPIRED, STATUS_QUEUE_FULL, STATUS_EXCEPTION
def task(arg1, arg2):
print(f'executing task with {arg1}, {arg2}')
return arg1
def finished_callback(return_value, status, *args, **kwargs):
print(f'Completed task return value: {return_value}, status {status}, args: {args}, kwargs: {kwargs}')
queue = QueueManager(name='test1', command_func=task, callback_func=finished_callback)
queue.add(kwargs={'arg1': 1, 'arg2': 2})
## Command Function
The command function can be any callable function (including a function that is part of a class). The function will be passed all the positional and keyword arguments that are supplied when adding the instance to the queue.
## Callback Function
The callback function can be any callable function (including a function that is part of a class). This OPTIONAL function is called after the command function either completes or times out. The callback will provide the return value of the command function (if any) as well as a status that will be one of the following:
- OK: Command function completed (may or may not be successful in your view, but the function completed and did not raise any errors)
- TIMEOUT: If the command function did not complete within the timeout period.
- EXPIRED: The item was NOT executed as it sat in the queue longer than the max time permitted.
- QUEUE_FULL: This is returned if a callback is provided when an item is attempted to be added but the queue is full. This item is NOT executed.
- EXCEPTION: An exception was raised during the execution of the callback. The exception is returned as the "return_value"
### Callback function parameters
The callback function must accept the "return_value" and "status" as positional arguments in that order. It is strongly recommended to include *args and **kwargs to catch all positional and keyword arguments that are sent after "return_value" and "status". The queue processor will send ALL arguments to the callback function that were also sent to the command function.
NOTE: You may send objects as parameters that will be updated by the command function! You may also have the command function be a member of a class instance. This allows you to act on data within an instance of a class so passing objects may not be required.
!!IMPORTANT!! The queue processor uses Python threads. In your command and callback function be sure to use thread locks appropriately if modifying data that may be accessed by a different thread. Also be sure not to create deadlocks!
Raw data
{
"_id": null,
"home_page": "",
"name": "queue-processor",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "queue,thread,async",
"author": "",
"author_email": "Thomas Dunteman <git@learningtopi.com>",
"download_url": "https://files.pythonhosted.org/packages/d6/33/2adfd9f0c2b33f09d5e20cbf85f9e2091b8dcdacb14e57bf7f5df19bc0e1/queue_processor-1.1.2.tar.gz",
"platform": null,
"description": "# queue_processor Python3 Queue\nHomepage: https://www.learningtopi.com/python-modules-applications/queue_processor/\n\nThis package is a simple threading based queue manager. A queue can be created\nwith a specific maximum length as well as default command and final callback functions.\nWhen an item is added to the queue, the queue_processor will create a thread to execute\nthe command function that was either provided when the item was added, or the default\nfunction for the queue. Arguments will be passed to the function and the return\nvalue will be captured. If a funal callback function is provided, the return value\nas well as the status will be returned along with a copy of the arguments.\n\nThe purpose of this class is to provide a modular and reusable queue that can be\nused in many different projects for any async tasks that need to be performed in\nsequence. The queue_processor operates as a FIFO queue and can provide a delay\nbetween tasks (i.e. for an IR transmitter where a pause is needed).\n\n## Use Example\n\n from queue_processor import QueueManager, STATUS_OK, STATUS_TIMEOUT, STATUS_EXPIRED, STATUS_QUEUE_FULL, STATUS_EXCEPTION\n\n def task(arg1, arg2):\n print(f'executing task with {arg1}, {arg2}')\n return arg1\n\n def finished_callback(return_value, status, *args, **kwargs):\n print(f'Completed task return value: {return_value}, status {status}, args: {args}, kwargs: {kwargs}')\n\n queue = QueueManager(name='test1', command_func=task, callback_func=finished_callback)\n queue.add(kwargs={'arg1': 1, 'arg2': 2})\n\n## Command Function\n\nThe command function can be any callable function (including a function that is part of a class). The function will be passed all the positional and keyword arguments that are supplied when adding the instance to the queue.\n\n## Callback Function\n\nThe callback function can be any callable function (including a function that is part of a class). This OPTIONAL function is called after the command function either completes or times out. The callback will provide the return value of the command function (if any) as well as a status that will be one of the following:\n- OK: Command function completed (may or may not be successful in your view, but the function completed and did not raise any errors)\n- TIMEOUT: If the command function did not complete within the timeout period.\n- EXPIRED: The item was NOT executed as it sat in the queue longer than the max time permitted.\n- QUEUE_FULL: This is returned if a callback is provided when an item is attempted to be added but the queue is full. This item is NOT executed.\n- EXCEPTION: An exception was raised during the execution of the callback. The exception is returned as the \"return_value\"\n\n### Callback function parameters\n\nThe callback function must accept the \"return_value\" and \"status\" as positional arguments in that order. It is strongly recommended to include *args and **kwargs to catch all positional and keyword arguments that are sent after \"return_value\" and \"status\". The queue processor will send ALL arguments to the callback function that were also sent to the command function.\n\nNOTE: You may send objects as parameters that will be updated by the command function! You may also have the command function be a member of a class instance. This allows you to act on data within an instance of a class so passing objects may not be required.\n\n!!IMPORTANT!! The queue processor uses Python threads. In your command and callback function be sure to use thread locks appropriately if modifying data that may be accessed by a different thread. Also be sure not to create deadlocks!\n",
"bugtrack_url": null,
"license": "",
"summary": "Python3 thread based async queue manager",
"version": "1.1.2",
"project_urls": {
"bug tracker": "https://github.com/LearningToPi/queue_processor/issues",
"homepage": "https://www.learningtopi.com/python-modules-applications/queue_processor/",
"source code": "https://github.com/LearningToPi/queue_processor"
},
"split_keywords": [
"queue",
"thread",
"async"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e8c18cf745a1e35b17f19dba9cb76bf8c442ce743ee3fae12aa71ad925dd7e31",
"md5": "e134c291732e1a466ee73a712d9f4b7d",
"sha256": "10c800fc71be44bc7da8fea3ce29e1abbad44f4f89e23bf80d686383ed7fc608"
},
"downloads": -1,
"filename": "queue_processor-1.1.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e134c291732e1a466ee73a712d9f4b7d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 9204,
"upload_time": "2023-12-26T03:06:22",
"upload_time_iso_8601": "2023-12-26T03:06:22.211957Z",
"url": "https://files.pythonhosted.org/packages/e8/c1/8cf745a1e35b17f19dba9cb76bf8c442ce743ee3fae12aa71ad925dd7e31/queue_processor-1.1.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d6332adfd9f0c2b33f09d5e20cbf85f9e2091b8dcdacb14e57bf7f5df19bc0e1",
"md5": "1ab881f9d19aa80a91f4f6850e026aa3",
"sha256": "ff391440d00523fd2c430d8c8219e3665ddaac81d3b6dc5649019565f8804329"
},
"downloads": -1,
"filename": "queue_processor-1.1.2.tar.gz",
"has_sig": false,
"md5_digest": "1ab881f9d19aa80a91f4f6850e026aa3",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 8693,
"upload_time": "2023-12-26T03:06:23",
"upload_time_iso_8601": "2023-12-26T03:06:23.252733Z",
"url": "https://files.pythonhosted.org/packages/d6/33/2adfd9f0c2b33f09d5e20cbf85f9e2091b8dcdacb14e57bf7f5df19bc0e1/queue_processor-1.1.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-12-26 03:06:23",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "LearningToPi",
"github_project": "queue_processor",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "queue-processor"
}
