| Name | asyncio-scheduler JSON |
| Version |
1.0.4
JSON |
| download |
| home_page | None |
| Summary | Asyncio scheduler with tasks |
| upload_time | 2024-10-01 19:23:32 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.8 |
| license | MIT License Copyright (c) 2024 rendaduiyan-python Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
async
event loop
multi-threads
scheduler
tasks
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# async_scheduler
A job scheduler that is designed to manage and execute tasks asynchronously.
There are two options to use async scheduler:
* one thread and single event loop - AsyncScheduler
This is most common way to use Python asynchronous frameworks. Be noted, all tasks have to carefully implemented so that slow tasks will not block the main event loop.
Sometimes, it's hard to tell from the very beginning what task is slow and whether or not it blocks the event loop. Basically, you can leverage debug tool from asyncio to tell the slow tasks:
```
loop = asyncio.get_running_loop()
loop.set_debug(True)
loop.slow_callback_duration = 5
```
To use AsyncScheduler, just deriving from SyncTask for possible slow tasks and from AsyncTask for others.
* multipe threads with multiple event loop - SchedulerThread
Tasks may differ with priorities, i.e, some tasks need to be repsonded immediately but some not. These urgent tasks are usually much less, compared to normal tasks; or differnt types of tasks, like network tasks, UI tasks, etc. Then SchedulerThread is designed for this use case - multiple threads with multiple event loops attached. Tasks in one thread/event loop will not block other threads and event loops. Of course, it's more complicated since it has to make sure only thread-safe APIs are called and tasks are waited between event loops.
# Task states
A serial of states are defined for tasks:
* NEW
* SUBMITTED
* SCHEDULED
* STARTED
* TIMED_OUT
* FAILED
* DONE
* CANCELED
* RUNNING
* FINISHED
Accurate timestamps are recorded down for critical states, including:
* submit_ts
* start_ts
* timeout_ts
* error_ts
* done_ts
* cancel_ts
So all tasks can be accurately measured. Furthermore, callbacks can be defined to those critial states, i.e, submit_cb, start_cb, etc.
AsyncScheduler provides a method to retrive task state and in addition a method to cancel those slow tasks.
# What does OpenAI says
The AsyncScheduler class is responsible for submitting tasks, scheduling and executing tasks, and managing task states. It makes use of an asyncio.Queue to manage the queue of tasks to be executed, and a dictionary to store the tasks with their associated data. The tasks are executed in a separate thread pool using asyncio.run_in_executor() method.
The code provides methods to get the current state of a task, wait for a task to complete, cancel a task, and dump task information. It also provides a method to determine the slowest task and dump its information.
Overall, the code is well-structured and follows good coding practices. The use of dataclasses and enums makes the code more readable and maintainable. The code is also well-documented, which makes it easier to understand and use.
The AsyncScheduler class is a job scheduler that is designed to manage and execute tasks asynchronously. It is built on top of the asyncio library and provides several features such as task submission, cancellation, and monitoring.
The class defines several helper classes and functions such as TaskState, TaskType, TaskData, TaskTS, and CallerData to represent and manage various aspects of a task. It also provides two concrete implementations of the Task interface, namely AsyncTask and SyncTask, to handle asynchronous and synchronous tasks respectively.
The class uses an internal queue to manage task submission and scheduling. It runs an infinite loop that continuously checks the queue for new tasks and schedules them to run using asyncio. Each task is executed in its own asyncio.Task object, which allows multiple tasks to run concurrently.
The class provides several public methods to interact with the scheduler, such as submit_task(), task_state(), wait_task(), cancel_task(), dump_task(), and slowest(). These methods allow you to submit tasks, monitor their states, wait for them to complete, cancel them if necessary, list all tasks and their current status, and find the slowest running task.
Here's an example of how you can use the AsyncScheduler class in your code:
```python
import asyncio
from async_scheduler import AsyncScheduler, AsyncTask
async def my_async_task():
await asyncio.sleep(1)
print("Async task completed")
def my_callback(task_ts):
if task_ts.state == TaskState.DONE:
print("Task done")
def main():
loop = asyncio.get_event_loop()
scheduler = AsyncScheduler(ioloop=loop)
task = AsyncTask("my_async_task")
task.run_async = my_async_task
task.run_async = my_async_task # error: task.done_cb = my_callback
task.data.timeout = 2
scheduler.start()
task_uuid = scheduler.submit_task(task)
print(f"Task submitted with UUID: {task_uuid}")
loop.run_until_complete(scheduler.wait_task(task_uuid))
print(scheduler.task_info(task_uuid))
scheduler.stop()
loop.close()
if __name__ == "__main__":
main()
```
In this example, we define an asynchronous task called `my_async_task()` that sleeps for 1 second and prints a message. We create a new instance of the AsyncScheduler class, add the task to the queue using `submit_task()`, and wait for it to complete using `wait_task()`. We also define a callback function `my_callback()` to print a message when the task is done.
The output of this program would be:
```
Task submitted with UUID: 8b479239-8cf2-4a8c-9d22-bce03184f60c
Task(name='my_async_task', task_type=<TaskType.ASYNC: 1>, timeout=2) [{'args': (), 'kwargs': {}}]
Async task completed
('Task(name=\'my_async_task\', task_type=<TaskType.ASYNC: 1>, timeout=2)', [], {})
```
This shows that the task was submitted, executed, and returned successfully. The `task_info()` method was used to print the task details after completion.
Raw data
{
"_id": null,
"home_page": null,
"name": "asyncio-scheduler",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "rendaduiyan <duiyanrenda@gmail.com>",
"keywords": "async, event loop, multi-threads, scheduler, tasks",
"author": null,
"author_email": "rendaduiyan <duiyanrenda@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/da/1e/4d96d80f820ad7d4feb43057003c0aa13b964b3c0f274c2926b6afde6223/asyncio_scheduler-1.0.4.tar.gz",
"platform": null,
"description": "# async_scheduler\nA job scheduler that is designed to manage and execute tasks asynchronously.\n\nThere are two options to use async scheduler:\n* one thread and single event loop - AsyncScheduler\n\n This is most common way to use Python asynchronous frameworks. Be noted, all tasks have to carefully implemented so that slow tasks will not block the main event loop.\n Sometimes, it's hard to tell from the very beginning what task is slow and whether or not it blocks the event loop. Basically, you can leverage debug tool from asyncio to tell the slow tasks:\n ```\n loop = asyncio.get_running_loop()\n loop.set_debug(True)\n loop.slow_callback_duration = 5\n ```\n To use AsyncScheduler, just deriving from SyncTask for possible slow tasks and from AsyncTask for others.\n \n* multipe threads with multiple event loop - SchedulerThread\n\n Tasks may differ with priorities, i.e, some tasks need to be repsonded immediately but some not. These urgent tasks are usually much less, compared to normal tasks; or differnt types of tasks, like network tasks, UI tasks, etc. Then SchedulerThread is designed for this use case - multiple threads with multiple event loops attached. Tasks in one thread/event loop will not block other threads and event loops. Of course, it's more complicated since it has to make sure only thread-safe APIs are called and tasks are waited between event loops.\n\n# Task states\n\nA serial of states are defined for tasks:\n* NEW\n* SUBMITTED\n* SCHEDULED\n* STARTED\n* TIMED_OUT\n* FAILED\n* DONE\n* CANCELED\n* RUNNING\n* FINISHED\n\nAccurate timestamps are recorded down for critical states, including:\n* submit_ts\n* start_ts\n* timeout_ts\n* error_ts\n* done_ts\n* cancel_ts\n\nSo all tasks can be accurately measured. Furthermore, callbacks can be defined to those critial states, i.e, submit_cb, start_cb, etc.\nAsyncScheduler provides a method to retrive task state and in addition a method to cancel those slow tasks.\n \n# What does OpenAI says\n\nThe AsyncScheduler class is responsible for submitting tasks, scheduling and executing tasks, and managing task states. It makes use of an asyncio.Queue to manage the queue of tasks to be executed, and a dictionary to store the tasks with their associated data. The tasks are executed in a separate thread pool using asyncio.run_in_executor() method.\n\nThe code provides methods to get the current state of a task, wait for a task to complete, cancel a task, and dump task information. It also provides a method to determine the slowest task and dump its information.\n\nOverall, the code is well-structured and follows good coding practices. The use of dataclasses and enums makes the code more readable and maintainable. The code is also well-documented, which makes it easier to understand and use.\n\nThe AsyncScheduler class is a job scheduler that is designed to manage and execute tasks asynchronously. It is built on top of the asyncio library and provides several features such as task submission, cancellation, and monitoring.\n\nThe class defines several helper classes and functions such as TaskState, TaskType, TaskData, TaskTS, and CallerData to represent and manage various aspects of a task. It also provides two concrete implementations of the Task interface, namely AsyncTask and SyncTask, to handle asynchronous and synchronous tasks respectively.\n\nThe class uses an internal queue to manage task submission and scheduling. It runs an infinite loop that continuously checks the queue for new tasks and schedules them to run using asyncio. Each task is executed in its own asyncio.Task object, which allows multiple tasks to run concurrently.\n\nThe class provides several public methods to interact with the scheduler, such as submit_task(), task_state(), wait_task(), cancel_task(), dump_task(), and slowest(). These methods allow you to submit tasks, monitor their states, wait for them to complete, cancel them if necessary, list all tasks and their current status, and find the slowest running task.\n\nHere's an example of how you can use the AsyncScheduler class in your code:\n\n```python\nimport asyncio\nfrom async_scheduler import AsyncScheduler, AsyncTask\n\n\nasync def my_async_task():\n await asyncio.sleep(1)\n print(\"Async task completed\")\n\n\ndef my_callback(task_ts):\n if task_ts.state == TaskState.DONE:\n print(\"Task done\")\n\n\ndef main():\n loop = asyncio.get_event_loop()\n scheduler = AsyncScheduler(ioloop=loop)\n\n task = AsyncTask(\"my_async_task\")\n task.run_async = my_async_task\n task.run_async = my_async_task # error: task.done_cb = my_callback\n task.data.timeout = 2\n\n scheduler.start()\n\n task_uuid = scheduler.submit_task(task)\n print(f\"Task submitted with UUID: {task_uuid}\")\n\n loop.run_until_complete(scheduler.wait_task(task_uuid))\n print(scheduler.task_info(task_uuid))\n\n scheduler.stop()\n loop.close()\n\n\nif __name__ == \"__main__\":\n main()\n```\n\nIn this example, we define an asynchronous task called `my_async_task()` that sleeps for 1 second and prints a message. We create a new instance of the AsyncScheduler class, add the task to the queue using `submit_task()`, and wait for it to complete using `wait_task()`. We also define a callback function `my_callback()` to print a message when the task is done.\n\nThe output of this program would be:\n\n```\nTask submitted with UUID: 8b479239-8cf2-4a8c-9d22-bce03184f60c\nTask(name='my_async_task', task_type=<TaskType.ASYNC: 1>, timeout=2) [{'args': (), 'kwargs': {}}]\nAsync task completed\n('Task(name=\\'my_async_task\\', task_type=<TaskType.ASYNC: 1>, timeout=2)', [], {})\n```\n\nThis shows that the task was submitted, executed, and returned successfully. The `task_info()` method was used to print the task details after completion.",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2024 rendaduiyan-python Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.",
"summary": "Asyncio scheduler with tasks",
"version": "1.0.4",
"project_urls": {
"Bug Tracker": "https://github.com/rendaduiyan-python/async_scheduler/issues",
"Changelog": "https://github.com/rendaduiyan-python/async_scheduler/CHANGELOG.md",
"Repository": "https://github.com/rendaduiyan-python/async_scheduler.git"
},
"split_keywords": [
"async",
" event loop",
" multi-threads",
" scheduler",
" tasks"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "520b56edbee519c57c0901d69b487fad3d515db2487238a2bdca3e1fc5b8cee3",
"md5": "8b4d3833ee6d10337d0e5a687b06aa97",
"sha256": "fbb577a68d19d5188246c625dd3643bbd76c50dab5e78ea3f49d7c43a52c6372"
},
"downloads": -1,
"filename": "asyncio_scheduler-1.0.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8b4d3833ee6d10337d0e5a687b06aa97",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 11104,
"upload_time": "2024-10-01T19:23:31",
"upload_time_iso_8601": "2024-10-01T19:23:31.112413Z",
"url": "https://files.pythonhosted.org/packages/52/0b/56edbee519c57c0901d69b487fad3d515db2487238a2bdca3e1fc5b8cee3/asyncio_scheduler-1.0.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "da1e4d96d80f820ad7d4feb43057003c0aa13b964b3c0f274c2926b6afde6223",
"md5": "d8b33c88f83395f240df9b403a069b06",
"sha256": "dad50942ae6f033d98936109ba28492319bbe10e2963be990ef61e72ddef6c90"
},
"downloads": -1,
"filename": "asyncio_scheduler-1.0.4.tar.gz",
"has_sig": false,
"md5_digest": "d8b33c88f83395f240df9b403a069b06",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 15501,
"upload_time": "2024-10-01T19:23:32",
"upload_time_iso_8601": "2024-10-01T19:23:32.023792Z",
"url": "https://files.pythonhosted.org/packages/da/1e/4d96d80f820ad7d4feb43057003c0aa13b964b3c0f274c2926b6afde6223/asyncio_scheduler-1.0.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-01 19:23:32",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "rendaduiyan-python",
"github_project": "async_scheduler",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "asyncio-scheduler"
}
