# Threads Utilities
### Threads utilities in Python.
[](https://coveralls.io/github/etuzon/pytohn-nrt-threads-utils)
[](https://app.deepsource.com/gh/etuzon/python-nrt-threads-utils/)
## Threads pool manager
### Description
Threads pool manager is a threads pool manager that manages a pool of task executors that execute tasks.
### Queue Placement Algorithms
| **Algorithm** | **Description** |
|-----------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `STRICT_PRIORITY` | Higher priority tasks are added to the queue before lower priority tasks. |
| `AVOID_STARVATION_PRIORITY` | Tasks are added to the queue in `STRICT_PRIORITY`,<br>and if tasks are not added to the last place in the queue in `avoid_starvation_amount` times,<br>the next task is added to the last place in the queue and will be flagged with `avoid_starvation` flag.<br>Other tasks will be added to the queue in `AVOID_STARVATION_PRIORITY` order after that flagged task, so this task will be executed before them. |
### Executors extension pool
The threads pool manager has an executors extension pool that extends the pool of task executors<br>
when the pool is full and at least one task executor run more than `executors_timeout_ms`.
## ThreadsPoolManager class
### Methods
| **Method** | **Description** | **Parameters** | **Returns** |
|-------------------------------------|-----------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------|
| `add_task` | Adds a task to the threads pool. | `task (ThreadTask \| MethodTask)` The task to add to the threads pool.<br>`task_id (Optional[str], Default: None)` Task identifier.<br>`priority (int, Default: 1)` Task Priority. Higher number is higher priority.<br>`queue_placement (QueuePlacementEnum)` Queue placement algorithm. | `None` |
| `get_task` | Gets a task from the threads pool. | `task_id (str)` Task identifier. | `ThreadTask \| MethodTask \| None` Return task if task is found, otherwise return None. |
| `is_task_exists` | Checks if task exists in the threads pool. | `task_id (str)` Task identifier. | `bool` True if task exists, otherwise False. |
| `reset_metrics` | Resets Metrics. | `None` | `None` |
| `start` | Starts the threads pool. | `None` | `None` |
| `shutdown` | Shuts down the threads pool. | `None` | `None` |
| `active_tasks_amount` | Returns the amount of active tasks in the threads pool. | `@propery` | `int` Active tasks amount. |
| `avoid_starvation_amount` | Returns Avoid Starvation Amount | `@propery` | `int` Avoid Starvation Amount. |
| `executors_extension_pool_size` | Returns Executors Extension Pool Size. | `@propery` | `int` Executors Extension Pool Size. |
| `executors_timeout_ms` | Returns Executors Timeout in milliseconds. | `@propery` | `int` Executors Timeout in milliseconds. |
| `executors_timeout_ms` | Sets Executors Timeout in milliseconds. | `@setter` Executors Timeout in milliseconds. | `None` |
| `finished_tasks` | Returns finished tasks.<br>Finished tasks list will be truncated after each call. | `@propery` | `List[ThreadTask \| MethodTask]` Finished tasks. |
| `is_executors_shutdown` | Returns True if executors pool is shutdown. | `@propery` | `bool` |
| `max_executors_extension_pool_size` | Returns Max Executors Extension Pool Size. | `@propery` | `int` Max Executors Extension Pool Size. |
| `max_executors_extension_pool_size` | Sets Max Executors Extension Pool Size. | `@setter` Max Executors Extension Pool Size. | `None` |
| `max_executors_pool_size` | Returns Max Executors Pool Size. | `@propery` | `int` Max Executors Pool Size. |
| `max_executors_pool_size` | Sets Max Executors Pool Size. | `@setter` Max Executors Pool Size. | `None` |
| `max_queue_size` | Returns Queue Size limitation. | `@propery` | `int` Max Queue Size. |
| `max_queue_size` | Sets Max Queue Size. | `@setter` Max Queue Size. | `None` |
| `metrics` | Returns Metrics. | `@propery` | `ThreadsPoolManagerMetrics` Metrics. |
| `name` | Returns Threads Pool Name. | `@propery` | `str` Threads Pool Name. |
| `name` | Sets Threads Pool Name. | `@setter` Threads Pool Name. | `None` |
| `queue` | Returns Queue. | `@propery` | `Queue` Queue. |
| `queue_size` | Returns Queue Size. | `@propery` | `int` Queue Size. |
| `shutdown` | Shuts down the threads pool. | `None` | `None` |
| `shutdown_executors` | Shuts down executors pool. | `None` | `None` |
| `start_executors` | Starts executors pool. | `None` | `None` |
## TaskExecutor class
### Methods
| **Method** | **Description** | **Parameters** | **Returns** |
|-------------------------|---------------------------------------------------------------------------------------|----------------|------------------------------|
| `start` | Starts the task executor. | `None` | `None` |
| `avoid_starvation_flag` | Returns Avoid Starvation Flag. | `@propery` | `bool` |
| `avoid_starvation_flag` | Sets Avoid Starvation Flag. | `@setter` | `None` |
| `exception` | Returns Exception, if it happened in method that was executed in MethodTask. | `@propery` | `Exception` |
| `stack_trace` | Returns stack trace, if exception happened in method that was executed in MethodTask. | `@propery` | `str` Exception stack trace. |
| `priority` | Returns priority. | `@propery` | `int` |
| `task` | Returns task. | `@propery` | `ThreadTask \| MethodTask` |
| `task_id` | Returns task id. | `@propery` | `str` |
## ThreadTask class
Task that runs a thread.
### Methods
| **Method** | **Description** | **Parameters** | **Returns** |
|-----------------|-----------------------------|----------------|--------------------------------|
| `execute` | Executes the task. | `None` | `None` |
| `task_instance` | Returns task instance. | `@propery` | `Thread` Task Thread instance. |
| `alive_date_ms` | Returns task alive date ms. | `@propery` | `int` Alive date in ms. |
| `start_date_ms` | Returns task start date ms. | `@propery` | `int` Start date in ms. |
| `start_date_ms` | Sets task start date ms. | `@setter` | `None` |
| `task_state` | Returns task state. | `@propery` | `TaskStateEnum` Task state. |
| `task_state` | Sets task state. | `@setter` | `None` |
## MethodTask class
Task that runs a method.
### Methods
| **Method** | **Description** | **Parameters** | **Returns** |
|-----------------|---------------------------------------------|----------------|------------------------------|
| `execute` - | Executes the task. | `None` | `None` |
| `alive_date_ms` | Returns task alive date ms. | `@propery` | `int` Alive date in ms. |
| `exception` | Returns Exception, if it happened. | `@propery` | `Exception` |
| `stack_trace` | Returns stack trace, if exception happened. | `@propery` | `str` Exception stack trace. |
| `start_date_ms` | Returns task start date ms. | `@propery` | `int` Start date in ms. |
| `start_date_ms` | Sets task start date ms. | `@setter` | `None` |
| `task_state` | Returns task state. | `@propery` | `TaskStateEnum` Task state. |
| `task_state` | Sets task state. | `@setter` | `None` |
## ThreadsPoolManagerMetrics class
dataclass that holds threads pool manager metrics.
### Properties
| **Property** | **Description** | **Type** |
|-------------------------------|------------------------------------------------------------------------------------------------------|------------------|
| `avoid_starvation_counter` | Count the times that `AVOID_STARVATION_PRORITY` flag has been raised. | `int` |
| `executed_method_counter` | Count the times that `MethodTask` has been executed. | `int` |
| `executed_task_counter` | Count the times that `ThreadTask\|MethodTask` has been executed. | `int` |
| `executed_thread_counter` | Count the times that `ThreadTask` has been executed. | `int` |
| `max_execution_date_ms` | Max execution date in ms. | `int` |
| `max_queue_size` | Max size that the queue has reached. | `int` |
| `method_tasks_counter_dict` | Count the times that `MethodTask` has been executed by task id.<br>key is task priority. | `dict[int, int]` |
| `tasks_priority_counter_dict` | Count the times that `ThreadTask\|MethodTask` has been executed by task id.<br>key is task priority. | `dict[int, int]` |
| `thread_tasks_counter_dict` | Count the times that `ThreadTask` has been executed by task id.<br>key is task priority. | `dict[int, int]` |
## TaskStateEnum Enum
Task state enum.
| **Enum** | **Description** |
|------------------|----------------------------|
| `QUEUE` | Task is in queue. |
| `EXECUTORS_POOL` | Task is in executors pool. |
| `EXECUTED` | Task is finished. |
## QueuePlacementEnum Enum
Queue placement enum.
| **Enum** | **Description** |
|-----------------------------|----------------------------|
| `STRICT_PRIORITY` | Strict Priority. |
| `AVOID_STARVATION_PRIORITY` | Avoid Starvation Priority. |
## Examples:
- ### Add tasks with `STRICT_PRIORITY` queue placement.
Create a threads pool manager with 2 executors pool size.
```python
from nrt_threads_utils.threads_pool_manager.threads_pool_manager import ThreadsPoolManager
threads_pool_manager = ThreadsPoolManager(executors_pool_size=2)
threads_pool_manager.start()
```
Add two tasks to the threads pool manager.<br>
The tasks will be executed as the two executors are empty.
```python
from nrt_threads_utils.threads_pool_manager.tasks import ThreadTask
t_1 = CustomThread()
t_2 = CustomThread()
threads_pool_manager.add_task(ThreadTask(t_1), priority=1)
threads_pool_manager.add_task(ThreadTask(t_1), priority=1)
```
Add 2 tasks in priority 1.<br>
Default queue placement is `STRICT_PRIORITY`.
```python
t_3 = CustomThread()
t_4 = CustomThread()
threads_pool_manager.add_task(ThreadTask(t_3), priority=1)
threads_pool_manager.add_task(ThreadTask(t_4), priority=1)
```
Add task in priority 2 (Higher priority)
Default queue placement is `STRICT_PRIORITY`.
```python
t_5 = CustomThread()
threads_pool_manager.add_task(ThreadTask(t_3), priority=2)
```
Task in priority 2 is executed before tasks in priority 1.
- ### Add tasks with `AVOID_STARVATION_PRIORITY` queue placement.
Create a threads pool manager with 2 executors pool size.
`avoid_starvation_amount` is set to 1.
```python
from nrt_threads_utils.threads_pool_manager.threads_pool_manager import ThreadsPoolManager
threads_pool_manager = ThreadsPoolManager(executors_pool_size=2)
threads_pool_manager.avoid_starvation_amount = 1
threads_pool_manager.start()
```
Add 2 tasks in priority 1.<br>
```python
from nrt_threads_utils.threads_pool_manager.tasks import ThreadTask
from nrt_threads_utils.threads_pool_manager.enums import QueuePlacementEnum
t_1 = CustomThread()
t_2 = CustomThread()
threads_pool_manager.add_task(
ThreadTask(t_1),
priority=1,
queue_placement=QueuePlacementEnum.AVOID_STARVATION_PRIORITY)
threads_pool_manager.add_task(
ThreadTask(t_2),
priority=1,
queue_placement=QueuePlacementEnum.AVOID_STARVATION_PRIORITY)
```
Add task in priority 2 (Higher priority)
```python
t_5 = CustomThread()
threads_pool_manager.add_task(
ThreadTask(t_3),
priority=2,
queue_placement=QueuePlacementEnum.AVOID_STARVATION_PRIORITY)
```
Task in priority 2 is executed before tasks in priority 1.<br>
Avoid starvation counter is increased by 1 because the task is not appended to the end of the queue.
Add another task in priority 2 (Higher priority)
```python
t_5 = CustomThread()
threads_pool_manager.add_task(
ThreadTask(t_3),
priority=2,
queue_placement=QueuePlacementEnum.AVOID_STARVATION_PRIORITY)
```
The task will flag with `avoid_starvation_flag` and will be added to the end of the queue.
Add the next tasks will be added in **Avoid Starvation Priority** after the flagged task.
- ### Metrics.
Create method that sleep for 10 seconds.<br><br>
**Code**
```python
from nrt_threads_utils.threads_pool_manager.threads_pool_manager import ThreadsPoolManager
from nrt_threads_utils.threads_pool_manager.tasks import MethodTask
from time import sleep
def sleep_10_sec():
sleep(10)
threads_pool_manager = \
ThreadsPoolManager(executors_pool_size=1)
try:
threads_pool_manager.start()
threads_pool_manager.add_task(MethodTask(sleep_10_sec), priority=1)
sleep(0.2)
threads_pool_manager.add_task(MethodTask(sleep_10_sec), priority=2)
threads_pool_manager.add_task(MethodTask(sleep_10_sec), priority=2)
metrics = threads_pool_manager.metrics
print(f'Max queue size: {metrics.max_queue_size}')
print(f'Max execution date ms {metrics.max_execution_date_ms}')
print(f'Executed tasks counter: {metrics.executed_tasks_counter}')
print(f'Executed threads counter: {metrics.executed_threads_counter}')
print(f'Executed methods counter: {metrics.executed_methods_counter}')
print(f'Avoid starvation counter: {metrics.avoid_starvation_counter}')
print(f'Tasks priority counter dict {metrics.tasks_priority_counter_dict}')
finally:
threads_pool_manager.shutdown()
threads_pool_manager.join()
```
**Output**
```
Max queue size: 2
Max execution date ms 10000
Executed tasks counter: 3
Executed threads counter: 0
Executed methods counter: 3
Avoid starvation counter: 0
Tasks priority counter dict {1: 1, 2: 2}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/etuzon/python-nrt-threads-utils",
"name": "nrt-threads-utils",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "python, python3, python-3, tool, tools, thread, threads, pool, threads-pool, util, nrt, nrt-utils, nrt-utilities, threads-utils, threads-utilities, nrt-utilities, nrt-threads-utils, nrt-threads-utilities",
"author": "Eyal Tuzon",
"author_email": "Eyal Tuzon <eyal.tuzon.dev@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/50/bf/eb9b6c7c6788d63c182cb80353c01fca346231d24588b9beadb0ac5fe6c8/nrt_threads_utils-2.0.5.tar.gz",
"platform": null,
"description": "# Threads Utilities\r\n\r\n### Threads utilities in Python.\r\n\r\n\r\n\r\n\r\n\r\n\r\n[](https://coveralls.io/github/etuzon/pytohn-nrt-threads-utils)\r\n\r\n\r\n[](https://app.deepsource.com/gh/etuzon/python-nrt-threads-utils/)\r\n\r\n## Threads pool manager\r\n\r\n### Description\r\n\r\nThreads pool manager is a threads pool manager that manages a pool of task executors that execute tasks.\r\n\r\n### Queue Placement Algorithms\r\n\r\n| **Algorithm** | **Description** |\r\n|-----------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\r\n| `STRICT_PRIORITY` | Higher priority tasks are added to the queue before lower priority tasks. |\r\n| `AVOID_STARVATION_PRIORITY` | Tasks are added to the queue in `STRICT_PRIORITY`,<br>and if tasks are not added to the last place in the queue in `avoid_starvation_amount` times,<br>the next task is added to the last place in the queue and will be flagged with `avoid_starvation` flag.<br>Other tasks will be added to the queue in `AVOID_STARVATION_PRIORITY` order after that flagged task, so this task will be executed before them. |\r\n\r\n### Executors extension pool\r\n\r\nThe threads pool manager has an executors extension pool that extends the pool of task executors<br>\r\nwhen the pool is full and at least one task executor run more than `executors_timeout_ms`.\r\n\r\n\r\n## ThreadsPoolManager class\r\n\r\n### Methods\r\n\r\n| **Method** | **Description** | **Parameters** | **Returns** |\r\n|-------------------------------------|-----------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------|\r\n| `add_task` | Adds a task to the threads pool. | `task (ThreadTask \\| MethodTask)` The task to add to the threads pool.<br>`task_id (Optional[str], Default: None)` Task identifier.<br>`priority (int, Default: 1)` Task Priority. Higher number is higher priority.<br>`queue_placement (QueuePlacementEnum)` Queue placement algorithm. | `None` |\r\n| `get_task` | Gets a task from the threads pool. | `task_id (str)` Task identifier. | `ThreadTask \\| MethodTask \\| None` Return task if task is found, otherwise return None. |\r\n| `is_task_exists` | Checks if task exists in the threads pool. | `task_id (str)` Task identifier. | `bool` True if task exists, otherwise False. |\r\n| `reset_metrics` | Resets Metrics. | `None` | `None` |\r\n| `start` | Starts the threads pool. | `None` | `None` |\r\n| `shutdown` | Shuts down the threads pool. | `None` | `None` |\r\n| `active_tasks_amount` | Returns the amount of active tasks in the threads pool. | `@propery` | `int` Active tasks amount. |\r\n| `avoid_starvation_amount` | Returns Avoid Starvation Amount | `@propery` | `int` Avoid Starvation Amount. |\r\n| `executors_extension_pool_size` | Returns Executors Extension Pool Size. | `@propery` | `int` Executors Extension Pool Size. |\r\n| `executors_timeout_ms` | Returns Executors Timeout in milliseconds. | `@propery` | `int` Executors Timeout in milliseconds. |\r\n| `executors_timeout_ms` | Sets Executors Timeout in milliseconds. | `@setter` Executors Timeout in milliseconds. | `None` |\r\n| `finished_tasks` | Returns finished tasks.<br>Finished tasks list will be truncated after each call. | `@propery` | `List[ThreadTask \\| MethodTask]` Finished tasks. |\r\n| `is_executors_shutdown` | Returns True if executors pool is shutdown. | `@propery` | `bool` |\r\n| `max_executors_extension_pool_size` | Returns Max Executors Extension Pool Size. | `@propery` | `int` Max Executors Extension Pool Size. |\r\n| `max_executors_extension_pool_size` | Sets Max Executors Extension Pool Size. | `@setter` Max Executors Extension Pool Size. | `None` |\r\n| `max_executors_pool_size` | Returns Max Executors Pool Size. | `@propery` | `int` Max Executors Pool Size. |\r\n| `max_executors_pool_size` | Sets Max Executors Pool Size. | `@setter` Max Executors Pool Size. | `None` |\r\n| `max_queue_size` | Returns Queue Size limitation. | `@propery` | `int` Max Queue Size. |\r\n| `max_queue_size` | Sets Max Queue Size. | `@setter` Max Queue Size. | `None` |\r\n| `metrics` | Returns Metrics. | `@propery` | `ThreadsPoolManagerMetrics` Metrics. |\r\n| `name` | Returns Threads Pool Name. | `@propery` | `str` Threads Pool Name. |\r\n| `name` | Sets Threads Pool Name. | `@setter` Threads Pool Name. | `None` |\r\n| `queue` | Returns Queue. | `@propery` | `Queue` Queue. |\r\n| `queue_size` | Returns Queue Size. | `@propery` | `int` Queue Size. |\r\n| `shutdown` | Shuts down the threads pool. | `None` | `None` |\r\n| `shutdown_executors` | Shuts down executors pool. | `None` | `None` |\r\n| `start_executors` | Starts executors pool. | `None` | `None` |\r\n\r\n## TaskExecutor class\r\n\r\n### Methods\r\n\r\n| **Method** | **Description** | **Parameters** | **Returns** |\r\n|-------------------------|---------------------------------------------------------------------------------------|----------------|------------------------------|\r\n| `start` | Starts the task executor. | `None` | `None` |\r\n| `avoid_starvation_flag` | Returns Avoid Starvation Flag. | `@propery` | `bool` |\r\n| `avoid_starvation_flag` | Sets Avoid Starvation Flag. | `@setter` | `None` |\r\n| `exception` | Returns Exception, if it happened in method that was executed in MethodTask. | `@propery` | `Exception` |\r\n| `stack_trace` | Returns stack trace, if exception happened in method that was executed in MethodTask. | `@propery` | `str` Exception stack trace. |\r\n| `priority` | Returns priority. | `@propery` | `int` |\r\n| `task` | Returns task. | `@propery` | `ThreadTask \\| MethodTask` |\r\n| `task_id` | Returns task id. | `@propery` | `str` |\r\n\r\n## ThreadTask class\r\n\r\nTask that runs a thread.\r\n\r\n### Methods\r\n\r\n| **Method** | **Description** | **Parameters** | **Returns** |\r\n|-----------------|-----------------------------|----------------|--------------------------------|\r\n| `execute` | Executes the task. | `None` | `None` |\r\n| `task_instance` | Returns task instance. | `@propery` | `Thread` Task Thread instance. |\r\n| `alive_date_ms` | Returns task alive date ms. | `@propery` | `int` Alive date in ms. |\r\n| `start_date_ms` | Returns task start date ms. | `@propery` | `int` Start date in ms. |\r\n| `start_date_ms` | Sets task start date ms. | `@setter` | `None` |\r\n| `task_state` | Returns task state. | `@propery` | `TaskStateEnum` Task state. |\r\n| `task_state` | Sets task state. | `@setter` | `None` |\r\n\r\n## MethodTask class\r\n\r\nTask that runs a method.\r\n\r\n### Methods\r\n\r\n| **Method** | **Description** | **Parameters** | **Returns** |\r\n|-----------------|---------------------------------------------|----------------|------------------------------|\r\n| `execute` - | Executes the task. | `None` | `None` |\r\n| `alive_date_ms` | Returns task alive date ms. | `@propery` | `int` Alive date in ms. |\r\n| `exception` | Returns Exception, if it happened. | `@propery` | `Exception` |\r\n| `stack_trace` | Returns stack trace, if exception happened. | `@propery` | `str` Exception stack trace. |\r\n| `start_date_ms` | Returns task start date ms. | `@propery` | `int` Start date in ms. |\r\n| `start_date_ms` | Sets task start date ms. | `@setter` | `None` |\r\n| `task_state` | Returns task state. | `@propery` | `TaskStateEnum` Task state. |\r\n| `task_state` | Sets task state. | `@setter` | `None` |\r\n\r\n\r\n## ThreadsPoolManagerMetrics class\r\n\r\ndataclass that holds threads pool manager metrics.\r\n\r\n### Properties\r\n\r\n| **Property** | **Description** | **Type** |\r\n|-------------------------------|------------------------------------------------------------------------------------------------------|------------------|\r\n| `avoid_starvation_counter` | Count the times that `AVOID_STARVATION_PRORITY` flag has been raised. | `int` |\r\n| `executed_method_counter` | Count the times that `MethodTask` has been executed. | `int` |\r\n| `executed_task_counter` | Count the times that `ThreadTask\\|MethodTask` has been executed. | `int` |\r\n| `executed_thread_counter` | Count the times that `ThreadTask` has been executed. | `int` |\r\n| `max_execution_date_ms` | Max execution date in ms. | `int` |\r\n| `max_queue_size` | Max size that the queue has reached. | `int` |\r\n| `method_tasks_counter_dict` | Count the times that `MethodTask` has been executed by task id.<br>key is task priority. | `dict[int, int]` |\r\n| `tasks_priority_counter_dict` | Count the times that `ThreadTask\\|MethodTask` has been executed by task id.<br>key is task priority. | `dict[int, int]` |\r\n| `thread_tasks_counter_dict` | Count the times that `ThreadTask` has been executed by task id.<br>key is task priority. | `dict[int, int]` |\r\n\r\n\r\n## TaskStateEnum Enum\r\n\r\nTask state enum.\r\n\r\n| **Enum** | **Description** |\r\n|------------------|----------------------------|\r\n| `QUEUE` | Task is in queue. |\r\n| `EXECUTORS_POOL` | Task is in executors pool. |\r\n| `EXECUTED` | Task is finished. |\r\n\r\n## QueuePlacementEnum Enum\r\n\r\nQueue placement enum.\r\n\r\n| **Enum** | **Description** |\r\n|-----------------------------|----------------------------|\r\n| `STRICT_PRIORITY` | Strict Priority. |\r\n| `AVOID_STARVATION_PRIORITY` | Avoid Starvation Priority. |\r\n\r\n## Examples:\r\n\r\n- ### Add tasks with `STRICT_PRIORITY` queue placement.\r\n\r\n Create a threads pool manager with 2 executors pool size.\r\n \r\n ```python\r\n from nrt_threads_utils.threads_pool_manager.threads_pool_manager import ThreadsPoolManager\r\n \r\n threads_pool_manager = ThreadsPoolManager(executors_pool_size=2)\r\n threads_pool_manager.start()\r\n ```\r\n \r\n \r\n Add two tasks to the threads pool manager.<br>\r\n The tasks will be executed as the two executors are empty.\r\n \r\n ```python\r\n from nrt_threads_utils.threads_pool_manager.tasks import ThreadTask\r\n \r\n t_1 = CustomThread()\r\n t_2 = CustomThread()\r\n \r\n threads_pool_manager.add_task(ThreadTask(t_1), priority=1)\r\n threads_pool_manager.add_task(ThreadTask(t_1), priority=1)\r\n ```\r\n \r\n \r\n \r\n Add 2 tasks in priority 1.<br>\r\n Default queue placement is `STRICT_PRIORITY`.\r\n \r\n ```python\r\n t_3 = CustomThread()\r\n t_4 = CustomThread()\r\n \r\n threads_pool_manager.add_task(ThreadTask(t_3), priority=1)\r\n threads_pool_manager.add_task(ThreadTask(t_4), priority=1)\r\n ```\r\n \r\n \r\n \r\n Add task in priority 2 (Higher priority)\r\n Default queue placement is `STRICT_PRIORITY`.\r\n \r\n ```python\r\n t_5 = CustomThread()\r\n \r\n threads_pool_manager.add_task(ThreadTask(t_3), priority=2)\r\n ```\r\n \r\n Task in priority 2 is executed before tasks in priority 1.\r\n \r\n \r\n\r\n- ### Add tasks with `AVOID_STARVATION_PRIORITY` queue placement.\r\n\r\n Create a threads pool manager with 2 executors pool size.\r\n `avoid_starvation_amount` is set to 1.\r\n \r\n ```python\r\n from nrt_threads_utils.threads_pool_manager.threads_pool_manager import ThreadsPoolManager\r\n \r\n threads_pool_manager = ThreadsPoolManager(executors_pool_size=2)\r\n threads_pool_manager.avoid_starvation_amount = 1\r\n threads_pool_manager.start()\r\n ```\r\n \r\n \r\n Add 2 tasks in priority 1.<br>\r\n \r\n ```python\r\n from nrt_threads_utils.threads_pool_manager.tasks import ThreadTask\r\n from nrt_threads_utils.threads_pool_manager.enums import QueuePlacementEnum\r\n \r\n t_1 = CustomThread()\r\n t_2 = CustomThread()\r\n \r\n threads_pool_manager.add_task(\r\n ThreadTask(t_1),\r\n priority=1,\r\n queue_placement=QueuePlacementEnum.AVOID_STARVATION_PRIORITY)\r\n threads_pool_manager.add_task(\r\n ThreadTask(t_2),\r\n priority=1,\r\n queue_placement=QueuePlacementEnum.AVOID_STARVATION_PRIORITY)\r\n ```\r\n \r\n \r\n Add task in priority 2 (Higher priority)\r\n \r\n ```python\r\n t_5 = CustomThread()\r\n \r\n threads_pool_manager.add_task(\r\n ThreadTask(t_3), \r\n priority=2,\r\n queue_placement=QueuePlacementEnum.AVOID_STARVATION_PRIORITY)\r\n ```\r\n \r\n Task in priority 2 is executed before tasks in priority 1.<br>\r\n Avoid starvation counter is increased by 1 because the task is not appended to the end of the queue.\r\n \r\n \r\n \r\n Add another task in priority 2 (Higher priority)\r\n \r\n ```python\r\n t_5 = CustomThread()\r\n \r\n threads_pool_manager.add_task(\r\n ThreadTask(t_3), \r\n priority=2,\r\n queue_placement=QueuePlacementEnum.AVOID_STARVATION_PRIORITY)\r\n ```\r\n \r\n The task will flag with `avoid_starvation_flag` and will be added to the end of the queue.\r\n\r\n \r\n \r\n Add the next tasks will be added in **Avoid Starvation Priority** after the flagged task.\r\n \r\n \r\n\r\n\r\n- ### Metrics.\r\n\r\n Create method that sleep for 10 seconds.<br><br>\r\n \r\n **Code**\r\n \r\n ```python\r\n from nrt_threads_utils.threads_pool_manager.threads_pool_manager import ThreadsPoolManager\r\n from nrt_threads_utils.threads_pool_manager.tasks import MethodTask\r\n from time import sleep\r\n \r\n \r\n def sleep_10_sec():\r\n sleep(10)\r\n \r\n \r\n threads_pool_manager = \\\r\n ThreadsPoolManager(executors_pool_size=1)\r\n \r\n try:\r\n threads_pool_manager.start()\r\n \r\n threads_pool_manager.add_task(MethodTask(sleep_10_sec), priority=1)\r\n sleep(0.2)\r\n threads_pool_manager.add_task(MethodTask(sleep_10_sec), priority=2)\r\n threads_pool_manager.add_task(MethodTask(sleep_10_sec), priority=2)\r\n \r\n metrics = threads_pool_manager.metrics\r\n print(f'Max queue size: {metrics.max_queue_size}')\r\n print(f'Max execution date ms {metrics.max_execution_date_ms}')\r\n print(f'Executed tasks counter: {metrics.executed_tasks_counter}')\r\n print(f'Executed threads counter: {metrics.executed_threads_counter}')\r\n print(f'Executed methods counter: {metrics.executed_methods_counter}')\r\n print(f'Avoid starvation counter: {metrics.avoid_starvation_counter}')\r\n print(f'Tasks priority counter dict {metrics.tasks_priority_counter_dict}')\r\n finally:\r\n threads_pool_manager.shutdown()\r\n threads_pool_manager.join()\r\n ```\r\n \r\n **Output**\r\n \r\n ```\r\n Max queue size: 2\r\n Max execution date ms 10000\r\n Executed tasks counter: 3\r\n Executed threads counter: 0\r\n Executed methods counter: 3\r\n Avoid starvation counter: 0\r\n Tasks priority counter dict {1: 1, 2: 2}\r\n ```\r\n",
"bugtrack_url": null,
"license": null,
"summary": "Threads utilities in Python",
"version": "2.0.5",
"project_urls": {
"Bug Tracker": "https://github.com/etuzon/python-nrt-threads-utils/issues",
"Homepage": "https://github.com/etuzon/python-nrt-threads-utils",
"documentation": "https://github.com/etuzon/python-nrt-threads-utils/wiki"
},
"split_keywords": [
"python",
" python3",
" python-3",
" tool",
" tools",
" thread",
" threads",
" pool",
" threads-pool",
" util",
" nrt",
" nrt-utils",
" nrt-utilities",
" threads-utils",
" threads-utilities",
" nrt-utilities",
" nrt-threads-utils",
" nrt-threads-utilities"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "45baf2abc3f57f01f2ccb97fdffd322a8c0498ae8edaef80d1f64cb8f25b50e5",
"md5": "08a5930efec4bc81e86db092803e32f2",
"sha256": "58372c4ebb376d6e3f852c5596ca188184644995893d7fbb879e612a7cdabc78"
},
"downloads": -1,
"filename": "nrt_threads_utils-2.0.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "08a5930efec4bc81e86db092803e32f2",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 18272,
"upload_time": "2024-07-27T15:26:26",
"upload_time_iso_8601": "2024-07-27T15:26:26.452498Z",
"url": "https://files.pythonhosted.org/packages/45/ba/f2abc3f57f01f2ccb97fdffd322a8c0498ae8edaef80d1f64cb8f25b50e5/nrt_threads_utils-2.0.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "50bfeb9b6c7c6788d63c182cb80353c01fca346231d24588b9beadb0ac5fe6c8",
"md5": "703f79f93a00ce97dd7b673edfe09617",
"sha256": "91274e3631999f7a7b8e050ea98daea35d34d1e40438fe056a13b7a5090aed7e"
},
"downloads": -1,
"filename": "nrt_threads_utils-2.0.5.tar.gz",
"has_sig": false,
"md5_digest": "703f79f93a00ce97dd7b673edfe09617",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 20872,
"upload_time": "2024-07-27T15:26:30",
"upload_time_iso_8601": "2024-07-27T15:26:30.755055Z",
"url": "https://files.pythonhosted.org/packages/50/bf/eb9b6c7c6788d63c182cb80353c01fca346231d24588b9beadb0ac5fe6c8/nrt_threads_utils-2.0.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-27 15:26:30",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "etuzon",
"github_project": "python-nrt-threads-utils",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"requirements": [],
"lcname": "nrt-threads-utils"
}
JSON
