# python-worker
[](https://pepy.tech/project/python-worker)
---
## Description
A package to simplify the thread declaration directly either by using decorator or pass it through function. It also allows you to stop the running thread (worker) from any layer
---
## Installation
```
pip install python-worker
```
---
## Changelogs
- v1.8:
- Refactoring codes
- flexible `worker` declaration
- v1.9:
- Added Asynchronous Worker for coroutine function using `@async_worker` decorator
- v1.10:
- Added `overload` typehints for `worker` and `async_worker`
- Added `restart` feature for worker
- v2.0:
- Added `process` worker to enable run your function in different GIL (Global Interpreter Lock) which could give you a performance boost
- v2.2:
- Added `async_process` worker to enable run your async function / coroutine in different GIL (Global Interpreter Lock) which could give you a performance boost
---
## Basic Guide
`@worker` will define a function as a thread object once it run
```
import time
from worker import worker
@worker
def go(n, sleepDur):
for i in range(n):
time.sleep(sleepDur)
print('done')
go(100, 0.1)
```
The function `go` will be running as a thread
---
## Asynchronous Guide
Well, if you have a coroutine function you can use `async_worker` instead
```
import asyncio
from worker import async_worker
@async_worker
async def go():
print("this is inside coroutine!")
for i in range(10):
time.sleep(0.5)
print(i)
print("done!")
return "result!"
go_worker = asyncio.run(go())
```
or run it as a process
```
import asyncio
from worker import async_process
@async_process
async def go():
print("this is inside coroutine!")
for i in range(10):
time.sleep(0.5)
print(i)
print("done!")
return "result!"
go_worker = asyncio.run(go())
```
---
## Process Guide
A new feature called `process` is simply putting your worker on different GIL (Global Interpreter Lock) which could give you a performance boost.
It's implementing `multiprocessing` instead of `multithreading` which at this stage is achieving the true form of `parallelism` which is run in different environment with your function call environment
```
import time
import os
from worker import process
@process
def go(n, sleepDur):
for i in range(n):
time.sleep(sleepDur)
print('done')
go(100, 0.1)
```
your function `go` will run in different process.
To check it, you can just print out the `os.getpid()`
```
import os
from worker import worker, process
@worker(multiproc=True)
def run_in_new_process_from_worker(parent_pid):
print(f"from {parent_pid} running in a new process {os.getpid()} - from worker.mutliproc==True")
return "return from process"
@process
def run_in_new_process(parent_pid):
print(f"from {parent_pid} running in a new process {os.getpid()} - from process")
return "return from process"
@worker
def run_in_new_thread(parent_pid):
print(f"from {parent_pid} running in a new thread {os.getpid()} - from worker.multiproc==False")
return "return from thread"
print(f"this is on main thread {os.getpid()}")
run_in_new_process_from_worker(os.getpid())
run_in_new_process(os.getpid())
run_in_new_thread(os.getpid())
```
then run the script
```
this is on main thread 29535
from 29535 running in a new process 29537 - from worker.mutliproc==True
from 29535 running in a new thread 29535 - from worker.multiproc==False
from 29535 running in a new process 29538 - from process
```
you can see the different of process id between running in a new process and thread
---
# Additional Guides
## Kill / Stop / Abort the running worker
You can abort some workers, all workers or even all threads..
### Abort specific workers
```
import time
from worker import worker, abort_worker
@worker
def go4(n=10):
for i in range(n):
time.sleep(1)
go4_worker = go4(10)
time.sleep(3)
abort_worker(go4_worker)
```
or just abort it from the instance
```
go4_worker.abort()
```
### Abort all workers (this only abort worker threads only)
```
from worker import abort_all_worker
abort_all_worker()
```
### Abort all threads (it will abort both all worker and non-worker threads)
```
from worker import abort_all_thread
abort_all_thread()
```
---
## Run undefined `@worker` function
```
import time
from worker import run_as_Worker
def go(n):
...
go_worker = run_as_Worker(target=go, args=(10,))
```
---
## Get Return Value
How to get the return of threaded function ?
```
@worker
def go(n):
time.sleep(n)
return "done"
go_worker = go(10)
# this will await the worker to finished and return the value
go_result = go_worker.await
# You can also use this if it's finished, dont have to await
go_result = go_worker.ret
```
## Check/Monitor All Workers
```
from worker import ThreadWorkerManager
## all created workers
ThreadWorkerManager.list()
## All active/running workers only
ThreadWorkerManager.list(active_only=True)
```
it will return the information
```
>>> ThreadWorkerManager.list()
==============================================================
ID |Name |Active|Address | WorkTime (s)
==============================================================
0 |worker |True |0x7fdf1a977af0 | 4.97
1 |worker1 |True |0x7fdf1a73d640 | 4.07
2 |worker2 |True |0x7fdf1a73d9d0 | 3.83
3 |worker3 |True |0x7fdf1a73dd00 | 3.62
4 |worker4 |True |0x7fdf1a74b070 | 3.38
==============================================================
```
---
## Python Interactive Shell - Keyboard Interrupt (CTRL+C)
When you run your scripts on interactive mode
```
python -i myScript.py
```
you could add an abort handler with keyboard interrupt to abort your thread.
#### Inside myScript.py
`ThreadWorkerManager.enableKeyboardInterrupt()` allows you to abort your running workers.
```
from worker import worker, ThreadWorkerManager
# enabling abort handler for worker into keyboard interrupt (CTRL+C)
ThreadWorkerManager.enableKeyboardInterrupt()
```
You could also activate exit thread which triggered by pressing the CTRL+Z. This also added an abort handler for worker into keyboard interrupt (CTRL+C).
```
ThreadWorkerManager.disableKeyboardInterrupt(enable_exit_thread=True)
```
Disabling abort handler for worker into keyboard interrupt (CTRL+C).
```
ThreadWorkerManager.disableKeyboardInterrupt()
```
Check handler status.
```
ThreadWorkerManager.keyboard_interrupt_handler_status
```
You also can choose which workers are allowed to be aborted on keyboard interrupt
#### Inside myScript.py
```
from worker import worker, ThreadWorkerManager
@worker("Uninterrupted", on_abort=lambda: print("ITS GREAT"), keyboard_interrupt=False)
def go_not_interrupted():
i = 0
while i < 1e3/2:
i += 10
print(i,"go_not_interrupted")
time.sleep(0.001)
return i
@worker("Interrupted", on_abort=lambda: print("ITS GREAT"), keyboard_interrupt=True)
def go_interrupted():
i = 0
while i < 1e3/2:
i += 10
print(i,"go_interrupted")
time.sleep(0.001)
return i
ThreadWorkerManagerManager.enableKeyboardInterrupt()
go_not_interrupted()
go_interrupted()
```
run in your terminal
```
python -i myScript.py
```
press CTRL+C while the process is running and see the results.
Raw data
{
"_id": null,
"home_page": "https://github.com/Danangjoyoo/python-worker",
"name": "python-worker",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "python, threading, worker, async worker, async thread, abort thread, thread stopper, thread manager, simple thread, thread monitor",
"author": "danangjoyoo (Agus Danangjoyo)",
"author_email": "<agus.danangjoyo.blog@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/98/50/484172a58ff43ad2e5119bf3b70e6e7594d88b0c002a1e2ecb59d55ce0a3/python-worker-2.2.4.tar.gz",
"platform": null,
"description": "# python-worker\n[](https://pepy.tech/project/python-worker)\n\n---\n## Description\nA package to simplify the thread declaration directly either by using decorator or pass it through function. It also allows you to stop the running thread (worker) from any layer\n\n---\n\n## Installation\n```\npip install python-worker\n```\n\n---\n\n## Changelogs\n- v1.8:\n - Refactoring codes\n - flexible `worker` declaration\n- v1.9:\n - Added Asynchronous Worker for coroutine function using `@async_worker` decorator\n- v1.10:\n - Added `overload` typehints for `worker` and `async_worker`\n - Added `restart` feature for worker\n- v2.0:\n - Added `process` worker to enable run your function in different GIL (Global Interpreter Lock) which could give you a performance boost\n- v2.2:\n - Added `async_process` worker to enable run your async function / coroutine in different GIL (Global Interpreter Lock) which could give you a performance boost\n\n\n---\n## Basic Guide\n`@worker` will define a function as a thread object once it run\n\n```\nimport time\nfrom worker import worker\n\n@worker\ndef go(n, sleepDur):\n for i in range(n):\n time.sleep(sleepDur)\n print('done')\n\ngo(100, 0.1)\n```\nThe function `go` will be running as a thread\n\n---\n\n## Asynchronous Guide\nWell, if you have a coroutine function you can use `async_worker` instead\n```\nimport asyncio\nfrom worker import async_worker\n\n@async_worker\nasync def go():\n print(\"this is inside coroutine!\")\n for i in range(10):\n time.sleep(0.5)\n print(i)\n print(\"done!\")\n return \"result!\"\n\ngo_worker = asyncio.run(go())\n```\n\nor run it as a process\n```\nimport asyncio\nfrom worker import async_process\n\n@async_process\nasync def go():\n print(\"this is inside coroutine!\")\n for i in range(10):\n time.sleep(0.5)\n print(i)\n print(\"done!\")\n return \"result!\"\n\ngo_worker = asyncio.run(go())\n```\n\n---\n\n## Process Guide\nA new feature called `process` is simply putting your worker on different GIL (Global Interpreter Lock) which could give you a performance boost.\nIt's implementing `multiprocessing` instead of `multithreading` which at this stage is achieving the true form of `parallelism` which is run in different environment with your function call environment\n\n```\nimport time\nimport os\nfrom worker import process\n\n@process\ndef go(n, sleepDur):\n for i in range(n):\n time.sleep(sleepDur)\n print('done')\n\ngo(100, 0.1)\n```\nyour function `go` will run in different process.\n\nTo check it, you can just print out the `os.getpid()`\n```\nimport os\nfrom worker import worker, process\n\n@worker(multiproc=True)\ndef run_in_new_process_from_worker(parent_pid):\n print(f\"from {parent_pid} running in a new process {os.getpid()} - from worker.mutliproc==True\")\n return \"return from process\"\n\n@process\ndef run_in_new_process(parent_pid):\n print(f\"from {parent_pid} running in a new process {os.getpid()} - from process\")\n return \"return from process\"\n\n@worker\ndef run_in_new_thread(parent_pid):\n print(f\"from {parent_pid} running in a new thread {os.getpid()} - from worker.multiproc==False\")\n return \"return from thread\"\n\n\nprint(f\"this is on main thread {os.getpid()}\")\n\nrun_in_new_process_from_worker(os.getpid())\nrun_in_new_process(os.getpid())\nrun_in_new_thread(os.getpid())\n\n```\n\nthen run the script\n```\nthis is on main thread 29535\nfrom 29535 running in a new process 29537 - from worker.mutliproc==True\nfrom 29535 running in a new thread 29535 - from worker.multiproc==False\nfrom 29535 running in a new process 29538 - from process\n```\n\nyou can see the different of process id between running in a new process and thread\n\n---\n\n# Additional Guides\n\n## Kill / Stop / Abort the running worker\nYou can abort some workers, all workers or even all threads..\n\n### Abort specific workers\n```\nimport time\nfrom worker import worker, abort_worker\n\n@worker\ndef go4(n=10):\n for i in range(n):\n time.sleep(1)\n\ngo4_worker = go4(10)\ntime.sleep(3)\nabort_worker(go4_worker)\n```\nor just abort it from the instance\n```\ngo4_worker.abort()\n```\n\n### Abort all workers (this only abort worker threads only)\n```\nfrom worker import abort_all_worker\n\nabort_all_worker()\n```\n\n### Abort all threads (it will abort both all worker and non-worker threads)\n```\nfrom worker import abort_all_thread\n\nabort_all_thread()\n```\n---\n## Run undefined `@worker` function\n```\nimport time\nfrom worker import run_as_Worker\n\ndef go(n):\n ...\n\ngo_worker = run_as_Worker(target=go, args=(10,))\n```\n\n---\n## Get Return Value\nHow to get the return of threaded function ?\n```\n@worker\ndef go(n):\n time.sleep(n)\n return \"done\"\n\ngo_worker = go(10)\n\n# this will await the worker to finished and return the value\n\ngo_result = go_worker.await\n\n# You can also use this if it's finished, dont have to await\n\ngo_result = go_worker.ret\n```\n\n## Check/Monitor All Workers\n```\nfrom worker import ThreadWorkerManager\n\n## all created workers\nThreadWorkerManager.list()\n\n## All active/running workers only\nThreadWorkerManager.list(active_only=True)\n```\nit will return the information\n```\n>>> ThreadWorkerManager.list()\n==============================================================\nID |Name |Active|Address | WorkTime (s)\n==============================================================\n0 |worker |True |0x7fdf1a977af0 | 4.97\n1 |worker1 |True |0x7fdf1a73d640 | 4.07\n2 |worker2 |True |0x7fdf1a73d9d0 | 3.83\n3 |worker3 |True |0x7fdf1a73dd00 | 3.62\n4 |worker4 |True |0x7fdf1a74b070 | 3.38\n==============================================================\n```\n\n---\n\n## Python Interactive Shell - Keyboard Interrupt (CTRL+C)\n When you run your scripts on interactive mode\n ```\n python -i myScript.py\n ```\n you could add an abort handler with keyboard interrupt to abort your thread.\n\n #### Inside myScript.py\n\n `ThreadWorkerManager.enableKeyboardInterrupt()` allows you to abort your running workers.\n ```\n from worker import worker, ThreadWorkerManager\n\n\n # enabling abort handler for worker into keyboard interrupt (CTRL+C)\n\n ThreadWorkerManager.enableKeyboardInterrupt()\n ```\n You could also activate exit thread which triggered by pressing the CTRL+Z. This also added an abort handler for worker into keyboard interrupt (CTRL+C).\n ```\n ThreadWorkerManager.disableKeyboardInterrupt(enable_exit_thread=True)\n ```\n Disabling abort handler for worker into keyboard interrupt (CTRL+C).\n ```\n ThreadWorkerManager.disableKeyboardInterrupt()\n ```\n Check handler status.\n ```\n ThreadWorkerManager.keyboard_interrupt_handler_status\n ```\n\n You also can choose which workers are allowed to be aborted on keyboard interrupt\n\n #### Inside myScript.py\n```\nfrom worker import worker, ThreadWorkerManager\n\n@worker(\"Uninterrupted\", on_abort=lambda: print(\"ITS GREAT\"), keyboard_interrupt=False)\ndef go_not_interrupted():\n i = 0\n while i < 1e3/2:\n i += 10\n print(i,\"go_not_interrupted\")\n time.sleep(0.001)\n return i\n\n@worker(\"Interrupted\", on_abort=lambda: print(\"ITS GREAT\"), keyboard_interrupt=True)\ndef go_interrupted():\n i = 0\n while i < 1e3/2:\n i += 10\n print(i,\"go_interrupted\")\n time.sleep(0.001)\n return i\n\nThreadWorkerManagerManager.enableKeyboardInterrupt()\ngo_not_interrupted()\ngo_interrupted()\n```\n run in your terminal\n ```\n python -i myScript.py\n ```\n press CTRL+C while the process is running and see the results.\n",
"bugtrack_url": null,
"license": null,
"summary": "Simplify and master control (run and stop) the python threads (workers)",
"version": "2.2.4",
"project_urls": {
"Homepage": "https://github.com/Danangjoyoo/python-worker"
},
"split_keywords": [
"python",
" threading",
" worker",
" async worker",
" async thread",
" abort thread",
" thread stopper",
" thread manager",
" simple thread",
" thread monitor"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "13875e7d547e40b5f532bc6c0c149673ac5e02d1d240bd6af460723d47fafa36",
"md5": "5587e03fed02a12e150fcaa17ea58a3c",
"sha256": "6f65735532e5281ebb1c22307b004ff6457d2aca76b9fda2e30005f119a78dcb"
},
"downloads": -1,
"filename": "python_worker-2.2.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "5587e03fed02a12e150fcaa17ea58a3c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 9367,
"upload_time": "2024-08-30T17:49:01",
"upload_time_iso_8601": "2024-08-30T17:49:01.020968Z",
"url": "https://files.pythonhosted.org/packages/13/87/5e7d547e40b5f532bc6c0c149673ac5e02d1d240bd6af460723d47fafa36/python_worker-2.2.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9850484172a58ff43ad2e5119bf3b70e6e7594d88b0c002a1e2ecb59d55ce0a3",
"md5": "78bba3af895571315d25bde60791f8bd",
"sha256": "804f0fcdd5ee24e93e017ed20c3355fc8f879690353de3143b9a83811dc98b83"
},
"downloads": -1,
"filename": "python-worker-2.2.4.tar.gz",
"has_sig": false,
"md5_digest": "78bba3af895571315d25bde60791f8bd",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 9409,
"upload_time": "2024-08-30T17:49:02",
"upload_time_iso_8601": "2024-08-30T17:49:02.611904Z",
"url": "https://files.pythonhosted.org/packages/98/50/484172a58ff43ad2e5119bf3b70e6e7594d88b0c002a1e2ecb59d55ce0a3/python-worker-2.2.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-30 17:49:02",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Danangjoyoo",
"github_project": "python-worker",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "python-worker"
}
JSON
