# tools-mp
This Python library provides a convenient one-line multiprocessing option that can be used as follows:
```Python
from tools_mp import multiprocess
from tools_mp import process
def any_function( arg_1, arg_2, ..., arg_n ):
do something
return result
args = [
{
'arg_1': arg_1,
'arg_2': arg_2,
...
'arg_n': arg_n,
} for arg_1, arg_2, ..., arg_n in arguments
]
data = multiprocess(
function = any_function,
args = args,
return_data = True,
verbose = True,
workers = 8,
**kwargs,
)
data = process(
function = any_function,
args = args,
return_data = True,
verbose = True,
workers = 8,
mp_threshold = 4, # <- if the number of tasks is less than this value,
# the process function will use single core processing, otherwise it
# will use multiprocessing. This can speed up the processing of small
# tasks significantly, as the overhead of multiprocessing is avoided.
**kwargs,
)
```
If verbose=True, the progress of the calculation will be visualized via the [tqdm](https://pypi.org/project/tqdm/) library.
If workers=None (default) all available CPU cores will be used for processing.
**kwargs will be passed to 'multiprocessing.Pool' as keyword arguments. This is important if you would like to initialize the workers with a specific function or variables for example.
# Installation
pip install tools-mp
Raw data
{
"_id": null,
"home_page": "https://github.com/paul-krug/tools-mp",
"name": "tools-mp",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "multiprocessing, Python",
"author": "Paul Krug",
"author_email": "paul_konstantin.krug@tu-dresden.de",
"download_url": "https://files.pythonhosted.org/packages/70/a0/e132c1c11587fbbb8d5bc93c77b88eb80da739e7a290debebce2cc5f05a9/tools_mp-0.2.0.tar.gz",
"platform": null,
"description": "# tools-mp\n\nThis Python library provides a convenient one-line multiprocessing option that can be used as follows:\n\n```Python\nfrom tools_mp import multiprocess\nfrom tools_mp import process\n\ndef any_function( arg_1, arg_2, ..., arg_n ):\n do something\n return result\n\nargs = [ \n {\n 'arg_1': arg_1,\n 'arg_2': arg_2,\n ...\n 'arg_n': arg_n,\n } for arg_1, arg_2, ..., arg_n in arguments\n ]\n\ndata = multiprocess(\n function = any_function,\n args = args,\n return_data = True,\n verbose = True,\n workers = 8,\n **kwargs,\n )\n\ndata = process(\n function = any_function,\n args = args,\n return_data = True,\n verbose = True,\n workers = 8,\n mp_threshold = 4, # <- if the number of tasks is less than this value,\n\t# the process function will use single core processing, otherwise it\n\t# will use multiprocessing. This can speed up the processing of small\n\t# tasks significantly, as the overhead of multiprocessing is avoided.\n **kwargs,\n )\n\n\n```\n\nIf verbose=True, the progress of the calculation will be visualized via the [tqdm](https://pypi.org/project/tqdm/) library.\nIf workers=None (default) all available CPU cores will be used for processing.\n**kwargs will be passed to 'multiprocessing.Pool' as keyword arguments. This is important if you would like to initialize the workers with a specific function or variables for example.\n\n\n\n# Installation\n\n pip install tools-mp\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Convenient multiprocessing function for Python",
"version": "0.2.0",
"project_urls": {
"Homepage": "https://github.com/paul-krug/tools-mp"
},
"split_keywords": [
"multiprocessing",
" python"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5537b5da2fd9d2d05e20e29fde14843d6970aea2255af78fd9730ced537309f3",
"md5": "660ee27b239b1a74e9c1f724dccfdd52",
"sha256": "c841456ce668358424cd65c0d8051720ff82b435a4d5da908bd91144b07c6d04"
},
"downloads": -1,
"filename": "tools_mp-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "660ee27b239b1a74e9c1f724dccfdd52",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 3861,
"upload_time": "2024-10-26T12:11:34",
"upload_time_iso_8601": "2024-10-26T12:11:34.380867Z",
"url": "https://files.pythonhosted.org/packages/55/37/b5da2fd9d2d05e20e29fde14843d6970aea2255af78fd9730ced537309f3/tools_mp-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "70a0e132c1c11587fbbb8d5bc93c77b88eb80da739e7a290debebce2cc5f05a9",
"md5": "e53c5a60ce4ad92770dc816232153414",
"sha256": "9df3602fa8da61d3ac9cfe29a523c275bad90381b7348a8e3668abcbf881b6ad"
},
"downloads": -1,
"filename": "tools_mp-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "e53c5a60ce4ad92770dc816232153414",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 3721,
"upload_time": "2024-10-26T12:11:35",
"upload_time_iso_8601": "2024-10-26T12:11:35.859019Z",
"url": "https://files.pythonhosted.org/packages/70/a0/e132c1c11587fbbb8d5bc93c77b88eb80da739e7a290debebce2cc5f05a9/tools_mp-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-26 12:11:35",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "paul-krug",
"github_project": "tools-mp",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "tools-mp"
}
JSON
