Multiprocessing on Dill
=======================
This project is a friendly fork – for Python 3 – of the Python Standard Library `multiprocessing
<https://docs.python.org/3/library/multiprocessing.html>`_ module, which uses the third-party
`dill <https://pypi.python.org/pypi/dill>`_ serializer instead of the standard ``pickle`` serializer. This overcomes
many shortcomings of ``pickle`` which prevent multiprocessing being used with lambdas, closures and other useful Python
objects.
The easiest way to use ``multiprocessing_on_dill`` in place of ``multiprocessing`` is simply to replace any import
statements like this::
import multiprocessing
with::
import multiprocessing_on_dill as multiprocessing
and import statements like this::
from multiprocessing import Pool
with::
from multiprocessing_on_dill import Pool
With such import changes in place, it will now be possible to use functions like ``Pool.map()`` with lambdas::
pool = Pool(12)
result = pool.map(lambda x: x*x, range(10000))
Everything else should be identical to the Python version.
You can determine from which version of the Python Standard Library ``multiprocessing_on_dill`` has been forked, by
examining the ``multiprocessing_on_dill.__version__`` attribute.
Future
======
It is our hope that one day the Python Standard Library ``pickle`` module will gain the additional capabilities of
``dill`` and there will no longer be a need for ``multiprocessing_on_dill`` to exist.
Raw data
{
"_id": null,
"home_page": "https://github.com/sixty-north/multiprocessing_on_dill",
"name": "multiprocessing_on_dill",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "multiprocessing parallel",
"author": "Robert Smallshire",
"author_email": "rob@sixty-north.com",
"download_url": "https://files.pythonhosted.org/packages/86/4d/4b135e2e5cd0194eb29f2ed36e9a77a07596787a9a8ac2279bd4445398f2/multiprocessing_on_dill-3.5.0a4.tar.gz",
"platform": "UNKNOWN",
"description": "Multiprocessing on Dill\n=======================\n\nThis project is a friendly fork \u2013 for Python 3 \u2013 of the Python Standard Library `multiprocessing\n<https://docs.python.org/3/library/multiprocessing.html>`_ module, which uses the third-party\n`dill <https://pypi.python.org/pypi/dill>`_ serializer instead of the standard ``pickle`` serializer. This overcomes\nmany shortcomings of ``pickle`` which prevent multiprocessing being used with lambdas, closures and other useful Python\nobjects.\n\nThe easiest way to use ``multiprocessing_on_dill`` in place of ``multiprocessing`` is simply to replace any import\nstatements like this::\n\n import multiprocessing\n\nwith::\n\n import multiprocessing_on_dill as multiprocessing\n\nand import statements like this::\n\n from multiprocessing import Pool\n\nwith::\n\n from multiprocessing_on_dill import Pool\n\nWith such import changes in place, it will now be possible to use functions like ``Pool.map()`` with lambdas::\n\n pool = Pool(12)\n result = pool.map(lambda x: x*x, range(10000))\n\nEverything else should be identical to the Python version.\n\nYou can determine from which version of the Python Standard Library ``multiprocessing_on_dill`` has been forked, by\nexamining the ``multiprocessing_on_dill.__version__`` attribute.\n\n\nFuture\n======\n\nIt is our hope that one day the Python Standard Library ``pickle`` module will gain the additional capabilities of\n``dill`` and there will no longer be a need for ``multiprocessing_on_dill`` to exist.",
"bugtrack_url": null,
"license": "PSFL",
"summary": "A friendly fork of multiprocessing which uses dill instead of pickle",
"version": "3.5.0a4",
"project_urls": {
"Download": "UNKNOWN",
"Homepage": "https://github.com/sixty-north/multiprocessing_on_dill"
},
"split_keywords": [
"multiprocessing",
"parallel"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "864d4b135e2e5cd0194eb29f2ed36e9a77a07596787a9a8ac2279bd4445398f2",
"md5": "cd9da5a57987eddb0040939296fe3c3d",
"sha256": "d6d50c300ff4bd408bb71eb78725e60231039ee9b3d0d9bb7697b9d0e15045e7"
},
"downloads": -1,
"filename": "multiprocessing_on_dill-3.5.0a4.tar.gz",
"has_sig": false,
"md5_digest": "cd9da5a57987eddb0040939296fe3c3d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 53330,
"upload_time": "2015-05-17T12:05:20",
"upload_time_iso_8601": "2015-05-17T12:05:20.894454Z",
"url": "https://files.pythonhosted.org/packages/86/4d/4b135e2e5cd0194eb29f2ed36e9a77a07596787a9a8ac2279bd4445398f2/multiprocessing_on_dill-3.5.0a4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2015-05-17 12:05:20",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "sixty-north",
"github_project": "multiprocessing_on_dill",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "multiprocessing_on_dill"
}
JSON
