.. image:: https://raw.githubusercontent.com/BrianPugh/lox/main/assets/lox_200w.png
.. image:: https://img.shields.io/pypi/v/lox.svg
:target: https://pypi.python.org/pypi/lox
.. image:: https://circleci.com/gh/BrianPugh/lox.svg?style=svg
:target: https://circleci.com/gh/BrianPugh/lox
.. image:: https://readthedocs.org/projects/lox/badge/?version=latest
:target: https://lox.readthedocs.io/en/latest/?badge=latest
:alt: Documentation Status
Threading and multiprocessing made easy.
* Free software: Apache-2.0 license
* Documentation: https://lox.readthedocs.io.
* Python >=3.6
**Lox** provides decorators and synchronization primitives to quickly add
concurrency to your projects.
Installation
------------
pip3 install --user lox
Features
--------
* **Multithreading**: Powerful, intuitive multithreading in just 2 additional lines of code.
* **Multiprocessing**: Truly parallel function execution with the same interface as **multithreading**.
* **Synchronization**: Advanced thread synchronization, communication, and resource management tools.
Todos
-----
* All objects except ``lox.process`` are for threads. These will eventually be multiprocess friendly.
Usage
-----
Easy Multithreading
^^^^^^^^^^^^^^^^^^^
>>> import lox
>>>
>>> @lox.thread(4) # Will operate with a maximum of 4 threads
... def foo(x,y):
... return x*y
>>> foo(3,4) # normal function calls still work
12
>>> for i in range(5):
... foo.scatter(i, i+1)
-ignore-
>>> # foo is currently being executed in 4 threads
>>> results = foo.gather() # block until results are ready
>>> print(results) # Results are in the same order as scatter() calls
[0, 2, 6, 12, 20]
Or, for example, if you aren't allowed to directly decorate the function you
would like multithreaded/multiprocessed, you can just directly invoke the
decorator:
.. code-block:: pycon
>>> # Lets say we don't have direct access to this function
... def foo(x, y):
... return x * y
...
>>>
>>> def my_func():
... foo_threaded = lox.thread(foo)
... for i in range(5):
... foo_threaded.scatter(i, i + 1)
... results = foo_threaded.gather()
... # foo is currently being executed in default 50 thread executor pool
... return results
...
This also makes it easier to dynamically control the number of
thread/processes in the executor pool. The syntax is a little weird, but
this is just explicitly invoking a decorator that has optional arguments:
.. code-block:: pycon
>>> # Set the number of executer threads to 10
>>> foo_threaded = lox.thread(10)(foo)
Easy Multiprocessing
^^^^^^^^^^^^^^^^^^^^
.. code-block:: pycon
>>> import lox
>>>
>>> @lox.process(4) # Will operate with a pool of 4 processes
... def foo(x, y):
... return x * y
...
>>> foo(3, 4) # normal function calls still work
12
>>> for i in range(5):
... foo.scatter(i, i + 1)
...
-ignore-
>>> # foo is currently being executed in 4 processes
>>> results = foo.gather() # block until results are ready
>>> print(results) # Results are in the same order as scatter() calls
[0, 2, 6, 12, 20]
Progress Bar Support (tqdm)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code-block:: pycon
>>> import lox
>>> from random import random
>>> from time import sleep
>>>
>>> @lox.thread(2)
... def foo(multiplier):
... sleep(multiplier * random())
...
>>> for i in range(10):
>>> foo.scatter(i)
>>> results = foo.gather(tqdm=True)
90%|████████████████████████████████▌ | 9/10 [00:03<00:00, 1.32it/s]
100%|███████████████████████████████████████| 10/10 [00:06<00:00, 1.46s/it]
=======
History
=======
0.11.0 (2022-04-07)
-------------------
* Set number of workers to 0 (in thread execution) if the environment variable ``LOX_DEBUG`` is set to a true-like value (``true``, ``1``, etc.). Makes it easier to set breakpoints in multithreaded code without having to manually edit the decorator.
0.10.0 (2021-12-18)
-------------------
* Remove dependency pinning.
* Allow ``@lox.thread(0)``. This will execute ``scatter`` calls in parent thread.
Useful for debugging breakpoints in parallelized code.
0.9.0 (2020-11-25)
------------------
* ``tqdm`` support on ``lox.process.gather``. See v0.8.0 release notes for usage.
0.8.0 (2020-11-25)
------------------
* ``tqdm`` support on ``lox.thread.gather``
* Can be a bool::
>>> my_func.gather(tqdm=True)
* Can be a ``tqdm`` object::
>>> from tqdm import tqdm
>>> pbar = tqdm(total=100)
>>> for _ in range(100):
>>> my_func.scatter()
>>> my_func.gather(tqdm=pbar)
0.7.0 (2020-07-20)
------------------
* Complete rework of workers
+ Fix memory leaks
* Drop support for python3.5
* Drop support for chaining in favor of simpler codebase
0.6.3 (2019-07-30)
------------------
* Alternative fix for 0.6.2.
0.6.2 (2019-07-21)
------------------
* Update dependencies
* Fix garbage-collecting exclusiviity
0.6.1 (2019-07-21)
------------------
* Fix memory leak in ``lox.process``.
0.6.0 (2019-07-21)
------------------
* ``lox.Announcement`` ``subscribe()`` calls now return another ``Announcement``
object that behaves like a queue instead of an actual queue. Allows for
many-queue-to-many-queue communications.
* New Object: ``lox.Funnel``. allows for waiting on many queues for a complete
set of inputs indicated by a job ID.
0.5.0 (2019-07-01)
------------------
* New Object: ``lox.Announcement``. Allows a one-to-many thread queue with
backlog support so that late subscribers can still get all (or most recent)
announcements before they subscribed.
* New Feature: ``lox.thread`` ``scatter`` calls can now be chained together.
``scatter`` now returns an ``int`` subclass that contains metadata to allow
chaining. Each scatter call can have a maximum of 1 previous ``scatter`` result.
* Documentation updates, theming, and logos
0.4.3 (2019-06-24)
------------------
* Garbage collect cached decorated object methods
0.4.2 (2019-06-23)
------------------
* Fixed multiple instances and successive scatter and gather calls to wrapped methods
0.4.1 (2019-06-23)
------------------
* Fixed broken workers and unit tests for workers
0.4.0 (2019-06-22)
------------------
* Semi-breaking change: **lox.thread** and **lox.process** now automatically pass
the object instance when decorating a method.
0.3.4 (2019-06-20)
------------------
* Print traceback in red when a thread crashes
0.3.3 (2019-06-19)
------------------
* Fix bug where thread in scatter of lox.thread double releases on empty queue
0.3.2 (2019-06-17)
------------------
* Fix manifest for installation from wheel
0.3.1 (2019-06-17)
------------------
* Fix package on pypi
0.3.0 (2019-06-01)
------------------
* Multiprocessing decorator. **lox.pool** renamed to **lox.thread**
* Substantial pytest bug fixes
* Documentation examples
* timeout for RWLock
0.2.1 (2019-05-25)
------------------
* Fix IndexSemaphore context manager
0.2.0 (2019-05-24)
------------------
* Added QLock
* Documentation syntax fixes
0.1.1 (2019-05-24)
------------------
* CICD test
0.1.0 (2019-05-24)
------------------
* First release on PyPI.
Raw data
{
"_id": null,
"home_page": "https://github.com/BrianPugh/lox",
"name": "lox",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "lox",
"author": "Brian Pugh",
"author_email": "bnp117@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/0f/b5/2bfa8da2a1dd6647c3ea0b8d7ae366bbb36b49f9f3858a253199daacb860/lox-0.12.0.tar.gz",
"platform": null,
"description": ".. image:: https://raw.githubusercontent.com/BrianPugh/lox/main/assets/lox_200w.png\n\n\n.. image:: https://img.shields.io/pypi/v/lox.svg\n :target: https://pypi.python.org/pypi/lox\n\n.. image:: https://circleci.com/gh/BrianPugh/lox.svg?style=svg\n :target: https://circleci.com/gh/BrianPugh/lox\n\n.. image:: https://readthedocs.org/projects/lox/badge/?version=latest\n :target: https://lox.readthedocs.io/en/latest/?badge=latest\n :alt: Documentation Status\n\n\nThreading and multiprocessing made easy.\n\n\n* Free software: Apache-2.0 license\n* Documentation: https://lox.readthedocs.io.\n* Python >=3.6\n\n\n**Lox** provides decorators and synchronization primitives to quickly add\nconcurrency to your projects.\n\nInstallation\n------------\n\n pip3 install --user lox\n\nFeatures\n--------\n\n* **Multithreading**: Powerful, intuitive multithreading in just 2 additional lines of code.\n\n* **Multiprocessing**: Truly parallel function execution with the same interface as **multithreading**.\n\n* **Synchronization**: Advanced thread synchronization, communication, and resource management tools.\n\nTodos\n-----\n\n* All objects except ``lox.process`` are for threads. These will eventually be multiprocess friendly.\n\nUsage\n-----\n\nEasy Multithreading\n^^^^^^^^^^^^^^^^^^^\n\n >>> import lox\n >>>\n >>> @lox.thread(4) # Will operate with a maximum of 4 threads\n ... def foo(x,y):\n ... return x*y\n >>> foo(3,4) # normal function calls still work\n 12\n >>> for i in range(5):\n ... foo.scatter(i, i+1)\n -ignore-\n >>> # foo is currently being executed in 4 threads\n >>> results = foo.gather() # block until results are ready\n >>> print(results) # Results are in the same order as scatter() calls\n [0, 2, 6, 12, 20]\n\nOr, for example, if you aren't allowed to directly decorate the function you\nwould like multithreaded/multiprocessed, you can just directly invoke the\ndecorator:\n\n.. code-block:: pycon\n\n >>> # Lets say we don't have direct access to this function\n ... def foo(x, y):\n ... return x * y\n ...\n >>>\n >>> def my_func():\n ... foo_threaded = lox.thread(foo)\n ... for i in range(5):\n ... foo_threaded.scatter(i, i + 1)\n ... results = foo_threaded.gather()\n ... # foo is currently being executed in default 50 thread executor pool\n ... return results\n ...\n\n\nThis also makes it easier to dynamically control the number of\nthread/processes in the executor pool. The syntax is a little weird, but\nthis is just explicitly invoking a decorator that has optional arguments:\n\n.. code-block:: pycon\n\n >>> # Set the number of executer threads to 10\n >>> foo_threaded = lox.thread(10)(foo)\n\n\nEasy Multiprocessing\n^^^^^^^^^^^^^^^^^^^^\n\n.. code-block:: pycon\n\n >>> import lox\n >>>\n >>> @lox.process(4) # Will operate with a pool of 4 processes\n ... def foo(x, y):\n ... return x * y\n ...\n >>> foo(3, 4) # normal function calls still work\n 12\n >>> for i in range(5):\n ... foo.scatter(i, i + 1)\n ...\n -ignore-\n >>> # foo is currently being executed in 4 processes\n >>> results = foo.gather() # block until results are ready\n >>> print(results) # Results are in the same order as scatter() calls\n [0, 2, 6, 12, 20]\n\n\nProgress Bar Support (tqdm)\n^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n.. code-block:: pycon\n\n >>> import lox\n >>> from random import random\n >>> from time import sleep\n >>>\n >>> @lox.thread(2)\n ... def foo(multiplier):\n ... sleep(multiplier * random())\n ...\n >>> for i in range(10):\n >>> foo.scatter(i)\n >>> results = foo.gather(tqdm=True)\n 90%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u258c | 9/10 [00:03<00:00, 1.32it/s]\n 100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 10/10 [00:06<00:00, 1.46s/it]\n\n\n=======\nHistory\n=======\n\n0.11.0 (2022-04-07)\n-------------------\n* Set number of workers to 0 (in thread execution) if the environment variable ``LOX_DEBUG`` is set to a true-like value (``true``, ``1``, etc.). Makes it easier to set breakpoints in multithreaded code without having to manually edit the decorator.\n\n0.10.0 (2021-12-18)\n-------------------\n* Remove dependency pinning.\n* Allow ``@lox.thread(0)``. This will execute ``scatter`` calls in parent thread.\n Useful for debugging breakpoints in parallelized code.\n\n0.9.0 (2020-11-25)\n------------------\n* ``tqdm`` support on ``lox.process.gather``. See v0.8.0 release notes for usage.\n\n0.8.0 (2020-11-25)\n------------------\n* ``tqdm`` support on ``lox.thread.gather``\n * Can be a bool::\n\n >>> my_func.gather(tqdm=True)\n\n * Can be a ``tqdm`` object::\n\n >>> from tqdm import tqdm\n >>> pbar = tqdm(total=100)\n >>> for _ in range(100):\n >>> my_func.scatter()\n >>> my_func.gather(tqdm=pbar)\n\n0.7.0 (2020-07-20)\n------------------\n* Complete rework of workers\n + Fix memory leaks\n* Drop support for python3.5\n* Drop support for chaining in favor of simpler codebase\n\n0.6.3 (2019-07-30)\n------------------\n* Alternative fix for 0.6.2.\n\n0.6.2 (2019-07-21)\n------------------\n* Update dependencies\n* Fix garbage-collecting exclusiviity\n\n0.6.1 (2019-07-21)\n------------------\n* Fix memory leak in ``lox.process``.\n\n0.6.0 (2019-07-21)\n------------------\n\n* ``lox.Announcement`` ``subscribe()`` calls now return another ``Announcement``\n object that behaves like a queue instead of an actual queue. Allows for\n many-queue-to-many-queue communications.\n\n* New Object: ``lox.Funnel``. allows for waiting on many queues for a complete\n set of inputs indicated by a job ID.\n\n0.5.0 (2019-07-01)\n------------------\n\n* New Object: ``lox.Announcement``. Allows a one-to-many thread queue with\n backlog support so that late subscribers can still get all (or most recent)\n announcements before they subscribed.\n\n* New Feature: ``lox.thread`` ``scatter`` calls can now be chained together.\n ``scatter`` now returns an ``int`` subclass that contains metadata to allow\n chaining. Each scatter call can have a maximum of 1 previous ``scatter`` result.\n\n* Documentation updates, theming, and logos\n\n0.4.3 (2019-06-24)\n------------------\n* Garbage collect cached decorated object methods\n\n0.4.2 (2019-06-23)\n------------------\n* Fixed multiple instances and successive scatter and gather calls to wrapped methods\n\n0.4.1 (2019-06-23)\n------------------\n* Fixed broken workers and unit tests for workers\n\n0.4.0 (2019-06-22)\n------------------\n* Semi-breaking change: **lox.thread** and **lox.process** now automatically pass\n the object instance when decorating a method.\n\n0.3.4 (2019-06-20)\n------------------\n* Print traceback in red when a thread crashes\n\n0.3.3 (2019-06-19)\n------------------\n* Fix bug where thread in scatter of lox.thread double releases on empty queue\n\n0.3.2 (2019-06-17)\n------------------\n\n* Fix manifest for installation from wheel\n\n0.3.1 (2019-06-17)\n------------------\n\n* Fix package on pypi\n\n0.3.0 (2019-06-01)\n------------------\n\n* Multiprocessing decorator. **lox.pool** renamed to **lox.thread**\n\n* Substantial pytest bug fixes\n\n* Documentation examples\n\n* timeout for RWLock\n\n0.2.1 (2019-05-25)\n------------------\n\n* Fix IndexSemaphore context manager\n\n0.2.0 (2019-05-24)\n------------------\n\n* Added QLock\n\n* Documentation syntax fixes\n\n0.1.1 (2019-05-24)\n------------------\n\n* CICD test\n\n0.1.0 (2019-05-24)\n------------------\n\n* First release on PyPI.\n",
"bugtrack_url": null,
"license": "MIT license",
"summary": "Threading and Multiprocessing for every project.",
"version": "0.12.0",
"project_urls": {
"Homepage": "https://github.com/BrianPugh/lox"
},
"split_keywords": [
"lox"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "029acc790ca4b853821b76acb5944d32036590a789e5f3b9e4f10a8962bcfda5",
"md5": "bf385688e299c6456320cac47cea2402",
"sha256": "ac0a392662f3a75cc9097655d26169d5e3564e2670431fd9884a7a09a09f6921"
},
"downloads": -1,
"filename": "lox-0.12.0-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "bf385688e299c6456320cac47cea2402",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 25372,
"upload_time": "2024-06-12T23:25:19",
"upload_time_iso_8601": "2024-06-12T23:25:19.576197Z",
"url": "https://files.pythonhosted.org/packages/02/9a/cc790ca4b853821b76acb5944d32036590a789e5f3b9e4f10a8962bcfda5/lox-0.12.0-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "0fb52bfa8da2a1dd6647c3ea0b8d7ae366bbb36b49f9f3858a253199daacb860",
"md5": "002b892b240577f79c73326824b86d8f",
"sha256": "cc7d5f867afb4dc7c2bce7bd6e90f4665c6df492863f35ff63229300b7219977"
},
"downloads": -1,
"filename": "lox-0.12.0.tar.gz",
"has_sig": false,
"md5_digest": "002b892b240577f79c73326824b86d8f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 37579,
"upload_time": "2024-06-12T23:25:21",
"upload_time_iso_8601": "2024-06-12T23:25:21.111867Z",
"url": "https://files.pythonhosted.org/packages/0f/b5/2bfa8da2a1dd6647c3ea0b8d7ae366bbb36b49f9f3858a253199daacb860/lox-0.12.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-06-12 23:25:21",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "BrianPugh",
"github_project": "lox",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"requirements": [],
"tox": true,
"lcname": "lox"
}
JSON
