cs.later


Namecs.later JSON
Version 20230125 PyPI version JSON
download
home_pagehttps://bitbucket.org/cameron_simpson/css/commits/all
SummaryQueue functions for execution later in priority and time order.
upload_time2023-01-25 10:28:47
maintainer
docs_urlNone
authorCameron Simpson
requires_python
licenseGNU General Public License v3 or later (GPLv3+)
keywords python3
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            Queue functions for execution later in priority and time order.

*Latest release 20230125*:
Later: use HasThreadState mixin, provide @uses_later decorator.

I use `Later` objects for convenient queuing of functions whose
execution occurs later in a priority order with capacity constraints.

Why not futures?
I already had this before futures came out,
I prefer its naming scheme and interface,
and futures did not then support prioritised execution.

Use is simple enough: create a `Later` instance and typically queue
functions with the `.defer()` method::

    L = Later(4)      # a Later with a parallelism of 4
    ...
    LF = L.defer(func, *args, **kwargs)
    ...
    x = LF()          # collect result

The `.defer` method and its siblings return a `LateFunction`,
which is a subclass of `cs.result.Result`.
As such it is a callable,
so to collect the result you just call the `LateFunction`.

## Function `defer(func, *a, **kw)`

Queue a function using the current default Later.
Return the `LateFunction`.

## Class `LateFunction(cs.result.Result, cs.fsm.FSM, cs.gvutils.DOTNodeMixin)`

State information about a pending function,
a subclass of `cs.result.Result`.

A `LateFunction` is callable,
so a synchronous call can be done like this:

    def func():
      return 3
    L = Later(4)
    LF = L.defer(func)
    x = LF()
    print(x)        # prints 3

Used this way, if the called function raises an exception it is visible:

    LF = L.defer()
    try:
      x = LF()
    except SomeException as e:
      # handle the exception ...

To avoid handling exceptions with try/except the .wait()
method should be used:

    LF = L.defer()
    x, exc_info = LF.wait()
    if exc_info:
      # handle exception
      exc_type, exc_value, exc_traceback = exc_info
      ...
    else:
      # use `x`, the function result

TODO: .cancel(), timeout for wait().

*Method `LateFunction.__init__(self, func, name=None, retry_delay=None)`*:
Initialise a `LateFunction`.

Parameters:
* `func` is the callable for later execution.
* `name`, if supplied, specifies an identifying name for the `LateFunction`.
* `retry_local`: time delay before retry of this function on RetryError.
  Default from `later.retry_delay`.

## Class `LatePool`

A context manager after the style of subprocess.Pool
but with deferred completion.

Example usage:

    L = Later(4)    # a 4 thread Later
    with LatePool(L) as LP:
      # several calls to LatePool.defer, perhaps looped
      LP.defer(func, *args, **kwargs)
      LP.defer(func, *args, **kwargs)
    # now we can LP.join() to block for all `LateFunctions`
    #
    # or iterate over LP to collect `LateFunction`s as they complete
    for LF in LP:
      result = LF()
      print(result)

*Method `LatePool.__init__(self, *, later, priority=None, delay=None, when=None, pfx=None, block=False)`*:
Initialise the `LatePool`.

Parameters:
* `later`: optional `Later` instance, default from `Later.default()`
* `priority`, `delay`, `when`, `name`, `pfx`:
  default values passed to Later.submit.
* `block`: if true, wait for `LateFunction` completion
  before leaving __exit__.

## Class `Later(cs.resources.MultiOpenMixin, cs.threads.HasThreadState, cs.context.ContextManagerMixin)`

A management class to queue function calls for later execution.

Methods are provided for submitting functions to run ASAP or
after a delay or after other pending functions. These methods
return `LateFunction`s, a subclass of `cs.result.Result`.

A Later instance' close method closes the Later for further
submission.
Shutdown does not imply that all submitted functions have
completed or even been dispatched.
Callers may wait for completion and optionally cancel functions.

TODO: __enter__ returns a SubLater, __exit__ closes the SubLater.

TODO: drop global default Later.

*Method `Later.__init__(self, capacity, name=None, inboundCapacity=0, retry_delay=None)`*:
Initialise the Later instance.

Parameters:
* `capacity`: resource contraint on this Later; if an int, it is used
  to size a Semaphore to constrain the number of dispatched functions
  which may be in play at a time; if not an int it is presumed to be a
  suitable Semaphore-like object, perhaps shared with other subsystems.
* `name`: optional identifying name for this instance.
* `inboundCapacity`: if >0, used as a limit on the number of
  undispatched functions that may be queued up; the default is 0 (no
  limit).  Calls to submit functions when the inbound limit is reached
  block until some functions are dispatched.
* `retry_delay`: time delay for requeued functions.
  Default: `DEFAULT_RETRY_DELAY`.

## Function `retry(retry_interval, func, *a, **kw)`

Call the callable `func` with the supplied arguments.

If it raises `RetryError`,
run `time.sleep(retry_interval)`
and then call again until it does not raise `RetryError`.

## Class `RetryError(builtins.Exception, builtins.BaseException)`

Exception raised by functions which should be resubmitted to the queue.

## Class `SubLater`

A class for managing a group of deferred tasks using an existing `Later`.

*Method `SubLater.__init__(self, *, later: cs.later.Later)`*:
Initialise the `SubLater` with its parent `Later`.

TODO: accept `discard=False` param to suppress the queue and
associated checks.

# Release Log



*Release 20230125*:
Later: use HasThreadState mixin, provide @uses_later decorator.

*Release 20221228*:
* Later: replace submittable checks with decorator accepting a force=True override.
* Later.defer_iterable: implement greedy vs nongreedy.

*Release 20220918*:
* Later.wait: new optional timeout, replaces hardwired 5s timeout; return the Event.finished return.
* Later: expose the finished Event as .finished_event.
* Later.finished_event logic fixes.

*Release 20220805*:
Update for recent changes to Result.

*Release 20220605*:
* Later: replace the default = _ThreadLocal with a default = ThreadState(current=None).
* Later: fold startup/shutdown/__enter__/__exit__ into the startup_shutdown context manager, fixes MultiOpenMixin misbehaviour.

*Release 20201021*:
* Later: subclass MultiOpenMixin.
* Later._defer: make a shallow copy of the keyword parameters as we do for the positional parameters.

*Release 20191007*:
Drop pipeline functionality, moved to new cs.pipeline module.

*Release 20181231*:
* New SubLater class to provide a grouping for deferred functions and an iteration to collect them as they complete.
* Drop WorkerThreadPool (leaks idle Threads, brings little benefit).
* Later: drop worker queue thread and semaphore, just try a dispatch on submit or complete.
* Later: drop tracking code. Drop capacity context manager, never used.

*Release 20181109*:
* Updates for cs.asynchron renamed to cs.result.
* Later: no longer subclass MultiOpenMixin, users now call close to end submission, shutdown to terminate activity and wait to await finalisation.
* Clean lint, add docstrings, minor bugfixes.

*Release 20160828*:
* Use "install_requires" instead of "requires" in DISTINFO.
* Add LatePool, a context manager after the flavour of subprocess.Pool.
* Python 2 fix.
* Rename NestingOpenCloseMixin to MultiOpenMixin - easier to type, say and remember, not to mention being more accurate.
* Add RetryError exception for use by Later.retriable.
* LateFunction: support RetryError exception from function, causing requeue.
* LateFunction: accept retry_delay parameter, used to delay function retry.
* Later.defer_iterable: accept `test_ready` callable to support deferring iteration until the callable returns truthiness.
* New function retry(retry_interval, func, *a, **kw) to call func until it does not raise RetryError.
* Later: wrap several methods in @MultiOpenMixin.is_opened.
* Assorted bugfixes and improvements.

*Release 20150115*:
First PyPI release.

            

Raw data

            {
    "_id": null,
    "home_page": "https://bitbucket.org/cameron_simpson/css/commits/all",
    "name": "cs.later",
    "maintainer": "",
    "docs_url": null,
    "requires_python": "",
    "maintainer_email": "",
    "keywords": "python3",
    "author": "Cameron Simpson",
    "author_email": "Cameron Simpson <cs@cskk.id.au>",
    "download_url": "https://files.pythonhosted.org/packages/52/23/bbbd26c7e5316f8549c84ee47416054b770347c8cc695be0142225fa77a4/cs.later-20230125.tar.gz",
    "platform": null,
    "description": "Queue functions for execution later in priority and time order.\n\n*Latest release 20230125*:\nLater: use HasThreadState mixin, provide @uses_later decorator.\n\nI use `Later` objects for convenient queuing of functions whose\nexecution occurs later in a priority order with capacity constraints.\n\nWhy not futures?\nI already had this before futures came out,\nI prefer its naming scheme and interface,\nand futures did not then support prioritised execution.\n\nUse is simple enough: create a `Later` instance and typically queue\nfunctions with the `.defer()` method::\n\n    L = Later(4)      # a Later with a parallelism of 4\n    ...\n    LF = L.defer(func, *args, **kwargs)\n    ...\n    x = LF()          # collect result\n\nThe `.defer` method and its siblings return a `LateFunction`,\nwhich is a subclass of `cs.result.Result`.\nAs such it is a callable,\nso to collect the result you just call the `LateFunction`.\n\n## Function `defer(func, *a, **kw)`\n\nQueue a function using the current default Later.\nReturn the `LateFunction`.\n\n## Class `LateFunction(cs.result.Result, cs.fsm.FSM, cs.gvutils.DOTNodeMixin)`\n\nState information about a pending function,\na subclass of `cs.result.Result`.\n\nA `LateFunction` is callable,\nso a synchronous call can be done like this:\n\n    def func():\n      return 3\n    L = Later(4)\n    LF = L.defer(func)\n    x = LF()\n    print(x)        # prints 3\n\nUsed this way, if the called function raises an exception it is visible:\n\n    LF = L.defer()\n    try:\n      x = LF()\n    except SomeException as e:\n      # handle the exception ...\n\nTo avoid handling exceptions with try/except the .wait()\nmethod should be used:\n\n    LF = L.defer()\n    x, exc_info = LF.wait()\n    if exc_info:\n      # handle exception\n      exc_type, exc_value, exc_traceback = exc_info\n      ...\n    else:\n      # use `x`, the function result\n\nTODO: .cancel(), timeout for wait().\n\n*Method `LateFunction.__init__(self, func, name=None, retry_delay=None)`*:\nInitialise a `LateFunction`.\n\nParameters:\n* `func` is the callable for later execution.\n* `name`, if supplied, specifies an identifying name for the `LateFunction`.\n* `retry_local`: time delay before retry of this function on RetryError.\n  Default from `later.retry_delay`.\n\n## Class `LatePool`\n\nA context manager after the style of subprocess.Pool\nbut with deferred completion.\n\nExample usage:\n\n    L = Later(4)    # a 4 thread Later\n    with LatePool(L) as LP:\n      # several calls to LatePool.defer, perhaps looped\n      LP.defer(func, *args, **kwargs)\n      LP.defer(func, *args, **kwargs)\n    # now we can LP.join() to block for all `LateFunctions`\n    #\n    # or iterate over LP to collect `LateFunction`s as they complete\n    for LF in LP:\n      result = LF()\n      print(result)\n\n*Method `LatePool.__init__(self, *, later, priority=None, delay=None, when=None, pfx=None, block=False)`*:\nInitialise the `LatePool`.\n\nParameters:\n* `later`: optional `Later` instance, default from `Later.default()`\n* `priority`, `delay`, `when`, `name`, `pfx`:\n  default values passed to Later.submit.\n* `block`: if true, wait for `LateFunction` completion\n  before leaving __exit__.\n\n## Class `Later(cs.resources.MultiOpenMixin, cs.threads.HasThreadState, cs.context.ContextManagerMixin)`\n\nA management class to queue function calls for later execution.\n\nMethods are provided for submitting functions to run ASAP or\nafter a delay or after other pending functions. These methods\nreturn `LateFunction`s, a subclass of `cs.result.Result`.\n\nA Later instance' close method closes the Later for further\nsubmission.\nShutdown does not imply that all submitted functions have\ncompleted or even been dispatched.\nCallers may wait for completion and optionally cancel functions.\n\nTODO: __enter__ returns a SubLater, __exit__ closes the SubLater.\n\nTODO: drop global default Later.\n\n*Method `Later.__init__(self, capacity, name=None, inboundCapacity=0, retry_delay=None)`*:\nInitialise the Later instance.\n\nParameters:\n* `capacity`: resource contraint on this Later; if an int, it is used\n  to size a Semaphore to constrain the number of dispatched functions\n  which may be in play at a time; if not an int it is presumed to be a\n  suitable Semaphore-like object, perhaps shared with other subsystems.\n* `name`: optional identifying name for this instance.\n* `inboundCapacity`: if >0, used as a limit on the number of\n  undispatched functions that may be queued up; the default is 0 (no\n  limit).  Calls to submit functions when the inbound limit is reached\n  block until some functions are dispatched.\n* `retry_delay`: time delay for requeued functions.\n  Default: `DEFAULT_RETRY_DELAY`.\n\n## Function `retry(retry_interval, func, *a, **kw)`\n\nCall the callable `func` with the supplied arguments.\n\nIf it raises `RetryError`,\nrun `time.sleep(retry_interval)`\nand then call again until it does not raise `RetryError`.\n\n## Class `RetryError(builtins.Exception, builtins.BaseException)`\n\nException raised by functions which should be resubmitted to the queue.\n\n## Class `SubLater`\n\nA class for managing a group of deferred tasks using an existing `Later`.\n\n*Method `SubLater.__init__(self, *, later: cs.later.Later)`*:\nInitialise the `SubLater` with its parent `Later`.\n\nTODO: accept `discard=False` param to suppress the queue and\nassociated checks.\n\n# Release Log\n\n\n\n*Release 20230125*:\nLater: use HasThreadState mixin, provide @uses_later decorator.\n\n*Release 20221228*:\n* Later: replace submittable checks with decorator accepting a force=True override.\n* Later.defer_iterable: implement greedy vs nongreedy.\n\n*Release 20220918*:\n* Later.wait: new optional timeout, replaces hardwired 5s timeout; return the Event.finished return.\n* Later: expose the finished Event as .finished_event.\n* Later.finished_event logic fixes.\n\n*Release 20220805*:\nUpdate for recent changes to Result.\n\n*Release 20220605*:\n* Later: replace the default = _ThreadLocal with a default = ThreadState(current=None).\n* Later: fold startup/shutdown/__enter__/__exit__ into the startup_shutdown context manager, fixes MultiOpenMixin misbehaviour.\n\n*Release 20201021*:\n* Later: subclass MultiOpenMixin.\n* Later._defer: make a shallow copy of the keyword parameters as we do for the positional parameters.\n\n*Release 20191007*:\nDrop pipeline functionality, moved to new cs.pipeline module.\n\n*Release 20181231*:\n* New SubLater class to provide a grouping for deferred functions and an iteration to collect them as they complete.\n* Drop WorkerThreadPool (leaks idle Threads, brings little benefit).\n* Later: drop worker queue thread and semaphore, just try a dispatch on submit or complete.\n* Later: drop tracking code. Drop capacity context manager, never used.\n\n*Release 20181109*:\n* Updates for cs.asynchron renamed to cs.result.\n* Later: no longer subclass MultiOpenMixin, users now call close to end submission, shutdown to terminate activity and wait to await finalisation.\n* Clean lint, add docstrings, minor bugfixes.\n\n*Release 20160828*:\n* Use \"install_requires\" instead of \"requires\" in DISTINFO.\n* Add LatePool, a context manager after the flavour of subprocess.Pool.\n* Python 2 fix.\n* Rename NestingOpenCloseMixin to MultiOpenMixin - easier to type, say and remember, not to mention being more accurate.\n* Add RetryError exception for use by Later.retriable.\n* LateFunction: support RetryError exception from function, causing requeue.\n* LateFunction: accept retry_delay parameter, used to delay function retry.\n* Later.defer_iterable: accept `test_ready` callable to support deferring iteration until the callable returns truthiness.\n* New function retry(retry_interval, func, *a, **kw) to call func until it does not raise RetryError.\n* Later: wrap several methods in @MultiOpenMixin.is_opened.\n* Assorted bugfixes and improvements.\n\n*Release 20150115*:\nFirst PyPI release.\n",
    "bugtrack_url": null,
    "license": "GNU General Public License v3 or later (GPLv3+)",
    "summary": "Queue functions for execution later in priority and time order.",
    "version": "20230125",
    "split_keywords": [
        "python3"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "5b15f864ca19e79d70078eeadb5d1881093e58c06c972d1f2d98b01fd2cab143",
                "md5": "64b6d143bd5da7f0aa5920c6e0618f07",
                "sha256": "f7ed31bd5d7251b946b4e85dd1ee9c3882edbd6e56dbd0ba602577e20f2b13cf"
            },
            "downloads": -1,
            "filename": "cs.later-20230125-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "64b6d143bd5da7f0aa5920c6e0618f07",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 14803,
            "upload_time": "2023-01-25T10:28:45",
            "upload_time_iso_8601": "2023-01-25T10:28:45.658202Z",
            "url": "https://files.pythonhosted.org/packages/5b/15/f864ca19e79d70078eeadb5d1881093e58c06c972d1f2d98b01fd2cab143/cs.later-20230125-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "5223bbbd26c7e5316f8549c84ee47416054b770347c8cc695be0142225fa77a4",
                "md5": "580c895283177d853f594f1ab79ad084",
                "sha256": "4dc92e8856d76f0681b5d69041729317355c3e979659aa41f63d8b88d7d46d82"
            },
            "downloads": -1,
            "filename": "cs.later-20230125.tar.gz",
            "has_sig": false,
            "md5_digest": "580c895283177d853f594f1ab79ad084",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 19182,
            "upload_time": "2023-01-25T10:28:47",
            "upload_time_iso_8601": "2023-01-25T10:28:47.591879Z",
            "url": "https://files.pythonhosted.org/packages/52/23/bbbd26c7e5316f8549c84ee47416054b770347c8cc695be0142225fa77a4/cs.later-20230125.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-01-25 10:28:47",
    "github": false,
    "gitlab": false,
    "bitbucket": false,
    "lcname": "cs.later"
}
        
Elapsed time: 0.11044s