An attempt at comingling async-code and nonasync-code-in-a-thread in an argonomic way.
*Latest release 20241221.1*:
Doc fix for amap().
One of the difficulties in adapting non-async code for use in
an async world is that anything asynchronous needs to be turtles
all the way down: a single blocking synchronous call anywhere
in the call stack blocks the async event loop.
This module presently provides:
- `@afunc`: a decorator to make a synchronous function asynchronous
- `@agen`: a decorator to make a synchronous generator asynchronous
- `amap(func,iterable)`: asynchronous mapping of `func` over an iterable
- `async_iter(iterable)`: return an asynchronous iterator of an iterable
## <a name="afunc"></a>`afunc(*da, **dkw)`
A decorator for a synchronous function which turns it into
an asynchronous function.
Example:
@afunc
def func(count):
time.sleep(count)
return count
slept = await func(5)
## <a name="agen"></a>`agen(*da, **dkw)`
A decorator for a synchronous generator which turns it into
an asynchronous generator.
Exceptions in the synchronous generator are reraised in the asynchronous
generator.
Example:
@agen
def gen(count):
for i in range(count):
yield i
time.sleep(1.0)
async for item in gen(5):
print(item)
## <a name="amap"></a>`amap(func: Callable[[Any], Any], it: Iterable, concurrent=False, unordered=False, indexed=False)`
An asynchronous generator yielding the results of `func(item)`
for each `item` in the iterable `it`.
`func` may be a synchronous or asynchronous callable.
If `concurrent` is `False` (the default), run each `func(item)`
call in series.
If `concurrent` is true run the function calls as `asyncio`
tasks concurrently.
If `unordered` is true (default `False`) yield results as
they arrive, otherwise yield results in the order of the items
in `it`, but as they arrive - tasks still evaluate concurrently
if `concurrent` is true.
If `indexed` is true (default `False`) yield 2-tuples of
`(i,result)` instead of just `result`, where `i` is the index
if each item from `it` counting from `0`.
Example of an async function to fetch URLs in parallel.
async def get_urls(urls : List[str]):
""" Fetch `urls` in parallel.
Yield `(url,response)` 2-tuples.
"""
async for i, response in amap(
requests.get, urls,
concurrent=True, unordered=True, indexed=True,
):
yield urls[i], response
## <a name="async_iter"></a>`async_iter(it: Iterable)`
Return an asynchronous iterator yielding items from the iterable `it`.
# Release Log
*Release 20241221.1*:
Doc fix for amap().
*Release 20241221*:
* Simpler implementation of @afunc.
* Simplify implementation of @agen by using async_iter.
* Docstring improvements.
*Release 20241220*:
* New async_iter(Iterable) returning an asynchronous iterator of a synchronous iterable.
* New amap(func,iterable) asynchronously mapping a function over an iterable.
*Release 20241215*:
* @afunc: now uses asyncio.to_thread() instead of wrapping @agen, drop the decorator parameters since no queue or polling are now used.
* @agen: nonpolling implementation - now uses asyncio.to_thread() for the next(genfunc) step, drop the decorator parameters since no queue or polling are now used.
*Release 20241214.1*:
Doc update.
*Release 20241214*:
Initial release with @agen and @afunc decorators.
Raw data
{
"_id": null,
"home_page": null,
"name": "cs-naysync",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "python3",
"author": null,
"author_email": "Cameron Simpson <cs@cskk.id.au>",
"download_url": "https://files.pythonhosted.org/packages/5a/c7/50afdb3f2a955e8e110e97265d6679b51a28c131db0e36895693dcec0812/cs_naysync-20241221.1.tar.gz",
"platform": null,
"description": "An attempt at comingling async-code and nonasync-code-in-a-thread in an argonomic way.\n\n*Latest release 20241221.1*:\nDoc fix for amap().\n\nOne of the difficulties in adapting non-async code for use in\nan async world is that anything asynchronous needs to be turtles\nall the way down: a single blocking synchronous call anywhere\nin the call stack blocks the async event loop.\n\nThis module presently provides:\n- `@afunc`: a decorator to make a synchronous function asynchronous\n- `@agen`: a decorator to make a synchronous generator asynchronous\n- `amap(func,iterable)`: asynchronous mapping of `func` over an iterable\n- `async_iter(iterable)`: return an asynchronous iterator of an iterable\n\n## <a name=\"afunc\"></a>`afunc(*da, **dkw)`\n\nA decorator for a synchronous function which turns it into\nan asynchronous function.\n\nExample:\n\n @afunc\n def func(count):\n time.sleep(count)\n return count\n\n slept = await func(5)\n\n## <a name=\"agen\"></a>`agen(*da, **dkw)`\n\nA decorator for a synchronous generator which turns it into\nan asynchronous generator.\nExceptions in the synchronous generator are reraised in the asynchronous\ngenerator.\n\nExample:\n\n @agen\n def gen(count):\n for i in range(count):\n yield i\n time.sleep(1.0)\n\n async for item in gen(5):\n print(item)\n\n## <a name=\"amap\"></a>`amap(func: Callable[[Any], Any], it: Iterable, concurrent=False, unordered=False, indexed=False)`\n\nAn asynchronous generator yielding the results of `func(item)`\nfor each `item` in the iterable `it`.\n\n`func` may be a synchronous or asynchronous callable.\n\nIf `concurrent` is `False` (the default), run each `func(item)`\ncall in series.\n\nIf `concurrent` is true run the function calls as `asyncio`\ntasks concurrently.\nIf `unordered` is true (default `False`) yield results as\nthey arrive, otherwise yield results in the order of the items\nin `it`, but as they arrive - tasks still evaluate concurrently\nif `concurrent` is true.\n\nIf `indexed` is true (default `False`) yield 2-tuples of\n`(i,result)` instead of just `result`, where `i` is the index\nif each item from `it` counting from `0`.\n\nExample of an async function to fetch URLs in parallel.\n\n async def get_urls(urls : List[str]):\n \"\"\" Fetch `urls` in parallel.\n Yield `(url,response)` 2-tuples.\n \"\"\"\n async for i, response in amap(\n requests.get, urls,\n concurrent=True, unordered=True, indexed=True,\n ):\n yield urls[i], response\n\n## <a name=\"async_iter\"></a>`async_iter(it: Iterable)`\n\nReturn an asynchronous iterator yielding items from the iterable `it`.\n\n# Release Log\n\n\n\n*Release 20241221.1*:\nDoc fix for amap().\n\n*Release 20241221*:\n* Simpler implementation of @afunc.\n* Simplify implementation of @agen by using async_iter.\n* Docstring improvements.\n\n*Release 20241220*:\n* New async_iter(Iterable) returning an asynchronous iterator of a synchronous iterable.\n* New amap(func,iterable) asynchronously mapping a function over an iterable.\n\n*Release 20241215*:\n* @afunc: now uses asyncio.to_thread() instead of wrapping @agen, drop the decorator parameters since no queue or polling are now used.\n* @agen: nonpolling implementation - now uses asyncio.to_thread() for the next(genfunc) step, drop the decorator parameters since no queue or polling are now used.\n\n*Release 20241214.1*:\nDoc update.\n\n*Release 20241214*:\nInitial release with @agen and @afunc decorators.\n",
"bugtrack_url": null,
"license": "GNU General Public License v3 or later (GPLv3+)",
"summary": "An attempt at comingling async-code and nonasync-code-in-a-thread in an argonomic way.",
"version": "20241221.1",
"project_urls": {
"MonoRepo Commits": "https://bitbucket.org/cameron_simpson/css/commits/branch/main",
"Monorepo Git Mirror": "https://github.com/cameron-simpson/css",
"Monorepo Hg/Mercurial Mirror": "https://hg.sr.ht/~cameron-simpson/css",
"Source": "https://github.com/cameron-simpson/css/blob/main/lib/python/cs/naysync.py"
},
"split_keywords": [
"python3"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "26bd5b059b76d6a1722830d73217938fc29fe1b5e65efb8e78f83ca621c14956",
"md5": "b2d7f29206ddb7077e5ff380e0cdc4cd",
"sha256": "d6c63c38e3eb143351ada515446bc8376fa9d8a5ed20d55a071293b5407e0115"
},
"downloads": -1,
"filename": "cs_naysync-20241221.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b2d7f29206ddb7077e5ff380e0cdc4cd",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 5249,
"upload_time": "2024-12-20T23:53:29",
"upload_time_iso_8601": "2024-12-20T23:53:29.020364Z",
"url": "https://files.pythonhosted.org/packages/26/bd/5b059b76d6a1722830d73217938fc29fe1b5e65efb8e78f83ca621c14956/cs_naysync-20241221.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5ac750afdb3f2a955e8e110e97265d6679b51a28c131db0e36895693dcec0812",
"md5": "22faad7e28a1827c03aa8cbdbc3c3608",
"sha256": "d558ffb4309959026f45e69df66627657f9d54138e567f0f5148443b76621ccb"
},
"downloads": -1,
"filename": "cs_naysync-20241221.1.tar.gz",
"has_sig": false,
"md5_digest": "22faad7e28a1827c03aa8cbdbc3c3608",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 4890,
"upload_time": "2024-12-20T23:53:30",
"upload_time_iso_8601": "2024-12-20T23:53:30.210851Z",
"url": "https://files.pythonhosted.org/packages/5a/c7/50afdb3f2a955e8e110e97265d6679b51a28c131db0e36895693dcec0812/cs_naysync-20241221.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-20 23:53:30",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cameron-simpson",
"github_project": "css",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "cs-naysync"
}
JSON
