# asyncio-pool-ng
[](https://pypi.org/project/asyncio-pool-ng/)
[](https://pypi.org/project/asyncio-pool-ng/)
[](https://opensource.org/licenses/MIT)
[](https://github.com/smithk86/asyncio-pool-ng/actions?query=workflow%3Aci)
[](https://github.com/psf/black)
## About
**AsyncioPoolNG** takes the ideas used in [asyncio-pool](https://github.com/gistart/asyncio-pool) and wraps them around an [asyncio.TaskGroup](https://docs.python.org/3/library/asyncio-task.html#task-groups).
`AsyncioPool` has three main functions `spawn`, `map`, and `itermap`.
1. `spawn`: Schedule an async function on the pool and get a future back which will eventually have either the result or the exception from the function.
2. `map`: Spawn an async function for each item in an iterable object, and return a set containing a future for each item.
- `asyncio.wait()` can be used to wait for the set of futures to complete.
- When the `AsyncioPool` closes, it will wait for all tasks to complete. All pending futures will be complete once it is closed.
3. `itermap`: Works similarly to `map` but returns an [Async Generator](https://docs.python.org/3/library/typing.html#typing.AsyncGenerator "Async Generator") which yields each future as it completes.
## Differences from asyncio-pool
1. `asyncio-pool-ng` implements [Python typing](https://typing.readthedocs.io/en/latest/) and passes validation checks with [mypy](http://mypy-lang.org/)'s strict mode. This helps IDEs and static type checkers know what type of result to expect when getting data from a completed future.
2. `asyncio-pool` uses callbacks to process data before returning it; `asyncio-pool-ng` only returns [Future](https://docs.python.org/3.10/library/asyncio-future.html#asyncio.Future) instances directly. The future will contain either a result or an exception which can then be handled as needed.
3. While `asyncio-pool` schedules [Coroutine](https://docs.python.org/3/library/typing.html#typing.Coroutine) instances directly, `asyncio-pool-ng` takes the callable and arguments, and creates the Coroutine instance at execution time.
## Example
```python title="example.py"
import asyncio
import logging
from random import random
from asyncio_pool import AsyncioPool
logging.basicConfig(level=logging.INFO)
async def worker(number: int) -> int:
await asyncio.sleep(random() / 2)
return number * 2
async def main() -> None:
result: int = 0
results: list[int] = []
async with AsyncioPool(2) as pool:
"""spawn task and wait for the results"""
result = await pool.spawn(worker, 5)
assert result == 10
logging.info(f"results for pool.spawn(worker, 5): {result}")
"""spawn task and get results later"""
future: asyncio.Future[int] = pool.spawn(worker, 5)
# do other stuff
result = await future
assert result == 10
"""map an async function to a set of values"""
futures: set[asyncio.Future[int]] = pool.map(worker, range(10))
await asyncio.wait(futures)
results = [x.result() for x in futures]
logging.info(f"results for pool.map(worker, range(10)): {results}")
results.sort()
assert results == [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
"""iterate futures as they complete"""
logging.info("results for pool.itermap(worker, range(10)):")
results = []
async for future in pool.itermap(worker, range(10)):
results.append(future.result())
logging.info(f"> {future.result()}")
results.sort()
assert results == [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
asyncio.run(main())
```
Raw data
{
"_id": null,
"home_page": "https://github.com/smithk86/asyncio-pool-ng",
"name": "asyncio-pool-ng",
"maintainer": null,
"docs_url": null,
"requires_python": "<4,>=3.11",
"maintainer_email": null,
"keywords": null,
"author": "Kyle Smith",
"author_email": "smithk86@smc3.com",
"download_url": "https://files.pythonhosted.org/packages/98/5f/6d473dd82f3f6d9af977343a5d38d4ffc9c948160fa4006ffd9641233de5/asyncio_pool_ng-0.8.1.tar.gz",
"platform": null,
"description": "# asyncio-pool-ng\n\n[](https://pypi.org/project/asyncio-pool-ng/)\n[](https://pypi.org/project/asyncio-pool-ng/)\n[](https://opensource.org/licenses/MIT)\n[](https://github.com/smithk86/asyncio-pool-ng/actions?query=workflow%3Aci)\n[](https://github.com/psf/black)\n\n## About\n\n**AsyncioPoolNG** takes the ideas used in [asyncio-pool](https://github.com/gistart/asyncio-pool) and wraps them around an [asyncio.TaskGroup](https://docs.python.org/3/library/asyncio-task.html#task-groups).\n\n`AsyncioPool` has three main functions `spawn`, `map`, and `itermap`.\n\n1. `spawn`: Schedule an async function on the pool and get a future back which will eventually have either the result or the exception from the function.\n2. `map`: Spawn an async function for each item in an iterable object, and return a set containing a future for each item.\n\n- `asyncio.wait()` can be used to wait for the set of futures to complete.\n- When the `AsyncioPool` closes, it will wait for all tasks to complete. All pending futures will be complete once it is closed.\n\n3. `itermap`: Works similarly to `map` but returns an [Async Generator](https://docs.python.org/3/library/typing.html#typing.AsyncGenerator \"Async Generator\") which yields each future as it completes.\n\n## Differences from asyncio-pool\n\n1. `asyncio-pool-ng` implements [Python typing](https://typing.readthedocs.io/en/latest/) and passes validation checks with [mypy](http://mypy-lang.org/)'s strict mode. This helps IDEs and static type checkers know what type of result to expect when getting data from a completed future.\n2. `asyncio-pool` uses callbacks to process data before returning it; `asyncio-pool-ng` only returns [Future](https://docs.python.org/3.10/library/asyncio-future.html#asyncio.Future) instances directly. The future will contain either a result or an exception which can then be handled as needed.\n3. While `asyncio-pool` schedules [Coroutine](https://docs.python.org/3/library/typing.html#typing.Coroutine) instances directly, `asyncio-pool-ng` takes the callable and arguments, and creates the Coroutine instance at execution time.\n\n## Example\n\n```python title=\"example.py\"\nimport asyncio\nimport logging\nfrom random import random\n\nfrom asyncio_pool import AsyncioPool\n\n\nlogging.basicConfig(level=logging.INFO)\n\n\nasync def worker(number: int) -> int:\n await asyncio.sleep(random() / 2)\n return number * 2\n\n\nasync def main() -> None:\n result: int = 0\n results: list[int] = []\n\n async with AsyncioPool(2) as pool:\n \"\"\"spawn task and wait for the results\"\"\"\n result = await pool.spawn(worker, 5)\n assert result == 10\n logging.info(f\"results for pool.spawn(worker, 5): {result}\")\n\n \"\"\"spawn task and get results later\"\"\"\n future: asyncio.Future[int] = pool.spawn(worker, 5)\n\n # do other stuff\n\n result = await future\n assert result == 10\n\n \"\"\"map an async function to a set of values\"\"\"\n futures: set[asyncio.Future[int]] = pool.map(worker, range(10))\n await asyncio.wait(futures)\n results = [x.result() for x in futures]\n logging.info(f\"results for pool.map(worker, range(10)): {results}\")\n results.sort()\n assert results == [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]\n\n \"\"\"iterate futures as they complete\"\"\"\n logging.info(\"results for pool.itermap(worker, range(10)):\")\n results = []\n async for future in pool.itermap(worker, range(10)):\n results.append(future.result())\n logging.info(f\"> {future.result()}\")\n\n results.sort()\n assert results == [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]\n\n\nasyncio.run(main())\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A pool of coroutine functions.",
"version": "0.8.1",
"project_urls": {
"Documentation": "https://smithk86.github.io/asyncio-pool-ng",
"Homepage": "https://github.com/smithk86/asyncio-pool-ng",
"Repository": "https://github.com/smithk86/asyncio-pool-ng"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e2d73db4d6bbf8a77e5a7b7ada9046343d6e3a16a95e8d400808637041040a2f",
"md5": "e0a0bd1a8a6a396adacec7e39c52842d",
"sha256": "8dba804c078b2a27b7885980d9b1cf99eaa077f00c07a4bf32a67233f09a1d7c"
},
"downloads": -1,
"filename": "asyncio_pool_ng-0.8.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e0a0bd1a8a6a396adacec7e39c52842d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4,>=3.11",
"size": 7374,
"upload_time": "2024-04-02T13:10:38",
"upload_time_iso_8601": "2024-04-02T13:10:38.871890Z",
"url": "https://files.pythonhosted.org/packages/e2/d7/3db4d6bbf8a77e5a7b7ada9046343d6e3a16a95e8d400808637041040a2f/asyncio_pool_ng-0.8.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "985f6d473dd82f3f6d9af977343a5d38d4ffc9c948160fa4006ffd9641233de5",
"md5": "bc875afb7b3fc3d921af1d717b3c8211",
"sha256": "b5eb07ce36822b1c15832bd0d6c450671ebcd2d7303a9dac29f36b8c6b667a7b"
},
"downloads": -1,
"filename": "asyncio_pool_ng-0.8.1.tar.gz",
"has_sig": false,
"md5_digest": "bc875afb7b3fc3d921af1d717b3c8211",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4,>=3.11",
"size": 7003,
"upload_time": "2024-04-02T13:10:40",
"upload_time_iso_8601": "2024-04-02T13:10:40.756264Z",
"url": "https://files.pythonhosted.org/packages/98/5f/6d473dd82f3f6d9af977343a5d38d4ffc9c948160fa4006ffd9641233de5/asyncio_pool_ng-0.8.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-02 13:10:40",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "smithk86",
"github_project": "asyncio-pool-ng",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "asyncio-pool-ng"
}
JSON
