[![Python versions](https://img.shields.io/badge/python-3.7%2C%203.8%2C%203.9%2C%203.10%2C%203.11%2C%203.12%2C%203.13-green.svg)]()
[![PyPI version](https://badge.fury.io/py/async-reduce.svg)](https://pypi.org/project/async-reduce/)
About Async-Reduce
==================
``async_reduce(coroutine)`` allows aggregate all *similar simultaneous*
ready to run `coroutine`s and reduce to running **only one** `coroutine`.
Other aggregated `coroutine`s will get result from single `coroutine`.
It can boost application performance in highly competitive execution of the
similar asynchronous operations and reduce load for inner systems.
Quick example
-------------
```python
from async_reduce import async_reduce
async def fetch_user_data(user_id: int) -> dict:
"""" Get user data from inner service """
url = 'http://inner-service/user/{}'.format(user_id)
return await http.get(url, timeout=10).json()
@web_server.router('/users/(\d+)')
async def handler_user_detail(request, user_id: int):
""" Handler for get detail information about user """
# all simultaneous requests of fetching user data for `user_id` will
# reduced to single request
user_data = await async_reduce(
fetch_user_data(user_id)
)
# sometimes ``async_reduce`` cannot detect similar coroutines and
# you should provide special argument `ident` for manually determination
user_statistics = await async_reduce(
DataBase.query('user_statistics').where(id=user_id).fetch_one(),
ident='db_user_statistics:{}'.format(user_id)
)
return Response(...)
```
In that example without using ``async_reduce`` if client performs **N**
simultaneous requests like `GET http://web_server/users/42` *web_server*
performs **N** requests to *inner-service* and **N** queries to *database*.
In total: **N** simultaneous requests emits **2 * N** requests to inner systems.
With ``async_reduce`` if client performs **N** simultaneous requests *web_server*
performs **one** request to *inner-service* and **one** query to *database*.
In total: **N** simultaneous requests emit only **2** requests to inner systems.
See other real [examples](https://github.com/sirkonst/async-reduce/tree/master/examples).
Similar coroutines determination
--------------------------------
``async_reduce(coroutine)`` tries to detect similar coroutines by hashing
local variables bounded on call. It does not work correctly if:
* one of the arguments is not hashable
* coroutine function is a method of class with specific state (like ORM)
* coroutine function has closure to unhashable variable
You can disable auto-determination by setting custom key to argument ``ident``.
Use as decorator
----------------
Also library provide special decorator ``@async_reduceable()``, example:
```python
from async_reduce import async_reduceable
@async_reduceable()
async def fetch_user_data(user_id: int) -> dict:
"""" Get user data from inner service """
url = 'http://inner-servicce/user/{}'.format(user_id)
return await http.get(url, timeout=10).json()
@web_server.router('/users/(\d+)')
async def handler_user_detail(request, user_id: int):
""" Handler for get detail information about user """
return await fetch_user_data(user_id)
```
Hooks
-----
Library supports hooks. Add-on hooks:
* **DebugHooks** - print about all triggered hooks
* **StatisticsOverallHooks** - general statistics on the use of `async_reduce`
* **StatisticsDetailHooks** - like `StatisticsOverallHooks` but detail statistics
about all `coroutine` processed by `async_reduce`
Example:
```python
from async_reduce import AsyncReducer
from async_reduce.hooks import DebugHooks
# define custom async_reduce with hooks
async_reduce = AsyncReducer(hooks=DebugHooks())
async def handler_user_detail(request, user_id: int):
user_data = await async_reduce(fetch_user_data(user_id))
```
See more detail example in [examples/example_hooks.py](https://github.com/sirkonst/async-reduce/blob/master/examples/example_hooks.py).
You can write custom hooks via inherit from [BaseHooks](https://github.com/sirkonst/async-reduce/blob/master/async_reduce/hooks/base.py).
Caveats
-------
* If single `coroutine` raises exceptions all aggregated `coroutine`s will get
same exception too
* If single `coroutine` is stuck all aggregated `coroutine`s will stuck too.
Limit execution time for `coroutine` and add retries (optional) to avoid it.
* Be careful when return mutable value from `coroutine` because single value
will shared. Prefer to use non-mutable value as coroutine return.
Development
-----------
See [DEVELOPMENT.md](https://github.com/sirkonst/async-reduce/blob/master/DEVELOPMENT.md).
Raw data
{
"_id": null,
"home_page": "https://github.com/sirkonst/async-reduce",
"name": "async-reduce",
"maintainer": "Konstantin Enchant",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "sirkonst@gmail.com",
"keywords": "asyncio, reduce",
"author": "Konstantin Enchant",
"author_email": "sirkonst@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/a8/81/ec55c99bf95349fd8bc24a9cb36903c0a842c4662ff803720e65d7c37d87/async_reduce-1.4.tar.gz",
"platform": null,
"description": "[![Python versions](https://img.shields.io/badge/python-3.7%2C%203.8%2C%203.9%2C%203.10%2C%203.11%2C%203.12%2C%203.13-green.svg)]()\n[![PyPI version](https://badge.fury.io/py/async-reduce.svg)](https://pypi.org/project/async-reduce/)\n\n\nAbout Async-Reduce\n==================\n\n``async_reduce(coroutine)`` allows aggregate all *similar simultaneous*\nready to run `coroutine`s and reduce to running **only one** `coroutine`.\nOther aggregated `coroutine`s will get result from single `coroutine`.\n\nIt can boost application performance in highly competitive execution of the\nsimilar asynchronous operations and reduce load for inner systems.\n\n\nQuick example\n-------------\n\n```python\nfrom async_reduce import async_reduce\n\n\nasync def fetch_user_data(user_id: int) -> dict:\n \"\"\"\" Get user data from inner service \"\"\"\n url = 'http://inner-service/user/{}'.format(user_id)\n\n return await http.get(url, timeout=10).json()\n\n\n@web_server.router('/users/(\\d+)')\nasync def handler_user_detail(request, user_id: int):\n \"\"\" Handler for get detail information about user \"\"\"\n\n # all simultaneous requests of fetching user data for `user_id` will\n # reduced to single request\n user_data = await async_reduce(\n fetch_user_data(user_id)\n )\n\n # sometimes ``async_reduce`` cannot detect similar coroutines and\n # you should provide special argument `ident` for manually determination\n user_statistics = await async_reduce(\n DataBase.query('user_statistics').where(id=user_id).fetch_one(),\n ident='db_user_statistics:{}'.format(user_id)\n )\n\n return Response(...)\n```\n\nIn that example without using ``async_reduce`` if client performs **N**\nsimultaneous requests like `GET http://web_server/users/42` *web_server*\nperforms **N** requests to *inner-service* and **N** queries to *database*.\nIn total: **N** simultaneous requests emits **2 * N** requests to inner systems.\n\nWith ``async_reduce`` if client performs **N** simultaneous requests *web_server*\nperforms **one** request to *inner-service* and **one** query to *database*.\nIn total: **N** simultaneous requests emit only **2** requests to inner systems.\n\nSee other real [examples](https://github.com/sirkonst/async-reduce/tree/master/examples).\n\n\nSimilar coroutines determination\n--------------------------------\n\n``async_reduce(coroutine)`` tries to detect similar coroutines by hashing\nlocal variables bounded on call. It does not work correctly if:\n\n* one of the arguments is not hashable\n* coroutine function is a method of class with specific state (like ORM)\n* coroutine function has closure to unhashable variable\n\nYou can disable auto-determination by setting custom key to argument ``ident``.\n\n\nUse as decorator\n----------------\n\nAlso library provide special decorator ``@async_reduceable()``, example:\n\n```python\nfrom async_reduce import async_reduceable\n\n\n@async_reduceable()\nasync def fetch_user_data(user_id: int) -> dict:\n \"\"\"\" Get user data from inner service \"\"\"\n url = 'http://inner-servicce/user/{}'.format(user_id)\n\n return await http.get(url, timeout=10).json()\n\n\n@web_server.router('/users/(\\d+)')\nasync def handler_user_detail(request, user_id: int):\n \"\"\" Handler for get detail information about user \"\"\"\n return await fetch_user_data(user_id)\n```\n\n\nHooks\n-----\n\nLibrary supports hooks. Add-on hooks:\n\n* **DebugHooks** - print about all triggered hooks\n* **StatisticsOverallHooks** - general statistics on the use of `async_reduce`\n* **StatisticsDetailHooks** - like `StatisticsOverallHooks` but detail statistics\nabout all `coroutine` processed by `async_reduce`\n\nExample:\n\n```python\nfrom async_reduce import AsyncReducer\nfrom async_reduce.hooks import DebugHooks\n\n# define custom async_reduce with hooks\nasync_reduce = AsyncReducer(hooks=DebugHooks())\n\n\nasync def handler_user_detail(request, user_id: int):\n user_data = await async_reduce(fetch_user_data(user_id))\n```\n\nSee more detail example in [examples/example_hooks.py](https://github.com/sirkonst/async-reduce/blob/master/examples/example_hooks.py).\n\nYou can write custom hooks via inherit from [BaseHooks](https://github.com/sirkonst/async-reduce/blob/master/async_reduce/hooks/base.py).\n\n\nCaveats\n-------\n\n* If single `coroutine` raises exceptions all aggregated `coroutine`s will get\nsame exception too\n\n* If single `coroutine` is stuck all aggregated `coroutine`s will stuck too.\nLimit execution time for `coroutine` and add retries (optional) to avoid it.\n\n* Be careful when return mutable value from `coroutine` because single value\nwill shared. Prefer to use non-mutable value as coroutine return.\n\n\nDevelopment\n-----------\n\nSee [DEVELOPMENT.md](https://github.com/sirkonst/async-reduce/blob/master/DEVELOPMENT.md).\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Reducer for similar simultaneously coroutines",
"version": "1.4",
"project_urls": {
"Homepage": "https://github.com/sirkonst/async-reduce"
},
"split_keywords": [
"asyncio",
" reduce"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1e09fa6b990a1c2f047230414ffc4a99504ff6a1f3f672bf109936deb313a9dd",
"md5": "579563eb84b47b54913dfdf1c8994bc4",
"sha256": "a94a95af5b4068241440bda4ceea9042dbb1798499656d49fece1d08e6c6b236"
},
"downloads": -1,
"filename": "async_reduce-1.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "579563eb84b47b54913dfdf1c8994bc4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 12240,
"upload_time": "2025-01-18T18:27:23",
"upload_time_iso_8601": "2025-01-18T18:27:23.983345Z",
"url": "https://files.pythonhosted.org/packages/1e/09/fa6b990a1c2f047230414ffc4a99504ff6a1f3f672bf109936deb313a9dd/async_reduce-1.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a881ec55c99bf95349fd8bc24a9cb36903c0a842c4662ff803720e65d7c37d87",
"md5": "9a7b11e8cea59e06e6ee2e0301ce97c3",
"sha256": "ce15bef1bd2b294f813f471c43a0f0fb7f77e27504e13b892dadd809430364d1"
},
"downloads": -1,
"filename": "async_reduce-1.4.tar.gz",
"has_sig": false,
"md5_digest": "9a7b11e8cea59e06e6ee2e0301ce97c3",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 13023,
"upload_time": "2025-01-18T18:27:25",
"upload_time_iso_8601": "2025-01-18T18:27:25.857138Z",
"url": "https://files.pythonhosted.org/packages/a8/81/ec55c99bf95349fd8bc24a9cb36903c0a842c4662ff803720e65d7c37d87/async_reduce-1.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-18 18:27:25",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "sirkonst",
"github_project": "async-reduce",
"travis_ci": false,
"coveralls": true,
"github_actions": false,
"tox": true,
"lcname": "async-reduce"
}