[]()
[](https://pypi.org/project/async-reduce/)
About Async-Reduce
==================
``async_reduce(coroutine)`` allows aggregate all *similar simultaneous*
ready to run `coroutine`s and reduce to running **only one** `coroutine`.
Other aggregated `coroutine`s will get result from single `coroutine`.
It can boost application performance in highly competitive execution of the
similar asynchronous operations and reduce load for inner systems.
Quick example
-------------
```python
from async_reduce import async_reduce
async def fetch_user_data(user_id: int) -> dict:
"""" Get user data from inner service """
url = 'http://inner-service/user/{}'.format(user_id)
return await http.get(url, timeout=10).json()
@web_server.router('/users/(\d+)')
async def handler_user_detail(request, user_id: int):
""" Handler for get detail information about user """
# all simultaneous requests of fetching user data for `user_id` will
# reduced to single request
user_data = await async_reduce(
fetch_user_data(user_id)
)
# sometimes ``async_reduce`` cannot detect similar coroutines and
# you should provide special argument `ident` for manually determination
user_statistics = await async_reduce(
DataBase.query('user_statistics').where(id=user_id).fetch_one(),
ident='db_user_statistics:{}'.format(user_id)
)
return Response(...)
```
In that example without using ``async_reduce`` if client performs **N**
simultaneous requests like `GET http://web_server/users/42` *web_server*
performs **N** requests to *inner-service* and **N** queries to *database*.
In total: **N** simultaneous requests emits **2 * N** requests to inner systems.
With ``async_reduce`` if client performs **N** simultaneous requests *web_server*
performs **one** request to *inner-service* and **one** query to *database*.
In total: **N** simultaneous requests emit only **2** requests to inner systems.
See other real [examples](https://github.com/sirkonst/async-reduce/tree/master/examples).
Similar coroutines determination
--------------------------------
``async_reduce(coroutine)`` tries to detect similar coroutines by hashing
local variables bounded on call. It does not work correctly if:
* one of the arguments is not hashable
* coroutine function is a method of class with specific state (like ORM)
* coroutine function has closure to unhashable variable
You can disable auto-determination by setting custom key to argument ``ident``.
Use as decorator
----------------
Also library provide special decorator ``@async_reduceable()``, example:
```python
from async_reduce import async_reduceable
@async_reduceable()
async def fetch_user_data(user_id: int) -> dict:
"""" Get user data from inner service """
url = 'http://inner-servicce/user/{}'.format(user_id)
return await http.get(url, timeout=10).json()
@web_server.router('/users/(\d+)')
async def handler_user_detail(request, user_id: int):
""" Handler for get detail information about user """
return await fetch_user_data(user_id)
```
Hooks
-----
Library supports hooks. Add-on hooks:
* **DebugHooks** - print about all triggered hooks
* **StatisticsOverallHooks** - general statistics on the use of `async_reduce`
* **StatisticsDetailHooks** - like `StatisticsOverallHooks` but detail statistics
about all `coroutine` processed by `async_reduce`
Example:
```python
from async_reduce import AsyncReducer
from async_reduce.hooks import DebugHooks
# define custom async_reduce with hooks
async_reduce = AsyncReducer(hooks=DebugHooks())
async def handler_user_detail(request, user_id: int):
user_data = await async_reduce(fetch_user_data(user_id))
```
See more detail example in [examples/example_hooks.py](https://github.com/sirkonst/async-reduce/blob/master/examples/example_hooks.py).
You can write custom hooks via inherit from [BaseHooks](https://github.com/sirkonst/async-reduce/blob/master/async_reduce/hooks/base.py).
Caveats
-------
* If single `coroutine` raises exceptions all aggregated `coroutine`s will get
same exception too
* If single `coroutine` is stuck all aggregated `coroutine`s will stuck too.
Limit execution time for `coroutine` and add retries (optional) to avoid it.
* Be careful when return mutable value from `coroutine` because single value
will shared. Prefer to use non-mutable value as coroutine return.
Development
-----------
See [DEVELOPMENT.md](https://github.com/sirkonst/async-reduce/blob/master/DEVELOPMENT.md).
Raw data
{
"_id": null,
"home_page": "https://github.com/sirkonst/async-reduce",
"name": "async-reduce",
"maintainer": "Konstantin Enchant",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "sirkonst@gmail.com",
"keywords": "asyncio,reduce",
"author": "Konstantin Enchant",
"author_email": "sirkonst@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/39/c6/3b183a249e2085f3c0c6ffe71711933b28879b72de2f5fe08be6ed290cdd/async_reduce-1.3.tar.gz",
"platform": null,
"description": "[]()\n[](https://pypi.org/project/async-reduce/)\n\n\nAbout Async-Reduce\n==================\n\n``async_reduce(coroutine)`` allows aggregate all *similar simultaneous*\nready to run `coroutine`s and reduce to running **only one** `coroutine`.\nOther aggregated `coroutine`s will get result from single `coroutine`.\n\nIt can boost application performance in highly competitive execution of the\nsimilar asynchronous operations and reduce load for inner systems.\n\n\nQuick example\n-------------\n\n```python\nfrom async_reduce import async_reduce\n\n\nasync def fetch_user_data(user_id: int) -> dict:\n \"\"\"\" Get user data from inner service \"\"\"\n url = 'http://inner-service/user/{}'.format(user_id)\n\n return await http.get(url, timeout=10).json()\n\n\n@web_server.router('/users/(\\d+)')\nasync def handler_user_detail(request, user_id: int):\n \"\"\" Handler for get detail information about user \"\"\"\n\n # all simultaneous requests of fetching user data for `user_id` will\n # reduced to single request\n user_data = await async_reduce(\n fetch_user_data(user_id)\n )\n\n # sometimes ``async_reduce`` cannot detect similar coroutines and\n # you should provide special argument `ident` for manually determination\n user_statistics = await async_reduce(\n DataBase.query('user_statistics').where(id=user_id).fetch_one(),\n ident='db_user_statistics:{}'.format(user_id)\n )\n\n return Response(...)\n```\n\nIn that example without using ``async_reduce`` if client performs **N**\nsimultaneous requests like `GET http://web_server/users/42` *web_server*\nperforms **N** requests to *inner-service* and **N** queries to *database*.\nIn total: **N** simultaneous requests emits **2 * N** requests to inner systems.\n\nWith ``async_reduce`` if client performs **N** simultaneous requests *web_server*\nperforms **one** request to *inner-service* and **one** query to *database*.\nIn total: **N** simultaneous requests emit only **2** requests to inner systems.\n\nSee other real [examples](https://github.com/sirkonst/async-reduce/tree/master/examples).\n\n\nSimilar coroutines determination\n--------------------------------\n\n``async_reduce(coroutine)`` tries to detect similar coroutines by hashing\nlocal variables bounded on call. It does not work correctly if:\n\n* one of the arguments is not hashable\n* coroutine function is a method of class with specific state (like ORM)\n* coroutine function has closure to unhashable variable\n\nYou can disable auto-determination by setting custom key to argument ``ident``.\n\n\nUse as decorator\n----------------\n\nAlso library provide special decorator ``@async_reduceable()``, example:\n\n```python\nfrom async_reduce import async_reduceable\n\n\n@async_reduceable()\nasync def fetch_user_data(user_id: int) -> dict:\n \"\"\"\" Get user data from inner service \"\"\"\n url = 'http://inner-servicce/user/{}'.format(user_id)\n\n return await http.get(url, timeout=10).json()\n\n\n@web_server.router('/users/(\\d+)')\nasync def handler_user_detail(request, user_id: int):\n \"\"\" Handler for get detail information about user \"\"\"\n return await fetch_user_data(user_id)\n```\n\n\nHooks\n-----\n\nLibrary supports hooks. Add-on hooks:\n\n* **DebugHooks** - print about all triggered hooks\n* **StatisticsOverallHooks** - general statistics on the use of `async_reduce`\n* **StatisticsDetailHooks** - like `StatisticsOverallHooks` but detail statistics\nabout all `coroutine` processed by `async_reduce`\n\nExample:\n\n```python\nfrom async_reduce import AsyncReducer\nfrom async_reduce.hooks import DebugHooks\n\n# define custom async_reduce with hooks\nasync_reduce = AsyncReducer(hooks=DebugHooks())\n\n\nasync def handler_user_detail(request, user_id: int):\n user_data = await async_reduce(fetch_user_data(user_id))\n```\n\nSee more detail example in [examples/example_hooks.py](https://github.com/sirkonst/async-reduce/blob/master/examples/example_hooks.py).\n\nYou can write custom hooks via inherit from [BaseHooks](https://github.com/sirkonst/async-reduce/blob/master/async_reduce/hooks/base.py).\n\n\nCaveats\n-------\n\n* If single `coroutine` raises exceptions all aggregated `coroutine`s will get\nsame exception too\n\n* If single `coroutine` is stuck all aggregated `coroutine`s will stuck too.\nLimit execution time for `coroutine` and add retries (optional) to avoid it.\n\n* Be careful when return mutable value from `coroutine` because single value\nwill shared. Prefer to use non-mutable value as coroutine return.\n\n\nDevelopment\n-----------\n\nSee [DEVELOPMENT.md](https://github.com/sirkonst/async-reduce/blob/master/DEVELOPMENT.md).\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Reducer for similar simultaneously coroutines",
"version": "1.3",
"project_urls": {
"Homepage": "https://github.com/sirkonst/async-reduce"
},
"split_keywords": [
"asyncio",
"reduce"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "98a9bebbb862a7ed384d6ba4faaf4ac8a0ac777b471f1efb3d2882b7487dd684",
"md5": "e2075d07f28086c585d40a17488d1d8d",
"sha256": "c1fc6d5a9e494c0e3b72ba5fa65b7963c86f76d3fb35ee3d2691a69435ec0521"
},
"downloads": -1,
"filename": "async_reduce-1.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e2075d07f28086c585d40a17488d1d8d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 12231,
"upload_time": "2023-10-04T19:33:19",
"upload_time_iso_8601": "2023-10-04T19:33:19.478914Z",
"url": "https://files.pythonhosted.org/packages/98/a9/bebbb862a7ed384d6ba4faaf4ac8a0ac777b471f1efb3d2882b7487dd684/async_reduce-1.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "39c63b183a249e2085f3c0c6ffe71711933b28879b72de2f5fe08be6ed290cdd",
"md5": "24f3f40320f606e9925b4b3fcd3e1a85",
"sha256": "9f2811d1e13f333d4f7f6801e3993ffe1b6082503e272e28189677431729d3bb"
},
"downloads": -1,
"filename": "async_reduce-1.3.tar.gz",
"has_sig": false,
"md5_digest": "24f3f40320f606e9925b4b3fcd3e1a85",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 12979,
"upload_time": "2023-10-04T19:33:20",
"upload_time_iso_8601": "2023-10-04T19:33:20.866016Z",
"url": "https://files.pythonhosted.org/packages/39/c6/3b183a249e2085f3c0c6ffe71711933b28879b72de2f5fe08be6ed290cdd/async_reduce-1.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-10-04 19:33:20",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "sirkonst",
"github_project": "async-reduce",
"travis_ci": false,
"coveralls": true,
"github_actions": false,
"tox": true,
"lcname": "async-reduce"
}
JSON
