# Simple aiohttp retry client
[](https://codecov.io/gh/inyutin/aiohttp_retry)
Python 3.7 or higher.
**Install**: `pip install aiohttp-retry`.
[](https://www.buymeacoffee.com/inyutin)
### Breaking API changes
- Everything between [2.7.0 - 2.8.3) is yanked.
There is a bug with evaluate_response_callback, it led to infinite retries
- 2.8.0 is incorrect and yanked.
https://github.com/inyutin/aiohttp_retry/issues/79
- Since 2.5.6 this is a new parameter in ```get_timeout``` func called "response".
If you have defined your own ```RetryOptions```, you should add this param into it.
Issue about this: https://github.com/inyutin/aiohttp_retry/issues/59
### Examples of usage:
```python
from aiohttp_retry import RetryClient, ExponentialRetry
async def main():
retry_options = ExponentialRetry(attempts=1)
retry_client = RetryClient(raise_for_status=False, retry_options=retry_options)
async with retry_client.get('https://ya.ru') as response:
print(response.status)
await retry_client.close()
```
```python
from aiohttp import ClientSession
from aiohttp_retry import RetryClient
async def main():
client_session = ClientSession()
retry_client = RetryClient(client_session=client_session)
async with retry_client.get('https://ya.ru') as response:
print(response.status)
await client_session.close()
```
```python
from aiohttp_retry import RetryClient, RandomRetry
async def main():
retry_options = RandomRetry(attempts=1)
retry_client = RetryClient(raise_for_status=False, retry_options=retry_options)
response = await retry_client.get('/ping')
print(response.status)
await retry_client.close()
```
```python
from aiohttp_retry import RetryClient
async def main():
async with RetryClient() as client:
async with client.get('https://ya.ru') as response:
print(response.status)
```
You can change parameters between attempts by passing multiple requests params:
```python
from aiohttp_retry import RetryClient, RequestParams, ExponentialRetry
async def main():
retry_client = RetryClient(raise_for_status=False)
async with retry_client.requests(
params_list=[
RequestParams(
method='GET',
url='https://ya.ru',
),
RequestParams(
method='GET',
url='https://ya.ru',
headers={'some_header': 'some_value'},
),
]
) as response:
print(response.status)
await retry_client.close()
```
You can also add some logic, F.E. logging, on failures by using trace mechanic.
```python
import logging
import sys
from types import SimpleNamespace
from aiohttp import ClientSession, TraceConfig, TraceRequestStartParams
from aiohttp_retry import RetryClient, ExponentialRetry
handler = logging.StreamHandler(sys.stdout)
logging.basicConfig(handlers=[handler])
logger = logging.getLogger(__name__)
retry_options = ExponentialRetry(attempts=2)
async def on_request_start(
session: ClientSession,
trace_config_ctx: SimpleNamespace,
params: TraceRequestStartParams,
) -> None:
current_attempt = trace_config_ctx.trace_request_ctx['current_attempt']
if retry_options.attempts <= current_attempt:
logger.warning('Wow! We are in last attempt')
async def main():
trace_config = TraceConfig()
trace_config.on_request_start.append(on_request_start)
retry_client = RetryClient(retry_options=retry_options, trace_configs=[trace_config])
response = await retry_client.get('https://httpstat.us/503', ssl=False)
print(response.status)
await retry_client.close()
```
Look tests for more examples. \
**Be aware: last request returns as it is.**
**If the last request ended with exception, that this exception will be raised from RetryClient request**
### Documentation
`RetryClient` takes the same arguments as ClientSession[[docs](https://docs.aiohttp.org/en/stable/client_reference.html)] \
`RetryClient` has methods:
- request
- get
- options
- head
- post
- put
- patch
- put
- delete
They are same as for `ClientSession`, but take one possible additional argument:
```python
class RetryOptionsBase:
def __init__(
self,
attempts: int = 3, # How many times we should retry
statuses: Optional[Iterable[int]] = None, # On which statuses we should retry
exceptions: Optional[Iterable[Type[Exception]]] = None, # On which exceptions we should retry
retry_all_server_errors: bool = True, # If should retry all 500 errors or not
# a callback that will run on response to decide if retry
evaluate_response_callback: Optional[EvaluateResponseCallbackType] = None,
):
...
@abc.abstractmethod
def get_timeout(self, attempt: int, response: Optional[Response] = None) -> float:
raise NotImplementedError
```
You can specify `RetryOptions` both for `RetryClient` and it's methods.
`RetryOptions` in methods override `RetryOptions` defined in `RetryClient` constructor.
**Important**: by default all 5xx responses are retried + statuses you specified as ```statuses``` param
If you will pass ```retry_all_server_errors=False``` than you can manually set what 5xx errors to retry.
You can define your own timeouts logic or use:
- ```ExponentialRetry``` with exponential backoff
- ```RandomRetry``` for random backoff
- ```ListRetry``` with backoff you predefine by list
- ```FibonacciRetry``` with backoff that looks like fibonacci sequence
- ```JitterRetry``` exponential retry with a bit of randomness
**Important**: you can proceed server response as an parameter for calculating next timeout.
However this response can be None, server didn't make a response or you have set up ```raise_for_status=True```
Look here for an example: https://github.com/inyutin/aiohttp_retry/issues/59
Additionally, you can specify ```evaluate_response_callback```. It receive a ```ClientResponse``` and decide to retry or not by returning a bool.
It can be useful, if server API sometimes response with malformed data.
#### Request Trace Context
`RetryClient` add *current attempt number* to `request_trace_ctx` (see examples,
for more info see [aiohttp doc](https://docs.aiohttp.org/en/stable/client_advanced.html#aiohttp-client-tracing)).
### Change parameters between retries
`RetryClient` also has a method called `requests`. This method should be used if you want to make requests with different params.
```python
@dataclass
class RequestParams:
method: str
url: _RAW_URL_TYPE
trace_request_ctx: Optional[Dict[str, Any]] = None
kwargs: Optional[Dict[str, Any]] = None
```
```python
def requests(
self,
params_list: List[RequestParams],
retry_options: Optional[RetryOptionsBase] = None,
raise_for_status: Optional[bool] = None,
) -> _RequestContext:
```
You can find an example of usage above or in tests.
But basically `RequestParams` is a structure to define params for `ClientSession.request` func.
`method`, `url`, `headers` `trace_request_ctx` defined outside kwargs, because they are popular.
There is also an old way to change URL between retries by specifying ```url``` as list of urls. Example:
```python
from aiohttp_retry import RetryClient
retry_client = RetryClient()
async with retry_client.get(url=['/internal_error', '/ping']) as response:
text = await response.text()
assert response.status == 200
assert text == 'Ok!'
await retry_client.close()
```
In this example we request ```/interval_error```, fail and then successfully request ```/ping```.
If you specify less urls than ```attempts``` number in ```RetryOptions```, ```RetryClient``` will request last url at last attempts.
This means that in example above we would request ```/ping``` once again in case of failure.
### Types
`aiohttp_retry` is a typed project. It should be fully compatablie with mypy.
It also introduce one special type:
```
ClientType = Union[ClientSession, RetryClient]
```
This type can be imported by ```from aiohttp_retry.types import ClientType```
Raw data
{
"_id": null,
"home_page": "https://github.com/inyutin/aiohttp_retry",
"name": "aiohttp-retry",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "aiohttp retry client",
"author": "Dmitry Inyutin",
"author_email": "inyutin.da@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/01/c1/d57818a0ed5b0313ad8c620638225ddd44094d0d606ee33f3df5105572cd/aiohttp_retry-2.8.3.tar.gz",
"platform": "any",
"description": "# Simple aiohttp retry client\n\n[](https://codecov.io/gh/inyutin/aiohttp_retry)\n\nPython 3.7 or higher.\n\n**Install**: `pip install aiohttp-retry`.\n\n[](https://www.buymeacoffee.com/inyutin)\n\n\n### Breaking API changes\n- Everything between [2.7.0 - 2.8.3) is yanked. \nThere is a bug with evaluate_response_callback, it led to infinite retries\n\n- 2.8.0 is incorrect and yanked.\nhttps://github.com/inyutin/aiohttp_retry/issues/79\n\n- Since 2.5.6 this is a new parameter in ```get_timeout``` func called \"response\". \nIf you have defined your own ```RetryOptions```, you should add this param into it.\nIssue about this: https://github.com/inyutin/aiohttp_retry/issues/59\n\n### Examples of usage:\n```python\nfrom aiohttp_retry import RetryClient, ExponentialRetry\n\nasync def main():\n retry_options = ExponentialRetry(attempts=1)\n retry_client = RetryClient(raise_for_status=False, retry_options=retry_options)\n async with retry_client.get('https://ya.ru') as response:\n print(response.status)\n \n await retry_client.close()\n```\n\n```python\nfrom aiohttp import ClientSession\nfrom aiohttp_retry import RetryClient \n\nasync def main():\n client_session = ClientSession()\n retry_client = RetryClient(client_session=client_session)\n async with retry_client.get('https://ya.ru') as response:\n print(response.status)\n\n await client_session.close()\n```\n\n```python\nfrom aiohttp_retry import RetryClient, RandomRetry\n\nasync def main():\n retry_options = RandomRetry(attempts=1)\n retry_client = RetryClient(raise_for_status=False, retry_options=retry_options)\n\n response = await retry_client.get('/ping')\n print(response.status)\n \n await retry_client.close()\n```\n\n```python\nfrom aiohttp_retry import RetryClient\n\nasync def main():\n async with RetryClient() as client:\n async with client.get('https://ya.ru') as response:\n print(response.status)\n```\n\nYou can change parameters between attempts by passing multiple requests params:\n```python\nfrom aiohttp_retry import RetryClient, RequestParams, ExponentialRetry\n\nasync def main():\n retry_client = RetryClient(raise_for_status=False)\n\n async with retry_client.requests(\n params_list=[\n RequestParams(\n method='GET',\n url='https://ya.ru',\n ),\n RequestParams(\n method='GET',\n url='https://ya.ru',\n headers={'some_header': 'some_value'},\n ),\n ]\n ) as response:\n print(response.status)\n \n await retry_client.close()\n```\n\nYou can also add some logic, F.E. logging, on failures by using trace mechanic.\n```python\nimport logging\nimport sys\nfrom types import SimpleNamespace\n\nfrom aiohttp import ClientSession, TraceConfig, TraceRequestStartParams\n\nfrom aiohttp_retry import RetryClient, ExponentialRetry\n\n\nhandler = logging.StreamHandler(sys.stdout)\nlogging.basicConfig(handlers=[handler])\nlogger = logging.getLogger(__name__)\nretry_options = ExponentialRetry(attempts=2)\n\n\nasync def on_request_start(\n session: ClientSession,\n trace_config_ctx: SimpleNamespace,\n params: TraceRequestStartParams,\n) -> None:\n current_attempt = trace_config_ctx.trace_request_ctx['current_attempt']\n if retry_options.attempts <= current_attempt:\n logger.warning('Wow! We are in last attempt')\n\n\nasync def main():\n trace_config = TraceConfig()\n trace_config.on_request_start.append(on_request_start)\n retry_client = RetryClient(retry_options=retry_options, trace_configs=[trace_config])\n\n response = await retry_client.get('https://httpstat.us/503', ssl=False)\n print(response.status)\n\n await retry_client.close()\n```\nLook tests for more examples. \\\n**Be aware: last request returns as it is.** \n**If the last request ended with exception, that this exception will be raised from RetryClient request**\n\n### Documentation\n`RetryClient` takes the same arguments as ClientSession[[docs](https://docs.aiohttp.org/en/stable/client_reference.html)] \\\n`RetryClient` has methods:\n- request\n- get\n- options\n- head\n- post\n- put\n- patch\n- put\n- delete\n\nThey are same as for `ClientSession`, but take one possible additional argument: \n```python\nclass RetryOptionsBase:\n def __init__(\n self,\n attempts: int = 3, # How many times we should retry\n statuses: Optional[Iterable[int]] = None, # On which statuses we should retry\n exceptions: Optional[Iterable[Type[Exception]]] = None, # On which exceptions we should retry\n retry_all_server_errors: bool = True, # If should retry all 500 errors or not\n # a callback that will run on response to decide if retry\n evaluate_response_callback: Optional[EvaluateResponseCallbackType] = None,\n ):\n ...\n\n @abc.abstractmethod\n def get_timeout(self, attempt: int, response: Optional[Response] = None) -> float:\n raise NotImplementedError\n\n```\nYou can specify `RetryOptions` both for `RetryClient` and it's methods. \n`RetryOptions` in methods override `RetryOptions` defined in `RetryClient` constructor.\n\n**Important**: by default all 5xx responses are retried + statuses you specified as ```statuses``` param\nIf you will pass ```retry_all_server_errors=False``` than you can manually set what 5xx errors to retry.\n\nYou can define your own timeouts logic or use: \n- ```ExponentialRetry``` with exponential backoff\n- ```RandomRetry``` for random backoff\n- ```ListRetry``` with backoff you predefine by list\n- ```FibonacciRetry``` with backoff that looks like fibonacci sequence\n- ```JitterRetry``` exponential retry with a bit of randomness\n\n**Important**: you can proceed server response as an parameter for calculating next timeout. \nHowever this response can be None, server didn't make a response or you have set up ```raise_for_status=True```\nLook here for an example: https://github.com/inyutin/aiohttp_retry/issues/59\n\nAdditionally, you can specify ```evaluate_response_callback```. It receive a ```ClientResponse``` and decide to retry or not by returning a bool.\nIt can be useful, if server API sometimes response with malformed data.\n\n#### Request Trace Context\n`RetryClient` add *current attempt number* to `request_trace_ctx` (see examples, \nfor more info see [aiohttp doc](https://docs.aiohttp.org/en/stable/client_advanced.html#aiohttp-client-tracing)).\n\n### Change parameters between retries\n`RetryClient` also has a method called `requests`. This method should be used if you want to make requests with different params.\n```python\n@dataclass\nclass RequestParams:\n method: str\n url: _RAW_URL_TYPE\n trace_request_ctx: Optional[Dict[str, Any]] = None\n kwargs: Optional[Dict[str, Any]] = None\n```\n\n```python\ndef requests(\n self,\n params_list: List[RequestParams],\n retry_options: Optional[RetryOptionsBase] = None,\n raise_for_status: Optional[bool] = None,\n) -> _RequestContext:\n```\n\nYou can find an example of usage above or in tests. \nBut basically `RequestParams` is a structure to define params for `ClientSession.request` func. \n`method`, `url`, `headers` `trace_request_ctx` defined outside kwargs, because they are popular. \n\nThere is also an old way to change URL between retries by specifying ```url``` as list of urls. Example:\n```python\nfrom aiohttp_retry import RetryClient\n\nretry_client = RetryClient()\nasync with retry_client.get(url=['/internal_error', '/ping']) as response:\n text = await response.text()\n assert response.status == 200\n assert text == 'Ok!'\n\nawait retry_client.close()\n```\n\nIn this example we request ```/interval_error```, fail and then successfully request ```/ping```.\nIf you specify less urls than ```attempts``` number in ```RetryOptions```, ```RetryClient``` will request last url at last attempts.\nThis means that in example above we would request ```/ping``` once again in case of failure.\n\n### Types\n\n`aiohttp_retry` is a typed project. It should be fully compatablie with mypy.\n\nIt also introduce one special type:\n```\nClientType = Union[ClientSession, RetryClient]\n```\n\nThis type can be imported by ```from aiohttp_retry.types import ClientType```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Simple retry client for aiohttp",
"version": "2.8.3",
"project_urls": {
"Homepage": "https://github.com/inyutin/aiohttp_retry"
},
"split_keywords": [
"aiohttp",
"retry",
"client"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5757af573003eca6248a5cbc538fc46bea5b249c2ac86f27140b0a621bcd3fde",
"md5": "ff2137444c4d54f3b8f75a282ff7c06a",
"sha256": "3aeeead8f6afe48272db93ced9440cf4eda8b6fd7ee2abb25357b7eb28525b45"
},
"downloads": -1,
"filename": "aiohttp_retry-2.8.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ff2137444c4d54f3b8f75a282ff7c06a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 9773,
"upload_time": "2022-08-11T21:13:40",
"upload_time_iso_8601": "2022-08-11T21:13:40.602435Z",
"url": "https://files.pythonhosted.org/packages/57/57/af573003eca6248a5cbc538fc46bea5b249c2ac86f27140b0a621bcd3fde/aiohttp_retry-2.8.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "01c1d57818a0ed5b0313ad8c620638225ddd44094d0d606ee33f3df5105572cd",
"md5": "bc17db5d56a609515765dcfae616758a",
"sha256": "9a8e637e31682ad36e1ff9f8bcba912fcfc7d7041722bc901a4b948da4d71ea9"
},
"downloads": -1,
"filename": "aiohttp_retry-2.8.3.tar.gz",
"has_sig": false,
"md5_digest": "bc17db5d56a609515765dcfae616758a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 10585,
"upload_time": "2022-08-11T21:13:41",
"upload_time_iso_8601": "2022-08-11T21:13:41.945324Z",
"url": "https://files.pythonhosted.org/packages/01/c1/d57818a0ed5b0313ad8c620638225ddd44094d0d606ee33f3df5105572cd/aiohttp_retry-2.8.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-08-11 21:13:41",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "inyutin",
"github_project": "aiohttp_retry",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "aiohttp-retry"
}
JSON
