---------------------------
Rever, A retrying decorator
---------------------------
A retry decorator can be useful in many situations. One example is when scraping web pages.
Suppose you have a function that retrieves the status code response of a GET request. If the status
code returns 200, then you are happy. But if not, then there here is what you might do:
1) You could write your retrying logic directly into your functions
>>> def get_response(webpage):
>>> response = function_to_get_webpage(webpage)
>>> status_code = function_to_get_status_code(response)
>>> if status_code == 200:
>>> return status_code
>>> else:
>>> time.sleep(3)
>>> num_tries -= 1
>>> if num_tries > 0:
>>> return get_response(webpage)
>>>
>>> if __name__ == "__main__":
>>> num_tries = 2
>>> get_response("http://www.google.com")
2) You could use a retrying decorator like rever
>>> from rever import rever
>>> @rever(times=2, pause=3, exception=MyException, raises=False)
>>> def get_response(webpage):
>>> response = function_to_get_webpage(webpage)
>>> status_code = function_to_get_status_code(response)
>>> if status_code == 200:
>>> return status_code
>>> else:
>>> raise MyException
>>>
>>> if __name__ == "__main__":
>>> get_response("http://www.google.com")
In the first example, you need to write out the retrying logic yourself. The second
example it is taken care of in the decorator; a nice way of keeping things separate.
Installation
------------
::
$ pip install rever
Keyword Arguments
-----------------
The rever decorator takes only keyword arguments.
Possible keyword arguments:
*backoff*:
description: if True subsequent pauses for each retry will increase exponentially
possible values: boolean
*total_pause*:
description: the total time you are willing to wait for all of your pauses between retrys
possible values: integer or float
*steps*:
description: related to backoff and is set at 10 because wikipedia says so: https://en.wikipedia.org/wiki/Exponential_backoff
possible values: integer
*exception*:
description: you can choose which exception or exceptions to catch
possible values: any Exception that gets raised by Python
*raises*:
description: if all the retrys fail, do you want to raise an exception or not?
possible values: boolean
*prior*:
description: if you want to call another function/script prior to retrying, you can do so but without any args or kwargs
possible values: a simple function...cannot take args or kwargs
**These arguments are used if *backoff* is set to False**:
*times*:
description: the number of times you want the function to retry
possible values: integer
*pause*:
description: the number of seconds you want to pause before your function retrys
possible values: integer or float
Examples & Explanation
----------------------
**default**
Default behavior
>>> @rever()
- rever will use exponential backoff
- rever will have a total pause time of 30 seconds (total time your function will pause)
- rever will have 10 steps (steps here means the number of times your function will retry)
- rever will catch any exception
- rever will ultimately raise an exception if all retrys fail
**exception**
Catch one specific exception
>>> @rever(exception=TypeError)
>>> @rever(exception=(TypeError, ))
- rever will use exponential backoff
- rever will have a total pause time of 30 seconds (total time your function will pause)
- rever will have 10 steps (steps here means the number of times your function will retry)
- rever will catch only *TypeError*
- rever will ultimately raise an exception if all retrys fail
Catch one of multiple specific exceptions
>>> @rever(exception=(TypeError, ConnectionError))
- rever will use exponential backoff
- rever will have a total pause time of 30 seconds (total time your function will pause)
- rever will have 10 steps (steps here means the number of times your function will retry)
- rever will catch any of only *TypeError* or *ConnectionError*
- rever will ultimately raise an exception if all retrys fail
raises
Raise an exception or do not
>>> @rever(raises=False)
- rever will use exponential backoff
- rever will have a total pause time of 30 seconds (total time your function will pause)
- rever will have 10 steps (steps here means the number of times your function will retry)
- rever will catch any exception
- rever will ultimately *not* raise an exception if all retrys fail
prior
Call a function prior to retrying
>>> @rever(prior=some_function_to_call_prior_to_retyring)
- rever will use exponential backoff
- rever will have a total pause time of 30 seconds (total time your function will pause)
- rever will have 10 steps (steps here means the number of times your function will retry)
- rever will catch any exception
- rever will ultimately raise an exception if all retrys fail
- *rever will call some function prior to each retry*
**Below used only if backoff is set to False, it is included for backwards compatibility**
times
Retry a certain number of times
>>> @rever(backoff=False, times=10)
- rever will *not* use exponential backoff
- rever will have a total pause time of *0* seconds (total time your function will pause)
- rever will retry *1* time (time here means the number of times your function will retry)
- rever will catch any exception
- rever will ultimately raise an exception if all retrys fail
pause
Pause for some number of seconds between each retry
>>> @rever(backoff=False, pause=5)
- rever will *not* use exponential backoff
- rever will have a total pause time of *5* seconds (total time your function will pause)
- rever will retry *1* time (time here means the number of times your function will retry)
- rever will catch any exception
- rever will ultimately raise an exception if all retrys fail
You can basically use any combination of keywords you would like
Testing
-------
To run tests, clone the github repository:
$ git clone https://github.com/liamcryan/rever
$ cd rever
$ pip install pytest
$ pytest
-------
History
-------
version 0.3.3 (5/12/2020)
-------------------------
- fix bad link in __version__
version 0.3.2 (5/12/2020)
-------------------------
- fix non-local rever_kwargs
- change structure from multiple to files to one __init__ file
- update README.rst with installation instructions
version 0.3.1 (7/13/2018)
-------------------------
- found bug when calling the same decorated function multiple times. In certain cases the 'times' keyword argument decreased to 0 triggering a ReachedMaxRetries exception despite the function only being called once
version 0.3.0 (8/23/2017)
-------------------------
- wanted to modify behaviour to exponential backoff as default rather than fixed intervals between retrys
- to replicate functionality of previous versions include a kwarg backoff=False in your decorator
version 0.2.1 (8/8/2017)
------------------------
- realized that any function wanting to return any value would return None, so fixed that :)
version 0.2.0 (6/26/2017)
-------------------------
- specify a function to call prior to retrying
- realized that the retry count was off by 1, now it should be correct
version 0.1.0 (6/24/2017)
-------------------------
- specify whether to raise exception after all retry attempts
- included some testing
- default pause is now zero seconds
version 0.0.1 (6/23/2017)
-------------------------
- retry decorator
- specify number of times to retry
- specify number of seconds to wait
- specify which exceptions to catch
Raw data
{
"_id": null,
"home_page": "http://github.com/liamcryan/rever",
"name": "rever",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "retry decorator rever",
"author": "Liam Cryan",
"author_email": "cryan.liam@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/ba/c7/b3682f108bebcb455f2c56240a03e725d956c897ee947eca9f8257913cdf/rever-0.3.3.tar.gz",
"platform": "",
"description": "---------------------------\nRever, A retrying decorator\n---------------------------\n\nA retry decorator can be useful in many situations. One example is when scraping web pages.\nSuppose you have a function that retrieves the status code response of a GET request. If the status\ncode returns 200, then you are happy. But if not, then there here is what you might do:\n\n1) You could write your retrying logic directly into your functions\n\n >>> def get_response(webpage):\n >>> response = function_to_get_webpage(webpage)\n >>> status_code = function_to_get_status_code(response)\n >>> if status_code == 200:\n >>> return status_code\n >>> else:\n >>> time.sleep(3)\n >>> num_tries -= 1\n >>> if num_tries > 0:\n >>> return get_response(webpage)\n >>>\n >>> if __name__ == \"__main__\":\n >>> num_tries = 2\n >>> get_response(\"http://www.google.com\")\n\n2) You could use a retrying decorator like rever\n\n >>> from rever import rever\n >>> @rever(times=2, pause=3, exception=MyException, raises=False)\n >>> def get_response(webpage):\n >>> response = function_to_get_webpage(webpage)\n >>> status_code = function_to_get_status_code(response)\n >>> if status_code == 200:\n >>> return status_code\n >>> else:\n >>> raise MyException\n >>>\n >>> if __name__ == \"__main__\":\n >>> get_response(\"http://www.google.com\")\n\n\nIn the first example, you need to write out the retrying logic yourself. The second\nexample it is taken care of in the decorator; a nice way of keeping things separate.\n\nInstallation\n------------\n\n::\n\n $ pip install rever\n\n\n\nKeyword Arguments\n-----------------\n\nThe rever decorator takes only keyword arguments.\n\nPossible keyword arguments:\n\n*backoff*:\n\n description: if True subsequent pauses for each retry will increase exponentially\n\n possible values: boolean\n\n*total_pause*:\n\n description: the total time you are willing to wait for all of your pauses between retrys\n\n possible values: integer or float\n\n*steps*:\n\n description: related to backoff and is set at 10 because wikipedia says so: https://en.wikipedia.org/wiki/Exponential_backoff\n\n possible values: integer\n\n*exception*:\n\n description: you can choose which exception or exceptions to catch\n\n possible values: any Exception that gets raised by Python\n\n*raises*:\n\n description: if all the retrys fail, do you want to raise an exception or not?\n\n possible values: boolean\n\n*prior*:\n\n description: if you want to call another function/script prior to retrying, you can do so but without any args or kwargs\n\n possible values: a simple function...cannot take args or kwargs\n\n**These arguments are used if *backoff* is set to False**:\n\n*times*:\n\n description: the number of times you want the function to retry\n\n possible values: integer\n\n*pause*:\n\n description: the number of seconds you want to pause before your function retrys\n\n possible values: integer or float\n\n\nExamples & Explanation\n----------------------\n\n**default**\n Default behavior\n\n >>> @rever()\n\n - rever will use exponential backoff\n - rever will have a total pause time of 30 seconds (total time your function will pause)\n - rever will have 10 steps (steps here means the number of times your function will retry)\n - rever will catch any exception\n - rever will ultimately raise an exception if all retrys fail\n\n**exception**\n Catch one specific exception\n\n >>> @rever(exception=TypeError)\n >>> @rever(exception=(TypeError, ))\n\n - rever will use exponential backoff\n - rever will have a total pause time of 30 seconds (total time your function will pause)\n - rever will have 10 steps (steps here means the number of times your function will retry)\n - rever will catch only *TypeError*\n - rever will ultimately raise an exception if all retrys fail\n\n Catch one of multiple specific exceptions\n\n >>> @rever(exception=(TypeError, ConnectionError))\n\n - rever will use exponential backoff\n - rever will have a total pause time of 30 seconds (total time your function will pause)\n - rever will have 10 steps (steps here means the number of times your function will retry)\n - rever will catch any of only *TypeError* or *ConnectionError*\n - rever will ultimately raise an exception if all retrys fail\n\nraises\n Raise an exception or do not\n\n >>> @rever(raises=False)\n\n - rever will use exponential backoff\n - rever will have a total pause time of 30 seconds (total time your function will pause)\n - rever will have 10 steps (steps here means the number of times your function will retry)\n - rever will catch any exception\n - rever will ultimately *not* raise an exception if all retrys fail\n\nprior\n Call a function prior to retrying\n\n >>> @rever(prior=some_function_to_call_prior_to_retyring)\n\n - rever will use exponential backoff\n - rever will have a total pause time of 30 seconds (total time your function will pause)\n - rever will have 10 steps (steps here means the number of times your function will retry)\n - rever will catch any exception\n - rever will ultimately raise an exception if all retrys fail\n - *rever will call some function prior to each retry*\n\n**Below used only if backoff is set to False, it is included for backwards compatibility**\n\ntimes\n Retry a certain number of times\n\n >>> @rever(backoff=False, times=10)\n\n - rever will *not* use exponential backoff\n - rever will have a total pause time of *0* seconds (total time your function will pause)\n - rever will retry *1* time (time here means the number of times your function will retry)\n - rever will catch any exception\n - rever will ultimately raise an exception if all retrys fail\n\npause\n Pause for some number of seconds between each retry\n\n >>> @rever(backoff=False, pause=5)\n\n - rever will *not* use exponential backoff\n - rever will have a total pause time of *5* seconds (total time your function will pause)\n - rever will retry *1* time (time here means the number of times your function will retry)\n - rever will catch any exception\n - rever will ultimately raise an exception if all retrys fail\n\n\nYou can basically use any combination of keywords you would like\n\nTesting\n-------\n\nTo run tests, clone the github repository:\n\n $ git clone https://github.com/liamcryan/rever\n $ cd rever\n $ pip install pytest\n $ pytest\n\n\n-------\nHistory\n-------\n\nversion 0.3.3 (5/12/2020)\n-------------------------\n\n- fix bad link in __version__\n\nversion 0.3.2 (5/12/2020)\n-------------------------\n\n- fix non-local rever_kwargs\n- change structure from multiple to files to one __init__ file\n- update README.rst with installation instructions\n\nversion 0.3.1 (7/13/2018)\n-------------------------\n\n- found bug when calling the same decorated function multiple times. In certain cases the 'times' keyword argument decreased to 0 triggering a ReachedMaxRetries exception despite the function only being called once\n\nversion 0.3.0 (8/23/2017)\n-------------------------\n\n- wanted to modify behaviour to exponential backoff as default rather than fixed intervals between retrys\n- to replicate functionality of previous versions include a kwarg backoff=False in your decorator\n\nversion 0.2.1 (8/8/2017)\n------------------------\n\n- realized that any function wanting to return any value would return None, so fixed that :)\n\nversion 0.2.0 (6/26/2017)\n-------------------------\n\n- specify a function to call prior to retrying\n- realized that the retry count was off by 1, now it should be correct\n\nversion 0.1.0 (6/24/2017)\n-------------------------\n\n- specify whether to raise exception after all retry attempts\n- included some testing\n- default pause is now zero seconds\n\nversion 0.0.1 (6/23/2017)\n-------------------------\n\n- retry decorator\n- specify number of times to retry\n- specify number of seconds to wait\n- specify which exceptions to catch\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Rever, a retrying decorator",
"version": "0.3.3",
"project_urls": {
"Homepage": "http://github.com/liamcryan/rever"
},
"split_keywords": [
"retry",
"decorator",
"rever"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "90b18253594d0027f74e6735540d8691c9b253e45e6259bda29e8f493772c05b",
"md5": "2fa0bb72195ee7b10e44f3232209f964",
"sha256": "2b69bd02fab2b57cb29cd31c017a56d7546ea1a11896c327ca44d8693236b4b8"
},
"downloads": -1,
"filename": "rever-0.3.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2fa0bb72195ee7b10e44f3232209f964",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 5315,
"upload_time": "2020-05-12T16:55:39",
"upload_time_iso_8601": "2020-05-12T16:55:39.610228Z",
"url": "https://files.pythonhosted.org/packages/90/b1/8253594d0027f74e6735540d8691c9b253e45e6259bda29e8f493772c05b/rever-0.3.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "bac7b3682f108bebcb455f2c56240a03e725d956c897ee947eca9f8257913cdf",
"md5": "48a177107c89cf6242ea54a09f2b8160",
"sha256": "958559c8e2d9cdecb26e4184642f7a65e5c02051a1629d07d6b3ef0fc731539c"
},
"downloads": -1,
"filename": "rever-0.3.3.tar.gz",
"has_sig": false,
"md5_digest": "48a177107c89cf6242ea54a09f2b8160",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 5525,
"upload_time": "2020-05-12T16:55:40",
"upload_time_iso_8601": "2020-05-12T16:55:40.880154Z",
"url": "https://files.pythonhosted.org/packages/ba/c7/b3682f108bebcb455f2c56240a03e725d956c897ee947eca9f8257913cdf/rever-0.3.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2020-05-12 16:55:40",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "liamcryan",
"github_project": "rever",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "pytest",
"specs": []
}
],
"lcname": "rever"
}
JSON
