| Name | pybreaker JSON |
| Version |
1.2.0
JSON |
| download |
| home_page | |
| Summary | Python implementation of the Circuit Breaker pattern |
| upload_time | 2024-02-14 22:16:04 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.8 |
| license | |
| keywords |
design
pattern
circuit
breaker
integration
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
PyBreaker
=========
PyBreaker is a Python implementation of the Circuit Breaker pattern, described
in Michael T. Nygard's book `Release It!`_.
In Nygard's words, *"circuit breakers exists to allow one subsystem to fail
without destroying the entire system. This is done by wrapping dangerous
operations (typically integration points) with a component that can circumvent
calls when the system is not healthy"*.
Features
--------
* Configurable list of excluded exceptions (e.g. business exceptions)
* Configurable failure threshold and reset timeout
* Support for several event listeners per circuit breaker
* Can guard generator functions
* Functions and properties for easy monitoring and management
* Thread-safe
* Optional redis backing
* Optional support for asynchronous Tornado calls
Requirements
------------
* `Python`_ 3.8+
Installation
------------
Run the following command line to download the latest stable version of
PyBreaker from `PyPI`_::
$ pip install pybreaker
If you are a `Git`_ user, you might want to install the current development
version in editable mode::
$ git clone git://github.com/danielfm/pybreaker.git
$ cd pybreaker
$ # run tests (on windows omit ./)
$ ./pw test
$ pip install -e .
Usage
-----
The first step is to create an instance of ``CircuitBreaker`` for each
integration point you want to protect against:
.. code:: python
import pybreaker
# Used in database integration points
db_breaker = pybreaker.CircuitBreaker(fail_max=5, reset_timeout=60)
``CircuitBreaker`` instances should live globally inside the application scope,
e.g., live across requests.
.. note::
Integration points to external services (i.e. databases, queues, etc) are
more likely to fail, so make sure to always use timeouts when accessing such
services if there's support at the API level.
If you'd like to use the Redis backing, initialize the ``CircuitBreaker`` with
a ``CircuitRedisStorage``:
.. code:: python
import pybreaker
import redis
redis = redis.StrictRedis()
db_breaker = pybreaker.CircuitBreaker(
fail_max=5,
reset_timeout=60,
state_storage=pybreaker.CircuitRedisStorage(pybreaker.STATE_CLOSED, redis))
**Do not** initialize the Redis connection with the ``decode_responses`` set to
``True``, this will force returning ASCII objects from redis and in Python3+ will
fail with:
`AttributeError: 'str' object has no attribute 'decode'`
.. note::
You may want to reuse a connection already created in your application, if you're
using ``django_redis`` for example:
.. code:: python
import pybreaker
from django_redis import get_redis_connection
db_breaker = pybreaker.CircuitBreaker(
fail_max=5,
reset_timeout=60,
state_storage=pybreaker.CircuitRedisStorage(pybreaker.STATE_CLOSED, get_redis_connection('default')))
.. note::
If you require multiple, independent CircuitBreakers and wish to store their states in Redis, it is essential to assign a ``unique namespace`` for each
CircuitBreaker instance. This can be achieved by specifying a distinct namespace parameter in the CircuitRedisStorage constructor. for example:
.. code:: python
import pybreaker
from django_redis import get_redis_connection
db_breaker = pybreaker.CircuitBreaker(
fail_max=5,
reset_timeout=60,
state_storage=pybreaker.CircuitRedisStorage(pybreaker.STATE_CLOSED, get_redis_connection('default'),namespace='unique_namespace'))
Event Listening
```````````````
There's no need to subclass ``CircuitBreaker`` if you just want to take action
when certain events occur. In that case, it's better to subclass
``CircuitBreakerListener`` instead:
.. code:: python
class DBListener(pybreaker.CircuitBreakerListener):
"Listener used by circuit breakers that execute database operations."
def before_call(self, cb, func, *args, **kwargs):
"Called before the circuit breaker `cb` calls `func`."
pass
def state_change(self, cb, old_state, new_state):
"Called when the circuit breaker `cb` state changes."
pass
def failure(self, cb, exc):
"Called when a function invocation raises a system error."
pass
def success(self, cb):
"Called when a function invocation succeeds."
pass
class LogListener(pybreaker.CircuitBreakerListener):
"Listener used to log circuit breaker events."
def state_change(self, cb, old_state, new_state):
msg = "State Change: CB: {0}, New State: {1}".format(cb.name, new_state)
logging.info(msg)
To add listeners to a circuit breaker:
.. code:: python
# At creation time...
db_breaker = pybreaker.CircuitBreaker(listeners=[DBListener(), LogListener()])
# ...or later
db_breaker.add_listeners(OneListener(), AnotherListener())
What Does a Circuit Breaker Do?
```````````````````````````````
Let's say you want to use a circuit breaker on a function that updates a row
in the ``customer`` database table:
.. code:: python
@db_breaker
def update_customer(cust):
# Do stuff here...
pass
# Will trigger the circuit breaker
updated_customer = update_customer(my_customer)
Or if you don't want to use the decorator syntax:
.. code:: python
def update_customer(cust):
# Do stuff here...
pass
# Will trigger the circuit breaker
updated_customer = db_breaker.call(update_customer, my_customer)
Or use it as a context manager and a `with` statement:
.. code:: python
# Will trigger the circuit breaker
with db_breaker.calling():
# Do stuff here...
pass
According to the default parameters, the circuit breaker ``db_breaker`` will
automatically open the circuit after 5 consecutive failures in
``update_customer``.
When the circuit is open, all calls to ``update_customer`` will fail immediately
(raising ``CircuitBreakerError``) without any attempt to execute the real
operation. If you want the original error to be thrown when the circuit trips,
set the ``throw_new_error_on_trip`` option to ``False``:
.. code:: python
pybreaker.CircuitBreaker(..., throw_new_error_on_trip=False)
After 60 seconds, the circuit breaker will allow the next call to
``update_customer`` pass through. If that call succeeds, the circuit is closed;
if it fails, however, the circuit is opened again until another timeout elapses.
Optional Tornado Support
````````````````````````
A circuit breaker can (optionally) be used to call asynchronous Tornado functions:
.. code:: python
from tornado import gen
@db_breaker(__pybreaker_call_async=True)
@gen.coroutine
def async_update(cust):
# Do async stuff here...
pass
Or if you don't want to use the decorator syntax:
.. code:: python
@gen.coroutine
def async_update(cust):
# Do async stuff here...
pass
updated_customer = db_breaker.call_async(async_update, my_customer)
Excluding Exceptions
````````````````````
By default, a failed call is any call that raises an exception. However, it's
common to raise exceptions to also indicate business exceptions, and those
exceptions should be ignored by the circuit breaker as they don't indicate
system errors:
.. code:: python
# At creation time...
db_breaker = CircuitBreaker(exclude=[CustomerValidationError])
# ...or later
db_breaker.add_excluded_exception(CustomerValidationError)
In that case, when any function guarded by that circuit breaker raises
``CustomerValidationError`` (or any exception derived from
``CustomerValidationError``), that call won't be considered a system failure.
So as to cover cases where the exception class alone is not enough to determine
whether it represents a system error, you may also pass a callable rather than
a type:
.. code:: python
db_breaker = CircuitBreaker(exclude=[lambda e: type(e) == HTTPError and e.status_code < 500])
You may mix types and filter callables freely.
Monitoring and Management
`````````````````````````
A circuit breaker provides properties and functions you can use to monitor and
change its current state:
.. code:: python
# Get the current number of consecutive failures
print(db_breaker.fail_counter)
# Get/set the maximum number of consecutive failures
print(db_breaker.fail_max)
db_breaker.fail_max = 10
# Get/set the current reset timeout period (in seconds)
print db_breaker.reset_timeout
db_breaker.reset_timeout = 60
# Get the current state, i.e., 'open', 'half-open', 'closed'
print(db_breaker.current_state)
# Closes the circuit
db_breaker.close()
# Half-opens the circuit
db_breaker.half_open()
# Opens the circuit
db_breaker.open()
These properties and functions might and should be exposed to the operations
staff somehow as they help them to detect problems in the system.
Contributing
-------------
Run tests::
$ ./pw test
Code formatting (black and isort) and linting (mypy) ::
$ ./pw format
$ ./pw lint
Above commands will automatically install the necessary tools inside *.pyprojectx*
and also install pre-commit hooks.
List available commands::
$ ./pw -i
.. _Python: http://python.org
.. _Jython: http://jython.org
.. _Release It!: https://pragprog.com/titles/mnee2/release-it-second-edition/
.. _PyPI: http://pypi.python.org
.. _Git: http://git-scm.com
Raw data
{
"_id": null,
"home_page": "",
"name": "pybreaker",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "design,pattern,circuit,breaker,integration",
"author": "",
"author_email": "Daniel Fernandes Martins <daniel.tritone@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/f6/9b/675a7cad98bb19a131b9aaa5a1a876607071c64a5e64c9656f0156cec53a/pybreaker-1.2.0.tar.gz",
"platform": null,
"description": "\nPyBreaker\n=========\n\nPyBreaker is a Python implementation of the Circuit Breaker pattern, described\nin Michael T. Nygard's book `Release It!`_.\n\nIn Nygard's words, *\"circuit breakers exists to allow one subsystem to fail\nwithout destroying the entire system. This is done by wrapping dangerous\noperations (typically integration points) with a component that can circumvent\ncalls when the system is not healthy\"*.\n\n\nFeatures\n--------\n\n* Configurable list of excluded exceptions (e.g. business exceptions)\n* Configurable failure threshold and reset timeout\n* Support for several event listeners per circuit breaker\n* Can guard generator functions\n* Functions and properties for easy monitoring and management\n* Thread-safe\n* Optional redis backing\n* Optional support for asynchronous Tornado calls\n\n\nRequirements\n------------\n\n* `Python`_ 3.8+\n\n\nInstallation\n------------\n\nRun the following command line to download the latest stable version of\nPyBreaker from `PyPI`_::\n\n $ pip install pybreaker\n\nIf you are a `Git`_ user, you might want to install the current development\nversion in editable mode::\n\n $ git clone git://github.com/danielfm/pybreaker.git\n $ cd pybreaker\n $ # run tests (on windows omit ./)\n $ ./pw test\n $ pip install -e .\n\n\nUsage\n-----\n\nThe first step is to create an instance of ``CircuitBreaker`` for each\nintegration point you want to protect against:\n\n.. code:: python\n\n import pybreaker\n\n # Used in database integration points\n db_breaker = pybreaker.CircuitBreaker(fail_max=5, reset_timeout=60)\n\n\n``CircuitBreaker`` instances should live globally inside the application scope,\ne.g., live across requests.\n\n.. note::\n\n Integration points to external services (i.e. databases, queues, etc) are\n more likely to fail, so make sure to always use timeouts when accessing such\n services if there's support at the API level.\n\nIf you'd like to use the Redis backing, initialize the ``CircuitBreaker`` with\na ``CircuitRedisStorage``:\n\n.. code:: python\n\n import pybreaker\n import redis\n\n redis = redis.StrictRedis()\n db_breaker = pybreaker.CircuitBreaker(\n fail_max=5,\n reset_timeout=60,\n state_storage=pybreaker.CircuitRedisStorage(pybreaker.STATE_CLOSED, redis))\n\n**Do not** initialize the Redis connection with the ``decode_responses`` set to\n``True``, this will force returning ASCII objects from redis and in Python3+ will\nfail with:\n\n `AttributeError: 'str' object has no attribute 'decode'`\n\n\n.. note::\n\n You may want to reuse a connection already created in your application, if you're\n using ``django_redis`` for example:\n\n.. code:: python\n\n import pybreaker\n from django_redis import get_redis_connection\n\n db_breaker = pybreaker.CircuitBreaker(\n fail_max=5,\n reset_timeout=60,\n state_storage=pybreaker.CircuitRedisStorage(pybreaker.STATE_CLOSED, get_redis_connection('default')))\n\n.. note::\n\n If you require multiple, independent CircuitBreakers and wish to store their states in Redis, it is essential to assign a ``unique namespace`` for each\n CircuitBreaker instance. This can be achieved by specifying a distinct namespace parameter in the CircuitRedisStorage constructor. for example:\n\n.. code:: python\n\n import pybreaker\n from django_redis import get_redis_connection\n\n db_breaker = pybreaker.CircuitBreaker(\n fail_max=5,\n reset_timeout=60,\n state_storage=pybreaker.CircuitRedisStorage(pybreaker.STATE_CLOSED, get_redis_connection('default'),namespace='unique_namespace'))\n\nEvent Listening\n```````````````\n\nThere's no need to subclass ``CircuitBreaker`` if you just want to take action\nwhen certain events occur. In that case, it's better to subclass\n``CircuitBreakerListener`` instead:\n\n.. code:: python\n\n class DBListener(pybreaker.CircuitBreakerListener):\n \"Listener used by circuit breakers that execute database operations.\"\n\n def before_call(self, cb, func, *args, **kwargs):\n \"Called before the circuit breaker `cb` calls `func`.\"\n pass\n\n def state_change(self, cb, old_state, new_state):\n \"Called when the circuit breaker `cb` state changes.\"\n pass\n\n def failure(self, cb, exc):\n \"Called when a function invocation raises a system error.\"\n pass\n\n def success(self, cb):\n \"Called when a function invocation succeeds.\"\n pass\n\n class LogListener(pybreaker.CircuitBreakerListener):\n \"Listener used to log circuit breaker events.\"\n\n def state_change(self, cb, old_state, new_state):\n msg = \"State Change: CB: {0}, New State: {1}\".format(cb.name, new_state)\n logging.info(msg)\n\n\nTo add listeners to a circuit breaker:\n\n.. code:: python\n\n # At creation time...\n db_breaker = pybreaker.CircuitBreaker(listeners=[DBListener(), LogListener()])\n\n # ...or later\n db_breaker.add_listeners(OneListener(), AnotherListener())\n\n\nWhat Does a Circuit Breaker Do?\n```````````````````````````````\n\nLet's say you want to use a circuit breaker on a function that updates a row\nin the ``customer`` database table:\n\n.. code:: python\n\n @db_breaker\n def update_customer(cust):\n # Do stuff here...\n pass\n\n # Will trigger the circuit breaker\n updated_customer = update_customer(my_customer)\n\n\nOr if you don't want to use the decorator syntax:\n\n.. code:: python\n\n def update_customer(cust):\n # Do stuff here...\n pass\n\n # Will trigger the circuit breaker\n updated_customer = db_breaker.call(update_customer, my_customer)\n\nOr use it as a context manager and a `with` statement:\n\n.. code:: python\n\n # Will trigger the circuit breaker\n with db_breaker.calling():\n # Do stuff here...\n pass\n\n\n\nAccording to the default parameters, the circuit breaker ``db_breaker`` will\nautomatically open the circuit after 5 consecutive failures in\n``update_customer``.\n\nWhen the circuit is open, all calls to ``update_customer`` will fail immediately\n(raising ``CircuitBreakerError``) without any attempt to execute the real\noperation. If you want the original error to be thrown when the circuit trips,\nset the ``throw_new_error_on_trip`` option to ``False``:\n\n.. code:: python\n\n pybreaker.CircuitBreaker(..., throw_new_error_on_trip=False)\n\n\nAfter 60 seconds, the circuit breaker will allow the next call to\n``update_customer`` pass through. If that call succeeds, the circuit is closed;\nif it fails, however, the circuit is opened again until another timeout elapses.\n\nOptional Tornado Support\n````````````````````````\nA circuit breaker can (optionally) be used to call asynchronous Tornado functions:\n\n.. code:: python\n\n from tornado import gen\n\n @db_breaker(__pybreaker_call_async=True)\n @gen.coroutine\n def async_update(cust):\n # Do async stuff here...\n pass\n\nOr if you don't want to use the decorator syntax:\n\n.. code:: python\n\n @gen.coroutine\n def async_update(cust):\n # Do async stuff here...\n pass\n\n updated_customer = db_breaker.call_async(async_update, my_customer)\n\n\nExcluding Exceptions\n````````````````````\n\nBy default, a failed call is any call that raises an exception. However, it's\ncommon to raise exceptions to also indicate business exceptions, and those\nexceptions should be ignored by the circuit breaker as they don't indicate\nsystem errors:\n\n.. code:: python\n\n # At creation time...\n db_breaker = CircuitBreaker(exclude=[CustomerValidationError])\n\n # ...or later\n db_breaker.add_excluded_exception(CustomerValidationError)\n\n\nIn that case, when any function guarded by that circuit breaker raises\n``CustomerValidationError`` (or any exception derived from\n``CustomerValidationError``), that call won't be considered a system failure.\n\nSo as to cover cases where the exception class alone is not enough to determine\nwhether it represents a system error, you may also pass a callable rather than\na type:\n\n.. code:: python\n\n db_breaker = CircuitBreaker(exclude=[lambda e: type(e) == HTTPError and e.status_code < 500])\n\nYou may mix types and filter callables freely.\n\n\nMonitoring and Management\n`````````````````````````\n\nA circuit breaker provides properties and functions you can use to monitor and\nchange its current state:\n\n.. code:: python\n\n # Get the current number of consecutive failures\n print(db_breaker.fail_counter)\n\n # Get/set the maximum number of consecutive failures\n print(db_breaker.fail_max)\n db_breaker.fail_max = 10\n\n # Get/set the current reset timeout period (in seconds)\n print db_breaker.reset_timeout\n db_breaker.reset_timeout = 60\n\n # Get the current state, i.e., 'open', 'half-open', 'closed'\n print(db_breaker.current_state)\n\n # Closes the circuit\n db_breaker.close()\n\n # Half-opens the circuit\n db_breaker.half_open()\n\n # Opens the circuit\n db_breaker.open()\n\n\nThese properties and functions might and should be exposed to the operations\nstaff somehow as they help them to detect problems in the system.\n\nContributing\n-------------\n\nRun tests::\n\n $ ./pw test\n\nCode formatting (black and isort) and linting (mypy) ::\n\n $ ./pw format\n $ ./pw lint\n\nAbove commands will automatically install the necessary tools inside *.pyprojectx*\nand also install pre-commit hooks.\n\nList available commands::\n\n $ ./pw -i\n\n.. _Python: http://python.org\n.. _Jython: http://jython.org\n.. _Release It!: https://pragprog.com/titles/mnee2/release-it-second-edition/\n.. _PyPI: http://pypi.python.org\n.. _Git: http://git-scm.com\n\n",
"bugtrack_url": null,
"license": "",
"summary": "Python implementation of the Circuit Breaker pattern",
"version": "1.2.0",
"project_urls": {
"Source": "http://github.com/danielfm/pybreaker"
},
"split_keywords": [
"design",
"pattern",
"circuit",
"breaker",
"integration"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6dbd8d9a1a0de00f10430a55b92910991ae8ea3dd407371054be12d619903474",
"md5": "edb2b55707fa920d3ec5cd0992b9bc82",
"sha256": "c3e7683e29ecb3d4421265aaea55504f1186a2fdc1f17b6b091d80d1e1eb5ede"
},
"downloads": -1,
"filename": "pybreaker-1.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "edb2b55707fa920d3ec5cd0992b9bc82",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 12327,
"upload_time": "2024-02-14T22:16:00",
"upload_time_iso_8601": "2024-02-14T22:16:00.672158Z",
"url": "https://files.pythonhosted.org/packages/6d/bd/8d9a1a0de00f10430a55b92910991ae8ea3dd407371054be12d619903474/pybreaker-1.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f69b675a7cad98bb19a131b9aaa5a1a876607071c64a5e64c9656f0156cec53a",
"md5": "5d3bdee79b5e1010aca96ea78f34392d",
"sha256": "18707776316f93a30c1be0e4fec1f8aa5ed19d7e395a218eb2f050c8524fb2dc"
},
"downloads": -1,
"filename": "pybreaker-1.2.0.tar.gz",
"has_sig": false,
"md5_digest": "5d3bdee79b5e1010aca96ea78f34392d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 14720,
"upload_time": "2024-02-14T22:16:04",
"upload_time_iso_8601": "2024-02-14T22:16:04.744484Z",
"url": "https://files.pythonhosted.org/packages/f6/9b/675a7cad98bb19a131b9aaa5a1a876607071c64a5e64c9656f0156cec53a/pybreaker-1.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-14 22:16:04",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "danielfm",
"github_project": "pybreaker",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "pybreaker"
}
