aiodebug
========
This is a tiny library for monitoring and testing asyncio programs.
Its monitoring features are meant to be always on in production.
Installation
--------------
``aiodebug`` is only compatible with Python 3.8 and higher. There are no plans to support older versions.
``aiodebug`` is `available on PyPI <https://pypi.org/project/aiodebug/>`_ and you can install it with:
::
pip install aiodebug
or
::
poetry add aiodebug
``aiodebug`` will use `logwood <https://github.com/qntln/logwood>`_ if it is installed, otherwise it will default
to the standard logging module.
Log warnings when callbacks block the event loop
------------------------------------------------
.. code-block:: python
import aiodebug.log_slow_callbacks
aiodebug.log_slow_callbacks.enable(0.05)
This will produce WARNING-level logs such as
.. code-block::
Executing <Task pending coro=<foo() running at /home/.../foo.py:37>
wait_for=<Future pending cb=[Task._wakeup()]>> took 0.069 seconds
asyncio already does this in debug mode, but you probably don't want to enable full-on debug mode in production.
Instead of defaulting to the logs, you may provide your own callback that gets called with the name of the
slow callback and its execution duration, and can do anything it needs with it. This might be useful
e.g. for structured JSON logging.
.. code-block:: python
import aiodebug.log_slow_callbacks
aiodebug.log_slow_callbacks.enable(
0.05,
on_slow_callback = lambda task_name, duration: json_logger.warning(
'Task blocked async loop for too long',
extra = {'task_name': task_name, 'duration': duration}
)
)
Track event loop lags in StatsD
------------------------------------------------
.. code-block:: python
import aiodebug.monitor_loop_lag
aiodebug.monitor_loop_lag.enable(statsd_client)
Tracks how much scheduled calls get delayed and sends the lags to StatsD.
.. image:: loop-lags.png
Dump stack traces of all threads if the event loop hangs for too long
-----------------------------------------------------------------------
.. code-block:: python
import aiodebug.hang_inspection
dumper = aiodebug.hang_inspection.start('/path/to/output/directory', interval = 0.25) # 0.25 is the default
...
await aiodebug.hang_inspection.stop_wait(dumper)
Enabling this function may help you in case one of your threads (sometimes) runs a CPU-bound operation that
completely stalls the event loop, but you don't know which thread it is or what it is doing.
Every time the event loop hangs (doesn't run a scheduled 'monitoring' task) for longer than the given
``interval``, aiodebug will create 3 stack traces, 1 second apart, in your output directory.
For example:
.. code-block::
-rw-r--r-- 1 user group 6.7K 4 Jan 09:41 stacktrace-20220104-094154.197418-0.txt
-rw-r--r-- 1 user group 7.0K 4 Jan 09:41 stacktrace-20220104-094155.206574-1.txt
-rw-r--r-- 1 user group 6.6K 4 Jan 09:41 stacktrace-20220104-094156.211781-2.txt
Each file then contains the Python stack traces of all threads that were running or waiting at the time.
You might be able to find your culprit blocking the event loop at the end of one of the traces.
Speed up or slow down time in the event loop
------------------------------------------------
This is mainly useful for testing.
.. code-block:: python
import aiodebug.testing.time_dilated_loop
loop = aiodebug.testing.time_dilated_loop.TimeDilatedLoop()
asyncio.set_event_loop(loop)
loop.time_dilation = 3
await asyncio.sleep(1) # Takes 0.333s of real time
loop.time_dilation = 0.1
await asyncio.sleep(1) # Takes 10s of real time
****
.. image:: quantlane.png
``aiodebug`` was made by `Quantlane <https://quantlane.com>`_, a systematic trading firm.
We design, build and run our own stock trading platform.
Raw data
{
"_id": null,
"home_page": "https://gitlab.com/quantlane/libs/aiodebug",
"name": "aiodebug",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "Quantlane",
"author_email": "code@quantlane.com",
"download_url": "",
"platform": "",
"description": "aiodebug\n========\n\nThis is a tiny library for monitoring and testing asyncio programs.\nIts monitoring features are meant to be always on in production.\n\n\nInstallation\n--------------\n``aiodebug`` is only compatible with Python 3.8 and higher. There are no plans to support older versions.\n\n``aiodebug`` is `available on PyPI <https://pypi.org/project/aiodebug/>`_ and you can install it with:\n\n::\n\n\tpip install aiodebug\n\nor\n\n::\n\n\tpoetry add aiodebug\n\n\n``aiodebug`` will use `logwood <https://github.com/qntln/logwood>`_ if it is installed, otherwise it will default\nto the standard logging module.\n\n\nLog warnings when callbacks block the event loop\n------------------------------------------------\n\n.. code-block:: python\n\n\timport aiodebug.log_slow_callbacks\n\n\taiodebug.log_slow_callbacks.enable(0.05)\n\nThis will produce WARNING-level logs such as\n\n.. code-block::\n\n\tExecuting <Task pending coro=<foo() running at /home/.../foo.py:37>\n\twait_for=<Future pending cb=[Task._wakeup()]>> took 0.069 seconds\n\nasyncio already does this in debug mode, but you probably don't want to enable full-on debug mode in production.\n\nInstead of defaulting to the logs, you may provide your own callback that gets called with the name of the\nslow callback and its execution duration, and can do anything it needs with it. This might be useful\ne.g. for structured JSON logging.\n\n.. code-block:: python\n\n\timport aiodebug.log_slow_callbacks\n\n\taiodebug.log_slow_callbacks.enable(\n\t\t0.05,\n\t\ton_slow_callback = lambda task_name, duration: json_logger.warning(\n\t\t\t'Task blocked async loop for too long',\n\t\t\textra = {'task_name': task_name, 'duration': duration}\n\t\t)\n\t)\n\nTrack event loop lags in StatsD\n------------------------------------------------\n\n.. code-block:: python\n\n\timport aiodebug.monitor_loop_lag\n\n\taiodebug.monitor_loop_lag.enable(statsd_client)\n\nTracks how much scheduled calls get delayed and sends the lags to StatsD.\n\n.. image:: loop-lags.png\n\n\nDump stack traces of all threads if the event loop hangs for too long\n-----------------------------------------------------------------------\n\n.. code-block:: python\n\n\timport aiodebug.hang_inspection\n\n\tdumper = aiodebug.hang_inspection.start('/path/to/output/directory', interval = 0.25) # 0.25 is the default\n\t...\n\tawait aiodebug.hang_inspection.stop_wait(dumper)\n\nEnabling this function may help you in case one of your threads (sometimes) runs a CPU-bound operation that\ncompletely stalls the event loop, but you don't know which thread it is or what it is doing.\n\nEvery time the event loop hangs (doesn't run a scheduled 'monitoring' task) for longer than the given\n``interval``, aiodebug will create 3 stack traces, 1 second apart, in your output directory.\nFor example:\n\n.. code-block::\n\n\t-rw-r--r-- 1 user group 6.7K 4 Jan 09:41 stacktrace-20220104-094154.197418-0.txt\n\t-rw-r--r-- 1 user group 7.0K 4 Jan 09:41 stacktrace-20220104-094155.206574-1.txt\n\t-rw-r--r-- 1 user group 6.6K 4 Jan 09:41 stacktrace-20220104-094156.211781-2.txt\n\nEach file then contains the Python stack traces of all threads that were running or waiting at the time.\nYou might be able to find your culprit blocking the event loop at the end of one of the traces.\n\n\nSpeed up or slow down time in the event loop\n------------------------------------------------\n\nThis is mainly useful for testing.\n\n.. code-block:: python\n\n\timport aiodebug.testing.time_dilated_loop\n\n\tloop = aiodebug.testing.time_dilated_loop.TimeDilatedLoop()\n\tasyncio.set_event_loop(loop)\n\n\tloop.time_dilation = 3\n\tawait asyncio.sleep(1) # Takes 0.333s of real time\n\n\tloop.time_dilation = 0.1\n\tawait asyncio.sleep(1) # Takes 10s of real time\n\n\n****\n\n\t.. image:: quantlane.png\n\n\t``aiodebug`` was made by `Quantlane <https://quantlane.com>`_, a systematic trading firm.\n\tWe design, build and run our own stock trading platform.\n\n\n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "A tiny library for monitoring and testing asyncio programs",
"version": "2.3.0",
"project_urls": {
"Homepage": "https://gitlab.com/quantlane/libs/aiodebug"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "39ac0b32c4f7c7b726ff292d1e38e0bd180675768bec6f2ec9c109d086c384a1",
"md5": "eebd813ee8dd4c7925fde536d937db19",
"sha256": "ed408cd90869b395f5bc642acffcbaed3b2668e89b40e684a95e0fb60a3257cd"
},
"downloads": -1,
"filename": "aiodebug-2.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "eebd813ee8dd4c7925fde536d937db19",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 7872,
"upload_time": "2022-01-04T13:33:46",
"upload_time_iso_8601": "2022-01-04T13:33:46.323914Z",
"url": "https://files.pythonhosted.org/packages/39/ac/0b32c4f7c7b726ff292d1e38e0bd180675768bec6f2ec9c109d086c384a1/aiodebug-2.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-01-04 13:33:46",
"github": false,
"gitlab": true,
"bitbucket": false,
"codeberg": false,
"gitlab_user": "quantlane",
"gitlab_project": "libs",
"lcname": "aiodebug"
}
JSON