PyLoot
===========
PyLoot is a memory leak detector based on [Dozer](https://github.com/mgedmin/dozer) and [vprof](https://github.com/nvdv/vprof) with added support for servers with multiple workers/processes.
This project is in active development and may contain bugs or otherwise work in ways not expected or intended.
# Installation
```shell script
$ pip install pyloot
```
# Basic API Usage
```python
from pyloot import PyLoot
loot = PyLoot()
"""
Collect objects still in `gc.get_objects` after a call to `gc.collect`.
"""
loot.collect_objects()
"""
Collecting data in a background thread every 30 seconds.
If gevent is detected, gevent.threadpool.spawn is used.
Otherwise, threading.Thread is used.
"""
loot.start()
"""
Stop running the collector background thread.
NOTE: This does not do a "final" collection.
To ensure objects were collected in a short lived execution, call collect_objects().
:param blocking: When true, wait until the thread has died
"""
loot.stop()
"""
Return a WSGI compatible application serving the PyLoot remote backend and
and the website.
:return: ::class::`PyLootServer`
"""
loot.get_wsgi()
```
# Running embedded within a server
**Starlette/FastApi/ASGI**
_see note below about bypassing the [multiprocessing check](#bypass-the-multiprocessing-check)_
```python
from pyloot import PyLoot
from fastapi import FastAPI
from starlette.applications import Starlette
from starlette.middleware.wsgi import WSGIMiddleware
app = FastAPI() # or Starlette()
pyloot = PyLoot()
app.on_event("startup")(pyloot.start)
app.mount("/_pyloot", WSGIMiddleware(pyloot.get_wsgi()))
```
**Flask/WSGI**
```python
from pyloot import PyLoot
from flask import Flask
from werkzeug.middleware.dispatcher import DispatcherMiddleware
app = Flask(__name__)
pyloot = PyLoot()
app.on_before_first_request(pyloot.start)
app = DispatcherMiddleware(app, {
'/_pyloot': pyloot.get_wsgi()
})
```
# Running in remote mode (multi-process servers)
```python
# Embedded code
from pyloot import PyLoot
...
pyloot = PyLoot(host="127.0.0.1", port=8000)
...
```
```shell script
# Start the remote server
$ pyloot --help
usage: pyloot [-h HOST] [-p PORT] [--help]
optional arguments:
-h HOST, --host HOST Host to listen on. (Default: 0.0.0.0)
-p PORT, --port PORT Port to listen on. (Default: 8000)
--help show this help message and exit
```
# Bypass the multiprocessing check
If pyloot detects it is running in a multiprocessing environment with an inmemory backend
it will refuse to serve the webpages/requests.
This environment is common for gunicorn servers running with multiple workers.
If you run pyloot embedded in a gunicorn server with multiple workers, statistics will be collected in each individual worker and a random worker will be selected when returning statistics.
When using multiple workers, pyloot will give the most accurate information using the http backend.
For dev servers or servers with really low traffic (e.g. <1 request per second), you can also reduce the workers to 1.
Pyloot cannot detect how many workers are running so the bypass is still needed when only 1 worker is used.
The WSGIMiddleware of starlette sets `environ["wsgi.multiprocess"]=True` regardless of the server.
This can be bypassed with a wrapper **use with caution**:
```python
pyloot = PyLoot()
def pyloot_wrapper(wsgi_environ, start_response):
pyloot_environ = wsgi_environ.copy()
pyloot_environ["wsgi.multiprocess"] = False
wsgi = pyloot.get_wsgi()
return wsgi(pyloot_environ, start_response)
app.mount("/_pyloot", WSGIMiddleware(pyloot_wrapper))
```
# Disabling gzip encoding
By default, the pyloot server will gzip encode the response metadata.
If pyloot is running behind a middleware that gzip encodes data, encoding can happen twice.
This will result in the following error being shown in the UI:
```text
Error parsing the response data. Check the server logs. If everything looks ok, you make need to disable gzip in pyloot. For more info see the README.
```
To disable gzip encoding do the following:
```python
from pyloot import PyLoot
from pyloot import PyLootServer
pyloot = PyLoot(server=PyLootServer(disable_response_gzip=True))
```
If a remote server is used, it must be configured directly on the server like so:
```python
from pyloot import PyLoot
from pyloot import PyLootServer
from pyloot import HTTPRemoteBackend
backend = HTTPRemoteBackend(host="127.0.0.1", port=8000)
pyloot = PyLoot(server=PyLootServer(backend=backend, disable_response_gzip=True))
```
# Screenshots
### View history of object counts by object group:
### Modify history page size
### Search history page
### View objects by group
### Modify objects fetch size
### Modify objects page size
### View an object, its attributes, __repr__, children, and parents
Raw data
{
"_id": null,
"home_page": "https://github.com/reallistic/pyloot",
"name": "pyloot",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "memory profiler multiprocessing wsgi asgi",
"author": "Michael Chase",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/8a/6b/bc5631c7faea2b7a0026a9c3cc3e64b27b9a200411d3519686d92d3d8c47/pyloot-0.1.0.tar.gz",
"platform": null,
"description": "PyLoot\n===========\n\nPyLoot is a memory leak detector based on [Dozer](https://github.com/mgedmin/dozer) and [vprof](https://github.com/nvdv/vprof) with added support for servers with multiple workers/processes.\n\n\nThis project is in active development and may contain bugs or otherwise work in ways not expected or intended.\n\n# Installation\n```shell script\n$ pip install pyloot\n```\n\n# Basic API Usage\n```python\nfrom pyloot import PyLoot\nloot = PyLoot()\n\n\n\"\"\"\nCollect objects still in `gc.get_objects` after a call to `gc.collect`.\n\"\"\"\nloot.collect_objects()\n\n\"\"\"\nCollecting data in a background thread every 30 seconds.\n\nIf gevent is detected, gevent.threadpool.spawn is used.\nOtherwise, threading.Thread is used.\n\"\"\"\nloot.start()\n\n\"\"\"\nStop running the collector background thread.\n\nNOTE: This does not do a \"final\" collection.\nTo ensure objects were collected in a short lived execution, call collect_objects().\n\n:param blocking: When true, wait until the thread has died\n\"\"\"\nloot.stop()\n\n\"\"\"\nReturn a WSGI compatible application serving the PyLoot remote backend and\nand the website.\n:return: ::class::`PyLootServer`\n\"\"\"\nloot.get_wsgi()\n```\n\n\n# Running embedded within a server\n**Starlette/FastApi/ASGI**\n\n_see note below about bypassing the [multiprocessing check](#bypass-the-multiprocessing-check)_\n\n```python\nfrom pyloot import PyLoot\nfrom fastapi import FastAPI\nfrom starlette.applications import Starlette\nfrom starlette.middleware.wsgi import WSGIMiddleware\n\napp = FastAPI() # or Starlette()\n\npyloot = PyLoot()\napp.on_event(\"startup\")(pyloot.start)\napp.mount(\"/_pyloot\", WSGIMiddleware(pyloot.get_wsgi()))\n```\n\n\n**Flask/WSGI**\n```python\nfrom pyloot import PyLoot\nfrom flask import Flask\nfrom werkzeug.middleware.dispatcher import DispatcherMiddleware\n\napp = Flask(__name__)\n\npyloot = PyLoot()\napp.on_before_first_request(pyloot.start)\n\napp = DispatcherMiddleware(app, {\n '/_pyloot': pyloot.get_wsgi()\n})\n```\n\n# Running in remote mode (multi-process servers)\n```python\n# Embedded code\nfrom pyloot import PyLoot\n...\npyloot = PyLoot(host=\"127.0.0.1\", port=8000)\n...\n```\n```shell script\n# Start the remote server\n$ pyloot --help\nusage: pyloot [-h HOST] [-p PORT] [--help]\n\noptional arguments:\n-h HOST, --host HOST Host to listen on. (Default: 0.0.0.0)\n-p PORT, --port PORT Port to listen on. (Default: 8000)\n--help show this help message and exit\n```\n\n# Bypass the multiprocessing check\nIf pyloot detects it is running in a multiprocessing environment with an inmemory backend\nit will refuse to serve the webpages/requests.\n\nThis environment is common for gunicorn servers running with multiple workers.\nIf you run pyloot embedded in a gunicorn server with multiple workers, statistics will be collected in each individual worker and a random worker will be selected when returning statistics.\nWhen using multiple workers, pyloot will give the most accurate information using the http backend.\nFor dev servers or servers with really low traffic (e.g. <1 request per second), you can also reduce the workers to 1.\nPyloot cannot detect how many workers are running so the bypass is still needed when only 1 worker is used.\n\nThe WSGIMiddleware of starlette sets `environ[\"wsgi.multiprocess\"]=True` regardless of the server.\nThis can be bypassed with a wrapper **use with caution**:\n\n```python\npyloot = PyLoot()\n\ndef pyloot_wrapper(wsgi_environ, start_response):\n pyloot_environ = wsgi_environ.copy()\n pyloot_environ[\"wsgi.multiprocess\"] = False\n wsgi = pyloot.get_wsgi()\n return wsgi(pyloot_environ, start_response)\n\napp.mount(\"/_pyloot\", WSGIMiddleware(pyloot_wrapper))\n```\n\n# Disabling gzip encoding\nBy default, the pyloot server will gzip encode the response metadata.\nIf pyloot is running behind a middleware that gzip encodes data, encoding can happen twice.\nThis will result in the following error being shown in the UI:\n\n```text\nError parsing the response data. Check the server logs. If everything looks ok, you make need to disable gzip in pyloot. For more info see the README.\n```\n\nTo disable gzip encoding do the following:\n\n```python\nfrom pyloot import PyLoot\nfrom pyloot import PyLootServer\n\npyloot = PyLoot(server=PyLootServer(disable_response_gzip=True))\n```\n\n\nIf a remote server is used, it must be configured directly on the server like so:\n\n```python\nfrom pyloot import PyLoot\nfrom pyloot import PyLootServer\nfrom pyloot import HTTPRemoteBackend\n\nbackend = HTTPRemoteBackend(host=\"127.0.0.1\", port=8000)\npyloot = PyLoot(server=PyLootServer(backend=backend, disable_response_gzip=True))\n```\n\n# Screenshots\n### View history of object counts by object group:\n\n\n### Modify history page size\n\n\n### Search history page\n\n\n### View objects by group\n\n\n### Modify objects fetch size\n\n\n### Modify objects page size\n\n\n### View an object, its attributes, __repr__, children, and parents\n\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Multiprocessing compatible memory leak debugger inspired by dozer/dowser",
"version": "0.1.0",
"split_keywords": [
"memory",
"profiler",
"multiprocessing",
"wsgi",
"asgi"
],
"urls": [
{
"comment_text": "",
"digests": {
"md5": "d1d4957a4dae3befabde405e065805be",
"sha256": "7173439ad6c4adbd7019af93fae79efc96b2a7ccaff5363238246c588cd05f12"
},
"downloads": -1,
"filename": "pyloot-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d1d4957a4dae3befabde405e065805be",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 189880,
"upload_time": "2022-12-05T20:28:14",
"upload_time_iso_8601": "2022-12-05T20:28:14.541007Z",
"url": "https://files.pythonhosted.org/packages/2a/fd/f4a4d9b8ea16d59a3d6a76469983d5a88cfeb58959e09c08402baae2da3a/pyloot-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"md5": "d5c3e78bce415e515e5195d7c4c2aae4",
"sha256": "ad7ae41f8e2cd55469d4a8743bbe301f39f05d0c93d67f4038fbd89cedb33d22"
},
"downloads": -1,
"filename": "pyloot-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "d5c3e78bce415e515e5195d7c4c2aae4",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 189501,
"upload_time": "2022-12-05T20:28:15",
"upload_time_iso_8601": "2022-12-05T20:28:15.803138Z",
"url": "https://files.pythonhosted.org/packages/8a/6b/bc5631c7faea2b7a0026a9c3cc3e64b27b9a200411d3519686d92d3d8c47/pyloot-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-12-05 20:28:15",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "reallistic",
"github_project": "pyloot",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "pyloot"
}
JSON
