# Django Requests Tracker
<p>
<a href="https://github.com/bensi94/Django-Requests-Tracker/actions/workflows/quality.yml?query=event%3Apush+branch%3Amain" target="_blank">
<img src="https://github.com/bensi94/Django-Requests-Tracker/actions/workflows/quality.yml/badge.svg?event=push&branch=main" alt="Test & Quality">
</a>
<a href="https://coverage-badge.samuelcolvin.workers.dev/redirect/bensi94/Django-Requests-Tracker" target="_blank">
<img src="https://coverage-badge.samuelcolvin.workers.dev/bensi94/Django-Requests-Tracker.svg" alt="Coverage">
</a>
<a href="https://pypi.org/project/requests-tracker/" target="_blank">
<img src="https://img.shields.io/pypi/v/requests-tracker?color=blue" alt="Package version"/>
</a>
<a href="https://pypi.org/project/requests-tracker/" target="_blank">
<img src="https://img.shields.io/pypi/pyversions/requests-tracker?color=blue" alt="Supported Python versions">
</a>
</p>
A convenient Django development tool based on the great [`Django Debug Toolbar`](https://github.com/jazzband/django-debug-toolbar) but aimed towards rest API development. It collects and displays information on requests, responses, SQL queries, headers, Django settings and more.
## Table of contents
1. [Features](#features)
1. [Requests list](#requests-list)
2. [Request details](#request-details)
2. [The example Project](#the-example-project)
3. [Installation](#installation)
1. [Install the package](#install-the-package)
2. [Configure project settings](#configure-project-settings)
3. [Configure URLs](#configure-urls)
4. [Optional: Configure static content for WSGI and ASGI servers, e.g. Uvicorn for async Django](#configure-static-content)
4. [Configuration](#configuration)
1. [IGNORE_SQL_PATTERNS](#ignore_sql_patterns)
2. [IGNORE_PATHS_PATTERNS](#ignore_paths_patterns)
3. [ENABLE_STACKTRACES](#enable_stacktraces)
4. [HIDE_IN_STACKTRACES](#hide_in_stacktraces)
5. [SQL_WARNING_THRESHOLD](#sql_warning_threshold)
6. [TRACK_SQL](#track_sql)
## Features
### Requests list
Django Requests Tracker registers every request sent to your Django application and displays them in a tidy list. Each element in the list contains information about the request's HTTP method, path, Django view, status code, database information and query count and execution time and duration.
The requests list can be:
* Searched by *path*, *Django view*, *sql* and *headers*. The search is quite simple and a request is only filtered from the list if the search term does not exist in any of theses elements.
* Ordered in ascending and descending order by *time*, *duration*, *Django view*, *query count*, *similar query count* and *duplicate query count*.
* Auto-refreshed so that new requests will automatically show up in the list.
* Manually refreshed.
* Cleared.
#### The requests list in action 🎥
### Request details
Each list element can be clicked where further information collected about the request such as SQL queries and headers can be found.
#### SQL queries
In request details, every SQL query executed in the context of the Django request should be shown, along with the execution time and a timeline bar that shows how big a chunk of the total time belongs to the given query. A stacktrace is shown for each query that helps with finding the origin of it.
Some queries are labelled with a tag `X similar queries` or `X duplicate queries` this can often indicate a problem and can be very handy when debugging or in development.
* `Similar Queries` means that the same query is executed more than once but with different parameters. This can for example happen when iterating over a list of IDs and fetching one item by ID at a time.
* `Duplicate Queries` means that the exact same query with the same parameters is executed more than once. This can for example happen when iterating over a list child items and fetching same parent multiple times. Also known as an N-plus-1 query which is quite common problem with ORMs.
#### The request details view in action 🎥
### Django Settings
Django settings very often contain some logic, and usage of environment variables and can even be spread out over multiple files. So it can be very beneficial to be able to see the current computed settings being used in the running process. Django Requests Tracker offers a simple way to view this. The view can be accessed by clicking on `Django settings` in the right corner of the requests tracker view.
All information determined to be sensitive, such as keys and passwords, are masked in the displayed settings.
<img width="1470" alt="Screenshot 2023-01-31 at 00 24 32" src="https://user-images.githubusercontent.com/20007971/215627287-4d62cc7d-1679-4fee-ad20-c52b59dccf34.png">
## The Example Project
This repository includes an [example project](example_project) to try out the package and see how it works. It can also be a great reference when adding the package to your project. To try it out, clone this project and follow the instructions on the [example project README](example_project/README.md)
## Installation
If any of the following steps are unclear, check out the [Example Project](example_project) for reference.
### Install the package
```bash
pip install requests-tracker
```
or install with you're chosen package tool, e.g.
[poetry](https://python-poetry.org/),
[pipenv](https://pipenv.pypa.io/en/latest/), etc.
### Configure project settings
#### Settings prerequisites
First, ensure that `django.contrib.staticfiles` is in your `INSTALLED_APPS` setting and configured properly:
```python
INSTALLED_APPS = [
# ...
"django.contrib.staticfiles",
# ...
]
STATIC_URL = "static/"
```
Second, ensure that your `TEMPLATES` setting contains a `DjangoTemplates` backend whose `APP_DIRS` options is set to True:
```python
TEMPLATES = [
{
"BACKEND": "django.template.backends.django.DjangoTemplates",
"APP_DIRS": True,
# ...
}
]
```
#### Install the app, add middleware and configure internal ips
* Add `requests_tracker` to your `INSTALLED_APPS` setting.
* Add `requests_tracker.middleware.requests_tracker_middleware` to your `MIDDLEWARE` setting.
* Add your internal IP addresses to `INTERNAL_IPS` setting.
```python
if DEBUG:
INSTALLED_APPS += ["requests_tracker"]
MIDDLEWARE += ["requests_tracker.middleware.requests_tracker_middleware"]
INTERNAL_IPS = ["127.0.0.1"]
```
⚠️ If using [Docker](https://www.docker.com/) the following will set your INTERNAL_IPS correctly in Debug mode:
```python
if DEBUG:
import socket # only if you haven't already imported this
hostname, _, ips = socket.gethostbyname_ex(socket.gethostname())
INTERNAL_IPS = [ip[: ip.rfind(".")] + ".1" for ip in ips] + ["127.0.0.1", "10.0.2.2"]
```
🚨 ️ It's recommended to only configure these settings in DEBUG mode.
Even though Django Requests Tracker will only track requests in DEBUG mode
it's still a good practice to only have it installed in DEBUG mode.
### Configure URLs
Add Django Requests Tracker URLs to your project's URLconf:
```python
if settings.DEBUG:
urlpatterns += [path("__requests_tracker__/", include("requests_tracker.urls"))]
```
🚨️ Again it's recommended to only add the URLs in DEBUG mode.
### Optional: Configure static content for WSGI and ASGI servers, e.g. Uvicorn for async Django <a name="configure-static-content"></a>
#### Add static root to settings
```python
# 🚨 Your project might not include BASE_DIR setting but likely some variation of it 🚨
BASE_DIR = Path(__file__).resolve().parent.parent
STATIC_ROOT = os.path.join(BASE_DIR, "static")
```
#### Add static root URLs to your project's URLconf:
```python
urlpatterns += static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)
```
#### Collect static files
```console
python manage.py collectstatic
```
## Configuration
Django Requests Tracker provides a few very simple settings. The settings are applied by setting `REQUESTS_TRACKER_CONFIG` setting in your `settings.py` file. `REQUESTS_TRACKER_CONFIG` takes a dictonary. Example:
```python
# settings.py
REQUESTS_TRACKER_CONFIG = {
"IGNORE_PATHS_PATTERNS": (".*/api/keep-alive.*",),
"ENABLE_STACKTRACES": False",
}
```
### `IGNORE_SQL_PATTERNS`
Takes a tuple of strings. Each string is a regular expression pattern.
If a SQL query matches any of the patterns it will be ignored and not
shown in the requests list or request details.
Default: `()`
Example:
```python
REQUESTS_TRACKER_CONFIG = {
"IGNORE_SQL_PATTERNS": (
r"^SELECT .* FROM django_migrations WHERE app = 'requests_tracker'",
r"^SELECT .* FROM django_migrations WHERE app = 'auth'",
),
}
```
### `IGNORE_PATHS_PATTERNS`
Takes a tuple of strings. Each string is a regular expression pattern.
If a request path matches any of the patterns it will be ignored and not tracked.
Default: `()`
Example:
```python
REQUESTS_TRACKER_CONFIG = {
"IGNORE_PATHS_PATTERNS": (
r".*/api/keep-alive.*",
),
}
```
### `SQL_WARNING_THRESHOLD`
Represents the threshold in milliseconds after which a SQL query is considered slow and
will be marked with a warning label in the SQL list.
Default: `500` (500 milliseconds)
Example:
```python
REQUESTS_TRACKER_CONFIG = {
"SQL_WARNING_THRESHOLD": 50,
}
```
### `ENABLE_STACKTRACES`
If set to `False` stacktraces will not be shown in the request details view.
Default: `True`
### `HIDE_IN_STACKTRACES`
Takes a tuple of strings. Each string represents a module name. If a module name is found
in a stacktrace that part of the stacktrace will be hidden.
Default:
```python
(
"socketserver",
"threading",
"wsgiref",
"requests_tracker",
"django.db",
"django.core.handlers",
"django.core.servers",
"django.utils.decorators",
"django.utils.deprecation",
"django.utils.functional",
)
```
### `TRACK_SQL`
If set to `False` SQL queries will not be tracked.
Default: `True`
Raw data
{
"_id": null,
"home_page": "https://github.com/bensi94/Django-Requests-Tracker",
"name": "requests-tracker",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4.0",
"maintainer_email": "",
"keywords": "django,python,sql,middlware,api,rest,debug,htmx,django-rest-framework,django-ninja,requests-tracker,django-requests-tracker,development",
"author": "Benedikt \u00d3skarsson",
"author_email": "bensi94@hotmail.com",
"download_url": "https://files.pythonhosted.org/packages/e9/10/35d214c4eaa479251ebb6f774842e476cd4162ca939e72bb1d943131fb2c/requests_tracker-0.3.3.tar.gz",
"platform": null,
"description": "# Django Requests Tracker\n\n<p>\n\n<a href=\"https://github.com/bensi94/Django-Requests-Tracker/actions/workflows/quality.yml?query=event%3Apush+branch%3Amain\" target=\"_blank\">\n <img src=\"https://github.com/bensi94/Django-Requests-Tracker/actions/workflows/quality.yml/badge.svg?event=push&branch=main\" alt=\"Test & Quality\">\n</a>\n<a href=\"https://coverage-badge.samuelcolvin.workers.dev/redirect/bensi94/Django-Requests-Tracker\" target=\"_blank\">\n <img src=\"https://coverage-badge.samuelcolvin.workers.dev/bensi94/Django-Requests-Tracker.svg\" alt=\"Coverage\">\n</a>\n<a href=\"https://pypi.org/project/requests-tracker/\" target=\"_blank\">\n <img src=\"https://img.shields.io/pypi/v/requests-tracker?color=blue\" alt=\"Package version\"/>\n</a>\n<a href=\"https://pypi.org/project/requests-tracker/\" target=\"_blank\">\n <img src=\"https://img.shields.io/pypi/pyversions/requests-tracker?color=blue\" alt=\"Supported Python versions\">\n</a>\n</p>\n\n\nA convenient Django development tool based on the great [`Django Debug Toolbar`](https://github.com/jazzband/django-debug-toolbar) but aimed towards rest API development. It collects and displays information on requests, responses, SQL queries, headers, Django settings and more.\n\n## Table of contents\n1. [Features](#features)\n 1. [Requests list](#requests-list)\n 2. [Request details](#request-details)\n2. [The example Project](#the-example-project)\n3. [Installation](#installation)\n 1. [Install the package](#install-the-package)\n 2. [Configure project settings](#configure-project-settings)\n 3. [Configure URLs](#configure-urls)\n 4. [Optional: Configure static content for WSGI and ASGI servers, e.g. Uvicorn for async Django](#configure-static-content)\n4. [Configuration](#configuration)\n 1. [IGNORE_SQL_PATTERNS](#ignore_sql_patterns)\n 2. [IGNORE_PATHS_PATTERNS](#ignore_paths_patterns)\n 3. [ENABLE_STACKTRACES](#enable_stacktraces)\n 4. [HIDE_IN_STACKTRACES](#hide_in_stacktraces)\n 5. [SQL_WARNING_THRESHOLD](#sql_warning_threshold)\n 6. [TRACK_SQL](#track_sql)\n\n## Features\n\n### Requests list\n\nDjango Requests Tracker registers every request sent to your Django application and displays them in a tidy list. Each element in the list contains information about the request's HTTP method, path, Django view, status code, database information and query count and execution time and duration.\n\nThe requests list can be:\n* Searched by *path*, *Django view*, *sql* and *headers*. The search is quite simple and a request is only filtered from the list if the search term does not exist in any of theses elements.\n* Ordered in ascending and descending order by *time*, *duration*, *Django view*, *query count*, *similar query count* and *duplicate query count*.\n* Auto-refreshed so that new requests will automatically show up in the list.\n* Manually refreshed.\n* Cleared.\n\n#### The requests list in action \ud83c\udfa5\n\n\n\n### Request details\n\nEach list element can be clicked where further information collected about the request such as SQL queries and headers can be found.\n\n#### SQL queries\n\nIn request details, every SQL query executed in the context of the Django request should be shown, along with the execution time and a timeline bar that shows how big a chunk of the total time belongs to the given query. A stacktrace is shown for each query that helps with finding the origin of it.\n\nSome queries are labelled with a tag `X similar queries` or `X duplicate queries` this can often indicate a problem and can be very handy when debugging or in development.\n\n* `Similar Queries` means that the same query is executed more than once but with different parameters. This can for example happen when iterating over a list of IDs and fetching one item by ID at a time.\n* `Duplicate Queries` means that the exact same query with the same parameters is executed more than once. This can for example happen when iterating over a list child items and fetching same parent multiple times. Also known as an N-plus-1 query which is quite common problem with ORMs.\n\n#### The request details view in action \ud83c\udfa5\n\n\n\n### Django Settings\n\nDjango settings very often contain some logic, and usage of environment variables and can even be spread out over multiple files. So it can be very beneficial to be able to see the current computed settings being used in the running process. Django Requests Tracker offers a simple way to view this. The view can be accessed by clicking on `Django settings` in the right corner of the requests tracker view.\n\nAll information determined to be sensitive, such as keys and passwords, are masked in the displayed settings.\n\n<img width=\"1470\" alt=\"Screenshot 2023-01-31 at 00 24 32\" src=\"https://user-images.githubusercontent.com/20007971/215627287-4d62cc7d-1679-4fee-ad20-c52b59dccf34.png\">\n\n## The Example Project\n\nThis repository includes an [example project](example_project) to try out the package and see how it works. It can also be a great reference when adding the package to your project. To try it out, clone this project and follow the instructions on the [example project README](example_project/README.md)\n\n## Installation\n\nIf any of the following steps are unclear, check out the [Example Project](example_project) for reference.\n\n### Install the package\n\n```bash\npip install requests-tracker\n```\n\nor install with you're chosen package tool, e.g.\n[poetry](https://python-poetry.org/),\n[pipenv](https://pipenv.pypa.io/en/latest/), etc.\n\n### Configure project settings\n\n#### Settings prerequisites\n\nFirst, ensure that `django.contrib.staticfiles` is in your `INSTALLED_APPS` setting and configured properly:\n```python\nINSTALLED_APPS = [\n # ...\n \"django.contrib.staticfiles\",\n # ...\n]\n\nSTATIC_URL = \"static/\"\n```\n\nSecond, ensure that your `TEMPLATES` setting contains a `DjangoTemplates` backend whose `APP_DIRS` options is set to True:\n```python\nTEMPLATES = [\n {\n \"BACKEND\": \"django.template.backends.django.DjangoTemplates\",\n \"APP_DIRS\": True,\n # ...\n }\n]\n```\n\n#### Install the app, add middleware and configure internal ips\n\n* Add `requests_tracker` to your `INSTALLED_APPS` setting.\n* Add `requests_tracker.middleware.requests_tracker_middleware` to your `MIDDLEWARE` setting.\n* Add your internal IP addresses to `INTERNAL_IPS` setting.\n```python\nif DEBUG:\n INSTALLED_APPS += [\"requests_tracker\"]\n MIDDLEWARE += [\"requests_tracker.middleware.requests_tracker_middleware\"]\n INTERNAL_IPS = [\"127.0.0.1\"]\n```\n\n\u26a0\ufe0f If using [Docker](https://www.docker.com/) the following will set your INTERNAL_IPS correctly in Debug mode:\n```python\nif DEBUG:\n import socket # only if you haven't already imported this\n hostname, _, ips = socket.gethostbyname_ex(socket.gethostname())\n INTERNAL_IPS = [ip[: ip.rfind(\".\")] + \".1\" for ip in ips] + [\"127.0.0.1\", \"10.0.2.2\"]\n```\n\n\ud83d\udea8 \ufe0f It's recommended to only configure these settings in DEBUG mode.\nEven though Django Requests Tracker will only track requests in DEBUG mode\nit's still a good practice to only have it installed in DEBUG mode.\n\n### Configure URLs\n\nAdd Django Requests Tracker URLs to your project's URLconf:\n```python\nif settings.DEBUG:\n urlpatterns += [path(\"__requests_tracker__/\", include(\"requests_tracker.urls\"))]\n```\n\n\ud83d\udea8\ufe0f Again it's recommended to only add the URLs in DEBUG mode.\n\n\n### Optional: Configure static content for WSGI and ASGI servers, e.g. Uvicorn for async Django <a name=\"configure-static-content\"></a>\n\n#### Add static root to settings\n```python\n# \ud83d\udea8 Your project might not include BASE_DIR setting but likely some variation of it \ud83d\udea8\nBASE_DIR = Path(__file__).resolve().parent.parent\n\nSTATIC_ROOT = os.path.join(BASE_DIR, \"static\")\n```\n\n#### Add static root URLs to your project's URLconf:\n```python\nurlpatterns += static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)\n```\n\n#### Collect static files\n```console\npython manage.py collectstatic\n```\n\n## Configuration\n\nDjango Requests Tracker provides a few very simple settings. The settings are applied by setting `REQUESTS_TRACKER_CONFIG` setting in your `settings.py` file. `REQUESTS_TRACKER_CONFIG` takes a dictonary. Example:\n\n```python\n# settings.py\n\nREQUESTS_TRACKER_CONFIG = {\n \"IGNORE_PATHS_PATTERNS\": (\".*/api/keep-alive.*\",),\n \"ENABLE_STACKTRACES\": False\",\n}\n```\n\n### `IGNORE_SQL_PATTERNS`\n\nTakes a tuple of strings. Each string is a regular expression pattern.\nIf a SQL query matches any of the patterns it will be ignored and not\nshown in the requests list or request details.\n\nDefault: `()`\n\nExample:\n```python\nREQUESTS_TRACKER_CONFIG = {\n \"IGNORE_SQL_PATTERNS\": (\n r\"^SELECT .* FROM django_migrations WHERE app = 'requests_tracker'\",\n r\"^SELECT .* FROM django_migrations WHERE app = 'auth'\",\n ),\n}\n```\n\n### `IGNORE_PATHS_PATTERNS`\n\nTakes a tuple of strings. Each string is a regular expression pattern.\nIf a request path matches any of the patterns it will be ignored and not tracked.\n\nDefault: `()`\n\nExample:\n```python\nREQUESTS_TRACKER_CONFIG = {\n \"IGNORE_PATHS_PATTERNS\": (\n r\".*/api/keep-alive.*\",\n ),\n}\n```\n\n### `SQL_WARNING_THRESHOLD`\n\nRepresents the threshold in milliseconds after which a SQL query is considered slow and\nwill be marked with a warning label in the SQL list.\n\nDefault: `500` (500 milliseconds)\n\nExample:\n```python\nREQUESTS_TRACKER_CONFIG = {\n \"SQL_WARNING_THRESHOLD\": 50,\n}\n```\n\n### `ENABLE_STACKTRACES`\n\nIf set to `False` stacktraces will not be shown in the request details view.\n\nDefault: `True`\n\n### `HIDE_IN_STACKTRACES`\n\nTakes a tuple of strings. Each string represents a module name. If a module name is found\nin a stacktrace that part of the stacktrace will be hidden.\n\nDefault:\n```python\n(\n \"socketserver\",\n \"threading\",\n \"wsgiref\",\n \"requests_tracker\",\n \"django.db\",\n \"django.core.handlers\",\n \"django.core.servers\",\n \"django.utils.decorators\",\n \"django.utils.deprecation\",\n \"django.utils.functional\",\n)\n```\n\n### `TRACK_SQL`\n\nIf set to `False` SQL queries will not be tracked.\n\nDefault: `True`\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "The Django Requests Tracker is designed for local Django development, particularly for Rest API development. It provides various debugging information, such as SQL queries, headers, and status codes.",
"version": "0.3.3",
"project_urls": {
"Documentation": "https://github.com/bensi94/Django-Requests-Tracker/blob/main/README.md",
"Homepage": "https://github.com/bensi94/Django-Requests-Tracker",
"Repository": "https://github.com/bensi94/Django-Requests-Tracker"
},
"split_keywords": [
"django",
"python",
"sql",
"middlware",
"api",
"rest",
"debug",
"htmx",
"django-rest-framework",
"django-ninja",
"requests-tracker",
"django-requests-tracker",
"development"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "16f5d2fd9443c1839edf0c17216e9ab03201c16468e82e2968504fc738cd6917",
"md5": "0e252e489be16c4fa81d3abb60737de7",
"sha256": "31d8924470ceea34be51743142c5248f1bf625d2ff95d1f0dccc2cfe14ecda0b"
},
"downloads": -1,
"filename": "requests_tracker-0.3.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "0e252e489be16c4fa81d3abb60737de7",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4.0",
"size": 58078,
"upload_time": "2023-11-04T01:24:09",
"upload_time_iso_8601": "2023-11-04T01:24:09.924248Z",
"url": "https://files.pythonhosted.org/packages/16/f5/d2fd9443c1839edf0c17216e9ab03201c16468e82e2968504fc738cd6917/requests_tracker-0.3.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e91035d214c4eaa479251ebb6f774842e476cd4162ca939e72bb1d943131fb2c",
"md5": "ea210fd5cbf4eada9e8af8a27be50575",
"sha256": "eb288d69ebcae49149b41d603960d101d7eb892627e3455a456fa1f9441d2a49"
},
"downloads": -1,
"filename": "requests_tracker-0.3.3.tar.gz",
"has_sig": false,
"md5_digest": "ea210fd5cbf4eada9e8af8a27be50575",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4.0",
"size": 49168,
"upload_time": "2023-11-04T01:24:11",
"upload_time_iso_8601": "2023-11-04T01:24:11.992345Z",
"url": "https://files.pythonhosted.org/packages/e9/10/35d214c4eaa479251ebb6f774842e476cd4162ca939e72bb1d943131fb2c/requests_tracker-0.3.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-04 01:24:11",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "bensi94",
"github_project": "Django-Requests-Tracker",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "requests-tracker"
}
JSON
