# django-requestlogs
django-requestlogs is a package providing middleware and other helpers for audit logging.
The middleware collects information about request-response cycle into log entries. The
collected information can be fully customized, but the out-of-the-box implementation
includes
- user ID and username
- request (path, method, payload..)
- response (status code, payload..)
- general information, such as timestamp, execution time
Finally the log entry is stored in predefined storage, which by default is configurable
using Django's logging system.
Once installed, log storage should start showing entries such as the following:
```
{'action_name': None, 'execution_time': '00:00:00.024900', 'timestamp': '2019-07-01T07:05:34.217703Z', 'ip_address': None, 'request': OrderedDict([('method', 'GET'), ('full_path', '/'), ('data', '{}'), ('query_params', '{}')]), 'response': OrderedDict([('status_code', 200), ('data', '{"ok": true}')]), 'user': OrderedDict([('id', 1), ('username', 'admin')])}
```
*Note that to get IP address logged as well, the optional dependency `django-ipware` must be installed.*
## Motivation
django-requestlogs attempts to provide tools for implementing audit logging (audit trail)
to systems that require such feature. These systems typically must have the ability to
tell "what information the end-user has accessed (and what information was sent to the
system)?". django-requestlogs hooks into the Django REST framework in the simplest
way possible while logging every request without the need of remembering to enable it
for each view separately.
Currently django-requestlogs package is primarily focusing on working seamlessly with
Django REST framework. While plain Django requests are also collected, storing their request
and response payloads is not fully supported.
# Requirements
- Django (1.11, 2.0, 2.1, 2.2, 3.0, 3.1, 3.2, 4.0)
- Django REST framework
Optional dependencies:
- django-ipware
- if installed, this is used for storing end-user's IP address
# Installation
Install using `pip`:
pip install django-requestlogs
Add `'requestlogs.middleware.RequestLogsMiddleware'` to `MIDDLEWARE` settings.
```python
MIDDLEWARE = [
...
'requestlogs.middleware.RequestLogsMiddleware',
]
```
Set `'requestlogs.views.exception_handler'` as rest_framework's exception handler
(this will make sure requestlog entry has all possible data available about the
request in case of a 500 error):
```python
REST_FRAMEWORK={
...
'EXCEPTION_HANDLER': 'requestlogs.views.exception_handler',
}
```
The middleware is now ready to start storing requestlog entries using the default
`STORAGE_CLASS`, which in fact just uses Python logger named `requestlogs`. Now you can,
for example, redirect these logs to a file with the following `LOGGING` configuration:
```python
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'requestlogs_to_file': {
'level': 'INFO',
'class': 'logging.FileHandler',
'filename': '/tmp/requestlogs.log',
},
},
'loggers': {
'requestlogs': {
'handlers': ['requestlogs_to_file'],
'level': 'INFO',
'propagate': False,
},
},
}
```
# Settings
Requestlogs can be customized using Django settings. The following shows the default values for the available settings:
```python
REQUESTLOGS = {
'STORAGE_CLASS': 'requestlogs.storages.LoggingStorage',
'ENTRY_CLASS': 'requestlogs.entries.RequestLogEntry',
'SERIALIZER_CLASS': 'requestlogs.storages.BaseEntrySerializer',
'SECRETS': ['password', 'token'],
'ATTRIBUTE_NAME': '_requestlog',
'METHODS': ('GET', 'PUT', 'PATCH', 'POST', 'DELETE'),
'JSON_ENSURE_ASCII': True,
'IGNORE_USER_FIELD': None,
'IGNORE_USERS': [],
'IGNORE_PATHS': None,
}
```
- **STORAGE_CLASS**
- Path to the Python class which will handle storing the log entries. Override this if you only need to reimplement the storage mechanism. This may be the case e.g. when choosing what data to store.
- **ENTRY_CLASS**
- Path to the Python class which handles the construction of the complete requestlogs entry. Override this for full customization of the requestlog entry behaviour.
- **SERIALIZER_CLASS**
- Path to the serializer class which is used to serialize the requestlog entry before storage. By default this is a subclass of `rest_framework.serializers.Serializer`.
- **SECRETS**
- List of keys in request/response data which will be replaced with `'***'` in the stored entry.
- **ATTRIBUTE_NAME**
- django-requestlogs internally attaches the entry object to the Django request object, and uses this attribute name. Override if it causes collisions.
- **METHODS**
- django-requestlogs will handle only HTTP methods defined by this setting. By default it handles all HTTP methods.
- **JSON_ENSURE_ASCII**
- whether to dump the json data (of request and response) with `ensure_ascii=True/False`. Default is `True`. Use `False` to change it so that characters are displayed as-is.
- **IGNORE_USER_FIELD**
- ignore requests (that is, "do not store requestlogs") from users by the given user object field . E.g. `'email'`. Used in combination with `IGNORE_USERS`.
- **IGNORE_USERS**
- ignore requests from these users. E.g. if `IGNORE_USER_FIELD` is set to `'email'`, `IGNORE_USERS` can be list of emails: `['email@email1.com', 'email@email2.com']`.
- **IGNORE_PATHS**
- ignore requests to these paths. Can be one of the following:
- Function or callable, which takes one parameter (the request path) and returns `true` or `false` whether the path should be ignored.
- Path to a function or callable (e.g. `'my_utils.ignore_paths_func'`)
- List of paths to ignore. In addition to exact path matches, this supports simple wildcards (leading and trailing), and `re.Pattern` objects (typically created using `re.compile(r'^/foo')`). Example:
['/foo/', '/admin/*', '*/bar', re.compile(r'/baz/?')]
# Logging with Request ID
django-requestlogs also contains a middleware and logging helpers to associate a
request-specific identifier (uuid) to logging messages. This aims to help
distinguishing messages to certain request-response cycle, which can be useful
in an application that receives a high number of requests.
The request id is added to the standard logging messages (Django application logs)
by specifying a custom formatter and using the provided logging filter.
The request id can be stored to requestlog entries as well.
The middleware to enable the request id logging does not require the core requestlogs
middleware to be installed.
Under the hood the request id is implemented with help of `threading.local()`.
## Installation
The feature is enabled by adding `requestlogs.middleware.RequestIdMiddleware`
to the `MIDDLEWARE` setting:
```python
MIDDLEWARE = [
...
'requestlogs.middleware.RequestLogsMiddleware',
'requestlogs.middleware.RequestIdMiddleware',
]
```
Once installed, the application logs should start showing messages with a format such as
the following:
```
2019-07-18 11:56:07,261 INFO 954fb004fb404751a2fa33326101442c urls:31 Handling GET request
2019-07-18 11:56:07,262 DEBUG 954fb004fb404751a2fa33326101442c urls:32 No parameters given
2019-07-18 11:56:07,262 INFO 954fb004fb404751a2fa33326101442c urls:33 All good
```
To add the request id to requestlog entries as well, you can use the provided serializer
class as a starting point:
```python
REQUESTLOGS = {
...
'SERIALIZER_CLASS': 'requestlogs.storages.RequestIdEntrySerializer',
}
```
## Configuration
The middleware has some additional configuration possiblities:
```python
REQUESTLOGS = {
...
'REQUEST_ID_HTTP_HEADER': 'X_DJANGO_REQUEST_ID',
'REQUEST_ID_ATTRIBUTE_NAME': 'request_id',
}
```
- **REQUEST_ID_HTTP_HEADER**
- If set, the value of this request header is used as request id (instead of it being
randomly generated). This must be a valid uuid. One use case for this feature is in
microservice architecture, where a micreservice calls another, internal microservice.
Having the log messages of both applications to be formatted with same request id
might be the preferred outcome.
- **REQUEST_ID_ATTRIBUTE_NAME**
- The attribute name which is used internally to attach request id to
`threading.locals()`. Override if it causes collisions.
To add the request id to logging messages of your Django application, use the provided
logging filter and include `request_id` to the log formatter.
Here is the complete logging configuration:
```python
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'requestlogs_to_file': {
'level': 'INFO',
'class': 'logging.FileHandler',
'filename': '/tmp/requestlogs.log',
},
'root': {
'class': 'logging.StreamHandler',
'filters': ['request_id_context'],
'formatter': 'verbose',
},
},
'loggers': {
'': {
'handlers': ['root'],
'level': 'DEBUG',
},
'requestlogs': {
'handlers': ['requestlogs_to_file'],
'level': 'INFO',
'propagate': False,
},
},
'filters': {
'request_id_context': {
'()': 'requestlogs.logging.RequestIdContext',
},
},
'formatters': {
'verbose': {
'format': '%(asctime)s %(levelname)s %(request_id)s %(module)s:%(lineno)s %(message)s'
},
},
}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/Raekkeri/django-requestlogs",
"name": "django-requestlogs",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "django,log,logging",
"author": "Teemu Husso",
"author_email": "teemu.husso@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/74/34/5cd94ac485b1559e21d0213cfffa3f199dfd87472168fd4c46d2fa10ba7a/django-requestlogs-0.8.tar.gz",
"platform": null,
"description": "# django-requestlogs\n\n\n\n\ndjango-requestlogs is a package providing middleware and other helpers for audit logging.\nThe middleware collects information about request-response cycle into log entries. The\ncollected information can be fully customized, but the out-of-the-box implementation\nincludes\n\n- user ID and username\n- request (path, method, payload..)\n- response (status code, payload..)\n- general information, such as timestamp, execution time\n\nFinally the log entry is stored in predefined storage, which by default is configurable\nusing Django's logging system.\n\nOnce installed, log storage should start showing entries such as the following:\n\n```\n{'action_name': None, 'execution_time': '00:00:00.024900', 'timestamp': '2019-07-01T07:05:34.217703Z', 'ip_address': None, 'request': OrderedDict([('method', 'GET'), ('full_path', '/'), ('data', '{}'), ('query_params', '{}')]), 'response': OrderedDict([('status_code', 200), ('data', '{\"ok\": true}')]), 'user': OrderedDict([('id', 1), ('username', 'admin')])}\n```\n\n*Note that to get IP address logged as well, the optional dependency `django-ipware` must be installed.*\n\n## Motivation\n\ndjango-requestlogs attempts to provide tools for implementing audit logging (audit trail)\nto systems that require such feature. These systems typically must have the ability to\ntell \"what information the end-user has accessed (and what information was sent to the\nsystem)?\". django-requestlogs hooks into the Django REST framework in the simplest\nway possible while logging every request without the need of remembering to enable it\nfor each view separately.\n\nCurrently django-requestlogs package is primarily focusing on working seamlessly with\nDjango REST framework. While plain Django requests are also collected, storing their request\nand response payloads is not fully supported.\n\n# Requirements\n\n- Django (1.11, 2.0, 2.1, 2.2, 3.0, 3.1, 3.2, 4.0)\n- Django REST framework\n\nOptional dependencies:\n\n- django-ipware\n - if installed, this is used for storing end-user's IP address\n\n# Installation\n\nInstall using `pip`:\n\n pip install django-requestlogs\n\nAdd `'requestlogs.middleware.RequestLogsMiddleware'` to `MIDDLEWARE` settings.\n\n```python\nMIDDLEWARE = [\n ...\n 'requestlogs.middleware.RequestLogsMiddleware',\n]\n```\n\nSet `'requestlogs.views.exception_handler'` as rest_framework's exception handler\n(this will make sure requestlog entry has all possible data available about the\nrequest in case of a 500 error):\n\n```python\nREST_FRAMEWORK={\n ...\n 'EXCEPTION_HANDLER': 'requestlogs.views.exception_handler',\n}\n```\n\nThe middleware is now ready to start storing requestlog entries using the default\n`STORAGE_CLASS`, which in fact just uses Python logger named `requestlogs`. Now you can,\nfor example, redirect these logs to a file with the following `LOGGING` configuration:\n\n```python\nLOGGING = {\n 'version': 1,\n 'disable_existing_loggers': False,\n 'handlers': {\n 'requestlogs_to_file': {\n 'level': 'INFO',\n 'class': 'logging.FileHandler',\n 'filename': '/tmp/requestlogs.log',\n },\n },\n 'loggers': {\n 'requestlogs': {\n 'handlers': ['requestlogs_to_file'],\n 'level': 'INFO',\n 'propagate': False,\n },\n },\n}\n```\n\n# Settings\n\nRequestlogs can be customized using Django settings. The following shows the default values for the available settings:\n\n```python\nREQUESTLOGS = {\n 'STORAGE_CLASS': 'requestlogs.storages.LoggingStorage',\n 'ENTRY_CLASS': 'requestlogs.entries.RequestLogEntry',\n 'SERIALIZER_CLASS': 'requestlogs.storages.BaseEntrySerializer',\n 'SECRETS': ['password', 'token'],\n 'ATTRIBUTE_NAME': '_requestlog',\n 'METHODS': ('GET', 'PUT', 'PATCH', 'POST', 'DELETE'),\n 'JSON_ENSURE_ASCII': True,\n 'IGNORE_USER_FIELD': None,\n 'IGNORE_USERS': [],\n 'IGNORE_PATHS': None,\n}\n```\n\n- **STORAGE_CLASS**\n - Path to the Python class which will handle storing the log entries. Override this if you only need to reimplement the storage mechanism. This may be the case e.g. when choosing what data to store.\n- **ENTRY_CLASS**\n - Path to the Python class which handles the construction of the complete requestlogs entry. Override this for full customization of the requestlog entry behaviour.\n- **SERIALIZER_CLASS**\n - Path to the serializer class which is used to serialize the requestlog entry before storage. By default this is a subclass of `rest_framework.serializers.Serializer`.\n- **SECRETS**\n - List of keys in request/response data which will be replaced with `'***'` in the stored entry.\n- **ATTRIBUTE_NAME**\n - django-requestlogs internally attaches the entry object to the Django request object, and uses this attribute name. Override if it causes collisions.\n- **METHODS**\n - django-requestlogs will handle only HTTP methods defined by this setting. By default it handles all HTTP methods.\n- **JSON_ENSURE_ASCII**\n - whether to dump the json data (of request and response) with `ensure_ascii=True/False`. Default is `True`. Use `False` to change it so that characters are displayed as-is.\n- **IGNORE_USER_FIELD**\n - ignore requests (that is, \"do not store requestlogs\") from users by the given user object field . E.g. `'email'`. Used in combination with `IGNORE_USERS`.\n- **IGNORE_USERS**\n - ignore requests from these users. E.g. if `IGNORE_USER_FIELD` is set to `'email'`, `IGNORE_USERS` can be list of emails: `['email@email1.com', 'email@email2.com']`.\n- **IGNORE_PATHS**\n - ignore requests to these paths. Can be one of the following:\n - Function or callable, which takes one parameter (the request path) and returns `true` or `false` whether the path should be ignored.\n - Path to a function or callable (e.g. `'my_utils.ignore_paths_func'`)\n - List of paths to ignore. In addition to exact path matches, this supports simple wildcards (leading and trailing), and `re.Pattern` objects (typically created using `re.compile(r'^/foo')`). Example:\n\n ['/foo/', '/admin/*', '*/bar', re.compile(r'/baz/?')]\n\n\n# Logging with Request ID\n\ndjango-requestlogs also contains a middleware and logging helpers to associate a\nrequest-specific identifier (uuid) to logging messages. This aims to help\ndistinguishing messages to certain request-response cycle, which can be useful\nin an application that receives a high number of requests.\n\nThe request id is added to the standard logging messages (Django application logs)\nby specifying a custom formatter and using the provided logging filter.\nThe request id can be stored to requestlog entries as well.\nThe middleware to enable the request id logging does not require the core requestlogs\nmiddleware to be installed.\n\nUnder the hood the request id is implemented with help of `threading.local()`.\n\n## Installation\n\nThe feature is enabled by adding `requestlogs.middleware.RequestIdMiddleware`\nto the `MIDDLEWARE` setting:\n\n```python\nMIDDLEWARE = [\n ...\n 'requestlogs.middleware.RequestLogsMiddleware',\n 'requestlogs.middleware.RequestIdMiddleware',\n]\n```\n\nOnce installed, the application logs should start showing messages with a format such as\nthe following:\n\n```\n2019-07-18 11:56:07,261 INFO 954fb004fb404751a2fa33326101442c urls:31 Handling GET request\n2019-07-18 11:56:07,262 DEBUG 954fb004fb404751a2fa33326101442c urls:32 No parameters given\n2019-07-18 11:56:07,262 INFO 954fb004fb404751a2fa33326101442c urls:33 All good\n```\n\nTo add the request id to requestlog entries as well, you can use the provided serializer\nclass as a starting point:\n\n```python\nREQUESTLOGS = {\n ...\n 'SERIALIZER_CLASS': 'requestlogs.storages.RequestIdEntrySerializer',\n}\n```\n\n## Configuration\n\nThe middleware has some additional configuration possiblities:\n\n\n```python\nREQUESTLOGS = {\n ...\n 'REQUEST_ID_HTTP_HEADER': 'X_DJANGO_REQUEST_ID',\n 'REQUEST_ID_ATTRIBUTE_NAME': 'request_id',\n}\n```\n- **REQUEST_ID_HTTP_HEADER**\n - If set, the value of this request header is used as request id (instead of it being\n randomly generated). This must be a valid uuid. One use case for this feature is in\n microservice architecture, where a micreservice calls another, internal microservice.\n Having the log messages of both applications to be formatted with same request id\n might be the preferred outcome.\n- **REQUEST_ID_ATTRIBUTE_NAME**\n - The attribute name which is used internally to attach request id to\n `threading.locals()`. Override if it causes collisions.\n\nTo add the request id to logging messages of your Django application, use the provided\nlogging filter and include `request_id` to the log formatter.\nHere is the complete logging configuration:\n\n```python\nLOGGING = {\n 'version': 1,\n 'disable_existing_loggers': False,\n 'handlers': {\n 'requestlogs_to_file': {\n 'level': 'INFO',\n 'class': 'logging.FileHandler',\n 'filename': '/tmp/requestlogs.log',\n },\n 'root': {\n 'class': 'logging.StreamHandler',\n 'filters': ['request_id_context'],\n 'formatter': 'verbose',\n },\n },\n 'loggers': {\n '': {\n 'handlers': ['root'],\n 'level': 'DEBUG',\n },\n 'requestlogs': {\n 'handlers': ['requestlogs_to_file'],\n 'level': 'INFO',\n 'propagate': False,\n },\n },\n 'filters': {\n 'request_id_context': {\n '()': 'requestlogs.logging.RequestIdContext',\n },\n },\n 'formatters': {\n 'verbose': {\n 'format': '%(asctime)s %(levelname)s %(request_id)s %(module)s:%(lineno)s %(message)s'\n },\n },\n}\n```\n",
"bugtrack_url": null,
"license": "",
"summary": "Audit logging for Django and Django Rest Framework",
"version": "0.8",
"project_urls": {
"Download": "https://github.com/raekkeri/django-requestlogs/tarball/0.8",
"Homepage": "https://github.com/Raekkeri/django-requestlogs"
},
"split_keywords": [
"django",
"log",
"logging"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "664a3b2adad8730cf2e47a6ffe573d5c7697ee63b665a474e8d80e32e3280be1",
"md5": "24404136e3ae4c18e363b3c808e6b4af",
"sha256": "5bebb80f401a948184a84a26db2dede72faecbafd96687110d5921e78c72d916"
},
"downloads": -1,
"filename": "django_requestlogs-0.8-py3-none-any.whl",
"has_sig": false,
"md5_digest": "24404136e3ae4c18e363b3c808e6b4af",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 11568,
"upload_time": "2024-01-03T19:43:28",
"upload_time_iso_8601": "2024-01-03T19:43:28.530077Z",
"url": "https://files.pythonhosted.org/packages/66/4a/3b2adad8730cf2e47a6ffe573d5c7697ee63b665a474e8d80e32e3280be1/django_requestlogs-0.8-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "74345cd94ac485b1559e21d0213cfffa3f199dfd87472168fd4c46d2fa10ba7a",
"md5": "bae9c6c626e1add26031bb4a35f79ac0",
"sha256": "727373fd1eaae7f63450aa65fd301ac243bb1bedc317d9d54389f60af92e1b5a"
},
"downloads": -1,
"filename": "django-requestlogs-0.8.tar.gz",
"has_sig": false,
"md5_digest": "bae9c6c626e1add26031bb4a35f79ac0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 14380,
"upload_time": "2024-01-03T19:43:30",
"upload_time_iso_8601": "2024-01-03T19:43:30.235458Z",
"url": "https://files.pythonhosted.org/packages/74/34/5cd94ac485b1559e21d0213cfffa3f199dfd87472168fd4c46d2fa10ba7a/django-requestlogs-0.8.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-03 19:43:30",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Raekkeri",
"github_project": "django-requestlogs",
"travis_ci": true,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "django-requestlogs"
}
JSON
