=====================
JSON log formatter 🪵
=====================
The library helps you to store logs in JSON format. Why is it important?
Well, it facilitates integration with **Logstash**.
Usage example:
.. code-block:: python
import logging
import json_log_formatter
formatter = json_log_formatter.JSONFormatter()
json_handler = logging.FileHandler(filename='/var/log/my-log.json')
json_handler.setFormatter(formatter)
logger = logging.getLogger('my_json')
logger.addHandler(json_handler)
logger.setLevel(logging.INFO)
logger.info('Sign up', extra={'referral_code': '52d6ce'})
try:
raise ValueError('something wrong')
except ValueError:
logger.error('Request failed', exc_info=True)
The log file will contain the following log record (inline).
.. code-block:: json
{
"message": "Sign up",
"time": "2015-09-01T06:06:26.524448",
"referral_code": "52d6ce"
}
{
"message": "Request failed",
"time": "2015-09-01T06:06:26.524449",
"exc_info": "Traceback (most recent call last): ..."
}
If you use a log collection and analysis system,
you might need to include the built-in
`log record attributes <https://docs.python.org/3/library/logging.html#logrecord-attributes>`_
with ``VerboseJSONFormatter``.
.. code-block:: python
json_handler.setFormatter(json_log_formatter.VerboseJSONFormatter())
logger.error('An error has occured')
.. code-block:: json
{
"filename": "tests.py",
"funcName": "test_file_name_is_testspy",
"levelname": "ERROR",
"lineno": 276,
"module": "tests",
"name": "my_json",
"pathname": "/Users/bob/json-log-formatter/tests.py",
"process": 3081,
"processName": "MainProcess",
"stack_info": null,
"thread": 4664270272,
"threadName": "MainThread",
"message": "An error has occured",
"time": "2021-07-04T21:05:42.767726"
}
If you need to flatten complex objects as strings, use ``FlatJSONFormatter``.
.. code-block:: python
json_handler.setFormatter(json_log_formatter.FlatJSONFormatter())
logger.error('An error has occured')
logger.info('Sign up', extra={'request': WSGIRequest({
'PATH_INFO': 'bogus',
'REQUEST_METHOD': 'bogus',
'CONTENT_TYPE': 'text/html; charset=utf8',
'wsgi.input': BytesIO(b''),
})})
.. code-block:: json
{
"message": "Sign up",
"time": "2024-10-01T00:59:29.332888+00:00",
"request": "<WSGIRequest: BOGUS '/bogus'>"
}
JSON libraries
--------------
You can use **ujson** or **simplejson** instead of built-in **json** library.
.. code-block:: python
import json_log_formatter
import ujson
formatter = json_log_formatter.JSONFormatter()
formatter.json_lib = ujson
Note, **ujson** doesn't support ``dumps(default=f)`` argument:
if it can't serialize an attribute, it might fail with ``TypeError`` or skip an attribute.
Django integration
------------------
Here is an example of how the JSON formatter can be used with Django.
.. code-block:: python
LOGGING['formatters']['json'] = {
'()': 'json_log_formatter.JSONFormatter',
}
LOGGING['handlers']['json_file'] = {
'level': 'INFO',
'class': 'logging.FileHandler',
'filename': '/var/log/my-log.json',
'formatter': 'json',
}
LOGGING['loggers']['my_json'] = {
'handlers': ['json_file'],
'level': 'INFO',
}
Let's try to log something.
.. code-block:: python
import logging
logger = logging.getLogger('my_json')
logger.info('Sign up', extra={'referral_code': '52d6ce'})
Custom formatter
----------------
You will likely need a custom log formatter. For instance, you want to log
a user ID, an IP address and ``time`` as ``django.utils.timezone.now()``.
To do so you should override ``JSONFormatter.json_record()``.
.. code-block:: python
class CustomisedJSONFormatter(json_log_formatter.JSONFormatter):
def json_record(self, message: str, extra: dict, record: logging.LogRecord) -> dict:
extra['message'] = message
extra['user_id'] = current_user_id()
extra['ip'] = current_ip()
# Include builtins
extra['level'] = record.levelname
extra['name'] = record.name
if 'time' not in extra:
extra['time'] = django.utils.timezone.now()
if record.exc_info:
extra['exc_info'] = self.formatException(record.exc_info)
return extra
Let's say you want ``datetime`` to be serialized as timestamp.
You can use **ujson** (which does it by default) and disable
ISO8601 date mutation.
.. code-block:: python
class CustomisedJSONFormatter(json_log_formatter.JSONFormatter):
json_lib = ujson
def mutate_json_record(self, json_record):
return json_record
Tests
-----
.. code-block:: console
$ pip install -r requirements.txt
$ tox
Raw data
{
"_id": null,
"home_page": "https://github.com/marselester/json-log-formatter",
"name": "JSON-log-formatter",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": null,
"author": "Marsel Mavletkulov",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/67/34/02eee63c3871b9f3ea340f58d0675dae8d9cc95a8a961f379f5a1b325911/json_log_formatter-1.1.tar.gz",
"platform": null,
"description": "=====================\nJSON log formatter \ud83e\udeb5\n=====================\n\nThe library helps you to store logs in JSON format. Why is it important?\nWell, it facilitates integration with **Logstash**.\n\nUsage example:\n\n.. code-block:: python\n\n import logging\n\n import json_log_formatter\n\n formatter = json_log_formatter.JSONFormatter()\n\n json_handler = logging.FileHandler(filename='/var/log/my-log.json')\n json_handler.setFormatter(formatter)\n\n logger = logging.getLogger('my_json')\n logger.addHandler(json_handler)\n logger.setLevel(logging.INFO)\n\n logger.info('Sign up', extra={'referral_code': '52d6ce'})\n\n try:\n raise ValueError('something wrong')\n except ValueError:\n logger.error('Request failed', exc_info=True)\n\nThe log file will contain the following log record (inline).\n\n.. code-block:: json\n\n {\n \"message\": \"Sign up\",\n \"time\": \"2015-09-01T06:06:26.524448\",\n \"referral_code\": \"52d6ce\"\n }\n {\n \"message\": \"Request failed\",\n \"time\": \"2015-09-01T06:06:26.524449\",\n \"exc_info\": \"Traceback (most recent call last): ...\"\n }\n\nIf you use a log collection and analysis system,\nyou might need to include the built-in\n`log record attributes <https://docs.python.org/3/library/logging.html#logrecord-attributes>`_\nwith ``VerboseJSONFormatter``.\n\n.. code-block:: python\n\n json_handler.setFormatter(json_log_formatter.VerboseJSONFormatter())\n logger.error('An error has occured')\n\n.. code-block:: json\n\n {\n \"filename\": \"tests.py\",\n \"funcName\": \"test_file_name_is_testspy\",\n \"levelname\": \"ERROR\",\n \"lineno\": 276,\n \"module\": \"tests\",\n \"name\": \"my_json\",\n \"pathname\": \"/Users/bob/json-log-formatter/tests.py\",\n \"process\": 3081,\n \"processName\": \"MainProcess\",\n \"stack_info\": null,\n \"thread\": 4664270272,\n \"threadName\": \"MainThread\",\n \"message\": \"An error has occured\",\n \"time\": \"2021-07-04T21:05:42.767726\"\n }\n\nIf you need to flatten complex objects as strings, use ``FlatJSONFormatter``.\n\n.. code-block:: python\n\n json_handler.setFormatter(json_log_formatter.FlatJSONFormatter())\n logger.error('An error has occured')\n\n logger.info('Sign up', extra={'request': WSGIRequest({\n 'PATH_INFO': 'bogus',\n 'REQUEST_METHOD': 'bogus',\n 'CONTENT_TYPE': 'text/html; charset=utf8',\n 'wsgi.input': BytesIO(b''),\n })})\n\n.. code-block:: json\n\n {\n \"message\": \"Sign up\",\n \"time\": \"2024-10-01T00:59:29.332888+00:00\",\n \"request\": \"<WSGIRequest: BOGUS '/bogus'>\"\n }\n\nJSON libraries\n--------------\n\nYou can use **ujson** or **simplejson** instead of built-in **json** library.\n\n.. code-block:: python\n\n import json_log_formatter\n import ujson\n\n formatter = json_log_formatter.JSONFormatter()\n formatter.json_lib = ujson\n\nNote, **ujson** doesn't support ``dumps(default=f)`` argument:\nif it can't serialize an attribute, it might fail with ``TypeError`` or skip an attribute.\n\nDjango integration\n------------------\n\nHere is an example of how the JSON formatter can be used with Django.\n\n.. code-block:: python\n\n LOGGING['formatters']['json'] = {\n '()': 'json_log_formatter.JSONFormatter',\n }\n LOGGING['handlers']['json_file'] = {\n 'level': 'INFO',\n 'class': 'logging.FileHandler',\n 'filename': '/var/log/my-log.json',\n 'formatter': 'json',\n }\n LOGGING['loggers']['my_json'] = {\n 'handlers': ['json_file'],\n 'level': 'INFO',\n }\n\nLet's try to log something.\n\n.. code-block:: python\n\n import logging\n\n logger = logging.getLogger('my_json')\n\n logger.info('Sign up', extra={'referral_code': '52d6ce'})\n\nCustom formatter\n----------------\n\nYou will likely need a custom log formatter. For instance, you want to log\na user ID, an IP address and ``time`` as ``django.utils.timezone.now()``.\nTo do so you should override ``JSONFormatter.json_record()``.\n\n.. code-block:: python\n\n class CustomisedJSONFormatter(json_log_formatter.JSONFormatter):\n def json_record(self, message: str, extra: dict, record: logging.LogRecord) -> dict:\n extra['message'] = message\n extra['user_id'] = current_user_id()\n extra['ip'] = current_ip()\n\n # Include builtins\n extra['level'] = record.levelname\n extra['name'] = record.name\n\n if 'time' not in extra:\n extra['time'] = django.utils.timezone.now()\n\n if record.exc_info:\n extra['exc_info'] = self.formatException(record.exc_info)\n\n return extra\n\nLet's say you want ``datetime`` to be serialized as timestamp.\nYou can use **ujson** (which does it by default) and disable\nISO8601 date mutation.\n\n.. code-block:: python\n\n class CustomisedJSONFormatter(json_log_formatter.JSONFormatter):\n json_lib = ujson\n\n def mutate_json_record(self, json_record):\n return json_record\n\nTests\n-----\n\n.. code-block:: console\n\n $ pip install -r requirements.txt\n $ tox\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "JSON log formatter",
"version": "1.1",
"project_urls": {
"Homepage": "https://github.com/marselester/json-log-formatter",
"repository": "https://github.com/marselester/json-log-formatter"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "673402eee63c3871b9f3ea340f58d0675dae8d9cc95a8a961f379f5a1b325911",
"md5": "683319090698cebd39b4e88826c4c0b0",
"sha256": "fe8fd801c58c1234df86211720921f60149105ef8d1e2a72966bb61da9bed584"
},
"downloads": -1,
"filename": "json_log_formatter-1.1.tar.gz",
"has_sig": false,
"md5_digest": "683319090698cebd39b4e88826c4c0b0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 5858,
"upload_time": "2024-10-02T01:15:50",
"upload_time_iso_8601": "2024-10-02T01:15:50.688704Z",
"url": "https://files.pythonhosted.org/packages/67/34/02eee63c3871b9f3ea340f58d0675dae8d9cc95a8a961f379f5a1b325911/json_log_formatter-1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-02 01:15:50",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "marselester",
"github_project": "json-log-formatter",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"tox": true,
"lcname": "json-log-formatter"
}
JSON
