pygelf
======
|travis| |coveralls| |pypi| |downloads|
.. |travis| image:: https://travis-ci.org/keeprocking/pygelf.svg?branch=master
:target: https://travis-ci.org/keeprocking/pygelf
.. |pypi| image:: https://badge.fury.io/py/pygelf.svg
:target: https://pypi.python.org/pypi/pygelf
.. |coveralls| image:: https://coveralls.io/repos/github/keeprocking/pygelf/badge.svg?branch=master
:target: https://coveralls.io/github/keeprocking/pygelf?branch=master
.. |downloads| image:: https://img.shields.io/pypi/dm/pygelf.svg
:target: https://pypi.python.org/pypi/pygelf
Python logging handlers with GELF (Graylog Extended Log Format) support.
Currently TCP, UDP, TLS (encrypted TCP) and HTTP logging handlers are supported.
Get pygelf
==========
.. code:: python
pip install pygelf
Usage
=====
.. code:: python
from pygelf import GelfTcpHandler, GelfUdpHandler, GelfTlsHandler, GelfHttpHandler
import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()
logger.addHandler(GelfTcpHandler(host='127.0.0.1', port=9401))
logger.addHandler(GelfUdpHandler(host='127.0.0.1', port=9402))
logger.addHandler(GelfTlsHandler(host='127.0.0.1', port=9403))
logger.addHandler(GelfHttpHandler(host='127.0.0.1', port=9404))
logger.info('hello gelf')
Message structure
=================
According to the GELF spec, each message has the following mandatory fields:
- **version**: '1.1', can be overridden when creating a logger
- **short_message**: the log message itself
- **timestamp**: current timestamp
- **level**: syslog-compliant_ log level number (e.g. WARNING will be sent as 4)
- **host**: hostname of the machine that sent the message
- **full_message**: this field contains stack trace and is being written **ONLY** when logging an exception, e.g.
.. code:: python
try:
1/0
except ZeroDivisionError as e:
logger.exception(e)
.. _syslog-compliant: https://en.wikipedia.org/wiki/Syslog#Severity_level
In debug mode (when handler was created with debug=True option) each message contains some extra fields (which are pretty self-explanatory):
- **_file**
- **_line**
- **_module**
- **_func**
- **_logger_name**
Configuration
=============
Each handler has the following parameters:
- **host**: IP address of the GELF input
- **port**: port of the GELF input
- **debug** (False by default): if true, each log message will include debugging info: module name, file name, line number, method name
- **version** ('1.1' by default): GELF protocol version, can be overridden
- **include_extra_fields** (False by default): if true, each log message will include all the extra fields set to LogRecord
- **json_default** (:code:`str` with exception for several :code:`datetime` objects): function that is called for objects that cannot be serialized to JSON natively by python. Default implementation is custom function that returns result of :code:`isoformat()` method for :code:`datetime.datetime`, :code:`datetime.time`, :code:`datetime.date` objects and result of :code:`str(obj)` call for other objects (which is string representation of an object with fallback to :code:`repr`)
Also, there are some handler-specific parameters.
UDP:
- **chunk\_size** (1300 by default) - maximum length of the message. If log length exceeds this value, it splits into multiple chunks (see https://www.graylog.org/resources/gelf/ section "chunked GELF") with the length equals to this value. This parameter must be less than the MTU_. If the logs don't seem to be delivered, try to reduce this value.
- **compress** (True by default) - if true, compress log messages before sending them to the server
.. _MTU: https://en.wikipedia.org/wiki/Maximum_transmission_unit
TLS:
- **validate** (False by default) - if true, validate server certificate. If server provides a certificate that doesn't exist in **ca_certs**, you won't be able to send logs over TLS
- **ca_certs** (None by default) - path to CA bundle file. This parameter is required if **validate** is true.
- **certfile** (None by default) - path to certificate file that will be used to identify ourselves to the remote endpoint. This is necessary when the remote server has client authentication required. If **certfile** contains the private key, it should be placed before the certificate.
- **keyfile** (None by default) - path to the private key. If the private key is stored in **certfile** this parameter can be None.
HTTP:
- **compress** (True by default) - if true, compress log messages before sending them to the server
- **path** ('/gelf' by default) - path of the HTTP input (http://docs.graylog.org/en/latest/pages/sending_data.html#gelf-via-http)
- **timeout** (5 by default) - amount of seconds that HTTP client should wait before it discards the request if the server doesn't respond
Static fields
=============
If you need to include some static fields into your logs, simply pass them to the handler constructor. Each additional field should start with underscore. You can't add field '\_id'.
Example:
.. code:: python
handler = GelfUdpHandler(host='127.0.0.1', port=9402, _app_name='pygelf', _something=11)
logger.addHandler(handler)
Dynamic fields
==============
If you need to include some dynamic fields into your logs, add them to record by using LoggingAdapter or logging.Filter and create handler with include_extra_fields set to True.
All the non-trivial fields of the record will be sent to graylog2 with '\_' added before the name
Example:
.. code:: python
class ContextFilter(logging.Filter):
def filter(self, record):
record.job_id = threading.local().process_id
return True
logger.addFilter(ContextFilter())
handler = GelfUdpHandler(host='127.0.0.1', port=9402, include_extra_fields=True)
logger.addHandler(handler)
Defining fields from environment
================================
If you need to include some fields from the environment into your logs, add them to record by using `additional_env_fields`.
The following example will add an `env` field to the logs, taking its value from the environment variable `FLASK_ENV`.
.. code:: python
handler = GelfTcpHandler(host='127.0.0.1', port=9402, include_extra_fields=True, additional_env_fields={env: 'FLASK_ENV'})
logger.addHandler(handler)
The following can also be used in defining logging from configuration files (yaml/ini):
.. code:: ini
[formatters]
keys=standard
[formatter_standard]
class=logging.Formatter
format=%(message)s
[handlers]
keys=graylog
[handler_graylog]
class=pygelf.GelfTcpHandler
formatter=standard
args=('127.0.0.1', '12201')
kwargs={'include_extra_fields': True, 'debug': True, 'additional_env_fields': {'env': 'FLASK_ENV'}}
[loggers]
keys=root
[logger_root]
level=WARN
handlers=graylog
Running tests
=============
To run tests, you'll need tox_. After installing, simply run it:
.. code::
tox
.. _tox: https://pypi.python.org/pypi/tox
Raw data
{
"_id": null,
"home_page": "https://github.com/keeprocking/pygelf",
"name": "pygelf",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "logging udp tcp ssl tls graylog2 graylog gelf",
"author": "Ivan Mukhin",
"author_email": "muhin.ivan@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/fe/d3/73d1fe74a156f9a0e519bedc87815ed309e64af19c73b94352e4c0959ddb/pygelf-0.4.2.tar.gz",
"platform": "",
"description": "pygelf\n======\n|travis| |coveralls| |pypi| |downloads|\n\n.. |travis| image:: https://travis-ci.org/keeprocking/pygelf.svg?branch=master\n :target: https://travis-ci.org/keeprocking/pygelf\n.. |pypi| image:: https://badge.fury.io/py/pygelf.svg\n :target: https://pypi.python.org/pypi/pygelf\n.. |coveralls| image:: https://coveralls.io/repos/github/keeprocking/pygelf/badge.svg?branch=master\n :target: https://coveralls.io/github/keeprocking/pygelf?branch=master\n.. |downloads| image:: https://img.shields.io/pypi/dm/pygelf.svg\n :target: https://pypi.python.org/pypi/pygelf\n\n\nPython logging handlers with GELF (Graylog Extended Log Format) support.\n\nCurrently TCP, UDP, TLS (encrypted TCP) and HTTP logging handlers are supported.\n\nGet pygelf\n==========\n.. code:: python\n\n pip install pygelf\n\nUsage\n=====\n\n.. code:: python\n\n from pygelf import GelfTcpHandler, GelfUdpHandler, GelfTlsHandler, GelfHttpHandler\n import logging\n\n\n logging.basicConfig(level=logging.INFO)\n logger = logging.getLogger()\n logger.addHandler(GelfTcpHandler(host='127.0.0.1', port=9401))\n logger.addHandler(GelfUdpHandler(host='127.0.0.1', port=9402))\n logger.addHandler(GelfTlsHandler(host='127.0.0.1', port=9403))\n logger.addHandler(GelfHttpHandler(host='127.0.0.1', port=9404))\n\n logger.info('hello gelf')\n\nMessage structure\n=================\n\nAccording to the GELF spec, each message has the following mandatory fields:\n\n- **version**: '1.1', can be overridden when creating a logger\n- **short_message**: the log message itself\n- **timestamp**: current timestamp\n- **level**: syslog-compliant_ log level number (e.g. WARNING will be sent as 4)\n- **host**: hostname of the machine that sent the message\n- **full_message**: this field contains stack trace and is being written **ONLY** when logging an exception, e.g.\n\n.. code:: python\n\n try:\n 1/0\n except ZeroDivisionError as e:\n logger.exception(e)\n\n.. _syslog-compliant: https://en.wikipedia.org/wiki/Syslog#Severity_level\n\nIn debug mode (when handler was created with debug=True option) each message contains some extra fields (which are pretty self-explanatory): \n\n- **_file**\n- **_line**\n- **_module**\n- **_func**\n- **_logger_name**\n\nConfiguration\n=============\n\nEach handler has the following parameters:\n\n- **host**: IP address of the GELF input\n- **port**: port of the GELF input\n- **debug** (False by default): if true, each log message will include debugging info: module name, file name, line number, method name\n- **version** ('1.1' by default): GELF protocol version, can be overridden\n- **include_extra_fields** (False by default): if true, each log message will include all the extra fields set to LogRecord\n- **json_default** (:code:`str` with exception for several :code:`datetime` objects): function that is called for objects that cannot be serialized to JSON natively by python. Default implementation is custom function that returns result of :code:`isoformat()` method for :code:`datetime.datetime`, :code:`datetime.time`, :code:`datetime.date` objects and result of :code:`str(obj)` call for other objects (which is string representation of an object with fallback to :code:`repr`)\n\nAlso, there are some handler-specific parameters.\n\nUDP:\n\n- **chunk\\_size** (1300 by default) - maximum length of the message. If log length exceeds this value, it splits into multiple chunks (see https://www.graylog.org/resources/gelf/ section \"chunked GELF\") with the length equals to this value. This parameter must be less than the MTU_. If the logs don't seem to be delivered, try to reduce this value.\n- **compress** (True by default) - if true, compress log messages before sending them to the server\n\n.. _MTU: https://en.wikipedia.org/wiki/Maximum_transmission_unit\n\nTLS:\n\n- **validate** (False by default) - if true, validate server certificate. If server provides a certificate that doesn't exist in **ca_certs**, you won't be able to send logs over TLS\n- **ca_certs** (None by default) - path to CA bundle file. This parameter is required if **validate** is true.\n- **certfile** (None by default) - path to certificate file that will be used to identify ourselves to the remote endpoint. This is necessary when the remote server has client authentication required. If **certfile** contains the private key, it should be placed before the certificate.\n- **keyfile** (None by default) - path to the private key. If the private key is stored in **certfile** this parameter can be None.\n\nHTTP:\n\n- **compress** (True by default) - if true, compress log messages before sending them to the server\n- **path** ('/gelf' by default) - path of the HTTP input (http://docs.graylog.org/en/latest/pages/sending_data.html#gelf-via-http)\n- **timeout** (5 by default) - amount of seconds that HTTP client should wait before it discards the request if the server doesn't respond\n\nStatic fields\n=============\n\nIf you need to include some static fields into your logs, simply pass them to the handler constructor. Each additional field should start with underscore. You can't add field '\\_id'.\n\nExample:\n\n.. code:: python\n\n handler = GelfUdpHandler(host='127.0.0.1', port=9402, _app_name='pygelf', _something=11)\n logger.addHandler(handler)\n\nDynamic fields\n==============\n\nIf you need to include some dynamic fields into your logs, add them to record by using LoggingAdapter or logging.Filter and create handler with include_extra_fields set to True.\nAll the non-trivial fields of the record will be sent to graylog2 with '\\_' added before the name\n\nExample:\n\n.. code:: python\n\n class ContextFilter(logging.Filter):\n\n def filter(self, record):\n record.job_id = threading.local().process_id\n return True\n\n logger.addFilter(ContextFilter())\n handler = GelfUdpHandler(host='127.0.0.1', port=9402, include_extra_fields=True)\n logger.addHandler(handler)\n\nDefining fields from environment\n================================\n\nIf you need to include some fields from the environment into your logs, add them to record by using `additional_env_fields`.\n\nThe following example will add an `env` field to the logs, taking its value from the environment variable `FLASK_ENV`.\n\n.. code:: python\n\n handler = GelfTcpHandler(host='127.0.0.1', port=9402, include_extra_fields=True, additional_env_fields={env: 'FLASK_ENV'})\n logger.addHandler(handler)\n\nThe following can also be used in defining logging from configuration files (yaml/ini):\n\n.. code:: ini\n\n [formatters]\n keys=standard\n\n [formatter_standard]\n class=logging.Formatter\n format=%(message)s\n\n [handlers]\n keys=graylog\n\n [handler_graylog]\n class=pygelf.GelfTcpHandler\n formatter=standard\n args=('127.0.0.1', '12201')\n kwargs={'include_extra_fields': True, 'debug': True, 'additional_env_fields': {'env': 'FLASK_ENV'}}\n\n [loggers]\n keys=root\n\n [logger_root]\n level=WARN\n handlers=graylog\n\nRunning tests\n=============\n\nTo run tests, you'll need tox_. After installing, simply run it:\n\n.. code::\n\n tox\n\n.. _tox: https://pypi.python.org/pypi/tox\n\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Logging handlers with GELF support",
"version": "0.4.2",
"project_urls": {
"Homepage": "https://github.com/keeprocking/pygelf"
},
"split_keywords": [
"logging",
"udp",
"tcp",
"ssl",
"tls",
"graylog2",
"graylog",
"gelf"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "03cd4afdddbc73f54ddf31d16137ef81c3d47192d75754b3115d925926081fd6",
"md5": "2e2a9738e3ec603a42589a35e06e6538",
"sha256": "ab57d1b26bffa014e29ae645ee51d2aa2f0c0cb419c522f2d24a237090b894a1"
},
"downloads": -1,
"filename": "pygelf-0.4.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2e2a9738e3ec603a42589a35e06e6538",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 8714,
"upload_time": "2021-10-06T23:32:03",
"upload_time_iso_8601": "2021-10-06T23:32:03.156888Z",
"url": "https://files.pythonhosted.org/packages/03/cd/4afdddbc73f54ddf31d16137ef81c3d47192d75754b3115d925926081fd6/pygelf-0.4.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "fed373d1fe74a156f9a0e519bedc87815ed309e64af19c73b94352e4c0959ddb",
"md5": "0e83829cddb7764203eab19832e29b00",
"sha256": "d0bb8f45ff648a9a187713f4a05c09f685fcb8add7b04bb7471f20071bd11aad"
},
"downloads": -1,
"filename": "pygelf-0.4.2.tar.gz",
"has_sig": false,
"md5_digest": "0e83829cddb7764203eab19832e29b00",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 11991,
"upload_time": "2021-10-06T23:32:05",
"upload_time_iso_8601": "2021-10-06T23:32:05.764876Z",
"url": "https://files.pythonhosted.org/packages/fe/d3/73d1fe74a156f9a0e519bedc87815ed309e64af19c73b94352e4c0959ddb/pygelf-0.4.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2021-10-06 23:32:05",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "keeprocking",
"github_project": "pygelf",
"travis_ci": true,
"coveralls": false,
"github_actions": false,
"tox": true,
"lcname": "pygelf"
}
JSON
