python-logstash-tradingstrategy
===============================
Python logging handler for `Logstash <git@github.com:tradingstrategy-ai/python-logstash.git>`_.
**Note**: This is a forked from the original `python-logstash project <git@github.com:vklochan/python-logstash.git>`_
for `Trading Strategy <https://tradingstrategy.ai>`_. It has been updated and modified with
some patches for advanced functionality.
Changelog
=========
0.5.1:
- Bette exception message if LogStash formatter has problems to make the message out as JSON,
to give a clue what could be wrong
0.5.0:
- Forked to have a new release
- Added ``extra_fields`` parameter to handler
0.4.7
- Add couple of sensitive fields to the skip_list
0.4.6
- Updated field names to match java counterparts supported by logstash crew
0.4.5
- Allow passing exchange's routing key to AMQP handler
0.4.4
- Fixed urllib import python3 compatibility.
- Added long type to easy_types.
0.4.3
- Added AMQP handler.
0.4.2
- Updated README
- Added ``tags`` parameter to handler
0.4.1
- Added TCP handler.
0.3.1
- Added support for Python 3
0.2.2
- Split Handler into Handler and Formatter classes
0.2.1
- Added support for the new JSON schema in Logstash 1.2.x. See details in
http://tobrunet.ch/2013/09/logstash-1-2-0-upgrade-notes-included/ and
https://logstash.jira.com/browse/LOGSTASH-675
- Added ``version`` parameter. Available values: 1 (Logstash 1.2.x version format), 0 - default (previous version).
Installation
============
Using pip::
pip install python-logstash
Usage
=====
``LogstashHandler`` is a custom logging handler which sends Logstash messages using UDP.
For example::
import logging
import logstash
import sys
host = 'localhost'
test_logger = logging.getLogger('python-logstash-logger')
test_logger.setLevel(logging.INFO)
test_logger.addHandler(logstash.LogstashHandler(host, 5959, version=1))
# test_logger.addHandler(logstash.TCPLogstashHandler(host, 5959, version=1))
test_logger.error('python-logstash: test logstash error message.')
test_logger.info('python-logstash: test logstash info message.')
test_logger.warning('python-logstash: test logstash warning message.')
# add extra field to logstash message
extra = {
'test_string': 'python version: ' + repr(sys.version_info),
'test_boolean': True,
'test_dict': {'a': 1, 'b': 'c'},
'test_float': 1.23,
'test_integer': 123,
'test_list': [1, 2, '3'],
}
test_logger.info('python-logstash: test extra fields', extra=extra)
When using ``extra`` field make sure you don't use reserved names. From `Python documentation <https://docs.python.org/2/library/logging.html>`_.
| "The keys in the dictionary passed in extra should not clash with the keys used by the logging system. (See the `Formatter <https://docs.python.org/2/library/logging.html#logging.Formatter>`_ documentation for more information on which keys are used by the logging system.)"
To use the AMQPLogstashHandler you will need to install pika first.
pip install pika
For example::
import logging
import logstash
test_logger = logging.getLogger('python-logstash-logger')
test_logger.setLevel(logging.INFO)
test_logger.addHandler(logstash.AMQPLogstashHandler(host='localhost', version=1))
test_logger.info('python-logstash: test logstash info message.')
try:
1/0
except:
test_logger.exception('python-logstash-logger: Exception with stack trace!')
You can also pass extra fields to the handler, so that those fields will appear in Logstash::
# Pass `application` field and `tags` to Logstash
# with every log message
tags = ["python"]
extra_fields = {"application": "my_application_name}
handler = logstash.UDPLogstashHandler(
logstash_server,
5959,
version=1,
tags=tags,
extra_fields==extra_fields,
)
Using with Django
=================
Modify your ``settings.py`` to integrate ``python-logstash`` with Django's logging::
LOGGING = {
...
'handlers': {
'logstash': {
'level': 'DEBUG',
'class': 'logstash.LogstashHandler',
'host': 'localhost',
'port': 5959, # Default value: 5959
'version': 1, # Version of logstash event schema. Default value: 0 (for backward compatibility of the library)
'message_type': 'logstash', # 'type' field in logstash message. Default value: 'logstash'.
'fqdn': False, # Fully qualified domain name. Default value: false.
'tags': ['tag1', 'tag2'], # list of tags. Default: None.
'extra_fields': {
'server_name': 'myserver.local',
'server_ip': '10.74.22.12',
}, # dict of extra fields, always added to the log. Default: None.
},
},
'loggers': {
'django.request': {
'handlers': ['logstash'],
'level': 'DEBUG',
'propagate': True,
},
},
...
}
Example Logstash Configuration
==============================
Example Logstash Configuration (``logstash.conf``) for Receiving Events from python-logstash is::
input {
udp {
port => 5959
codec => json
}
}
output {
stdout {
codec => rubydebug
}
}
For TCP input you need to change the logstash's input to ``tcp`` and modify django log handler's class to ``logstash.TCPLogstashHandler``
Release
=======
.. code-block:: shell
python3 -m venv venv
source venv/bin/activate
pip install setuptools twine
python setup.py bdist_wheel
twine build
Raw data
{
"_id": null,
"home_page": "https://github.com/tradingstrategy-ai/python-logstash/",
"name": "python-logstash-tradingstrategy",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "Mikko Ohtamaa",
"author_email": "mikko@opensourcehacker.com",
"download_url": "https://files.pythonhosted.org/packages/15/93/066168e695b831ecc6cab9b6601ad67549d123171aa2129742184c32fad0/python-logstash-tradingstrategy-0.5.1.tar.gz",
"platform": null,
"description": "python-logstash-tradingstrategy\n===============================\n\nPython logging handler for `Logstash <git@github.com:tradingstrategy-ai/python-logstash.git>`_.\n\n**Note**: This is a forked from the original `python-logstash project <git@github.com:vklochan/python-logstash.git>`_\nfor `Trading Strategy <https://tradingstrategy.ai>`_. It has been updated and modified with\nsome patches for advanced functionality.\n\nChangelog\n=========\n\n0.5.1:\n - Bette exception message if LogStash formatter has problems to make the message out as JSON,\n to give a clue what could be wrong\n0.5.0:\n - Forked to have a new release\n - Added ``extra_fields`` parameter to handler\n0.4.7\n - Add couple of sensitive fields to the skip_list\n0.4.6\n - Updated field names to match java counterparts supported by logstash crew\n0.4.5\n - Allow passing exchange's routing key to AMQP handler\n0.4.4\n - Fixed urllib import python3 compatibility.\n - Added long type to easy_types.\n0.4.3\n - Added AMQP handler.\n0.4.2\n - Updated README\n - Added ``tags`` parameter to handler\n0.4.1\n - Added TCP handler.\n0.3.1\n - Added support for Python 3\n0.2.2\n - Split Handler into Handler and Formatter classes\n0.2.1\n - Added support for the new JSON schema in Logstash 1.2.x. See details in\n http://tobrunet.ch/2013/09/logstash-1-2-0-upgrade-notes-included/ and\n https://logstash.jira.com/browse/LOGSTASH-675\n\n - Added ``version`` parameter. Available values: 1 (Logstash 1.2.x version format), 0 - default (previous version).\n\n\nInstallation\n============\n\nUsing pip::\n\n pip install python-logstash\n\nUsage\n=====\n\n``LogstashHandler`` is a custom logging handler which sends Logstash messages using UDP.\n\nFor example::\n\n import logging\n import logstash\n import sys\n\n host = 'localhost'\n\n test_logger = logging.getLogger('python-logstash-logger')\n test_logger.setLevel(logging.INFO)\n test_logger.addHandler(logstash.LogstashHandler(host, 5959, version=1))\n # test_logger.addHandler(logstash.TCPLogstashHandler(host, 5959, version=1))\n\n test_logger.error('python-logstash: test logstash error message.')\n test_logger.info('python-logstash: test logstash info message.')\n test_logger.warning('python-logstash: test logstash warning message.')\n\n # add extra field to logstash message\n extra = {\n 'test_string': 'python version: ' + repr(sys.version_info),\n 'test_boolean': True,\n 'test_dict': {'a': 1, 'b': 'c'},\n 'test_float': 1.23,\n 'test_integer': 123,\n 'test_list': [1, 2, '3'],\n }\n test_logger.info('python-logstash: test extra fields', extra=extra)\n\nWhen using ``extra`` field make sure you don't use reserved names. From `Python documentation <https://docs.python.org/2/library/logging.html>`_.\n | \"The keys in the dictionary passed in extra should not clash with the keys used by the logging system. (See the `Formatter <https://docs.python.org/2/library/logging.html#logging.Formatter>`_ documentation for more information on which keys are used by the logging system.)\"\n\nTo use the AMQPLogstashHandler you will need to install pika first.\n\n pip install pika\n\nFor example::\n\n import logging\n import logstash\n\n test_logger = logging.getLogger('python-logstash-logger')\n test_logger.setLevel(logging.INFO)\n test_logger.addHandler(logstash.AMQPLogstashHandler(host='localhost', version=1))\n\n test_logger.info('python-logstash: test logstash info message.')\n try:\n 1/0\n except:\n test_logger.exception('python-logstash-logger: Exception with stack trace!')\n\n\nYou can also pass extra fields to the handler, so that those fields will appear in Logstash::\n\n # Pass `application` field and `tags` to Logstash\n # with every log message\n tags = [\"python\"]\n extra_fields = {\"application\": \"my_application_name}\n handler = logstash.UDPLogstashHandler(\n logstash_server,\n 5959,\n version=1,\n tags=tags,\n extra_fields==extra_fields,\n )\n\n\nUsing with Django\n=================\n\nModify your ``settings.py`` to integrate ``python-logstash`` with Django's logging::\n\n LOGGING = {\n ...\n 'handlers': {\n 'logstash': {\n 'level': 'DEBUG',\n 'class': 'logstash.LogstashHandler',\n 'host': 'localhost',\n 'port': 5959, # Default value: 5959\n 'version': 1, # Version of logstash event schema. Default value: 0 (for backward compatibility of the library)\n 'message_type': 'logstash', # 'type' field in logstash message. Default value: 'logstash'.\n 'fqdn': False, # Fully qualified domain name. Default value: false.\n 'tags': ['tag1', 'tag2'], # list of tags. Default: None.\n 'extra_fields': {\n 'server_name': 'myserver.local',\n 'server_ip': '10.74.22.12',\n }, # dict of extra fields, always added to the log. Default: None.\n },\n },\n 'loggers': {\n 'django.request': {\n 'handlers': ['logstash'],\n 'level': 'DEBUG',\n 'propagate': True,\n },\n },\n ...\n }\n\nExample Logstash Configuration\n==============================\n\nExample Logstash Configuration (``logstash.conf``) for Receiving Events from python-logstash is::\n\n input {\n udp {\n port => 5959\n codec => json\n }\n }\n output {\n stdout {\n codec => rubydebug\n }\n }\n\nFor TCP input you need to change the logstash's input to ``tcp`` and modify django log handler's class to ``logstash.TCPLogstashHandler``\n\nRelease\n=======\n\n.. code-block:: shell\n\n python3 -m venv venv\n source venv/bin/activate\n pip install setuptools twine\n python setup.py bdist_wheel\n twine build\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Python logging handler for Logstash (forked)",
"version": "0.5.1",
"project_urls": {
"Homepage": "https://github.com/tradingstrategy-ai/python-logstash/"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1593066168e695b831ecc6cab9b6601ad67549d123171aa2129742184c32fad0",
"md5": "db59cfb28c62519ee91b247fba0d3022",
"sha256": "f98b43ac53adaef0bd4e1da84d2c2ab8e8ceebba76b412a9d63c89c1ea9207a4"
},
"downloads": -1,
"filename": "python-logstash-tradingstrategy-0.5.1.tar.gz",
"has_sig": false,
"md5_digest": "db59cfb28c62519ee91b247fba0d3022",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 9798,
"upload_time": "2023-04-29T18:49:40",
"upload_time_iso_8601": "2023-04-29T18:49:40.470311Z",
"url": "https://files.pythonhosted.org/packages/15/93/066168e695b831ecc6cab9b6601ad67549d123171aa2129742184c32fad0/python-logstash-tradingstrategy-0.5.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-04-29 18:49:40",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "tradingstrategy-ai",
"github_project": "python-logstash",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "python-logstash-tradingstrategy"
}
JSON
