Overview
========
``libranet_logging`` is an easy to use logging-configuration reading the configuration
for loggers, handlers, filters and formatters from a logging.yml-file.
Overview urls
-------------
- git-repository: https://github.com/Libranet/libranet_logging
- changelog: <url>
- sphinx-docs: <url>
- code-coverage: <url>
- pypi: <url>
Why use logging?
----------------
Logfiles are your best-friend
- during development, where debugmode is developmentmode
- more important: while running in PRD,
- it shows how the application is being used, by whom, and if it's successfull
- allows to become pro-active. There is no need to wait for bugreports from users.
- most important: during urgent troubleshooting on PRD (AKA panic-mode)
- heisenbugs, difficult to reproduce.
Goal of libranet_logging
------------------------
Make it as easy as possible to enable and properly use the full power of the python logging-framework
python logging-module contains:
- loggers, hierarchical
- handlers
- formatters
- filters
Think of logger=message-channel, handler=subscriber to a channel
Minimize the need to make changes in code
Move all config out of code and into a config-file "logging.yml"
- logging to a file should be as simple as:
>>> import logging
>>> logging.getLogger('panicmode')
Features
--------
- load logging-configuration from a yaml-config
- validate yaml-file for missing keys, invalid values
- configurable via env-variables
- sane defaults if env-var is not set
- when logging to console, have colorized logging,
- but nowhere else
- configurable colors (avoid blue on black)
- integrate python-warnings
- add sample email-logger
- add sample syslog-logger
- avoid empty files that will remain empty
- cleanup dedicated file-handlers based on root-loglevel
- future ideas:
- integrate with kibana
- log as json, structlog
- https://logmatic.io/blog/python-logging-with-json-steroids/
- https://medium.com/@sanchitsokhey/centralised-logging-for-django-gunicorn-and-celery-using-elk-stack-76b13c54414c
- https://logmatic.io/blog/beyond-application-monitoring-discover-logging-best-practices/
- in code throw out all
- formatting,
- handler-config,
- setting loglevel
- debug-flags like::
if DEBUG:
log.debug(....)
Changelog
=========
1.3 (2023-01-24)
----------------
- No longer try to be smart about interactive mode or not.
You can set the active logging-handlers by setting the env-var ``LOG_HANDLERS```
- Remove ``disable_console`` as input-parameter to ``libranet_logging.initialize()``.
- Change default separator from ``;`` to ``|``. Use set via env-var ``LOG_HANDLERS``.
1.2 (2021-06-06)
----------------
- Fix logo. [WVH]
- Add .gitlab-ci.yml [WVH]
1.1 (2020-02-13)
----------------
- Fix error ``ModuleNotFoundError: No module named 'libranet_logging.version'``. [WVH]
1.0 (2020-02-12)
----------------
- Move ``__version__``-attribute to ``__init__``. [WVH]
- Package ``libranet_logging`` forked from WVH's unreleased package. [WVH]
0.5 (2019-08-19)
----------------
- Add docstrings and type-hinting.
- Fix a series of issues reported by pylint.
- Change function-signature of ``libranet_logging.yaml.read_yml``: change ``vars`` into ``variables``
to avoid shadowing the buitlin ``vars()``-function::
>>> config = read_yaml(path, vars=None)
>>> config = read_yaml(path, variables=None)
0.4 (2019-07-31)
----------------
- Add ``version.py`` with a ``__version__``-attribute, rework version-management.
- In ``setup.py`` set minimum-requirements for cerberus >=1.3.1. [WVH]
0.3 (2019-05-28)
----------------
- In sphinx-docs, add link to coverage-report
on https://example.com/docs/libranet_logging-coverage [WVH]
- In ``Makefile`` add step`` copy-cov`` to copy coverage-report to apache-webdirectory. [WVH]
- In ``libranet_logging.yaml.read_yaml`` cdefault vars to empty dicts when not provided. [WVH]
- In ``libranet_logging.logconfig.logging_schema`` rename ``valueschema`` to ``valuesrules``
to avoid DeprecationWarnings. [WVH]
0.2 (2019-03-28)
----------------
- Make ``libranet_logging.yaml.read_yaml`` compatible with PyYAML 5.1,
but keep backwards-compatibility with older versions. [WVH]
Cfr::
- https://github.com/yaml/pyyaml/blob/master/CHANGES
- https://github.com/yaml/pyyaml/pull/257
- https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load(input)-Deprecation
0.1 (2019-03-28)
----------------
- Add support for simple string-formatting in the loggin.yml. [WVH]
- Use ``isort`` to manage the imports. Add isort-config to ``setup.cfg``. [WVH]
- Introduce environment-variable ``PYTHON_CONSOLE_FORMATTER`` to select which
console-formatter to use. [WVH]
- Rename ``colored``-console-formatter into ``console_color`` and add
``console_bw``-formatter for simpe black & white logging in the console. [WVH]
- Add ``flask_wtf``-handler in default ``logging.yml``. [WVH]
- If the log-directory does not yet exist, we now create it. [WVH]
- We should have a user-specific default location to avoid interference between users.
The log-directory will be first taken from th direct function-parameters, then
from the ``logging.yml`` if present. If not present, from the env-var
``PYTHON_LOG_CONFIG``, and in case of no env-var we default to ``$HOME/logs``
instead of ``var/tmp/python``. [WVH]
- Support setting the log-directory via the ``initialize``-function. [WVH]
- Fix failing test ``test_initialize_without_logging_tree``.
It was failing when the env-var ``PYTHON_ENABLE_LOGGING_TREE`` was not set. [WVH]
- Fix failing test of the click-command ``cli.print_logging_tree``. [WVH]
- Add new testing-dependency ``pytest-click``. [WVH]
- Convert ``cli.print_logging_tree`` to a ``click``-command, accepting an optional ``path``-argument.
If the environment-variable ``PYTHON_LOG_CONFIG`` is set, we use that value as the path-default. [WVH]
- Add ``click`` as a new dependency. [WVH]
- Add documentation about unittesting. [WVH]
- We now support arrays in environment-variables. Environment-variables
containing a ``;`` are now converted to a list similar to the
default value if that env-variable was not set. [WVH]
- Fix ``filters.RegexFilter`` to use ``search()`` instead of ``match()``.
Cfr. https://docs.python.org/3/library/re.html#search-vs-match [WVH]
- Add passing unit-tests. [WVH]
- In ``initialize()`` allow Path-parameters as input instead of only string-paths. [WVH]
- Generally make the code robust in case of loading a logging.yml with schema-errors. [WVH]
- Add console-entrypoint ``libranet-logging-print-logging-tree``
to initialize the logging and print the logging-tree to the standard output.
Add corresponding function in new ``libranet_logging.cli``-module. [WVH]
- Add function-paramater ``use_print=False`` to ``logconfig.show_logging_tree``
to enable printing to standard output instead of logging to the configured loggers. [WVH]
- If we call ``initialize()`` without providing a path of setting the environmant-variable ``PYTHON_LOG_CONFIG``,
we now use the default ``logging.yml`` shipped with this ``libranet_logging``-package. [WVH]
- Add logger ``libranet_logging`` to our default ``logging.yml``. [WVH]
- Instantiate the correct logger using ``__name__`` instead of logging to the root-logger. [WVH]
- Add ``recommonmark`` and update ``docs.conf.py`` to allow markdown in docs.
Cfr.https://recommonmark.readthedocs.io/en/latest/ [WVH]
- In ``setup.py`` and ``docs/pip-requirements`` add ``sphinx``-related dependencies. [WVH]
- Simplify public api:
- Rename function ``loglevel.create_loglevel`` into ``loglevel.create``.
- Rename function ``logconfig.initialize_logging`` into ``logconfig.initialize``.
[WVH]
- Run ``Black`` on the code. ``Black`` is a code-formatter for Python.
Cfr. https://github.com/ambv/black [WVH]
- Add some basic Sphinx-based documentation. [WVH]
- Factor out creating new loglevels into its own ``loglevel``-module. [WVH]
- Factor out logging-filters into its own ``filters``-module. [WVH]
- Add third-party dependency ``colorlog``. This is not a code-dependency
but rather a dependency of ``logging.yml``. [WVH]
- Add third-party dependencies ``cerberus``, ``logging_tree`` and ``PyYAML``. [WVH]
- Move logging-related code from ``libdl.utils`` into its own ``libranet_logging``-package. [WVH]
- Package created via ``cookiecutter templates/cookiecutter-libranet-python-package``.
[Wouter Vanden Hove <wouter@wvhconsulting.org>]
Contributors
============
.. Append your name if you have contributed to this package.
- Wouter Vanden Hove <wouter@wvhconsulting.org>
Raw data
{
"_id": null,
"home_page": "https://github.com/Libranet/libranet_logging",
"name": "libranet-logging",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "python logging yaml loggers handlers filters",
"author": "Wouter Vanden Hove",
"author_email": "wouter@wvhconsulting.org",
"download_url": "https://files.pythonhosted.org/packages/2b/7b/251a9fc30d8117c14863a180b666cd5a5ea16ca82a30235458a1271becd0/libranet_logging-1.3.tar.gz",
"platform": null,
"description": "Overview\n========\n\n``libranet_logging`` is an easy to use logging-configuration reading the configuration\nfor loggers, handlers, filters and formatters from a logging.yml-file.\n\nOverview urls\n-------------\n - git-repository: https://github.com/Libranet/libranet_logging\n - changelog: <url>\n - sphinx-docs: <url>\n - code-coverage: <url>\n - pypi: <url>\n\n\nWhy use logging?\n----------------\nLogfiles are your best-friend\n\n - during development, where debugmode is developmentmode\n\n - more important: while running in PRD,\n - it shows how the application is being used, by whom, and if it's successfull\n - allows to become pro-active. There is no need to wait for bugreports from users.\n\n - most important: during urgent troubleshooting on PRD (AKA panic-mode)\n - heisenbugs, difficult to reproduce.\n\n\nGoal of libranet_logging\n------------------------\nMake it as easy as possible to enable and properly use the full power of the python logging-framework\n\npython logging-module contains:\n - loggers, hierarchical\n - handlers\n - formatters\n - filters\n\nThink of logger=message-channel, handler=subscriber to a channel\n\nMinimize the need to make changes in code\n\nMove all config out of code and into a config-file \"logging.yml\"\n\n - logging to a file should be as simple as:\n >>> import logging\n >>> logging.getLogger('panicmode')\n\n\nFeatures\n--------\n\n - load logging-configuration from a yaml-config\n\n - validate yaml-file for missing keys, invalid values\n\n - configurable via env-variables\n - sane defaults if env-var is not set\n\n - when logging to console, have colorized logging,\n - but nowhere else\n - configurable colors (avoid blue on black)\n\n - integrate python-warnings\n - add sample email-logger\n - add sample syslog-logger\n\n - avoid empty files that will remain empty\n - cleanup dedicated file-handlers based on root-loglevel\n\n - future ideas:\n - integrate with kibana\n - log as json, structlog\n\n - https://logmatic.io/blog/python-logging-with-json-steroids/\n - https://medium.com/@sanchitsokhey/centralised-logging-for-django-gunicorn-and-celery-using-elk-stack-76b13c54414c\n - https://logmatic.io/blog/beyond-application-monitoring-discover-logging-best-practices/\n\n\n - in code throw out all\n - formatting,\n - handler-config,\n - setting loglevel\n - debug-flags like::\n\n if DEBUG:\n log.debug(....)\n\n\nChangelog\n=========\n\n1.3 (2023-01-24)\n----------------\n\n- No longer try to be smart about interactive mode or not.\n You can set the active logging-handlers by setting the env-var ``LOG_HANDLERS```\n\n- Remove ``disable_console`` as input-parameter to ``libranet_logging.initialize()``.\n\n- Change default separator from ``;`` to ``|``. Use set via env-var ``LOG_HANDLERS``.\n\n\n1.2 (2021-06-06)\n----------------\n\n- Fix logo. [WVH]\n\n- Add .gitlab-ci.yml [WVH]\n\n\n1.1 (2020-02-13)\n----------------\n\n- Fix error ``ModuleNotFoundError: No module named 'libranet_logging.version'``. [WVH]\n\n\n1.0 (2020-02-12)\n----------------\n\n- Move ``__version__``-attribute to ``__init__``. [WVH]\n\n- Package ``libranet_logging`` forked from WVH's unreleased package. [WVH]\n\n\n0.5 (2019-08-19)\n----------------\n\n- Add docstrings and type-hinting.\n\n- Fix a series of issues reported by pylint.\n\n- Change function-signature of ``libranet_logging.yaml.read_yml``: change ``vars`` into ``variables``\n to avoid shadowing the buitlin ``vars()``-function::\n\n >>> config = read_yaml(path, vars=None)\n >>> config = read_yaml(path, variables=None)\n\n\n0.4 (2019-07-31)\n----------------\n- Add ``version.py`` with a ``__version__``-attribute, rework version-management.\n\n- In ``setup.py`` set minimum-requirements for cerberus >=1.3.1. [WVH]\n\n\n0.3 (2019-05-28)\n----------------\n\n- In sphinx-docs, add link to coverage-report\n on https://example.com/docs/libranet_logging-coverage [WVH]\n\n- In ``Makefile`` add step`` copy-cov`` to copy coverage-report to apache-webdirectory. [WVH]\n\n- In ``libranet_logging.yaml.read_yaml`` cdefault vars to empty dicts when not provided. [WVH]\n\n- In ``libranet_logging.logconfig.logging_schema`` rename ``valueschema`` to ``valuesrules``\n to avoid DeprecationWarnings. [WVH]\n\n\n0.2 (2019-03-28)\n----------------\n\n- Make ``libranet_logging.yaml.read_yaml`` compatible with PyYAML 5.1,\n but keep backwards-compatibility with older versions. [WVH]\n Cfr::\n\n - https://github.com/yaml/pyyaml/blob/master/CHANGES\n - https://github.com/yaml/pyyaml/pull/257\n - https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load(input)-Deprecation\n\n\n0.1 (2019-03-28)\n----------------\n\n- Add support for simple string-formatting in the loggin.yml. [WVH]\n\n- Use ``isort`` to manage the imports. Add isort-config to ``setup.cfg``. [WVH]\n\n- Introduce environment-variable ``PYTHON_CONSOLE_FORMATTER`` to select which\n console-formatter to use. [WVH]\n\n- Rename ``colored``-console-formatter into ``console_color`` and add\n ``console_bw``-formatter for simpe black & white logging in the console. [WVH]\n\n- Add ``flask_wtf``-handler in default ``logging.yml``. [WVH]\n\n- If the log-directory does not yet exist, we now create it. [WVH]\n\n- We should have a user-specific default location to avoid interference between users.\n The log-directory will be first taken from th direct function-parameters, then\n from the ``logging.yml`` if present. If not present, from the env-var\n ``PYTHON_LOG_CONFIG``, and in case of no env-var we default to ``$HOME/logs``\n instead of ``var/tmp/python``. [WVH]\n\n- Support setting the log-directory via the ``initialize``-function. [WVH]\n\n- Fix failing test ``test_initialize_without_logging_tree``.\n It was failing when the env-var ``PYTHON_ENABLE_LOGGING_TREE`` was not set. [WVH]\n\n- Fix failing test of the click-command ``cli.print_logging_tree``. [WVH]\n\n- Add new testing-dependency ``pytest-click``. [WVH]\n\n- Convert ``cli.print_logging_tree`` to a ``click``-command, accepting an optional ``path``-argument.\n If the environment-variable ``PYTHON_LOG_CONFIG`` is set, we use that value as the path-default. [WVH]\n\n- Add ``click`` as a new dependency. [WVH]\n\n- Add documentation about unittesting. [WVH]\n\n- We now support arrays in environment-variables. Environment-variables\n containing a ``;`` are now converted to a list similar to the\n default value if that env-variable was not set. [WVH]\n\n- Fix ``filters.RegexFilter`` to use ``search()`` instead of ``match()``.\n Cfr. https://docs.python.org/3/library/re.html#search-vs-match [WVH]\n\n- Add passing unit-tests. [WVH]\n\n- In ``initialize()`` allow Path-parameters as input instead of only string-paths. [WVH]\n\n- Generally make the code robust in case of loading a logging.yml with schema-errors. [WVH]\n\n- Add console-entrypoint ``libranet-logging-print-logging-tree``\n to initialize the logging and print the logging-tree to the standard output.\n Add corresponding function in new ``libranet_logging.cli``-module. [WVH]\n\n- Add function-paramater ``use_print=False`` to ``logconfig.show_logging_tree``\n to enable printing to standard output instead of logging to the configured loggers. [WVH]\n\n- If we call ``initialize()`` without providing a path of setting the environmant-variable ``PYTHON_LOG_CONFIG``,\n we now use the default ``logging.yml`` shipped with this ``libranet_logging``-package. [WVH]\n\n- Add logger ``libranet_logging`` to our default ``logging.yml``. [WVH]\n\n- Instantiate the correct logger using ``__name__`` instead of logging to the root-logger. [WVH]\n\n- Add ``recommonmark`` and update ``docs.conf.py`` to allow markdown in docs.\n Cfr.https://recommonmark.readthedocs.io/en/latest/ [WVH]\n\n- In ``setup.py`` and ``docs/pip-requirements`` add ``sphinx``-related dependencies. [WVH]\n\n- Simplify public api:\n\n - Rename function ``loglevel.create_loglevel`` into ``loglevel.create``.\n\n - Rename function ``logconfig.initialize_logging`` into ``logconfig.initialize``.\n\n [WVH]\n\n- Run ``Black`` on the code. ``Black`` is a code-formatter for Python.\n Cfr. https://github.com/ambv/black [WVH]\n- Add some basic Sphinx-based documentation. [WVH]\n\n- Factor out creating new loglevels into its own ``loglevel``-module. [WVH]\n\n- Factor out logging-filters into its own ``filters``-module. [WVH]\n\n- Add third-party dependency ``colorlog``. This is not a code-dependency\n but rather a dependency of ``logging.yml``. [WVH]\n\n- Add third-party dependencies ``cerberus``, ``logging_tree`` and ``PyYAML``. [WVH]\n\n- Move logging-related code from ``libdl.utils`` into its own ``libranet_logging``-package. [WVH]\n\n- Package created via ``cookiecutter templates/cookiecutter-libranet-python-package``.\n [Wouter Vanden Hove <wouter@wvhconsulting.org>]\n\n\nContributors\n============\n\n.. Append your name if you have contributed to this package.\n\n- Wouter Vanden Hove <wouter@wvhconsulting.org>\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Easy-to-use logging-configuration using a logging.yml-file.",
"version": "1.3",
"split_keywords": [
"python",
"logging",
"yaml",
"loggers",
"handlers",
"filters"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2b7b251a9fc30d8117c14863a180b666cd5a5ea16ca82a30235458a1271becd0",
"md5": "893108a10c4c9bbc41b9196ae90f2c1d",
"sha256": "a44a693ec541476c90061d54565d3e2501fc52df599bc94b35ed464a7fe00dd1"
},
"downloads": -1,
"filename": "libranet_logging-1.3.tar.gz",
"has_sig": false,
"md5_digest": "893108a10c4c9bbc41b9196ae90f2c1d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 85970,
"upload_time": "2023-01-24T12:01:36",
"upload_time_iso_8601": "2023-01-24T12:01:36.929001Z",
"url": "https://files.pythonhosted.org/packages/2b/7b/251a9fc30d8117c14863a180b666cd5a5ea16ca82a30235458a1271becd0/libranet_logging-1.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-01-24 12:01:36",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "Libranet",
"github_project": "libranet_logging",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "libranet-logging"
}
JSON
