# Loggate
[](https://pypi.org/project/loggate/)
The complex logging system with support of log metadata and delivery to [Grafana Loki](https://grafana.com/oss/loki/).
This library supports threading & asyncio modules.
## Simple stdout/stderr colorized output
One example is more than a thousand words.
```python
from loggate import setup_logging, get_logger
setup_logging(level='DEBUG')
logger = get_logger('component', meta={'version': '1.0.0'})
logger.debug('Loading resources for the component')
logger.info('Initialize of the component')
logger.warning('The component is not ready')
logger.error('The component failed.',
meta={'inputs': {'A': 1, 'B': 2}})
logger.critical('The component unexpected failed.',
meta={'attrs': {'A': 1, 'B': 2}})
```
*Console output:*
### Exceptions
```python
from loggate import setup_logging, get_logger
setup_logging()
logger = get_logger('component', meta={'version': '1.0.0'})
try:
raise Exception('Some error')
except Exception as ex:
logger.error('The component failed.', exc_info=True)
```
*Console output:*
## Advanced configuration
The Loggate supports a declarative configuration alike as [logging.config](https://docs.python.org/3/library/logging.config.html).
But this support profiles as well. It's mean we can declare many logging profiles and switch between them. Default profile is called `default`.
We can use yaml file as configuration file (see next):
```yaml
profiles:
default:
# Default profile
filters:
warning:
# This is a filter for stdout logger, that enable only logs with level lower than WARNING.
# For logs with WARNING and higher we use stderr logger.
class: loggate.LowerLogLevelFilter
level: WARNING
formatters:
colored:
# This is stdout/sterr formatter.
class: loggate.LogColorFormatter
loki:
# This is formatter of loki messages.
class: loggate.loki.LokiLogFormatter
handlers:
stdout:
# This is a stdout handler
class: logging.StreamHandler
stream: ext://sys.stdout
formatter: colored
filters:
- warning
stderr:
# This is a stderr handler
class: logging.StreamHandler
stream: ext://sys.stderr
formatter: colored
level: WARNING
loki:
# This is a loki handler
class: loggate.loki.LokiThreadHandler # for asyncio use loggate.loki.LokiHandler
formatter: loki
max_queue_size: 1000 # Default is 0 = unlimit
# send_retry: [5, 5, 10, 10, 30, 30, 60, 60, 120]
urls:
- "http://loki1:3100/loki/api/v1/push"
- "http://loki2:3100/loki/api/v1/push"
- "http://loki3:3100/loki/api/v1/push"
meta:
# loggate handlers accept metadata as well. Standard logging handlers do not!
stage: dev
ip: 192.168.1.10
loggers:
root:
level: INFO
handlers:
- stdout
- stderr
- loki
'urllib3.connectionpool':
level: WARNING
```
```python
import yaml
from loggate import setup_logging, get_logger
def get_yaml(filename):
with open(filename, 'r+') as fd:
return yaml.safe_load(fd)
schema = get_yaml('test.yaml')
setup_logging(profiles=schema.get('profiles'))
logger = get_logger('component')
logger.debug('Loading resources for the component')
logger.info('Initialize of the component')
logger.warning('The component is not ready')
```
The console output is the same as above, but now we send logs to Loki as well.
*Loki output:*
# Description
## Methods
- `get_logger`
- `name` - Return logger for this name. Empty name returns root logger.
- `meta` - Metadata (dict), which are sent only by this logger.
- `getLogger` - only alias for `get_logger`
- `setup_logging` - init setup of application logging.
- `profiles` - Profiles (dict) of logging profiles. When we do not set this parameter, application use predefined profile with log `INFO` level (this level can be set by parameter `level`).
- `default_profile` - name of the default profile (default: `default`)
- `level` - This is special parameter for situation when application use predefined profile (default `INFO`).
## Filters
### Class `loggate.LowerLogLevelFilter`
This filters out all logs which are higher than `level`.
- `level` - log level
## Formatters
### Class `loggate.LogColorFormatter`
Colorized formatter for stdout/stderr.
- `fmt` - message format (default: `%(LEVEL_COLOR)s%(asctime)s\t [%(levelname)s] %(name)s:%(COLOR_RESET)s %(message)s`)
- `datefmt` - datetime format (default: `%Y-%m-%d %H:%M:%S`)
- `style` - style of templating (default: `%`).
- `validate` - validate the input format (default: True)
- `INDENTATION_TRACEBACK` - default: `\t\t\t`
- `INDENTATION_METADATA` - default: `\t\t\t\t`
- `COLOR_DEBUG`, ..., `COLOR_CRITICAL` - set color of this levels (e.g. `\x1b[1;31m`, see [more colors](https://dev.to/ifenna__/adding-colors-to-bash-scripts-48g4)).
- `COLOR_METADATA` - set color metadata
- `COLOR_TRACEBACK` - set color of tracebacks
- `COLOR_...` - set custom color
### Class `loggate.loki.LokiLogFormatter`
This is special loki formatter, this converts log records to jsons.
## Handlers
### Class `loggate.loki.LokiHandler`
This handler send log records to Loki server. This is blocking implementation of handler.
It means, when we call log method (`debug`, ... `critical`) the message is sent in the same thread. We should use
this only for tiny scripts where other ways have a big overhead.
- `level` - This handler sends only log records with log level equal or higher than this (default: all = `logging.NOTSET`).
- `urls` - List of loki entrypoints.
- `strategy` - Deploy strategy (default: `random`).
- `random` - At the beginning the handler choose random Loki server and others are fallbacks.
- `fallbacks` - The handler uses the first Loki server and others are fallbacks.
- `all` - The handler send the log record to all loki servers.
- `auth` - The Loki authentication, the list with two items (`username`, `password`).
- `timeout` - Timeout for one delivery try (default: 5s).
- `ssl_verify` - Enable ssl verify (default: True).
- `max_queue_size` - Size of sending queue. The default is 0 = unlimited. Privileged messages have got a limit 110% of `max_queue_size`.
- `send_retry` - Comma separated list of seconds for resend. The last item of this list is used as default for all other sending.
- `loki_tags` - the list of metadata keys, which are sent to Loki server as label (defailt: [`logger`, `level`]).
- `meta` - Metadata (dict), which are sent only by this handler.
### Class `loggate.loki.LokiAsyncioHandler`
This is non-bloking extending of LokiHandler. We register an extra asyncio task for sending messages to the Loki server.
Parameters are the same as `loggate.loki.LokiHandler`. This handler uses `urllib.requests` module in default ([aiohttp](https://pypi.org/project/aiohttp/) as optional).
Unfortunately `urllib.requests` module does not support asyncio, it means the sending itself is blocking.
The `loggate.loki.Loki AsyncioHandler` can use the optional dependency [aiohttp](https://pypi.org/project/aiohttp/) for non-bloking sending.
### Class `loggate.loki.LokiThreadHandler`
This is non-bloking extending of LokiHandler. We register and start an extra thread for sending messages to the Loki server.
Parameters are the same as `loggate.loki.LokiHandler`.
## Profiles
The structure of profiles (parameter `profiles` of `setup_logging`).
```yaml
<profile_name>:
filters:
<filter_name>:
class: <filter_class>
<filter_attribute_name>: <filter_attribute_value>
formatters:
<formatter_name>:
class: <formatter_class>
<formatter_attribute_name>: <formatter_attribute_value>
handlers:
<handler_name>:
class: <handler_class>
<handler_attribute_name>: <handler_attribute_value>
loggers:
<logger_name>|root: # definition of root logger
level: <log_level>
handlers:
- <name_of_handler>|<definition_of_handler>
disabled: True|False # default: False
propagate: True|False # default: True
meta: <logger_metadata>
```
Raw data
{
"_id": null,
"home_page": "https://github.com/calcite/loggate",
"name": "loggate",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.7",
"maintainer_email": null,
"keywords": "log, logging, logger, loki",
"author": "Martin Korbel",
"author_email": "mkorbel@alps.cz",
"download_url": "https://files.pythonhosted.org/packages/30/24/077853a29d0ea59288f0d2e5ad1b4c96b53bfa455ebd8ce38f198b09dd05/loggate-1.11.0.tar.gz",
"platform": null,
"description": "# Loggate\n[](https://pypi.org/project/loggate/)\n\n\n\nThe complex logging system with support of log metadata and delivery to [Grafana Loki](https://grafana.com/oss/loki/). \nThis library supports threading & asyncio modules. \n\n## Simple stdout/stderr colorized output\nOne example is more than a thousand words.\n\n```python\nfrom loggate import setup_logging, get_logger\n\nsetup_logging(level='DEBUG')\nlogger = get_logger('component', meta={'version': '1.0.0'})\n\nlogger.debug('Loading resources for the component')\nlogger.info('Initialize of the component')\nlogger.warning('The component is not ready')\nlogger.error('The component failed.',\n meta={'inputs': {'A': 1, 'B': 2}})\nlogger.critical('The component unexpected failed.',\n meta={'attrs': {'A': 1, 'B': 2}})\n```\n*Console output:*\n\n\n\n\n### Exceptions\n\n```python\nfrom loggate import setup_logging, get_logger\n\nsetup_logging()\nlogger = get_logger('component', meta={'version': '1.0.0'})\n\ntry:\n raise Exception('Some error')\nexcept Exception as ex:\n logger.error('The component failed.', exc_info=True)\n```\n*Console output:*\n\n\n\n\n## Advanced configuration\nThe Loggate supports a declarative configuration alike as [logging.config](https://docs.python.org/3/library/logging.config.html).\nBut this support profiles as well. It's mean we can declare many logging profiles and switch between them. Default profile is called `default`.\n\nWe can use yaml file as configuration file (see next):\n```yaml\nprofiles:\n default: \n # Default profile\n filters:\n warning: \n # This is a filter for stdout logger, that enable only logs with level lower than WARNING. \n # For logs with WARNING and higher we use stderr logger. \n class: loggate.LowerLogLevelFilter\n level: WARNING\n\n formatters:\n colored:\n # This is stdout/sterr formatter. \n class: loggate.LogColorFormatter\n loki:\n # This is formatter of loki messages.\n class: loggate.loki.LokiLogFormatter\n\n handlers:\n stdout:\n # This is a stdout handler\n class: logging.StreamHandler\n stream: ext://sys.stdout\n formatter: colored\n filters:\n - warning\n stderr:\n # This is a stderr handler\n class: logging.StreamHandler\n stream: ext://sys.stderr\n formatter: colored\n level: WARNING \n loki:\n # This is a loki handler\n class: loggate.loki.LokiThreadHandler # for asyncio use loggate.loki.LokiHandler \n formatter: loki\n max_queue_size: 1000 # Default is 0 = unlimit\n # send_retry: [5, 5, 10, 10, 30, 30, 60, 60, 120]\n urls:\n - \"http://loki1:3100/loki/api/v1/push\"\n - \"http://loki2:3100/loki/api/v1/push\"\n - \"http://loki3:3100/loki/api/v1/push\"\n meta:\n # loggate handlers accept metadata as well. Standard logging handlers do not!\n stage: dev\n ip: 192.168.1.10\n \n\n loggers:\n root: \n level: INFO\n handlers:\n - stdout\n - stderr \n - loki\n 'urllib3.connectionpool': \n level: WARNING\n\n```\n\n```python\nimport yaml\nfrom loggate import setup_logging, get_logger\n\n\ndef get_yaml(filename):\n with open(filename, 'r+') as fd:\n return yaml.safe_load(fd)\n\n\nschema = get_yaml('test.yaml')\nsetup_logging(profiles=schema.get('profiles'))\n\nlogger = get_logger('component')\n\nlogger.debug('Loading resources for the component')\nlogger.info('Initialize of the component')\nlogger.warning('The component is not ready')\n```\nThe console output is the same as above, but now we send logs to Loki as well.\n\n*Loki output:*\n\n\n\n\n\n# Description\n## Methods\n- `get_logger`\n - `name` - Return logger for this name. Empty name returns root logger.\n - `meta` - Metadata (dict), which are sent only by this logger.\n\n- `getLogger` - only alias for `get_logger`\n- `setup_logging` - init setup of application logging.\n - `profiles` - Profiles (dict) of logging profiles. When we do not set this parameter, application use predefined profile with log `INFO` level (this level can be set by parameter `level`). \n - `default_profile` - name of the default profile (default: `default`)\n - `level` - This is special parameter for situation when application use predefined profile (default `INFO`). \n\n## Filters\n### Class `loggate.LowerLogLevelFilter`\nThis filters out all logs which are higher than `level`.\n- `level` - log level\n\n## Formatters\n### Class `loggate.LogColorFormatter`\nColorized formatter for stdout/stderr.\n- `fmt` - message format (default: `%(LEVEL_COLOR)s%(asctime)s\\t [%(levelname)s] %(name)s:%(COLOR_RESET)s %(message)s`)\n- `datefmt` - datetime format (default: `%Y-%m-%d %H:%M:%S`)\n- `style` - style of templating (default: `%`). \n- `validate` - validate the input format (default: True)\n- `INDENTATION_TRACEBACK` - default: `\\t\\t\\t`\n- `INDENTATION_METADATA` - default: `\\t\\t\\t\\t`\n- `COLOR_DEBUG`, ..., `COLOR_CRITICAL` - set color of this levels (e.g. `\\x1b[1;31m`, see [more colors](https://dev.to/ifenna__/adding-colors-to-bash-scripts-48g4)).\n- `COLOR_METADATA` - set color metadata\n- `COLOR_TRACEBACK` - set color of tracebacks\n- `COLOR_...` - set custom color\n\n### Class `loggate.loki.LokiLogFormatter`\nThis is special loki formatter, this converts log records to jsons.\n\n\n## Handlers\n### Class `loggate.loki.LokiHandler`\nThis handler send log records to Loki server. This is blocking implementation of handler.\nIt means, when we call log method (`debug`, ... `critical`) the message is sent in the same thread. We should use\nthis only for tiny scripts where other ways have a big overhead.\n- `level` - This handler sends only log records with log level equal or higher than this (default: all = `logging.NOTSET`).\n- `urls` - List of loki entrypoints.\n- `strategy` - Deploy strategy (default: `random`).\n - `random` - At the beginning the handler choose random Loki server and others are fallbacks.\n - `fallbacks` - The handler uses the first Loki server and others are fallbacks.\n - `all` - The handler send the log record to all loki servers.\n- `auth` - The Loki authentication, the list with two items (`username`, `password`).\n- `timeout` - Timeout for one delivery try (default: 5s).\n- `ssl_verify` - Enable ssl verify (default: True).\n- `max_queue_size` - Size of sending queue. The default is 0 = unlimited. Privileged messages have got a limit 110% of `max_queue_size`.\n- `send_retry` - Comma separated list of seconds for resend. The last item of this list is used as default for all other sending.\n- `loki_tags` - the list of metadata keys, which are sent to Loki server as label (defailt: [`logger`, `level`]).\n- `meta` - Metadata (dict), which are sent only by this handler. \n\n### Class `loggate.loki.LokiAsyncioHandler`\nThis is non-bloking extending of LokiHandler. We register an extra asyncio task for sending messages to the Loki server.\nParameters are the same as `loggate.loki.LokiHandler`. This handler uses `urllib.requests` module in default ([aiohttp](https://pypi.org/project/aiohttp/) as optional). \nUnfortunately `urllib.requests` module does not support asyncio, it means the sending itself is blocking.\nThe `loggate.loki.Loki AsyncioHandler` can use the optional dependency [aiohttp](https://pypi.org/project/aiohttp/) for non-bloking sending.\n\n### Class `loggate.loki.LokiThreadHandler`\nThis is non-bloking extending of LokiHandler. We register and start an extra thread for sending messages to the Loki server.\nParameters are the same as `loggate.loki.LokiHandler`.\n\n## Profiles\nThe structure of profiles (parameter `profiles` of `setup_logging`).\n\n```yaml\n<profile_name>:\n \n filters:\n <filter_name>:\n class: <filter_class>\n <filter_attribute_name>: <filter_attribute_value>\n \n formatters:\n <formatter_name>:\n class: <formatter_class>\n <formatter_attribute_name>: <formatter_attribute_value>\n \n handlers:\n <handler_name>:\n class: <handler_class>\n <handler_attribute_name>: <handler_attribute_value>\n\n loggers:\n <logger_name>|root: # definition of root logger\n level: <log_level>\n handlers: \n - <name_of_handler>|<definition_of_handler>\n disabled: True|False # default: False\n propagate: True|False # default: True\n meta: <logger_metadata> \n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "The complex logging system.",
"version": "1.11.0",
"project_urls": {
"Documentation": "https://github.com/calcite/loggate",
"Homepage": "https://github.com/calcite/loggate",
"Repository": "https://github.com/calcite/loggate"
},
"split_keywords": [
"log",
" logging",
" logger",
" loki"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1df8404b61deca9164a2a9b29c81bf9375c3345eebe62a0d9f655a9f27b143d7",
"md5": "b8ece5d045977e0acad3507ee72f2426",
"sha256": "395af7714ab866985e6ca6617d4f071e90e2d08b5434e6daa8068e5aa3f122d6"
},
"downloads": -1,
"filename": "loggate-1.11.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b8ece5d045977e0acad3507ee72f2426",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.7",
"size": 19192,
"upload_time": "2024-08-08T08:51:00",
"upload_time_iso_8601": "2024-08-08T08:51:00.053248Z",
"url": "https://files.pythonhosted.org/packages/1d/f8/404b61deca9164a2a9b29c81bf9375c3345eebe62a0d9f655a9f27b143d7/loggate-1.11.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3024077853a29d0ea59288f0d2e5ad1b4c96b53bfa455ebd8ce38f198b09dd05",
"md5": "e394f18c6bf49a5ee380039bccf74f9c",
"sha256": "6935d54e75b21feb4d3ee35569d23e8df3c6aab50932b1432b341b28c0f9da10"
},
"downloads": -1,
"filename": "loggate-1.11.0.tar.gz",
"has_sig": false,
"md5_digest": "e394f18c6bf49a5ee380039bccf74f9c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.7",
"size": 17699,
"upload_time": "2024-08-08T08:51:05",
"upload_time_iso_8601": "2024-08-08T08:51:05.571566Z",
"url": "https://files.pythonhosted.org/packages/30/24/077853a29d0ea59288f0d2e5ad1b4c96b53bfa455ebd8ce38f198b09dd05/loggate-1.11.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-08 08:51:05",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "calcite",
"github_project": "loggate",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "loggate"
}
JSON
