# Alliance Auth Loki Logger
Python logging handler and formatter for [loki](https://grafana.com/oss/loki/)
for django. Supports blocking calls and non blocking ones, using threading.
Build on top of [django-loki-reloaded](https://github.com/zepc007/django-loki).
# Why?
A single location for all logs across auth, Separate to auth! With search and notifications etc. Complete with python trace type data for everything.
# Installation
Have a [loki instance configured and running](https://github.com/grafana/loki)
### Bare Metal:
```shell
pip install allianceauth-loki-logging
```
or
```shell
pip install git+https://github.com/Solar-Helix-Independent-Transport/allianceauth-loki-logging.git
```
### Docker
add this to your requirements file and rebuild your image
```
allianceauth-loki-logging>=1.0.0
```
or
```
allianceauth-loki-logging @ git+https://github.com/Solar-Helix-Independent-Transport/allianceauth-loki-logging.git
```
# Usage
`LokiHandler` is a custom logging handler that pushes log messages to Loki.
Modify your settings to integrate `allianceauth_loki_logging` with Django's logging:
in your `local.py` add this at the end, Be sure to read the comments and update any that need to be updated. Specifically the url for loki.
```python
LOKI_URL = "'http://loki:3100/loki/api/v1/push'
### Override the defaults from base.py
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'verbose': {
'format': "[%(asctime)s] %(levelname)s [%(name)s:%(lineno)s] %(message)s",
'datefmt': "%d/%b/%Y %H:%M:%S"
},
'simple': {
'format': '%(levelname)s %(message)s'
},
},
'handlers': {
'extension_file': {
'level': 'INFO',
'class': 'logging.handlers.RotatingFileHandler',
'filename': os.path.join(BASE_DIR, 'log/extensions.log'),
'formatter': 'verbose',
'maxBytes': 1024 * 1024 * 5, # edit this line to change max log file size
'backupCount': 5, # edit this line to change number of log backups
},
'console': {
'level': 'DEBUG' if DEBUG else 'INFO', # edit this line to change logging level to console
'class': 'logging.StreamHandler',
'formatter': 'verbose',
},
'notifications': { # creates notifications for users with logging_notifications permission
'level': 'ERROR', # edit this line to change logging level to notifications
'class': 'allianceauth.notifications.handlers.NotificationHandler',
'formatter': 'verbose',
},
},
'loggers': {
'allianceauth': {
'handlers': ['notifications'],
'level': 'ERROR',
},
'extensions': {
'handlers': ['extension_file'],
'level': 'DEBUG' if DEBUG else 'INFO',
}
}
}
### LOKI Specific settings
LOGGING['formatters']['loki'] = {
'class': 'allianceauth_loki_logging.LokiFormatter' # required
}
print(f"Configuring Loki Log job to: {os.path.basename(os.sys.argv[0])}")
LOGGING['handlers']['loki'] = {
'level': 'DEBUG' if DEBUG else 'INFO', # Required # We are auto setting the log level to only record debug when in debug.
'class': 'allianceauth_loki_logging.LokiHandler', # Required
'formatter': 'loki', #Required
'timeout': 1, # Post request timeout, default is 0.5. Optional
# Loki url. Defaults to localhost. Optional.
'url': , LOKI_URL,
# Extra tags / labels to attach to the log. Optional, but usefull to differentiate instances.
'tags': {
"job":os.path.basename(os.sys.argv[0]), # Auto set the job to differentiate between celery, gunicorn, manage.py etc.
# you could add extra tags here if you were running multiple auths and needed to be able to tell them apart in a single loki instance eg:
# "auth": "CoolAuth 1",
},
# Push mode. Can be 'sync' or 'thread'. Sync is blocking, thread is non-blocking. Defaults to sync. Optional.
'mode': 'thread',
}
LOGGING['root'] = { # Set the root logger
'handlers': ['loki', 'console'],
'level': 'DEBUG' if DEBUG else 'INFO', # Auto set the log level to only record debug when in debug
}
WORKER_HIJACK_ROOT_LOGGER = False # Do not overide with celery logging.
```
## Diagnosing issues with logs not being pushed in HIGHLY threaded environments
add the following to your loki config to bypass the rate limits.
```yaml
limits_config:
max_streams_per_user: 0
max_global_streams_per_user: 0
```
Raw data
{
"_id": null,
"home_page": "https://github.com/Solar-Helix-Independent-Transport/allianceauth-loki-logging",
"name": "allianceauth-loki-logging",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "python,loki,grafana,logging,metrics,threaded",
"author": "aaronkable",
"author_email": "aaronkable@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/0d/ef/fef3d7fa71d7e8d9db5ff1e142b5f7e06964a80fa119bcc39ee1bc3a6949/allianceauth-loki-logging-1.0.1.tar.gz",
"platform": null,
"description": "# Alliance Auth Loki Logger\n\nPython logging handler and formatter for [loki](https://grafana.com/oss/loki/)\nfor django. Supports blocking calls and non blocking ones, using threading.\n\nBuild on top of [django-loki-reloaded](https://github.com/zepc007/django-loki).\n\n# Why?\n\nA single location for all logs across auth, Separate to auth! With search and notifications etc. Complete with python trace type data for everything.\n\n\n\n\n\n\n\n# Installation\n\nHave a [loki instance configured and running](https://github.com/grafana/loki)\n\n### Bare Metal:\n\n```shell\npip install allianceauth-loki-logging\n```\n\nor\n\n```shell\npip install git+https://github.com/Solar-Helix-Independent-Transport/allianceauth-loki-logging.git\n```\n\n### Docker\n\nadd this to your requirements file and rebuild your image\n\n```\nallianceauth-loki-logging>=1.0.0\n```\n\nor\n\n```\nallianceauth-loki-logging @ git+https://github.com/Solar-Helix-Independent-Transport/allianceauth-loki-logging.git\n```\n\n# Usage\n\n`LokiHandler` is a custom logging handler that pushes log messages to Loki.\n\nModify your settings to integrate `allianceauth_loki_logging` with Django's logging:\n\nin your `local.py` add this at the end, Be sure to read the comments and update any that need to be updated. Specifically the url for loki.\n\n```python\nLOKI_URL = \"'http://loki:3100/loki/api/v1/push'\n### Override the defaults from base.py\nLOGGING = {\n 'version': 1,\n 'disable_existing_loggers': False,\n 'formatters': {\n 'verbose': {\n 'format': \"[%(asctime)s] %(levelname)s [%(name)s:%(lineno)s] %(message)s\",\n 'datefmt': \"%d/%b/%Y %H:%M:%S\"\n },\n 'simple': {\n 'format': '%(levelname)s %(message)s'\n },\n },\n 'handlers': {\n 'extension_file': {\n 'level': 'INFO',\n 'class': 'logging.handlers.RotatingFileHandler',\n 'filename': os.path.join(BASE_DIR, 'log/extensions.log'),\n 'formatter': 'verbose',\n 'maxBytes': 1024 * 1024 * 5, # edit this line to change max log file size\n 'backupCount': 5, # edit this line to change number of log backups\n },\n 'console': {\n 'level': 'DEBUG' if DEBUG else 'INFO', # edit this line to change logging level to console\n 'class': 'logging.StreamHandler',\n 'formatter': 'verbose',\n },\n 'notifications': { # creates notifications for users with logging_notifications permission\n 'level': 'ERROR', # edit this line to change logging level to notifications\n 'class': 'allianceauth.notifications.handlers.NotificationHandler',\n 'formatter': 'verbose',\n },\n },\n 'loggers': {\n 'allianceauth': {\n 'handlers': ['notifications'],\n 'level': 'ERROR',\n },\n 'extensions': {\n 'handlers': ['extension_file'], \n 'level': 'DEBUG' if DEBUG else 'INFO',\n }\n }\n}\n\n### LOKI Specific settings\nLOGGING['formatters']['loki'] = {\n 'class': 'allianceauth_loki_logging.LokiFormatter' # required\n}\n\nprint(f\"Configuring Loki Log job to: {os.path.basename(os.sys.argv[0])}\")\n\nLOGGING['handlers']['loki'] = {\n 'level': 'DEBUG' if DEBUG else 'INFO', # Required # We are auto setting the log level to only record debug when in debug.\n 'class': 'allianceauth_loki_logging.LokiHandler', # Required\n 'formatter': 'loki', #Required\n 'timeout': 1, # Post request timeout, default is 0.5. Optional\n # Loki url. Defaults to localhost. Optional.\n 'url': , LOKI_URL,\n # Extra tags / labels to attach to the log. Optional, but usefull to differentiate instances.\n 'tags': {\n \"job\":os.path.basename(os.sys.argv[0]), # Auto set the job to differentiate between celery, gunicorn, manage.py etc.\n # you could add extra tags here if you were running multiple auths and needed to be able to tell them apart in a single loki instance eg:\n # \"auth\": \"CoolAuth 1\",\n }, \n # Push mode. Can be 'sync' or 'thread'. Sync is blocking, thread is non-blocking. Defaults to sync. Optional.\n 'mode': 'thread',\n}\n\nLOGGING['root'] = { # Set the root logger\n 'handlers': ['loki', 'console'],\n 'level': 'DEBUG' if DEBUG else 'INFO', # Auto set the log level to only record debug when in debug\n}\n\nWORKER_HIJACK_ROOT_LOGGER = False # Do not overide with celery logging.\n```\n\n## Diagnosing issues with logs not being pushed in HIGHLY threaded environments\n\nadd the following to your loki config to bypass the rate limits.\n\n```yaml\nlimits_config:\n max_streams_per_user: 0\n max_global_streams_per_user: 0\n```\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A non-blocking django logging handler for Loki",
"version": "1.0.1",
"project_urls": {
"Homepage": "https://github.com/Solar-Helix-Independent-Transport/allianceauth-loki-logging"
},
"split_keywords": [
"python",
"loki",
"grafana",
"logging",
"metrics",
"threaded"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0deffef3d7fa71d7e8d9db5ff1e142b5f7e06964a80fa119bcc39ee1bc3a6949",
"md5": "ac838079d60a7f094e3be4eb9991974f",
"sha256": "a3b91afd0ff53e95a1d705233f2a8d6112d3637a90d9facaacb55d4a94a3da13"
},
"downloads": -1,
"filename": "allianceauth-loki-logging-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "ac838079d60a7f094e3be4eb9991974f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 5436,
"upload_time": "2024-01-29T11:54:22",
"upload_time_iso_8601": "2024-01-29T11:54:22.013558Z",
"url": "https://files.pythonhosted.org/packages/0d/ef/fef3d7fa71d7e8d9db5ff1e142b5f7e06964a80fa119bcc39ee1bc3a6949/allianceauth-loki-logging-1.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-29 11:54:22",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Solar-Helix-Independent-Transport",
"github_project": "allianceauth-loki-logging",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "allianceauth-loki-logging"
}
JSON
