# powerflex-logging-utilities
<!-- Badges (images) related to Python package information -->
[   ](https://pypi.org/project/powerflex-logging-utilities/)
Helpful code for logging in Python by PowerFlex.
| Module | Description |
|-----------------|--------------------------------------------|
| forbid_toplevel_logging | Disable logging with the top-level root logging functions such as `logging.info`.
| log_slow_callbacks | Either warn or info log when an async callback runs for too long.
| init_loggers | A function for easily setting up logging to a file and to stdout.
| Class | Description |
|-----------------|--------------------------------------------|
| JsonFormatter | A JSON log formatter to enable structured logging. It depends on the `python-json-logger` package.
| TraceLogger | A Python Logger subclass that adds a TRACE logging level
| AsyncNatsLogLevelListener | A NATS interface for changing the program's log level by sending a NATS request
# Installation
You can install from [PyPi](https://pypi.org/project/powerflex-logging-utilities/) directly:
```shellscript
pip install powerflex-logging-utilities
```
# Sample usage
## Initializing Loggers
Setup **all Loggers** to output JSON to stdout and to a file:
```python
import logging
import sys
from powerflex_logging_utilities import (
JsonFormatter,
init_loggers,
TraceLogger,
)
LOG_LEVEL = "DEBUG"
FILE_LOG_LEVEL = "TRACE"
LOG_FILE = "./logs/trace.log"
MAX_LOG_FILE_MB = 10
MAX_TOTAL_LOG_FILE_MB = 10000
root_logger = logging.getLogger()
# Log warnings with the py.warnings logger
logging.captureWarnings(True)
# Fix iPython autocomplete
logging.getLogger("parso").propagate=False
init_loggers.init_loggers(
[root_logger],
log_level=LOG_LEVEL,
file_log_level=FILE_LOG_LEVEL,
filename=LOG_FILE,
max_bytes=1000 * 1000 * MAX_LOG_FILE_MB,
backup_count=MAX_TOTAL_LOG_FILE_MB // MAX_LOG_FILE_MB,
stream=sys.stdout,
formatter=JsonFormatter,
info_logger=root_logger,
)
# Either use logging.getLogger or don't initialize a logger until your root logger is configured.
logging.setLoggerClass(TraceLogger)
logger = logging.getLogger(__name__)
```
This uses Python's logger propagation feature.
We only need to configure the root Logger in order to make sure all other Loggers output in the desired format.
You can pass `formatter_kwargs` to enable logging with a different JSON serializer.
To use:
```skip_phmdoctest
logger = logging.getLogger(__name__)
logger.info("hello world")
```
### Explicitly listing loggers
You can also list the loggers you'd like to configure instead of configuring
the root logger.
This could be useful if you configure your package's main logger
`logging.getLogger("package")`. You can then use Python's logger propagation by calling
`logging.getLogger("package.submodule.a.b.c")` to get Logger instances for all
other submodules.
```python
import logging
from powerflex_logging_utilities import (
JsonFormatter,
init_loggers,
)
logger = logging.getLogger("your_package_name")
# Log warnings with the py.warnings logger
logging.captureWarnings(True)
init_loggers.init_loggers(
[logger, "asyncio", "py.warnings"],
log_level="DEBUG",
file_log_level="TRACE",
filename="./logs/trace-no-root.log",
formatter=JsonFormatter,
info_logger=logger,
)
```
**NOTICE**: if you use this method, any loggers you do not explicitly list will have non-JSON output.
## Using several other utilities
```python
import logging
from powerflex_logging_utilities import (
forbid_toplevel_logging,
log_slow_callbacks,
)
logger = logging.getLogger(__name__)
# Log slow async callbacks with two log levels
log_slow_callbacks.log_slow_callbacks(logger)
# Forbid functions such as logging.info since they implicitly use the root logger
forbid_toplevel_logging.forbid_logging_with_logging_toplevel()
```
## Using the JSON formatter
```python
import logging
import sys
from powerflex_logging_utilities import JsonFormatter
log_handler = logging.StreamHandler(stream=sys.stdout)
log_handler.setLevel("DEBUG")
log_handler.setFormatter(JsonFormatter())
logger = logging.getLogger(__name__)
logger.addHandler(log_handler)
logger.setLevel("DEBUG")
logger.info("hello world", extra={
"data": ["log structured data", ":D"],
1: "handles non string key",
})
```
```skip_phmdoctest
{
"message": "hello world",
"name": "__main__",
"module": "<ipython-input-10-b016ce80d46f>",
"lineno": 1,
"funcName": "<cell line: 1>",
"filename": "<ipython-input-10-b016ce80d46f>",
"asctime": "2022-05-12 01:04:16,824",
"data": [
"log structured data",
":D"
],
"severity": "INFO",
"1": "handles non string key"
}
```
# Using pipenv
1. Run `make setup-with-pipenv` to install all dependencies.
Make sure you have the version of Python specified in `.tool-versions` or simply change this file to your Python version (must be 3.8+).
2. Run `pipenv shell` or run the following `make` commands with `pipenv run make ...`.
You could also alias `pmake` to `pipenv run make` for convenience.
# Tests
There is 100% code coverage.
```
make test-unit
```
To test in several versions of Python, run:
```
tox
```
To download several versions of Python, use `pyenv` or `asdf`
To use `pyenv`, install it [here](https://github.com/pyenv/pyenv#installation) and run the following script:
```
./install_python_versions_pyenv.sh
```
To use `asdf`, install the core parts [here](http://asdf-vm.com/guide/getting-started.html) and run the following commands:
```
./install_python_versions_asdf.sh
```
## Testing the code in this README
```
make test-readme
```
# Checking code quality
The Github Actions will run all of the following checks on the code.
## Code formatting
```
make format-fix
```
## Linting
```
make lint
```
## Type checking
```
make type-check-strict
```
# Releasing to [PyPi.org](https://pypi.org/project/powerflex-logging-utilities/)
1. Make sure all code checks have passed with `make commitready`.
1. Make sure you commit all code you wish to release with `git commit`.
1. Set the version in [`./src/powerflex_monitoring/VERSION`](./src/powerflex_logging_utilities/VERSION)
Please attempt to follow [semantic versioning](https://semver.org/).
1. Run `make bump-version` to commit the change to the `VERSION` file.
1. Run `make release` to upload the package to pypi.org and to push a new git tag
Raw data
{
"_id": null,
"home_page": "https://github.com/edf-re/powerflex_logging_utilities_py",
"name": "powerflex-logging-utilities",
"maintainer": null,
"docs_url": null,
"requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,!=3.7.*,>2.7",
"maintainer_email": null,
"keywords": "NATS, NATS request, aiodebug, async, JSON logging",
"author": null,
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/da/ab/39f121543b5a151ebe7ddebb6a20fac73a2873f3997e310ba699b45e5bab/powerflex-logging-utilities-1.3.1.tar.gz",
"platform": null,
"description": "# powerflex-logging-utilities\n\n<!-- Badges (images) related to Python package information -->\n[   ](https://pypi.org/project/powerflex-logging-utilities/)\n\nHelpful code for logging in Python by PowerFlex.\n\n| Module | Description |\n|-----------------|--------------------------------------------|\n| forbid_toplevel_logging | Disable logging with the top-level root logging functions such as `logging.info`.\n| log_slow_callbacks | Either warn or info log when an async callback runs for too long.\n| init_loggers | A function for easily setting up logging to a file and to stdout.\n\n| Class | Description |\n|-----------------|--------------------------------------------|\n| JsonFormatter | A JSON log formatter to enable structured logging. It depends on the `python-json-logger` package.\n| TraceLogger | A Python Logger subclass that adds a TRACE logging level\n| AsyncNatsLogLevelListener | A NATS interface for changing the program's log level by sending a NATS request\n\n# Installation\n\nYou can install from [PyPi](https://pypi.org/project/powerflex-logging-utilities/) directly:\n```shellscript\npip install powerflex-logging-utilities\n```\n\n# Sample usage\n\n## Initializing Loggers\n\nSetup **all Loggers** to output JSON to stdout and to a file:\n \n```python\nimport logging\nimport sys\n\nfrom powerflex_logging_utilities import (\n JsonFormatter,\n init_loggers,\n TraceLogger,\n)\n\nLOG_LEVEL = \"DEBUG\"\nFILE_LOG_LEVEL = \"TRACE\"\nLOG_FILE = \"./logs/trace.log\"\n\nMAX_LOG_FILE_MB = 10\nMAX_TOTAL_LOG_FILE_MB = 10000\n\nroot_logger = logging.getLogger()\n\n# Log warnings with the py.warnings logger\nlogging.captureWarnings(True)\n\n# Fix iPython autocomplete\nlogging.getLogger(\"parso\").propagate=False\n\ninit_loggers.init_loggers(\n [root_logger],\n log_level=LOG_LEVEL,\n file_log_level=FILE_LOG_LEVEL,\n filename=LOG_FILE,\n max_bytes=1000 * 1000 * MAX_LOG_FILE_MB,\n backup_count=MAX_TOTAL_LOG_FILE_MB // MAX_LOG_FILE_MB,\n stream=sys.stdout,\n formatter=JsonFormatter,\n info_logger=root_logger,\n)\n\n# Either use logging.getLogger or don't initialize a logger until your root logger is configured.\nlogging.setLoggerClass(TraceLogger)\nlogger = logging.getLogger(__name__)\n```\n\nThis uses Python's logger propagation feature.\nWe only need to configure the root Logger in order to make sure all other Loggers output in the desired format.\n\nYou can pass `formatter_kwargs` to enable logging with a different JSON serializer.\n\nTo use:\n\n```skip_phmdoctest\nlogger = logging.getLogger(__name__)\nlogger.info(\"hello world\")\n```\n\n### Explicitly listing loggers\n\nYou can also list the loggers you'd like to configure instead of configuring\nthe root logger.\n\nThis could be useful if you configure your package's main logger\n`logging.getLogger(\"package\")`. You can then use Python's logger propagation by calling\n`logging.getLogger(\"package.submodule.a.b.c\")` to get Logger instances for all\nother submodules.\n\n```python\nimport logging\n\nfrom powerflex_logging_utilities import (\n JsonFormatter,\n init_loggers,\n)\n\nlogger = logging.getLogger(\"your_package_name\")\n\n# Log warnings with the py.warnings logger\nlogging.captureWarnings(True)\n\ninit_loggers.init_loggers(\n [logger, \"asyncio\", \"py.warnings\"],\n log_level=\"DEBUG\",\n file_log_level=\"TRACE\",\n filename=\"./logs/trace-no-root.log\",\n formatter=JsonFormatter,\n info_logger=logger,\n)\n```\n\n**NOTICE**: if you use this method, any loggers you do not explicitly list will have non-JSON output.\n\n## Using several other utilities\n\n```python\nimport logging\nfrom powerflex_logging_utilities import (\n forbid_toplevel_logging,\n log_slow_callbacks,\n)\n\nlogger = logging.getLogger(__name__)\n\n# Log slow async callbacks with two log levels\nlog_slow_callbacks.log_slow_callbacks(logger)\n\n# Forbid functions such as logging.info since they implicitly use the root logger\nforbid_toplevel_logging.forbid_logging_with_logging_toplevel()\n```\n\n## Using the JSON formatter\n\n```python\nimport logging\nimport sys\nfrom powerflex_logging_utilities import JsonFormatter\n\nlog_handler = logging.StreamHandler(stream=sys.stdout)\nlog_handler.setLevel(\"DEBUG\")\nlog_handler.setFormatter(JsonFormatter())\nlogger = logging.getLogger(__name__)\nlogger.addHandler(log_handler)\nlogger.setLevel(\"DEBUG\")\n\nlogger.info(\"hello world\", extra={\n \"data\": [\"log structured data\", \":D\"],\n 1: \"handles non string key\",\n})\n```\n\n```skip_phmdoctest\n{\n \"message\": \"hello world\",\n \"name\": \"__main__\",\n \"module\": \"<ipython-input-10-b016ce80d46f>\",\n \"lineno\": 1,\n \"funcName\": \"<cell line: 1>\",\n \"filename\": \"<ipython-input-10-b016ce80d46f>\",\n \"asctime\": \"2022-05-12 01:04:16,824\",\n \"data\": [\n \"log structured data\",\n \":D\"\n ],\n \"severity\": \"INFO\",\n \"1\": \"handles non string key\"\n}\n```\n\n# Using pipenv\n\n1. Run `make setup-with-pipenv` to install all dependencies.\n Make sure you have the version of Python specified in `.tool-versions` or simply change this file to your Python version (must be 3.8+).\n2. Run `pipenv shell` or run the following `make` commands with `pipenv run make ...`.\n You could also alias `pmake` to `pipenv run make` for convenience.\n\n# Tests\n\nThere is 100% code coverage.\n\n```\nmake test-unit\n```\n\nTo test in several versions of Python, run:\n\n```\ntox\n```\n\nTo download several versions of Python, use `pyenv` or `asdf` \n\nTo use `pyenv`, install it [here](https://github.com/pyenv/pyenv#installation) and run the following script:\n\n```\n./install_python_versions_pyenv.sh\n```\n\nTo use `asdf`, install the core parts [here](http://asdf-vm.com/guide/getting-started.html) and run the following commands:\n\n```\n./install_python_versions_asdf.sh\n```\n\n## Testing the code in this README\n\n```\nmake test-readme\n```\n\n# Checking code quality\n\nThe Github Actions will run all of the following checks on the code.\n\n## Code formatting\n\n```\nmake format-fix\n```\n\n## Linting\n\n```\nmake lint\n```\n\n## Type checking\n\n```\nmake type-check-strict\n```\n\n\n# Releasing to [PyPi.org](https://pypi.org/project/powerflex-logging-utilities/)\n\n1. Make sure all code checks have passed with `make commitready`.\n1. Make sure you commit all code you wish to release with `git commit`.\n1. Set the version in [`./src/powerflex_monitoring/VERSION`](./src/powerflex_logging_utilities/VERSION)\n Please attempt to follow [semantic versioning](https://semver.org/).\n1. Run `make bump-version` to commit the change to the `VERSION` file.\n1. Run `make release` to upload the package to pypi.org and to push a new git tag\n\n\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "Helpful code for logging in Python",
"version": "1.3.1",
"project_urls": {
"Homepage": "https://github.com/edf-re/powerflex_logging_utilities_py",
"Issue Tracker": "https://github.com/edf-re/powerflex_logging_utilities_py/issues"
},
"split_keywords": [
"nats",
" nats request",
" aiodebug",
" async",
" json logging"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "91db82cea4b215fb87acfa3e74d0c19fded063a7e8ff34a11e5a632901d2db29",
"md5": "11736b100d20fd664ad8c7729c58f25e",
"sha256": "a952f374c6a22f47e46d81eafd06c15a23998afd93b17356fa6703a35dfd16bd"
},
"downloads": -1,
"filename": "powerflex_logging_utilities-1.3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "11736b100d20fd664ad8c7729c58f25e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,!=3.7.*,>2.7",
"size": 13649,
"upload_time": "2024-04-10T17:23:30",
"upload_time_iso_8601": "2024-04-10T17:23:30.139040Z",
"url": "https://files.pythonhosted.org/packages/91/db/82cea4b215fb87acfa3e74d0c19fded063a7e8ff34a11e5a632901d2db29/powerflex_logging_utilities-1.3.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "daab39f121543b5a151ebe7ddebb6a20fac73a2873f3997e310ba699b45e5bab",
"md5": "8755cbf9336c756c967b5bfc6516fb50",
"sha256": "2f7416081d3149b5207673cc2b4ac8c776697f74fcc877d42bd61394022fa2be"
},
"downloads": -1,
"filename": "powerflex-logging-utilities-1.3.1.tar.gz",
"has_sig": false,
"md5_digest": "8755cbf9336c756c967b5bfc6516fb50",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,!=3.7.*,>2.7",
"size": 12923,
"upload_time": "2024-04-10T17:23:33",
"upload_time_iso_8601": "2024-04-10T17:23:33.564781Z",
"url": "https://files.pythonhosted.org/packages/da/ab/39f121543b5a151ebe7ddebb6a20fac73a2873f3997e310ba699b45e5bab/powerflex-logging-utilities-1.3.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-10 17:23:33",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "edf-re",
"github_project": "powerflex_logging_utilities_py",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"tox": true,
"lcname": "powerflex-logging-utilities"
}
JSON
