| Name | uologging JSON |
| Version |
0.8.1
JSON |
| download |
| home_page | None |
| Summary | Auto-Configuration solution for Python built-in logging. |
| upload_time | 2024-04-10 21:36:24 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | None |
| license | None |
| keywords |
nts
uo
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
UOLogging is a solution for configuring Python's built-in logging module, including a utility for tracing multithreaded downloads.
# Install
```
pip install uologging[all]
```
> ℹ Use the '`[all]` suffix to download the (†very small) `humanize` package as well.
>
>> `humanize` makes logs human-readable when dealing with large numbers.
>>
>> † The `humanize` package is 78.5 kB as of version 4.9.0.
## Enable console logging
Simply call "`init_console()`" to initializing Python's root logger to log to console:
# ⚠ Inadvisable: Enable logging for ALL python packages/modules
uologging.init_console()
> ⚠ WARNING: It is inadvisable to "init" the overall root logger except for debugging.
> Why? The console can get *very noisy* when using 3rd party libraries (that use Python `logging` module).
In general, you will want to specify your package name. To enable logging within your package only, you can provide your package name.
> The handy invocation of `__name__.split('.')[0]` will provide your package's name from *anywhere within your package*.
# ✅ Best Practice: Enable logging only for your package.
my_package_name = __name__.split('.')[0]
uologging.init_console(my_package_name)
## Enable (Linux) syslog logging
Similarly, you can call "`init_syslog()`":
# Best Practice: Enable logging for your python package
my_package_name = __name__.split('.')[0]
uologging.init_syslog(my_package_name)
# Inadvisable: Enable logging for ALL python packages/modules
uologging.init_syslog()
## Set Logging Verbosity
> Per Python logging suggestion: WARNING, ERROR, and CRITICAL messages are all logged by default.
If you are interested in seeing the DEBUG and INFO log messages, you'll need to update the logging verbosity in your application.
We provide the method set_verbosity() for this purpose.
Higher number means more logging.
> Choices are [0,2].
> Default is 0. Default will captures WARNING, ERROR, and CRITICAL logs.
> Provide 1 to also capture INFO logs.
> Provide 2 to also capture DEBUG logs.
# Enable maximum logging for your python package
my_package_name = __name__.split('.')[0]
uologging.set_verbosity(2, args.verbosity_flag, my_package_name)
# Enable maximum logging for ALL python packages/modules
uologging.set_verbosity(2)
## argparse 'verbosity flag'
For CLI tools, we provide an integration with argparse to set the logging verbosity.
This integration enables the tool's user to add `-vv` at the command-line for maximum logging verbosity.
> `-v` will enable INFO messages, but not DEBUG.
The verbosity_flag can be gathered via argparse using "`add_verbosity_flag(parser)`":
import uologging
import argparse
parser = argparse.ArgumentParser()
uologging.add_verbosity_flag(parser)
args = parser.parse_args(['-vv'])
# args.verbosity_flag == 2
Now, simply call "`set_verbosity()`" with `args.verbosity_flag` for your package:
my_package_name = __name__.split('.')[0]
uologging.set_verbosity(args.verbosity_flag, my_package_name)
### Example: Configuring CLI tool with console & syslog logging
Let's imagine you have a package "`examplepkg`" with a CLI tool in the "`mytool`" module.
# my_cli_tool.py
import argparse
import uologging
# Parse CLI arguments, '-vv' will result in maximum logging verbosity.
parser = argparse.ArgumentParser()
uologging.add_verbosity_flag(parser)
args = parser.parse_args()
# Initialize logging
my_package_name = __name__.split('.')[0]
uologging.init_console(my_package_name)
uologging.init_syslog(my_package_name)
uologging.set_verbosity(args.verbosity_flag, my_package_name)
### Logging messages format
The formatting for log messages is specified in the (private) `uologging._logging_format` variable.
Here are a couple of lines showing what you can expect your logs to looks like:
2022-01-07 15:40:09 DEBUG Some simle message for you [hello.py:10]
2022-01-07 15:40:09 DEBUG Finished: example.hello:hello((),{}) [hello.py:10]
2022-01-07 15:40:09 DEBUG example.hello:hello((),{}) execution time: 0.00 sec [hello.py:10]
## Tracing a function
There is a simple `trace` decorator you can use in your python modules to log the 'execution time' of any of your functions.
> The trace decorator logs at DEBUG severity.
> So, call `set_verbosity(>=2)` to see the trace messages in your logs.
# hello.py
import logging
import uologging
logger = logging.getLogger(__name__)
@uologging.trace(logger)
def hello():
print('hello!')
hello()
Expect the following messages to be logged:
2022-01-07 15:40:09 DEBUG Starting: example.hello:hello((),{}) [hello.py:10]
hello!
2022-01-07 15:40:09 DEBUG Finished: example.hello:hello((),{}) [hello.py:10]
2022-01-07 15:40:09 DEBUG example.hello:hello((),{}) execution time: 0.00 sec [hello.py:10]
## Tracing Concurrent Downloads
Assume you have some service, and you need to make *several http requests* to download *lots of gobs of data*.
You want to do it concurrently, to save on time.
> ℹ 'Downloading data via several http requests' is an example of an '*embarrassingly parallel*' problem.
> I.e. for each concurrent worker we add, we gain 1x speedup Lots of speedup.
> Ex. using *8 workers will give 8x speedup.*
For this big, concurrent download, you would like to see:
1. *download progress indications*, to know that the download is indeed happening, and
2. when the download is completed, *how much in total was downloaded.*
There is the `uologging.DownloadTracer` for that!
> The download tracer logs a message for every 1 MB downloaded by default (customizable via '`threshold_bytes`').
Trace a concurrent downloads using the `DownloadTracer` with [python `requests` package](https://pypi.org/project/requests/) like the following:
```python
from concurrent.futures import ThreadPoolExecutor
import requests
import uologging
download_tracer = DownloadTracer('MyService', threshold_bytes=750000)
ALL_MY_URLS = ['http://ex1.io', 'http://ex2.io', 'http://ex3.io'] # ... Replace with your actual list of URLs
def http_get(url):
respons = requests.get(url)
download_tracer.trace(len(response.content))
return response
with ThreadPoolExecutor(max_workers=4) as executor:
map(http_get, ALL_MY_URLS)
```
## `logging` Best Practices
Use the Python logging package per the following best practices:
1. `logger = logging.getLogger(__name__)` to get the logger for each module/script.
2. Then, use `logger.debug()`, `logger.info()`, `logger.warning()`, etc to add tracing to your Python modules/scripts.
### Example
A trivial example demonstrating best practices:
# hello.py
import logging
logger = logging.getLogger(__name__)
def hello():
logger.debug('About to say "hello!"')
print('hello!')
logger.debug('Said "hello!"')
Raw data
{
"_id": null,
"home_page": null,
"name": "uologging",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "NTS, UO",
"author": null,
"author_email": "University of Oregon <ntsjenkins@uoregon.edu>",
"download_url": "https://files.pythonhosted.org/packages/f4/c6/98b0cac78473f006c2d7c616967125ba0084356078aea899691cab90f2f3/uologging-0.8.1.tar.gz",
"platform": null,
"description": "UOLogging is a solution for configuring Python's built-in logging module, including a utility for tracing multithreaded downloads.\n\n# Install\n\n```\npip install uologging[all]\n```\n\n> \u2139 Use the '`[all]` suffix to download the (\u2020very small) `humanize` package as well.\n>\n>> `humanize` makes logs human-readable when dealing with large numbers.\n>>\n>> \u2020 The `humanize` package is 78.5 kB as of version 4.9.0.\n\n## Enable console logging\n\nSimply call \"`init_console()`\" to initializing Python's root logger to log to console:\n\n # \u26a0 Inadvisable: Enable logging for ALL python packages/modules\n uologging.init_console()\n\n> \u26a0 WARNING: It is inadvisable to \"init\" the overall root logger except for debugging. \n> Why? The console can get *very noisy* when using 3rd party libraries (that use Python `logging` module).\n\nIn general, you will want to specify your package name. To enable logging within your package only, you can provide your package name.\n\n> The handy invocation of `__name__.split('.')[0]` will provide your package's name from *anywhere within your package*.\n\n # \u2705 Best Practice: Enable logging only for your package.\n my_package_name = __name__.split('.')[0]\n uologging.init_console(my_package_name)\n\n\n\n## Enable (Linux) syslog logging\n\nSimilarly, you can call \"`init_syslog()`\":\n\n # Best Practice: Enable logging for your python package\n my_package_name = __name__.split('.')[0]\n uologging.init_syslog(my_package_name)\n\n # Inadvisable: Enable logging for ALL python packages/modules\n uologging.init_syslog()\n\n## Set Logging Verbosity\n\n> Per Python logging suggestion: WARNING, ERROR, and CRITICAL messages are all logged by default.\n\nIf you are interested in seeing the DEBUG and INFO log messages, you'll need to update the logging verbosity in your application.\nWe provide the method set_verbosity() for this purpose.\nHigher number means more logging. \n\n> Choices are [0,2].\n> Default is 0. Default will captures WARNING, ERROR, and CRITICAL logs.\n> Provide 1 to also capture INFO logs. \n> Provide 2 to also capture DEBUG logs.\n\n # Enable maximum logging for your python package\n my_package_name = __name__.split('.')[0]\n uologging.set_verbosity(2, args.verbosity_flag, my_package_name)\n\n # Enable maximum logging for ALL python packages/modules\n uologging.set_verbosity(2)\n\n## argparse 'verbosity flag'\n\nFor CLI tools, we provide an integration with argparse to set the logging verbosity.\nThis integration enables the tool's user to add `-vv` at the command-line for maximum logging verbosity.\n\n> `-v` will enable INFO messages, but not DEBUG.\n\nThe verbosity_flag can be gathered via argparse using \"`add_verbosity_flag(parser)`\":\n\n import uologging\n import argparse\n\n parser = argparse.ArgumentParser()\n uologging.add_verbosity_flag(parser)\n\n args = parser.parse_args(['-vv'])\n # args.verbosity_flag == 2\n\nNow, simply call \"`set_verbosity()`\" with `args.verbosity_flag` for your package:\n\n my_package_name = __name__.split('.')[0]\n uologging.set_verbosity(args.verbosity_flag, my_package_name)\n\n\n### Example: Configuring CLI tool with console & syslog logging\n\nLet's imagine you have a package \"`examplepkg`\" with a CLI tool in the \"`mytool`\" module.\n\n # my_cli_tool.py\n import argparse\n import uologging\n\n # Parse CLI arguments, '-vv' will result in maximum logging verbosity.\n parser = argparse.ArgumentParser()\n uologging.add_verbosity_flag(parser)\n args = parser.parse_args()\n\n # Initialize logging\n my_package_name = __name__.split('.')[0]\n uologging.init_console(my_package_name)\n uologging.init_syslog(my_package_name)\n uologging.set_verbosity(args.verbosity_flag, my_package_name)\n\n### Logging messages format\n\nThe formatting for log messages is specified in the (private) `uologging._logging_format` variable.\n\nHere are a couple of lines showing what you can expect your logs to looks like:\n\n 2022-01-07 15:40:09 DEBUG Some simle message for you [hello.py:10]\n 2022-01-07 15:40:09 DEBUG Finished: example.hello:hello((),{}) [hello.py:10] \n 2022-01-07 15:40:09 DEBUG example.hello:hello((),{}) execution time: 0.00 sec [hello.py:10] \n\n\n## Tracing a function\n\nThere is a simple `trace` decorator you can use in your python modules to log the 'execution time' of any of your functions.\n\n> The trace decorator logs at DEBUG severity.\n> So, call `set_verbosity(>=2)` to see the trace messages in your logs.\n\n # hello.py\n import logging\n import uologging\n\n logger = logging.getLogger(__name__)\n\n @uologging.trace(logger)\n def hello():\n print('hello!')\n \n hello()\n\nExpect the following messages to be logged:\n\n 2022-01-07 15:40:09 DEBUG Starting: example.hello:hello((),{}) [hello.py:10]\n hello!\n 2022-01-07 15:40:09 DEBUG Finished: example.hello:hello((),{}) [hello.py:10] \n 2022-01-07 15:40:09 DEBUG example.hello:hello((),{}) execution time: 0.00 sec [hello.py:10]\n\n## Tracing Concurrent Downloads\n\nAssume you have some service, and you need to make *several http requests* to download *lots of gobs of data*.\nYou want to do it concurrently, to save on time.\n\n> \u2139 'Downloading data via several http requests' is an example of an '*embarrassingly parallel*' problem. \n> I.e. for each concurrent worker we add, we gain 1x speedup Lots of speedup. \n> Ex. using *8 workers will give 8x speedup.*\n\nFor this big, concurrent download, you would like to see:\n1. *download progress indications*, to know that the download is indeed happening, and\n2. when the download is completed, *how much in total was downloaded.*\n\nThere is the `uologging.DownloadTracer` for that!\n\n> The download tracer logs a message for every 1 MB downloaded by default (customizable via '`threshold_bytes`').\n\nTrace a concurrent downloads using the `DownloadTracer` with [python `requests` package](https://pypi.org/project/requests/) like the following:\n\n```python\nfrom concurrent.futures import ThreadPoolExecutor\nimport requests\nimport uologging\n\ndownload_tracer = DownloadTracer('MyService', threshold_bytes=750000)\nALL_MY_URLS = ['http://ex1.io', 'http://ex2.io', 'http://ex3.io'] # ... Replace with your actual list of URLs\n\ndef http_get(url):\n respons = requests.get(url)\n download_tracer.trace(len(response.content))\n return response\n\nwith ThreadPoolExecutor(max_workers=4) as executor:\n map(http_get, ALL_MY_URLS)\n```\n\n## `logging` Best Practices\n\nUse the Python logging package per the following best practices:\n\n1. `logger = logging.getLogger(__name__)` to get the logger for each module/script.\n2. Then, use `logger.debug()`, `logger.info()`, `logger.warning()`, etc to add tracing to your Python modules/scripts.\n\n### Example\n\nA trivial example demonstrating best practices:\n\n # hello.py\n import logging\n\n logger = logging.getLogger(__name__)\n\n def hello():\n logger.debug('About to say \"hello!\"')\n print('hello!')\n logger.debug('Said \"hello!\"')\n",
"bugtrack_url": null,
"license": null,
"summary": "Auto-Configuration solution for Python built-in logging.",
"version": "0.8.1",
"project_urls": null,
"split_keywords": [
"nts",
" uo"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5160ddcb6c883d0c98bc7eb765ab070cbfe94f105a07223c835eb9c86143e887",
"md5": "d85f4e9b7573f2b9eb34166701edb235",
"sha256": "7aee0ec37bad1b3771f8833b72f12307a7d446bfee092c961cc9076e6eed4a12"
},
"downloads": -1,
"filename": "uologging-0.8.1-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "d85f4e9b7573f2b9eb34166701edb235",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 8165,
"upload_time": "2024-04-10T21:36:22",
"upload_time_iso_8601": "2024-04-10T21:36:22.837067Z",
"url": "https://files.pythonhosted.org/packages/51/60/ddcb6c883d0c98bc7eb765ab070cbfe94f105a07223c835eb9c86143e887/uologging-0.8.1-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f4c698b0cac78473f006c2d7c616967125ba0084356078aea899691cab90f2f3",
"md5": "62526bf8dc9ee8fbeba949abf1078513",
"sha256": "fd2d18a2b514a64eb608456e432c4eb643c88ed84ea77c9d1024ee8cf816c157"
},
"downloads": -1,
"filename": "uologging-0.8.1.tar.gz",
"has_sig": false,
"md5_digest": "62526bf8dc9ee8fbeba949abf1078513",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 32464,
"upload_time": "2024-04-10T21:36:24",
"upload_time_iso_8601": "2024-04-10T21:36:24.034780Z",
"url": "https://files.pythonhosted.org/packages/f4/c6/98b0cac78473f006c2d7c616967125ba0084356078aea899691cab90f2f3/uologging-0.8.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-10 21:36:24",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "uologging"
}
