| Name | backgroundlog JSON |
| Version |
0.2.4
JSON |
| download |
| home_page | None |
| Summary | A logging handler wrapper to improve logging performace by sending writes to a background thread |
| upload_time | 2025-07-12 16:34:52 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.10 |
| license | None |
| keywords |
logging
log
thread
threading
non-blocking
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# backgroundlog
Thread-based log handler for better performance
[](https://opensource.org/licenses/MIT)
[](https://github.com/diegojromerolopez/backgroundlog/graphs/commit-activity)
[](https://www.python.org/)
[](https://github.com/psf/black)
[](https://pycqa.github.io/isort/)
[](http://mypy-lang.org/)
[](https://pypi.python.org/pypi/backgroundlog/)
[](https://pypi.python.org/pypi/backgroundlog/)
[](https://pypi.python.org/pypi/backgroundlog/)
[](https://pypi.python.org/pypi/backgroundlog/)
Do not have your python program slowed down by your logging.
## Introduction
Most of the time we log on disk, and all I/O is time consuming.
By leveraging a thread, it is possible to speed up your python application
considerably.
## Use
### Default use
```python
import logging
from backgroundlog.handlers.thread_handler import ThreadHandler
# Setting up the logging thread handler
file_handler = logging.FileHandler('/var/log/myapp.log', mode="a", encoding="utf-8")
thread_handler = ThreadHandler(file_handler)
# Creating a new logger
bg_logger = logging.getLogger('bg_logger')
bg_logger.setLevel(logging.INFO)
# Adding the thread handler
bg_logger.addHandler(thread_handler)
# Using the logger
bg_logger.info('This is a log message')
```
### Options
#### Set a queue size
```python
from backgroundlog.handlers.thread_handler import ThreadHandler
thread_handler = ThreadHandler(file_handler, queue_size=5000)
```
By default, the queue size is 1000.
#### Set a blocking policy by logging record levels
When putting the records in the queue, it could reach the queue size.
We provide a way to deal with this issue: set a blocking policy
by logging record level, and in the case of a non-blocking policy,
the record will be discarded and we will increment a dropped log record.
##### Only info, error and critical records are blocking:
```python
from backgroundlog.handlers.thread_handler import ThreadHandler
from logging import CRITICAL, ERROR, INFO
thread_handler = ThreadHandler(file_handler, blocking_levels={INFO, ERROR, CRITICAL})
```
##### Only error and critical records are blocking
```python
from backgroundlog.handlers.thread_handler import ThreadHandler
from logging import CRITICAL, ERROR
thread_handler = ThreadHandler(file_handler, blocking_levels={ERROR, CRITICAL})
```
##### Only critical records are blocking
```python
from backgroundlog.handlers.thread_handler import ThreadHandler
from logging import CRITICAL
thread_handler = ThreadHandler(file_handler, blocking_levels={CRITICAL})
```
##### No records are blocking
```python
from backgroundlog.handlers.thread_handler import ThreadHandler
thread_handler = ThreadHandler(file_handler, blocking_levels=None)
```
By default, the error and critical records are blocking, the rest are not.
## Performance testing
We have done several local testing with different logging handlers.
See the file
[run_performance_comparison.py](/backgroundlog/performance/run_performance_comparison.py) for
a full catalog of the performance tests we run.
All tests are 100_000 iterations of creating the same logging message,
and were run with Python 3.13.5 in a Macbook Pro M1 with 16 GB of RAM:
| Logging Handler | Spent Time | | vs. Baseline |
|-------------------------------|---------------|-------------|--------------|
| | Mean Time (s) | Std Dev (s) | |
| StreamHandler | 0.687 | 0.006 | baseline |
| FileHandler | 0.687 | 0.007 | -0.067% |
| ThreadHandler (StreamHandler) | 0.477 | 0.003 | -30.646% |
| ThreadHandler (FileHandler) | 0.475 | 0.001 | -30.865% |
As you see there is a ~30% of improvement when running the thread handler.
It is not much, but in some contexts it can be useful for sure.
## Dependencies
This package has no dependencies.
## Python version support
Minimum version support is 3.10.
## License
[MIT](LICENSE) license, but if you need any other contact me.
Raw data
{
"_id": null,
"home_page": null,
"name": "backgroundlog",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "logging, log, thread, threading, non-blocking",
"author": null,
"author_email": "\"Diego J. Romero L\u00f3pez\" <diegojromerolopez@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/ff/e2/2b57e93a6a8b3bfb94497fd1a7801055473760dddd61d3685121522bdb32/backgroundlog-0.2.4.tar.gz",
"platform": null,
"description": "# backgroundlog\nThread-based log handler for better performance\n\n\n[](https://opensource.org/licenses/MIT)\n[](https://github.com/diegojromerolopez/backgroundlog/graphs/commit-activity)\n[](https://www.python.org/)\n[](https://github.com/psf/black)\n[](https://pycqa.github.io/isort/)\n[](http://mypy-lang.org/)\n[](https://pypi.python.org/pypi/backgroundlog/)\n[](https://pypi.python.org/pypi/backgroundlog/)\n[](https://pypi.python.org/pypi/backgroundlog/)\n[](https://pypi.python.org/pypi/backgroundlog/)\n\nDo not have your python program slowed down by your logging.\n\n## Introduction\nMost of the time we log on disk, and all I/O is time consuming.\nBy leveraging a thread, it is possible to speed up your python application\nconsiderably.\n\n## Use\n\n### Default use\n\n```python\nimport logging\nfrom backgroundlog.handlers.thread_handler import ThreadHandler\n\n# Setting up the logging thread handler\nfile_handler = logging.FileHandler('/var/log/myapp.log', mode=\"a\", encoding=\"utf-8\")\nthread_handler = ThreadHandler(file_handler)\n\n# Creating a new logger\nbg_logger = logging.getLogger('bg_logger')\nbg_logger.setLevel(logging.INFO)\n\n# Adding the thread handler\nbg_logger.addHandler(thread_handler)\n\n# Using the logger\nbg_logger.info('This is a log message')\n```\n\n### Options\n\n#### Set a queue size\n\n```python\nfrom backgroundlog.handlers.thread_handler import ThreadHandler\n\nthread_handler = ThreadHandler(file_handler, queue_size=5000)\n```\n\nBy default, the queue size is 1000.\n\n#### Set a blocking policy by logging record levels\n\nWhen putting the records in the queue, it could reach the queue size.\nWe provide a way to deal with this issue: set a blocking policy\nby logging record level, and in the case of a non-blocking policy,\nthe record will be discarded and we will increment a dropped log record.\n\n##### Only info, error and critical records are blocking:\n\n```python\nfrom backgroundlog.handlers.thread_handler import ThreadHandler\nfrom logging import CRITICAL, ERROR, INFO\n\nthread_handler = ThreadHandler(file_handler, blocking_levels={INFO, ERROR, CRITICAL})\n```\n\n##### Only error and critical records are blocking\n\n```python\nfrom backgroundlog.handlers.thread_handler import ThreadHandler\nfrom logging import CRITICAL, ERROR\n\nthread_handler = ThreadHandler(file_handler, blocking_levels={ERROR, CRITICAL})\n```\n\n##### Only critical records are blocking\n\n```python\nfrom backgroundlog.handlers.thread_handler import ThreadHandler\nfrom logging import CRITICAL\n\nthread_handler = ThreadHandler(file_handler, blocking_levels={CRITICAL})\n```\n\n##### No records are blocking\n\n```python\nfrom backgroundlog.handlers.thread_handler import ThreadHandler\n\nthread_handler = ThreadHandler(file_handler, blocking_levels=None)\n```\n\nBy default, the error and critical records are blocking, the rest are not.\n\n## Performance testing\n\nWe have done several local testing with different logging handlers.\nSee the file\n[run_performance_comparison.py](/backgroundlog/performance/run_performance_comparison.py) for\na full catalog of the performance tests we run.\n\nAll tests are 100_000 iterations of creating the same logging message,\nand were run with Python 3.13.5 in a Macbook Pro M1 with 16 GB of RAM:\n\n| Logging Handler | Spent Time | | vs. Baseline |\n|-------------------------------|---------------|-------------|--------------|\n| | Mean Time (s) | Std Dev (s) | |\n| StreamHandler | 0.687 | 0.006 | baseline |\n| FileHandler | 0.687 | 0.007 | -0.067% |\n| ThreadHandler (StreamHandler) | 0.477 | 0.003 | -30.646% |\n| ThreadHandler (FileHandler) | 0.475 | 0.001 | -30.865% |\n\nAs you see there is a ~30% of improvement when running the thread handler.\nIt is not much, but in some contexts it can be useful for sure.\n\n## Dependencies\nThis package has no dependencies.\n\n## Python version support\nMinimum version support is 3.10.\n\n## License\n[MIT](LICENSE) license, but if you need any other contact me.\n",
"bugtrack_url": null,
"license": null,
"summary": "A logging handler wrapper to improve logging performace by sending writes to a background thread",
"version": "0.2.4",
"project_urls": {
"Bug Tracker": "https://github.com/diegojromerolopez/backgroundlog/issues",
"documentation": "https://github.com/diegojromerolopez/backgroundlog/blob/main/README.md",
"repository": "https://github.com/diegojromerolopez/backgroundlog"
},
"split_keywords": [
"logging",
" log",
" thread",
" threading",
" non-blocking"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "ab298685658ba96f8c4d8206f4c7f98c3aaac199a177951441a5faa24d0b9cf1",
"md5": "f6616583cc55c73ed326e9afa13ae9e6",
"sha256": "022ae936018344c299361cad5456fcba184d8047e93614681ef6c6f71ff814c4"
},
"downloads": -1,
"filename": "backgroundlog-0.2.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f6616583cc55c73ed326e9afa13ae9e6",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 8948,
"upload_time": "2025-07-12T16:34:51",
"upload_time_iso_8601": "2025-07-12T16:34:51.494960Z",
"url": "https://files.pythonhosted.org/packages/ab/29/8685658ba96f8c4d8206f4c7f98c3aaac199a177951441a5faa24d0b9cf1/backgroundlog-0.2.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "ffe22b57e93a6a8b3bfb94497fd1a7801055473760dddd61d3685121522bdb32",
"md5": "e59884c9962a291966d76b02b0f14f6c",
"sha256": "1004011dfd65b35799cbf11f80073f4f653148de13fef5da029c87084222298d"
},
"downloads": -1,
"filename": "backgroundlog-0.2.4.tar.gz",
"has_sig": false,
"md5_digest": "e59884c9962a291966d76b02b0f14f6c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 8756,
"upload_time": "2025-07-12T16:34:52",
"upload_time_iso_8601": "2025-07-12T16:34:52.795146Z",
"url": "https://files.pythonhosted.org/packages/ff/e2/2b57e93a6a8b3bfb94497fd1a7801055473760dddd61d3685121522bdb32/backgroundlog-0.2.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-12 16:34:52",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "diegojromerolopez",
"github_project": "backgroundlog",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "backgroundlog"
}
