| Name | backgroundlog JSON |
| Version |
0.2.6
JSON |
| download |
| home_page | None |
| Summary | A logging handler wrapper to improve logging performace by sending writes to a background thread |
| upload_time | 2025-10-12 19:33:06 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.10 |
| license | None |
| keywords |
logging
log
thread
threading
non-blocking
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# backgroundlog
Thread-based log handler for better performance
[](https://opensource.org/licenses/MIT)
[](https://github.com/diegojromerolopez/backgroundlog/graphs/commit-activity)
[](https://www.python.org/)
[](https://github.com/psf/black)
[](https://pycqa.github.io/isort/)
[](http://mypy-lang.org/)
[](https://pypi.python.org/pypi/backgroundlog/)
[](https://pypi.python.org/pypi/backgroundlog/)
[](https://pypi.python.org/pypi/backgroundlog/)
[](https://pypi.python.org/pypi/backgroundlog/)
Do not have your python program slowed down by your logging.
## Introduction
Most of the time we log on disk, and all I/O is time consuming.
By leveraging a thread, it is possible to speed up your python application
considerably.
## Use
### Default use
```python
import logging
from backgroundlog.handlers.thread_handler import ThreadHandler
# Setting up the logging thread handler
file_handler = logging.FileHandler('/var/log/myapp.log', mode="a", encoding="utf-8")
thread_handler = ThreadHandler(file_handler)
# Creating a new logger
bg_logger = logging.getLogger('bg_logger')
bg_logger.setLevel(logging.INFO)
# Adding the thread handler
bg_logger.addHandler(thread_handler)
# Using the logger
bg_logger.info('This is a log message')
```
### Options
#### Set a queue size
```python
from backgroundlog.handlers.thread_handler import ThreadHandler
thread_handler = ThreadHandler(file_handler, queue_size=5000)
```
By default, the queue size is 1000.
#### Set a blocking policy by logging record levels
When putting the records in the queue, it could reach the queue size.
We provide a way to deal with this issue: set a blocking policy
by logging record level, and in the case of a non-blocking policy,
the record will be discarded and we will increment a dropped log record.
##### Only info, error and critical records are blocking:
```python
from backgroundlog.handlers.thread_handler import ThreadHandler
from logging import CRITICAL, ERROR, INFO
thread_handler = ThreadHandler(file_handler, blocking_levels={INFO, ERROR, CRITICAL})
```
##### Only error and critical records are blocking
```python
from backgroundlog.handlers.thread_handler import ThreadHandler
from logging import CRITICAL, ERROR
thread_handler = ThreadHandler(file_handler, blocking_levels={ERROR, CRITICAL})
```
##### Only critical records are blocking
```python
from backgroundlog.handlers.thread_handler import ThreadHandler
from logging import CRITICAL
thread_handler = ThreadHandler(file_handler, blocking_levels={CRITICAL})
```
##### No records are blocking
```python
from backgroundlog.handlers.thread_handler import ThreadHandler
thread_handler = ThreadHandler(file_handler, blocking_levels=None)
```
By default, the error and critical records are blocking, the rest are not.
## Performance testing
We have done several local testing with different logging handlers.
See the file
[run_performance_comparison.py](/backgroundlog/performance/run_performance_comparison.py) for
a full catalog of the performance tests we run.
We used two versions of Python depending on if they had or not the global interpret lock:
- Python 3.15.3 (with GIL)
- Python 3.15.3t (without GIL)
All tests are 100_000 iterations of creating the same logging message,
and were run in a Macbook Pro M1 with 16 GB of RAM.
### Python 3.15.3 (with GIL)
| Logging Handler | Spent Time | | vs. Baseline |
|-------------------------------|----------------|--------------|--------------|
| | Mean Time (ms) | Std Dev (ms) | |
| StreamHandler | 0.685 | 0.006 | baseline |
| FileHandler | 0.685 | 0.018 | +0.03% |
| ThreadHandler (StreamHandler) | 0.487 | 0.03 | -28.911% |
| ThreadHandler (FileHandler) | 0.475 | 0.002 | -30.66% |
There is a ~30% of improvement when running the thread handler.
### Python 3.15.3t (without GIL)
| Logging Handler | Spent Time | | vs. Baseline |
|-------------------------------|----------------|--------------|--------------|
| | Mean Time (ms) | Std Dev (ms) | |
| StreamHandler | 0.539 | 0.004 | baseline |
| FileHandler | 0.545 | 0.013 | +1.109% |
| ThreadHandler (StreamHandler) | 0.344 | 0.002 | -36.301% |
| ThreadHandler (FileHandler) | 0.339 | 0.001 | -37.118% |
There is a ~36% of improvement when running the thread handler. +6% with respect of the
Python version with GIL.
### Conclusions
Not blocking the main flow of your program by making use of backgroundlog
gives you a 30% speed gain.
## Dependencies
This package has no dependencies.
## Python version support
Minimum version support is 3.10.
## License
[MIT](LICENSE) license, but if you need any other contact me.
Raw data
{
"_id": null,
"home_page": null,
"name": "backgroundlog",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "logging, log, thread, threading, non-blocking",
"author": null,
"author_email": "\"Diego J. Romero L\u00f3pez\" <diegojromerolopez@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/48/bc/7bc7febc28732858e876f4727ada003b14251ba7f54eb98a07292fe7fc9a/backgroundlog-0.2.6.tar.gz",
"platform": null,
"description": "# backgroundlog\nThread-based log handler for better performance\n\n\n[](https://opensource.org/licenses/MIT)\n[](https://github.com/diegojromerolopez/backgroundlog/graphs/commit-activity)\n[](https://www.python.org/)\n[](https://github.com/psf/black)\n[](https://pycqa.github.io/isort/)\n[](http://mypy-lang.org/)\n[](https://pypi.python.org/pypi/backgroundlog/)\n[](https://pypi.python.org/pypi/backgroundlog/)\n[](https://pypi.python.org/pypi/backgroundlog/)\n[](https://pypi.python.org/pypi/backgroundlog/)\n\nDo not have your python program slowed down by your logging.\n\n## Introduction\nMost of the time we log on disk, and all I/O is time consuming.\nBy leveraging a thread, it is possible to speed up your python application\nconsiderably.\n\n## Use\n\n### Default use\n\n```python\nimport logging\nfrom backgroundlog.handlers.thread_handler import ThreadHandler\n\n# Setting up the logging thread handler\nfile_handler = logging.FileHandler('/var/log/myapp.log', mode=\"a\", encoding=\"utf-8\")\nthread_handler = ThreadHandler(file_handler)\n\n# Creating a new logger\nbg_logger = logging.getLogger('bg_logger')\nbg_logger.setLevel(logging.INFO)\n\n# Adding the thread handler\nbg_logger.addHandler(thread_handler)\n\n# Using the logger\nbg_logger.info('This is a log message')\n```\n\n### Options\n\n#### Set a queue size\n\n```python\nfrom backgroundlog.handlers.thread_handler import ThreadHandler\n\nthread_handler = ThreadHandler(file_handler, queue_size=5000)\n```\n\nBy default, the queue size is 1000.\n\n#### Set a blocking policy by logging record levels\n\nWhen putting the records in the queue, it could reach the queue size.\nWe provide a way to deal with this issue: set a blocking policy\nby logging record level, and in the case of a non-blocking policy,\nthe record will be discarded and we will increment a dropped log record.\n\n##### Only info, error and critical records are blocking:\n\n```python\nfrom backgroundlog.handlers.thread_handler import ThreadHandler\nfrom logging import CRITICAL, ERROR, INFO\n\nthread_handler = ThreadHandler(file_handler, blocking_levels={INFO, ERROR, CRITICAL})\n```\n\n##### Only error and critical records are blocking\n\n```python\nfrom backgroundlog.handlers.thread_handler import ThreadHandler\nfrom logging import CRITICAL, ERROR\n\nthread_handler = ThreadHandler(file_handler, blocking_levels={ERROR, CRITICAL})\n```\n\n##### Only critical records are blocking\n\n```python\nfrom backgroundlog.handlers.thread_handler import ThreadHandler\nfrom logging import CRITICAL\n\nthread_handler = ThreadHandler(file_handler, blocking_levels={CRITICAL})\n```\n\n##### No records are blocking\n\n```python\nfrom backgroundlog.handlers.thread_handler import ThreadHandler\n\nthread_handler = ThreadHandler(file_handler, blocking_levels=None)\n```\n\nBy default, the error and critical records are blocking, the rest are not.\n\n## Performance testing\n\nWe have done several local testing with different logging handlers.\nSee the file\n[run_performance_comparison.py](/backgroundlog/performance/run_performance_comparison.py) for\na full catalog of the performance tests we run.\n\nWe used two versions of Python depending on if they had or not the global interpret lock:\n- Python 3.15.3 (with GIL)\n- Python 3.15.3t (without GIL)\n\nAll tests are 100_000 iterations of creating the same logging message,\nand were run in a Macbook Pro M1 with 16 GB of RAM.\n\n### Python 3.15.3 (with GIL)\n\n| Logging Handler | Spent Time | | vs. Baseline |\n|-------------------------------|----------------|--------------|--------------|\n| | Mean Time (ms) | Std Dev (ms) | |\n| StreamHandler | 0.685 | 0.006 | baseline |\n| FileHandler | 0.685 | 0.018 | +0.03% |\n| ThreadHandler (StreamHandler) | 0.487 | 0.03 | -28.911% |\n| ThreadHandler (FileHandler) | 0.475 | 0.002 | -30.66% |\n\nThere is a ~30% of improvement when running the thread handler.\n\n### Python 3.15.3t (without GIL)\n\n| Logging Handler | Spent Time | | vs. Baseline |\n|-------------------------------|----------------|--------------|--------------|\n| | Mean Time (ms) | Std Dev (ms) | |\n| StreamHandler | 0.539 | 0.004 | baseline |\n| FileHandler | 0.545 | 0.013 | +1.109% |\n| ThreadHandler (StreamHandler) | 0.344 | 0.002 | -36.301% |\n| ThreadHandler (FileHandler) | 0.339 | 0.001 | -37.118% |\n\nThere is a ~36% of improvement when running the thread handler. +6% with respect of the\nPython version with GIL.\n\n### Conclusions\nNot blocking the main flow of your program by making use of backgroundlog\ngives you a 30% speed gain.\n\n## Dependencies\nThis package has no dependencies.\n\n## Python version support\nMinimum version support is 3.10.\n\n## License\n[MIT](LICENSE) license, but if you need any other contact me.\n",
"bugtrack_url": null,
"license": null,
"summary": "A logging handler wrapper to improve logging performace by sending writes to a background thread",
"version": "0.2.6",
"project_urls": {
"Bug Tracker": "https://github.com/diegojromerolopez/backgroundlog/issues",
"documentation": "https://github.com/diegojromerolopez/backgroundlog/blob/main/README.md",
"repository": "https://github.com/diegojromerolopez/backgroundlog"
},
"split_keywords": [
"logging",
" log",
" thread",
" threading",
" non-blocking"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "60b53a3ac0483ef1749ddd06770bddba54d816802770b414d36ca0be4ec6563e",
"md5": "d0d47f4e2a73ed87a5727e9fb26c85b2",
"sha256": "8ee267f0f07669d38fe16d4a2ace7ad25b34f9ae1fb2d998264a92fbca663953"
},
"downloads": -1,
"filename": "backgroundlog-0.2.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d0d47f4e2a73ed87a5727e9fb26c85b2",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 8818,
"upload_time": "2025-10-12T19:33:05",
"upload_time_iso_8601": "2025-10-12T19:33:05.186467Z",
"url": "https://files.pythonhosted.org/packages/60/b5/3a3ac0483ef1749ddd06770bddba54d816802770b414d36ca0be4ec6563e/backgroundlog-0.2.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "48bc7bc7febc28732858e876f4727ada003b14251ba7f54eb98a07292fe7fc9a",
"md5": "7420b92544645b846e60190373089893",
"sha256": "3c6073600df61219961e4d3f85d48e352d0f87b4e8943520ecffe7cf0682c5f1"
},
"downloads": -1,
"filename": "backgroundlog-0.2.6.tar.gz",
"has_sig": false,
"md5_digest": "7420b92544645b846e60190373089893",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 9000,
"upload_time": "2025-10-12T19:33:06",
"upload_time_iso_8601": "2025-10-12T19:33:06.639603Z",
"url": "https://files.pythonhosted.org/packages/48/bc/7bc7febc28732858e876f4727ada003b14251ba7f54eb98a07292fe7fc9a/backgroundlog-0.2.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-12 19:33:06",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "diegojromerolopez",
"github_project": "backgroundlog",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "backgroundlog"
}
