# Flask Real Time Log Viewer
## Real time log view
Display logs in real time through a browser with flask.
## Background
Looking at a few different projects and writes up showing how to stream logs to a browser and found
they relied on WebSockets (wss), and Redis. Which complicates simple projects and increases expense.
Flask / Python at large has a few hurdles
* Websockets in gunicorns works, but only if you have 1 worker
* This limits performance even with threads
* uWSGI - maybe if you're willing to compile it
* Dockerized slim images require the addition of gcc slowing the build down
* Flasks Werkzeug isn't production hardened.
So the criteria for this project was
* A simple web based real time log viewer
* Could run across workers (forks)
* Did not rely on WSS (Websockets)
This was designed using gunicorns but should work with any wsgi server.
## Install & Configuration
This is a flask blueprint, and might not suit everyones setup but feel free to suggest changes.
```sh
pip install flask-log-viewer
```
Within your main flask app configure the following
```python
from flask_log_viewer.viewer import log_viewer_blueprint
import os
log_directory = "/app/logs" #Directory logs are stored in
app = Flask(__name__)
log_viewer = log_viewer_blueprint(base_path=log_directory, allowed_directories=[log_directory])
app.register_blueprint(log_viewer, url_prefix='/logs')
......
......
```
This will now create the following urls
```
/logs/stream/<string: log_file>
```
Thus http://hostname/logs/stream/foo.log will look for a /app/logs/foo.log and begin tailing it emitting http stream events. Feel free to change the prefix as needed.
If you have logs in subdirectories like /app/logs/monday/foo.log
* ensure /app/logs/monday is added to allowed_directories
* the url will now be http://hostname/logs/stream/monday/foo.log
## Using the included viewer
A reference viewer is included, using htmx to make it easier. This can be copied and modified to meet your needs and frameworks.
The viewer is at http://hostname/logs/view/ without a log name, a random set of strings is streamed back.
Include a log file with http://hostname/logs/view/file-name
## How logs are tailed
Under the covers we are using pygtail https://pypi.org/project/pygtail/ | https://github.com/bgreenlee/pygtail
from Brad Greenlee. Pygtail is GPL licensed, use accordingly
>Pygtail monitors log files for rotation, keeps track of the seek position, ensuring every time it's called, it goes back to the last position of the file being read from, which is stored on disk as 'file-name.log.offset'
This is useful in Server Side Events as sockets close and additional requests can be made by a client that can end up at any WSGI worker.
However it can also prevent a file from being re-read at the moment.
Future //TODO: provide a method to reset
## Files
What to look for if git cloning
```
├── LICENSE
├── README.md
├── app.py
├── app_logger
│ └── logger.py
├── flask_log_viewer
│ ├── static
│ │ └── js
│ │ └── ansi_html.js
│ ├── templates
│ │ └── viewer.html
│ └── viewer.py
├── poetry.lock
└── pyproject.toml
```
* app.py sample flask app
* app_logger - testing logger
* flask_log_viewer
* viewer.py blueprint code
* templates/viewer.html - a htmx log viewer
* static/js/ansi_html.js - basic ansi colors to html converter
## Running with gunicorn
Using gevent with gunicorns reduces the amount of connect closes you receive and is recommended for production level hosting.
```
gunicorn -k gevent -w 4 -b 0.0.0.0:3001 --access-logfile - --error-logfile - app:app
```
Raw data
{
"_id": null,
"home_page": "https://github.com/thevgergroup/flask-log-viewer",
"name": "flask-log-viewer",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.11,<4.0",
"maintainer_email": "",
"keywords": "flask,log,viewer,tail,logviewer,log-viewer,log-tail,logtail,log-tailer,htmx",
"author": "patrick o'leary",
"author_email": "pjaol@pjaol.com",
"download_url": "https://files.pythonhosted.org/packages/52/46/faeee74e9339a6e5de59a3ef50bda4ddd040cd2a46415120e855e358f9a6/flask_log_viewer-0.2.2.tar.gz",
"platform": null,
"description": "# Flask Real Time Log Viewer\n\n\n## Real time log view\n\n\nDisplay logs in real time through a browser with flask.\n\n\n## Background\nLooking at a few different projects and writes up showing how to stream logs to a browser and found\nthey relied on WebSockets (wss), and Redis. Which complicates simple projects and increases expense. \nFlask / Python at large has a few hurdles\n\n* Websockets in gunicorns works, but only if you have 1 worker\n * This limits performance even with threads \n* uWSGI - maybe if you're willing to compile it\n * Dockerized slim images require the addition of gcc slowing the build down\n* Flasks Werkzeug isn't production hardened. \n\nSo the criteria for this project was\n* A simple web based real time log viewer\n* Could run across workers (forks) \n* Did not rely on WSS (Websockets)\n\nThis was designed using gunicorns but should work with any wsgi server.\n\n\n## Install & Configuration\n\nThis is a flask blueprint, and might not suit everyones setup but feel free to suggest changes.\n\n```sh\npip install flask-log-viewer\n\n```\n\nWithin your main flask app configure the following\n\n```python\n\nfrom flask_log_viewer.viewer import log_viewer_blueprint\nimport os\n\nlog_directory = \"/app/logs\" #Directory logs are stored in\napp = Flask(__name__)\n\nlog_viewer = log_viewer_blueprint(base_path=log_directory, allowed_directories=[log_directory])\n\napp.register_blueprint(log_viewer, url_prefix='/logs')\n......\n......\n\n```\n\nThis will now create the following urls\n\n```\n/logs/stream/<string: log_file>\n```\nThus http://hostname/logs/stream/foo.log will look for a /app/logs/foo.log and begin tailing it emitting http stream events. Feel free to change the prefix as needed.\n\nIf you have logs in subdirectories like /app/logs/monday/foo.log \n* ensure /app/logs/monday is added to allowed_directories\n* the url will now be http://hostname/logs/stream/monday/foo.log\n\n\n## Using the included viewer\nA reference viewer is included, using htmx to make it easier. This can be copied and modified to meet your needs and frameworks. \n\nThe viewer is at http://hostname/logs/view/ without a log name, a random set of strings is streamed back.\nInclude a log file with http://hostname/logs/view/file-name\n\n## How logs are tailed\nUnder the covers we are using pygtail https://pypi.org/project/pygtail/ | https://github.com/bgreenlee/pygtail\nfrom Brad Greenlee. Pygtail is GPL licensed, use accordingly \n\n>Pygtail monitors log files for rotation, keeps track of the seek position, ensuring every time it's called, it goes back to the last position of the file being read from, which is stored on disk as 'file-name.log.offset'\n\nThis is useful in Server Side Events as sockets close and additional requests can be made by a client that can end up at any WSGI worker. \n\nHowever it can also prevent a file from being re-read at the moment. \nFuture //TODO: provide a method to reset\n\n## Files \n\nWhat to look for if git cloning \n\n```\n\u251c\u2500\u2500 LICENSE\n\u251c\u2500\u2500 README.md\n\u251c\u2500\u2500 app.py\n\u251c\u2500\u2500 app_logger\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 logger.py\n\u251c\u2500\u2500 flask_log_viewer\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 static\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2514\u2500\u2500 js\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2514\u2500\u2500 ansi_html.js\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 templates\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2514\u2500\u2500 viewer.html\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 viewer.py\n\u251c\u2500\u2500 poetry.lock\n\u2514\u2500\u2500 pyproject.toml\n```\n\n* app.py sample flask app\n* app_logger - testing logger\n* flask_log_viewer\n * viewer.py blueprint code\n * templates/viewer.html - a htmx log viewer \n * static/js/ansi_html.js - basic ansi colors to html converter\n\n\n## Running with gunicorn\nUsing gevent with gunicorns reduces the amount of connect closes you receive and is recommended for production level hosting.\n\n\n```\ngunicorn -k gevent -w 4 -b 0.0.0.0:3001 --access-logfile - --error-logfile - app:app \n```\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Real time log viewer for flask applications",
"version": "0.2.2",
"project_urls": {
"Homepage": "https://github.com/thevgergroup/flask-log-viewer",
"Repository": "https://github.com/thevgergroup/flask-log-viewer.git"
},
"split_keywords": [
"flask",
"log",
"viewer",
"tail",
"logviewer",
"log-viewer",
"log-tail",
"logtail",
"log-tailer",
"htmx"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8c3723b6b171bfab39cdfaa76c3e5d02055d103c36d0066a49c5f7ce278260c1",
"md5": "5f685700efcd54ff7a6697b397e4e7d1",
"sha256": "6c2e6598ee5f26ade93d9b91c9beb928889bc3d36f260379647f246cc25e0e9c"
},
"downloads": -1,
"filename": "flask_log_viewer-0.2.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "5f685700efcd54ff7a6697b397e4e7d1",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11,<4.0",
"size": 7183,
"upload_time": "2024-01-28T00:22:34",
"upload_time_iso_8601": "2024-01-28T00:22:34.284763Z",
"url": "https://files.pythonhosted.org/packages/8c/37/23b6b171bfab39cdfaa76c3e5d02055d103c36d0066a49c5f7ce278260c1/flask_log_viewer-0.2.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5246faeee74e9339a6e5de59a3ef50bda4ddd040cd2a46415120e855e358f9a6",
"md5": "4baef507d0dc436c89be4e3b676e079e",
"sha256": "f8d87d8cfa8a5dc5c4fefb25d13aec0924154a716efdc536a8ae10d0dbf106f0"
},
"downloads": -1,
"filename": "flask_log_viewer-0.2.2.tar.gz",
"has_sig": false,
"md5_digest": "4baef507d0dc436c89be4e3b676e079e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11,<4.0",
"size": 6252,
"upload_time": "2024-01-28T00:22:35",
"upload_time_iso_8601": "2024-01-28T00:22:35.282968Z",
"url": "https://files.pythonhosted.org/packages/52/46/faeee74e9339a6e5de59a3ef50bda4ddd040cd2a46415120e855e358f9a6/flask_log_viewer-0.2.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-28 00:22:35",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "thevgergroup",
"github_project": "flask-log-viewer",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "flask-log-viewer"
}
JSON
