# tail-jsonl
Tail JSONL Logs
## Background
I wanted to find a tool that could:
1. Convert a stream of JSONL logs into a readable `logfmt`-like output with minimal configuration
1. Show exceptions on their own line
I investigated a lot of alternatives such as: [humanlog](https://github.com/humanlogio/humanlog), [lnav](https://docs.lnav.org/en/latest/formats.html#), [goaccess](https://goaccess.io/get-started), [angle-grinder](https://github.com/rcoh/angle-grinder#rendering), [jq](https://github.com/stedolan/jq), [textualog](https://github.com/rhuygen/textualog), [logss](https://github.com/todoesverso/logss), etc. but nothing would both cleanly format the JSONL data and show the exception.
## Installation
[Install with `pipx`](https://pypi.org/project/pipx/)
```sh
pipx install tail-jsonl
```
## Usage
Pipe JSONL output from any file, kubernetes (such as [stern](https://github.com/stern/stern)), Docker, etc.
```sh
# Example piping input in shell
echo '{"message": "message", "timestamp": "2023-01-01T01:01:01.0123456Z", "level": "debug", "data": true, "more-data": [null, true, -123.123]}' | tail-jsonl
cat tests/data/logs.jsonl | tail-jsonl
# Optionally, pre-filter or format with jq, grep, awk, or other tools
cat tests/data/logs.jsonl | jq '.record' --compact-output | tail-jsonl
# An example stern command (also consider -o=extjson)
stern envvars --context staging --container gateway --since="60m" --output raw | tail-jsonl
# Or with Docker Compose (note that awk, cut, and grep all buffer. For awk, add '; system("")')
docker compose logs --follow | awk 'match($0, / \| \{.+/) { print substr($0, RSTART+3, RLENGTH); system("") }' | tail-jsonl
```
## Configuration
Optionally, specify a path to a custom configuration file. For an example configuration file see: [./tests/config_default.toml](https://github.com/KyleKing/tail-jsonl/blob/main/tests/config_default.toml)
```sh
echo '...' | tail-jsonl --config-path=~/.tail-jsonl.toml
```
## Project Status
See the `Open Issues` and/or the [CODE_TAG_SUMMARY]. For release history, see the [CHANGELOG].
## Contributing
We welcome pull requests! For your pull request to be accepted smoothly, we suggest that you first open a GitHub issue to discuss your idea. For resources on getting started with the code base, see the below documentation:
- [DEVELOPER_GUIDE]
- [STYLE_GUIDE]
## Code of Conduct
We follow the [Contributor Covenant Code of Conduct][contributor-covenant].
### Open Source Status
We try to reasonably meet most aspects of the "OpenSSF scorecard" from [Open Source Insights](https://deps.dev/pypi/tail-jsonl)
## Responsible Disclosure
If you have any security issue to report, please contact the project maintainers privately. You can reach us at [dev.act.kyle@gmail.com](mailto:dev.act.kyle@gmail.com).
## License
[LICENSE]
[changelog]: https://tail-jsonl.kyleking.me/docs/CHANGELOG
[code_tag_summary]: https://tail-jsonl.kyleking.me/docs/CODE_TAG_SUMMARY
[contributor-covenant]: https://www.contributor-covenant.org
[developer_guide]: https://tail-jsonl.kyleking.me/docs/DEVELOPER_GUIDE
[license]: https://github.com/kyleking/tail-jsonl/blob/main/LICENSE
[style_guide]: https://tail-jsonl.kyleking.me/docs/STYLE_GUIDE
Raw data
{
"_id": null,
"home_page": "https://github.com/kyleking/tail-jsonl",
"name": "tail_jsonl",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0.0,>=3.9.13",
"maintainer_email": null,
"keywords": "calcipy_template",
"author": "Kyle King",
"author_email": "dev.act.kyle@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/e0/e1/7c2a02532759fa47c682e1a53bcfc84cf70c584f3d467bc544914077acf2/tail_jsonl-1.2.6.tar.gz",
"platform": null,
"description": "# tail-jsonl\n\nTail JSONL Logs\n\n## Background\n\nI wanted to find a tool that could:\n\n1. Convert a stream of JSONL logs into a readable `logfmt`-like output with minimal configuration\n1. Show exceptions on their own line\n\nI investigated a lot of alternatives such as: [humanlog](https://github.com/humanlogio/humanlog), [lnav](https://docs.lnav.org/en/latest/formats.html#), [goaccess](https://goaccess.io/get-started), [angle-grinder](https://github.com/rcoh/angle-grinder#rendering), [jq](https://github.com/stedolan/jq), [textualog](https://github.com/rhuygen/textualog), [logss](https://github.com/todoesverso/logss), etc. but nothing would both cleanly format the JSONL data and show the exception.\n\n\n\n## Installation\n\n[Install with `pipx`](https://pypi.org/project/pipx/)\n\n```sh\npipx install tail-jsonl\n```\n\n## Usage\n\nPipe JSONL output from any file, kubernetes (such as [stern](https://github.com/stern/stern)), Docker, etc.\n\n```sh\n# Example piping input in shell\necho '{\"message\": \"message\", \"timestamp\": \"2023-01-01T01:01:01.0123456Z\", \"level\": \"debug\", \"data\": true, \"more-data\": [null, true, -123.123]}' | tail-jsonl\ncat tests/data/logs.jsonl | tail-jsonl\n\n# Optionally, pre-filter or format with jq, grep, awk, or other tools\ncat tests/data/logs.jsonl | jq '.record' --compact-output | tail-jsonl\n\n# An example stern command (also consider -o=extjson)\nstern envvars --context staging --container gateway --since=\"60m\" --output raw | tail-jsonl\n\n# Or with Docker Compose (note that awk, cut, and grep all buffer. For awk, add '; system(\"\")')\ndocker compose logs --follow | awk 'match($0, / \\| \\{.+/) { print substr($0, RSTART+3, RLENGTH); system(\"\") }' | tail-jsonl\n```\n\n## Configuration\n\nOptionally, specify a path to a custom configuration file. For an example configuration file see: [./tests/config_default.toml](https://github.com/KyleKing/tail-jsonl/blob/main/tests/config_default.toml)\n\n```sh\necho '...' | tail-jsonl --config-path=~/.tail-jsonl.toml\n```\n\n## Project Status\n\nSee the `Open Issues` and/or the [CODE_TAG_SUMMARY]. For release history, see the [CHANGELOG].\n\n## Contributing\n\nWe welcome pull requests! For your pull request to be accepted smoothly, we suggest that you first open a GitHub issue to discuss your idea. For resources on getting started with the code base, see the below documentation:\n\n- [DEVELOPER_GUIDE]\n- [STYLE_GUIDE]\n\n## Code of Conduct\n\nWe follow the [Contributor Covenant Code of Conduct][contributor-covenant].\n\n### Open Source Status\n\nWe try to reasonably meet most aspects of the \"OpenSSF scorecard\" from [Open Source Insights](https://deps.dev/pypi/tail-jsonl)\n\n## Responsible Disclosure\n\nIf you have any security issue to report, please contact the project maintainers privately. You can reach us at [dev.act.kyle@gmail.com](mailto:dev.act.kyle@gmail.com).\n\n## License\n\n[LICENSE]\n\n[changelog]: https://tail-jsonl.kyleking.me/docs/CHANGELOG\n[code_tag_summary]: https://tail-jsonl.kyleking.me/docs/CODE_TAG_SUMMARY\n[contributor-covenant]: https://www.contributor-covenant.org\n[developer_guide]: https://tail-jsonl.kyleking.me/docs/DEVELOPER_GUIDE\n[license]: https://github.com/kyleking/tail-jsonl/blob/main/LICENSE\n[style_guide]: https://tail-jsonl.kyleking.me/docs/STYLE_GUIDE\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Pretty Print Tailed JSONL Logs",
"version": "1.2.6",
"project_urls": {
"Bug Tracker": "https://github.com/kyleking/tail-jsonl/issues",
"Changelog": "https://github.com/kyleking/tail-jsonl/blob/main/docs/docs/CHANGELOG.md",
"Documentation": "https://tail-jsonl.kyleking.me",
"Homepage": "https://github.com/kyleking/tail-jsonl",
"Repository": "https://github.com/kyleking/tail-jsonl"
},
"split_keywords": [
"calcipy_template"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "cb210522d50c819aaa28ba34d9c36631f90762661317e9de713cba33c4d7a445",
"md5": "e368b1aa400cd6fe9c0de5ab24ef0f0a",
"sha256": "95f1b63c0f3ad2510d7f20a46a1c45e13f715a25b816e1c467b1b1ed96f0121b"
},
"downloads": -1,
"filename": "tail_jsonl-1.2.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e368b1aa400cd6fe9c0de5ab24ef0f0a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0.0,>=3.9.13",
"size": 8098,
"upload_time": "2024-10-05T15:14:21",
"upload_time_iso_8601": "2024-10-05T15:14:21.448872Z",
"url": "https://files.pythonhosted.org/packages/cb/21/0522d50c819aaa28ba34d9c36631f90762661317e9de713cba33c4d7a445/tail_jsonl-1.2.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e0e17c2a02532759fa47c682e1a53bcfc84cf70c584f3d467bc544914077acf2",
"md5": "44917e972751845f3311914b5cd92b79",
"sha256": "a447ba24d8f925e5f806e6048dd079af94e99069ec94594f1d3960cd7c7e5c32"
},
"downloads": -1,
"filename": "tail_jsonl-1.2.6.tar.gz",
"has_sig": false,
"md5_digest": "44917e972751845f3311914b5cd92b79",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0.0,>=3.9.13",
"size": 7848,
"upload_time": "2024-10-05T15:14:23",
"upload_time_iso_8601": "2024-10-05T15:14:23.097366Z",
"url": "https://files.pythonhosted.org/packages/e0/e1/7c2a02532759fa47c682e1a53bcfc84cf70c584f3d467bc544914077acf2/tail_jsonl-1.2.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-05 15:14:23",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "kyleking",
"github_project": "tail-jsonl",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "tail_jsonl"
}
JSON
