| Name | splitlog JSON |
| Version |
4.0.0
JSON |
| download |
| home_page | https://github.com/splitlog/splitlog.git |
| Summary | Utility to split aggregated logs from Apache Hadoop Yarn applications into a folder hierarchy |
| upload_time | 2024-10-26 18:08:04 |
| maintainer | None |
| docs_url | None |
| author | Sebastian Klemke |
| requires_python | <4.0.0,>=3.9.0 |
| license | MIT |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
splitlog
========
Hadoop Yarn application logs aggregate all container logs of a Yarn application into a single file. This makes it very
difficult to use Unix command line tools to analyze these logs: Grep will search over all containers and context
provided for hits often does not include Yarn container name or host name. `splitlog` splits a combined logfile for all
containers of an application into a file system hierarchy suitable for further analysis:
```
out
└── hadoopnode
├── container_1671326373437_0001_01_000001
│ ├── directory.info
│ ├── launch_container.sh
│ ├── prelaunch.err
│ ├── prelaunch.out
│ ├── stderr
│ ├── stdout
│ └── syslog
├── container_1671326373437_0001_01_000002
│ ├── directory.info
│ ├── launch_container.sh
│ ├── prelaunch.err
│ ├── prelaunch.out
│ ├── stderr
│ ├── stdout
│ └── syslog
└── container_1671326373437_0001_01_000003
├── directory.info
├── launch_container.sh
├── prelaunch.err
├── prelaunch.out
├── stderr
├── stdout
└── syslog
4 directories, 21 files
```
Installation
------------
Python 3.9+ must be available. Installation via [pipx](https://pypi.org/project/pipx/):
```shell script
pipx install splitlog
```
How to use
----------
Read logs from standard input:
```shell script
yarn logs -applicationId application_1582815261257_232080 | splitlog
```
Read logs from file `application_1582815261257_232080.log`:
```shell script
splitlog -i application_1582815261257_232080.log
```
Raw data
{
"_id": null,
"home_page": "https://github.com/splitlog/splitlog.git",
"name": "splitlog",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0.0,>=3.9.0",
"maintainer_email": null,
"keywords": null,
"author": "Sebastian Klemke",
"author_email": "pypi@nerdheim.de",
"download_url": "https://files.pythonhosted.org/packages/6b/0d/1b2685d1c996b0e5803e3a9f440ff9bc8d483c85aea419e039de835523d4/splitlog-4.0.0.tar.gz",
"platform": null,
"description": "splitlog\n========\n \nHadoop Yarn application logs aggregate all container logs of a Yarn application into a single file. This makes it very\ndifficult to use Unix command line tools to analyze these logs: Grep will search over all containers and context\nprovided for hits often does not include Yarn container name or host name. `splitlog` splits a combined logfile for all\ncontainers of an application into a file system hierarchy suitable for further analysis:\n\n```\nout\n\u2514\u2500\u2500 hadoopnode\n \u251c\u2500\u2500 container_1671326373437_0001_01_000001\n \u2502 \u251c\u2500\u2500 directory.info\n \u2502 \u251c\u2500\u2500 launch_container.sh\n \u2502 \u251c\u2500\u2500 prelaunch.err\n \u2502 \u251c\u2500\u2500 prelaunch.out\n \u2502 \u251c\u2500\u2500 stderr\n \u2502 \u251c\u2500\u2500 stdout\n \u2502 \u2514\u2500\u2500 syslog\n \u251c\u2500\u2500 container_1671326373437_0001_01_000002\n \u2502 \u251c\u2500\u2500 directory.info\n \u2502 \u251c\u2500\u2500 launch_container.sh\n \u2502 \u251c\u2500\u2500 prelaunch.err\n \u2502 \u251c\u2500\u2500 prelaunch.out\n \u2502 \u251c\u2500\u2500 stderr\n \u2502 \u251c\u2500\u2500 stdout\n \u2502 \u2514\u2500\u2500 syslog\n \u2514\u2500\u2500 container_1671326373437_0001_01_000003\n \u251c\u2500\u2500 directory.info\n \u251c\u2500\u2500 launch_container.sh\n \u251c\u2500\u2500 prelaunch.err\n \u251c\u2500\u2500 prelaunch.out\n \u251c\u2500\u2500 stderr\n \u251c\u2500\u2500 stdout\n \u2514\u2500\u2500 syslog\n\n4 directories, 21 files\n```\n \nInstallation\n------------\nPython 3.9+ must be available. Installation via [pipx](https://pypi.org/project/pipx/):\n\n```shell script\npipx install splitlog\n```\n \nHow to use\n----------\n\nRead logs from standard input:\n```shell script\nyarn logs -applicationId application_1582815261257_232080 | splitlog\n```\n\nRead logs from file `application_1582815261257_232080.log`:\n```shell script\nsplitlog -i application_1582815261257_232080.log\n```\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Utility to split aggregated logs from Apache Hadoop Yarn applications into a folder hierarchy",
"version": "4.0.0",
"project_urls": {
"Homepage": "https://github.com/splitlog/splitlog.git",
"Repository": "https://github.com/splitlog/splitlog.git"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3db4e5f6c5c60e50e2e4d18ac22dbf3ab9bcef078844e9a056ad088b5df102f2",
"md5": "46ea19eeb43c9342bcfa224906c534d6",
"sha256": "c8b75a16bcd363c160d36f607c5d8e5e41a19123837956e176523010b15604b0"
},
"downloads": -1,
"filename": "splitlog-4.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "46ea19eeb43c9342bcfa224906c534d6",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0.0,>=3.9.0",
"size": 8846,
"upload_time": "2024-10-26T18:08:02",
"upload_time_iso_8601": "2024-10-26T18:08:02.853997Z",
"url": "https://files.pythonhosted.org/packages/3d/b4/e5f6c5c60e50e2e4d18ac22dbf3ab9bcef078844e9a056ad088b5df102f2/splitlog-4.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "6b0d1b2685d1c996b0e5803e3a9f440ff9bc8d483c85aea419e039de835523d4",
"md5": "9dc2b7b7f5a4bd6b1be802342fcf0c51",
"sha256": "07864cf5d6713358022f2a23c0dc3e6c35f63adceddb490bc247ad7855ee91c2"
},
"downloads": -1,
"filename": "splitlog-4.0.0.tar.gz",
"has_sig": false,
"md5_digest": "9dc2b7b7f5a4bd6b1be802342fcf0c51",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0.0,>=3.9.0",
"size": 8082,
"upload_time": "2024-10-26T18:08:04",
"upload_time_iso_8601": "2024-10-26T18:08:04.964330Z",
"url": "https://files.pythonhosted.org/packages/6b/0d/1b2685d1c996b0e5803e3a9f440ff9bc8d483c85aea419e039de835523d4/splitlog-4.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-26 18:08:04",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "splitlog",
"github_project": "splitlog",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "splitlog"
}
