| Name | pylogsentinel JSON |
| Version |
0.2.1
JSON |
| download |
| home_page | None |
| Summary | Lightweight cron-friendly log monitoring utility with regex rules and per-rule shell actions |
| upload_time | 2025-10-12 18:35:42 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.9 |
| license | None |
| keywords |
log
monitoring
regex
cron
alerting
sentinel
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# pylogsentinel
A lightweight, zero-dependency (standard library only) log monitoring sentinel intended
to be invoked periodically (e.g. once per minute via `cron`). It incrementally scans
configured log files for user-defined regular expression rules and triggers shell
actions with rich contextual environment variables.
`pylogsentinel` is intentionally simple: no daemons, no background threads, no
long-running processes. Each run processes just the _new_ bytes appended since the
previous run (tracked per-file, per-inode) and then exits.
## Key Features
- INI-style configuration (`pylogsentinel.conf`).
- Two log discovery modes: static paths or shell command
- Per-rule regular expressions with optional flags (`/pattern/i` style).
- Per-rule or actions executed as shell commands with rich contextual environment variables.
- Reliable state tracking via inode-based files (`<inode>` in `state_dir`).
- Safe handling of truncation & rotation (offset reset if file shrinks).
- Strict lock file (`LOCK`) prevents concurrent overlapping runs.
- Skip-actions mode (`--skip-actions`) processes new data but skips executing actions (use on the first run to avoid firing on historical log lines).
- Human-friendly size option for `max_block_size` (e.g. `512K`, `10M`).
- Automatic configuration file discovery when -c/--config is not supplied.
- Directory path monitoring uses the system 'file' command; only paths whose description contains 'text' are monitored (explicit file paths are always processed).
- Python 3.9+ (only standard library imports).
## Installation
```sh
pip install pylogsentinel
```
## Quick Start
1. Write a configuration in a standard location:
```
touch /etc/pylogsentinel.conf
chmod 600 /etc/pylogsentinel.conf
```
2. Edit paths, rules, and actions to your needs.
3. First build state without executing actions (initial run with --skip-actions):
```
python -m pylogsentinel -c /etc/pylogsentinel.conf --skip-actions
```
If you omit the `-c` option (or the specified file does not exist), pylogsentinel will look for a configuration file in this order:
- `~/.pylogsentinel.conf`
- `~/.config/pylogsentinel.conf`
- `/etc/pylogsentinel.conf`
- `/usr/local/etc/pylogsentinel.conf`.
4. Add a cron entry (run every minute):
```
* * * * * /usr/bin/env python3 -m pylogsentinel -c /etc/pylogsentinel.conf
```
## Configuration File Reference
The configuration is an INI-style file. An example:
```
[system]
state_dir = /var/run/pylogsentinel
max_block_size = 10M
[logs]
# Choose exactly one, `paths` or `cmd`:
paths = /var/log /custom/app/logs/app.log
cmd = find /var/log -type f -name '*.log'
[action.default]
cmd = echo -e "Matched $RULE_ID in $FILE at line $LINE, context:\n\n$CONTEXT" | mail -s "Sentinel alert" root
[action.another]
cmd = echo "Another action"
[rule.error]
description = Error-like conditions
pattern = /(error|fatal|exception|kill|crash)/i
action = another # if omitted, use default action
```
### `[system]` Section
| Option | Required | Description | Default |
| ---------------- | -------- | ------------------------------------------------------------------------------------------------------- | ------- |
| `state_dir` | Yes | Directory storing lock + per-inode state files. Must be writable. (Defined under `[system]`.) | (none) |
| `max_block_size` | No | Maximum newly appended bytes to read per file per run. Supports suffixes `K`, `M`, `G` (in `[system]`). | `10M` |
### `[logs]` Section
Supply **either**:
- `paths`: Space-separated list of file and/or directory paths. Directories are traversed recursively; only files whose `file -b` output contains the substring "text" are monitored (others are skipped). Explicit file paths are always processed.
- `cmd`: A shell command producing **one path per line** on stdout.
Exactly one must be present. If using `cmd`, ensure it returns relatively quickly (recommended under a second) as it executes on every run.
### `[rule.<rule_id>]` Sections
| Field | Required | Description |
| ------------- | -------- | -------------------------------------------------------------------------------------------------------------- |
| `pattern` | Yes | Regular expression in the mandatory form `/pattern/flags` (flags optional). Supported flags: `i` (IGNORECASE). |
| `description` | No | Human-readable description for environment variable `RULE_DESCRIPTION`. |
| `action` | No | The `action_id` to invoke. Defaults to `default` if omitted. |
At least one rule is required.
#### Pattern Forms Examples
| Pattern | Meaning |
| ----------------------- | --------------------------------------------------- |
| `/error/i` | Case-insensitive match of "error" |
| `/^panic:.+/` | Match lines starting with "panic:" (case-sensitive) |
| `/^panic:.+/i` | Same, case-insensitive |
| `/\b(?:CRIT\|FATAL)\b/` | Word boundary critical terms |
If you supply an invalid regex or an unsupported flag, startup will fail with a configuration error.
### `[action.<action_id>]` Sections
Each action defines a shell `cmd` executed when a matching rule triggers. The special action id `default` **must** exist; it is used as a fallback when a rule references it explicitly or omits `action`.
Environment variables available to the command:
| Variable | Description |
| ------------------ | ---------------------------------------------------------------------- |
| `RULE_ID` | The rule identifier (e.g. `error`). |
| `RULE_PATTERN` | The normalized pattern string (e.g. `/error/i`). |
| `RULE_DESCRIPTION` | Description text or placeholder string if not provided. |
| `FILE` | Absolute path of the log file where the match occurred. |
| `LINE` | 1-based absolute line number within the file at time of scan. |
| `CONTEXT` | Concatenated lines around the match (default radius 2 before & after). |
Your `cmd` can reference these with typical shell expansion, for example:
```
[action.notify]
cmd = printf "%s\n%s\n" "$RULE_DESCRIPTION" "$CONTEXT" | mail -s "Alert $RULE_ID: $FILE:$LINE" ops@example.com
```
## State Tracking
For each processed file:
1. The file's inode (`st_ino`) is determined.
2. A state file named `<inode>` inside `state_dir` stores:
```
<byte_offset> <line_number>
```
3. On the next run only new bytes (up to `max_block_size`) beyond `<byte_offset>` are read.
4. If the file shrinks (truncation), the stored offset is treated as invalid and scanning restarts from the beginning with line numbers reset.
This scheme tolerates typical log rotation patterns.
## Lock File
A lock file named `LOCK` in `state_dir` prevents overlapping runs. If it exists the process exits immediately. Remove a stale lock only after verifying no instance is active.
## Dry Run
`--skip-actions` processes new data but does not run the actions (recommended for the very first run to prevent alert storms from historical data).
## Exit Codes
| Code | Meaning |
| ---- | -------------------- |
| 0 | Success |
| 2 | Configuration error |
| 130 | Interrupted (Ctrl+C) |
| 1 | Other runtime error |
## License
MIT License (see LICENSE).
Raw data
{
"_id": null,
"home_page": null,
"name": "pylogsentinel",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "log, monitoring, regex, cron, alerting, sentinel",
"author": null,
"author_email": "Jan Tuomi <jan@jantuomi.fi>",
"download_url": "https://files.pythonhosted.org/packages/f1/cd/c0ea326245137ac15e2ed6124f2b79a09038b2f96b18b24ede72118f3822/pylogsentinel-0.2.1.tar.gz",
"platform": null,
"description": "# pylogsentinel\n\nA lightweight, zero-dependency (standard library only) log monitoring sentinel intended\nto be invoked periodically (e.g. once per minute via `cron`). It incrementally scans\nconfigured log files for user-defined regular expression rules and triggers shell\nactions with rich contextual environment variables.\n\n`pylogsentinel` is intentionally simple: no daemons, no background threads, no\nlong-running processes. Each run processes just the _new_ bytes appended since the\nprevious run (tracked per-file, per-inode) and then exits.\n\n## Key Features\n\n- INI-style configuration (`pylogsentinel.conf`).\n- Two log discovery modes: static paths or shell command\n- Per-rule regular expressions with optional flags (`/pattern/i` style).\n- Per-rule or actions executed as shell commands with rich contextual environment variables.\n- Reliable state tracking via inode-based files (`<inode>` in `state_dir`).\n- Safe handling of truncation & rotation (offset reset if file shrinks).\n- Strict lock file (`LOCK`) prevents concurrent overlapping runs.\n- Skip-actions mode (`--skip-actions`) processes new data but skips executing actions (use on the first run to avoid firing on historical log lines).\n- Human-friendly size option for `max_block_size` (e.g. `512K`, `10M`).\n- Automatic configuration file discovery when -c/--config is not supplied.\n- Directory path monitoring uses the system 'file' command; only paths whose description contains 'text' are monitored (explicit file paths are always processed).\n- Python 3.9+ (only standard library imports).\n\n## Installation\n\n```sh\npip install pylogsentinel\n```\n\n## Quick Start\n\n1. Write a configuration in a standard location:\n ```\n touch /etc/pylogsentinel.conf\n chmod 600 /etc/pylogsentinel.conf\n ```\n2. Edit paths, rules, and actions to your needs.\n3. First build state without executing actions (initial run with --skip-actions):\n\n ```\n python -m pylogsentinel -c /etc/pylogsentinel.conf --skip-actions\n ```\n\n If you omit the `-c` option (or the specified file does not exist), pylogsentinel will look for a configuration file in this order:\n - `~/.pylogsentinel.conf`\n - `~/.config/pylogsentinel.conf`\n - `/etc/pylogsentinel.conf`\n - `/usr/local/etc/pylogsentinel.conf`.\n\n4. Add a cron entry (run every minute):\n\n ```\n * * * * * /usr/bin/env python3 -m pylogsentinel -c /etc/pylogsentinel.conf\n ```\n\n## Configuration File Reference\n\nThe configuration is an INI-style file. An example:\n\n```\n[system]\nstate_dir = /var/run/pylogsentinel\nmax_block_size = 10M\n\n[logs]\n# Choose exactly one, `paths` or `cmd`:\npaths = /var/log /custom/app/logs/app.log\ncmd = find /var/log -type f -name '*.log'\n\n[action.default]\ncmd = echo -e \"Matched $RULE_ID in $FILE at line $LINE, context:\\n\\n$CONTEXT\" | mail -s \"Sentinel alert\" root\n\n[action.another]\ncmd = echo \"Another action\"\n\n[rule.error]\ndescription = Error-like conditions\npattern = /(error|fatal|exception|kill|crash)/i\naction = another # if omitted, use default action\n```\n\n### `[system]` Section\n\n| Option | Required | Description | Default |\n| ---------------- | -------- | ------------------------------------------------------------------------------------------------------- | ------- |\n| `state_dir` | Yes | Directory storing lock + per-inode state files. Must be writable. (Defined under `[system]`.) | (none) |\n| `max_block_size` | No | Maximum newly appended bytes to read per file per run. Supports suffixes `K`, `M`, `G` (in `[system]`). | `10M` |\n\n### `[logs]` Section\n\nSupply **either**:\n\n- `paths`: Space-separated list of file and/or directory paths. Directories are traversed recursively; only files whose `file -b` output contains the substring \"text\" are monitored (others are skipped). Explicit file paths are always processed.\n- `cmd`: A shell command producing **one path per line** on stdout.\n\nExactly one must be present. If using `cmd`, ensure it returns relatively quickly (recommended under a second) as it executes on every run.\n\n### `[rule.<rule_id>]` Sections\n\n| Field | Required | Description |\n| ------------- | -------- | -------------------------------------------------------------------------------------------------------------- |\n| `pattern` | Yes | Regular expression in the mandatory form `/pattern/flags` (flags optional). Supported flags: `i` (IGNORECASE). |\n| `description` | No | Human-readable description for environment variable `RULE_DESCRIPTION`. |\n| `action` | No | The `action_id` to invoke. Defaults to `default` if omitted. |\n\nAt least one rule is required.\n\n#### Pattern Forms Examples\n\n| Pattern | Meaning |\n| ----------------------- | --------------------------------------------------- |\n| `/error/i` | Case-insensitive match of \"error\" |\n| `/^panic:.+/` | Match lines starting with \"panic:\" (case-sensitive) |\n| `/^panic:.+/i` | Same, case-insensitive |\n| `/\\b(?:CRIT\\|FATAL)\\b/` | Word boundary critical terms |\n\nIf you supply an invalid regex or an unsupported flag, startup will fail with a configuration error.\n\n### `[action.<action_id>]` Sections\n\nEach action defines a shell `cmd` executed when a matching rule triggers. The special action id `default` **must** exist; it is used as a fallback when a rule references it explicitly or omits `action`.\n\nEnvironment variables available to the command:\n\n| Variable | Description |\n| ------------------ | ---------------------------------------------------------------------- |\n| `RULE_ID` | The rule identifier (e.g. `error`). |\n| `RULE_PATTERN` | The normalized pattern string (e.g. `/error/i`). |\n| `RULE_DESCRIPTION` | Description text or placeholder string if not provided. |\n| `FILE` | Absolute path of the log file where the match occurred. |\n| `LINE` | 1-based absolute line number within the file at time of scan. |\n| `CONTEXT` | Concatenated lines around the match (default radius 2 before & after). |\n\nYour `cmd` can reference these with typical shell expansion, for example:\n\n```\n[action.notify]\ncmd = printf \"%s\\n%s\\n\" \"$RULE_DESCRIPTION\" \"$CONTEXT\" | mail -s \"Alert $RULE_ID: $FILE:$LINE\" ops@example.com\n```\n\n## State Tracking\n\nFor each processed file:\n\n1. The file's inode (`st_ino`) is determined.\n2. A state file named `<inode>` inside `state_dir` stores:\n ```\n <byte_offset> <line_number>\n ```\n3. On the next run only new bytes (up to `max_block_size`) beyond `<byte_offset>` are read.\n4. If the file shrinks (truncation), the stored offset is treated as invalid and scanning restarts from the beginning with line numbers reset.\n\nThis scheme tolerates typical log rotation patterns.\n\n## Lock File\n\nA lock file named `LOCK` in `state_dir` prevents overlapping runs. If it exists the process exits immediately. Remove a stale lock only after verifying no instance is active.\n\n## Dry Run\n\n`--skip-actions` processes new data but does not run the actions (recommended for the very first run to prevent alert storms from historical data).\n\n## Exit Codes\n\n| Code | Meaning |\n| ---- | -------------------- |\n| 0 | Success |\n| 2 | Configuration error |\n| 130 | Interrupted (Ctrl+C) |\n| 1 | Other runtime error |\n\n## License\n\nMIT License (see LICENSE).\n",
"bugtrack_url": null,
"license": null,
"summary": "Lightweight cron-friendly log monitoring utility with regex rules and per-rule shell actions",
"version": "0.2.1",
"project_urls": {
"Homepage": "https://github.com/jantuomi/pylogsentinel",
"Issues": "https://github.com/jantuomi/pylogsentinel/issues",
"Repository": "https://github.com/jantuomi/pylogsentinel.git"
},
"split_keywords": [
"log",
" monitoring",
" regex",
" cron",
" alerting",
" sentinel"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "02c9e93d881507a1a7ca293d8893212c66e87d43988edf596b96a7079fd647a3",
"md5": "a2a8b025bbab4ef6bab9b0a3d72d1407",
"sha256": "8eba954b3872dd5e04375bfe6d09cd3b3313ba0e881234a6fcd32dd869cd60c4"
},
"downloads": -1,
"filename": "pylogsentinel-0.2.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "a2a8b025bbab4ef6bab9b0a3d72d1407",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 13308,
"upload_time": "2025-10-12T18:35:41",
"upload_time_iso_8601": "2025-10-12T18:35:41.490108Z",
"url": "https://files.pythonhosted.org/packages/02/c9/e93d881507a1a7ca293d8893212c66e87d43988edf596b96a7079fd647a3/pylogsentinel-0.2.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "f1cdc0ea326245137ac15e2ed6124f2b79a09038b2f96b18b24ede72118f3822",
"md5": "4563fcf03e1a4ea5986dc104f826630a",
"sha256": "2d67f058e74bd74caf621fd86abb53352c58219c333989044d7c48740eedc432"
},
"downloads": -1,
"filename": "pylogsentinel-0.2.1.tar.gz",
"has_sig": false,
"md5_digest": "4563fcf03e1a4ea5986dc104f826630a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 15051,
"upload_time": "2025-10-12T18:35:42",
"upload_time_iso_8601": "2025-10-12T18:35:42.590655Z",
"url": "https://files.pythonhosted.org/packages/f1/cd/c0ea326245137ac15e2ed6124f2b79a09038b2f96b18b24ede72118f3822/pylogsentinel-0.2.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-12 18:35:42",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "jantuomi",
"github_project": "pylogsentinel",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "pylogsentinel"
}
