# pipen-verbose
Add verbosal information in logs for [pipen][1].
## Additional information
- Following process properties if not `None` and different from pipeline-level configurations: `scheduler`, `lang`, `forks`, `cache`, `dirsig`, `size`, `template`
- Ellapsed time for a process. Note that this is time ellapsed from process initialization to completion, no matter the jobs are cached or not, so this is not the real running time for the jobs.
- Process `envs` if set.
- Computed input data for processes.
- The indices of failed jobs if any.
- The stderr, paths to script, stdout file, stderr file, of the first failed jobs if any.
- The input/output data of the first job.
## Installation
```
pip install -U pipen-verbose
```
## Enabling/Disabling the plugin
The plugin is registered via entrypoints. It's by default enabled. To disable it:
`plugins=[..., "no:verbose"]`, or uninstall this plugin.
## Usage
`example.py`
```python
from pipen import Proc, Pipen
class Process(Proc):
input = 'a'
input_data = range(10)
output = 'b:file:a.txt'
cache = False
script = 'echo {{in.a}} > {{out.b}}'
Pipen().run(Process)
```
```
> python example.py
[09/12/21 22:57:01] I main _____________________________________ __
[09/12/21 22:57:01] I main ___ __ \___ _/__ __ \__ ____/__ | / /
[09/12/21 22:57:01] I main __ /_/ /__ / __ /_/ /_ __/ __ |/ /
[09/12/21 22:57:01] I main _ ____/__/ / _ ____/_ /___ _ /| /
[09/12/21 22:57:01] I main /_/ /___/ /_/ /_____/ /_/ |_/
[09/12/21 22:57:01] I main
[09/12/21 22:57:01] I main version: 0.1.0
[09/12/21 22:57:01] I main
[09/12/21 22:57:01] I main ╭═════════════════════════════ PIPEN-0 ══════════════════════════════╮
[09/12/21 22:57:01] I main ║ # procs = 1 ║
[09/12/21 22:57:01] I main ║ plugins = ['main', 'verbose-0.0.1'] ║
[09/12/21 22:57:01] I main ║ profile = default ║
[09/12/21 22:57:01] I main ║ outdir = ./Pipen-output ║
[09/12/21 22:57:01] I main ║ cache = True ║
[09/12/21 22:57:01] I main ║ dirsig = 1 ║
[09/12/21 22:57:01] I main ║ error_strategy = ignore ║
[09/12/21 22:57:01] I main ║ forks = 1 ║
[09/12/21 22:57:01] I main ║ lang = bash ║
[09/12/21 22:57:01] I main ║ loglevel = info ║
[09/12/21 22:57:01] I main ║ num_retries = 3 ║
[09/12/21 22:57:01] I main ║ plugin_opts = {} ║
[09/12/21 22:57:01] I main ║ plugins = None ║
[09/12/21 22:57:01] I main ║ scheduler = local ║
[09/12/21 22:57:01] I main ║ scheduler_opts = {} ║
[09/12/21 22:57:01] I main ║ submission_batch = 8 ║
[09/12/21 22:57:01] I main ║ template = liquid ║
[09/12/21 22:57:01] I main ║ template_opts = {} ║
[09/12/21 22:57:01] I main ║ workdir = ./.pipen ║
[09/12/21 22:57:01] I main ╰════════════════════════════════════════════════════════════════════╯
[09/12/21 22:57:02] I main
[09/12/21 22:57:02] I main ╭═════════════════════════════ Process ══════════════════════════════╮
[09/12/21 22:57:02] I main ║ Undescribed ║
[09/12/21 22:57:02] I main ╰════════════════════════════════════════════════════════════════════╯
[09/12/21 22:57:02] I main Process: Workdir: '.pipen/pipen-0/process'
[09/12/21 22:57:02] I main Process: <<< [START]
[09/12/21 22:57:02] I main Process: >>> [END]
[09/12/21 22:57:02] I verbose Process: cache: False
[09/12/21 22:57:02] I verbose Process: size : 10
[09/12/21 22:57:02] I verbose Process: [0/9] in.a: 0
[09/12/21 22:57:02] I verbose Process: [0/9] out.b:
/home/pwwang/github/pipen-verbose/Pipen-output/Process/0/a.txt
[09/12/21 22:57:04] I verbose Process: Time elapsed: 00:00:02.043s
[09/12/21 22:57:04] I main
```
[1]: https://github.com/pwwang/pipen
Raw data
{
"_id": null,
"home_page": "https://github.com/pwwang/pipen-verbose",
"name": "pipen-verbose",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.9",
"maintainer_email": null,
"keywords": null,
"author": "pwwang",
"author_email": "pwwang@pwwang.com",
"download_url": "https://files.pythonhosted.org/packages/09/9e/31f0927cfd802ed2e347c943814b101a23eb6dec1efcbe75a9f063c84cc7/pipen_verbose-0.12.1.tar.gz",
"platform": null,
"description": "# pipen-verbose\n\nAdd verbosal information in logs for [pipen][1].\n\n## Additional information\n\n- Following process properties if not `None` and different from pipeline-level configurations: `scheduler`, `lang`, `forks`, `cache`, `dirsig`, `size`, `template`\n- Ellapsed time for a process. Note that this is time ellapsed from process initialization to completion, no matter the jobs are cached or not, so this is not the real running time for the jobs.\n- Process `envs` if set.\n- Computed input data for processes.\n- The indices of failed jobs if any.\n- The stderr, paths to script, stdout file, stderr file, of the first failed jobs if any.\n- The input/output data of the first job.\n\n## Installation\n\n```\npip install -U pipen-verbose\n```\n\n## Enabling/Disabling the plugin\n\nThe plugin is registered via entrypoints. It's by default enabled. To disable it:\n`plugins=[..., \"no:verbose\"]`, or uninstall this plugin.\n\n## Usage\n\n`example.py`\n```python\nfrom pipen import Proc, Pipen\n\nclass Process(Proc):\n input = 'a'\n input_data = range(10)\n output = 'b:file:a.txt'\n cache = False\n script = 'echo {{in.a}} > {{out.b}}'\n\nPipen().run(Process)\n```\n\n```\n> python example.py\n[09/12/21 22:57:01] I main _____________________________________ __\n[09/12/21 22:57:01] I main ___ __ \\___ _/__ __ \\__ ____/__ | / /\n[09/12/21 22:57:01] I main __ /_/ /__ / __ /_/ /_ __/ __ |/ /\n[09/12/21 22:57:01] I main _ ____/__/ / _ ____/_ /___ _ /| /\n[09/12/21 22:57:01] I main /_/ /___/ /_/ /_____/ /_/ |_/\n[09/12/21 22:57:01] I main\n[09/12/21 22:57:01] I main version: 0.1.0\n[09/12/21 22:57:01] I main\n[09/12/21 22:57:01] I main \u256d\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550 PIPEN-0 \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u256e\n[09/12/21 22:57:01] I main \u2551 # procs = 1 \u2551\n[09/12/21 22:57:01] I main \u2551 plugins = ['main', 'verbose-0.0.1'] \u2551\n[09/12/21 22:57:01] I main \u2551 profile = default \u2551\n[09/12/21 22:57:01] I main \u2551 outdir = ./Pipen-output \u2551\n[09/12/21 22:57:01] I main \u2551 cache = True \u2551\n[09/12/21 22:57:01] I main \u2551 dirsig = 1 \u2551\n[09/12/21 22:57:01] I main \u2551 error_strategy = ignore \u2551\n[09/12/21 22:57:01] I main \u2551 forks = 1 \u2551\n[09/12/21 22:57:01] I main \u2551 lang = bash \u2551\n[09/12/21 22:57:01] I main \u2551 loglevel = info \u2551\n[09/12/21 22:57:01] I main \u2551 num_retries = 3 \u2551\n[09/12/21 22:57:01] I main \u2551 plugin_opts = {} \u2551\n[09/12/21 22:57:01] I main \u2551 plugins = None \u2551\n[09/12/21 22:57:01] I main \u2551 scheduler = local \u2551\n[09/12/21 22:57:01] I main \u2551 scheduler_opts = {} \u2551\n[09/12/21 22:57:01] I main \u2551 submission_batch = 8 \u2551\n[09/12/21 22:57:01] I main \u2551 template = liquid \u2551\n[09/12/21 22:57:01] I main \u2551 template_opts = {} \u2551\n[09/12/21 22:57:01] I main \u2551 workdir = ./.pipen \u2551\n[09/12/21 22:57:01] I main \u2570\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u256f\n[09/12/21 22:57:02] I main\n[09/12/21 22:57:02] I main \u256d\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550 Process \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u256e\n[09/12/21 22:57:02] I main \u2551 Undescribed \u2551\n[09/12/21 22:57:02] I main \u2570\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u256f\n[09/12/21 22:57:02] I main Process: Workdir: '.pipen/pipen-0/process'\n[09/12/21 22:57:02] I main Process: <<< [START]\n[09/12/21 22:57:02] I main Process: >>> [END]\n[09/12/21 22:57:02] I verbose Process: cache: False\n[09/12/21 22:57:02] I verbose Process: size : 10\n[09/12/21 22:57:02] I verbose Process: [0/9] in.a: 0\n[09/12/21 22:57:02] I verbose Process: [0/9] out.b:\n /home/pwwang/github/pipen-verbose/Pipen-output/Process/0/a.txt\n[09/12/21 22:57:04] I verbose Process: Time elapsed: 00:00:02.043s\n[09/12/21 22:57:04] I main\n```\n\n[1]: https://github.com/pwwang/pipen\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Add verbosal information in logs for pipen.",
"version": "0.12.1",
"project_urls": {
"Homepage": "https://github.com/pwwang/pipen-verbose",
"Repository": "https://github.com/pwwang/pipen-verbose"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "66a06953ff406993bd9537205a4bae9d0f4d939ac9cd70b6eada01ca87b374ec",
"md5": "9661eae3c181d8af29f74668ab450971",
"sha256": "dec16ce629e3824da641d6213d9023f5235de91b796f8d6963e5eb9756be8b51"
},
"downloads": -1,
"filename": "pipen_verbose-0.12.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9661eae3c181d8af29f74668ab450971",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.9",
"size": 4603,
"upload_time": "2024-09-28T23:48:15",
"upload_time_iso_8601": "2024-09-28T23:48:15.507088Z",
"url": "https://files.pythonhosted.org/packages/66/a0/6953ff406993bd9537205a4bae9d0f4d939ac9cd70b6eada01ca87b374ec/pipen_verbose-0.12.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "099e31f0927cfd802ed2e347c943814b101a23eb6dec1efcbe75a9f063c84cc7",
"md5": "4a8aa009fe0a995dfaa63e829b1f85b7",
"sha256": "8533142e4fce10fb7395a8aa6d20116ece182e40b46f65ebc74f7415e5d0f5ff"
},
"downloads": -1,
"filename": "pipen_verbose-0.12.1.tar.gz",
"has_sig": false,
"md5_digest": "4a8aa009fe0a995dfaa63e829b1f85b7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.9",
"size": 4983,
"upload_time": "2024-09-28T23:48:16",
"upload_time_iso_8601": "2024-09-28T23:48:16.769584Z",
"url": "https://files.pythonhosted.org/packages/09/9e/31f0927cfd802ed2e347c943814b101a23eb6dec1efcbe75a9f063c84cc7/pipen_verbose-0.12.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-28 23:48:16",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "pwwang",
"github_project": "pipen-verbose",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "pipen-verbose"
}
JSON
