# Pyallel
Run and handle the output of multiple executables in `pyallel` (as in parallel)
https://github.com/Danthewaann/pyallel/assets/22531177/8685eb92-aac5-440a-9170-30fd1460c53f
Tested on Linux and MacOS only
# Installation
Pre-built executables are available on the [Releases](https://github.com/Danthewaann/pyallel/releases) page.
`pyallel` can also be installed using pip (requires Python >=3.8):
```bash
pip install pyallel
```
# Quick start
Once installed, you can run `pyallel` to see usage information, like so:
```
usage: pyallel [-h] [-t] [-n] [-V] [--colour {yes,no,auto}] [commands ...]
Run and handle the output of multiple executables in pyallel (as in parallel)
positional arguments:
commands list of quoted commands to run in parallel e.g "mypy ." "black ."
each command is executed inside a shell, so shell syntax is supported as
if you were running the command directly in a shell, some examples are below
"MYPY_FORCE_COLOR=1 mypy ." <- provide environment variables
"mypy | tee -a mypy.log" <- use pipes to redirect output
"cat > test.log < other.log" <- use input and output redirection
"mypy .; pytest ." <- run commands one at a time in sequence
"echo \$SHELL" or "\$(echo mypy .)" <- expand variables and commands to evaluate (must be escaped)
"pytest . && mypy . || echo failed!" <- use AND (&&) and OR (||) to run commands conditionally
PROCESS GROUPS
--------------
commands can be grouped using the group separator symbol (:::)
%(prog)s "echo boil kettle" "sleep 1" ::: "echo make coffee"
the above will print "boil kettle" and sleep for 1 second first before printing "make coffee"
command groups are ran in the sequence you provide them, and if a command group fails
(if a command fails inside the command group) the rest of the command groups in the sequence are not run
COMMAND MODIFIERS
-----------------
modifiers can be set for commands to augment their behaviour using the command modifier symbol (::)
lines (only used in interactive mode):
the lines modifier allows you to specify how many lines the command output can take up on the screen
%(prog)s "lines=90 :: echo running long command..." "echo running other command..."
90 is expressed as a percentage value, which must be between 1 and 100 inclusive
options:
-h, --help show this help message and exit
-t, --no-timer don't time how long each command is taking
-n, --non-interactive
run in non-interactive mode
-V, --version print version and exit
--colour {yes,no,auto}
colour terminal output, defaults to "auto"
```
Currently you can provide a variable number of `commands` to run to `pyallel`, like so:
> [!IMPORTANT]
> If you need to provide arguments to a command, you must surround the command and it's arguments in quotes!
```bash
pyallel "MYPY_FORCE_COLOR=1 mypy ." \
"black --check --diff ." \
"pytest ."
```
# Build
You can also build an executable with the following (executables will be written to `./dist`):
> [!NOTE]
> The `arch=x86_64` values in the following code blocks can be replaced with `arch=aarch64` and
> any other architecture that is supported by docker to build an executable for that given architecture
> [!NOTE]
> To build aarch64 binaries on an x86_64 host machine, you will need to run the following
> commands to setup qemu to allow this to work
```bash
sudo apt-get install qemu binfmt-support qemu-user-static && \
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
```
#### Build for generic linux
```bash
docker build --tag pyallel --build-arg 'arch=x86_64' --build-arg "uid=$(id -u)" . && \
docker run -e 'arch=x86_64' --rm --volume "$(pwd):/src" pyallel
```
#### Build for alpine linux
```bash
docker build --tag pyallel-alpine --build-arg 'arch=x86_64' --build-arg "uid=$(id -u)" --file Dockerfile.alpine . && \
docker run -e 'arch=x86_64' --rm --volume "$(pwd):/src" pyallel-alpine
```
#### Build locally
```bash
python -m venv .venv && \
source .venv/bin/activate && \
pip install . -r requirements_build.txt && \
./build.sh
```
#### Build all
```bash
./build_all.sh
```
## TODOs
- [x] Add support to have commands depend on other commands (some commands must complete
before a given command can start)
- [x] Add support to state how many lines a command can use for it's output in interactive mode
- [ ] Add a debug mode that logs debug information to a log file
- [ ] Maybe add support to allow the user to provide stdin for commands that request it
(such as a REPL)
- [ ] Add custom parsing of command output to support filtering for errors (like vim's
`errorformat`)
- [ ] Allow list of files to be provided to supply as input arguments to each command
- [ ] Allow input to be piped into `pyallel` via stdin to supply as standard input to each
command
Raw data
{
"_id": null,
"home_page": "https://github.com/Danthewaann/pyallel",
"name": "pyallel",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.14,>=3.8",
"maintainer_email": null,
"keywords": "parallel, command, runner, executable, shell, terminal",
"author": "Daniel Black",
"author_email": "danielcrblack@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/d4/b9/09b2bb9d35b2cca0153d551783b08407db1fb2022404a2930a92967ed44d/pyallel-1.3.1.tar.gz",
"platform": null,
"description": "# Pyallel\n\nRun and handle the output of multiple executables in `pyallel` (as in parallel)\n\nhttps://github.com/Danthewaann/pyallel/assets/22531177/8685eb92-aac5-440a-9170-30fd1460c53f\n\nTested on Linux and MacOS only\n\n# Installation\n\nPre-built executables are available on the [Releases](https://github.com/Danthewaann/pyallel/releases) page.\n\n`pyallel` can also be installed using pip (requires Python >=3.8):\n\n```bash\npip install pyallel\n```\n\n# Quick start\n\nOnce installed, you can run `pyallel` to see usage information, like so:\n\n```\nusage: pyallel [-h] [-t] [-n] [-V] [--colour {yes,no,auto}] [commands ...]\n\nRun and handle the output of multiple executables in pyallel (as in parallel)\n\npositional arguments:\n commands list of quoted commands to run in parallel e.g \"mypy .\" \"black .\"\n\n each command is executed inside a shell, so shell syntax is supported as\n if you were running the command directly in a shell, some examples are below\n\n \"MYPY_FORCE_COLOR=1 mypy .\" <- provide environment variables\n \"mypy | tee -a mypy.log\" <- use pipes to redirect output\n \"cat > test.log < other.log\" <- use input and output redirection\n \"mypy .; pytest .\" <- run commands one at a time in sequence\n \"echo \\$SHELL\" or \"\\$(echo mypy .)\" <- expand variables and commands to evaluate (must be escaped)\n \"pytest . && mypy . || echo failed!\" <- use AND (&&) and OR (||) to run commands conditionally\n\n PROCESS GROUPS\n --------------\n commands can be grouped using the group separator symbol (:::)\n\n %(prog)s \"echo boil kettle\" \"sleep 1\" ::: \"echo make coffee\"\n\n the above will print \"boil kettle\" and sleep for 1 second first before printing \"make coffee\"\n\n command groups are ran in the sequence you provide them, and if a command group fails\n (if a command fails inside the command group) the rest of the command groups in the sequence are not run\n\n COMMAND MODIFIERS\n -----------------\n modifiers can be set for commands to augment their behaviour using the command modifier symbol (::)\n\n lines (only used in interactive mode):\n the lines modifier allows you to specify how many lines the command output can take up on the screen\n \n %(prog)s \"lines=90 :: echo running long command...\" \"echo running other command...\"\n\n 90 is expressed as a percentage value, which must be between 1 and 100 inclusive\n\noptions:\n -h, --help show this help message and exit\n -t, --no-timer don't time how long each command is taking\n -n, --non-interactive\n run in non-interactive mode\n -V, --version print version and exit\n --colour {yes,no,auto}\n colour terminal output, defaults to \"auto\"\n```\n\nCurrently you can provide a variable number of `commands` to run to `pyallel`, like so:\n\n> [!IMPORTANT]\n> If you need to provide arguments to a command, you must surround the command and it's arguments in quotes!\n\n```bash\npyallel \"MYPY_FORCE_COLOR=1 mypy .\" \\\n \"black --check --diff .\" \\\n \"pytest .\"\n```\n\n# Build\n\nYou can also build an executable with the following (executables will be written to `./dist`):\n\n> [!NOTE]\n> The `arch=x86_64` values in the following code blocks can be replaced with `arch=aarch64` and\n> any other architecture that is supported by docker to build an executable for that given architecture\n\n> [!NOTE]\n> To build aarch64 binaries on an x86_64 host machine, you will need to run the following\n> commands to setup qemu to allow this to work\n\n```bash\nsudo apt-get install qemu binfmt-support qemu-user-static && \\\ndocker run --rm --privileged multiarch/qemu-user-static --reset -p yes\n```\n\n#### Build for generic linux\n\n```bash\ndocker build --tag pyallel --build-arg 'arch=x86_64' --build-arg \"uid=$(id -u)\" . && \\\n docker run -e 'arch=x86_64' --rm --volume \"$(pwd):/src\" pyallel\n```\n\n#### Build for alpine linux\n\n```bash\ndocker build --tag pyallel-alpine --build-arg 'arch=x86_64' --build-arg \"uid=$(id -u)\" --file Dockerfile.alpine . && \\\n docker run -e 'arch=x86_64' --rm --volume \"$(pwd):/src\" pyallel-alpine\n```\n\n#### Build locally\n\n```bash\npython -m venv .venv && \\\n source .venv/bin/activate && \\\n pip install . -r requirements_build.txt && \\\n ./build.sh\n```\n\n#### Build all\n\n```bash\n./build_all.sh\n```\n\n## TODOs\n\n- [x] Add support to have commands depend on other commands (some commands must complete\n before a given command can start)\n- [x] Add support to state how many lines a command can use for it's output in interactive mode\n- [ ] Add a debug mode that logs debug information to a log file\n- [ ] Maybe add support to allow the user to provide stdin for commands that request it\n (such as a REPL)\n- [ ] Add custom parsing of command output to support filtering for errors (like vim's\n `errorformat`)\n- [ ] Allow list of files to be provided to supply as input arguments to each command\n- [ ] Allow input to be piped into `pyallel` via stdin to supply as standard input to each\n command\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Run and handle the output of multiple executables in pyallel (as in parallel)",
"version": "1.3.1",
"project_urls": {
"Homepage": "https://github.com/Danthewaann/pyallel",
"Repository": "https://github.com/Danthewaann/pyallel"
},
"split_keywords": [
"parallel",
" command",
" runner",
" executable",
" shell",
" terminal"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5f599d2f9cf7f36ec2876e0198394c6c72e3fda37a51adaaffb46e518b462b08",
"md5": "ee3a1e8b89961fecc5c2c2a3703c1e28",
"sha256": "6c0d7c1379d76978f2014d1c70d330d17d77fe8ffb9193ba405028568f31a359"
},
"downloads": -1,
"filename": "pyallel-1.3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ee3a1e8b89961fecc5c2c2a3703c1e28",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.14,>=3.8",
"size": 14252,
"upload_time": "2024-12-21T19:54:05",
"upload_time_iso_8601": "2024-12-21T19:54:05.977204Z",
"url": "https://files.pythonhosted.org/packages/5f/59/9d2f9cf7f36ec2876e0198394c6c72e3fda37a51adaaffb46e518b462b08/pyallel-1.3.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d4b909b2bb9d35b2cca0153d551783b08407db1fb2022404a2930a92967ed44d",
"md5": "ac63494114a8387044c02e8fffb2c5de",
"sha256": "bf057778dc06838ee5876666e591246077cb04702e0caf9a53fc9c0b4c324956"
},
"downloads": -1,
"filename": "pyallel-1.3.1.tar.gz",
"has_sig": false,
"md5_digest": "ac63494114a8387044c02e8fffb2c5de",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.14,>=3.8",
"size": 12719,
"upload_time": "2024-12-21T19:54:07",
"upload_time_iso_8601": "2024-12-21T19:54:07.199792Z",
"url": "https://files.pythonhosted.org/packages/d4/b9/09b2bb9d35b2cca0153d551783b08407db1fb2022404a2930a92967ed44d/pyallel-1.3.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-21 19:54:07",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Danthewaann",
"github_project": "pyallel",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "pyallel"
}
JSON
