| Name | pypyp JSON |
| Version |
1.2.0
JSON |
| download |
| home_page | |
| Summary | Easily run Python at the shell! Magical, but never mysterious. |
| upload_time | 2024-03-03 06:01:04 |
| maintainer | |
| docs_url | None |
| author | Shantanu Jain |
| requires_python | >=3.8 |
| license | |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# pyp
[](https://github.com/hauntsaninja/pyp/actions/workflows/tests.yml)
[](https://coveralls.io/github/hauntsaninja/pyp?branch=master)
[](http://mypy-lang.org/)
Easily run Python at the shell! Magical, but never mysterious.
## Installation
Run `pip install pypyp` <sup>(note the extra "yp"!)</sup>
pyp requires Python 3.8 or above.
## How it works
pyp will statically analyse the input code to detect undefined variables. Based on what it finds,
it will proceed to transform the AST of the input code as needed. We then compile and execute the
result, or if using `--explain`, unparse the AST back to source code.
## Examples
This section will walk you through the details of using pyp, and hopefully replace your needs
for many common shell utilities. For a cheatsheet / tldr, run `pyp --help`.
#### pyp can easily be used to apply Python code to each line in the input.
Just use one of the magic variables `x`, `l` or `line` to refer to the current line.
```sh
# pyp like cut
ls | pyp 'x[:3]'
ps x | pyp 'line.split()[4]'
```
#### pyp can be used to easily apply Python code to the entire input as well.
Use the magic variable `lines` for a list of rstripped lines or `stdin` for `sys.stdin`.
```sh
# pyp like wc -c
cat /usr/share/dict/words | pyp 'len(stdin.read())'
# pyp like awk
seq 1 5 | pyp 'sum(map(int, lines))'
```
#### pyp will automatically import modules you use.
```sh
# pyp like sh
echo echo echo | pyp 'subprocess.run(lines[0], shell=True); pass'
# pyp like jq
curl -s 'https://api.github.com/repos/hauntsaninja/pyp/commits?per_page=1' | pyp 'json.load(stdin)[0]["commit"]["author"]'
# pyp like egrep
cat /usr/share/dict/words | pyp 'x if re.search("(p|m)yth", x) else None'
```
For `collections`, `math`, `itertools`, `pathlib.Path`, `pprint.pp`, pyp will figure it out even
if you don't use the qualified name.
```sh
# pyp like bc
pyp 'sqrt(5)'
# pyp like ${x##*.}
ls | pyp 'Path(x).suffix'
```
#### pyp can give you access to loop indices using the magic variables `i`, `idx` or `index`.
```sh
# pyp like line numbers
cat setup.py | pyp 'f"{idx+1: >3} {x}"'
```
#### Note so far you haven't had to call `print`!
By default, pyp will print the last expression in your code — except if it evaluates to `None`
(or the last statement is `pass`).
And you can always explicitly call `print` yourself, in which case pyp will stay out of your way.
```sh
# pyp like grep
cat /usr/share/dict/words | pyp 'x if "python" in x else None'
cat /usr/share/dict/words | pyp 'if "python" in x: print(x); "this will not be printed"'
```
#### pyp will attempt to intelligently print dicts and iterables.
This makes the output of pyp easier to compose with shell tools.
Again, explicit printing will stop this magic, but pyp makes the function `pypprint` available if
you do want to explicitly opt back in.
```sh
# pyp like tail
ls | pyp 'lines[-10:]'
# pyp like sort
ls | pyp 'sorted(lines)'
ls | pyp 'print(f"Sorting {len(lines)} lines"); pypprint(sorted(lines))'
# pyp like sort | uniq
ls | pyp 'sorted(set(lines))'
```
#### pyp lets you run snippets of Python before and after processing input.
Note if you run into trouble with semicolons and want a new line (without using a multiline string
in your shell), you can just pass another string to pyp. You can also always pipe pyp to pyp!
```sh
# pyp like anything!
ps aux | pyp -b 'd = defaultdict(list)' 'user, pid, *_ = x.split()' 'd[user].append(pid)' -a 'del d["root"]' -a 'd'
```
#### pyp can be magical, but it doesn't have to be mysterious!
Use `--explain` or `--script` and pyp will output a script equivalent to what it would run. This can also serve as a
useful starting point for more complex scripts.
```sh
pyp --explain -b 'd = defaultdict(list)' 'user, pid, *_ = x.split()' 'd[user].append(pid)' -a 'del d["root"]' -a 'd'
```
```py
#!/usr/bin/env python3
from collections import defaultdict
from pyp import pypprint
import sys
d = defaultdict(list)
for x in sys.stdin:
x = x.rstrip('\n')
(user, pid, *_) = x.split()
d[user].append(pid)
del d['root']
if d is not None:
pypprint(d)
```
And if your command hits an exception, pyp will reconstruct a traceback into the generated code.
#### pyp is configurable.
Point the environment variable `PYP_CONFIG_PATH` to a file containing, for example:
```py
import numpy as np
import tensorflow as tf
from pipetools import *
def p95(data):
return np.percentile(data, 95)
class PotentiallyUsefulClass: ...
```
When attempting to define undefined names, pyp will statically\* analyse this file as a source of
possible definitions. This means that if you don't use `tf`, we won't import `tensorflow`! And of
course, `--explain` will show you exactly what gets run (and hence what doesn't!):
```sh
pyp --explain 'print(p95(list(map(float, stdin))))'
```
```py
#!/usr/bin/env python3
import sys
import numpy as np
def p95(data):
return np.percentile(data, 95)
stdin = sys.stdin
print(p95(list(map(float, stdin))))
```
Note, importing things from libraries like [pipetools](https://0101.github.io/pipetools/doc/index.html)
in your configuration can allow you to achieve high levels of syntax sugar:
```sh
seq 1 110 | pyp 'lines > foreach(int) | where(X > 100) | group_by(X % 3) | sort_by(X[0])'
```
<sub>\*If you use wildcard imports, we will need to import those modules if there remain undefined
names, though we skip this in the happy path. If this matters to you, definitely don't
`from tensorflow import *` in your config! </sub>
#### pyp lets you configure your own magic!
If definitions in your config file depend on magic variables, pyp will substitute them in the
way that makes sense. For example, put the following in your config...
```py
n = int(x)
f = x.split()
j = json.load(stdin)
import pandas as pd
csv = pd.read_csv(stdin)
```
...to make pyp easier than ever for your custom use cases:
```sh
ps | pyp 'f[3]'
cat commits.json | pyp 'j[0]["commit"]["author"]'
< cities.csv pyp 'csv.to_string()'
```
#### I have questions!
There's additional documentation and examples at [FAQ](https://github.com/hauntsaninja/pyp/blob/master/FAQ.md).
If that doesn't answer your question, please open an issue!
## Related projects
### [Pyed Piper](https://github.com/thepyedpiper/pyp) aka Python Power at the Prompt
pypyp takes inspiration (and the command name!) from here.
Pyed Piper [had been dead and Python 2 only](https://code.google.com/archive/p/pyp/) for a decade
when pypyp was written; it appears it's recently been resurrected. Pyed Piper is further away
from normal Python syntax and APIs than pypyp aims to be. In particular, Pyed Piper has an emphasis
on piping within Python similar to the way you can combine `pypyp` with `pipetools` in the
configuration example above.
### [Pyped](https://github.com/ksamuel/Pyped)
Pyped is very similar; enough so that I probably wouldn't have written pyp had I known
about it. But I'm glad I didn't, since Pyped doesn't do the AST introspection and manipulation
that we do. This means:
- Pyped relies on you to pass in flags to tell it what to do, when pyp can unambiguously infer
intention from the input.
- It doesn't provide easy automatic printing, or smart printing of iterables and dicts.
- It hardcodes a list of imports and installs some libraries on your system. This project's
automatic import will work for any library you use.
- It doesn't have anything like `--explain`/`--script`.
However,
- It has some conveniences, like regex splitting of input, that you'd have to do for yourself here.
- It supports Python 2 and early versions of Python 3.
- It's been around for much longer.
### [piep](http://gfxmonk.net/dist/doc/piep/) / [spy](https://github.com/edk0/spy) / [pyfil](https://github.com/ninjaaron/pyfil) / [pythonpy](https://github.com/fish2000/pythonpy-fork) / [oneliner](https://python-oneliner.readthedocs.io/en/latest/)
Since writing pyp, it turns out there are more alternatives out there than I thought :-) Some quick
notes:
- Most of them rely on the user passing in flags, like Pyped.
- Most of them have limitations around automatic printing, like only being able to automatically
print single expressions or not handling iterables and dicts well.
- Some of them have custom syntax for in-process command chaining, which can be convenient.
- Some of them have specialised support for things like JSON input or running shell commands.
- Some of them expose the input in interesting ways with custom line / file / stream objects.
- Some of them have more advanced options for error handling (although none of them have pyp's
great tracebacks).
- None of them have powerful configuration like pyp.
- None of them have anything like `--explain`.
For whatever it's worth, I've listed the projects above in approximate order of my
personal preference.
### [mario](https://github.com/python-mario/mario)
`mario` is a featureful take on shell processing with Python. It doesn't use undefined name
detection, instead relying on a pluggable subcommand system. While the subcommands can be more
verbose than pyp, `mario` makes up some ground by automatic application of functions and a custom
command chaining syntax. The result can feel a little DSL-like, while pyp tries to feel very close
to writing Python.
Consider using `mario` if:
- You find yourself stringing together long sequences of pyp commands and want to be able to
command chain within a single process out of the box.
- You find yourself often needing to reuse complex pyp commands or doing a lot of domain specific
shell processing that you wish you could reuse with a single command.
- You want to easily be able to use async functions.
Consider pyp if:
- You want to minimise keystrokes for things that should be quick and easy.
- You want something minimal and lightweight that feels very close to Python. You don't want to have
to remember commands.
- You're happy using Python libraries to do domain specific heavy lifting, for easy command chaining
or syntax sugar. You don't mind (or want to be able to) fall back to a script via `--script` to deal
with complexity.
### [xonsh](https://xon.sh/)
`xonsh` is a shell whose language is a superset of Python; this is more ambitious and pretty
different from pyp. pyp is easier to use for the one-liner piping use case, but if you need
more Python in your shell, check out `xonsh`.
### [awk](https://www.gnu.org/software/gawk/manual/gawk.html)
If `awk` works for you, how did you end up here?
Raw data
{
"_id": null,
"home_page": "",
"name": "pypyp",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "",
"author": "Shantanu Jain",
"author_email": "hauntsaninja@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/88/af/8c8a089edfc8690fdc00e404386d090aea1a16acf0adfffdc361cd699de6/pypyp-1.2.0.tar.gz",
"platform": null,
"description": "# pyp\n\n[](https://github.com/hauntsaninja/pyp/actions/workflows/tests.yml)\n[](https://coveralls.io/github/hauntsaninja/pyp?branch=master)\n[](http://mypy-lang.org/)\n\nEasily run Python at the shell! Magical, but never mysterious.\n\n## Installation\n\nRun `pip install pypyp` <sup>(note the extra \"yp\"!)</sup>\n\npyp requires Python 3.8 or above.\n\n## How it works\n\npyp will statically analyse the input code to detect undefined variables. Based on what it finds,\nit will proceed to transform the AST of the input code as needed. We then compile and execute the\nresult, or if using `--explain`, unparse the AST back to source code.\n\n## Examples\n\nThis section will walk you through the details of using pyp, and hopefully replace your needs\nfor many common shell utilities. For a cheatsheet / tldr, run `pyp --help`.\n\n#### pyp can easily be used to apply Python code to each line in the input.\nJust use one of the magic variables `x`, `l` or `line` to refer to the current line.\n\n```sh\n# pyp like cut\nls | pyp 'x[:3]'\nps x | pyp 'line.split()[4]'\n```\n\n#### pyp can be used to easily apply Python code to the entire input as well.\nUse the magic variable `lines` for a list of rstripped lines or `stdin` for `sys.stdin`.\n\n```sh\n# pyp like wc -c\ncat /usr/share/dict/words | pyp 'len(stdin.read())'\n\n# pyp like awk\nseq 1 5 | pyp 'sum(map(int, lines))'\n```\n\n#### pyp will automatically import modules you use.\n\n```sh\n# pyp like sh\necho echo echo | pyp 'subprocess.run(lines[0], shell=True); pass'\n\n# pyp like jq\ncurl -s 'https://api.github.com/repos/hauntsaninja/pyp/commits?per_page=1' | pyp 'json.load(stdin)[0][\"commit\"][\"author\"]'\n\n# pyp like egrep\ncat /usr/share/dict/words | pyp 'x if re.search(\"(p|m)yth\", x) else None'\n```\n\nFor `collections`, `math`, `itertools`, `pathlib.Path`, `pprint.pp`, pyp will figure it out even\nif you don't use the qualified name.\n```sh\n# pyp like bc\npyp 'sqrt(5)'\n\n# pyp like ${x##*.}\nls | pyp 'Path(x).suffix'\n```\n\n#### pyp can give you access to loop indices using the magic variables `i`, `idx` or `index`.\n\n```sh\n# pyp like line numbers\ncat setup.py | pyp 'f\"{idx+1: >3} {x}\"'\n```\n\n#### Note so far you haven't had to call `print`!\nBy default, pyp will print the last expression in your code \u2014 except if it evaluates to `None`\n(or the last statement is `pass`).\nAnd you can always explicitly call `print` yourself, in which case pyp will stay out of your way.\n\n```sh\n# pyp like grep\ncat /usr/share/dict/words | pyp 'x if \"python\" in x else None'\ncat /usr/share/dict/words | pyp 'if \"python\" in x: print(x); \"this will not be printed\"'\n```\n\n#### pyp will attempt to intelligently print dicts and iterables.\nThis makes the output of pyp easier to compose with shell tools.\nAgain, explicit printing will stop this magic, but pyp makes the function `pypprint` available if\nyou do want to explicitly opt back in.\n\n```sh\n# pyp like tail\nls | pyp 'lines[-10:]'\n\n# pyp like sort\nls | pyp 'sorted(lines)'\nls | pyp 'print(f\"Sorting {len(lines)} lines\"); pypprint(sorted(lines))'\n\n# pyp like sort | uniq\nls | pyp 'sorted(set(lines))'\n```\n\n#### pyp lets you run snippets of Python before and after processing input.\nNote if you run into trouble with semicolons and want a new line (without using a multiline string\nin your shell), you can just pass another string to pyp. You can also always pipe pyp to pyp!\n\n```sh\n# pyp like anything!\nps aux | pyp -b 'd = defaultdict(list)' 'user, pid, *_ = x.split()' 'd[user].append(pid)' -a 'del d[\"root\"]' -a 'd'\n```\n\n#### pyp can be magical, but it doesn't have to be mysterious!\nUse `--explain` or `--script` and pyp will output a script equivalent to what it would run. This can also serve as a\nuseful starting point for more complex scripts.\n```sh\npyp --explain -b 'd = defaultdict(list)' 'user, pid, *_ = x.split()' 'd[user].append(pid)' -a 'del d[\"root\"]' -a 'd'\n```\n```py\n#!/usr/bin/env python3\nfrom collections import defaultdict\nfrom pyp import pypprint\nimport sys\nd = defaultdict(list)\nfor x in sys.stdin:\n x = x.rstrip('\\n')\n (user, pid, *_) = x.split()\n d[user].append(pid)\ndel d['root']\nif d is not None:\n pypprint(d)\n```\nAnd if your command hits an exception, pyp will reconstruct a traceback into the generated code.\n\n#### pyp is configurable.\n\nPoint the environment variable `PYP_CONFIG_PATH` to a file containing, for example:\n```py\nimport numpy as np\nimport tensorflow as tf\nfrom pipetools import *\n\ndef p95(data):\n return np.percentile(data, 95)\n\nclass PotentiallyUsefulClass: ...\n```\n\nWhen attempting to define undefined names, pyp will statically\\* analyse this file as a source of\npossible definitions. This means that if you don't use `tf`, we won't import `tensorflow`! And of\ncourse, `--explain` will show you exactly what gets run (and hence what doesn't!):\n\n```sh\npyp --explain 'print(p95(list(map(float, stdin))))'\n```\n```py\n#!/usr/bin/env python3\nimport sys\nimport numpy as np\n\ndef p95(data):\n return np.percentile(data, 95)\nstdin = sys.stdin\nprint(p95(list(map(float, stdin))))\n```\n\nNote, importing things from libraries like [pipetools](https://0101.github.io/pipetools/doc/index.html)\nin your configuration can allow you to achieve high levels of syntax sugar:\n```sh\nseq 1 110 | pyp 'lines > foreach(int) | where(X > 100) | group_by(X % 3) | sort_by(X[0])'\n```\n\n<sub>\\*If you use wildcard imports, we will need to import those modules if there remain undefined\nnames, though we skip this in the happy path. If this matters to you, definitely don't\n`from tensorflow import *` in your config! </sub>\n\n#### pyp lets you configure your own magic!\n\nIf definitions in your config file depend on magic variables, pyp will substitute them in the\nway that makes sense. For example, put the following in your config...\n```py\nn = int(x)\nf = x.split()\nj = json.load(stdin)\n\nimport pandas as pd\ncsv = pd.read_csv(stdin)\n```\n\n...to make pyp easier than ever for your custom use cases:\n```sh\nps | pyp 'f[3]'\n\ncat commits.json | pyp 'j[0][\"commit\"][\"author\"]'\n\n< cities.csv pyp 'csv.to_string()'\n```\n\n#### I have questions!\n\nThere's additional documentation and examples at [FAQ](https://github.com/hauntsaninja/pyp/blob/master/FAQ.md).\nIf that doesn't answer your question, please open an issue!\n\n## Related projects\n\n### [Pyed Piper](https://github.com/thepyedpiper/pyp) aka Python Power at the Prompt\n\npypyp takes inspiration (and the command name!) from here.\n\nPyed Piper [had been dead and Python 2 only](https://code.google.com/archive/p/pyp/) for a decade\nwhen pypyp was written; it appears it's recently been resurrected. Pyed Piper is further away\nfrom normal Python syntax and APIs than pypyp aims to be. In particular, Pyed Piper has an emphasis\non piping within Python similar to the way you can combine `pypyp` with `pipetools` in the\nconfiguration example above.\n\n### [Pyped](https://github.com/ksamuel/Pyped)\n\nPyped is very similar; enough so that I probably wouldn't have written pyp had I known\nabout it. But I'm glad I didn't, since Pyped doesn't do the AST introspection and manipulation\nthat we do. This means:\n- Pyped relies on you to pass in flags to tell it what to do, when pyp can unambiguously infer\n intention from the input.\n- It doesn't provide easy automatic printing, or smart printing of iterables and dicts.\n- It hardcodes a list of imports and installs some libraries on your system. This project's\nautomatic import will work for any library you use.\n- It doesn't have anything like `--explain`/`--script`.\n\nHowever,\n- It has some conveniences, like regex splitting of input, that you'd have to do for yourself here.\n- It supports Python 2 and early versions of Python 3.\n- It's been around for much longer.\n\n### [piep](http://gfxmonk.net/dist/doc/piep/) / [spy](https://github.com/edk0/spy) / [pyfil](https://github.com/ninjaaron/pyfil) / [pythonpy](https://github.com/fish2000/pythonpy-fork) / [oneliner](https://python-oneliner.readthedocs.io/en/latest/)\n\nSince writing pyp, it turns out there are more alternatives out there than I thought :-) Some quick\nnotes:\n- Most of them rely on the user passing in flags, like Pyped.\n- Most of them have limitations around automatic printing, like only being able to automatically\nprint single expressions or not handling iterables and dicts well.\n- Some of them have custom syntax for in-process command chaining, which can be convenient.\n- Some of them have specialised support for things like JSON input or running shell commands.\n- Some of them expose the input in interesting ways with custom line / file / stream objects.\n- Some of them have more advanced options for error handling (although none of them have pyp's\n great tracebacks).\n- None of them have powerful configuration like pyp.\n- None of them have anything like `--explain`.\n\nFor whatever it's worth, I've listed the projects above in approximate order of my\npersonal preference.\n\n### [mario](https://github.com/python-mario/mario)\n\n`mario` is a featureful take on shell processing with Python. It doesn't use undefined name\ndetection, instead relying on a pluggable subcommand system. While the subcommands can be more\nverbose than pyp, `mario` makes up some ground by automatic application of functions and a custom\ncommand chaining syntax. The result can feel a little DSL-like, while pyp tries to feel very close\nto writing Python.\n\nConsider using `mario` if:\n- You find yourself stringing together long sequences of pyp commands and want to be able to\ncommand chain within a single process out of the box.\n- You find yourself often needing to reuse complex pyp commands or doing a lot of domain specific\nshell processing that you wish you could reuse with a single command.\n- You want to easily be able to use async functions.\n\nConsider pyp if:\n- You want to minimise keystrokes for things that should be quick and easy.\n- You want something minimal and lightweight that feels very close to Python. You don't want to have\nto remember commands.\n- You're happy using Python libraries to do domain specific heavy lifting, for easy command chaining\nor syntax sugar. You don't mind (or want to be able to) fall back to a script via `--script` to deal\nwith complexity.\n\n### [xonsh](https://xon.sh/)\n\n`xonsh` is a shell whose language is a superset of Python; this is more ambitious and pretty\ndifferent from pyp. pyp is easier to use for the one-liner piping use case, but if you need\nmore Python in your shell, check out `xonsh`.\n\n### [awk](https://www.gnu.org/software/gawk/manual/gawk.html)\n\nIf `awk` works for you, how did you end up here?\n\n",
"bugtrack_url": null,
"license": "",
"summary": "Easily run Python at the shell! Magical, but never mysterious.",
"version": "1.2.0",
"project_urls": {
"changelog": "https://github.com/hauntsaninja/pyp/blob/master/CHANGELOG.md",
"homepage": "https://github.com/hauntsaninja/pyp",
"repository": "https://github.com/hauntsaninja/pyp"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e06c6a9eac381dd1f0d2e878ad6213fca24cfe5b3846db46b26d620301ac7624",
"md5": "5d5e6e42cf292167d130cd54a6894388",
"sha256": "45e2a0fbe5b7b5bb0be7e95bcb5b23d43ddae8c561410c9c336e6773005c70b1"
},
"downloads": -1,
"filename": "pypyp-1.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "5d5e6e42cf292167d130cd54a6894388",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 15487,
"upload_time": "2024-03-03T06:01:02",
"upload_time_iso_8601": "2024-03-03T06:01:02.142042Z",
"url": "https://files.pythonhosted.org/packages/e0/6c/6a9eac381dd1f0d2e878ad6213fca24cfe5b3846db46b26d620301ac7624/pypyp-1.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "88af8c8a089edfc8690fdc00e404386d090aea1a16acf0adfffdc361cd699de6",
"md5": "b615f787c6914e425c6c244fb9bb6354",
"sha256": "d1571ed626bd65686248be7f7332dd750f413b15de1ae80ac1ddb353bd34486d"
},
"downloads": -1,
"filename": "pypyp-1.2.0.tar.gz",
"has_sig": false,
"md5_digest": "b615f787c6914e425c6c244fb9bb6354",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 18775,
"upload_time": "2024-03-03T06:01:04",
"upload_time_iso_8601": "2024-03-03T06:01:04.131187Z",
"url": "https://files.pythonhosted.org/packages/88/af/8c8a089edfc8690fdc00e404386d090aea1a16acf0adfffdc361cd699de6/pypyp-1.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-03 06:01:04",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "hauntsaninja",
"github_project": "pyp",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "pypyp"
}
