# shell-functools
[](https://github.com/sharkdp/shell-functools/actions/workflows/ci.yml)
*A collection of functional programming tools for the shell.*
This project provides higher order functions like `map`, `filter`, `foldl`, `sort_by` and `take_while` as simple command-line tools.
Following the UNIX philosophy, these commands are designed to be composed via pipes. A
[large collection](#available-function-arguments) of functions such as `basename`, `replace`, `contains` or `is_dir` are provided as
arguments to these commands.
## Contents
* [Demo](#demo)
* [Quick start](#quick-start)
* [Documentation and examples](#documentation-and-examples)
* [Usage of `map`](#usage-of-map)
* [Usage of `filter`](#usage-of-filter)
* [Usage of `foldl`](#usage-of-foldl)
* [Usage of `foldl1`](#usage-of-foldl1)
* [Usage of `sort_by`](#usage-of-sort_by)
* [Chaining commands](#chaining-commands)
* [Lazy evaluation](#lazy-evaluation)
* [Working with columns](#working-with-columns)
* [Available function arguments](#available-function-arguments)
## Demo
<a href="https://asciinema.org/a/6zsp3hEPpM7tmWHrjThl7idqh" target="_blank"><img src="https://asciinema.org/a/6zsp3hEPpM7tmWHrjThl7idqh.png" width="600" /></a>
## Quick start
If you want to try it out on your own, run:
``` bash
pip install shell-functools
```
## Documentation and examples
### Usage of `map`
The `map` command takes a [function argument](#available-function-arguments) and applies it to every line of input:
``` bash
> ls
document.txt
folder
image.jpg
> ls | map abspath
/tmp/demo/document.txt
/tmp/demo/folder
/tmp/demo/image.jpg
```
### Usage of `filter`
The `filter` command takes a [function argument](#available-function-arguments) with a `Bool`ean return type. It applies that function to each input line and shows only those that returned `true`:
``` bash
> find
.
./folder
./folder/me.jpg
./folder/subdirectory
./folder/subdirectory/song.mp3
./document.txt
./image.jpg
> find | filter is_file
./folder/me.jpg
./folder/subdirectory/song.mp3
./document.txt
./image.jpg
```
### Usage of `foldl`
The `foldl` command takes a [function argument](#available-function-arguments) and an initial value. The given function must be a binary function with two arguments, like `add` or `append`. The `foldl` command then applies this function iteratively by keeping an internal accumulator:
Add up the numbers from 0 to 100:
``` bash
> seq 100 | foldl add 0
5050
```
Multiply the numbers from 1 to 10:
``` bash
> seq 10 | foldl mul 1
3628800
```
Append the numbers from 1 to 10 in a string:
``` bash
> seq 10 | map append " " | foldl append ""
1 2 3 4 5 6 7 8 9 10
```
### Usage of `foldl1`
The `foldl1` command is a variant of `foldl` that uses the first input as the initial value.
This can be used to shorten the example above to:
``` bash
> seq 100 | foldl1 add
> seq 10 | foldl1 mul
> seq 10 | map append " " | foldl1 append
```
### Usage of `sort_by`
The `sort_by` command also takes a [function argument](#available-function-arguments). In the
background, it calls the function on each input line and uses the results to sort the *original input*.
Consider the following scenario:
``` bash
> ls
a.mp4 b.tar.gz c.txt
> ls | map filesize
7674860
126138
2214
```
We can use the `filesize` function to sort the entries by size:
```
> ls | sort_by filesize
c.txt
b.tar.gz
a.mp4
```
### Chaining commands
All of these commands can be composed by using standard UNIX pipes:
``` bash
> find
.
./folder
./folder/me.jpg
./folder/subdirectory
./folder/subdirectory/song.mp3
./document.txt
./image.jpg
> find | filter is_file | map basename | map append ".bak"
me.jpg.bak
song.mp3.bak
document.txt.bak
image.jpg.bak
```
### Lazy evaluation
All commands support lazy evaluation (i.e. they consume input in a streaming way) and never perform
unnecessary work (they exit early if the *output* pipe is closed).
As an example, suppose we want to compute the sum of all odd squares lower than 10000. Assuming we
have a command that prints the numbers from 1 to infinity (use `alias infinity="seq 999999999"` for
an approximation), we can write:
``` bash
> infinity | filter odd | map pow 2 | take_while less_than 10000 | foldl1 add
166650
```
### Working with columns
The `--column` / `-c` option can be used to apply a given function to a certain *column* in the input line (columns are separated by tabs). Column arrays can be created by using functions such as `duplicate`, `split sep` or `split_ext`:
``` bash
> ls | filter is_file | map split_ext
document txt
image jpg
> ls | filter is_file | map split_ext | map -c1 to_upper
DOCUMENT txt
IMAGE jpg
> ls | filter is_file | map split_ext | map -c1 to_upper | map join .
DOCUMENT.txt
IMAGE.jpg
```
Here is a more complicated example:
``` bash
> find -name '*.jpg'
./folder/me.jpg
./image.jpg
> find -name '*.jpg' | map duplicate
./folder/me.jpg ./folder/me.jpg
./image.jpg ./image.jpg
> find -name '*.jpg' | map duplicate | map -c2 basename
./folder/me.jpg me.jpg
./image.jpg image.jpg
> find -name '*.jpg' | map duplicate | map -c2 basename | map -c2 prepend "thumb_"
./folder/me.jpg thumb_me.jpg
./image.jpg thumb_image.jpg
> find -name '*.jpg' | map duplicate | map -c2 basename | map -c2 prepend "thumb_" | map run convert
Running 'convert' with arguments ['./folder/me.jpg', 'thumb_me.jpg']
Running 'convert' with arguments ['./image.jpg', 'thumb_image.jpg']
```
Get the login shell of user `shark`:
``` bash
> cat /etc/passwd | map split : | filter -c1 equal shark | map index 6
/usr/bin/zsh
```
### Available function arguments
You can call `ft-functions`, to get an overview of all available arguments to `map`, `filter`, etc.:
#### File and Directory operations ####
```
abspath :: Path → Path
dirname :: Path → Path
basename :: Path → Path
is_dir :: Path → Bool
is_file :: Path → Bool
is_link :: Path → Bool
is_executable :: Path → Bool
exists :: Path → Bool
has_ext ext :: Path → Bool
strip_ext :: Path → String
replace_ext new_ext :: Path → Path
split_ext :: Path → Array
```
#### Logical operations ####
```
non_empty :: * → Bool
nonempty :: * → Bool
```
#### Arithmetic operations ####
```
add num :: Int → Int
sub num :: Int → Int
mul num :: Int → Int
even :: Int → Bool
odd :: Int → Bool
pow num :: Int → Int
```
#### Comparison operations ####
```
eq other :: * → Bool
equal other :: * → Bool
equals other :: * → Bool
ne other :: * → Bool
not_equal other :: * → Bool
not_equals other :: * → Bool
ge i :: Int → Bool
greater_equal i :: Int → Bool
greater_equals i :: Int → Bool
gt i :: Int → Bool
greater i :: Int → Bool
greater_than i :: Int → Bool
le i :: Int → Bool
less_equal i :: Int → Bool
less_equals i :: Int → Bool
lt i :: Int → Bool
less i :: Int → Bool
less_than i :: Int → Bool
```
#### String operations ####
```
reverse :: String → String
append suffix :: String → String
strip :: String → String
substr start end :: String → String
take count :: String → String
to_lower :: String → String
to_upper :: String → String
replace old new :: String → String
prepend prefix :: String → String
capitalize :: String → String
drop count :: String → String
duplicate :: String → Array
contains substring :: String → Bool
starts_with pattern :: String → Bool
startswith pattern :: String → Bool
ends_with pattern :: String → Bool
endswith pattern :: String → Bool
len :: String → Int
length :: String → Int
format format_str :: * → String
```
#### Array operations ####
```
at idx :: Array → String
index idx :: Array → String
join separator :: Array → String
split separator :: String → Array
reverse :: Array → Array
```
#### Other operations ####
```
const value :: * → *
run command :: Array → !
id :: * → *
identity :: * → *
```
Raw data
{
"_id": null,
"home_page": null,
"name": "shell-functools",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "command-line, filesystem, functional-programming, shell, string-manipulation",
"author": null,
"author_email": "David Peter <mail@david-peter.de>",
"download_url": "https://files.pythonhosted.org/packages/d8/20/44d68070637ed02a176068b4c028438b4a65b0f4bf95b72e1720f62a98ae/shell_functools-0.3.1.tar.gz",
"platform": null,
"description": "# shell-functools\n\n[](https://github.com/sharkdp/shell-functools/actions/workflows/ci.yml)\n\n*A collection of functional programming tools for the shell.*\n\nThis project provides higher order functions like `map`, `filter`, `foldl`, `sort_by` and `take_while` as simple command-line tools.\nFollowing the UNIX philosophy, these commands are designed to be composed via pipes. A\n[large collection](#available-function-arguments) of functions such as `basename`, `replace`, `contains` or `is_dir` are provided as\narguments to these commands.\n\n## Contents\n\n* [Demo](#demo)\n* [Quick start](#quick-start)\n* [Documentation and examples](#documentation-and-examples)\n * [Usage of `map`](#usage-of-map)\n * [Usage of `filter`](#usage-of-filter)\n * [Usage of `foldl`](#usage-of-foldl)\n * [Usage of `foldl1`](#usage-of-foldl1)\n * [Usage of `sort_by`](#usage-of-sort_by)\n * [Chaining commands](#chaining-commands)\n * [Lazy evaluation](#lazy-evaluation)\n * [Working with columns](#working-with-columns)\n * [Available function arguments](#available-function-arguments)\n\n## Demo\n\n<a href=\"https://asciinema.org/a/6zsp3hEPpM7tmWHrjThl7idqh\" target=\"_blank\"><img src=\"https://asciinema.org/a/6zsp3hEPpM7tmWHrjThl7idqh.png\" width=\"600\" /></a>\n\n## Quick start\n\nIf you want to try it out on your own, run:\n``` bash\npip install shell-functools\n```\n\n## Documentation and examples\n\n### Usage of `map`\n\nThe `map` command takes a [function argument](#available-function-arguments) and applies it to every line of input:\n``` bash\n> ls\ndocument.txt\nfolder\nimage.jpg\n\n> ls | map abspath\n/tmp/demo/document.txt\n/tmp/demo/folder\n/tmp/demo/image.jpg\n```\n\n### Usage of `filter`\n\nThe `filter` command takes a [function argument](#available-function-arguments) with a `Bool`ean return type. It applies that function to each input line and shows only those that returned `true`:\n``` bash\n> find\n.\n./folder\n./folder/me.jpg\n./folder/subdirectory\n./folder/subdirectory/song.mp3\n./document.txt\n./image.jpg\n\n> find | filter is_file\n./folder/me.jpg\n./folder/subdirectory/song.mp3\n./document.txt\n./image.jpg\n```\n\n### Usage of `foldl`\n\nThe `foldl` command takes a [function argument](#available-function-arguments) and an initial value. The given function must be a binary function with two arguments, like `add` or `append`. The `foldl` command then applies this function iteratively by keeping an internal accumulator:\n\nAdd up the numbers from 0 to 100:\n``` bash\n> seq 100 | foldl add 0\n5050\n```\n\nMultiply the numbers from 1 to 10:\n``` bash\n> seq 10 | foldl mul 1\n3628800\n```\n\nAppend the numbers from 1 to 10 in a string:\n``` bash\n> seq 10 | map append \" \" | foldl append \"\"\n1 2 3 4 5 6 7 8 9 10\n```\n\n### Usage of `foldl1`\n\nThe `foldl1` command is a variant of `foldl` that uses the first input as the initial value.\nThis can be used to shorten the example above to:\n``` bash\n> seq 100 | foldl1 add\n> seq 10 | foldl1 mul\n> seq 10 | map append \" \" | foldl1 append\n```\n\n### Usage of `sort_by`\n\nThe `sort_by` command also takes a [function argument](#available-function-arguments). In the\nbackground, it calls the function on each input line and uses the results to sort the *original input*.\nConsider the following scenario:\n``` bash\n> ls\na.mp4 b.tar.gz c.txt\n> ls | map filesize\n7674860\n126138\n2214\n```\n\nWe can use the `filesize` function to sort the entries by size:\n```\n> ls | sort_by filesize\nc.txt\nb.tar.gz\na.mp4\n```\n\n### Chaining commands\n\nAll of these commands can be composed by using standard UNIX pipes:\n``` bash\n> find\n.\n./folder\n./folder/me.jpg\n./folder/subdirectory\n./folder/subdirectory/song.mp3\n./document.txt\n./image.jpg\n\n> find | filter is_file | map basename | map append \".bak\"\nme.jpg.bak\nsong.mp3.bak\ndocument.txt.bak\nimage.jpg.bak\n```\n\n### Lazy evaluation\n\nAll commands support lazy evaluation (i.e. they consume input in a streaming way) and never perform\nunnecessary work (they exit early if the *output* pipe is closed).\n\nAs an example, suppose we want to compute the sum of all odd squares lower than 10000. Assuming we\nhave a command that prints the numbers from 1 to infinity (use `alias infinity=\"seq 999999999\"` for\nan approximation), we can write:\n``` bash\n> infinity | filter odd | map pow 2 | take_while less_than 10000 | foldl1 add\n166650\n```\n\n### Working with columns\n\nThe `--column` / `-c` option can be used to apply a given function to a certain *column* in the input line (columns are separated by tabs). Column arrays can be created by using functions such as `duplicate`, `split sep` or `split_ext`:\n\n``` bash\n> ls | filter is_file | map split_ext\ndocument\ttxt\nimage\tjpg\n\n> ls | filter is_file | map split_ext | map -c1 to_upper\nDOCUMENT\ttxt\nIMAGE\tjpg\n\n> ls | filter is_file | map split_ext | map -c1 to_upper | map join .\nDOCUMENT.txt\nIMAGE.jpg\n```\n\nHere is a more complicated example:\n``` bash\n> find -name '*.jpg'\n./folder/me.jpg\n./image.jpg\n\n> find -name '*.jpg' | map duplicate\n./folder/me.jpg ./folder/me.jpg\n./image.jpg ./image.jpg\n\n> find -name '*.jpg' | map duplicate | map -c2 basename\n./folder/me.jpg me.jpg\n./image.jpg image.jpg\n\n> find -name '*.jpg' | map duplicate | map -c2 basename | map -c2 prepend \"thumb_\"\n./folder/me.jpg\t thumb_me.jpg\n./image.jpg thumb_image.jpg\n\n> find -name '*.jpg' | map duplicate | map -c2 basename | map -c2 prepend \"thumb_\" | map run convert\nRunning 'convert' with arguments ['./folder/me.jpg', 'thumb_me.jpg']\nRunning 'convert' with arguments ['./image.jpg', 'thumb_image.jpg']\n```\n\nGet the login shell of user `shark`:\n``` bash\n> cat /etc/passwd | map split : | filter -c1 equal shark | map index 6\n/usr/bin/zsh\n```\n\n\n### Available function arguments\n\nYou can call `ft-functions`, to get an overview of all available arguments to `map`, `filter`, etc.:\n\n#### File and Directory operations ####\n```\nabspath :: Path \u2192 Path\ndirname :: Path \u2192 Path\nbasename :: Path \u2192 Path\nis_dir :: Path \u2192 Bool\nis_file :: Path \u2192 Bool\nis_link :: Path \u2192 Bool\nis_executable :: Path \u2192 Bool\nexists :: Path \u2192 Bool\nhas_ext ext :: Path \u2192 Bool\nstrip_ext :: Path \u2192 String\nreplace_ext new_ext :: Path \u2192 Path\nsplit_ext :: Path \u2192 Array\n```\n#### Logical operations ####\n```\nnon_empty :: * \u2192 Bool\nnonempty :: * \u2192 Bool\n```\n#### Arithmetic operations ####\n```\nadd num :: Int \u2192 Int\nsub num :: Int \u2192 Int\nmul num :: Int \u2192 Int\neven :: Int \u2192 Bool\nodd :: Int \u2192 Bool\npow num :: Int \u2192 Int\n```\n#### Comparison operations ####\n```\neq other :: * \u2192 Bool\nequal other :: * \u2192 Bool\nequals other :: * \u2192 Bool\nne other :: * \u2192 Bool\nnot_equal other :: * \u2192 Bool\nnot_equals other :: * \u2192 Bool\nge i :: Int \u2192 Bool\ngreater_equal i :: Int \u2192 Bool\ngreater_equals i :: Int \u2192 Bool\ngt i :: Int \u2192 Bool\ngreater i :: Int \u2192 Bool\ngreater_than i :: Int \u2192 Bool\nle i :: Int \u2192 Bool\nless_equal i :: Int \u2192 Bool\nless_equals i :: Int \u2192 Bool\nlt i :: Int \u2192 Bool\nless i :: Int \u2192 Bool\nless_than i :: Int \u2192 Bool\n```\n#### String operations ####\n```\nreverse :: String \u2192 String\nappend suffix :: String \u2192 String\nstrip :: String \u2192 String\nsubstr start end :: String \u2192 String\ntake count :: String \u2192 String\nto_lower :: String \u2192 String\nto_upper :: String \u2192 String\nreplace old new :: String \u2192 String\nprepend prefix :: String \u2192 String\ncapitalize :: String \u2192 String\ndrop count :: String \u2192 String\nduplicate :: String \u2192 Array\ncontains substring :: String \u2192 Bool\nstarts_with pattern :: String \u2192 Bool\nstartswith pattern :: String \u2192 Bool\nends_with pattern :: String \u2192 Bool\nendswith pattern :: String \u2192 Bool\nlen :: String \u2192 Int\nlength :: String \u2192 Int\nformat format_str :: * \u2192 String\n```\n#### Array operations ####\n```\nat idx :: Array \u2192 String\nindex idx :: Array \u2192 String\njoin separator :: Array \u2192 String\nsplit separator :: String \u2192 Array\nreverse :: Array \u2192 Array\n```\n#### Other operations ####\n```\nconst value :: * \u2192 *\nrun command :: Array \u2192 !\nid :: * \u2192 *\nidentity :: * \u2192 *\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A collection of functional programming tools for the shell.",
"version": "0.3.1",
"project_urls": {
"Home": "https://github.com/sharkdp/shell-functools"
},
"split_keywords": [
"command-line",
" filesystem",
" functional-programming",
" shell",
" string-manipulation"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "7645f69d53d9321649e94c910b337825c1b17247aad4397f4fa3e34659cb2f2d",
"md5": "8852cc1aac86cdd63df4b8d7359578e9",
"sha256": "51346afca1b5889e11d09aa80cc1f3d87a5112c0f7bba44c76355a5381384a3b"
},
"downloads": -1,
"filename": "shell_functools-0.3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8852cc1aac86cdd63df4b8d7359578e9",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 16172,
"upload_time": "2024-11-22T18:56:42",
"upload_time_iso_8601": "2024-11-22T18:56:42.639114Z",
"url": "https://files.pythonhosted.org/packages/76/45/f69d53d9321649e94c910b337825c1b17247aad4397f4fa3e34659cb2f2d/shell_functools-0.3.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "d82044d68070637ed02a176068b4c028438b4a65b0f4bf95b72e1720f62a98ae",
"md5": "3cf9d391a94487dc813f93b003652cdf",
"sha256": "821423f10f034de09303cdb2739ccf381d93697bf52763fcff8b0e4058336456"
},
"downloads": -1,
"filename": "shell_functools-0.3.1.tar.gz",
"has_sig": false,
"md5_digest": "3cf9d391a94487dc813f93b003652cdf",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 228002,
"upload_time": "2024-11-22T18:56:44",
"upload_time_iso_8601": "2024-11-22T18:56:44.447567Z",
"url": "https://files.pythonhosted.org/packages/d8/20/44d68070637ed02a176068b4c028438b4a65b0f4bf95b72e1720f62a98ae/shell_functools-0.3.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-22 18:56:44",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "sharkdp",
"github_project": "shell-functools",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "shell-functools"
}
JSON
