<!-- Cover -->
<div align="center">
<img src="https://raw.githubusercontent.com/pyrustic/misc/master/assets/backstage/cover.jpg" alt="Demo" width="640">
<p align="center">
By © Jorge Royan / <a rel="nofollow" class="external free" href="http://www.royan.com.ar">http://www.royan.com.ar</a>, <a href="https://creativecommons.org/licenses/by-sa/3.0" title="Creative Commons Attribution-Share Alike 3.0">CC BY-SA 3.0</a>, <a href="https://commons.wikimedia.org/w/index.php?curid=23405928">Link</a>
</p>
</div>
<!-- Intro Text -->
# Pyrustic Backstage
<b> Three-speed scripting language and task automation tool </b>
This project is part of the [Pyrustic Open Ecosystem](https://pyrustic.github.io).
> [Installation](#installation) [Demo](#demo) [Latest](https://github.com/pyrustic/backstage/tags) [Modules](https://github.com/pyrustic/backstage/tree/master/docs/modules#readme)
## Table of contents
- [Overview](#overview)
- [Structure of the script file](#structure-of-the-script-file)
- [Spawning processes and branching subtasks](#spawning-processes-and-branching-subtasks)
- [Data types and control flow](#data-types-and-control-flow)
- [Namespaces and persistence](#namespaces-and-persistence)
- [Variable interpolation and escaping](#variable-interpolation-and-escaping)
- [Environment variables and language syntax](#environment-variables-and-language-syntax)
- [File and directory manipulation](#file-and-directory-manipulation)
- [Exception handling and tests](#exception-handling-and-tests)
- [Interfacing with Python](#interfacing-with-python)
- [Exception handling and tests](#exception-handling-and-tests)
- [Command line interface and developer experience](#command-line-interface-and-developer-experience)
- [Miscellaneous](#miscellaneous)
- [Demo](#demo)
- [Installation](#installation)
# Overview
**Backstage** is a cross-platform **automation tool** that looks for a `backstage.tasks` file in the current working directory to run a specific task defined in that file on demand. A task can be a sequence or pipeline of processes to be spawned, instructions for performing file and directory manipulation, or something more sophisticated.
The `backstage.tasks` file uses the [Jesth](https://github.com/pyrustic/jesth) (**J**ust **E**xtract **S**ections **T**hen **H**ack) file format that acts like a broken [INI file](https://en.wikipedia.org/wiki/INI_file) parser that only extract sections each made of a header and a body which is just a list of lines.
Using an eponymous **three-speed scripting language** designed for the automation tool, the programmer can, inside the `backstage.tasks` file, define, coordinate and use the various resources at his disposal to automate things.
The three-speed scripting language concept is inspired from the three forward gear ratios of early automobiles [transmission system](https://en.wikipedia.org/wiki/Manual_transmission). In the following subsections, we will explore each metaphorical gear of the **Backstage** scripting language, then we will briefly expose the automation tool itself.
## First gear
In first gear, a `backstage.tasks` file is intended to store a list of tasks related to a specific project, each task exposing a list of commands or subtasks to be executed. Here, a command represents a process or pipeline of processes to be spawned. [Environment variables](#environment-variables-and-language-syntax) can be used in commands via [variable interpolation](https://en.wikipedia.org/wiki/String_interpolation). No other logic is involved.
### Example
```
[task1]
# three commands to run sequentially
$ git commit -m 'Update'
$ python -m my.package.module
$ program1 arg1 | program2 {HOME}
---
# run the subtask 'task2'
& task2
---
# run the subtask 'task3' in a new thread
~ task3
[task2]
$ program val1 {CWD}
$ git push origin master
[task3]
# some heavy computation
$ engine -x 5000
$ engine --cleanup
[_task4]
# This is a private task (with an underscore as prefix)
$ clean dir
```
## Second gear
In second gear, the `backstage.tasks` file not only stores the tasks like in first gear, but here logic intervenes, variables are defined, control flow is used, built-in commands are called, et cetera. Basically, in second gear, **Backstage** unleashes its power and allows the programmer to anticipate problems, make sophisticated combinations of subtasks, in short to write a real **script to automate things**.
### Example
```
[task1]
# commit changes
$ git commit -m 'Update'
---
# tell user if 'Commit' has been success
if R == 0
# print 'Success !'
: Success !
else
: Failed to commit changes
---
# say hello ten times
set age = 42
set name = `John Doe`
from 1 to 10
$ python -m say.hello {name} {age}
---
# create a file in user home
set pathname = {HOME}/iliad.txt
create file pathname
---
# append some data to a file
set data = `\nHello World !`
append data to pathname
---
# browse current working directory
browse files and dirs in CWD
: Directory -> {R}
: Files ->
for item in files
: - {item}
: Dirs ->
for item in dirs
: - {item}
```
## Third gear
In third gear, in addition with whatever can be done with previous gears, the programmer can directly from the `backstage.tasks` file, call [Python](https://www.python.org/about/) functions with arguments and get the return ! Thanks to this third gear, any too complex or overly verbose calculation can be written in **Python** and called from **Backstage**. This [functionality](#interfacing-with-python) alone proves that **Backstage** is all about making the programmer's life better, not pretending to replace existing mature solutions that actually work.
### Example
```
[task1]
interface with package.coffeemaker alias coffeebro
set sugar_cubes (int) = 1 + 1 + 1
set extra = `milk`
call coffeebro.make(sugar_cubes, extra, 42)
if R == 1
: Coffee successfully made !
else
: Oops, failed to make coffee...
: Exception -> {EXCEPTION}
: Traceback -> {TRACEBACK}
```
## Automation tool
The scripting language help to define tasks in the `backstage.tasks` file that is intended to be consumed by the automation tool. As an automation tool, **Backstage** exposes a [command line interface](#command-line-interface-and-developer-experience) that allows the user to **discover** available tasks, **run** a task with arguments, read a task [documentation](#embedded-documentation-and-tests), use a [glob](https://en.wikipedia.org/wiki/Glob_(programming))-like syntax to **search** for a task by its name or by a keyword (case-insensitive mode) that is part of the documentation of the task, et cetera.
### Example 1
Let's assume that there is a `backstage.tasks` file in the directory `/home/alex/project`. This `backstage.tasks` file contains three public tasks and one private tasks (prefixed with an underscore).
This example shows how one could use the automation tool to run a task defined in the `backstage.tasks` file:
```bash
$ cd /home/alex/project
$ backstage -c
Available tasks (3):
make_coffee task1 task2
$ backstage make_coffee
Making coffee...
$ backstage mak*
Making coffee...
$ backstage make_coffee sugar=3
Making coffee with 3 sugar cubes...
```
### Example 2
This is the contents of a `backstage.tasks` file located at `/home/alex/project`:
```
[task]
# define 'x' as a variable with an 'int' assignment tag
set x (int) = 1 + 1 + 1
# default name (by default, the assignment tag is 'str')
set default_name = `John Doe`
# print some text
: Hi and Welcome !
: I can print your name {x} times in a row !
# take user input (always a 'str')
> name : `What is your name ? `
# control flow
if name == EMPTY
set name = {default_name}
# iteration
from 1 to x
: {R} - Ave {name} !
# branch subtask '_task2' and pass it an argument
& _task2 {name}
[_task2]
# define the variable 'name' (first argument passed to this task)
set name = {ARGS[0]}
# Just say Goodbye !
: Goodbye {name} !
```
Let's run the task named `task` from the command line:
```bash
$ cd /home/alex/project
$ backstage task
Hi and Welcome !
I can print your name 3 times in a row !
What is your name ? Alex
1 - Ave Alex !
2 - Ave Alex !
3 - Ave Alex !
Goodbye Alex !
```
> **Note:** You can reproduce this example as it. Just [install](#installation) **Backstage**, copy-paste the script in a `backstage.tasks` file, then run `backstage task` in the command line.
## And more
There is more to talk about **Backstage**, like the ability to embed documentation and tests in the `backstage.tasks` file and access them from the command line. Backstage is enough versatile to do the job of a trivial task runner or to automate things with its scripting language that has a built-in bridge to the powerful Python ecosystem.
In the following sections, we will explore this project in depth. You can also jump to the [demo](#demo) to start playing with **Backstage** !
# Structure of the script file
As stated in the [Overview](#overview) section, a `backstage.tasks` file is basically a [JesthFile](https://github.com/pyrustic/jesth).
In a `backstage.tasks` file, a section represents a task. The section title is the name of the task and the section body is made of commands to run and the constructs of the **Backstage** scripting language. A valid task name is an alphanumeric string that can contains an underscore.
```
You can write here at the top of the script,
a description of the script,
the date of its creation,
or any useful information.
[task1]
# body of task1
...
[task2]
# body of task2
...
[_private]
# prefix a task name with an underscore
# to turn it into a private task that
# won't appear in the list of available tasks
# when you will type 'backstage --check' in the command line
```
As you can guess, a line starting with `#` is a comment. But this is only true inside a task body, because in fact not all sections are tasks as described before: a section can also be an embedded test or documentation.
## Embedded documentation
You can embed documentation inside the `backstage.tasks` file. To create a documentation for a task, create a section which name is postfixed with `.help`:
```
[task1]
pass
[task1.help]
This is the description line.
Usage:
backstage task <option> <path>
Options:
-m, --msg Show message
-x, --exit Exit blah blah
```
From the **command line**, you can read the documentation of an arbitrary task:
```bash
$ backstage -h task1
This is the description line.
Usage:
backstage task <option> <path>
Options:
-m, --msg Show message
-x, --exit Exit blah blah
```
## Embedded tests
You can embed tests inside the `backstage.tasks` file. To create a test for a task, create a section which name is postfixed with `.test`:
```
[task1.test]
# perform some test here
# ...
assert some_var == some_var
```
From the **command line**, you can run the test of an arbitrary task:
```bash
$ backstage --test task1
```
or the tests of a bunch of tasks:
```bash
$ backstage --test task1 task2 task3
```
or run all tests defined in the `backstage.tasks` file:
```bash
$ backstage --test
```
# Spawning processes and branching subtasks
You can write commands to spawn a process or a pipeline of processes:
```
[task]
# spawn Git to perform a 'commit'
$ git commit -m "Update"
# spawn a pipeline of three processes
$ program1 arg1 | program2 arg2 | program3
```
Commands to spawn processes support variable interpolation:
```
[task]
# use HOME environment variable
$ ls {HOME}
# access the first index of the ARGS list
$ program {ARGS[0]}
```
You can **push** data to the **input** of a process:
```
[task]
# define variables name and age
set name = `John Doe`
set age (int) = 40 + 2
# name and age will be pushed to the input of the next spawned process
push name age
$ program1 {HOME}
# from now you can access via the environment variable R,
# the return code (exit status code)
: The return code is {R}
```
You can **capture** a process:
```
[task]
# you can capture a process,
# so you will get a direct access to the output and error
($) program2
# print the content of OUTPUT and ERROR
: Output -> {OUTPUT}
: Error -> {ERROR}
```
You can redirect **STDOUT** and **STDERR**:
```
[task]
# redirect STDOUT and STDERR
set STDOUT = `/path/to/file_out`
set STDERR = `/path/to/file_err`
$ program1
---
# cross platform DEVNULL:
set STDOUT = /dev/null
$ program2
```
## Branching subtasks
You can branch a subtask and pass arguments to it:
```
[task1]
# branch 'task2'
& task2 {HOME}
[task2]
# from task2, we can access arguments passed to it.
# ARGS (environment variable) is a list of arguments.
: Arguments -> {ARGS}
```
Whenever you branch a subtask, a new instance of this subtask is created, with its own variables. You can share data with a subtask with one of these three ways:
- pass arguments to the subtask while branching it;
- use the global [namespace](#namespaces-and-persistence);
- use the database [namespace](#namespaces-and-persistence).
A subtask can also return data that is cached in the `R` environment variable:
```
[task1]
& task2 "John Doe"
# from now, R contains 'Hello John Doe !'
[task2]
set result = `Hello {ARGS} !`
return result
```
### Multithreading
Branching a subtask is done in the main thread. But one can create a new thread for a subtask:
```
[task1]
# run an instance of task2 in a new thread
~ task2
# the next command won't wait 'task2' to complete
$ git commit -m "Update"
[task2]
# this task sleeps for 5 seconds
sleep 5
```
# Data types and control flow
In the next subsections we will talk about data types then control flow.
## Data types
**Backstage** supports [variables](https://en.wikipedia.org/wiki/Variable_(computer_science)) and let the programmer set, use, clear, and drop variables.
Under the hood, **Backstage** works with five **Python** data types: `str`, `list` (one-dimensional), `dict` (one-dimensional), `int`, and `float`. But these data types aren't intended to be directly used by the programmer. Instead, `assignment tags` are used to tell the interpreter how to treat a variable.
These `assignment tags` are: `raw`, `str`, `list`, `dict`, `int`, `float`, `date`, `time`, `dtime`, and `tstamp`.
```
[task]
# this is a string
set var = 42
# this is another string
set var (str) = `Hello World`
# this is a list
set var (list) = `reg green blue`
# this is the same list but edited
set var[0] = `yellow`
# var -> yellow green blue
# this is a dict
set var (dict) = `user="John Doe" age=42 location=Kernel`
# this is the same dict but edited
set var.user = `Jane Doe`
# var -> user='Jane Doe' age=42 location=Kernel
# this is an integer
set var (int) = 40 + 2
# var -> 42
# this is a raw string
set regex (raw) = `[\S\s]*?`
# get the current timestamp
set now (int) = {NOW}
# now -> 1662569326
# convert it into datetime
set var (dtime) = {now}
# var -> 2022-09-07 17:48:46
# go from a datetime to timestamp
set var (tstamp) = `2022-09-07 17:48:46`
# var -> 1662569326
# extract the time part of a timestamp
set var (time) = 1662569326
# var -> 17:48:46
```
> Note that all variables have a **string** representation and [backticks](https://en.wikipedia.org/wiki/Backtick) are used optionally as delimiters that will be ignored by the interpreter. So you can put backticks around an integer, and you can insert a list in a string.
### Here document
**Backstage** supports [here document](https://en.wikipedia.org/wiki/Here_document) for strings inside the `backstage.tasks` file:
```
[task]
# this is a text with two lines
set text (str) = `First line\nSecond line`
# this is another text with three lines
set text = `January\nFebruary\nMarch`
# this is not a text with two lines
set var (raw) = `Hello\nWorld`
# this is not a text with two lines
set var (str) = `Hello\\nWorld`
```
### Variable interpolation
[Variable interpolation](https://en.wikipedia.org/wiki/String_interpolation) is supported with the ability to access from a list or a string, the value at an arbitrary index, or from a dictionary, the value of a key.
```
[task]
# let's play with 'str' variables
set x = `red`
set var = `{x} green blue`
# var -> red green blue
# get the value of element at index 0
set value = {var[0]}
# value -> r
---
# let's play with a list
set x = `red`
set var (list) = `{x} green blue`
# get the value of element at index 0
set value = {var[0]}
# value -> red
# get the value of elements from index 1 to the end
set value = {var[1:]}
# value -> green blue
---
# let's play with a dict
set x (int) = 40 + 2
set var (dict) = `name="John Doe" age={x}`
# get the value of key 'name'
set value = {var.name}
# value -> John Doe
# get the value of key 'age'
set value = {var.age}
# value -> 42
---
# cancel the variable interpolation
set var1 = `Hello`
set var2 = `{{var1}} World`
# var2 -> {var1} World
```
## Control flow
**Backstage** implements conditionals and loops. A wide range of operators are available to compare values.
### Conditionals
```
[task]
set var1 = 1
set var2 = 1
set x = 2
set regex (raw) = `[\S\s]*?`
set text = `Hello world`
set y = `Hello`
# conditionals support the classic
# operators: == != > < >= <=
if var1 == 1
$ program1
elif var2 == x
$ program2
else
$ program3
# Backstage supports logical and/or
if var1 == var2 and var1 >= 3
$ program1
# use the 'matches' operator
# or the negated one: !matches
if regex matches text
: Matched !
elif regex !matches text
: Mismatched !
# you can use the in operator
# and also the negated one: !in
if y in text
pass
elif y !in text
pass
```
### Loops
```
[task]
# From To loop
from 10 to 0
: {R}
# For loop - iterate over a string
set text = `Hello World`
for char in text
# N is an environment variable that serves as counter
# for all loops
: {N}- {char}
# For loop - iterate over a list
set data (list) = `red green blue`
for item in data
: {item}
# For loop - iterate over a dict
set data (dict) = `name="John Doe" age=42`
for item in data
: key -> {item[0]} value -> {item[1]}
# For loop - iterate over a file
set path = `/home/alex/iliad.txt`
for line in path (file)
: Line {N}
: {line}
:
# while loop
set var = 1
while var == 1
: One Time Hello
break
# browse loop
set path = `/home/alex`
browse files and dirs in path
: Directory -> {R}
for item in files
: {item}
for item in dirs
: {item}
```
# Namespaces and persistence
[Namespaces](https://en.wikipedia.org/wiki/Namespace) are implemented in **Backstage** to provide convenient management of variables by defining three [scopes](https://en.wikipedia.org/wiki/Scope_(computer_science)):
- `L` for **Local** scope;
- `G` for **Global** scope;
- `D` for **Database** scope.
By default, variables exist in the **Local** namespace and are only accessible to the running task.
To share data with a subtask, one can expose arbitrary variables that will be copied into the **Global** namespace which is readable and writable (thread-safe) by all subtasks.
```
[task]
# by default, variables are defined in Local,
# i.e. they are only visible in the scope of the current task
set var = 42
: Var contains {var}
: Var still contains {L:var}
# you can make local variables public
expose var
# from now, you can get a thread-safe access to var
# from any running task:
: Global var contains {G:var}
# branch _task2
& _task2
[_task2]
: I got {G:var} !
```
Data can also be **persisted**:
```
[task]
set var = 42
store var
# from now, 'var' can be accessed by all tasks
# in this runtime but also in future runtimes
: Var contains {D:var}
# if you aren't sure about the existence
# of a variable, just do this:
default var
# it also works with a bunch of variables:
default var1 L:var2 D:var3 G:var4
# from now, if 'var1' hasn't been manually defined
# by the programmer, it will be
# automatically initialized and
# its value will be an empty string
```
Persisted variables are stored in `.backstage/database.json`.
# Variable interpolation and escaping
During the string interpolation of a command that spawn processes or branch a subtask, variables that are of the `str` type are automatically [shell-escaped](https://en.wikipedia.org/wiki/Escape_character).
```
[task]
# This is a 'str' variable (backticks aren't quotes, by the way!)
set name = `John Doe`
set colors (list) = `red green blue`
: Welcome {name} !
# Welcome John Doe
$ program name={name} -c {colors}
# program name='John Doe' -c red green blue
# Notice the quotes automatically added around the name John Doe
```
# Environment variables and language syntax
Environment variables are local to each instance of task. They are defined as uppercase strings. One can edit their contents but can't create new environment variables.
This is the exhaustive list of environment variables:
|Variables|Description|
|---|---|
|`ARGS`|List of arguments passed to this task from the command line|
|`CWD`|Current working directory|
|`DATE`|The current date in the **YYYY-MM-DD** format|
|`EMPTY`|Just an empty string|
|`ERROR`|Error string from a process previously spawned|
|`EXCEPTION`|Name of the last exception raised|
|`FALSE`|The integer **0**|
|`HOME`|The path to `$HOME`. Example: `/home/alex`|
|`LINE`|The current line (1-based numbering) of execution in the task body|
|`N`|Counter for `while`, `for`, `from`, and `browse` loops|
|`NOW`|Current timestamp in seconds|
|`ONE`|The integer **1**|
|`OS`|The running operating system: `aix`, `linux`, `win32`, `cygwin`, `darwin`|
|`OUTPUT`|Output string from a process previously spawned|
|`R`|The **return** of Python functions, built-in commands, statements, constructs, or process exit status codes|
|`RANDOM`|Random integer between **0** and **255** (closed interval)|
|`SPACE`|One space ` ` character|
|`STDERR`|Use this variable to perform **STDERR** redirection|
|`STDIN`|Use this variable to perform **STDIN** redirection|
|`STDOUT`|Use this variable to perform **STDOUT** redirection|
|`TASK`|The name of the currently running task|
|`TIME`|The current time in the **HH:MM:SS** format|
|`TIMEOUT`|Timeout in seconds for commands that spawn processes. Default value: **30** seconds|
|`TMP`|Temporary directory. **Attention**, this directory will automatically disappear at the end of the runtime ! So think twice before moving files inside|
|`TRACEBACK`|[Traceback](https://en.wikipedia.org/wiki/Stack_trace#Python) of the last exception raised|
|`TRASH`|Path to the trash: `$HOME/PyrusticData/trash`|
|`TRUE`|The integer **1**|
|`ZERO`|The integer **0**|
> Note that the `TRACEBACK` and `EXCEPTION` variables are cleared after the next successful command. **Backstage** also generates for convenience, `ARG0`, `ARG1`, `ARGx`, according to the contents of `ARGS`. For example, if `ARGS`contains two arguments, you can expect that `ARG0` and `ARG1` exist.
## Language syntax
In this section we will explore the built-in commands, statements, keywords, symbols, and language constructs that make **Backstage**.
> Note that wherever a built-in command or statement expects a **variable** that will be **read**, for convenience you can instead of supplying a variable name, define an **inline** `int` or `float` literal.
> Also, consider that the `R` environment variable is your friend, since it is used to cache the data returned by a statement, a construct, or a command.
### APPEND
Append data to a file.
**Usage:** `append <var> to <filename_var>`
### ASSERT
Test is a condition is true.
**Usage:** `assert (<var1>|<regex_var>) (==|!=|<=|>=|<|>|in|!in|rin|!rin|matches|!matches) <var2> [and|or] ...`
**Example:** `assert regex_var matches text_var and var1 in list`
Note that `!` is used to express negation and `rin` is a Regex-based `in`. A regexly-in operator ;)
### BRANCH
Branch a subtask. The syntax is similar to the one to spawn processes, i.e., a string of words. The syntax supports variable interpolation.
**Usage:** `& <subtask> [<argument> ...]`
**Example:** `& subtask1 name="John Doe" age=42 city={city}`
> In this example, the `city` variable will be automatically shell-escaped during its interpolation.
### BREAK
Break a loop.
**Usage:** `break`
### BROWSE
Loop construct to browse a directory.
**Usage:** `browse [files] [and] [dirs] in <dirname_var>`
**Example:**
```
[task]
browse files and dirs in dirname
: Root -> {R}
for item in files
: {item}
for item in dirs
: {item}
browse files in dirname
pass
browse dirs in dirname
pass
```
### CALL
Call a **Python** function from **Backstage** with arguments, then get the return !
**Usage:** `call <module>.<function>[(<argument_var>, ...)]`
**Example:**
```
[task]
# interface with the Python module
interface with package.coffee_module alias coffeemaker
# call the 'make' function with arguments then get the return
call coffeemaker.make(sugar_cube, extra, 42)
: Result -> {R}
```
### CD
Change directory.
**Usage:** `cd <dirname_var>`
### CHECK
Return the data type (`str`, `list`, `dict`, `int`, `float`) of a variable if it exists, else return an empty string.
**Usage:** `check <var>`
**Example:**
```
[task]
check myvar
if R == EMPTY
: This variable doesn't exist at all !
else
: 'myvar' exists, its data type is {R}
```
### CLEAR
Clear the content of a variable or a list of variables.
**Usage:** `clear <var> ...`
**Example:** `clear var1 var2 var3`
### COMMENT
Comment.
**Usage:** `# <comment>`
### CONFIG
Read and write configuration options (`FailFast`, `ReportException`, `ShowTraceback`, `TestMode`, `AutoLineBreak`).
**Usage:** `config <option> ...`
**Example:**
```
[task]
config FailFast=1 AutoLineBreak=0
config TestMode
if R == 1
: Test Mode On
elif R == 0
: Test Mode Off
```
### COPY
Copy a file or a directory tree to a new destination.
**Usage:** `copy <src_path_var> to <dest_path_var>`
### COUNT
Count `chars`, `items`, and `lines` in the content of a variable or inside a file (if the `(file)` tag is applied).
**Usage:** `count (chars|items|lines) in (<var>|<filename_var>) [(file)]`
**Example:**
```
[task]
set path = /home/alex/iliad.txt
count chars in path (file)
if R == 0
: The file is empty !
```
### CREATE
Create a new file or directory.
**Usage:** `create (dir|file) <path_var>`
### DEFAULT
Define an empty variable (or a bunch of variables) if it doesn't exist yet in the namespace.
**Usage:** `default <var> ...`
**Example:**
```
[task]
# default two variables in the Local namespace
default var1 L:var2
# default one variable in the Database namespace
default D:name
# from now, 'D:name' can be safely accessed
# even though the 'database.json' file supposed
# to contain the 'name' value was inadvertently deleted.
```
### DROP
Destroy a variable (or a bunch of variables).
**Usage:** `drop <var> ...`
### ELIF
Part of the `if` conditional construct.
**Usage:** `elif (<var1>|<regex_var>) (==|!=|<=|>=|<|>|in|!in|rin|!rin|matches|!matches) <var2> [and|or] ...`
### ELSE
Part of the `if` conditional construct.
**Usage:** `else`
### ENTER
Invite user to submit data.
**Usage:** `> [<var> [: <text>]]`
**Example:**
```
[task]
> name : Please enter your name
# have you spotted the space at the end the line above ?
# the same line can be rewritten like this:
> name : `Please enter your name `
# backticks serve as delimiters that will be ignored
# this one is also possible:
set msg = `Please enter your name `
> name : {msg}
# even this:
set info (dict) = name="John Doe" age=42
> info.name : `Please enter your name`
```
### EXIT
Exit.
**Usage:** `exit`
### EXPOSE
Copy a variable (or a bunch of variables) into the **Global** namespace.
**Usage:** `expose <var> ...`
### FAIL
Deliberately fail. It breaks the running task and mark it as a failure.
**Usage:** `fail`
### FIND
Find files and or directories paths.
**Usage 1:** `find [all] (paths|files|dirs) in <dirname_var>`
**Usage 2:** `find ... matching <regex_var>`
**Usage 3:** `find ... [and] (accessed|modified|created) (at|after|before|between) <timestamp_var> [and <timestamp_var>]`
**Example:** `find files in dirname matching regex and accessed between timestamp1 and timestamp2`
### FOR
A `for` loop to iterate the content of a variable or the content of a file (if you apply the `(file)` tag).
**Usage:** `for (char|item|line) in (<var>|<filename_var>) [(file)]`
**Example:**
```
[task]
# this code iterates over each character of the Iliad,
# and outputs it as it,
# with one twist: each line starts with its index (0-based)
set path = `/home/alex/iliad.txt`
# the print statement (:) won't anymore
# automatically add a line break !
config AutoLineBreak=0
for line in path (file)
: `{N} `
for char in line
: {char}
: \n
```
### FROM
A loop to go from an integer to another one. If the `start` integer is superior to the `end` integer, the count will decrease.
**Usage:** `from <start> to <end>`
**Example:**
```
[task]
from 10 to 0
# here, N will go from 0 to 10
# but R will go from 10 to 0
# because N is a counter for all loops
# while R is a cache for whatever is returned
# by a command, a statement, or a construct
: {N}\t{R}
# As you can see, I can add a Tab \t since
# Backstage supports natively here document ;)
# To get a simple backslash followed by a 't': \\t
```
### GET
Get a `char`, an `item`, or a `line` at index `x` (including negative index) from a target. The target can be the content of a variable or a file (if you apply the `(file)` tag).
**Usage:** `get (char|item|line) <index> from (<var>|<filename_var>) [(file)]`
### IF
Conditional construct.
**Usage:** `if (<var1>|<regex_var>) (==|!=|<=|>=|<|>|in|!in|rin|!rin|matches|!matches) <var2> [and|or] ...`
**Example:**
```
[task]
default var1 var2 var3 var4
if var1 == var2 or var3 == var4
pass
elif EMPTY == EMPTY
pass
else
pass
```
### INTERFACE
Interface with a **Python** module.
**Usage:** `interface with [<package>.]<module> [alias <name>]`
**Example:**
```
[task]
interface with package.mymodule alias module
default var1 var2
call module.function(var1, var2)
```
### LINE
Draw a line.
**Usage:** `(=|-) ...`
**Example:** `----------` or `==========`
### MOVE
Move a file or a directory tree to a new destination.
**Usage:** `move <src_path_var> to <dest_path_var>`
### PASS
Placeholder for the code that you might write in the future. This statement does nothing. It is the same as the eponymous one in **Python**.
**Usage:** `pass`
### POKE
Poke a file or directory to get access to a `dict` of properties if this path exists. Available properties: `size` `mtime` `ctime` `atime` `nlink` `uid` `gid` `mode` `ino` `dev`.
**Usage:** `poke <path_var>`
**Example:**
```
[task]
set path = /home/alex/iliad.txt
poke path
if R == EMPTY
: Oops ! This file doesn't exist
else
: File size -> {R.size}
```
### PREPEND
Prepend data to a file
**Usage:** `prepend <var> to <filename_var>`
### PRINT
Print data. You can use backquotes as delimiters. This statement supports variable interpolation.
**Usage:** `: <text>`
**Example:**
```
[task]
: Hello World !
# Have you spotted the two extra spaces characters ?
: ` Hello World ! `
# hehehe, got you ! ;)
```
### PUSH
Push variables into the input of the next spawned process.
**Usage:** `push <var> ...`
### READ
Read all or a specific line index (including negative index) from a file.
**Usage:** `read (*|<index>) from <filename_var>`
### REPLACE
Replace some pattern in a text with a replacement value.
**Usage:** `replace <regex_var> in <text_var> with <replacement_var>`
### RETURN
Return from a task with a value.
**Usage:** `return [<var>]`
### SET
Define a new variable or update the content of an existing variable. You don't can't specify a data type, but instead you can apply an assigment tag that is one of: `(raw)` `(str)` `(list)` `(dict)` `(int)` `(float)` `(date)` `(time)` `(dtime)` `(tstamp)`. Note that backquotes can be used as delimiters for the value (right side of the equal sign). These delimiters will be ignored. Backticks aren't quotes. This statement supports variable interpolation.
**Usage:** `set <var> [(raw)|(str)|(list)|(dict)|(int)|(float)|(date)|(time)|(dtime)|(tstamp))] = <value>`
**Example:** `set var (int) = 1 + 2`
### SLEEP
Sleep for `x` seconds.
**Usage:** `sleep <seconds>`
### SPAWN
Spawn a new process.
**Usage:** `$ <program> [<argument> ...]`
**Example:** `$ program1 arg {var} | program2 `
### SPLIT
Split with a regex pattern a text into a list.
**Usage:** `split <text_var> with <regex_var>`
### SPOT
Count the number of occurrences of a regex pattern inside a text.
**Usage:** `spot <regex_var> in <text_var>`
### STORE
Store a variable (or a bunch of variables) in the **Database** namespace. A stored variable can be accessed like this: `D:var`
**Usage:** `store <var> ...`
### THREAD
Branch a subtask... but in a new thread.
**Usage:** `~ <subtask> [<argument> ...]`
### WHILE
The `while` loop. Use `break` to break it, and check `N` if you need a counter.
**Usage:** `while (<var1>|<regex_var>) (==|!=|<=|>=|<|>|in|!in|rin|!rin|matches|!matches) <var2> [and|or] ...`
### WRITE
Erase the content of a file to write some data inside.
**Usage:** `write <var> to <filename_var>`
# File and directory manipulation
Let's explore how file and directory manipulatin is performed with **Backstage**.
## Resource creation
Create a file:
```
[task]
# create a file
set path = /home/alex/iliad.txt
create file path
```
Create a directory:
```
[task]
# create a directory
set path = /home/alex/new/directory
create dir path
```
## File edition
```
[task]
set path = /home/alex/iliad.txt
set var = Hello World
# write data
write var to path
# append data to a file
append var to path
# prepend data to a file
prepend var to path
```
## Read the content of a file
```
[task]
set path = /home/alex/iliad.txt
# read all from 'iliad.txt'
read * from path
: {R}
# read the line at index 3
set index (int) = 1 + 1 + 1
read index from path
: {R}
# just want to read the last line ?
read -1 from path
: {R}
```
## Iterating the content of a file
```
[task]
set path = /home/alex/iliad.txt
# iterate over the characters in a file
for char in path (file)
: Character -> char
# iterate over the lines in a file
for line in path (file)
: {line}
```
## Browse a folder
```
[task]
set folder = /home/alex
browse files and dirs in folder
: Directory -> {R}
for item in files
: {item}
for item in dirs
: {item}
```
## Find resources
The `find` statement is like Glob but on steroid:
```
[task]
set folder = /home/alex
set regex (raw) = `[\S\s]*?`
set timestamp1 = 1223322233
find files in folder matching regex and accessed between timestamp1 and NOW
: Results -> {R}
```
## Read resource properties
You can get from **Backstage** the properties of an arbitrary resource, like its size:
```
[task]
set path = /home/alex/iliad.txt
# poke a file
poke path
: Creation timestamp -> {R.ctime}
: Size -> {R.size}
```
# Interfacing with Python
Interfacing with **Python** is as simple as this:
```
[task]
interface with python.module as my_module
set name = `John Doe`
set age (int) = 40 + 2
call my_module.function(name, age)
: Return -> {R}
```
> **Allowed return data types:** `str`, `list` (one-dimensional),`tuple` (one-dimensional), `dict` (one-dimensional), `int`, and `float`. **Python** functions can also return `True`, `False`, and `None`, which will be converted to **1**, **0** and an **empty string**, respectively.
# Exception handling and tests
Whenever an exception is raised, the variables `EXCEPTION` and `TRACEBACK` are updated and **Backstage** continues calmly its execution.
Note that the variables `TRACEBACK` and `EXCEPTION` are cleared after the next successful command.
If you want the execution to stop whenever an exception is raised, just set `1` to the `FailFast` configuration option.
```
[task]
config FailFast=1
```
If you want to read a report of an exception when it's raised, just set `1` to the `ReportException` configuration option.
```
[task]
config ReportException=1
```
If you want to read the verbose [traceback](https://en.wikipedia.org/wiki/Stack_trace#Python) of an exception when it's raised, just set `1` to the `ShowTraceback` configuration option.
```
[task]
config ShowTraceback=1
```
You can edit these configuration options in the same command:
```
[task]
config FailFast=1 ReportException=1 ShowTraceback=0
```
You can read the current value of an arbitrary configuration option:
```
[task]
config FailFast
: FailFast -> {R}
```
## Debug mode
Instead of manually setting the `ReportException` configuration option to `1`, you can simply run a task in debug mode:
```bash
$ backstage -d task arg
```
## Tests
To create a test, just postfix `.test` to the name of a task. Then from the command line, just run the test `backstage --test task`.
### Example
```
[task]
set val (int) = {ARGS[0]} + {ARGS[1]}
return val
[task.test]
# here we branch the task with the arguments 40 and 2
& task 40 2
# we expect 42 as return
assert R == 42
```
# Command line interface and developer experience
```bash
$ backstage --help
Welcome to Pyrustic Backstage !
Ultimate task automation tool for hackers.
Usage:
backstage
backstage <task> [<arg> ...]
backstage <option> [<arg> ...]
Options:
-i, --intro Show file introductory text
-c, --check Show the list of tasks
-C, --Check Show the descriptive list of tasks
-d, --debug <task> [<arg> ...] Run task in debug mode
-t, --test [<task> ...] Run tests
-T, --Test [<task> ...] Run tests in debug mode
-s, --search <task> Search for a task by its name
-S, --Search <task> Search for a task by keyword
-h, --help [<task>] Show help text
The <task> string can use a glob-like syntax that allows
wildcards '*' and '?'. Therefore, 'task1' is identical to 'task*'.
Visit the webpage: https://github.com/pyrustic/backstage
```
## Developer experience
**Backstage** will do its best to help you understand raised exceptions:
```bash
$ backstage task1
ZeroDivisionError at line 3 of [task1] !
division by zero
$ backstage task2
InterpretationError at line 7 of [task2] !
Usage: sleep <seconds>
```
When you run **Backstage** in the loop mode, you can enjoy the autocomplete functionality (use the Tab key to complete your input) and also the history functionality (use Up and Down arrows).
```bash
$ backstage
Welcome to Pyrustic Backstage !
Ultimate task automation tool for hackers.
Press 'Ctrl-c' or 'Ctrl-d' to quit.
Type '--help' or '-h' to show more information.
(backstage) task(Tab Tab)
task task1 task2 task3
(backstage) --h(Tab)
...
```
# Miscellaneous
In the following subsections, we will explore some miscellaneous information.
## Dogfooding
**Backstage** itself as a project relies on a `backstage.tasks` file (check the root of this repository). You are reading a document about **Backstage** that has been updated with **Backstage** !
## Dependencies
**Backstage** relies on these **Python** packages:
- [Subrun](https://github.com/pyrustic/subrun) to spawn new processes;
- [Shared](https://github.com/pyrustic/shared) to store data;
- [Jesth](https://github.com/pyrustic/jesth) to parse `backstage.tasks` files;
- [Oscan](https://github.com/pyrustic/jesth) to extract tokens from the script.
## Indentation
Four (4) spaces by [indent](https://en.wikipedia.org/wiki/Indentation_(typesetting)#Indentation_in_programming). Period.
## Python 3
Inside the script file, you don't have to type `python3` in a command to spawn the **Python** interpreter. Just type `python` to spawn the same interpreter that is running **Backstage**:
```
[task]
$ python -m my.package.module
```
## Shell
**Backstage** doesn't rely on any **Shell**. But you can still pipe commands !
Example, if you want to change the current working directory:
```
[task]
# instead of doing this
$ cd {HOME}
# use the built-in 'cd'
cd HOME
# you can still spawn programs commonly used in the shell
$ ls
# or make this complex stuff (successfully performed on Ubuntu)
$ python -m this | tail --lines=+3 | sort
```
## Data cache
**Backstage** stores data in an automatically created directory `.backstage` located in the current working directory. Inside this directory you can find the `execution.log` and `database.json` files.
## Automatic line break
If you don't want anymore an extra line break at the end of printed strings, you can turn off this functionality:
```
[task]
# turn off auto line break
config AutoLineBreak=0
: `Hello `
: `World`
# turn on auto line break
config AutoLineBreak=1
: Hello World
```
## Lines
You can draw lines with the characters `=` or `-`. If you pick one, only this one is allowed to appear on the same line.
```
[task1]
$ program1
$ program2
-----------------
[task2]
pass
=================
```
# Demo
The demo is a [repository](https://github.com/pyrustic/project) that contains a [backstage.tasks](https://github.com/pyrustic/project/blob/master/backstage.tasks#L1) file similar to the one used to build, package and publish my projects. Your mission, if you accept it, is to clone the demo repository and run the `backstage.tasks` which contains tasks to create a new **Python** `Hello Friend !` project, build it, perform versioning, init **Git** , perform **Git** Commit and **Git** Push, and even push the latest built package to [PyPI](https://pypi.org) !
```bash
# 1- clone the repository
$ git clone https://github.com/pyrustic/project
$ cd project
# 2- install backstage
$ pip install backstage
# 3- install buildver
$ pip install buildver
# 4- install setupinit
$ pip install setupinit
# 5- list the tasks available in the `backstage.tasks` file
$ backstage -c
Available tasks (11):
build check clean gendoc gitcommit gitinit gitpush init
release test upload2pypi
# 6- descriptive list of tasks
$ backstage -C
# 7- initialize the project
$ backstage init
Successfully initialized !
# 8- run the project
$ python3 -m project
Hello Friend !
# 9- build the project
$ backstage build
building v0.0.1 ...
Successfully built 'project' v0.0.1 !
VERSION file updated from 0.0.1 to 0.0.2
# 10- check the project
$ backstage check
project v0.0.2 (source)
.whl v0.0.1 (package) built 28 secs ago
# 11- initialize Git
$ backstage gitinit
Origin: https://github.com/pyrustic/project.git
# 12- perform a Git Commit
$ backstage gitcommit
# 13- perform a Git Push
$ backstage gitpush
# 14- upload to PyPI
$ backstage upload2pypi
```
> **Note:** Commands `9`, `12`, `13`, and `14` can be replaced with one command: `backstage release`
# Installation
**Backstage** is **cross platform** and versions under **1.0.0** will be considered **Beta** at best. It is built on [Ubuntu](https://ubuntu.com/download/desktop) with [Python 3.8](https://www.python.org/downloads/) and should work on **Python 3.5** or **newer**.
## For the first time
```bash
$ pip install backstage
```
## Upgrade
```bash
$ pip install backstage --upgrade --upgrade-strategy eager
```
<br>
<br>
<br>
[Back to top](#readme)
Raw data
{
"_id": null,
"home_page": "https://github.com/pyrustic/backstage",
"name": "backstage",
"maintainer": "Pyrustic Evangelist",
"docs_url": null,
"requires_python": ">=3.5",
"maintainer_email": "rusticalex@yahoo.com",
"keywords": "scripting,language,script,project-manager,manager,packaging,release,build,project-init,lightweight,pyrustic,tool,CLI,setup,versioning,git,language-agnostic,productivity",
"author": "Pyrustic Evangelist",
"author_email": "rusticalex@yahoo.com",
"download_url": "https://files.pythonhosted.org/packages/d8/50/56b18c657e683aaa9a9a61ee168d4ad0385ba06422fb98f46649ec2ea6cb/backstage-0.0.22.tar.gz",
"platform": null,
"description": "<!-- Cover -->\n<div align=\"center\">\n <img src=\"https://raw.githubusercontent.com/pyrustic/misc/master/assets/backstage/cover.jpg\" alt=\"Demo\" width=\"640\">\n <p align=\"center\">\n By \u00a9 Jorge Royan / <a rel=\"nofollow\" class=\"external free\" href=\"http://www.royan.com.ar\">http://www.royan.com.ar</a>, <a href=\"https://creativecommons.org/licenses/by-sa/3.0\" title=\"Creative Commons Attribution-Share Alike 3.0\">CC BY-SA 3.0</a>, <a href=\"https://commons.wikimedia.org/w/index.php?curid=23405928\">Link</a>\n </p>\n</div>\n\n\n\n<!-- Intro Text -->\n# Pyrustic Backstage\n<b> Three-speed scripting language and task automation tool </b>\n \nThis project is part of the [Pyrustic Open Ecosystem](https://pyrustic.github.io).\n> [Installation](#installation) [Demo](#demo) [Latest](https://github.com/pyrustic/backstage/tags) [Modules](https://github.com/pyrustic/backstage/tree/master/docs/modules#readme)\n\n\n## Table of contents\n- [Overview](#overview)\n- [Structure of the script file](#structure-of-the-script-file)\n- [Spawning processes and branching subtasks](#spawning-processes-and-branching-subtasks)\n- [Data types and control flow](#data-types-and-control-flow)\n- [Namespaces and persistence](#namespaces-and-persistence)\n- [Variable interpolation and escaping](#variable-interpolation-and-escaping)\n- [Environment variables and language syntax](#environment-variables-and-language-syntax)\n- [File and directory manipulation](#file-and-directory-manipulation)\n- [Exception handling and tests](#exception-handling-and-tests)\n- [Interfacing with Python](#interfacing-with-python)\n- [Exception handling and tests](#exception-handling-and-tests)\n- [Command line interface and developer experience](#command-line-interface-and-developer-experience)\n- [Miscellaneous](#miscellaneous)\n- [Demo](#demo)\n- [Installation](#installation)\n\n# Overview\n**Backstage** is a cross-platform **automation tool** that looks for a `backstage.tasks` file in the current working directory to run a specific task defined in that file on demand. A task can be a sequence or pipeline of processes to be spawned, instructions for performing file and directory manipulation, or something more sophisticated.\n\nThe `backstage.tasks` file uses the [Jesth](https://github.com/pyrustic/jesth) (**J**ust **E**xtract **S**ections **T**hen **H**ack) file format that acts like a broken [INI file](https://en.wikipedia.org/wiki/INI_file) parser that only extract sections each made of a header and a body which is just a list of lines.\n\nUsing an eponymous **three-speed scripting language** designed for the automation tool, the programmer can, inside the `backstage.tasks` file, define, coordinate and use the various resources at his disposal to automate things.\n\nThe three-speed scripting language concept is inspired from the three forward gear ratios of early automobiles [transmission system](https://en.wikipedia.org/wiki/Manual_transmission). In the following subsections, we will explore each metaphorical gear of the **Backstage** scripting language, then we will briefly expose the automation tool itself.\n\n## First gear\nIn first gear, a `backstage.tasks` file is intended to store a list of tasks related to a specific project, each task exposing a list of commands or subtasks to be executed. Here, a command represents a process or pipeline of processes to be spawned. [Environment variables](#environment-variables-and-language-syntax) can be used in commands via [variable interpolation](https://en.wikipedia.org/wiki/String_interpolation). No other logic is involved.\n\n### Example\n```\n[task1]\n# three commands to run sequentially\n$ git commit -m 'Update'\n$ python -m my.package.module\n$ program1 arg1 | program2 {HOME}\n---\n# run the subtask 'task2'\n& task2\n---\n# run the subtask 'task3' in a new thread\n~ task3\n\n[task2]\n$ program val1 {CWD}\n$ git push origin master\n\n[task3]\n# some heavy computation\n$ engine -x 5000\n$ engine --cleanup\n\n[_task4]\n# This is a private task (with an underscore as prefix)\n$ clean dir\n```\n\n## Second gear\nIn second gear, the `backstage.tasks` file not only stores the tasks like in first gear, but here logic intervenes, variables are defined, control flow is used, built-in commands are called, et cetera. Basically, in second gear, **Backstage** unleashes its power and allows the programmer to anticipate problems, make sophisticated combinations of subtasks, in short to write a real **script to automate things**.\n\n### Example\n```\n[task1]\n# commit changes\n$ git commit -m 'Update'\n---\n# tell user if 'Commit' has been success\nif R == 0\n # print 'Success !'\n : Success !\nelse\n : Failed to commit changes\n---\n# say hello ten times\nset age = 42\nset name = `John Doe`\nfrom 1 to 10\n $ python -m say.hello {name} {age}\n---\n# create a file in user home\nset pathname = {HOME}/iliad.txt\ncreate file pathname\n---\n# append some data to a file\nset data = `\\nHello World !`\nappend data to pathname\n---\n# browse current working directory\nbrowse files and dirs in CWD\n : Directory -> {R}\n : Files ->\n for item in files\n : - {item}\n : Dirs ->\n for item in dirs\n : - {item}\n```\n\n## Third gear\nIn third gear, in addition with whatever can be done with previous gears, the programmer can directly from the `backstage.tasks` file, call [Python](https://www.python.org/about/) functions with arguments and get the return ! Thanks to this third gear, any too complex or overly verbose calculation can be written in **Python** and called from **Backstage**. This [functionality](#interfacing-with-python) alone proves that **Backstage** is all about making the programmer's life better, not pretending to replace existing mature solutions that actually work.\n\n### Example\n```\n[task1]\ninterface with package.coffeemaker alias coffeebro\n\nset sugar_cubes (int) = 1 + 1 + 1\nset extra = `milk`\n\ncall coffeebro.make(sugar_cubes, extra, 42)\n\nif R == 1\n : Coffee successfully made !\nelse\n : Oops, failed to make coffee...\n : Exception -> {EXCEPTION}\n : Traceback -> {TRACEBACK}\n```\n\n## Automation tool\nThe scripting language help to define tasks in the `backstage.tasks` file that is intended to be consumed by the automation tool. As an automation tool, **Backstage** exposes a [command line interface](#command-line-interface-and-developer-experience) that allows the user to **discover** available tasks, **run** a task with arguments, read a task [documentation](#embedded-documentation-and-tests), use a [glob](https://en.wikipedia.org/wiki/Glob_(programming))-like syntax to **search** for a task by its name or by a keyword (case-insensitive mode) that is part of the documentation of the task, et cetera.\n\n### Example 1\nLet's assume that there is a `backstage.tasks` file in the directory `/home/alex/project`. This `backstage.tasks` file contains three public tasks and one private tasks (prefixed with an underscore).\n\nThis example shows how one could use the automation tool to run a task defined in the `backstage.tasks` file:\n\n```bash\n$ cd /home/alex/project\n\n$ backstage -c\nAvailable tasks (3):\n make_coffee task1 task2\n\n$ backstage make_coffee\nMaking coffee...\n\n$ backstage mak*\nMaking coffee...\n\n$ backstage make_coffee sugar=3\nMaking coffee with 3 sugar cubes...\n```\n\n### Example 2\nThis is the contents of a `backstage.tasks` file located at `/home/alex/project`:\n\n```\n[task]\n# define 'x' as a variable with an 'int' assignment tag\nset x (int) = 1 + 1 + 1\n\n# default name (by default, the assignment tag is 'str')\nset default_name = `John Doe`\n\n# print some text\n: Hi and Welcome !\n: I can print your name {x} times in a row !\n\n# take user input (always a 'str')\n> name : `What is your name ? `\n\n# control flow\nif name == EMPTY\n set name = {default_name}\n\n# iteration\nfrom 1 to x\n : {R} - Ave {name} !\n\n# branch subtask '_task2' and pass it an argument\n& _task2 {name}\n\n\n[_task2]\n# define the variable 'name' (first argument passed to this task)\nset name = {ARGS[0]}\n\n# Just say Goodbye !\n: Goodbye {name} !\n\n```\n\nLet's run the task named `task` from the command line:\n\n```bash\n$ cd /home/alex/project\n\n$ backstage task\nHi and Welcome !\nI can print your name 3 times in a row !\nWhat is your name ? Alex\n1 - Ave Alex !\n2 - Ave Alex !\n3 - Ave Alex !\nGoodbye Alex !\n```\n\n> **Note:** You can reproduce this example as it. Just [install](#installation) **Backstage**, copy-paste the script in a `backstage.tasks` file, then run `backstage task` in the command line.\n\n## And more\nThere is more to talk about **Backstage**, like the ability to embed documentation and tests in the `backstage.tasks` file and access them from the command line. Backstage is enough versatile to do the job of a trivial task runner or to automate things with its scripting language that has a built-in bridge to the powerful Python ecosystem.\n\nIn the following sections, we will explore this project in depth. You can also jump to the [demo](#demo) to start playing with **Backstage** !\n\n# Structure of the script file\nAs stated in the [Overview](#overview) section, a `backstage.tasks` file is basically a [JesthFile](https://github.com/pyrustic/jesth).\n\nIn a `backstage.tasks` file, a section represents a task. The section title is the name of the task and the section body is made of commands to run and the constructs of the **Backstage** scripting language. A valid task name is an alphanumeric string that can contains an underscore.\n\n```\nYou can write here at the top of the script,\na description of the script,\nthe date of its creation,\nor any useful information.\n\n[task1]\n# body of task1\n...\n\n[task2]\n# body of task2\n...\n\n[_private]\n# prefix a task name with an underscore\n# to turn it into a private task that\n# won't appear in the list of available tasks\n# when you will type 'backstage --check' in the command line\n```\n\nAs you can guess, a line starting with `#` is a comment. But this is only true inside a task body, because in fact not all sections are tasks as described before: a section can also be an embedded test or documentation.\n\n\n## Embedded documentation\nYou can embed documentation inside the `backstage.tasks` file. To create a documentation for a task, create a section which name is postfixed with `.help`:\n\n```\n[task1]\npass\n\n\n[task1.help]\nThis is the description line.\n\nUsage:\n backstage task <option> <path>\n\nOptions:\n -m, --msg Show message\n -x, --exit Exit blah blah\n\n```\n\nFrom the **command line**, you can read the documentation of an arbitrary task:\n\n```bash\n$ backstage -h task1\nThis is the description line.\n\nUsage:\n backstage task <option> <path>\n\nOptions:\n -m, --msg Show message\n -x, --exit Exit blah blah\n```\n\n## Embedded tests\n\nYou can embed tests inside the `backstage.tasks` file. To create a test for a task, create a section which name is postfixed with `.test`:\n\n```\n[task1.test]\n# perform some test here\n\n# ...\n\nassert some_var == some_var\n```\n\nFrom the **command line**, you can run the test of an arbitrary task:\n\n```bash\n$ backstage --test task1\n```\n\nor the tests of a bunch of tasks:\n\n```bash\n$ backstage --test task1 task2 task3\n```\n\nor run all tests defined in the `backstage.tasks` file:\n\n```bash\n$ backstage --test\n```\n\n\n# Spawning processes and branching subtasks\nYou can write commands to spawn a process or a pipeline of processes:\n\n```\n[task]\n# spawn Git to perform a 'commit'\n$ git commit -m \"Update\"\n\n# spawn a pipeline of three processes\n$ program1 arg1 | program2 arg2 | program3\n```\n\nCommands to spawn processes support variable interpolation:\n```\n[task]\n# use HOME environment variable\n$ ls {HOME}\n\n# access the first index of the ARGS list\n$ program {ARGS[0]}\n```\n\nYou can **push** data to the **input** of a process:\n\n```\n[task]\n# define variables name and age\nset name = `John Doe`\nset age (int) = 40 + 2\n\n# name and age will be pushed to the input of the next spawned process\npush name age\n$ program1 {HOME}\n\n# from now you can access via the environment variable R,\n# the return code (exit status code)\n: The return code is {R}\n```\n\n\nYou can **capture** a process:\n\n```\n[task]\n# you can capture a process,\n# so you will get a direct access to the output and error\n($) program2\n\n# print the content of OUTPUT and ERROR\n: Output -> {OUTPUT}\n: Error -> {ERROR}\n```\n\n\nYou can redirect **STDOUT** and **STDERR**:\n\n```\n[task]\n\n# redirect STDOUT and STDERR\nset STDOUT = `/path/to/file_out`\nset STDERR = `/path/to/file_err`\n\n$ program1\n\n---\n\n# cross platform DEVNULL:\nset STDOUT = /dev/null\n\n$ program2\n```\n\n## Branching subtasks\nYou can branch a subtask and pass arguments to it:\n\n```\n[task1]\n# branch 'task2'\n& task2 {HOME}\n\n[task2]\n# from task2, we can access arguments passed to it.\n# ARGS (environment variable) is a list of arguments.\n: Arguments -> {ARGS}\n```\n\nWhenever you branch a subtask, a new instance of this subtask is created, with its own variables. You can share data with a subtask with one of these three ways:\n- pass arguments to the subtask while branching it;\n- use the global [namespace](#namespaces-and-persistence);\n- use the database [namespace](#namespaces-and-persistence).\n\nA subtask can also return data that is cached in the `R` environment variable:\n\n```\n[task1]\n& task2 \"John Doe\"\n# from now, R contains 'Hello John Doe !'\n\n[task2]\nset result = `Hello {ARGS} !`\nreturn result\n```\n\n### Multithreading\nBranching a subtask is done in the main thread. But one can create a new thread for a subtask:\n\n```\n[task1]\n# run an instance of task2 in a new thread\n~ task2\n\n# the next command won't wait 'task2' to complete\n$ git commit -m \"Update\"\n\n\n[task2]\n# this task sleeps for 5 seconds\nsleep 5\n```\n\n\n# Data types and control flow\nIn the next subsections we will talk about data types then control flow.\n\n## Data types\n**Backstage** supports [variables](https://en.wikipedia.org/wiki/Variable_(computer_science)) and let the programmer set, use, clear, and drop variables.\n\nUnder the hood, **Backstage** works with five **Python** data types: `str`, `list` (one-dimensional), `dict` (one-dimensional), `int`, and `float`. But these data types aren't intended to be directly used by the programmer. Instead, `assignment tags` are used to tell the interpreter how to treat a variable.\n\nThese `assignment tags` are: `raw`, `str`, `list`, `dict`, `int`, `float`, `date`, `time`, `dtime`, and `tstamp`.\n\n```\n[task]\n# this is a string\nset var = 42\n\n# this is another string\nset var (str) = `Hello World`\n\n# this is a list\nset var (list) = `reg green blue`\n\n# this is the same list but edited\nset var[0] = `yellow`\n# var -> yellow green blue\n\n# this is a dict\nset var (dict) = `user=\"John Doe\" age=42 location=Kernel`\n\n# this is the same dict but edited\nset var.user = `Jane Doe`\n# var -> user='Jane Doe' age=42 location=Kernel\n\n# this is an integer\nset var (int) = 40 + 2\n# var -> 42\n\n# this is a raw string\nset regex (raw) = `[\\S\\s]*?`\n\n# get the current timestamp\nset now (int) = {NOW}\n# now -> 1662569326\n\n# convert it into datetime\nset var (dtime) = {now}\n# var -> 2022-09-07 17:48:46\n\n# go from a datetime to timestamp\nset var (tstamp) = `2022-09-07 17:48:46`\n# var -> 1662569326\n\n# extract the time part of a timestamp\nset var (time) = 1662569326\n# var -> 17:48:46\n```\n\n> Note that all variables have a **string** representation and [backticks](https://en.wikipedia.org/wiki/Backtick) are used optionally as delimiters that will be ignored by the interpreter. So you can put backticks around an integer, and you can insert a list in a string.\n\n### Here document\n**Backstage** supports [here document](https://en.wikipedia.org/wiki/Here_document) for strings inside the `backstage.tasks` file:\n```\n[task]\n# this is a text with two lines\nset text (str) = `First line\\nSecond line`\n\n# this is another text with three lines\nset text = `January\\nFebruary\\nMarch`\n\n# this is not a text with two lines\nset var (raw) = `Hello\\nWorld`\n\n# this is not a text with two lines\nset var (str) = `Hello\\\\nWorld`\n```\n\n### Variable interpolation\n[Variable interpolation](https://en.wikipedia.org/wiki/String_interpolation) is supported with the ability to access from a list or a string, the value at an arbitrary index, or from a dictionary, the value of a key.\n\n```\n[task]\n# let's play with 'str' variables\nset x = `red`\nset var = `{x} green blue`\n# var -> red green blue\n\n# get the value of element at index 0\nset value = {var[0]}\n# value -> r\n\n---\n\n# let's play with a list\nset x = `red`\nset var (list) = `{x} green blue`\n\n# get the value of element at index 0\nset value = {var[0]}\n# value -> red\n\n# get the value of elements from index 1 to the end\nset value = {var[1:]}\n# value -> green blue\n\n---\n\n# let's play with a dict\nset x (int) = 40 + 2\nset var (dict) = `name=\"John Doe\" age={x}`\n\n# get the value of key 'name'\nset value = {var.name}\n# value -> John Doe\n\n# get the value of key 'age'\nset value = {var.age}\n# value -> 42\n\n---\n\n# cancel the variable interpolation\nset var1 = `Hello` \nset var2 = `{{var1}} World`\n# var2 -> {var1} World\n```\n\n## Control flow\n**Backstage** implements conditionals and loops. A wide range of operators are available to compare values.\n### Conditionals\n```\n[task]\nset var1 = 1\nset var2 = 1\nset x = 2\nset regex (raw) = `[\\S\\s]*?`\nset text = `Hello world`\nset y = `Hello`\n\n# conditionals support the classic\n# operators: == != > < >= <= \nif var1 == 1\n $ program1\nelif var2 == x\n $ program2\nelse\n $ program3\n \n# Backstage supports logical and/or \nif var1 == var2 and var1 >= 3\n $ program1\n\n# use the 'matches' operator\n# or the negated one: !matches\nif regex matches text\n : Matched !\nelif regex !matches text\n : Mismatched !\n \n# you can use the in operator\n# and also the negated one: !in\nif y in text\n pass\nelif y !in text\n pass\n\n```\n\n\n### Loops\n```\n[task]\n\n# From To loop\nfrom 10 to 0\n : {R}\n \n# For loop - iterate over a string\nset text = `Hello World`\nfor char in text\n # N is an environment variable that serves as counter\n # for all loops\n : {N}- {char}\n\n# For loop - iterate over a list\nset data (list) = `red green blue`\nfor item in data\n : {item}\n \n# For loop - iterate over a dict\nset data (dict) = `name=\"John Doe\" age=42`\nfor item in data\n : key -> {item[0]} value -> {item[1]}\n \n# For loop - iterate over a file\nset path = `/home/alex/iliad.txt`\nfor line in path (file)\n : Line {N}\n : {line}\n :\n \n# while loop\nset var = 1\nwhile var == 1\n : One Time Hello\n break\n \n# browse loop\nset path = `/home/alex`\nbrowse files and dirs in path\n : Directory -> {R}\n for item in files\n : {item}\n for item in dirs\n : {item} \n```\n\n\n\n# Namespaces and persistence\n[Namespaces](https://en.wikipedia.org/wiki/Namespace) are implemented in **Backstage** to provide convenient management of variables by defining three [scopes](https://en.wikipedia.org/wiki/Scope_(computer_science)): \n- `L` for **Local** scope;\n- `G` for **Global** scope;\n- `D` for **Database** scope.\n\nBy default, variables exist in the **Local** namespace and are only accessible to the running task.\n\nTo share data with a subtask, one can expose arbitrary variables that will be copied into the **Global** namespace which is readable and writable (thread-safe) by all subtasks.\n\n```\n[task]\n# by default, variables are defined in Local,\n# i.e. they are only visible in the scope of the current task\nset var = 42\n: Var contains {var}\n: Var still contains {L:var}\n\n# you can make local variables public\nexpose var\n# from now, you can get a thread-safe access to var \n# from any running task:\n: Global var contains {G:var}\n\n# branch _task2\n& _task2\n\n[_task2]\n: I got {G:var} !\n\n```\n\nData can also be **persisted**:\n\n```\n[task]\nset var = 42\nstore var\n\n# from now, 'var' can be accessed by all tasks\n# in this runtime but also in future runtimes\n: Var contains {D:var}\n\n# if you aren't sure about the existence\n# of a variable, just do this:\ndefault var\n\n# it also works with a bunch of variables:\ndefault var1 L:var2 D:var3 G:var4\n\n# from now, if 'var1' hasn't been manually defined\n# by the programmer, it will be\n# automatically initialized and\n# its value will be an empty string\n```\n\nPersisted variables are stored in `.backstage/database.json`.\n\n\n# Variable interpolation and escaping\nDuring the string interpolation of a command that spawn processes or branch a subtask, variables that are of the `str` type are automatically [shell-escaped](https://en.wikipedia.org/wiki/Escape_character).\n\n```\n[task]\n# This is a 'str' variable (backticks aren't quotes, by the way!)\nset name = `John Doe` \nset colors (list) = `red green blue`\n\n: Welcome {name} !\n# Welcome John Doe\n\n$ program name={name} -c {colors}\n# program name='John Doe' -c red green blue\n\n# Notice the quotes automatically added around the name John Doe\n```\n\n# Environment variables and language syntax\nEnvironment variables are local to each instance of task. They are defined as uppercase strings. One can edit their contents but can't create new environment variables.\n\nThis is the exhaustive list of environment variables:\n\n|Variables|Description|\n|---|---|\n|`ARGS`|List of arguments passed to this task from the command line|\n|`CWD`|Current working directory|\n|`DATE`|The current date in the **YYYY-MM-DD** format|\n|`EMPTY`|Just an empty string|\n|`ERROR`|Error string from a process previously spawned|\n|`EXCEPTION`|Name of the last exception raised|\n|`FALSE`|The integer **0**|\n|`HOME`|The path to `$HOME`. Example: `/home/alex`|\n|`LINE`|The current line (1-based numbering) of execution in the task body|\n|`N`|Counter for `while`, `for`, `from`, and `browse` loops|\n|`NOW`|Current timestamp in seconds|\n|`ONE`|The integer **1**|\n|`OS`|The running operating system: `aix`, `linux`, `win32`, `cygwin`, `darwin`|\n|`OUTPUT`|Output string from a process previously spawned|\n|`R`|The **return** of Python functions, built-in commands, statements, constructs, or process exit status codes|\n|`RANDOM`|Random integer between **0** and **255** (closed interval)|\n|`SPACE`|One space ` ` character|\n|`STDERR`|Use this variable to perform **STDERR** redirection|\n|`STDIN`|Use this variable to perform **STDIN** redirection|\n|`STDOUT`|Use this variable to perform **STDOUT** redirection|\n|`TASK`|The name of the currently running task|\n|`TIME`|The current time in the **HH:MM:SS** format|\n|`TIMEOUT`|Timeout in seconds for commands that spawn processes. Default value: **30** seconds|\n|`TMP`|Temporary directory. **Attention**, this directory will automatically disappear at the end of the runtime ! So think twice before moving files inside|\n|`TRACEBACK`|[Traceback](https://en.wikipedia.org/wiki/Stack_trace#Python) of the last exception raised|\n|`TRASH`|Path to the trash: `$HOME/PyrusticData/trash`|\n|`TRUE`|The integer **1**|\n|`ZERO`|The integer **0**|\n\n> Note that the `TRACEBACK` and `EXCEPTION` variables are cleared after the next successful command. **Backstage** also generates for convenience, `ARG0`, `ARG1`, `ARGx`, according to the contents of `ARGS`. For example, if `ARGS`contains two arguments, you can expect that `ARG0` and `ARG1` exist.\n\n## Language syntax\nIn this section we will explore the built-in commands, statements, keywords, symbols, and language constructs that make **Backstage**.\n\n> Note that wherever a built-in command or statement expects a **variable** that will be **read**, for convenience you can instead of supplying a variable name, define an **inline** `int` or `float` literal.\n\n> Also, consider that the `R` environment variable is your friend, since it is used to cache the data returned by a statement, a construct, or a command.\n\n### APPEND\nAppend data to a file.\n\n**Usage:** `append <var> to <filename_var>`\n\n\n### ASSERT\nTest is a condition is true.\n\n**Usage:** `assert (<var1>|<regex_var>) (==|!=|<=|>=|<|>|in|!in|rin|!rin|matches|!matches) <var2> [and|or] ...`\n\n**Example:** `assert regex_var matches text_var and var1 in list`\n\nNote that `!` is used to express negation and `rin` is a Regex-based `in`. A regexly-in operator ;)\n\n\n### BRANCH\nBranch a subtask. The syntax is similar to the one to spawn processes, i.e., a string of words. The syntax supports variable interpolation.\n\n**Usage:** `& <subtask> [<argument> ...]`\n\n**Example:** `& subtask1 name=\"John Doe\" age=42 city={city}`\n\n> In this example, the `city` variable will be automatically shell-escaped during its interpolation.\n\n\n### BREAK\nBreak a loop.\n\n**Usage:** `break`\n\n### BROWSE\nLoop construct to browse a directory.\n\n**Usage:** `browse [files] [and] [dirs] in <dirname_var>`\n\n**Example:**\n```\n[task]\nbrowse files and dirs in dirname\n : Root -> {R}\n for item in files\n : {item}\n for item in dirs\n : {item}\n \nbrowse files in dirname\n pass\n \nbrowse dirs in dirname\n pass\n```\n\n### CALL\nCall a **Python** function from **Backstage** with arguments, then get the return !\n\n**Usage:** `call <module>.<function>[(<argument_var>, ...)]`\n\n**Example:** \n```\n[task]\n# interface with the Python module\ninterface with package.coffee_module alias coffeemaker\n\n# call the 'make' function with arguments then get the return\ncall coffeemaker.make(sugar_cube, extra, 42)\n: Result -> {R}\n```\n\n### CD\nChange directory.\n\n**Usage:** `cd <dirname_var>`\n\n\n### CHECK\nReturn the data type (`str`, `list`, `dict`, `int`, `float`) of a variable if it exists, else return an empty string.\n\n**Usage:** `check <var>`\n\n**Example:**\n```\n[task]\ncheck myvar\nif R == EMPTY\n : This variable doesn't exist at all !\nelse\n : 'myvar' exists, its data type is {R}\n```\n\n### CLEAR\nClear the content of a variable or a list of variables.\n\n**Usage:** `clear <var> ...`\n\n**Example:** `clear var1 var2 var3`\n\n### COMMENT\nComment.\n\n**Usage:** `# <comment>`\n\n\n### CONFIG\nRead and write configuration options (`FailFast`, `ReportException`, `ShowTraceback`, `TestMode`, `AutoLineBreak`).\n\n**Usage:** `config <option> ...`\n\n**Example:**\n```\n[task]\nconfig FailFast=1 AutoLineBreak=0\nconfig TestMode\nif R == 1\n : Test Mode On\nelif R == 0\n : Test Mode Off\n```\n\n\n### COPY\nCopy a file or a directory tree to a new destination.\n\n**Usage:** `copy <src_path_var> to <dest_path_var>`\n\n### COUNT\nCount `chars`, `items`, and `lines` in the content of a variable or inside a file (if the `(file)` tag is applied).\n\n**Usage:** `count (chars|items|lines) in (<var>|<filename_var>) [(file)]`\n\n**Example:**\n```\n[task]\nset path = /home/alex/iliad.txt\ncount chars in path (file)\nif R == 0\n : The file is empty !\n```\n\n### CREATE\nCreate a new file or directory.\n\n**Usage:** `create (dir|file) <path_var>`\n\n### DEFAULT\nDefine an empty variable (or a bunch of variables) if it doesn't exist yet in the namespace.\n\n**Usage:** `default <var> ...`\n\n**Example:**\n\n```\n[task]\n# default two variables in the Local namespace\ndefault var1 L:var2\n\n# default one variable in the Database namespace\ndefault D:name\n\n# from now, 'D:name' can be safely accessed\n# even though the 'database.json' file supposed\n# to contain the 'name' value was inadvertently deleted.\n```\n\n### DROP\nDestroy a variable (or a bunch of variables). \n\n**Usage:** `drop <var> ...`\n\n### ELIF\nPart of the `if` conditional construct.\n\n**Usage:** `elif (<var1>|<regex_var>) (==|!=|<=|>=|<|>|in|!in|rin|!rin|matches|!matches) <var2> [and|or] ...`\n\n### ELSE\nPart of the `if` conditional construct.\n\n**Usage:** `else`\n\n### ENTER\nInvite user to submit data.\n\n**Usage:** `> [<var> [: <text>]]`\n\n**Example:**\n```\n[task]\n\n> name : Please enter your name \n# have you spotted the space at the end the line above ?\n\n# the same line can be rewritten like this:\n> name : `Please enter your name `\n# backticks serve as delimiters that will be ignored\n\n# this one is also possible:\nset msg = `Please enter your name `\n> name : {msg}\n\n# even this:\nset info (dict) = name=\"John Doe\" age=42\n> info.name : `Please enter your name`\n```\n\n### EXIT\nExit.\n\n**Usage:** `exit`\n\n### EXPOSE\nCopy a variable (or a bunch of variables) into the **Global** namespace.\n\n**Usage:** `expose <var> ...`\n\n### FAIL\nDeliberately fail. It breaks the running task and mark it as a failure.\n\n**Usage:** `fail`\n\n### FIND\nFind files and or directories paths.\n\n**Usage 1:** `find [all] (paths|files|dirs) in <dirname_var>`\n\n**Usage 2:** `find ... matching <regex_var>`\n\n**Usage 3:** `find ... [and] (accessed|modified|created) (at|after|before|between) <timestamp_var> [and <timestamp_var>]`\n\n**Example:** `find files in dirname matching regex and accessed between timestamp1 and timestamp2`\n\n### FOR\nA `for` loop to iterate the content of a variable or the content of a file (if you apply the `(file)` tag).\n\n**Usage:** `for (char|item|line) in (<var>|<filename_var>) [(file)]`\n\n**Example:**\n```\n[task]\n# this code iterates over each character of the Iliad,\n# and outputs it as it,\n# with one twist: each line starts with its index (0-based)\n\nset path = `/home/alex/iliad.txt`\n\n# the print statement (:) won't anymore\n# automatically add a line break !\nconfig AutoLineBreak=0\n\nfor line in path (file)\n : `{N} `\n for char in line\n : {char}\n : \\n\n```\n\n\n### FROM\nA loop to go from an integer to another one. If the `start` integer is superior to the `end` integer, the count will decrease.\n\n**Usage:** `from <start> to <end>`\n\n**Example:**\n```\n[task]\nfrom 10 to 0\n # here, N will go from 0 to 10\n # but R will go from 10 to 0\n # because N is a counter for all loops\n # while R is a cache for whatever is returned\n # by a command, a statement, or a construct\n : {N}\\t{R}\n\n# As you can see, I can add a Tab \\t since\n# Backstage supports natively here document ;)\n\n# To get a simple backslash followed by a 't': \\\\t\n```\n\n### GET\nGet a `char`, an `item`, or a `line` at index `x` (including negative index) from a target. The target can be the content of a variable or a file (if you apply the `(file)` tag).\n\n**Usage:** `get (char|item|line) <index> from (<var>|<filename_var>) [(file)]`\n\n### IF\nConditional construct.\n\n**Usage:** `if (<var1>|<regex_var>) (==|!=|<=|>=|<|>|in|!in|rin|!rin|matches|!matches) <var2> [and|or] ...`\n\n**Example:**\n```\n[task]\ndefault var1 var2 var3 var4\nif var1 == var2 or var3 == var4\n pass\nelif EMPTY == EMPTY\n pass\nelse\n pass\n```\n\n### INTERFACE\nInterface with a **Python** module.\n\n**Usage:** `interface with [<package>.]<module> [alias <name>]`\n\n**Example:** \n\n```\n[task]\ninterface with package.mymodule alias module\ndefault var1 var2\ncall module.function(var1, var2)\n```\n\n### LINE\nDraw a line.\n\n**Usage:** `(=|-) ...`\n\n**Example:** `----------` or `==========`\n\n\n### MOVE\nMove a file or a directory tree to a new destination.\n\n**Usage:** `move <src_path_var> to <dest_path_var>`\n\n### PASS\nPlaceholder for the code that you might write in the future. This statement does nothing. It is the same as the eponymous one in **Python**.\n\n**Usage:** `pass`\n\n### POKE\nPoke a file or directory to get access to a `dict` of properties if this path exists. Available properties: `size` `mtime` `ctime` `atime` `nlink` `uid` `gid` `mode` `ino` `dev`.\n\n**Usage:** `poke <path_var>`\n\n**Example:**\n```\n[task]\nset path = /home/alex/iliad.txt\npoke path\nif R == EMPTY\n : Oops ! This file doesn't exist\nelse\n : File size -> {R.size}\n```\n\n### PREPEND\nPrepend data to a file\n\n**Usage:** `prepend <var> to <filename_var>`\n\n### PRINT\nPrint data. You can use backquotes as delimiters. This statement supports variable interpolation.\n\n**Usage:** `: <text>`\n\n**Example:**\n```\n[task]\n: Hello World ! \n# Have you spotted the two extra spaces characters ?\n\n: ` Hello World ! `\n# hehehe, got you ! ;)\n```\n\n### PUSH\nPush variables into the input of the next spawned process.\n\n**Usage:** `push <var> ...`\n\n### READ\nRead all or a specific line index (including negative index) from a file.\n\n**Usage:** `read (*|<index>) from <filename_var>`\n\n### REPLACE\nReplace some pattern in a text with a replacement value.\n\n**Usage:** `replace <regex_var> in <text_var> with <replacement_var>`\n\n### RETURN\nReturn from a task with a value.\n \n**Usage:** `return [<var>]`\n\n### SET\nDefine a new variable or update the content of an existing variable. You don't can't specify a data type, but instead you can apply an assigment tag that is one of: `(raw)` `(str)` `(list)` `(dict)` `(int)` `(float)` `(date)` `(time)` `(dtime)` `(tstamp)`. Note that backquotes can be used as delimiters for the value (right side of the equal sign). These delimiters will be ignored. Backticks aren't quotes. This statement supports variable interpolation.\n\n**Usage:** `set <var> [(raw)|(str)|(list)|(dict)|(int)|(float)|(date)|(time)|(dtime)|(tstamp))] = <value>`\n\n**Example:** `set var (int) = 1 + 2`\n\n\n### SLEEP\nSleep for `x` seconds.\n\n**Usage:** `sleep <seconds>`\n\n### SPAWN\nSpawn a new process.\n\n**Usage:** `$ <program> [<argument> ...]`\n\n**Example:** `$ program1 arg {var} | program2 `\n\n### SPLIT\nSplit with a regex pattern a text into a list.\n\n**Usage:** `split <text_var> with <regex_var>`\n\n### SPOT\nCount the number of occurrences of a regex pattern inside a text.\n\n**Usage:** `spot <regex_var> in <text_var>`\n\n### STORE\nStore a variable (or a bunch of variables) in the **Database** namespace. A stored variable can be accessed like this: `D:var`\n\n**Usage:** `store <var> ...`\n\n### THREAD\nBranch a subtask... but in a new thread. \n\n**Usage:** `~ <subtask> [<argument> ...]`\n\n### WHILE\nThe `while` loop. Use `break` to break it, and check `N` if you need a counter. \n\n**Usage:** `while (<var1>|<regex_var>) (==|!=|<=|>=|<|>|in|!in|rin|!rin|matches|!matches) <var2> [and|or] ...`\n\n### WRITE\nErase the content of a file to write some data inside.\n\n**Usage:** `write <var> to <filename_var>`\n\n\n# File and directory manipulation\nLet's explore how file and directory manipulatin is performed with **Backstage**.\n\n## Resource creation\n\nCreate a file:\n```\n[task]\n# create a file\nset path = /home/alex/iliad.txt\ncreate file path\n```\n\nCreate a directory:\n\n```\n[task]\n# create a directory\nset path = /home/alex/new/directory\ncreate dir path\n```\n\n## File edition\n\n```\n[task]\nset path = /home/alex/iliad.txt\nset var = Hello World\n\n# write data\nwrite var to path\n\n# append data to a file\nappend var to path\n\n# prepend data to a file\nprepend var to path\n```\n\n## Read the content of a file\n```\n[task]\nset path = /home/alex/iliad.txt\n\n# read all from 'iliad.txt'\nread * from path\n: {R}\n\n# read the line at index 3\nset index (int) = 1 + 1 + 1\nread index from path\n: {R}\n\n# just want to read the last line ?\nread -1 from path \n: {R}\n```\n\n## Iterating the content of a file\n```\n[task]\nset path = /home/alex/iliad.txt\n\n# iterate over the characters in a file\nfor char in path (file)\n : Character -> char\n\n# iterate over the lines in a file\nfor line in path (file)\n : {line}\n```\n\n## Browse a folder\n\n```\n[task]\nset folder = /home/alex\n\nbrowse files and dirs in folder\n : Directory -> {R}\n for item in files\n : {item}\n for item in dirs\n : {item}\n```\n\n## Find resources\nThe `find` statement is like Glob but on steroid:\n```\n[task]\nset folder = /home/alex\nset regex (raw) = `[\\S\\s]*?`\nset timestamp1 = 1223322233\n\nfind files in folder matching regex and accessed between timestamp1 and NOW\n: Results -> {R}\n\n```\n\n## Read resource properties\nYou can get from **Backstage** the properties of an arbitrary resource, like its size:\n\n```\n[task]\nset path = /home/alex/iliad.txt\n\n# poke a file\npoke path\n: Creation timestamp -> {R.ctime}\n: Size -> {R.size}\n\n```\n\n\n# Interfacing with Python\nInterfacing with **Python** is as simple as this:\n```\n[task]\ninterface with python.module as my_module\nset name = `John Doe`\nset age (int) = 40 + 2\ncall my_module.function(name, age)\n: Return -> {R}\n```\n\n> **Allowed return data types:** `str`, `list` (one-dimensional),`tuple` (one-dimensional), `dict` (one-dimensional), `int`, and `float`. **Python** functions can also return `True`, `False`, and `None`, which will be converted to **1**, **0** and an **empty string**, respectively.\n\n# Exception handling and tests\nWhenever an exception is raised, the variables `EXCEPTION` and `TRACEBACK` are updated and **Backstage** continues calmly its execution.\n\nNote that the variables `TRACEBACK` and `EXCEPTION` are cleared after the next successful command.\n\nIf you want the execution to stop whenever an exception is raised, just set `1` to the `FailFast` configuration option.\n\n```\n[task]\nconfig FailFast=1\n```\n\nIf you want to read a report of an exception when it's raised, just set `1` to the `ReportException` configuration option.\n\n```\n[task]\nconfig ReportException=1\n```\n\nIf you want to read the verbose [traceback](https://en.wikipedia.org/wiki/Stack_trace#Python) of an exception when it's raised, just set `1` to the `ShowTraceback` configuration option.\n\n```\n[task]\nconfig ShowTraceback=1\n```\n\nYou can edit these configuration options in the same command:\n\n```\n[task]\nconfig FailFast=1 ReportException=1 ShowTraceback=0\n```\nYou can read the current value of an arbitrary configuration option:\n\n```\n[task]\nconfig FailFast\n: FailFast -> {R}\n```\n\n## Debug mode\nInstead of manually setting the `ReportException` configuration option to `1`, you can simply run a task in debug mode:\n\n```bash\n$ backstage -d task arg\n```\n\n## Tests\nTo create a test, just postfix `.test` to the name of a task. Then from the command line, just run the test `backstage --test task`.\n\n### Example\n```\n[task]\nset val (int) = {ARGS[0]} + {ARGS[1]}\nreturn val\n\n[task.test]\n# here we branch the task with the arguments 40 and 2\n& task 40 2\n# we expect 42 as return \nassert R == 42\n```\n\n# Command line interface and developer experience\n\n```bash\n$ backstage --help\nWelcome to Pyrustic Backstage !\nUltimate task automation tool for hackers.\n\nUsage:\n backstage\n backstage <task> [<arg> ...]\n backstage <option> [<arg> ...]\n \nOptions:\n -i, --intro Show file introductory text\n -c, --check Show the list of tasks\n -C, --Check Show the descriptive list of tasks\n -d, --debug <task> [<arg> ...] Run task in debug mode\n -t, --test [<task> ...] Run tests\n -T, --Test [<task> ...] Run tests in debug mode\n -s, --search <task> Search for a task by its name\n -S, --Search <task> Search for a task by keyword\n -h, --help [<task>] Show help text\n\n The <task> string can use a glob-like syntax that allows \n wildcards '*' and '?'. Therefore, 'task1' is identical to 'task*'.\n \nVisit the webpage: https://github.com/pyrustic/backstage\n```\n\n## Developer experience\n**Backstage** will do its best to help you understand raised exceptions:\n\n```bash\n$ backstage task1\nZeroDivisionError at line 3 of [task1] !\ndivision by zero\n\n$ backstage task2\nInterpretationError at line 7 of [task2] !\nUsage: sleep <seconds>\n```\n\nWhen you run **Backstage** in the loop mode, you can enjoy the autocomplete functionality (use the Tab key to complete your input) and also the history functionality (use Up and Down arrows).\n\n```bash\n$ backstage\nWelcome to Pyrustic Backstage !\nUltimate task automation tool for hackers.\nPress 'Ctrl-c' or 'Ctrl-d' to quit.\nType '--help' or '-h' to show more information.\n\n(backstage) task(Tab Tab)\ntask task1 task2 task3\n\n(backstage) --h(Tab)\n\n...\n\n```\n\n# Miscellaneous\nIn the following subsections, we will explore some miscellaneous information.\n\n## Dogfooding\n**Backstage** itself as a project relies on a `backstage.tasks` file (check the root of this repository). You are reading a document about **Backstage** that has been updated with **Backstage** !\n\n## Dependencies\n**Backstage** relies on these **Python** packages:\n- [Subrun](https://github.com/pyrustic/subrun) to spawn new processes;\n- [Shared](https://github.com/pyrustic/shared) to store data;\n- [Jesth](https://github.com/pyrustic/jesth) to parse `backstage.tasks` files;\n- [Oscan](https://github.com/pyrustic/jesth) to extract tokens from the script.\n\n## Indentation\nFour (4) spaces by [indent](https://en.wikipedia.org/wiki/Indentation_(typesetting)#Indentation_in_programming). Period.\n\n## Python 3\nInside the script file, you don't have to type `python3` in a command to spawn the **Python** interpreter. Just type `python` to spawn the same interpreter that is running **Backstage**:\n```\n[task]\n$ python -m my.package.module\n```\n\n## Shell\n**Backstage** doesn't rely on any **Shell**. But you can still pipe commands !\n\nExample, if you want to change the current working directory:\n```\n[task]\n# instead of doing this\n$ cd {HOME}\n\n# use the built-in 'cd'\ncd HOME\n\n# you can still spawn programs commonly used in the shell\n$ ls\n\n# or make this complex stuff (successfully performed on Ubuntu)\n$ python -m this | tail --lines=+3 | sort\n```\n\n## Data cache\n**Backstage** stores data in an automatically created directory `.backstage` located in the current working directory. Inside this directory you can find the `execution.log` and `database.json` files.\n\n## Automatic line break\nIf you don't want anymore an extra line break at the end of printed strings, you can turn off this functionality:\n\n```\n[task]\n# turn off auto line break\nconfig AutoLineBreak=0\n: `Hello `\n: `World`\n# turn on auto line break\nconfig AutoLineBreak=1\n: Hello World\n```\n\n\n## Lines\nYou can draw lines with the characters `=` or `-`. If you pick one, only this one is allowed to appear on the same line.\n```\n[task1]\n\n$ program1\n$ program2\n\n-----------------\n\n[task2]\npass\n\n=================\n```\n\n\n# Demo\nThe demo is a [repository](https://github.com/pyrustic/project) that contains a [backstage.tasks](https://github.com/pyrustic/project/blob/master/backstage.tasks#L1) file similar to the one used to build, package and publish my projects. Your mission, if you accept it, is to clone the demo repository and run the `backstage.tasks` which contains tasks to create a new **Python** `Hello Friend !` project, build it, perform versioning, init **Git** , perform **Git** Commit and **Git** Push, and even push the latest built package to [PyPI](https://pypi.org) !\n\n```bash\n# 1- clone the repository\n$ git clone https://github.com/pyrustic/project\n$ cd project\n\n# 2- install backstage\n$ pip install backstage\n\n# 3- install buildver\n$ pip install buildver\n\n# 4- install setupinit\n$ pip install setupinit\n\n# 5- list the tasks available in the `backstage.tasks` file\n$ backstage -c\nAvailable tasks (11):\n build check clean gendoc gitcommit gitinit gitpush init\n release test upload2pypi\n\n# 6- descriptive list of tasks\n$ backstage -C\n\n# 7- initialize the project\n$ backstage init\nSuccessfully initialized !\n\n# 8- run the project\n$ python3 -m project\nHello Friend !\n\n# 9- build the project\n$ backstage build\nbuilding v0.0.1 ...\nSuccessfully built 'project' v0.0.1 !\nVERSION file updated from 0.0.1 to 0.0.2\n\n# 10- check the project\n$ backstage check\nproject v0.0.2 (source)\n.whl v0.0.1 (package) built 28 secs ago\n\n# 11- initialize Git\n$ backstage gitinit\nOrigin: https://github.com/pyrustic/project.git\n\n# 12- perform a Git Commit\n$ backstage gitcommit\n\n# 13- perform a Git Push\n$ backstage gitpush\n\n# 14- upload to PyPI\n$ backstage upload2pypi\n```\n\n> **Note:** Commands `9`, `12`, `13`, and `14` can be replaced with one command: `backstage release`\n\n\n\n# Installation\n**Backstage** is **cross platform** and versions under **1.0.0** will be considered **Beta** at best. It is built on [Ubuntu](https://ubuntu.com/download/desktop) with [Python 3.8](https://www.python.org/downloads/) and should work on **Python 3.5** or **newer**.\n\n## For the first time\n\n```bash\n$ pip install backstage\n```\n\n## Upgrade\n```bash\n$ pip install backstage --upgrade --upgrade-strategy eager\n\n```\n\n<br>\n<br>\n<br>\n\n[Back to top](#readme)\n\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Three-speed scripting language and task automation tool",
"version": "0.0.22",
"project_urls": {
"Homepage": "https://github.com/pyrustic/backstage"
},
"split_keywords": [
"scripting",
"language",
"script",
"project-manager",
"manager",
"packaging",
"release",
"build",
"project-init",
"lightweight",
"pyrustic",
"tool",
"cli",
"setup",
"versioning",
"git",
"language-agnostic",
"productivity"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7a797075eeb86fd652af1b2508e8c5df673497b78de1ee864f0353223b8da2a9",
"md5": "6e052fb671e123e1408af8b9e2feac9a",
"sha256": "74611102dd69e9db76e9ad6fb34c28f0133ff6840147f58d4479711133a5bca3"
},
"downloads": -1,
"filename": "backstage-0.0.22-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6e052fb671e123e1408af8b9e2feac9a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.5",
"size": 100508,
"upload_time": "2023-06-09T10:10:13",
"upload_time_iso_8601": "2023-06-09T10:10:13.569931Z",
"url": "https://files.pythonhosted.org/packages/7a/79/7075eeb86fd652af1b2508e8c5df673497b78de1ee864f0353223b8da2a9/backstage-0.0.22-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d85056b18c657e683aaa9a9a61ee168d4ad0385ba06422fb98f46649ec2ea6cb",
"md5": "bd6e37743cda11085a8341718c264991",
"sha256": "1742d946f199cb2ac0bbb8c7fc2f3810570b54cfbb38338812cc9d329ed54e5b"
},
"downloads": -1,
"filename": "backstage-0.0.22.tar.gz",
"has_sig": false,
"md5_digest": "bd6e37743cda11085a8341718c264991",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.5",
"size": 66691,
"upload_time": "2023-06-09T10:10:16",
"upload_time_iso_8601": "2023-06-09T10:10:16.525272Z",
"url": "https://files.pythonhosted.org/packages/d8/50/56b18c657e683aaa9a9a61ee168d4ad0385ba06422fb98f46649ec2ea6cb/backstage-0.0.22.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-09 10:10:16",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "pyrustic",
"github_project": "backstage",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "backstage"
}
JSON
