Prysk
======================
.. image:: https://img.shields.io/github/actions/workflow/status/nicoretti/prysk/ci.yaml
:target: https://github.com/Nicoretti/prysk/actions
.. image:: https://img.shields.io/coverallsCoverage/github/Nicoretti/prysk
:target: https://coveralls.io/github/Nicoretti/prysk
.. image:: https://img.shields.io/badge/imports-isort-ef8336.svg
:target: https://pycqa.github.io/isort/
.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
:target: https://github.com/psf/black
.. image:: https://img.shields.io/pypi/v/prysk
:target: https://pypi.org/project/prysk/
.. image:: https://img.shields.io/badge/docs-available-blue.svg
:target: https://nicoretti.github.io/prysk/
Prysk is a fork of the popular snapshot testing tool Cram_.
Even though Cram_ is pretty complete and mature for everyday use,
Prysk wants to continue pushing its development forward.
.. _Cram: https://bitheap.org/cram
Prysk tests look like snippets of interactive shell sessions. Prysk runs
each command and compares the command output in the test with the
command's actual output.
Here's a snippet from `Prysk's own test suite`_:
.. code-block:: console
Set up prysk alias and example tests:
$ . "$TESTDIR"/setup.sh
Usage:
$ prysk -h
[Uu]sage: prysk \[OPTIONS\] TESTS\.\.\. (re)
[Oo]ptions: (re)
-h, --help show this help message and exit
-V, --version show version information and exit
-q, --quiet don't print diffs
-v, --verbose show filenames and test status
-i, --interactive interactively merge changed test output
-d, --debug write script output directly to the terminal
-y, --yes answer yes to all questions
-n, --no answer no to all questions
-E, --preserve-env don't reset common environment variables
--keep-tmpdir keep temporary directories
--shell=PATH shell to use for running tests (default: /bin/sh)
--shell-opts=OPTS arguments to invoke shell with
--indent=NUM number of spaces to use for indentation (default: 2)
--xunit-file=PATH path to write xUnit XML output
--dos2unix convert DOS/Windows line endings to UNIX line endings
--color mode which shall be used for coloring the output
The format in a nutshell:
* Prysk tests use the ``.t`` file extension.
* Lines beginning with two spaces, a dollar sign (``$``), and a space are run
in the shell.
* Lines beginning with two spaces, a greater than sign (``>``), and a space
allow multi-line commands.
* All other lines beginning with two spaces are considered command
output.
* Output lines ending with a space and the keyword ``(re)`` are
matched as `Perl-compatible regular expressions`_.
* Lines ending with a space and the keyword ``(glob)`` are matched
with a glob-like syntax. The only special characters supported are
``*`` and ``?``. Both characters can be escaped using ``\``, and the
backslash can be escaped itself.
* Output lines ending with either of the above keywords are always
first matched literally with actual command output.
* Lines ending with a space and the keyword ``(no-eol)`` will match
actual output that doesn't end in a newline.
* Actual output lines containing unprintable characters are escaped
and suffixed with a space and the keyword ``(esc)``. Lines matching
unprintable output must also contain the keyword.
* Anything else is a comment.
.. _Prysk's own test suite: https://github.com/Nicoretti/prysk/blob/master/test/integration/prysk/usage.t
.. _Perl-compatible regular expressions: https://en.wikipedia.org/wiki/Perl_Compatible_Regular_Expressions
Usage
-----
Prysk will print a dot for each passing test. If a test fails, a
`unified context diff`_ is printed showing the test's expected output
and the actual output. Skipped tests (empty tests and tests that exit
with return code ``80``) are marked with ``s`` instead of a dot.
For example, if we run Prysk on `its own example tests`_:
.. code-block:: diff
.s.!
--- examples/fail.t
+++ examples/fail.t.err
@@ -3,21 +3,22 @@
$ echo 1
1
$ echo 1
- 2
+ 1
$ echo 1
1
Invalid regex:
$ echo 1
- +++ (re)
+ 1
Offset regular expression:
$ printf 'foo\nbar\nbaz\n\n1\nA\n@\n'
foo
+ bar
baz
\d (re)
[A-Z] (re)
- #
+ @
s.
# Ran 6 tests, 2 skipped, 1 failed.
Prysk will also write the test with its actual output to
``examples/fail.t.err``, allowing you to use other diff tools. This
file is automatically removed the next time the test passes.
When you're first writing a test, you might just write the commands
and run the test to see what happens. If you run Prysk with ``-i`` or
``--interactive``, you'll be prompted to merge the actual output back
into the test. This makes it easy to quickly prototype new tests.
Is the same as invoking Prysk with ``--verbose`` and ``--indent=4``.
Note that the following environment variables are reset before tests
are run:
* ``TMPDIR``, ``TEMP``, and ``TMP`` are set to the test runner's
``tmp`` directory. In test output, occurrences of this directory are
replaced by ``$TMPDIR``.
* ``LANG``, ``LC_ALL``, and ``LANGUAGE`` are set to ``C``.
* ``TZ`` is set to ``GMT``.
* ``COLUMNS`` is set to ``80``. (Note: When using ``--shell=zsh``,
this cannot be reset. It will reflect the actual terminal's width.)
* ``CDPATH`` and ``GREP_OPTIONS`` are set to an empty string.
Prysk also provides the following environment variables to tests:
* ``PRYSK_TEMP``, set to the test runner's temporary directory.
* ``TESTDIR``, set to the directory containing the test file.
* ``TESTFILE``, set to the basename of the current test file.
* ``TESTSHELL``, set to the value specified by ``--shell``.
Also note that care should be taken with commands that close the test
shell's ``stdin``. For example, if you're trying to invoke ``ssh`` in
a test, try adding the ``-n`` option to prevent it from closing
``stdin``. Similarly, if you invoke a daemon process that inherits
``stdout`` and fails to close it, it may cause Prysk to hang while
waiting for the test shell's ``stdout`` to be fully closed.
.. _unified context diff: https://en.wikipedia.org/wiki/Diff#Unified_format
.. _its own example tests: https://github.com/nicoretti/prysk/tree/master/examples
Raw data
{
"_id": null,
"home_page": "https://www.prysk.net/",
"name": "prysk",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0.0,>=3.8",
"maintainer_email": null,
"keywords": "test framework, snapshot testing, testing, cli, functional tests",
"author": "Nicola Coretti",
"author_email": "nico.coretti@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/50/fc/bdbe0ceeccddcff9d9a9119e0707424d427fae84c2e77913a7e15c92458e/prysk-0.19.0.tar.gz",
"platform": null,
"description": "Prysk\n======================\n.. image:: https://img.shields.io/github/actions/workflow/status/nicoretti/prysk/ci.yaml\n :target: https://github.com/Nicoretti/prysk/actions\n\n.. image:: https://img.shields.io/coverallsCoverage/github/Nicoretti/prysk\n :target: https://coveralls.io/github/Nicoretti/prysk\n\n.. image:: https://img.shields.io/badge/imports-isort-ef8336.svg\n :target: https://pycqa.github.io/isort/\n\n.. image:: https://img.shields.io/badge/code%20style-black-000000.svg\n :target: https://github.com/psf/black\n\n.. image:: https://img.shields.io/pypi/v/prysk\n :target: https://pypi.org/project/prysk/\n\n.. image:: https://img.shields.io/badge/docs-available-blue.svg\n :target: https://nicoretti.github.io/prysk/\n\nPrysk is a fork of the popular snapshot testing tool Cram_.\nEven though Cram_ is pretty complete and mature for everyday use,\nPrysk wants to continue pushing its development forward.\n\n.. _Cram: https://bitheap.org/cram\n\nPrysk tests look like snippets of interactive shell sessions. Prysk runs\neach command and compares the command output in the test with the\ncommand's actual output.\n\nHere's a snippet from `Prysk's own test suite`_:\n\n.. code-block:: console\n\n Set up prysk alias and example tests:\n\n $ . \"$TESTDIR\"/setup.sh\n\n Usage:\n\n $ prysk -h\n [Uu]sage: prysk \\[OPTIONS\\] TESTS\\.\\.\\. (re)\n\n [Oo]ptions: (re)\n -h, --help show this help message and exit\n -V, --version show version information and exit\n -q, --quiet don't print diffs\n -v, --verbose show filenames and test status\n -i, --interactive interactively merge changed test output\n -d, --debug write script output directly to the terminal\n -y, --yes answer yes to all questions\n -n, --no answer no to all questions\n -E, --preserve-env don't reset common environment variables\n --keep-tmpdir keep temporary directories\n --shell=PATH shell to use for running tests (default: /bin/sh)\n --shell-opts=OPTS arguments to invoke shell with\n --indent=NUM number of spaces to use for indentation (default: 2)\n --xunit-file=PATH path to write xUnit XML output\n --dos2unix convert DOS/Windows line endings to UNIX line endings\n --color mode which shall be used for coloring the output\n\nThe format in a nutshell:\n\n* Prysk tests use the ``.t`` file extension.\n\n* Lines beginning with two spaces, a dollar sign (``$``), and a space are run\n in the shell.\n\n* Lines beginning with two spaces, a greater than sign (``>``), and a space\n allow multi-line commands.\n\n* All other lines beginning with two spaces are considered command\n output.\n\n* Output lines ending with a space and the keyword ``(re)`` are\n matched as `Perl-compatible regular expressions`_.\n\n* Lines ending with a space and the keyword ``(glob)`` are matched\n with a glob-like syntax. The only special characters supported are\n ``*`` and ``?``. Both characters can be escaped using ``\\``, and the\n backslash can be escaped itself.\n\n* Output lines ending with either of the above keywords are always\n first matched literally with actual command output.\n\n* Lines ending with a space and the keyword ``(no-eol)`` will match\n actual output that doesn't end in a newline.\n\n* Actual output lines containing unprintable characters are escaped\n and suffixed with a space and the keyword ``(esc)``. Lines matching\n unprintable output must also contain the keyword.\n\n* Anything else is a comment.\n\n.. _Prysk's own test suite: https://github.com/Nicoretti/prysk/blob/master/test/integration/prysk/usage.t\n.. _Perl-compatible regular expressions: https://en.wikipedia.org/wiki/Perl_Compatible_Regular_Expressions\n\nUsage\n-----\n\nPrysk will print a dot for each passing test. If a test fails, a\n`unified context diff`_ is printed showing the test's expected output\nand the actual output. Skipped tests (empty tests and tests that exit\nwith return code ``80``) are marked with ``s`` instead of a dot.\n\nFor example, if we run Prysk on `its own example tests`_:\n\n.. code-block:: diff\n\n .s.!\n --- examples/fail.t\n +++ examples/fail.t.err\n @@ -3,21 +3,22 @@\n $ echo 1\n 1\n $ echo 1\n - 2\n + 1\n $ echo 1\n 1\n\n Invalid regex:\n\n $ echo 1\n - +++ (re)\n + 1\n\n Offset regular expression:\n\n $ printf 'foo\\nbar\\nbaz\\n\\n1\\nA\\n@\\n'\n foo\n + bar\n baz\n\n \\d (re)\n [A-Z] (re)\n - #\n + @\n s.\n # Ran 6 tests, 2 skipped, 1 failed.\n\nPrysk will also write the test with its actual output to\n``examples/fail.t.err``, allowing you to use other diff tools. This\nfile is automatically removed the next time the test passes.\n\nWhen you're first writing a test, you might just write the commands\nand run the test to see what happens. If you run Prysk with ``-i`` or\n``--interactive``, you'll be prompted to merge the actual output back\ninto the test. This makes it easy to quickly prototype new tests.\n\nIs the same as invoking Prysk with ``--verbose`` and ``--indent=4``.\n\nNote that the following environment variables are reset before tests\nare run:\n\n* ``TMPDIR``, ``TEMP``, and ``TMP`` are set to the test runner's\n ``tmp`` directory. In test output, occurrences of this directory are\n replaced by ``$TMPDIR``.\n\n* ``LANG``, ``LC_ALL``, and ``LANGUAGE`` are set to ``C``.\n\n* ``TZ`` is set to ``GMT``.\n\n* ``COLUMNS`` is set to ``80``. (Note: When using ``--shell=zsh``,\n this cannot be reset. It will reflect the actual terminal's width.)\n\n* ``CDPATH`` and ``GREP_OPTIONS`` are set to an empty string.\n\nPrysk also provides the following environment variables to tests:\n\n* ``PRYSK_TEMP``, set to the test runner's temporary directory.\n\n* ``TESTDIR``, set to the directory containing the test file.\n\n* ``TESTFILE``, set to the basename of the current test file.\n\n* ``TESTSHELL``, set to the value specified by ``--shell``.\n\nAlso note that care should be taken with commands that close the test\nshell's ``stdin``. For example, if you're trying to invoke ``ssh`` in\na test, try adding the ``-n`` option to prevent it from closing\n``stdin``. Similarly, if you invoke a daemon process that inherits\n``stdout`` and fails to close it, it may cause Prysk to hang while\nwaiting for the test shell's ``stdout`` to be fully closed.\n\n.. _unified context diff: https://en.wikipedia.org/wiki/Diff#Unified_format\n.. _its own example tests: https://github.com/nicoretti/prysk/tree/master/examples\n",
"bugtrack_url": null,
"license": "GNU GPLv2 or any later version",
"summary": "Functional tests for command line applications",
"version": "0.19.0",
"project_urls": {
"Homepage": "https://www.prysk.net/",
"Repository": "https://github.com/prysk/prysk"
},
"split_keywords": [
"test framework",
" snapshot testing",
" testing",
" cli",
" functional tests"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c713d9d13cc2d5ebd79f71dbf5c52f22c801c7bf4f503d1a573710f9ac505d7e",
"md5": "e0586ec5b9c411403713abf95458fb7a",
"sha256": "18da45337724c819a9084d45286dd38687e44931a7219606f27f2cfda43c077b"
},
"downloads": -1,
"filename": "prysk-0.19.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e0586ec5b9c411403713abf95458fb7a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0.0,>=3.8",
"size": 25944,
"upload_time": "2024-03-23T09:51:42",
"upload_time_iso_8601": "2024-03-23T09:51:42.263221Z",
"url": "https://files.pythonhosted.org/packages/c7/13/d9d13cc2d5ebd79f71dbf5c52f22c801c7bf4f503d1a573710f9ac505d7e/prysk-0.19.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "50fcbdbe0ceeccddcff9d9a9119e0707424d427fae84c2e77913a7e15c92458e",
"md5": "e4d3845383f7b73e6ba212bae941a456",
"sha256": "5d605bf679d347b4e5506be056532c5456bf36fd77efefd42a0fed9fe0da1c73"
},
"downloads": -1,
"filename": "prysk-0.19.0.tar.gz",
"has_sig": false,
"md5_digest": "e4d3845383f7b73e6ba212bae941a456",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0.0,>=3.8",
"size": 71633,
"upload_time": "2024-03-23T09:51:43",
"upload_time_iso_8601": "2024-03-23T09:51:43.912096Z",
"url": "https://files.pythonhosted.org/packages/50/fc/bdbe0ceeccddcff9d9a9119e0707424d427fae84c2e77913a7e15c92458e/prysk-0.19.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-23 09:51:43",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "prysk",
"github_project": "prysk",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "prysk"
}
JSON
