frysk
======================
.. image:: https://img.shields.io/github/actions/workflow/status/nicoretti/frysk/ci.yaml
:target: https://github.com/Nicoretti/frysk/actions
.. image:: https://img.shields.io/coverallsCoverage/github/Nicoretti/frysk
:target: https://coveralls.io/github/Nicoretti/frysk
.. image:: https://img.shields.io/badge/imports-isort-ef8336.svg
:target: https://pycqa.github.io/isort/
.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
:target: https://github.com/psf/black
.. image:: https://img.shields.io/pypi/v/frysk
:target: https://pypi.org/project/frysk/
.. image:: https://img.shields.io/badge/docs-available-blue.svg
:target: https://nicoretti.github.io/frysk/
frysk is a fork of the popular snapshot testing tool Cram_.
Even though Cram_ is pretty complete and mature for everyday use,
frysk wants to continue pushing its development forward.
.. _Cram: https://bitheap.org/cram
frysk tests look like snippets of interactive shell sessions. frysk runs
each command and compares the command output in the test with the
command's actual output.
Here's a snippet from `frysk's own test suite`_:
.. code-block:: console
Set up frysk alias and example tests:
$ . "$TESTDIR"/setup.sh
Usage:
$ frysk -h
[Uu]sage: frysk \[OPTIONS\] TESTS\.\.\. (re)
[Oo]ptions: (re)
-h, --help show this help message and exit
-V, --version show version information and exit
-q, --quiet don't print diffs
-v, --verbose show filenames and test status
-i, --interactive interactively merge changed test output
-d, --debug write script output directly to the terminal
-y, --yes answer yes to all questions
-n, --no answer no to all questions
-E, --preserve-env don't reset common environment variables
--keep-tmpdir keep temporary directories
--shell=PATH shell to use for running tests (default: /bin/sh)
--shell-opts=OPTS arguments to invoke shell with
--indent=NUM number of spaces to use for indentation (default: 2)
--xunit-file=PATH path to write xUnit XML output
--dos2unix convert DOS/Windows line endings to UNIX line endings
--color mode which shall be used for coloring the output
The format in a nutshell:
* frysk tests use the ``.t`` file extension.
* Lines beginning with two spaces, a dollar sign (``$``), and a space are run
in the shell.
* Lines beginning with two spaces, a greater than sign (``>``), and a space
allow multi-line commands.
* All other lines beginning with two spaces are considered command
output.
* Output lines ending with a space and the keyword ``(re)`` are
matched as `Perl-compatible regular expressions`_.
* Lines ending with a space and the keyword ``(glob)`` are matched
with a glob-like syntax. The only special characters supported are
``*`` and ``?``. Both characters can be escaped using ``\``, and the
backslash can be escaped itself.
* Output lines ending with either of the above keywords are always
first matched literally with actual command output.
* Lines ending with a space and the keyword ``(no-eol)`` will match
actual output that doesn't end in a newline.
* Actual output lines containing unprintable characters are escaped
and suffixed with a space and the keyword ``(esc)``. Lines matching
unprintable output must also contain the keyword.
* Anything else is a comment.
.. _frysk's own test suite: https://github.com/Nicoretti/frysk/blob/master/test/integration/frysk/usage.t
.. _Perl-compatible regular expressions: https://en.wikipedia.org/wiki/Perl_Compatible_Regular_Expressions
Usage
-----
frysk will print a dot for each passing test. If a test fails, a
`unified context diff`_ is printed showing the test's expected output
and the actual output. Skipped tests (empty tests and tests that exit
with return code ``80``) are marked with ``s`` instead of a dot.
For example, if we run frysk on `its own example tests`_:
.. code-block:: diff
.s.!
--- examples/fail.t
+++ examples/fail.t.err
@@ -3,21 +3,22 @@
$ echo 1
1
$ echo 1
- 2
+ 1
$ echo 1
1
Invalid regex:
$ echo 1
- +++ (re)
+ 1
Offset regular expression:
$ printf 'foo\nbar\nbaz\n\n1\nA\n@\n'
foo
+ bar
baz
\d (re)
[A-Z] (re)
- #
+ @
s.
# Ran 6 tests, 2 skipped, 1 failed.
frysk will also write the test with its actual output to
``examples/fail.t.err``, allowing you to use other diff tools. This
file is automatically removed the next time the test passes.
When you're first writing a test, you might just write the commands
and run the test to see what happens. If you run frysk with ``-i`` or
``--interactive``, you'll be prompted to merge the actual output back
into the test. This makes it easy to quickly prototype new tests.
Is the same as invoking frysk with ``--verbose`` and ``--indent=4``.
Note that the following environment variables are reset before tests
are run:
* ``TMPDIR``, ``TEMP``, and ``TMP`` are set to the test runner's
``tmp`` directory. In test output, occurrences of this directory are
replaced by ``$TMPDIR``.
* ``LANG``, ``LC_ALL``, and ``LANGUAGE`` are set to ``C``.
* ``TZ`` is set to ``GMT``.
* ``COLUMNS`` is set to ``80``. (Note: When using ``--shell=zsh``,
this cannot be reset. It will reflect the actual terminal's width.)
* ``CDPATH`` and ``GREP_OPTIONS`` are set to an empty string.
frysk also provides the following environment variables to tests:
* ``frysk_TEMP``, set to the test runner's temporary directory.
* ``TESTDIR``, set to the directory containing the test file.
* ``TESTFILE``, set to the basename of the current test file.
* ``TESTSHELL``, set to the value specified by ``--shell``.
Also note that care should be taken with commands that close the test
shell's ``stdin``. For example, if you're trying to invoke ``ssh`` in
a test, try adding the ``-n`` option to prevent it from closing
``stdin``. Similarly, if you invoke a daemon process that inherits
``stdout`` and fails to close it, it may cause frysk to hang while
waiting for the test shell's ``stdout`` to be fully closed.
.. _unified context diff: https://en.wikipedia.org/wiki/Diff#Unified_format
.. _its own example tests: https://github.com/nicoretti/frysk/tree/master/examples
Raw data
{
"_id": null,
"home_page": "https://nicoretti.github.io/frysk/",
"name": "frysk",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7,<4.0.0",
"maintainer_email": "",
"keywords": "test framework,snapshot testing,testing,cli,functional tests",
"author": "Mehul Kar",
"author_email": "mehul.kar@vercel.org",
"download_url": "https://files.pythonhosted.org/packages/76/1a/996f8fc334a3ee984bf62ecd0daf91e85ed61abb4c87af5a7cf7cfd8fa45/frysk-0.2.0.tar.gz",
"platform": null,
"description": "frysk\n======================\n.. image:: https://img.shields.io/github/actions/workflow/status/nicoretti/frysk/ci.yaml\n :target: https://github.com/Nicoretti/frysk/actions\n\n.. image:: https://img.shields.io/coverallsCoverage/github/Nicoretti/frysk\n :target: https://coveralls.io/github/Nicoretti/frysk\n\n.. image:: https://img.shields.io/badge/imports-isort-ef8336.svg\n :target: https://pycqa.github.io/isort/\n\n.. image:: https://img.shields.io/badge/code%20style-black-000000.svg\n :target: https://github.com/psf/black\n\n.. image:: https://img.shields.io/pypi/v/frysk\n :target: https://pypi.org/project/frysk/\n\n.. image:: https://img.shields.io/badge/docs-available-blue.svg\n :target: https://nicoretti.github.io/frysk/\n\nfrysk is a fork of the popular snapshot testing tool Cram_.\nEven though Cram_ is pretty complete and mature for everyday use,\nfrysk wants to continue pushing its development forward.\n\n.. _Cram: https://bitheap.org/cram\n\nfrysk tests look like snippets of interactive shell sessions. frysk runs\neach command and compares the command output in the test with the\ncommand's actual output.\n\nHere's a snippet from `frysk's own test suite`_:\n\n.. code-block:: console\n\n Set up frysk alias and example tests:\n\n $ . \"$TESTDIR\"/setup.sh\n\n Usage:\n\n $ frysk -h\n [Uu]sage: frysk \\[OPTIONS\\] TESTS\\.\\.\\. (re)\n\n [Oo]ptions: (re)\n -h, --help show this help message and exit\n -V, --version show version information and exit\n -q, --quiet don't print diffs\n -v, --verbose show filenames and test status\n -i, --interactive interactively merge changed test output\n -d, --debug write script output directly to the terminal\n -y, --yes answer yes to all questions\n -n, --no answer no to all questions\n -E, --preserve-env don't reset common environment variables\n --keep-tmpdir keep temporary directories\n --shell=PATH shell to use for running tests (default: /bin/sh)\n --shell-opts=OPTS arguments to invoke shell with\n --indent=NUM number of spaces to use for indentation (default: 2)\n --xunit-file=PATH path to write xUnit XML output\n --dos2unix convert DOS/Windows line endings to UNIX line endings\n --color mode which shall be used for coloring the output\n\nThe format in a nutshell:\n\n* frysk tests use the ``.t`` file extension.\n\n* Lines beginning with two spaces, a dollar sign (``$``), and a space are run\n in the shell.\n\n* Lines beginning with two spaces, a greater than sign (``>``), and a space\n allow multi-line commands.\n\n* All other lines beginning with two spaces are considered command\n output.\n\n* Output lines ending with a space and the keyword ``(re)`` are\n matched as `Perl-compatible regular expressions`_.\n\n* Lines ending with a space and the keyword ``(glob)`` are matched\n with a glob-like syntax. The only special characters supported are\n ``*`` and ``?``. Both characters can be escaped using ``\\``, and the\n backslash can be escaped itself.\n\n* Output lines ending with either of the above keywords are always\n first matched literally with actual command output.\n\n* Lines ending with a space and the keyword ``(no-eol)`` will match\n actual output that doesn't end in a newline.\n\n* Actual output lines containing unprintable characters are escaped\n and suffixed with a space and the keyword ``(esc)``. Lines matching\n unprintable output must also contain the keyword.\n\n* Anything else is a comment.\n\n.. _frysk's own test suite: https://github.com/Nicoretti/frysk/blob/master/test/integration/frysk/usage.t\n.. _Perl-compatible regular expressions: https://en.wikipedia.org/wiki/Perl_Compatible_Regular_Expressions\n\nUsage\n-----\n\nfrysk will print a dot for each passing test. If a test fails, a\n`unified context diff`_ is printed showing the test's expected output\nand the actual output. Skipped tests (empty tests and tests that exit\nwith return code ``80``) are marked with ``s`` instead of a dot.\n\nFor example, if we run frysk on `its own example tests`_:\n\n.. code-block:: diff\n\n .s.!\n --- examples/fail.t\n +++ examples/fail.t.err\n @@ -3,21 +3,22 @@\n $ echo 1\n 1\n $ echo 1\n - 2\n + 1\n $ echo 1\n 1\n\n Invalid regex:\n\n $ echo 1\n - +++ (re)\n + 1\n\n Offset regular expression:\n\n $ printf 'foo\\nbar\\nbaz\\n\\n1\\nA\\n@\\n'\n foo\n + bar\n baz\n\n \\d (re)\n [A-Z] (re)\n - #\n + @\n s.\n # Ran 6 tests, 2 skipped, 1 failed.\n\nfrysk will also write the test with its actual output to\n``examples/fail.t.err``, allowing you to use other diff tools. This\nfile is automatically removed the next time the test passes.\n\nWhen you're first writing a test, you might just write the commands\nand run the test to see what happens. If you run frysk with ``-i`` or\n``--interactive``, you'll be prompted to merge the actual output back\ninto the test. This makes it easy to quickly prototype new tests.\n\nIs the same as invoking frysk with ``--verbose`` and ``--indent=4``.\n\nNote that the following environment variables are reset before tests\nare run:\n\n* ``TMPDIR``, ``TEMP``, and ``TMP`` are set to the test runner's\n ``tmp`` directory. In test output, occurrences of this directory are\n replaced by ``$TMPDIR``.\n\n* ``LANG``, ``LC_ALL``, and ``LANGUAGE`` are set to ``C``.\n\n* ``TZ`` is set to ``GMT``.\n\n* ``COLUMNS`` is set to ``80``. (Note: When using ``--shell=zsh``,\n this cannot be reset. It will reflect the actual terminal's width.)\n\n* ``CDPATH`` and ``GREP_OPTIONS`` are set to an empty string.\n\nfrysk also provides the following environment variables to tests:\n\n* ``frysk_TEMP``, set to the test runner's temporary directory.\n\n* ``TESTDIR``, set to the directory containing the test file.\n\n* ``TESTFILE``, set to the basename of the current test file.\n\n* ``TESTSHELL``, set to the value specified by ``--shell``.\n\nAlso note that care should be taken with commands that close the test\nshell's ``stdin``. For example, if you're trying to invoke ``ssh`` in\na test, try adding the ``-n`` option to prevent it from closing\n``stdin``. Similarly, if you invoke a daemon process that inherits\n``stdout`` and fails to close it, it may cause frysk to hang while\nwaiting for the test shell's ``stdout`` to be fully closed.\n\n.. _unified context diff: https://en.wikipedia.org/wiki/Diff#Unified_format\n.. _its own example tests: https://github.com/nicoretti/frysk/tree/master/examples\n",
"bugtrack_url": null,
"license": "GNU GPLv2 or any later version",
"summary": "Functional tests for command line applications (fork)",
"version": "0.2.0",
"project_urls": {
"Homepage": "https://nicoretti.github.io/frysk/",
"Repository": "https://github.com/Nicoretti/frysk"
},
"split_keywords": [
"test framework",
"snapshot testing",
"testing",
"cli",
"functional tests"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "318254875e103987b73110bead579fe10da5f3ffcf671cbc2692b8e7f1d7d6fa",
"md5": "ff9176494d4c1e9360d2b5f6b4529d40",
"sha256": "d7a57724751c165d6dd032a977b7a2654b921cd366e36ea3e1c7bc72b8409151"
},
"downloads": -1,
"filename": "frysk-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ff9176494d4c1e9360d2b5f6b4529d40",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7,<4.0.0",
"size": 88556,
"upload_time": "2023-11-04T06:39:02",
"upload_time_iso_8601": "2023-11-04T06:39:02.435730Z",
"url": "https://files.pythonhosted.org/packages/31/82/54875e103987b73110bead579fe10da5f3ffcf671cbc2692b8e7f1d7d6fa/frysk-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "761a996f8fc334a3ee984bf62ecd0daf91e85ed61abb4c87af5a7cf7cfd8fa45",
"md5": "b47fc05f0c289ea2858a98b0d14fc39f",
"sha256": "ab6d3748ef5928cb06293f2705cc06ccc127ebee618a18c76fe599cd24a86215"
},
"downloads": -1,
"filename": "frysk-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "b47fc05f0c289ea2858a98b0d14fc39f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7,<4.0.0",
"size": 74042,
"upload_time": "2023-11-04T06:39:04",
"upload_time_iso_8601": "2023-11-04T06:39:04.339173Z",
"url": "https://files.pythonhosted.org/packages/76/1a/996f8fc334a3ee984bf62ecd0daf91e85ed61abb4c87af5a7cf7cfd8fa45/frysk-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-04 06:39:04",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Nicoretti",
"github_project": "frysk",
"github_not_found": true,
"lcname": "frysk"
}
JSON
