===============
pytest-workflow
===============
.. Badges have empty alts. So nothing shows up if they do not work.
.. This fixes readthedocs issues with badges.
.. image:: https://img.shields.io/pypi/v/pytest-workflow.svg
:target: https://pypi.org/project/pytest-workflow/
:alt:
.. image:: https://img.shields.io/conda/v/conda-forge/pytest-workflow.svg
:target: https://anaconda.org/conda-forge/pytest-workflow
:alt:
.. image:: https://img.shields.io/pypi/pyversions/pytest-workflow.svg
:target: https://pypi.org/project/pytest-workflow/
:alt:
.. image:: https://img.shields.io/pypi/l/pytest-workflow.svg
:target: https://github.com/LUMC/pytest-workflow/blob/master/LICENSE
:alt:
.. image:: https://travis-ci.org/LUMC/pytest-workflow.svg?branch=develop
:target: https://travis-ci.org/LUMC/pytest-workflow
:alt:
.. image:: https://codecov.io/gh/LUMC/pytest-workflow/branch/develop/graph/badge.svg
:target: https://codecov.io/gh/LUMC/pytest-workflow
:alt:
.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.3757727.svg
:target: https://doi.org/10.5281/zenodo.3757727
:alt: More information on how to cite pytest-workflow here.
pytest-workflow is a workflow-system agnostic testing framework that aims
to make pipeline/workflow testing easy by using YAML files for the test
configuration. Whether you write your pipelines in WDL, snakemake, nextflow,
bash or any other workflow framework, pytest-workflow makes testing easy.
pytest-workflow is build on top of the pytest test framework.
For our complete documentation and examples checkout our
`readthedocs page <https://pytest-workflow.readthedocs.io/>`_.
Installation
============
Pytest-workflow requires Python 3.7 or higher. It is tested on Python 3.7,
3.8, 3.9, 3.10 and 3.11.
- Make sure your virtual environment is activated.
- Install using pip ``pip install pytest-workflow``
- Create a ``tests`` directory in the root of your repository.
- Create your test yaml files in the ``tests`` directory.
Pytest-workflow is also available as a `conda package on conda-forge
<https://anaconda.org/conda-forge/pytest-workflow>`_. Follow
`these instructions
<http://conda-forge.org/docs/user/introduction.html#how-can-i-install-packages-from-conda-forge>`_
to set up channels properly in order to use conda-forge. Alternatively,
you can `set up the channels correctly for use with bioconda
<https://bioconda.github.io/index.html#set-up-channels>`_. After that ``conda
install pytest-workflow`` can be used to install pytest-workflow.
Quickstart
==========
Run ``pytest`` from an environment with pytest-workflow installed.
Pytest will automatically gather files in the ``tests`` directory starting with
``test`` and ending in ``.yaml`` or ``.yml``.
To check the progress of a workflow while it is running you can use ``tail -f``
on the ``stdout`` or ``stderr`` file of the workflow. The locations of these
files are reported in the log as soon as a workflow is started.
For debugging pipelines using the ``--kwd`` or ``--keep-workflow-wd`` flag is
recommended. This will keep the workflow directory and logs after the test run
so it is possible to check where the pipeline crashed. The ``-v`` flag can come
in handy as well as it gives a complete overview of succeeded and failed tests.
Below is an example of a YAML file that defines a test:
.. code-block:: yaml
- name: Touch a file
command: touch test.file
files:
- path: test.file
This will run ``touch test.file`` and check afterwards if a file with path:
``test.file`` is present. It will also check if the ``command`` has exited
with exit code ``0``, which is the only default test that is run. Testing
workflows that exit with another exit code is also possible. Several other
predefined tests as well as custom tests are possible.
.. code-block:: yaml
- name: moo file # The name of the workflow (required)
command: bash moo_workflow.sh # The command to execute the workflow (required)
files: # A list of files to check (optional)
- path: "moo.txt" # File path. (Required for each file)
contains: # A list of strings that should be in the file (optional)
- "moo"
must_not_contain: # A list of strings that should NOT be in the file (optional)
- "Cock a doodle doo"
md5sum: e583af1f8b00b53cda87ae9ead880224 # Md5sum of the file (optional)
encoding: UTF-8 # Encoding for the text file (optional). Defaults to system locale.
- name: simple echo # A second workflow. Notice the starting `-` which means
command: "echo moo" # that workflow items are in a list. You can add as much workflows as you want
files:
- path: "moo.txt"
should_exist: false # Whether a file should be there or not. (optional, if not given defaults to true)
stdout: # Options for testing stdout (optional)
contains: # List of strings which should be in stdout (optional)
- "moo"
must_not_contain: # List of strings that should NOT be in stout (optional)
- "Cock a doodle doo"
encoding: ASCII # Encoding for stdout (optional). Defaults to system locale.
- name: mission impossible # Also failing workflows can be tested
tags: # A list of tags that can be used to select which test
- should fail # is run with pytest using the `--tag` flag.
command: bash impossible.sh
exit_code: 2 # What the exit code should be (optional, if not given defaults to 0)
files:
- path: "fail.log" # Multiple files can be tested for each workflow
- path: "TomCruise.txt.gz" # Gzipped files can also be searched, provided their extension is '.gz'
contains:
- "starring"
extract_md5sum: e27c52f6b5f8152aa3ef58be7bdacc4d # Md5sum of the uncompressed file (optional)
stderr: # Options for testing stderr (optional)
contains: # A list of strings which should be in stderr (optional)
- "BSOD error, please contact the IT crowd"
must_not_contain: # A list of strings which should NOT be in stderr (optional)
- "Mission accomplished!"
encoding: UTF-16 # Encoding for stderr (optional). Defaults to system locale.
- name: regex tests
command: echo Hello, world
stdout:
contains_regex: # A list of regex patterns that should be in stdout (optional)
- 'Hello.*' # Note the single quotes, these are required for complex regexes
- 'Hello .*' # This will fail, since there is a comma after Hello, not a space
must_not_contain_regex: # A list of regex patterns that should not be in stdout (optional)
- '^He.*' # This will fail, since the regex matches Hello, world
- '^Hello .*' # Complex regexes will break yaml if double quotes are used
For more information on how Python parses regular expressions, see the `Python
documentation <https://docs.python.org/3/library/re.html>`_.
Documentation for more advanced use cases including the custom tests can be
found on our `readthedocs page <https://pytest-workflow.readthedocs.io/>`_.
Raw data
{
"_id": null,
"home_page": "https://github.com/LUMC/pytest-workflow",
"name": "pytest-workflow",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "pytest workflow pipeline yaml yml wdl cromwell snakemake",
"author": "Leiden University Medical Center",
"author_email": "sasc@lumc.nl",
"download_url": "https://files.pythonhosted.org/packages/50/f8/6fbcfa9b39ae3ff0f66bbaae8157f2a3cb4addf0487d536965945b776740/pytest-workflow-2.1.0.tar.gz",
"platform": null,
"description": "===============\npytest-workflow\n===============\n\n.. Badges have empty alts. So nothing shows up if they do not work.\n.. This fixes readthedocs issues with badges.\n.. image:: https://img.shields.io/pypi/v/pytest-workflow.svg\n :target: https://pypi.org/project/pytest-workflow/\n :alt:\n\n.. image:: https://img.shields.io/conda/v/conda-forge/pytest-workflow.svg\n :target: https://anaconda.org/conda-forge/pytest-workflow\n :alt:\n\n.. image:: https://img.shields.io/pypi/pyversions/pytest-workflow.svg\n :target: https://pypi.org/project/pytest-workflow/\n :alt:\n\n.. image:: https://img.shields.io/pypi/l/pytest-workflow.svg\n :target: https://github.com/LUMC/pytest-workflow/blob/master/LICENSE\n :alt:\n\n.. image:: https://travis-ci.org/LUMC/pytest-workflow.svg?branch=develop\n :target: https://travis-ci.org/LUMC/pytest-workflow\n :alt:\n\n.. image:: https://codecov.io/gh/LUMC/pytest-workflow/branch/develop/graph/badge.svg\n :target: https://codecov.io/gh/LUMC/pytest-workflow\n :alt:\n\n.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.3757727.svg\n :target: https://doi.org/10.5281/zenodo.3757727\n :alt: More information on how to cite pytest-workflow here.\n\npytest-workflow is a workflow-system agnostic testing framework that aims \nto make pipeline/workflow testing easy by using YAML files for the test \nconfiguration. Whether you write your pipelines in WDL, snakemake, nextflow, \nbash or any other workflow framework, pytest-workflow makes testing easy.\npytest-workflow is build on top of the pytest test framework.\n\nFor our complete documentation and examples checkout our\n`readthedocs page <https://pytest-workflow.readthedocs.io/>`_.\n\n\nInstallation\n============\nPytest-workflow requires Python 3.7 or higher. It is tested on Python 3.7,\n3.8, 3.9, 3.10 and 3.11.\n\n- Make sure your virtual environment is activated.\n- Install using pip ``pip install pytest-workflow``\n- Create a ``tests`` directory in the root of your repository.\n- Create your test yaml files in the ``tests`` directory.\n\nPytest-workflow is also available as a `conda package on conda-forge\n<https://anaconda.org/conda-forge/pytest-workflow>`_. Follow\n`these instructions \n<http://conda-forge.org/docs/user/introduction.html#how-can-i-install-packages-from-conda-forge>`_ \nto set up channels properly in order to use conda-forge. Alternatively,\nyou can `set up the channels correctly for use with bioconda \n<https://bioconda.github.io/index.html#set-up-channels>`_. After that ``conda\ninstall pytest-workflow`` can be used to install pytest-workflow. \n\nQuickstart\n==========\n\nRun ``pytest`` from an environment with pytest-workflow installed.\nPytest will automatically gather files in the ``tests`` directory starting with\n``test`` and ending in ``.yaml`` or ``.yml``.\n\nTo check the progress of a workflow while it is running you can use ``tail -f``\non the ``stdout`` or ``stderr`` file of the workflow. The locations of these\nfiles are reported in the log as soon as a workflow is started.\n\nFor debugging pipelines using the ``--kwd`` or ``--keep-workflow-wd`` flag is\nrecommended. This will keep the workflow directory and logs after the test run\nso it is possible to check where the pipeline crashed. The ``-v`` flag can come\nin handy as well as it gives a complete overview of succeeded and failed tests.\n\nBelow is an example of a YAML file that defines a test:\n\n.. code-block:: yaml\n\n - name: Touch a file\n command: touch test.file\n files:\n - path: test.file\n\nThis will run ``touch test.file`` and check afterwards if a file with path:\n``test.file`` is present. It will also check if the ``command`` has exited\nwith exit code ``0``, which is the only default test that is run. Testing\nworkflows that exit with another exit code is also possible. Several other\npredefined tests as well as custom tests are possible.\n\n.. code-block:: yaml\n\n - name: moo file # The name of the workflow (required)\n command: bash moo_workflow.sh # The command to execute the workflow (required)\n files: # A list of files to check (optional)\n - path: \"moo.txt\" # File path. (Required for each file)\n contains: # A list of strings that should be in the file (optional)\n - \"moo\"\n must_not_contain: # A list of strings that should NOT be in the file (optional)\n - \"Cock a doodle doo\"\n md5sum: e583af1f8b00b53cda87ae9ead880224 # Md5sum of the file (optional)\n encoding: UTF-8 # Encoding for the text file (optional). Defaults to system locale.\n\n - name: simple echo # A second workflow. Notice the starting `-` which means\n command: \"echo moo\" # that workflow items are in a list. You can add as much workflows as you want\n files:\n - path: \"moo.txt\"\n should_exist: false # Whether a file should be there or not. (optional, if not given defaults to true)\n stdout: # Options for testing stdout (optional)\n contains: # List of strings which should be in stdout (optional)\n - \"moo\"\n must_not_contain: # List of strings that should NOT be in stout (optional)\n - \"Cock a doodle doo\"\n encoding: ASCII # Encoding for stdout (optional). Defaults to system locale.\n\n - name: mission impossible # Also failing workflows can be tested\n tags: # A list of tags that can be used to select which test\n - should fail # is run with pytest using the `--tag` flag.\n command: bash impossible.sh\n exit_code: 2 # What the exit code should be (optional, if not given defaults to 0)\n files:\n - path: \"fail.log\" # Multiple files can be tested for each workflow\n - path: \"TomCruise.txt.gz\" # Gzipped files can also be searched, provided their extension is '.gz'\n contains:\n - \"starring\"\n extract_md5sum: e27c52f6b5f8152aa3ef58be7bdacc4d # Md5sum of the uncompressed file (optional)\n stderr: # Options for testing stderr (optional)\n contains: # A list of strings which should be in stderr (optional)\n - \"BSOD error, please contact the IT crowd\"\n must_not_contain: # A list of strings which should NOT be in stderr (optional)\n - \"Mission accomplished!\"\n encoding: UTF-16 # Encoding for stderr (optional). Defaults to system locale.\n\n - name: regex tests\n command: echo Hello, world\n stdout:\n contains_regex: # A list of regex patterns that should be in stdout (optional)\n - 'Hello.*' # Note the single quotes, these are required for complex regexes\n - 'Hello .*' # This will fail, since there is a comma after Hello, not a space\n\n must_not_contain_regex: # A list of regex patterns that should not be in stdout (optional)\n - '^He.*' # This will fail, since the regex matches Hello, world\n - '^Hello .*' # Complex regexes will break yaml if double quotes are used\n\nFor more information on how Python parses regular expressions, see the `Python\ndocumentation <https://docs.python.org/3/library/re.html>`_.\n\nDocumentation for more advanced use cases including the custom tests can be\nfound on our `readthedocs page <https://pytest-workflow.readthedocs.io/>`_.\n",
"bugtrack_url": null,
"license": "AGPL-3.0-or-later",
"summary": "A pytest plugin for configuring workflow/pipeline tests using YAML files",
"version": "2.1.0",
"project_urls": {
"Homepage": "https://github.com/LUMC/pytest-workflow"
},
"split_keywords": [
"pytest",
"workflow",
"pipeline",
"yaml",
"yml",
"wdl",
"cromwell",
"snakemake"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "29cbab26957b835d03533d314bb5b5b6f866585a561a1a8f446afc6847772f46",
"md5": "8613f7d32a3c36228194a97619ded24b",
"sha256": "1ea86807f4207921249ae2eff4db2c4cc3c88a1bf8ac9538a0e8231e98f7b2e4"
},
"downloads": -1,
"filename": "pytest_workflow-2.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8613f7d32a3c36228194a97619ded24b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 40074,
"upload_time": "2024-03-18T16:02:41",
"upload_time_iso_8601": "2024-03-18T16:02:41.753799Z",
"url": "https://files.pythonhosted.org/packages/29/cb/ab26957b835d03533d314bb5b5b6f866585a561a1a8f446afc6847772f46/pytest_workflow-2.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "50f86fbcfa9b39ae3ff0f66bbaae8157f2a3cb4addf0487d536965945b776740",
"md5": "fac3514b3321c71d7761f4efa9fa9bcf",
"sha256": "dc86ad9a5f94482aec14926788f6b78b428be68ee0428cbca22f89b6326f8b7a"
},
"downloads": -1,
"filename": "pytest-workflow-2.1.0.tar.gz",
"has_sig": false,
"md5_digest": "fac3514b3321c71d7761f4efa9fa9bcf",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 50328,
"upload_time": "2024-03-18T16:02:44",
"upload_time_iso_8601": "2024-03-18T16:02:44.418654Z",
"url": "https://files.pythonhosted.org/packages/50/f8/6fbcfa9b39ae3ff0f66bbaae8157f2a3cb4addf0487d536965945b776740/pytest-workflow-2.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-18 16:02:44",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "LUMC",
"github_project": "pytest-workflow",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "pyyaml",
"specs": []
},
{
"name": "pytest",
"specs": [
[
">=",
"7.0.0"
]
]
},
{
"name": "jsonschema",
"specs": []
},
{
"name": "xopen",
"specs": [
[
">=",
"1.7.0"
]
]
},
{
"name": "zstandard",
"specs": []
}
],
"tox": true,
"lcname": "pytest-workflow"
}
JSON
