=================
zope.testrunner
=================
.. image:: https://img.shields.io/pypi/v/zope.testrunner.svg
:target: https://pypi.org/project/zope.testrunner/
:alt: Latest release
.. image:: https://img.shields.io/pypi/pyversions/zope.testrunner.svg
:target: https://pypi.org/project/zope.testrunner/
:alt: Supported Python versions
.. image:: https://github.com/zopefoundation/zope.testrunner/actions/workflows/tests.yml/badge.svg
:target: https://github.com/zopefoundation/zope.testrunner/actions/workflows/tests.yml
.. image:: https://coveralls.io/repos/github/zopefoundation/zope.testrunner/badge.svg?branch=master
:target: https://coveralls.io/github/zopefoundation/zope.testrunner?branch=master
.. image:: https://readthedocs.org/projects/zopetestrunner/badge/?version=latest
:target: https://zopetestrunner.readthedocs.io/en/latest/?badge=latest
:alt: Documentation Status
.. contents::
This package provides a flexible test runner with layer support.
Detailed documentation is hosted at https://zopetestrunner.readthedocs.io
=======================
Using zope.testrunner
=======================
.. IMPORTANT: This document is included in the long_description for
rendering on PyPI, so it cannot use Sphinx-only features.
Installation
============
Buildout-based projects
-----------------------
zope.testrunner is often used for projects that use buildout_::
[buildout]
develop = .
parts = ... test ...
[test]
recipe = zc.recipe.testrunner
eggs = mypackage
The usual buildout process ::
python bootstrap.py
bin/buildout
creates a ``bin/test`` script that will run the tests for *mypackage*.
.. tip::
zc.recipe.testrunner_ takes care to specify the right
``--test-path`` option in the generated script. You can add
other options (such as ``--tests-pattern``) too; check
zc.recipe.testrunner_'s documentation for details.
Virtualenv-based projects
-------------------------
``pip install zope.testrunner`` and you'll get a ``zope-testrunner``
script. Run your tests with ::
zope-testrunner --test-path=path/to/your/source/tree
Your source code needs to be available for the testrunner to import,
so you need to run ``python setup.py install`` or ``pip install -e
.`` into the same virtualenv_.
Some useful command-line options to get you started
===================================================
-p show a progress indicator
-v increase verbosity
-c colorize the output
-t test specify test names (one or more regexes)
-m module specify test modules (one or more regexes)
-s package specify test packages (one or more regexes)
--list-tests show names of tests instead of running them
-x stop on first error or failure
-D, --pdb enable post-mortem debugging of test failures
--xml path generate XML reports to be written at the given path
--help show *all* command-line options (there are many more!)
For example ::
bin/test -pvc -m test_foo -t TestBar
runs all TestBar tests from a module called test_foo.py.
Writing tests
=============
``zope.testrunner`` expects to find your tests inside your package
directory, in a subpackage or module named ``tests``. Test modules
in a test subpackage should be named ``test*.py``.
.. tip::
You can change these assumptions with ``--tests-pattern`` and
``--test-file-pattern`` test runner options.
Tests themselves should be classes inheriting from
``unittest.TestCase``, and if you wish to use doctests, please tell
the test runner where to find them and what options to use for them
in by supplying a function named ``test_suite``.
Example::
import unittest
import doctest
class TestArithmetic(unittest.TestCase):
def test_two_plus_two(self):
self.assertEqual(2 + 2, 4)
def doctest_string_formatting():
"""Test Python string formatting
>>> print('{} + {}'.format(2, 2))
2 + 2
"""
def test_suite():
return unittest.TestSuite([
unittest.defaultTestLoader.loadTestsFromName(__name__),
doctest.DocTestSuite(),
doctest.DocFileSuite('../README.txt',
optionflags=doctest.ELLIPSIS),
])
Test grouping
=============
In addition to per-package and per-module filtering, zope.testrunner
has other mechanisms for grouping tests:
* **layers** allow you to have shared setup/teardown code to be used
by a group of tests, that is executed only once, and not for each
test. Layers are orthogonal to the usual package/module structure
and are specified by setting the ``layer`` attribute on test
suites.
* **levels** allow you to group slow-running tests and not run them
by default. They're specified by setting the ``level`` attribute
on test suites to an int.
Other features
==============
zope.testrunner can profile your tests, measure test coverage,
check for memory leaks, integrate with subunit_, shuffle the
test execution order, and run multiple tests in parallel.
.. _buildout: https://buildout.readthedocs.io
.. _virtualenv: https://virtualenv.pypa.io/
.. _zc.recipe.testrunner: https://pypi.python.org/pypi/zc.recipe.testrunner
.. _subunit: https://pypi.python.org/pypi/python-subunit
===========================
zope.testrunner Changelog
===========================
6.4 (2024-02-27)
================
- Add PEP 440 support (implicit namespaces).
(`#160 <https://github.com/zopefoundation/zope.testrunner/issues/160>`_)
6.3.1 (2024-02-12)
==================
- Fix XML tests when running in distribution resp. separately.
(`#163 <https://github.com/zopefoundation/zope.testrunner/issues/163>`_)
6.3 (2024-02-07)
================
- Exit cleanly when using the test runner ``--version`` argument.
(`#102 <https://github.com/zopefoundation/zope.testrunner/issues/102>`_)
- Add new ``--xml <path>`` option to write JUnit-like XML reports.
Code comes from ``collective.xmltestreport``, but be aware that here ``--xml``
is not a boolean, but expects a path!
(`#148 <https://github.com/zopefoundation/zope.testrunner/issues/148>`_).
- Add support for Python 3.13a3.
6.2.1 (2023-12-22)
==================
- Work around Python 3.12.1+ no longer calling ``startTest`` for skipped tests
(`#157 <https://github.com/zopefoundation/zope.testrunner/issues/157>`_).
6.2 (2023-11-08)
================
- Add support for Python 3.12.
- Update code and tests to ``python-subunit >= 1.4.3`` thus requiring at least
this version.
6.1 (2023-08-26)
================
- Add preliminary support for Python 3.12b4.
(`#149 <https://github.com/zopefoundation/zope.testrunner/issues/149>`_)
6.0 (2023-03-28)
================
- Drop support for Python 2.7, 3.5, 3.6.
5.6 (2022-12-09)
================
- Add support for Python 3.11.
- Inline a small part of ``random.Random.shuffle`` which was deprecated in
Python 3.9 and removed in 3.11 (`#119
<https://github.com/zopefoundation/zope.testrunner/issues/119>`_).
- Don't trigger post mortem debugger for skipped tests. ( `#141
<https://github.com/zopefoundation/zope.testrunner/issues/141>`_).
5.5.1 (2022-09-07)
==================
- Fix: let ``--at-level=level`` with ``level <= 0`` run the tests
at all levels (rather than at no level)
`#138 <https://github.com/zopefoundation/zope.testrunner/issues/138>`_.
5.5 (2022-06-24)
================
- Use ``sys._current_frames`` (rather than ``threading.enumerate``)
as base for new thread detection, fixes
`#130 <https://github.com/zopefoundation/zope.testrunner/issues/130>`_.
- New option ``--gc-after-test``. It calls for a garbage collection
after each test and can be used to track down ``ResourceWarning``s
and cyclic garbage.
With ``rv = gc.collect()``, ``!`` is output on verbosity level 1 when
``rv`` is non zero (i.e. when cyclic structures have been released),
``[``*rv*``]`` on higher verbosity levels and
a detailed cyclic garbage analysis on verbosity level 4+.
For details, see
`#133 <https://github.com/zopefoundation/zope.testrunner/pull/133>`_.
- Allow the filename for the logging configuration to be specified
via the envvar ``ZOPE_TESTRUNNER_LOG_INI``.
If not defined, the configuration continues to be locked for
in file ``log.ini`` of the current working directory.
Remember the logging configuration file in envvar
``ZOPE_TESTRUNNER_LOG_INI`` to allow spawned child processes
to recreate the logging configuration.
For details, see
`#134 <https://github.com/zopefoundation/zope.testrunner/pull/134>`_.
5.4.0 (2021-11-19)
==================
- Improve ``--help`` documentation for ``--package-path`` option
(`#121 <https://github.com/zopefoundation/zope.testrunner/pull/121>`_).
- Do not disable existing loggers during logsupport initialization
(`#120 <https://github.com/zopefoundation/zope.testrunner/pull/120>`_).
- Fix tests with testtools >= 2.5.0 (`#125
<https://github.com/zopefoundation/zope.testrunner/issues/125>`_).
- Add support for Python 3.10.
5.3.0 (2021-03-17)
==================
- Add support for Python 3.9.
- Fix `package init file missing` warning
(`#112 <https://github.com/zopefoundation/zope.testrunner/pull/112>`_).
- Make standard streams provide a `buffer` attribute on Python 3 when using
`--buffer` or testing under subunit.
5.2 (2020-06-29)
================
- Add support for Python 3.8.
- When a layer is run in a subprocess, read its stderr in a thread to avoid
a deadlock if its stderr output (containing failing and erroring test IDs)
overflows the capacity of a pipe (`#105
<https://github.com/zopefoundation/zope.testrunner/issues/105>`_).
5.1 (2019-10-19)
================
- Recover more gracefully when layer setUp or tearDown fails, producing
useful subunit output.
- Prevent a spurious warning from the ``--require-unique`` option if the
``--module`` option was not used.
- Add optional buffering of standard output and standard error during tests,
requested via the ``--buffer`` option or enabled by default for subunit.
- Fix incorrect failure counts in per-layer summary output, broken in 4.0.1.
5.0 (2019-03-19)
================
- Fix test failures and deprecation warnings occurring when using Python 3.8a1.
(`#89 <https://github.com/zopefoundation/zope.testrunner/pull/89>`_)
- Drop support for Python 3.4.
4.9.2 (2018-11-24)
==================
- Fix ``TypeError: a bytes-like object is required, not 'str'``
running tests in parallel on Python 3. See `issue 80
<https://github.com/zopefoundation/zope.testrunner/issues/80>`_.
4.9.1 (2018-11-21)
==================
- Fix AssertionError in _DummyThread.isAlive on Python 3 (`#81
<https://github.com/zopefoundation/zope.testrunner/issues/81>`_).
4.9 (2018-10-05)
================
- Drop support for Python 3.3.
- Add support for Python 3.7.
- Enable test coverage reporting on coveralls.io and in tox.ini.
- Host documentation at https://zopetestrunner.readthedocs.io
- Remove untested support for the ``--pychecker`` option. See
`issue 63 <https://github.com/zopefoundation/zope.testrunner/issues/63>`_.
- Update the command line interface to use ``argparse`` instead of
``optparse``. See `issue 61
<https://github.com/zopefoundation/zope.testrunner/issues/61>`_.
- Use ipdb instead of pdb for post-mortem debugging if available
(`#10 <https://github.com/zopefoundation/zope.testrunner/issues/10>`_).
- Add a --require-unique option to check for duplicate test IDs. See
`LP #682771
<https://bugs.launchpad.net/launchpad/+bug/682771>`_.
- Reintroduce optional support for ``subunit``, now with support for both
version 1 and version 2 of its protocol.
- Handle string in exception values when formatting chained exceptions.
(`#74 <https://github.com/zopefoundation/zope.testrunner/pull/74>`_)
4.8.1 (2017-11-12)
==================
- Enable ``DeprecationWarning`` earlier, when discovering test
modules. This lets warnings that are raised on import (such as those
produced by ``zope.deprecation.moved``) be reported. See `issue 57
<https://github.com/zopefoundation/zope.testrunner/issues/57>`_.
4.8.0 (2017-11-10)
==================
- Automatically enable ``DeprecationWarning`` when running tests. This
is recommended by the Python core developers and matches the
behaviour of the ``unittest`` module. This can be overridden with
Python command-line options (``-W``) or environment variables
(``PYTHONWARNINGS``). See `issue 54
<https://github.com/zopefoundation/zope.testrunner/issues/54>`_.
4.7.0 (2017-05-30)
==================
- Drop all support for ``subunit``.
4.6.0 (2016-12-28)
==================
- Make the ``subunit`` support purely optional: applications which have
been getting the dependencies via ``zope.testrunner`` should either add
``zope.testrunner[subunit]`` to their ``install_requires`` or else
depend directly on ``python-subunit``.
- New option ``--ignore-new-thread=<regexp>`` to suppress "New thread(s)"
warnings.
- Support Python 3.6.
4.5.1 (2016-06-20)
==================
- Fixed: Using the ``-j`` option to run tests in multiple processes
caused tests that used the ``multiprocessing`` package to hang
(because the testrunner replaced ``sys.stdin`` with an unclosable
object).
- Drop conditional dependency on ``unittest2`` (redundant after dropping
support for Python 2.6).
4.5.0 (2016-05-02)
==================
- Stop tests for all layers when test fails/errors when started with
-x/--stop-on-error
(`#37 <https://github.com/zopefoundation/zope.testrunner/pull/37>`_).
- Drop support for Python 2.6 and 3.2.
4.4.10 (2015-11-10)
===================
- Add support for Python 3.5
(`#31 <https://github.com/zopefoundation/zope.testrunner/pull/31>`_).
- Insert extra paths (from ``--path``) to the front of sys.argv
(`#32 <https://github.com/zopefoundation/zope.testrunner/issues/32>`_).
4.4.9 (2015-05-21)
==================
- When using ``-j``, parallelize all the tests, including the first test layer
(`#28 <https://github.com/zopefoundation/zope.testrunner/issues/28>`_).
4.4.8 (2015-05-01)
==================
- Support skipped tests in subunit output
(`#25 <https://github.com/zopefoundation/zope.testrunner/pull/25>`_).
- More efficient test filtering
(`#26 <https://github.com/zopefoundation/zope.testrunner/pull/26>`_).
4.4.7 (2015-04-02)
==================
- Work around a bug in PyPy3's curses module
(`#24 <https://github.com/zopefoundation/zope.testrunner/issues/24>`_).
4.4.6 (2015-01-21)
==================
- Restore support for instance-based test layers that regressed in 4.4.5
(`#20 <https://github.com/zopefoundation/zope.testrunner/pull/20>`_).
4.4.5 (2015-01-06)
==================
- Sort related layers close to each other to reduce the number of unnecessary
teardowns (fixes `#14
<https://github.com/zopefoundation/zope.testrunner/issues/14>`_).
- Run the unit test layer first (fixes `LP #497871
<https://bugs.launchpad.net/zope.testrunner/+bug/497871>`__).
4.4.4 (2014-12-27)
==================
- When looking for the right location of test code, start with longest
location paths first. This fixes problems with nested code locations.
4.4.3 (2014-03-19)
==================
- Added support for Python 3.4.
4.4.2 (2014-02-22)
==================
- Drop support for Python 3.1.
- Fix post-mortem debugging when a non-printable exception happens
(https://github.com/zopefoundation/zope.testrunner/issues/8).
4.4.1 (2013-07-10)
==================
- Updated ``boostrap.py`` to version 2.2.
- Fix nondeterministic test failures on Python 3.3
- Tear down layers after ``post_mortem`` debugging is finished.
- Fix tests that write to source directory, it might be read-only.
4.4.0 (2013-06-06)
==================
- Fix tests selection when the negative "!" pattern is used several times
(LP #1160965)
- Moved tests into a 'tests' subpackage.
- Made ``python -m zope.testrunner`` work again.
- Support 'skip' feature of unittest2 (which became the new unittest in Python
2.7).
- Better diagnostics when communication with subprocess fails
(https://github.com/zopefoundation/zope.testrunner/issues/5).
- Do not break subprocess execution when the test suite changes the working
directory (https://github.com/zopefoundation/zope.testrunner/issues/6).
- Count test module import errors as errors (LP #1026576).
4.3.3 (2013-03-03)
==================
- Running layers in sub-processes did not use to work when run via
``python setup.py ftest`` since it tried to run setup.py with all the
command line options. It now detects ``setup.py`` runs and we run the test
runner directly.
4.3.2 (2013-03-03)
==================
- Fix ``SkipLayers`` class in cases where the distribution specifies a
``test_suite`` value.
4.3.1 (2013-03-02)
==================
- Fixed a bug in the `ftest` command and added a test.
- Fixed a trivial test failure with Python 3 of the previous release.
4.3.0 (2013-03-02)
==================
- Expose `ftest` distutils command via an entry point.
- Added tests for ``zope.testrunner.eggsupport``.
4.2.0 (2013-02-12)
==================
- Dropped use of 2to3, rewrote source code to be compatible with all Python
versions. Introduced a dependency on `six`_.
4.1.1 (2013-02-08)
==================
- Dropped use of zope.fixers (LP: #1118877).
- Fixed tox test error reporting; fixed tests on Pythons 2.6, 3.1, 3.2, 3.3 and
PyPy 1.9.
- Fix --shuffle ordering on Python 3.2 to be the same as it was on older Python
versions.
- Fix --shuffle nondeterminism when multiple test layers are present.
Note: this will likely change the order of tests for the same --shuffle-seed.
- New option: --profile-directory. Use it in the test suite so that tests
executed by detox in parallel don't conflict.
- Use a temporary coverage directory in the test suite so that tests
executed by detox in parallel don't conflict.
- Fix --post-mortem (aka -D, --pdb) when a test module cannot be imported
or is invalid (LP #1119363).
4.1.0 (2013-02-07)
==================
- Replaced deprecated ``zope.interface.implements`` usage with equivalent
``zope.interface.implementer`` decorator.
- Dropped support for Python 2.4 and 2.5.
- Made StartUpFailure compatible with unittest.TextTestRunner() (LP #1118344).
4.0.4 (2011-10-25)
==================
- Work around sporadic timing-related issues in the subprocess buffering
tests. Thanks to Jonathan Ballet for the patch!
4.0.3 (2011-03-17)
==================
- Added back support for Python <= 2.6 which was broken in 4.0.2.
4.0.2 (2011-03-16)
==================
- Added back Python 3 support which was broken in 4.0.1.
- Fixed `Unexpected success`_ support by implementing the whole concept.
- Added support for the new __pycache__ directories in Python 3.2.
4.0.1 (2011-02-21)
==================
- LP #719369: An `Unexpected success`_ (concept introduced in Python 2.7) is
no longer handled as success but as failure. This is a workaround. The
whole unexpected success concept might be implemented later.
.. _`Unexpected success`: http://www.voidspace.org.uk/python/articles/unittest2.shtml#more-skipping
4.0.0 (2010-10-19)
==================
- Show more information about layers whose setup fails (LP #638153).
4.0.0b5 (2010-07-20)
====================
- Update fix for LP #221151 to a spelling compatible with Python 2.4.
- Timestamps are now always included in subunit output (r114849).
- LP #591309: fix a crash when subunit reports test failures containing
UTF8-encoded data.
4.0.0b4 (2010-06-23)
====================
- Package as a zipfile to work around Python 2.4 distutils bug (no
feature changes or bugfixes in ``zope.testrunner`` itself).
4.0.0b3 (2010-06-16)
====================
- LP #221151: keep ``unittest.TestCase.shortDescription`` happy by supplying
a ``_testMethodDoc`` attribute.
- LP #595052: keep the distribution installable under Python 2.4: its
distutils appears to munge the empty ``__init__.py`` file in the
``foo.bar`` egg used for testing into a directory.
- LP #580083: fix the ``bin/test`` script to run only tests from
``zope.testrunner``.
- LP #579019: When layers were run in parallel, their tearDown was
not called. Additionally, the first layer which was run in the main
thread did not have its tearDown called either.
4.0.0b2 (2010-05-03)
====================
- Having 'sampletests' in the MANIFEST.in gave warnings, but doesn't actually
seem to include any more files, so I removed it.
- Moved zope.testing.exceptions to zope.testrunner.exceptions. Now
zope.testrunner no longer requires zope.testing except for when running
its own tests.
4.0.0b1 (2010-04-29)
====================
- Initial release of the testrunner from zope.testrunner as its own module.
(Previously it was part of zope.testing.)
.. _six: http://pypi.python.org/pypi/six
Raw data
{
"_id": null,
"home_page": "https://github.com/zopefoundation/zope.testrunner",
"name": "zope.testrunner",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "",
"author": "Zope Foundation and Contributors",
"author_email": "zope-dev@zope.dev",
"download_url": "https://files.pythonhosted.org/packages/61/c0/3b4e89cde05380a1d9e60d197f01ecee1c58b00a430e0f2a95e17102b54e/zope.testrunner-6.4.tar.gz",
"platform": null,
"description": "=================\n zope.testrunner\n=================\n\n.. image:: https://img.shields.io/pypi/v/zope.testrunner.svg\n :target: https://pypi.org/project/zope.testrunner/\n :alt: Latest release\n\n.. image:: https://img.shields.io/pypi/pyversions/zope.testrunner.svg\n :target: https://pypi.org/project/zope.testrunner/\n :alt: Supported Python versions\n\n.. image:: https://github.com/zopefoundation/zope.testrunner/actions/workflows/tests.yml/badge.svg\n :target: https://github.com/zopefoundation/zope.testrunner/actions/workflows/tests.yml\n\n.. image:: https://coveralls.io/repos/github/zopefoundation/zope.testrunner/badge.svg?branch=master\n :target: https://coveralls.io/github/zopefoundation/zope.testrunner?branch=master\n\n.. image:: https://readthedocs.org/projects/zopetestrunner/badge/?version=latest\n :target: https://zopetestrunner.readthedocs.io/en/latest/?badge=latest\n :alt: Documentation Status\n\n.. contents::\n\nThis package provides a flexible test runner with layer support.\n\nDetailed documentation is hosted at https://zopetestrunner.readthedocs.io\n\n\n=======================\n Using zope.testrunner\n=======================\n\n.. IMPORTANT: This document is included in the long_description for\n rendering on PyPI, so it cannot use Sphinx-only features.\n\nInstallation\n============\n\nBuildout-based projects\n-----------------------\n\nzope.testrunner is often used for projects that use buildout_::\n\n [buildout]\n develop = .\n parts = ... test ...\n\n [test]\n recipe = zc.recipe.testrunner\n eggs = mypackage\n\nThe usual buildout process ::\n\n python bootstrap.py\n bin/buildout\n\ncreates a ``bin/test`` script that will run the tests for *mypackage*.\n\n.. tip::\n\n zc.recipe.testrunner_ takes care to specify the right\n ``--test-path`` option in the generated script. You can add\n other options (such as ``--tests-pattern``) too; check\n zc.recipe.testrunner_'s documentation for details.\n\n\nVirtualenv-based projects\n-------------------------\n\n``pip install zope.testrunner`` and you'll get a ``zope-testrunner``\nscript. Run your tests with ::\n\n zope-testrunner --test-path=path/to/your/source/tree\n\nYour source code needs to be available for the testrunner to import,\nso you need to run ``python setup.py install`` or ``pip install -e\n.`` into the same virtualenv_.\n\n\nSome useful command-line options to get you started\n===================================================\n\n-p show a progress indicator\n-v increase verbosity\n-c colorize the output\n-t test specify test names (one or more regexes)\n-m module specify test modules (one or more regexes)\n-s package specify test packages (one or more regexes)\n--list-tests show names of tests instead of running them\n-x stop on first error or failure\n-D, --pdb enable post-mortem debugging of test failures\n--xml path generate XML reports to be written at the given path\n--help show *all* command-line options (there are many more!)\n\nFor example ::\n\n bin/test -pvc -m test_foo -t TestBar\n\nruns all TestBar tests from a module called test_foo.py.\n\nWriting tests\n=============\n\n``zope.testrunner`` expects to find your tests inside your package\ndirectory, in a subpackage or module named ``tests``. Test modules\nin a test subpackage should be named ``test*.py``.\n\n.. tip::\n\n You can change these assumptions with ``--tests-pattern`` and\n ``--test-file-pattern`` test runner options.\n\nTests themselves should be classes inheriting from\n``unittest.TestCase``, and if you wish to use doctests, please tell\nthe test runner where to find them and what options to use for them\nin by supplying a function named ``test_suite``.\n\nExample::\n\n import unittest\n import doctest\n\n class TestArithmetic(unittest.TestCase):\n\n def test_two_plus_two(self):\n self.assertEqual(2 + 2, 4)\n\n\n def doctest_string_formatting():\n \"\"\"Test Python string formatting\n\n >>> print('{} + {}'.format(2, 2))\n 2 + 2\n\n \"\"\"\n\n def test_suite():\n return unittest.TestSuite([\n unittest.defaultTestLoader.loadTestsFromName(__name__),\n doctest.DocTestSuite(),\n doctest.DocFileSuite('../README.txt',\n optionflags=doctest.ELLIPSIS),\n ])\n\n\n\nTest grouping\n=============\n\nIn addition to per-package and per-module filtering, zope.testrunner\nhas other mechanisms for grouping tests:\n\n* **layers** allow you to have shared setup/teardown code to be used\n by a group of tests, that is executed only once, and not for each\n test. Layers are orthogonal to the usual package/module structure\n and are specified by setting the ``layer`` attribute on test\n suites.\n\n* **levels** allow you to group slow-running tests and not run them\n by default. They're specified by setting the ``level`` attribute\n on test suites to an int.\n\n\nOther features\n==============\n\nzope.testrunner can profile your tests, measure test coverage,\ncheck for memory leaks, integrate with subunit_, shuffle the\ntest execution order, and run multiple tests in parallel.\n\n.. _buildout: https://buildout.readthedocs.io\n.. _virtualenv: https://virtualenv.pypa.io/\n.. _zc.recipe.testrunner: https://pypi.python.org/pypi/zc.recipe.testrunner\n.. _subunit: https://pypi.python.org/pypi/python-subunit\n\n\n===========================\n zope.testrunner Changelog\n===========================\n\n6.4 (2024-02-27)\n================\n\n- Add PEP 440 support (implicit namespaces).\n (`#160 <https://github.com/zopefoundation/zope.testrunner/issues/160>`_)\n\n\n6.3.1 (2024-02-12)\n==================\n\n- Fix XML tests when running in distribution resp. separately.\n (`#163 <https://github.com/zopefoundation/zope.testrunner/issues/163>`_)\n\n\n6.3 (2024-02-07)\n================\n\n- Exit cleanly when using the test runner ``--version`` argument.\n (`#102 <https://github.com/zopefoundation/zope.testrunner/issues/102>`_)\n\n- Add new ``--xml <path>`` option to write JUnit-like XML reports.\n Code comes from ``collective.xmltestreport``, but be aware that here ``--xml``\n is not a boolean, but expects a path!\n (`#148 <https://github.com/zopefoundation/zope.testrunner/issues/148>`_).\n\n- Add support for Python 3.13a3.\n\n\n6.2.1 (2023-12-22)\n==================\n\n- Work around Python 3.12.1+ no longer calling ``startTest`` for skipped tests\n (`#157 <https://github.com/zopefoundation/zope.testrunner/issues/157>`_).\n\n\n6.2 (2023-11-08)\n================\n\n- Add support for Python 3.12.\n\n- Update code and tests to ``python-subunit >= 1.4.3`` thus requiring at least\n this version.\n\n\n6.1 (2023-08-26)\n================\n\n- Add preliminary support for Python 3.12b4.\n (`#149 <https://github.com/zopefoundation/zope.testrunner/issues/149>`_)\n\n\n6.0 (2023-03-28)\n================\n\n- Drop support for Python 2.7, 3.5, 3.6.\n\n\n5.6 (2022-12-09)\n================\n\n- Add support for Python 3.11.\n\n- Inline a small part of ``random.Random.shuffle`` which was deprecated in\n Python 3.9 and removed in 3.11 (`#119\n <https://github.com/zopefoundation/zope.testrunner/issues/119>`_).\n\n- Don't trigger post mortem debugger for skipped tests. ( `#141\n <https://github.com/zopefoundation/zope.testrunner/issues/141>`_).\n\n\n5.5.1 (2022-09-07)\n==================\n\n- Fix: let ``--at-level=level`` with ``level <= 0`` run the tests\n at all levels (rather than at no level)\n `#138 <https://github.com/zopefoundation/zope.testrunner/issues/138>`_.\n\n\n5.5 (2022-06-24)\n================\n\n- Use ``sys._current_frames`` (rather than ``threading.enumerate``)\n as base for new thread detection, fixes\n `#130 <https://github.com/zopefoundation/zope.testrunner/issues/130>`_.\n\n- New option ``--gc-after-test``. It calls for a garbage collection\n after each test and can be used to track down ``ResourceWarning``s\n and cyclic garbage.\n With ``rv = gc.collect()``, ``!`` is output on verbosity level 1 when\n ``rv`` is non zero (i.e. when cyclic structures have been released),\n ``[``*rv*``]`` on higher verbosity levels and\n a detailed cyclic garbage analysis on verbosity level 4+.\n For details, see\n `#133 <https://github.com/zopefoundation/zope.testrunner/pull/133>`_.\n\n- Allow the filename for the logging configuration to be specified\n via the envvar ``ZOPE_TESTRUNNER_LOG_INI``.\n If not defined, the configuration continues to be locked for\n in file ``log.ini`` of the current working directory.\n Remember the logging configuration file in envvar\n ``ZOPE_TESTRUNNER_LOG_INI`` to allow spawned child processes\n to recreate the logging configuration.\n For details, see\n `#134 <https://github.com/zopefoundation/zope.testrunner/pull/134>`_.\n\n\n5.4.0 (2021-11-19)\n==================\n\n- Improve ``--help`` documentation for ``--package-path`` option\n (`#121 <https://github.com/zopefoundation/zope.testrunner/pull/121>`_).\n\n- Do not disable existing loggers during logsupport initialization\n (`#120 <https://github.com/zopefoundation/zope.testrunner/pull/120>`_).\n\n- Fix tests with testtools >= 2.5.0 (`#125\n <https://github.com/zopefoundation/zope.testrunner/issues/125>`_).\n\n- Add support for Python 3.10.\n\n\n5.3.0 (2021-03-17)\n==================\n\n- Add support for Python 3.9.\n\n- Fix `package init file missing` warning\n (`#112 <https://github.com/zopefoundation/zope.testrunner/pull/112>`_).\n\n- Make standard streams provide a `buffer` attribute on Python 3 when using\n `--buffer` or testing under subunit.\n\n\n5.2 (2020-06-29)\n================\n\n- Add support for Python 3.8.\n\n- When a layer is run in a subprocess, read its stderr in a thread to avoid\n a deadlock if its stderr output (containing failing and erroring test IDs)\n overflows the capacity of a pipe (`#105\n <https://github.com/zopefoundation/zope.testrunner/issues/105>`_).\n\n\n5.1 (2019-10-19)\n================\n\n- Recover more gracefully when layer setUp or tearDown fails, producing\n useful subunit output.\n\n- Prevent a spurious warning from the ``--require-unique`` option if the\n ``--module`` option was not used.\n\n- Add optional buffering of standard output and standard error during tests,\n requested via the ``--buffer`` option or enabled by default for subunit.\n\n- Fix incorrect failure counts in per-layer summary output, broken in 4.0.1.\n\n\n5.0 (2019-03-19)\n================\n\n- Fix test failures and deprecation warnings occurring when using Python 3.8a1.\n (`#89 <https://github.com/zopefoundation/zope.testrunner/pull/89>`_)\n\n- Drop support for Python 3.4.\n\n\n4.9.2 (2018-11-24)\n==================\n\n- Fix ``TypeError: a bytes-like object is required, not 'str'``\n running tests in parallel on Python 3. See `issue 80\n <https://github.com/zopefoundation/zope.testrunner/issues/80>`_.\n\n\n4.9.1 (2018-11-21)\n==================\n\n- Fix AssertionError in _DummyThread.isAlive on Python 3 (`#81\n <https://github.com/zopefoundation/zope.testrunner/issues/81>`_).\n\n\n4.9 (2018-10-05)\n================\n\n- Drop support for Python 3.3.\n\n- Add support for Python 3.7.\n\n- Enable test coverage reporting on coveralls.io and in tox.ini.\n\n- Host documentation at https://zopetestrunner.readthedocs.io\n\n- Remove untested support for the ``--pychecker`` option. See\n `issue 63 <https://github.com/zopefoundation/zope.testrunner/issues/63>`_.\n\n- Update the command line interface to use ``argparse`` instead of\n ``optparse``. See `issue 61\n <https://github.com/zopefoundation/zope.testrunner/issues/61>`_.\n\n- Use ipdb instead of pdb for post-mortem debugging if available\n (`#10 <https://github.com/zopefoundation/zope.testrunner/issues/10>`_).\n\n- Add a --require-unique option to check for duplicate test IDs. See\n `LP #682771\n <https://bugs.launchpad.net/launchpad/+bug/682771>`_.\n\n- Reintroduce optional support for ``subunit``, now with support for both\n version 1 and version 2 of its protocol.\n\n- Handle string in exception values when formatting chained exceptions.\n (`#74 <https://github.com/zopefoundation/zope.testrunner/pull/74>`_)\n\n\n4.8.1 (2017-11-12)\n==================\n\n- Enable ``DeprecationWarning`` earlier, when discovering test\n modules. This lets warnings that are raised on import (such as those\n produced by ``zope.deprecation.moved``) be reported. See `issue 57\n <https://github.com/zopefoundation/zope.testrunner/issues/57>`_.\n\n\n4.8.0 (2017-11-10)\n==================\n\n- Automatically enable ``DeprecationWarning`` when running tests. This\n is recommended by the Python core developers and matches the\n behaviour of the ``unittest`` module. This can be overridden with\n Python command-line options (``-W``) or environment variables\n (``PYTHONWARNINGS``). See `issue 54\n <https://github.com/zopefoundation/zope.testrunner/issues/54>`_.\n\n4.7.0 (2017-05-30)\n==================\n\n- Drop all support for ``subunit``.\n\n\n4.6.0 (2016-12-28)\n==================\n\n- Make the ``subunit`` support purely optional: applications which have\n been getting the dependencies via ``zope.testrunner`` should either add\n ``zope.testrunner[subunit]`` to their ``install_requires`` or else\n depend directly on ``python-subunit``.\n\n- New option ``--ignore-new-thread=<regexp>`` to suppress \"New thread(s)\"\n warnings.\n\n- Support Python 3.6.\n\n\n4.5.1 (2016-06-20)\n==================\n\n- Fixed: Using the ``-j`` option to run tests in multiple processes\n caused tests that used the ``multiprocessing`` package to hang\n (because the testrunner replaced ``sys.stdin`` with an unclosable\n object).\n\n- Drop conditional dependency on ``unittest2`` (redundant after dropping\n support for Python 2.6).\n\n\n4.5.0 (2016-05-02)\n==================\n\n- Stop tests for all layers when test fails/errors when started with\n -x/--stop-on-error\n (`#37 <https://github.com/zopefoundation/zope.testrunner/pull/37>`_).\n\n- Drop support for Python 2.6 and 3.2.\n\n\n4.4.10 (2015-11-10)\n===================\n\n- Add support for Python 3.5\n (`#31 <https://github.com/zopefoundation/zope.testrunner/pull/31>`_).\n\n- Insert extra paths (from ``--path``) to the front of sys.argv\n (`#32 <https://github.com/zopefoundation/zope.testrunner/issues/32>`_).\n\n\n4.4.9 (2015-05-21)\n==================\n\n- When using ``-j``, parallelize all the tests, including the first test layer\n (`#28 <https://github.com/zopefoundation/zope.testrunner/issues/28>`_).\n\n\n4.4.8 (2015-05-01)\n==================\n\n- Support skipped tests in subunit output\n (`#25 <https://github.com/zopefoundation/zope.testrunner/pull/25>`_).\n\n- More efficient test filtering\n (`#26 <https://github.com/zopefoundation/zope.testrunner/pull/26>`_).\n\n\n4.4.7 (2015-04-02)\n==================\n\n- Work around a bug in PyPy3's curses module\n (`#24 <https://github.com/zopefoundation/zope.testrunner/issues/24>`_).\n\n\n4.4.6 (2015-01-21)\n==================\n\n- Restore support for instance-based test layers that regressed in 4.4.5\n (`#20 <https://github.com/zopefoundation/zope.testrunner/pull/20>`_).\n\n\n4.4.5 (2015-01-06)\n==================\n\n- Sort related layers close to each other to reduce the number of unnecessary\n teardowns (fixes `#14\n <https://github.com/zopefoundation/zope.testrunner/issues/14>`_).\n\n- Run the unit test layer first (fixes `LP #497871\n <https://bugs.launchpad.net/zope.testrunner/+bug/497871>`__).\n\n\n4.4.4 (2014-12-27)\n==================\n\n- When looking for the right location of test code, start with longest\n location paths first. This fixes problems with nested code locations.\n\n\n4.4.3 (2014-03-19)\n==================\n\n- Added support for Python 3.4.\n\n\n4.4.2 (2014-02-22)\n==================\n\n- Drop support for Python 3.1.\n\n- Fix post-mortem debugging when a non-printable exception happens\n (https://github.com/zopefoundation/zope.testrunner/issues/8).\n\n\n4.4.1 (2013-07-10)\n==================\n\n- Updated ``boostrap.py`` to version 2.2.\n\n- Fix nondeterministic test failures on Python 3.3\n\n- Tear down layers after ``post_mortem`` debugging is finished.\n\n- Fix tests that write to source directory, it might be read-only.\n\n\n4.4.0 (2013-06-06)\n==================\n\n- Fix tests selection when the negative \"!\" pattern is used several times\n (LP #1160965)\n\n- Moved tests into a 'tests' subpackage.\n\n- Made ``python -m zope.testrunner`` work again.\n\n- Support 'skip' feature of unittest2 (which became the new unittest in Python\n 2.7).\n\n- Better diagnostics when communication with subprocess fails\n (https://github.com/zopefoundation/zope.testrunner/issues/5).\n\n- Do not break subprocess execution when the test suite changes the working\n directory (https://github.com/zopefoundation/zope.testrunner/issues/6).\n\n- Count test module import errors as errors (LP #1026576).\n\n\n4.3.3 (2013-03-03)\n==================\n\n- Running layers in sub-processes did not use to work when run via\n ``python setup.py ftest`` since it tried to run setup.py with all the\n command line options. It now detects ``setup.py`` runs and we run the test\n runner directly.\n\n\n4.3.2 (2013-03-03)\n==================\n\n- Fix ``SkipLayers`` class in cases where the distribution specifies a\n ``test_suite`` value.\n\n\n4.3.1 (2013-03-02)\n==================\n\n- Fixed a bug in the `ftest` command and added a test.\n\n- Fixed a trivial test failure with Python 3 of the previous release.\n\n\n4.3.0 (2013-03-02)\n==================\n\n- Expose `ftest` distutils command via an entry point.\n\n- Added tests for ``zope.testrunner.eggsupport``.\n\n\n4.2.0 (2013-02-12)\n==================\n\n- Dropped use of 2to3, rewrote source code to be compatible with all Python\n versions. Introduced a dependency on `six`_.\n\n\n4.1.1 (2013-02-08)\n==================\n\n- Dropped use of zope.fixers (LP: #1118877).\n\n- Fixed tox test error reporting; fixed tests on Pythons 2.6, 3.1, 3.2, 3.3 and\n PyPy 1.9.\n\n- Fix --shuffle ordering on Python 3.2 to be the same as it was on older Python\n versions.\n\n- Fix --shuffle nondeterminism when multiple test layers are present.\n Note: this will likely change the order of tests for the same --shuffle-seed.\n\n- New option: --profile-directory. Use it in the test suite so that tests\n executed by detox in parallel don't conflict.\n\n- Use a temporary coverage directory in the test suite so that tests\n executed by detox in parallel don't conflict.\n\n- Fix --post-mortem (aka -D, --pdb) when a test module cannot be imported\n or is invalid (LP #1119363).\n\n\n4.1.0 (2013-02-07)\n==================\n\n- Replaced deprecated ``zope.interface.implements`` usage with equivalent\n ``zope.interface.implementer`` decorator.\n\n- Dropped support for Python 2.4 and 2.5.\n\n- Made StartUpFailure compatible with unittest.TextTestRunner() (LP #1118344).\n\n\n4.0.4 (2011-10-25)\n==================\n\n- Work around sporadic timing-related issues in the subprocess buffering\n tests. Thanks to Jonathan Ballet for the patch!\n\n\n4.0.3 (2011-03-17)\n==================\n\n- Added back support for Python <= 2.6 which was broken in 4.0.2.\n\n\n4.0.2 (2011-03-16)\n==================\n\n- Added back Python 3 support which was broken in 4.0.1.\n\n- Fixed `Unexpected success`_ support by implementing the whole concept.\n\n- Added support for the new __pycache__ directories in Python 3.2.\n\n\n4.0.1 (2011-02-21)\n==================\n\n- LP #719369: An `Unexpected success`_ (concept introduced in Python 2.7) is\n no longer handled as success but as failure. This is a workaround. The\n whole unexpected success concept might be implemented later.\n\n.. _`Unexpected success`: http://www.voidspace.org.uk/python/articles/unittest2.shtml#more-skipping\n\n\n4.0.0 (2010-10-19)\n==================\n\n- Show more information about layers whose setup fails (LP #638153).\n\n\n4.0.0b5 (2010-07-20)\n====================\n\n- Update fix for LP #221151 to a spelling compatible with Python 2.4.\n\n- Timestamps are now always included in subunit output (r114849).\n\n- LP #591309: fix a crash when subunit reports test failures containing\n UTF8-encoded data.\n\n\n4.0.0b4 (2010-06-23)\n====================\n\n- Package as a zipfile to work around Python 2.4 distutils bug (no\n feature changes or bugfixes in ``zope.testrunner`` itself).\n\n\n4.0.0b3 (2010-06-16)\n====================\n\n- LP #221151: keep ``unittest.TestCase.shortDescription`` happy by supplying\n a ``_testMethodDoc`` attribute.\n\n- LP #595052: keep the distribution installable under Python 2.4: its\n distutils appears to munge the empty ``__init__.py`` file in the\n ``foo.bar`` egg used for testing into a directory.\n\n- LP #580083: fix the ``bin/test`` script to run only tests from\n ``zope.testrunner``.\n\n- LP #579019: When layers were run in parallel, their tearDown was\n not called. Additionally, the first layer which was run in the main\n thread did not have its tearDown called either.\n\n\n4.0.0b2 (2010-05-03)\n====================\n\n- Having 'sampletests' in the MANIFEST.in gave warnings, but doesn't actually\n seem to include any more files, so I removed it.\n\n- Moved zope.testing.exceptions to zope.testrunner.exceptions. Now\n zope.testrunner no longer requires zope.testing except for when running\n its own tests.\n\n\n4.0.0b1 (2010-04-29)\n====================\n\n- Initial release of the testrunner from zope.testrunner as its own module.\n (Previously it was part of zope.testing.)\n\n\n.. _six: http://pypi.python.org/pypi/six\n",
"bugtrack_url": null,
"license": "ZPL 2.1",
"summary": "Zope testrunner script.",
"version": "6.4",
"project_urls": {
"Homepage": "https://github.com/zopefoundation/zope.testrunner"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e301298e83a81c4d95587998d96181475befd617ad13ee99a2a2a1dbc6dee8cd",
"md5": "21b0ad96be37bbd442e58389d427dab0",
"sha256": "7d3a3cc5b2dba944d326ba7c742c45b86ac8b0b5ba2c81640b56ad26f21cb8c9"
},
"downloads": -1,
"filename": "zope.testrunner-6.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "21b0ad96be37bbd442e58389d427dab0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 231688,
"upload_time": "2024-02-27T10:01:49",
"upload_time_iso_8601": "2024-02-27T10:01:49.013647Z",
"url": "https://files.pythonhosted.org/packages/e3/01/298e83a81c4d95587998d96181475befd617ad13ee99a2a2a1dbc6dee8cd/zope.testrunner-6.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "61c03b4e89cde05380a1d9e60d197f01ecee1c58b00a430e0f2a95e17102b54e",
"md5": "f0b6d0e5ed971866f280876184499e70",
"sha256": "0b859fc74d6f2b6c5dd8adf9dee4ec7405f73f290ecab2570ac636f8361228c8"
},
"downloads": -1,
"filename": "zope.testrunner-6.4.tar.gz",
"has_sig": false,
"md5_digest": "f0b6d0e5ed971866f280876184499e70",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 159953,
"upload_time": "2024-02-27T10:01:53",
"upload_time_iso_8601": "2024-02-27T10:01:53.118714Z",
"url": "https://files.pythonhosted.org/packages/61/c0/3b4e89cde05380a1d9e60d197f01ecee1c58b00a430e0f2a95e17102b54e/zope.testrunner-6.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-27 10:01:53",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "zopefoundation",
"github_project": "zope.testrunner",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "zope.testrunner"
}
JSON
