mutmut - python mutation tester
===============================
.. image:: https://github.com/boxed/mutmut/actions/workflows/tests.yml/badge.svg
:target: https://github.com/boxed/mutmut/actions/workflows/tests.yml
.. image:: https://readthedocs.org/projects/mutmut/badge/?version=latest
:target: https://mutmut.readthedocs.io/en/latest/?badge=latest
:alt: Documentation Status
Mutmut is a mutation testing system for Python, with a strong focus on ease
of use. If you don't know what mutation testing is try starting with
`this article <https://kodare.net/2016/12/01/mutmut-a-python-mutation-testing-system.html>`_.
Some highlight features:
- Found mutants can be applied on disk with a simple command making it very
easy to work with the results
- Remembers work that has been done, so you can work incrementally
- Knows which tests to execute, speeding up mutation testing
- Interactive terminal based UI
- Parallel and fast execution
.. image:: browse_screenshot.png
If you want to mutate code outside of functions, you can try using mutmut 2,
which has a different execution model than mutmut 3+.
Requirements
------------
Mutmut must be run on a system with `fork` support. This means that if you want
to run on windows, you must run inside WSL.
Install and run
---------------
You can get started with a simple:
.. code-block:: console
pip install mutmut
mutmut run
This will by run pytest on tests in the "tests" or "test" folder and
it will try to figure out where the code to mutate is.
.. code-block:: ini
[mutmut]
paths_to_mutate=src/
tests_dir=tests/
You can stop the mutation run at any time and mutmut will restart where you
left off. It will continue where it left off, and re-test functions that were
modified since last run.
To work with the results, use `mutmut browse` where you can see the mutants,
retest them when you've updated your tests.
You can also write a mutant to disk from the `browse` interface, or via
`mutmut apply <mutant>`. You should **REALLY** have the file you mutate under
source code control and committed before you apply a mutant!
Wildcards for testing mutants
-----------------------------
Unix filename pattern matching style on mutants is supported. Example:
.. code-block:: console
mutmut run "my_module*"
mutmut run "my_module.my_function*"
In the `browse` TUI you can press `f` to retest a function, and `m` to retest
an entire module.
"also copy" files
-----------------
To run the full test suite some files are often needed above the tests and the
source. You can configure to copy extra files that you need by adding
directories and files to `also_copy` in your `setup.cfg`:
.. code-block:: ini
also_copy=
iommi/snapshots/
conftest.py
Limit stack depth
-----------------
In big code bases some functions are called incidentally by huge swaths of the
codebase, but you really don't want tests that hit those executions to count
for mutation testing purposes. Incidentally tested functions lead to slow
mutation testing as hundreds of tests can be checked for things that should
have clean and fast unit tests, and it leads to bad test suites as any
introduced bug in those base functions will lead to many tests that fail which
are hard to understand how they relate to the function with the change.
You can configure mutmut to only count a test as being relevant for a function
if the stack depth from the test to the function is below some limit. In your
`setup.cfg` add:
.. code-block:: ini
max_stack_depth=8
A lower value will increase mutation speed and lead to more localized tests,
but will also lead to more surviving mutants that would otherwise have been
caught.
Exclude files from mutation
---------------------------
You can exclude files from mutation in `setup.cfg`:
.. code-block::
do_not_mutate=
*__tests.py
Whitelisting
------------
You can mark lines like this:
.. code-block:: python
some_code_here() # pragma: no mutate
to stop mutation on those lines. Some cases we've found where you need to
whitelist lines are:
- The version string on your library. You really shouldn't have a test for this :P
- Optimizing break instead of continue. The code runs fine when mutating break
to continue, but it's slower.
Example mutations
-----------------
- Integer literals are changed by adding 1. So 0 becomes 1, 5 becomes 6, etc.
- `<` is changed to `<=`
- break is changed to continue and vice versa
In general the idea is that the mutations should be as subtle as possible.
See `__init__.py` for the full list.
Workflow
--------
This section describes how to work with mutmut to enhance your test suite.
1. Run mutmut with `mutmut run`. A full run is preferred but if you're just
getting started you can exit in the middle and start working with what you
have found so far.
2. Show the mutants with `mutmut browse`
3. Find a mutant you want to work on and write a test to try to kill it.
4. Press `r` to rerun the mutant and see if you successfully managed to kill it.
Mutmut keeps the data of what it has done and the mutants in the `mutants/`
directory.If you want to make sure you run a full mutmut run you can delete
this directory to start from scratch.
Raw data
{
"_id": null,
"home_page": "https://github.com/boxed/mutmut",
"name": "mutmut",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "mutmut mutant mutation test testing",
"author": "Anders Hovm\u00f6ller",
"author_email": "boxed@killingar.net",
"download_url": "https://files.pythonhosted.org/packages/83/54/9c9e7bcc7ebf07eef023945749fa73652f80d5521391f656a535c4362ebe/mutmut-3.2.1.tar.gz",
"platform": null,
"description": "mutmut - python mutation tester\n===============================\n\n.. image:: https://github.com/boxed/mutmut/actions/workflows/tests.yml/badge.svg\n :target: https://github.com/boxed/mutmut/actions/workflows/tests.yml\n\n.. image:: https://readthedocs.org/projects/mutmut/badge/?version=latest\n :target: https://mutmut.readthedocs.io/en/latest/?badge=latest\n :alt: Documentation Status\n\n\nMutmut is a mutation testing system for Python, with a strong focus on ease\nof use. If you don't know what mutation testing is try starting with\n`this article <https://kodare.net/2016/12/01/mutmut-a-python-mutation-testing-system.html>`_.\n\nSome highlight features:\n\n- Found mutants can be applied on disk with a simple command making it very\n easy to work with the results\n- Remembers work that has been done, so you can work incrementally\n- Knows which tests to execute, speeding up mutation testing\n- Interactive terminal based UI\n- Parallel and fast execution\n\n.. image:: browse_screenshot.png\n\n\nIf you want to mutate code outside of functions, you can try using mutmut 2,\nwhich has a different execution model than mutmut 3+.\n\n\nRequirements\n------------\n\nMutmut must be run on a system with `fork` support. This means that if you want\nto run on windows, you must run inside WSL.\n\n\n\nInstall and run\n---------------\n\nYou can get started with a simple:\n\n.. code-block:: console\n\n pip install mutmut\n mutmut run\n\nThis will by run pytest on tests in the \"tests\" or \"test\" folder and\nit will try to figure out where the code to mutate is.\n\n\n\n.. code-block:: ini\n\n [mutmut]\n paths_to_mutate=src/\n tests_dir=tests/\n\nYou can stop the mutation run at any time and mutmut will restart where you\nleft off. It will continue where it left off, and re-test functions that were\nmodified since last run.\n\nTo work with the results, use `mutmut browse` where you can see the mutants,\nretest them when you've updated your tests.\n\nYou can also write a mutant to disk from the `browse` interface, or via\n`mutmut apply <mutant>`. You should **REALLY** have the file you mutate under\nsource code control and committed before you apply a mutant!\n\n\nWildcards for testing mutants\n-----------------------------\n\nUnix filename pattern matching style on mutants is supported. Example:\n\n.. code-block:: console\n\n mutmut run \"my_module*\"\n mutmut run \"my_module.my_function*\"\n\nIn the `browse` TUI you can press `f` to retest a function, and `m` to retest\nan entire module.\n\n\n\"also copy\" files\n-----------------\n\nTo run the full test suite some files are often needed above the tests and the\nsource. You can configure to copy extra files that you need by adding\ndirectories and files to `also_copy` in your `setup.cfg`:\n\n.. code-block:: ini\n\n also_copy=\n iommi/snapshots/\n conftest.py\n\n\nLimit stack depth\n-----------------\n\nIn big code bases some functions are called incidentally by huge swaths of the\ncodebase, but you really don't want tests that hit those executions to count\nfor mutation testing purposes. Incidentally tested functions lead to slow\nmutation testing as hundreds of tests can be checked for things that should\nhave clean and fast unit tests, and it leads to bad test suites as any\nintroduced bug in those base functions will lead to many tests that fail which\nare hard to understand how they relate to the function with the change.\n\nYou can configure mutmut to only count a test as being relevant for a function\nif the stack depth from the test to the function is below some limit. In your\n`setup.cfg` add:\n\n.. code-block:: ini\n\n max_stack_depth=8\n\nA lower value will increase mutation speed and lead to more localized tests,\nbut will also lead to more surviving mutants that would otherwise have been\ncaught.\n\n\nExclude files from mutation\n---------------------------\n\nYou can exclude files from mutation in `setup.cfg`:\n\n.. code-block::\n\n do_not_mutate=\n *__tests.py\n\n\nWhitelisting\n------------\n\nYou can mark lines like this:\n\n.. code-block:: python\n\n some_code_here() # pragma: no mutate\n\nto stop mutation on those lines. Some cases we've found where you need to\nwhitelist lines are:\n\n- The version string on your library. You really shouldn't have a test for this :P\n- Optimizing break instead of continue. The code runs fine when mutating break\n to continue, but it's slower.\n\n\nExample mutations\n-----------------\n\n- Integer literals are changed by adding 1. So 0 becomes 1, 5 becomes 6, etc.\n- `<` is changed to `<=`\n- break is changed to continue and vice versa\n\nIn general the idea is that the mutations should be as subtle as possible.\nSee `__init__.py` for the full list.\n\n\nWorkflow\n--------\n\nThis section describes how to work with mutmut to enhance your test suite.\n\n1. Run mutmut with `mutmut run`. A full run is preferred but if you're just\n getting started you can exit in the middle and start working with what you\n have found so far.\n2. Show the mutants with `mutmut browse`\n3. Find a mutant you want to work on and write a test to try to kill it.\n4. Press `r` to rerun the mutant and see if you successfully managed to kill it.\n\nMutmut keeps the data of what it has done and the mutants in the `mutants/`\ndirectory.If you want to make sure you run a full mutmut run you can delete\nthis directory to start from scratch.\n",
"bugtrack_url": null,
"license": "BSD",
"summary": "mutation testing for Python 3",
"version": "3.2.1",
"project_urls": {
"Homepage": "https://github.com/boxed/mutmut"
},
"split_keywords": [
"mutmut",
"mutant",
"mutation",
"test",
"testing"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5576f6f74dc05d4a7716729ed4c3f9fb39587fa2ae5d7c74d21b11fd370f175d",
"md5": "1b72fa531dc4bef7f2efe75cddd5637c",
"sha256": "d9f64a4fc5e4d5c9e9fe73f11c4c54b0939bb84eb2f15a0c4a02f3231b76d622"
},
"downloads": -1,
"filename": "mutmut-3.2.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1b72fa531dc4bef7f2efe75cddd5637c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 22027,
"upload_time": "2024-11-13T07:27:42",
"upload_time_iso_8601": "2024-11-13T07:27:42.270130Z",
"url": "https://files.pythonhosted.org/packages/55/76/f6f74dc05d4a7716729ed4c3f9fb39587fa2ae5d7c74d21b11fd370f175d/mutmut-3.2.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "83549c9e7bcc7ebf07eef023945749fa73652f80d5521391f656a535c4362ebe",
"md5": "d6c8f199c79228606745635fcc1537bd",
"sha256": "76cfed66dbe4d5502e66583e4e0c6c5bbf07399e20a5ea544d2462837de7bf92"
},
"downloads": -1,
"filename": "mutmut-3.2.1.tar.gz",
"has_sig": false,
"md5_digest": "d6c8f199c79228606745635fcc1537bd",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 115486,
"upload_time": "2024-11-13T07:27:44",
"upload_time_iso_8601": "2024-11-13T07:27:44.237575Z",
"url": "https://files.pythonhosted.org/packages/83/54/9c9e7bcc7ebf07eef023945749fa73652f80d5521391f656a535c4362ebe/mutmut-3.2.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-13 07:27:44",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "boxed",
"github_project": "mutmut",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"test_requirements": [],
"tox": true,
"lcname": "mutmut"
}
JSON
