| Name | pytest-checklist JSON |
| Version |
0.3.3
JSON |
| download |
| home_page | None |
| Summary | Pytest plugin to track and report unit/function coverage. |
| upload_time | 2024-03-12 15:52:05 |
| maintainer | None |
| docs_url | None |
| author | Jack Klimov |
| requires_python | >=3.9 |
| license | None |
| keywords |
plugin
pytest
unit test
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Checklist: Pytest Plugin to Show Unit Coverage
**Code** coverage tools like
[coverage.py](https://coverage.readthedocs.io/en/7.0.1/) show you the
instrumented code coverage of your tests, however it won't tell you if you've
written specific unit tests for each of your code's **units** (here unit meaning
function).
This package implements a mechanism for measuring and reporting unit coverage.
Instead of instrumenting your code you will need to mark tests with a
**pointer** to the unit that it is covering.
You can think of it like an automatic checklist maintainer for each
function in your codebase!
You can still use standard coverage tools for measuring actual branch
coverage.
This package works by collecting all of the pointed-to units during test
execution and persists these to the pytest cache (typically somewhere under
`.pytest_cache`). Then in subsequent runs you need only report the results.
## Usage
### Writing Tests
First you must write tests and associate ("point") them to "targets"
(i.e. functions or "units") in your source code.
For example if you have in your code this module `mypackage/widget.py`:
``` python
def foo(in):
return in * 3
```
Then in your test suite you would write a unit test for this function
and mark it as relating to that unit, e.g. in `tests/test_widget.py`:
``` python
from mypackage.widget import foo
@pytest.mark.pointer(foo)
def test_foo():
assert foo(3) == 9
```
This registers that you have a unit test that covers the function.
NOTE: that this just helps you keep track of having declared a test
for a function, not that it is actually tested properly.
---
You can also write pointers like this:
```python
@pytest.mark.pointer(target=foo)
```
You can ignore files by using the ignore glob patterns (see below).
You can ignore individual functions using comments like this:
``` python
def foo(in): # nochecklist:
return in * 3
```
Only the `nochecklist:` token is required. You can add a comment on why
after it:
``` python
def foo(in): # nochecklist: not testable
return in * 3
```
---
You can mark multiple tests as covering a function, e.g.:
```python
@pytest.mark.pointer(foo)
def test_foo_caseA():
...
@pytest.mark.pointer(foo)
def test_foo_caseB():
...
```
But currently you can't mark a single test as covering multiple
functions. Only the first mark in the decorator stack is used.
#### Tips
We recommend adding this to the top of your test file to make typing
less and to reduce visua clutter:
```python
import pytest
pointer = pytest.mark.pointer
@pointer(func)
def test_func():
...
```
### Invocation
This package adds a couple new options to the `pytest` CLI:
`--checklist-disabled` (default `False`)
When this is given will explicitly disable this plugin from all
collection and reporting. Useful for running non-unit tests.
`--checklist-collect=STR` (default `src`)
This explicitly indicates to collect target coverage results. If not specified,
but `--checklist-report` is given results will be collected using the default.
`--checklist-report` (default `False`)
When this flag is given a textual report will be given at the end of the test
run. Note that even if this is not given the coverage checks will still be run.
`--checklist-func-min-pass=INT` (default `1`)
This flag controls the number of target test pointer marks are needed to
get a "passing" target.
`--checklist-fail-under=FLOAT` (default `100.0`)
This flag controls the percentage of passing targets are needed for the entire
coverage check to pass. The percentage is always displayed even without
`--checklist-report`. If this test is failed then the test process exits with
code 1, which is useful for things like CI.
`--checklist-exclude=STR` (default `''`)
Specify files via a comma separated list of glob pattern relative to
the `--checklist-collect` root directory to ignore. For example
`utils.py,no_unit/*.py`. Because excluded files will not be collected,
targets in them will not show up in the ignored target section. If you
want to ignore specific targets use the inline comments.
`--checklist-report-ignored` (default `False`)
When this flag is given the final report will also display the ignored
targets that were collected but will not fail. Note that anything
excluded will not be in this collection.
`--checklist-report-passing` (default `False`)
When this flag is given the final report will display all the passing
targets. Otherwise, only the failing target lines will be shown.
#### Example
Here is an example from this project (at a past point) source code
under the `src` folder, requiring 1 pointer test per collected unit in
the code, for all functions.
```
pytest --color=yes --verbose --import-mode=importlib --capture=no --tb=native --test-data=test_data --checklist-collect src/pytest_checklist --checklist-report --checklist-func-min-pass=1 --checklist-fail-under=100
========================================== test session starts ==========================================
platform linux -- Python 3.9.18, pytest-8.0.2, pluggy-1.4.0 -- /home/user/pytest-checklist/.hatch/pytest-checklist/bin/python
cachedir: .pytest_cache
rootdir: /home/user/pytest-checklist
configfile: pytest.ini
plugins: checklist-0.3.2
collected 6 items
tests/test_app.py::test_resolve_ignore_patterns PASSED
tests/test_app.py::test_is_passing PASSED
tests/test_collector.py::test_detect_files PASSED
tests/test_collector.py::test_resolve_fq_modules PASSED
tests/test_collector.py::test_resolve_fq_targets PASSED
tests/test_data.py::test_data_dir PASSED
----------------------
Checklist unit coverage
========================================
List of functions in project and the number of tests for them
1 ··· pytest_checklist.app.is_passing
1 ··· pytest_checklist.app.resolve_ignore_patterns
0 ··· pytest_checklist.collector.Target.fq_name
0 ··· pytest_checklist.collector.MethodQualNamesCollector.visit_FunctionDef
0 ··· pytest_checklist.collector.resolve_fq_targets
1 ··· pytest_checklist.collector.detect_files
0 ··· pytest_checklist.collector.MethodQualNamesCollector.__init__
0 ··· pytest_checklist.collector.collect_case_passes
1 ··· pytest_checklist.collector.resolve_fq_modules
0 ··· pytest_checklist.pointer.resolve_target_pointer
0 ··· pytest_checklist.pointer.resolve_pointer_mark_target
0 ··· pytest_checklist.report.make_report
Checklist unit coverage failed. Target was 100.0, achieved 33.33333333333333.
END Checklist unit coverage
========================================
```
## Installation
Just install from this git repository:
``` shell
pip install git+https://github.com/examol-corp/pytest-checklist.git
```
## Contributing
You must install [hatch](https://hatch.pypa.io/latest/).
### Install Hooks
Uses the [lefthook](https://github.com/evilmartians/lefthook) hook
runner.
You will need to run this once to have hooks run on git pre-commit:
```sh
lefthook install
```
### Python Bootstrapping
If you want you can bootstrap python installations with Hatch:
```sh
hatch python install --private 3.12
```
Be sure to check the documentation to make sure your site
configuration of Hatch makes sense.
If you don't do this you will be responsible for installing a version
of python for development declared in the environment.
### Testing, linting, etc.
You can just run all QA with:
```sh
lefthook run pre-commit
```
Or individually:
```sh
hatch run format_check
hatch run lint
hatch run typecheck
```
You can run the other tasks manually:
```sh
hatch run format
hatch run test
```
### Building and Publishing
```sh
hatch build
hatch publish
```
Raw data
{
"_id": null,
"home_page": null,
"name": "pytest-checklist",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "Samuel Lotz <salotz@salotz.info>",
"keywords": "plugin,pytest,unit test",
"author": "Jack Klimov",
"author_email": "Samuel Lotz <salotz@salotz.info>",
"download_url": "https://files.pythonhosted.org/packages/f8/6f/880fcc22b0a5f38b8036f173dac3d2ca4958a359e1232035239ebf78fcb1/pytest_checklist-0.3.3.tar.gz",
"platform": null,
"description": "# Checklist: Pytest Plugin to Show Unit Coverage\n\n**Code** coverage tools like\n[coverage.py](https://coverage.readthedocs.io/en/7.0.1/) show you the\ninstrumented code coverage of your tests, however it won't tell you if you've\nwritten specific unit tests for each of your code's **units** (here unit meaning\nfunction).\n\nThis package implements a mechanism for measuring and reporting unit coverage.\nInstead of instrumenting your code you will need to mark tests with a\n**pointer** to the unit that it is covering.\n\nYou can think of it like an automatic checklist maintainer for each\nfunction in your codebase!\n\nYou can still use standard coverage tools for measuring actual branch\ncoverage.\n\nThis package works by collecting all of the pointed-to units during test\nexecution and persists these to the pytest cache (typically somewhere under\n`.pytest_cache`). Then in subsequent runs you need only report the results.\n\n## Usage\n\n### Writing Tests\n\nFirst you must write tests and associate (\"point\") them to \"targets\"\n(i.e. functions or \"units\") in your source code.\n\nFor example if you have in your code this module `mypackage/widget.py`:\n\n``` python\ndef foo(in):\n return in * 3\n```\n\nThen in your test suite you would write a unit test for this function\nand mark it as relating to that unit, e.g. in `tests/test_widget.py`:\n\n``` python\n\nfrom mypackage.widget import foo\n\n@pytest.mark.pointer(foo)\ndef test_foo():\n assert foo(3) == 9\n```\n\nThis registers that you have a unit test that covers the function.\n\nNOTE: that this just helps you keep track of having declared a test\nfor a function, not that it is actually tested properly.\n\n---\n\nYou can also write pointers like this:\n\n```python\n@pytest.mark.pointer(target=foo)\n```\n\nYou can ignore files by using the ignore glob patterns (see below).\n\nYou can ignore individual functions using comments like this:\n\n``` python\ndef foo(in): # nochecklist:\n return in * 3\n```\n\nOnly the `nochecklist:` token is required. You can add a comment on why\nafter it:\n\n``` python\ndef foo(in): # nochecklist: not testable\n return in * 3\n```\n\n---\n\nYou can mark multiple tests as covering a function, e.g.:\n\n```python\n\n@pytest.mark.pointer(foo)\ndef test_foo_caseA():\n ...\n\n@pytest.mark.pointer(foo)\ndef test_foo_caseB():\n ...\n\n```\n\nBut currently you can't mark a single test as covering multiple\nfunctions. Only the first mark in the decorator stack is used.\n\n#### Tips\n\nWe recommend adding this to the top of your test file to make typing\nless and to reduce visua clutter:\n\n```python\nimport pytest\n\npointer = pytest.mark.pointer\n\n@pointer(func)\ndef test_func():\n ...\n```\n\n### Invocation\n\nThis package adds a couple new options to the `pytest` CLI:\n\n`--checklist-disabled` (default `False`)\n\nWhen this is given will explicitly disable this plugin from all\ncollection and reporting. Useful for running non-unit tests.\n\n\n`--checklist-collect=STR` (default `src`)\n\nThis explicitly indicates to collect target coverage results. If not specified,\nbut `--checklist-report` is given results will be collected using the default.\n\n\n`--checklist-report` (default `False`)\n\nWhen this flag is given a textual report will be given at the end of the test\nrun. Note that even if this is not given the coverage checks will still be run.\n\n`--checklist-func-min-pass=INT` (default `1`)\n\nThis flag controls the number of target test pointer marks are needed to\nget a \"passing\" target.\n\n`--checklist-fail-under=FLOAT` (default `100.0`)\n\nThis flag controls the percentage of passing targets are needed for the entire\ncoverage check to pass. The percentage is always displayed even without\n`--checklist-report`. If this test is failed then the test process exits with\ncode 1, which is useful for things like CI.\n\n`--checklist-exclude=STR` (default `''`)\n\nSpecify files via a comma separated list of glob pattern relative to\nthe `--checklist-collect` root directory to ignore. For example\n`utils.py,no_unit/*.py`. Because excluded files will not be collected,\ntargets in them will not show up in the ignored target section. If you\nwant to ignore specific targets use the inline comments.\n\n`--checklist-report-ignored` (default `False`)\n\nWhen this flag is given the final report will also display the ignored\ntargets that were collected but will not fail. Note that anything\nexcluded will not be in this collection.\n\n`--checklist-report-passing` (default `False`)\n\nWhen this flag is given the final report will display all the passing\ntargets. Otherwise, only the failing target lines will be shown.\n\n\n#### Example\n\nHere is an example from this project (at a past point) source code\nunder the `src` folder, requiring 1 pointer test per collected unit in\nthe code, for all functions.\n\n```\npytest --color=yes --verbose --import-mode=importlib --capture=no --tb=native --test-data=test_data --checklist-collect src/pytest_checklist --checklist-report --checklist-func-min-pass=1 --checklist-fail-under=100\n========================================== test session starts ==========================================\nplatform linux -- Python 3.9.18, pytest-8.0.2, pluggy-1.4.0 -- /home/user/pytest-checklist/.hatch/pytest-checklist/bin/python\ncachedir: .pytest_cache\nrootdir: /home/user/pytest-checklist\nconfigfile: pytest.ini\nplugins: checklist-0.3.2\ncollected 6 items\n\ntests/test_app.py::test_resolve_ignore_patterns PASSED\ntests/test_app.py::test_is_passing PASSED\ntests/test_collector.py::test_detect_files PASSED\ntests/test_collector.py::test_resolve_fq_modules PASSED\ntests/test_collector.py::test_resolve_fq_targets PASSED\ntests/test_data.py::test_data_dir PASSED\n\n----------------------\nChecklist unit coverage\n========================================\n \n \n \n List of functions in project and the number of tests for them\n \n \n 1 \u00b7\u00b7\u00b7 pytest_checklist.app.is_passing \n 1 \u00b7\u00b7\u00b7 pytest_checklist.app.resolve_ignore_patterns \n 0 \u00b7\u00b7\u00b7 pytest_checklist.collector.Target.fq_name \n 0 \u00b7\u00b7\u00b7 pytest_checklist.collector.MethodQualNamesCollector.visit_FunctionDef \n 0 \u00b7\u00b7\u00b7 pytest_checklist.collector.resolve_fq_targets \n 1 \u00b7\u00b7\u00b7 pytest_checklist.collector.detect_files \n 0 \u00b7\u00b7\u00b7 pytest_checklist.collector.MethodQualNamesCollector.__init__ \n 0 \u00b7\u00b7\u00b7 pytest_checklist.collector.collect_case_passes \n 1 \u00b7\u00b7\u00b7 pytest_checklist.collector.resolve_fq_modules \n 0 \u00b7\u00b7\u00b7 pytest_checklist.pointer.resolve_target_pointer \n 0 \u00b7\u00b7\u00b7 pytest_checklist.pointer.resolve_pointer_mark_target \n 0 \u00b7\u00b7\u00b7 pytest_checklist.report.make_report \n \n \n \nChecklist unit coverage failed. Target was 100.0, achieved 33.33333333333333.\n\nEND Checklist unit coverage\n========================================\n```\n\n## Installation\n\nJust install from this git repository:\n\n``` shell\npip install git+https://github.com/examol-corp/pytest-checklist.git\n```\n\n## Contributing\n\nYou must install [hatch](https://hatch.pypa.io/latest/).\n\n### Install Hooks\n\nUses the [lefthook](https://github.com/evilmartians/lefthook) hook\nrunner.\n\nYou will need to run this once to have hooks run on git pre-commit:\n\n```sh\nlefthook install\n```\n### Python Bootstrapping\n\nIf you want you can bootstrap python installations with Hatch:\n\n```sh\nhatch python install --private 3.12\n```\n\nBe sure to check the documentation to make sure your site\nconfiguration of Hatch makes sense.\n\nIf you don't do this you will be responsible for installing a version\nof python for development declared in the environment.\n\n### Testing, linting, etc.\n\n\nYou can just run all QA with:\n\n```sh\nlefthook run pre-commit\n```\n\nOr individually:\n\n```sh\nhatch run format_check\nhatch run lint\nhatch run typecheck\n```\n\nYou can run the other tasks manually:\n\n```sh\nhatch run format\nhatch run test\n```\n\n### Building and Publishing\n\n```sh\nhatch build\nhatch publish\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "Pytest plugin to track and report unit/function coverage.",
"version": "0.3.3",
"project_urls": {
"homepage": "https://github.com/examol-corp/pytest-checklist",
"repository": "https://github.com/examol-corp/pytest-checklist"
},
"split_keywords": [
"plugin",
"pytest",
"unit test"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "7f50d093669d151f093bcec4ce61cbae961c74a80e98c95b73dde6019a821491",
"md5": "f81b807002ae345610e1b2eef4e3049b",
"sha256": "1643241fd6adf238a203a98f7877dc8069cd397130f2c2e0b03fe023ae7b8d5f"
},
"downloads": -1,
"filename": "pytest_checklist-0.3.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f81b807002ae345610e1b2eef4e3049b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 11634,
"upload_time": "2024-03-12T15:52:07",
"upload_time_iso_8601": "2024-03-12T15:52:07.067411Z",
"url": "https://files.pythonhosted.org/packages/7f/50/d093669d151f093bcec4ce61cbae961c74a80e98c95b73dde6019a821491/pytest_checklist-0.3.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "f86f880fcc22b0a5f38b8036f173dac3d2ca4958a359e1232035239ebf78fcb1",
"md5": "26160825bcb45a3897f1013da4faba99",
"sha256": "e6345cf020731e0e6bb08817e66290c6ed0519b71d5ef49fed65c37ad8989489"
},
"downloads": -1,
"filename": "pytest_checklist-0.3.3.tar.gz",
"has_sig": false,
"md5_digest": "26160825bcb45a3897f1013da4faba99",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 10243,
"upload_time": "2024-03-12T15:52:05",
"upload_time_iso_8601": "2024-03-12T15:52:05.591776Z",
"url": "https://files.pythonhosted.org/packages/f8/6f/880fcc22b0a5f38b8036f173dac3d2ca4958a359e1232035239ebf78fcb1/pytest_checklist-0.3.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-12 15:52:05",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "examol-corp",
"github_project": "pytest-checklist",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "pytest-checklist"
}
