# pycobertura
A code coverage diff tool for Cobertura reports.
[](https://pypi.python.org/pypi/pycobertura)
- [About](#about)
- [Install](#install)
- [CLI usage](#cli-usage)
- [Library usage](#library-usage)
- [How to contribute?](#how-to-contribute)
- [FAQ](#faq)
## About
pycobertura is a generic [Cobertura](http://cobertura.github.io/cobertura/)
report parser. It was also designed to help prevent code coverage from
decreasing with the `pycobertura diff` command: any line changed should be
tested and uncovered changes should be clearly visible without letting legacy
uncovered code get in the way so developers can focus solely on their changes.
Features:
* show coverage summary of a cobertura file
* output in plain text or HTML
* compare two cobertura files and show changes in coverage
* colorized diff output
* diff exit status of non-zero if coverage worsened or if any changes were left uncovered
* fail based on uncovered lines rather than on decrease of coverage rate
([see why](#why-is-the-number-of-uncovered-lines-used-as-the-metric-to-check-if-code-coverage-worsened-rather-than-the-line-rate))
NOTE: The API is unstable any may be subject to changes until it reaches 1.0.
## Install
```shell
pip install pycobertura
```
## CLI usage
pycobertura provides a command line interface `pycobertura` to report on coverage files.
Alternatively, pycobertura can also be invoked as a module with `python -m pycobertura`, see [PEP 338](https://peps.python.org/pep-0338/).
### Help commands
Different help screens are available depending on what you need help about.
```shell
$ pycobertura --help
Usage: pycobertura [OPTIONS] COMMAND [ARGS]...
Options:
--help Show this message and exit.
Commands:
diff The diff command compares and shows the changes between two...
show show coverage summary of a Cobertura report
```
```shell
pycobertura show --help
```
```shell
pycobertura diff --help
```
### Command `show`
The `show` command displays the report summary of a coverage file.
```shell
$ pycobertura show coverage.xml
Filename Stmts Miss Cover Missing
------------------------- ------- ------ ------- ---------
pycobertura/__init__.py 1 0 100.00%
pycobertura/cli.py 18 0 100.00%
pycobertura/cobertura.py 93 0 100.00%
pycobertura/reporters.py 129 0 100.00%
pycobertura/utils.py 12 0 100.00%
TOTAL 253 0 100.00%
```
*NOTE*: For the csv format we chose as default delimiter the semicolon, because the common is used internally in the "Missing" column,
therefore the "," delimiter might lead to problems when rendering the csv in an editor.
```shell
$ pycobertura show --format csv coverage.xml --delimiter ";"
Filename;Stmts;Miss;Cover;Missing
pycobertura/__init__.py;2;0;100.00%;
pycobertura/cli.py;132;88;33.33%;274-3758
pycobertura/cobertura.py;216;35;83.80%;423, 648-696
pycobertura/filesystem.py;91;6;93.41%;343-625
pycobertura/reporters.py;214;110;48.60%;291, 295-3284
pycobertura/utils.py;122;15;87.70%;38, 262-486
pycobertura/templates/__init__.py;0;0;100.00%;
pycobertura/templates/filters.py;17;10;41.18%;261-544
TOTAL;794;264;66.75%;
```
```shell
$ pycobertura show --format markdown coverage.xml
| Filename | Stmts | Miss | Cover | Missing |
|-----------------------------------|---------|--------|---------|--------------------|
| pycobertura/__init__.py | 2 | 0 | 100.00% | |
| pycobertura/cli.py | 132 | 88 | 33.33% | 274-3758 |
| pycobertura/cobertura.py | 216 | 35 | 83.80% | 423, 648-696 |
| pycobertura/filesystem.py | 91 | 6 | 93.41% | 343-625 |
| pycobertura/reporters.py | 222 | 114 | 48.65% | 286, 290, 331-4065 |
| pycobertura/utils.py | 122 | 15 | 87.70% | 38, 262-486 |
| pycobertura/templates/__init__.py | 0 | 0 | 100.00% | |
| pycobertura/templates/filters.py | 17 | 10 | 41.18% | 261-544 |
| TOTAL | 802 | 268 | 66.58% | |
```
The following is a screenshot of the HTML version of another coverage file
which also include the source code with highlighted source code to indicate
whether lines were covered (green) or not (red).
```shell
pycobertura show --format html --output coverage.html coverage.xml
```
The following shows how to generate a JSON version of another coverage file.
```shell
$ pycobertura show --format json --output coverage.json tests/cobertura.xml
{
"files": [
{
"Filename": "Main.java",
"Stmts": 15,
"Miss": 0,
"Cover": "100.00%",
"Missing": ""
},
{
"Filename": "search/BinarySearch.java",
"Stmts": 12,
"Miss": 1,
"Cover": "91.67%",
"Missing": "24"
},
{
"Filename": "search/ISortedArraySearch.java",
"Stmts": 0,
"Miss": 0,
"Cover": "100.00%",
"Missing": ""
},
{
"Filename": "search/LinearSearch.java",
"Stmts": 7,
"Miss": 2,
"Cover": "71.43%",
"Missing": "19-24"
}
],
"total": {
"Filename": "TOTAL",
"Stmts": 34,
"Miss": 3,
"Cover": "90.00%"
}
}
```
```shell
$ pycobertura show --format yaml --output coverage.yaml tests/cobertura.xml
files:
- Filename: Main.java
Stmts: 15
Miss: 0
Cover: 100.00%
Missing: ''
- Filename: search/BinarySearch.java
Stmts: 12
Miss: 1
Cover: 91.67%
Missing: '24'
- Filename: search/ISortedArraySearch.java
Stmts: 0
Miss: 0
Cover: 100.00%
Missing: ''
- Filename: search/LinearSearch.java
Stmts: 7
Miss: 2
Cover: 71.43%
Missing: 19-24
total:
Filename: TOTAL
Stmts: 34
Miss: 3
Cover: 90.00%
```
The following shows how to generate GitHub annotations given a coverage file.
```shell
$ pycobertura show --format github-annotation tests/cobertura.xml
::notice file=dummy/dummy.py,line=5,endLine=6,title=pycobertura::not covered
::notice file=dummy/dummy4.py,line=1,endLine=6,title=pycobertura::not covered
```
If you run it in GitHub Actions/Apps, the above log generates check annotations.
### Command `diff`
You can also use the `diff` command to show the difference between two coverage
files. To properly compute the `Missing` column, it is necessary to provide the
source code that was used to generate each of the passed Cobertura reports
([see why](#why-do-i-need-to-provide-the-path-to-the-source-code-directory)).
```shell
$ pycobertura diff coverage.old.xml coverage.new.xml --source1 old_source/ --source2 new_source/
Filename Stmts Miss Cover Missing
---------------- ------- ------ -------- ---------
dummy/dummy.py - -2 +50.00% -2, -5
dummy/dummy2.py +2 - +100.00%
TOTAL +2 -2 +50.00%
```
The column `Missing` will show line numbers prefixed with either a plus sign
`+` or a minus sign `-`. When prefixed with a plus sign, the line was
introduced as uncovered and is shown in red, when prefixed as a minus sign, the
line is no longer uncovered and is rendered in green.
Analogously you can also show the diff in csv format with the option to define a delimiter (\t should also work here) via:
```shell
pycobertura diff tests/dummy.source1/coverage.xml tests/dummy.source2/coverage.xml --format csv --delimiter ";"
```
*NOTE*: The default delimiter is semicolon (;). The delimiters comma (,) and newline (/n) are *very risky* to use and are not supported by this library. The comma delimiter is used in the "Missing" column in the list datastructure to separate the different items. Therefore the "," delimiter might lead to problems when rendering the csv in an editor. As a missing value in csv corresponds to an empty value, the newline delimiter can as well lead to problems when rendering the csv in an editor.
Analogously you can also show the diff in markdown format via:
```shell
pycobertura diff --format markdown tests/dummy.source1/coverage.xml tests/dummy.source2/coverage.xml
```
This will lead to the following output
This screenshot shows how the HTML output only applies coverage highlighting to
the parts of the code where the coverage has changed (from covered to
uncovered, or vice versa).
```shell
pycobertura diff --format html --output coverage.html ./master/coverage.xml ./myfeature/coverage.xml
```
This screenshot shows how the JSON output only applies coverage highlighting to
the parts of the code where the coverage has changed (from covered to
uncovered, or vice versa).
```shell
pycobertura diff --format json tests/dummy.source1/coverage.xml tests/dummy.source2/coverage.xml
```
This screenshot shows how the YAML output only applies coverage highlighting to
the parts of the code where the coverage has changed (from covered to
uncovered, or vice versa).
```shell
pycobertura diff --format yaml tests/dummy.source1/coverage.xml tests/dummy.source2/coverage.xml
```
The following shows how to generate GitHub annotations given a coverage file.
```shell
$ pycobertura diff --format github-annotation tests/dummy.source1/coverage.xml tests/dummy.source2/coverage.xml
::notice file=dummy/dummy2.py,line=5,endLine=5,title=pycobertura::not covered
::notice file=dummy/dummy3.py,line=1,endLine=2,title=pycobertura::not covered
```
If you run it in GitHub Actions/Apps, the above log generates check annotations.
### Option to exclude files via --ignore-regex-option
You can specify the `--ignore-regex` option to exclude files that have a certain pattern either by specifying the path to a `.gitignore` file or by entering a Python regex.
Examples:
```
pycobertura show tests/cobertura.xml
Filename Stmts Miss Cover Missing
------------------------------ ------- ------ ------- ---------
Main.java 15 0 100.00%
search/BinarySearch.java 12 1 91.67% 24
search/ISortedArraySearch.java 0 0 100.00%
search/LinearSearch.java 7 2 71.43% 19-24
TOTAL 34 3 90.00%
```
```
pycobertura show tests/cobertura.xml --ignore-regex ".*Search"
Filename Stmts Miss Cover Missing
---------- ------- ------ ------- ---------
Main.java 15 0 100.00%
TOTAL 34 3 90.00%
```
> NOTE: Putting just a `*`, e.g. "*Search" above will lead to an error, you need to really follow Python regex conventions to use this option.
You can also specify a `.testgitignore` with the following content:
```
# Test file for ignore regexes
**/__pycache__
**/dummy*
*.xml
**/*Search*
```
and then use
```
pycobertura show tests/cobertura.xml --ignore-regex "tests/.testgitignore"
Filename Stmts Miss Cover Missing
---------- ------- ------ ------- ---------
Main.java 15 0 100.00%
TOTAL 34 3 90.00%
```
The same will also work for `diff` commands.
#### `diff` exit codes
Upon exit, the `diff` command may return various exit codes:
* 0: all changes are covered, no new uncovered statements have been introduced
* 1: some exception occurred (likely due to inappropriate usage or a bug in pycobertura)
* 2: the changes worsened the overall coverage
* 3: the changes introduced uncovered statements but the overall coverage is still better than before
You can capture the exit code by using the `$?` syntax from your shell:
```bash
pycobertura ...
PYCOBERTURA_EXIT_CODE=$?
echo $PYCOBERTURA_EXIT_CODE
```
## Library usage
Using it as a library in your Python application is easy:
```python
from pycobertura import Cobertura
cobertura = Cobertura('coverage.xml')
cobertura.version == '4.0.2'
cobertura.line_rate() == 1.0 # 100%
cobertura.files() == [
'pycobertura/__init__.py',
'pycobertura/cli.py',
'pycobertura/cobertura.py',
'pycobertura/reporters.py',
'pycobertura/utils.py',
]
cobertura.line_rate('pycobertura/cli.py') == 1.0
from pycobertura import TextReporter
tr = TextReporter(cobertura)
tr.generate() == """\
Filename Stmts Miss Cover Missing
------------------------- ------- ------ ------- ---------
pycobertura/__init__.py 1 0 100.00%
pycobertura/cli.py 18 0 100.00%
pycobertura/cobertura.py 93 0 100.00%
pycobertura/reporters.py 129 0 100.00%
pycobertura/utils.py 12 0 100.00%
TOTAL 253 0 100.00%"""
from pycobertura import TextReporterDelta
coverage1 = Cobertura('coverage1.xml')
coverage2 = Cobertura('coverage2.xml')
delta = TextReporterDelta(coverage1, coverage2)
delta.generate() == """\
Filename Stmts Miss Cover Missing
---------------- ------- ------ -------- ---------
dummy/dummy.py - -2 +50.00% -2, -5
dummy/dummy2.py +2 - +100.00%
TOTAL +2 -2 +50.00%"""
```
## How to contribute?
Found a bug/typo? Got a patch? Have an idea? Please use Github issues or fork
pycobertura and submit a pull request (PR). All contributions are welcome!
If you submit a PR:
* ensure the description of your PR illustrates your changes clearly by showing
what the problem was and how you fixed it (before/after)
* make sure your changes are covered with one or more tests
* add a descriptive note in the CHANGES file under the `Unreleased` section
* update the README accordingly if your changes outdate the documentation
* make sure all tests are passing using `tox`
```shell
pip install tox
tox
```
## FAQ
### How does pycobertura work?
Pycobertura takes two different Cobertura reports and compares them line by
line. If the coverage status of a line changed from covered to uncovered or
vice versa, then pycobertura will report it. Sometimes you have no code changes
at all, the only changes were to add more tests and pycobertura will show you
the progress.
Pycobertura was initially designed as a general purpose Cobertura parser and
can generate a summary table for a single Cobertura file (the `show` command).
### I only have one Cobertura report, can I just see my uncovered changes?
Yes. All you have to do is pass your same coverage report twice and provide the
path to the two different code bases:
```shell
pycobertura diff coverage.xml coverage.xml --source1 master/ --source2 myfeature/
```
But keep in mind that this will not show you if your changes have introduced a
drop in coverage elsewhere in the code base.
### Why doesn't pycobertura use git to diff the source given revision SHAs rather than passing paths to the source code?
Because we would have to support N version control systems (VCS). It is easy
enough to generate a directory that contains the source code at a given commit
or branch name that it's not a top priority for pycobertura to be VCS-aware:
```shell
git archive --prefix=source1/ ${BRANCH_OR_COMMIT1} | tar -xf -
git archive --prefix=source2/ ${BRANCH_OR_COMMIT2} | tar -xf -
pycobertura diff --source1 source1/ --source2 source2/ coverage1.xml coverage2.xml -o output.html
rm -rf source1/ source2/
```
Mercurial has `hg archive` and Subversion has `svn export`. These are simple
pre-steps to running `pycobertura diff`.
Also, the code repository may not always be available at the time pycobertura
is run. Typically, in Continuous Delivery pipelines, only
[artifacts](http://en.wikipedia.org/wiki/Artifact_%28software_development%29)
are available.
### Why do I need to provide the path to the source code directory?
With the command `pycobertura show`, you don't need to provide the source code
directory, unless you want the HTML output which will conveniently render the
highlighted source code for you.
But with `pycobertura diff`, if you care about *which* lines are covered or
uncovered (and not just a global count), then you will need to provide the
source for *each* of the reports.
To better understand why, let's assume we have 2 Cobertura reports with the
following info:
Report A:
```shell
line 1, hit
line 2, miss
line 3, hit
```
and Report B:
```shell
line 1, hit
line 2, miss
line 3, hit
line 4, miss
line 5, hit
```
How can you tell which lines need to be highlighted? Naively, you'd assume that
lines 4-5 were added and these should be the highlighted lines, the ones part
of your coverage diff. Well, that doesn't quite work.
The code for Report A is:
```python
if foo is True: # line 1
total += 1 # line 2
return total # line 3
```
The code for Report B is:
```python
if foo is False: # line 1 # new line
total -= 1 # line 2 # new line
elif foo is True: # line 3 # modified line
total += 1 # line 4, unchanged
return total # line 5, unchanged
```
The code change are lines 1-3 and these are the ones you want to highlight.
Lines 4-5 don't need to be highlighted (unless coverage status changed
in-between).
So, to accurately highlight the lines that have changed, the coverage reports
alone are not sufficient and this is why you need to provide the path to the
source that was used to generate each of the Cobertura reports and diff them to
see which lines actually changed to report accurate coverage.
### When should I use pycobertura?
pycobertura was built as a tool to educate developers about the testing culture
in such way that any code change should have one or more tests along with it.
You can use pycobertura in your Continuous Integration (CI) or Continuous
Delivery (CD) pipeline which would fail a build if the code changes worsened
the coverage. For example, when a pull request is submitted, the new code
should have equal or better coverage than the branch it's going to be merged
into. Or if code navigates through a release pipeline and the new code has
worse coverage than what's already in Production, then the release is aborted.
When a build is triggered by your CI/CD pipeline, each testing stage would
typically store an
[artifact](http://en.wikipedia.org/wiki/Artifact_%28software_development%29) of
the source code and another one of the Cobertura report. An extra stage in the
pipeline could ensure that the coverage did not worsen. This can be done by
retrieving the artifacts of the current build as well as the "target" artifacts
(code and Cobertura report of Production or target branch of a pull request).
Then `pycobertura diff` will take care of failing the build if the coverage
worsened (return a non-zero exit code) and then the pycobertura report can be
published as an artifact to make it available to developers to look at.
The step could look like this:
```shell
# Download artifacts of current build
curl -o coverage.${BUILD_ID}.xml https://ciserver/artifacts/${BUILD_ID}/coverage.xml
curl -o source.${BUILD_ID}.zip https://ciserver/artifacts/${BUILD_ID}/source.zip
# Download artifacts of already-in-Prod build
curl -o coverage.${PROD_BUILD}.xml https://ciserver/artifacts/${PROD_BUILD}/coverage.xml
curl -o source.${PROD_BUILD}.zip https://ciserver/artifacts/${PROD_BUILD}/source.zip
unzip source.${BUILD_ID}.zip -d source.${BUILD_ID}
unzip source.${PROD_BUILD}.zip -d source.${PROD_BUILD}
# Compare
pycobertura diff --format html \
--output pycobertura-diff.${BUILD_ID}.html \
--source1 source.${PROD_BUILD} \
--source2 source.${BUILD_ID} \
coverage.${PROD_BUILD}.xml \
coverage.${BUILD_ID}.xml
# Upload the pycobertura report artifact
curl -F filedata=@pycobertura-diff.${BUILD_ID}.html http://ciserver/artifacts/${BUILD_ID}/
```
### Why is the number of uncovered lines used as the metric to check if code coverage worsened rather than the line rate?
The line rate (percentage of covered lines) can legitimately go down for a
number of reasons. To illustrate, suppose we have this code coverage report for
version A of our code:
```shell
line 1: hit
line 2: hit
line 3: miss
line 4: hit
line 5: hit
```
Here, the line rate is 80% and uncovered lines is 1 (miss). Later in version B
of our code, we legitimately delete a covered line and the following coverage
report is generated:
```shell
line 1: hit
### line deleted ###
line 2: miss
line 3: hit
line 4: hit
```
The line rate decreased from 80% to 75% but uncovered lines is still at 1. In
this case, failing the build based on line rate is inappropriate, thus making
the line rate the wrong metric to look at when validating coverage.
The basic idea is that a code base may have technical debt of N uncovered lines
and you want to prevent N from ever going up.
### pycobertura sounds cool, but how to I generate a Cobertura file?
Depending on your programing language, you need to find a tool that measures
code coverage and generates a Cobertura report which is an XML representation
of your code coverage results.
In Python, [coverage.py](http://nedbatchelder.com/code/coverage/) is a great
tool for measuring code coverage and plugins such as
[pytest-cov](https://pypi.python.org/pypi/pytest-cov) for
[pytest](http://pytest.org/latest/) and
[nosexcover](https://pypi.python.org/pypi/nosexcover) for
[nose](https://nose.readthedocs.org/en/latest/) are available to generate
Cobertura reports while running tests.
[Istanbul](https://github.com/gotwarlost/istanbul) can generate
Cobertura reports for Javascript and also has plugins for
[CoffeeScript](https://github.com/benbria/coffee-coverage) if that is your
cup of tea.
Cobertura is a very common file format available in many testing tools for
pretty much all programing languages. pycobertura is language agnostic and
should work with reports generated by tools in any language. But it was mostly
developed and tested against reports generated with the `pytest-cov` plugin in
Python. If you see issues, please [create a
ticket](https://github.com/SurveyMonkey/pycobertura/issues).
## Logo credit
The pycobertura logo, Aysha, was graciously
[donated](https://github.com/arasatasaygin/openlogos/issues/19) by
[Open Logos](http://openlogos.org/). 🙇 Check them out!
Raw data
{
"_id": null,
"home_page": "https://github.com/aconrad/pycobertura",
"name": "pycobertura",
"maintainer": "\"Alex Conrad\"",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "\"alexandre.conrad@gmail.com\"",
"keywords": "\"cobertura coverage diff report parser parse xml\"",
"author": "\"Alex Conrad\"",
"author_email": "\"alexandre.conrad@gmail.com\"",
"download_url": "https://files.pythonhosted.org/packages/5e/57/2d33605526a1cff9805a5283a348346fbde8ff9a838cd457dee25ea933c0/pycobertura-3.3.1.tar.gz",
"platform": null,
"description": "# pycobertura\n\n\n\nA code coverage diff tool for Cobertura reports.\n\n\n[](https://pypi.python.org/pypi/pycobertura)\n\n- [About](#about)\n- [Install](#install)\n- [CLI usage](#cli-usage)\n- [Library usage](#library-usage)\n- [How to contribute?](#how-to-contribute)\n- [FAQ](#faq)\n\n## About\n\npycobertura is a generic [Cobertura](http://cobertura.github.io/cobertura/)\nreport parser. It was also designed to help prevent code coverage from\ndecreasing with the `pycobertura diff` command: any line changed should be\ntested and uncovered changes should be clearly visible without letting legacy\nuncovered code get in the way so developers can focus solely on their changes.\n\nFeatures:\n\n* show coverage summary of a cobertura file\n* output in plain text or HTML\n* compare two cobertura files and show changes in coverage\n* colorized diff output\n* diff exit status of non-zero if coverage worsened or if any changes were left uncovered\n* fail based on uncovered lines rather than on decrease of coverage rate\n ([see why](#why-is-the-number-of-uncovered-lines-used-as-the-metric-to-check-if-code-coverage-worsened-rather-than-the-line-rate))\n\nNOTE: The API is unstable any may be subject to changes until it reaches 1.0.\n\n## Install\n\n```shell\npip install pycobertura\n```\n\n## CLI usage\n\npycobertura provides a command line interface `pycobertura` to report on coverage files.\n\nAlternatively, pycobertura can also be invoked as a module with `python -m pycobertura`, see [PEP 338](https://peps.python.org/pep-0338/).\n\n### Help commands\n\nDifferent help screens are available depending on what you need help about.\n\n```shell\n$ pycobertura --help\nUsage: pycobertura [OPTIONS] COMMAND [ARGS]...\n\nOptions:\n --help Show this message and exit.\n\nCommands:\n diff The diff command compares and shows the changes between two...\n show show coverage summary of a Cobertura report\n```\n\n```shell\npycobertura show --help\n```\n\n```shell\npycobertura diff --help\n```\n\n### Command `show`\n\nThe `show` command displays the report summary of a coverage file.\n\n```shell\n$ pycobertura show coverage.xml\nFilename Stmts Miss Cover Missing\n------------------------- ------- ------ ------- ---------\npycobertura/__init__.py 1 0 100.00%\npycobertura/cli.py 18 0 100.00%\npycobertura/cobertura.py 93 0 100.00%\npycobertura/reporters.py 129 0 100.00%\npycobertura/utils.py 12 0 100.00%\nTOTAL 253 0 100.00%\n```\n\n*NOTE*: For the csv format we chose as default delimiter the semicolon, because the common is used internally in the \"Missing\" column,\ntherefore the \",\" delimiter might lead to problems when rendering the csv in an editor.\n```shell\n$ pycobertura show --format csv coverage.xml --delimiter \";\"\nFilename;Stmts;Miss;Cover;Missing\npycobertura/__init__.py;2;0;100.00%;\npycobertura/cli.py;132;88;33.33%;274-3758\npycobertura/cobertura.py;216;35;83.80%;423, 648-696\npycobertura/filesystem.py;91;6;93.41%;343-625\npycobertura/reporters.py;214;110;48.60%;291, 295-3284\npycobertura/utils.py;122;15;87.70%;38, 262-486\npycobertura/templates/__init__.py;0;0;100.00%;\npycobertura/templates/filters.py;17;10;41.18%;261-544\nTOTAL;794;264;66.75%;\n```\n\n```shell\n$ pycobertura show --format markdown coverage.xml \n| Filename | Stmts | Miss | Cover | Missing |\n|-----------------------------------|---------|--------|---------|--------------------|\n| pycobertura/__init__.py | 2 | 0 | 100.00% | |\n| pycobertura/cli.py | 132 | 88 | 33.33% | 274-3758 |\n| pycobertura/cobertura.py | 216 | 35 | 83.80% | 423, 648-696 |\n| pycobertura/filesystem.py | 91 | 6 | 93.41% | 343-625 |\n| pycobertura/reporters.py | 222 | 114 | 48.65% | 286, 290, 331-4065 |\n| pycobertura/utils.py | 122 | 15 | 87.70% | 38, 262-486 |\n| pycobertura/templates/__init__.py | 0 | 0 | 100.00% | |\n| pycobertura/templates/filters.py | 17 | 10 | 41.18% | 261-544 |\n| TOTAL | 802 | 268 | 66.58% | |\n```\n\nThe following is a screenshot of the HTML version of another coverage file\nwhich also include the source code with highlighted source code to indicate\nwhether lines were covered (green) or not (red).\n\n```shell\npycobertura show --format html --output coverage.html coverage.xml\n```\n\n\n\nThe following shows how to generate a JSON version of another coverage file.\n\n```shell\n$ pycobertura show --format json --output coverage.json tests/cobertura.xml\n{\n \"files\": [\n {\n \"Filename\": \"Main.java\",\n \"Stmts\": 15,\n \"Miss\": 0,\n \"Cover\": \"100.00%\",\n \"Missing\": \"\"\n },\n {\n \"Filename\": \"search/BinarySearch.java\",\n \"Stmts\": 12,\n \"Miss\": 1,\n \"Cover\": \"91.67%\",\n \"Missing\": \"24\"\n },\n {\n \"Filename\": \"search/ISortedArraySearch.java\",\n \"Stmts\": 0,\n \"Miss\": 0,\n \"Cover\": \"100.00%\",\n \"Missing\": \"\"\n },\n {\n \"Filename\": \"search/LinearSearch.java\",\n \"Stmts\": 7,\n \"Miss\": 2,\n \"Cover\": \"71.43%\",\n \"Missing\": \"19-24\"\n }\n ],\n \"total\": {\n \"Filename\": \"TOTAL\",\n \"Stmts\": 34,\n \"Miss\": 3,\n \"Cover\": \"90.00%\"\n }\n}\n```\n\n```shell\n$ pycobertura show --format yaml --output coverage.yaml tests/cobertura.xml\nfiles:\n- Filename: Main.java\n Stmts: 15\n Miss: 0\n Cover: 100.00%\n Missing: ''\n- Filename: search/BinarySearch.java\n Stmts: 12\n Miss: 1\n Cover: 91.67%\n Missing: '24'\n- Filename: search/ISortedArraySearch.java\n Stmts: 0\n Miss: 0\n Cover: 100.00%\n Missing: ''\n- Filename: search/LinearSearch.java\n Stmts: 7\n Miss: 2\n Cover: 71.43%\n Missing: 19-24\ntotal:\n Filename: TOTAL\n Stmts: 34\n Miss: 3\n Cover: 90.00%\n\n```\n\nThe following shows how to generate GitHub annotations given a coverage file.\n\n```shell\n$ pycobertura show --format github-annotation tests/cobertura.xml\n::notice file=dummy/dummy.py,line=5,endLine=6,title=pycobertura::not covered\n::notice file=dummy/dummy4.py,line=1,endLine=6,title=pycobertura::not covered\n```\n\nIf you run it in GitHub Actions/Apps, the above log generates check annotations.\n\n\n\n### Command `diff`\n\nYou can also use the `diff` command to show the difference between two coverage\nfiles. To properly compute the `Missing` column, it is necessary to provide the\nsource code that was used to generate each of the passed Cobertura reports\n([see why](#why-do-i-need-to-provide-the-path-to-the-source-code-directory)).\n\n```shell\n$ pycobertura diff coverage.old.xml coverage.new.xml --source1 old_source/ --source2 new_source/\nFilename Stmts Miss Cover Missing\n---------------- ------- ------ -------- ---------\ndummy/dummy.py - -2 +50.00% -2, -5\ndummy/dummy2.py +2 - +100.00%\nTOTAL +2 -2 +50.00%\n```\n\nThe column `Missing` will show line numbers prefixed with either a plus sign\n`+` or a minus sign `-`. When prefixed with a plus sign, the line was\nintroduced as uncovered and is shown in red, when prefixed as a minus sign, the\nline is no longer uncovered and is rendered in green.\n\nAnalogously you can also show the diff in csv format with the option to define a delimiter (\\t should also work here) via:\n\n```shell\npycobertura diff tests/dummy.source1/coverage.xml tests/dummy.source2/coverage.xml --format csv --delimiter \";\"\n```\n\n\n\n*NOTE*: The default delimiter is semicolon (;). The delimiters comma (,) and newline (/n) are *very risky* to use and are not supported by this library. The comma delimiter is used in the \"Missing\" column in the list datastructure to separate the different items. Therefore the \",\" delimiter might lead to problems when rendering the csv in an editor. As a missing value in csv corresponds to an empty value, the newline delimiter can as well lead to problems when rendering the csv in an editor.\n\nAnalogously you can also show the diff in markdown format via:\n\n```shell\npycobertura diff --format markdown tests/dummy.source1/coverage.xml tests/dummy.source2/coverage.xml\n```\n\nThis will lead to the following output\n\n\nThis screenshot shows how the HTML output only applies coverage highlighting to\nthe parts of the code where the coverage has changed (from covered to\nuncovered, or vice versa).\n\n```shell\npycobertura diff --format html --output coverage.html ./master/coverage.xml ./myfeature/coverage.xml\n```\n\n\n\nThis screenshot shows how the JSON output only applies coverage highlighting to\nthe parts of the code where the coverage has changed (from covered to\nuncovered, or vice versa).\n\n```shell\npycobertura diff --format json tests/dummy.source1/coverage.xml tests/dummy.source2/coverage.xml\n```\n\n\n\nThis screenshot shows how the YAML output only applies coverage highlighting to\nthe parts of the code where the coverage has changed (from covered to\nuncovered, or vice versa).\n\n```shell\npycobertura diff --format yaml tests/dummy.source1/coverage.xml tests/dummy.source2/coverage.xml\n```\n\n\n\nThe following shows how to generate GitHub annotations given a coverage file.\n\n```shell\n$ pycobertura diff --format github-annotation tests/dummy.source1/coverage.xml tests/dummy.source2/coverage.xml\n::notice file=dummy/dummy2.py,line=5,endLine=5,title=pycobertura::not covered\n::notice file=dummy/dummy3.py,line=1,endLine=2,title=pycobertura::not covered\n```\n\nIf you run it in GitHub Actions/Apps, the above log generates check annotations.\n\n\n\n\n### Option to exclude files via --ignore-regex-option\nYou can specify the `--ignore-regex` option to exclude files that have a certain pattern either by specifying the path to a `.gitignore` file or by entering a Python regex.\nExamples:\n```\npycobertura show tests/cobertura.xml\nFilename Stmts Miss Cover Missing\n------------------------------ ------- ------ ------- ---------\nMain.java 15 0 100.00%\nsearch/BinarySearch.java 12 1 91.67% 24\nsearch/ISortedArraySearch.java 0 0 100.00%\nsearch/LinearSearch.java 7 2 71.43% 19-24\nTOTAL 34 3 90.00%\n```\n```\npycobertura show tests/cobertura.xml --ignore-regex \".*Search\"\nFilename Stmts Miss Cover Missing\n---------- ------- ------ ------- ---------\nMain.java 15 0 100.00%\nTOTAL 34 3 90.00%\n```\n> NOTE: Putting just a `*`, e.g. \"*Search\" above will lead to an error, you need to really follow Python regex conventions to use this option.\nYou can also specify a `.testgitignore` with the following content:\n```\n# Test file for ignore regexes\n\n**/__pycache__\n**/dummy*\n\n*.xml\n\n**/*Search*\n```\nand then use\n```\npycobertura show tests/cobertura.xml --ignore-regex \"tests/.testgitignore\"\nFilename Stmts Miss Cover Missing\n---------- ------- ------ ------- ---------\nMain.java 15 0 100.00%\nTOTAL 34 3 90.00%\n```\nThe same will also work for `diff` commands.\n\n#### `diff` exit codes\n\nUpon exit, the `diff` command may return various exit codes:\n\n* 0: all changes are covered, no new uncovered statements have been introduced\n* 1: some exception occurred (likely due to inappropriate usage or a bug in pycobertura)\n* 2: the changes worsened the overall coverage\n* 3: the changes introduced uncovered statements but the overall coverage is still better than before\n\nYou can capture the exit code by using the `$?` syntax from your shell:\n\n```bash\npycobertura ...\nPYCOBERTURA_EXIT_CODE=$?\necho $PYCOBERTURA_EXIT_CODE\n```\n\n## Library usage\n\nUsing it as a library in your Python application is easy:\n\n```python\nfrom pycobertura import Cobertura\ncobertura = Cobertura('coverage.xml')\n\ncobertura.version == '4.0.2'\ncobertura.line_rate() == 1.0 # 100%\ncobertura.files() == [\n 'pycobertura/__init__.py',\n 'pycobertura/cli.py',\n 'pycobertura/cobertura.py',\n 'pycobertura/reporters.py',\n 'pycobertura/utils.py',\n]\ncobertura.line_rate('pycobertura/cli.py') == 1.0\n\nfrom pycobertura import TextReporter\ntr = TextReporter(cobertura)\ntr.generate() == \"\"\"\\\nFilename Stmts Miss Cover Missing\n------------------------- ------- ------ ------- ---------\npycobertura/__init__.py 1 0 100.00%\npycobertura/cli.py 18 0 100.00%\npycobertura/cobertura.py 93 0 100.00%\npycobertura/reporters.py 129 0 100.00%\npycobertura/utils.py 12 0 100.00%\nTOTAL 253 0 100.00%\"\"\"\n\nfrom pycobertura import TextReporterDelta\n\ncoverage1 = Cobertura('coverage1.xml')\ncoverage2 = Cobertura('coverage2.xml')\ndelta = TextReporterDelta(coverage1, coverage2)\ndelta.generate() == \"\"\"\\\nFilename Stmts Miss Cover Missing\n---------------- ------- ------ -------- ---------\ndummy/dummy.py - -2 +50.00% -2, -5\ndummy/dummy2.py +2 - +100.00%\nTOTAL +2 -2 +50.00%\"\"\"\n```\n\n## How to contribute?\n\nFound a bug/typo? Got a patch? Have an idea? Please use Github issues or fork\npycobertura and submit a pull request (PR). All contributions are welcome!\n\nIf you submit a PR:\n\n* ensure the description of your PR illustrates your changes clearly by showing\n what the problem was and how you fixed it (before/after)\n* make sure your changes are covered with one or more tests\n* add a descriptive note in the CHANGES file under the `Unreleased` section\n* update the README accordingly if your changes outdate the documentation\n* make sure all tests are passing using `tox`\n\n```shell\npip install tox\ntox\n```\n\n## FAQ\n\n### How does pycobertura work?\n\nPycobertura takes two different Cobertura reports and compares them line by\nline. If the coverage status of a line changed from covered to uncovered or\nvice versa, then pycobertura will report it. Sometimes you have no code changes\nat all, the only changes were to add more tests and pycobertura will show you\nthe progress.\n\nPycobertura was initially designed as a general purpose Cobertura parser and\ncan generate a summary table for a single Cobertura file (the `show` command).\n\n### I only have one Cobertura report, can I just see my uncovered changes?\n\nYes. All you have to do is pass your same coverage report twice and provide the\npath to the two different code bases:\n\n```shell\npycobertura diff coverage.xml coverage.xml --source1 master/ --source2 myfeature/\n```\n\nBut keep in mind that this will not show you if your changes have introduced a\ndrop in coverage elsewhere in the code base.\n\n### Why doesn't pycobertura use git to diff the source given revision SHAs rather than passing paths to the source code?\n\nBecause we would have to support N version control systems (VCS). It is easy\nenough to generate a directory that contains the source code at a given commit\nor branch name that it's not a top priority for pycobertura to be VCS-aware:\n\n```shell\ngit archive --prefix=source1/ ${BRANCH_OR_COMMIT1} | tar -xf -\ngit archive --prefix=source2/ ${BRANCH_OR_COMMIT2} | tar -xf -\npycobertura diff --source1 source1/ --source2 source2/ coverage1.xml coverage2.xml -o output.html\nrm -rf source1/ source2/\n```\n\nMercurial has `hg archive` and Subversion has `svn export`. These are simple\npre-steps to running `pycobertura diff`.\n\nAlso, the code repository may not always be available at the time pycobertura\nis run. Typically, in Continuous Delivery pipelines, only\n[artifacts](http://en.wikipedia.org/wiki/Artifact_%28software_development%29)\nare available.\n\n### Why do I need to provide the path to the source code directory?\n\nWith the command `pycobertura show`, you don't need to provide the source code\ndirectory, unless you want the HTML output which will conveniently render the\nhighlighted source code for you.\n\nBut with `pycobertura diff`, if you care about *which* lines are covered or\nuncovered (and not just a global count), then you will need to provide the\nsource for *each* of the reports.\n\nTo better understand why, let's assume we have 2 Cobertura reports with the\nfollowing info:\n\nReport A:\n\n```shell\nline 1, hit\nline 2, miss\nline 3, hit\n```\n\nand Report B:\n\n```shell\nline 1, hit\nline 2, miss\nline 3, hit\nline 4, miss\nline 5, hit\n```\n\nHow can you tell which lines need to be highlighted? Naively, you'd assume that\nlines 4-5 were added and these should be the highlighted lines, the ones part\nof your coverage diff. Well, that doesn't quite work.\n\nThe code for Report A is:\n\n```python\nif foo is True: # line 1\n total += 1 # line 2\nreturn total # line 3\n```\n\nThe code for Report B is:\n\n```python\nif foo is False: # line 1 # new line\n total -= 1 # line 2 # new line\nelif foo is True: # line 3 # modified line\n total += 1 # line 4, unchanged\nreturn total # line 5, unchanged\n```\n\nThe code change are lines 1-3 and these are the ones you want to highlight.\nLines 4-5 don't need to be highlighted (unless coverage status changed\nin-between).\n\nSo, to accurately highlight the lines that have changed, the coverage reports\nalone are not sufficient and this is why you need to provide the path to the\nsource that was used to generate each of the Cobertura reports and diff them to\nsee which lines actually changed to report accurate coverage.\n\n### When should I use pycobertura?\n\npycobertura was built as a tool to educate developers about the testing culture\nin such way that any code change should have one or more tests along with it.\n\nYou can use pycobertura in your Continuous Integration (CI) or Continuous\nDelivery (CD) pipeline which would fail a build if the code changes worsened\nthe coverage. For example, when a pull request is submitted, the new code\nshould have equal or better coverage than the branch it's going to be merged\ninto. Or if code navigates through a release pipeline and the new code has\nworse coverage than what's already in Production, then the release is aborted.\n\nWhen a build is triggered by your CI/CD pipeline, each testing stage would\ntypically store an\n[artifact](http://en.wikipedia.org/wiki/Artifact_%28software_development%29) of\nthe source code and another one of the Cobertura report. An extra stage in the\npipeline could ensure that the coverage did not worsen. This can be done by\nretrieving the artifacts of the current build as well as the \"target\" artifacts\n(code and Cobertura report of Production or target branch of a pull request).\nThen `pycobertura diff` will take care of failing the build if the coverage\nworsened (return a non-zero exit code) and then the pycobertura report can be\npublished as an artifact to make it available to developers to look at.\n\nThe step could look like this:\n\n```shell\n# Download artifacts of current build\ncurl -o coverage.${BUILD_ID}.xml https://ciserver/artifacts/${BUILD_ID}/coverage.xml\ncurl -o source.${BUILD_ID}.zip https://ciserver/artifacts/${BUILD_ID}/source.zip\n\n# Download artifacts of already-in-Prod build\ncurl -o coverage.${PROD_BUILD}.xml https://ciserver/artifacts/${PROD_BUILD}/coverage.xml\ncurl -o source.${PROD_BUILD}.zip https://ciserver/artifacts/${PROD_BUILD}/source.zip\n\nunzip source.${BUILD_ID}.zip -d source.${BUILD_ID}\nunzip source.${PROD_BUILD}.zip -d source.${PROD_BUILD}\n\n# Compare\npycobertura diff --format html \\\n --output pycobertura-diff.${BUILD_ID}.html \\\n --source1 source.${PROD_BUILD} \\\n --source2 source.${BUILD_ID} \\\n coverage.${PROD_BUILD}.xml \\\n coverage.${BUILD_ID}.xml\n\n# Upload the pycobertura report artifact\ncurl -F filedata=@pycobertura-diff.${BUILD_ID}.html http://ciserver/artifacts/${BUILD_ID}/\n```\n\n### Why is the number of uncovered lines used as the metric to check if code coverage worsened rather than the line rate?\n\nThe line rate (percentage of covered lines) can legitimately go down for a\nnumber of reasons. To illustrate, suppose we have this code coverage report for\nversion A of our code:\n\n```shell\nline 1: hit\nline 2: hit\nline 3: miss\nline 4: hit\nline 5: hit\n```\n\nHere, the line rate is 80% and uncovered lines is 1 (miss). Later in version B\nof our code, we legitimately delete a covered line and the following coverage\nreport is generated:\n\n```shell\nline 1: hit\n### line deleted ###\nline 2: miss\nline 3: hit\nline 4: hit\n```\n\nThe line rate decreased from 80% to 75% but uncovered lines is still at 1. In\nthis case, failing the build based on line rate is inappropriate, thus making\nthe line rate the wrong metric to look at when validating coverage.\n\nThe basic idea is that a code base may have technical debt of N uncovered lines\nand you want to prevent N from ever going up.\n\n### pycobertura sounds cool, but how to I generate a Cobertura file?\n\nDepending on your programing language, you need to find a tool that measures\ncode coverage and generates a Cobertura report which is an XML representation\nof your code coverage results.\n\nIn Python, [coverage.py](http://nedbatchelder.com/code/coverage/) is a great\ntool for measuring code coverage and plugins such as\n[pytest-cov](https://pypi.python.org/pypi/pytest-cov) for\n[pytest](http://pytest.org/latest/) and\n[nosexcover](https://pypi.python.org/pypi/nosexcover) for\n[nose](https://nose.readthedocs.org/en/latest/) are available to generate\nCobertura reports while running tests.\n\n[Istanbul](https://github.com/gotwarlost/istanbul) can generate\nCobertura reports for Javascript and also has plugins for\n[CoffeeScript](https://github.com/benbria/coffee-coverage) if that is your\ncup of tea.\n\nCobertura is a very common file format available in many testing tools for\npretty much all programing languages. pycobertura is language agnostic and\nshould work with reports generated by tools in any language. But it was mostly\ndeveloped and tested against reports generated with the `pytest-cov` plugin in\nPython. If you see issues, please [create a\nticket](https://github.com/SurveyMonkey/pycobertura/issues).\n\n## Logo credit\n\nThe pycobertura logo, Aysha, was graciously\n[donated](https://github.com/arasatasaygin/openlogos/issues/19) by\n[Open Logos](http://openlogos.org/). \ud83d\ude47\u200d Check them out!\n",
"bugtrack_url": null,
"license": "",
"summary": "\"A Cobertura coverage parser that can diff reports and show coverage progress.\"",
"version": "3.3.1",
"project_urls": {
"Homepage": "https://github.com/aconrad/pycobertura"
},
"split_keywords": [
"\"cobertura",
"coverage",
"diff",
"report",
"parser",
"parse",
"xml\""
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "dc262ebe8cf529b4f4fbcac4c1847efc512e891aa9edd7e946103cd487a89f16",
"md5": "d4fbbe9d1d342bcbec56c64cb1d3fa90",
"sha256": "47b7829d3ea35bf30fb243e20201f599607d32b698ba85ef33acb3cbd2a8582e"
},
"downloads": -1,
"filename": "pycobertura-3.3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d4fbbe9d1d342bcbec56c64cb1d3fa90",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 33231,
"upload_time": "2024-02-17T21:36:00",
"upload_time_iso_8601": "2024-02-17T21:36:00.151881Z",
"url": "https://files.pythonhosted.org/packages/dc/26/2ebe8cf529b4f4fbcac4c1847efc512e891aa9edd7e946103cd487a89f16/pycobertura-3.3.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5e572d33605526a1cff9805a5283a348346fbde8ff9a838cd457dee25ea933c0",
"md5": "d466dc69c4db662186c99dd8aa493f1d",
"sha256": "f0e9d6ba36dd706ba3601c6de525ac4be6df35a59718956dc4158a9fd42dbbd2"
},
"downloads": -1,
"filename": "pycobertura-3.3.1.tar.gz",
"has_sig": false,
"md5_digest": "d466dc69c4db662186c99dd8aa493f1d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 497231,
"upload_time": "2024-02-17T21:36:02",
"upload_time_iso_8601": "2024-02-17T21:36:02.780782Z",
"url": "https://files.pythonhosted.org/packages/5e/57/2d33605526a1cff9805a5283a348346fbde8ff9a838cd457dee25ea933c0/pycobertura-3.3.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-17 21:36:02",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "aconrad",
"github_project": "pycobertura",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"tox": true,
"lcname": "pycobertura"
}
JSON
