# pytest-failure-tracker
A pytest plugin for tracking test failures over multiple test runs, making it easier to identify flaky tests and frequently failing tests.
## Installation
You can install `pytest-failure-tracker` via pip:
```
pip install pytest-failure-tracker
```
## Usage
To use the plugin, simply run pytest with the `--track-failures` option:
```
pytest --track-failures
```
This will enable failure tracking for your test suite.
## Features
### Failure Tracking
The plugin tracks the following information for each test:
- Number of passes
- Number of failures
- Number of skips
- Timestamp of the last failure
- Traceback of the last failure
This data is stored in a JSON file (`test_results.json`) in your project directory.
### Summary Report
After each test run with `--track-failures` enabled, the plugin will display a summary report in the terminal. This report includes:
- Total number of runs for each test
- Number of passes, failures, and skips
- Failure rate
- Timestamp of the last failure
- Traceback of the last failure
### Persistent Storage
Test results are stored persistently, allowing you to track failures across multiple test runs and even across different sessions.
## Example Output
```
============================= Test Failure Tracking Summary =============================
test_module.py::test_example:
Total runs: 10
Passes: 8
Failures: 2
Skips: 0
Failure rate: 20.00%
Last failure: 2024-07-16T14:30:00.123456
Last failure traceback:
File "test_module.py", line 15, in test_example
assert False, "This test intentionally fails sometimes"
AssertionError: This test intentionally fails sometimes
test_module.py::test_another_example:
Total runs: 10
Passes: 10
Failures: 0
Skips: 0
Failure rate: 0.00%
```
## Configuration
Currently, the plugin doesn't require any additional configuration beyond the `--track-failures` command-line option.
## Limitations and Considerations
- The plugin treats parameterized tests as separate tests.
- There is currently no built-in way to reset or clear the tracking data. You can manually delete the `test_results.json` file to reset.
- The plugin doesn't currently provide trend analysis over time.
## Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
## License
[MIT](https://izikeros.mit-license.org/) © [Krystian Safjan](https://safjan.com).
TODO: expand documentation with:
1. More detailed examples of how to interpret the results
2. Any known issues or edge cases
3. A section on how to configure the plugin if you add any configuration options in the future
4. A more detailed contribution guide if you want to encourage open-source contributions
5. A changelog to track versions and updates
Raw data
{
"_id": null,
"home_page": "https://github.com/izikeros/pytest-failure-tracker",
"name": "pytest-failure-tracker",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": null,
"author": "Krystian Safjan",
"author_email": "ksafjan@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/d1/c3/fb29102156736bf03dea4d6e2264fc42a762a0c6f1f28e97c513da25f0c6/pytest_failure_tracker-0.2.3.tar.gz",
"platform": null,
"description": "# pytest-failure-tracker\n\n\n\n\n\nA pytest plugin for tracking test failures over multiple test runs, making it easier to identify flaky tests and frequently failing tests.\n\n## Installation\n\nYou can install `pytest-failure-tracker` via pip:\n\n```\npip install pytest-failure-tracker\n```\n\n## Usage\n\nTo use the plugin, simply run pytest with the `--track-failures` option:\n\n```\npytest --track-failures\n```\n\nThis will enable failure tracking for your test suite.\n\n## Features\n\n### Failure Tracking\n\nThe plugin tracks the following information for each test:\n\n- Number of passes\n- Number of failures\n- Number of skips\n- Timestamp of the last failure\n- Traceback of the last failure\n\nThis data is stored in a JSON file (`test_results.json`) in your project directory.\n\n### Summary Report\n\nAfter each test run with `--track-failures` enabled, the plugin will display a summary report in the terminal. This report includes:\n\n- Total number of runs for each test\n- Number of passes, failures, and skips\n- Failure rate\n- Timestamp of the last failure\n- Traceback of the last failure\n\n### Persistent Storage\n\nTest results are stored persistently, allowing you to track failures across multiple test runs and even across different sessions.\n\n## Example Output\n\n```\n============================= Test Failure Tracking Summary =============================\ntest_module.py::test_example:\n Total runs: 10\n Passes: 8\n Failures: 2\n Skips: 0\n Failure rate: 20.00%\n Last failure: 2024-07-16T14:30:00.123456\n Last failure traceback:\n File \"test_module.py\", line 15, in test_example\n assert False, \"This test intentionally fails sometimes\"\n AssertionError: This test intentionally fails sometimes\n\ntest_module.py::test_another_example:\n Total runs: 10\n Passes: 10\n Failures: 0\n Skips: 0\n Failure rate: 0.00%\n```\n\n## Configuration\n\nCurrently, the plugin doesn't require any additional configuration beyond the `--track-failures` command-line option.\n\n## Limitations and Considerations\n\n- The plugin treats parameterized tests as separate tests.\n- There is currently no built-in way to reset or clear the tracking data. You can manually delete the `test_results.json` file to reset.\n- The plugin doesn't currently provide trend analysis over time.\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## License\n\n[MIT](https://izikeros.mit-license.org/) \u00a9 [Krystian Safjan](https://safjan.com).\n\n\nTODO: expand documentation with:\n\n1. More detailed examples of how to interpret the results\n2. Any known issues or edge cases\n3. A section on how to configure the plugin if you add any configuration options in the future\n4. A more detailed contribution guide if you want to encourage open-source contributions\n5. A changelog to track versions and updates\n",
"bugtrack_url": null,
"license": null,
"summary": "A pytest plugin for tracking test failures over multiple runs",
"version": "0.2.3",
"project_urls": {
"Homepage": "https://github.com/izikeros/pytest-failure-tracker"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "872486565c43455678246692af56c5e5d0699691e1004fab40dbd1b0755c279c",
"md5": "1d39b7221c63face85db456b07a9c7cb",
"sha256": "fbecb7462e2b20e2e26055bd05996407f22247be3e7e85483b7ae6e86860f57f"
},
"downloads": -1,
"filename": "pytest_failure_tracker-0.2.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1d39b7221c63face85db456b07a9c7cb",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 5040,
"upload_time": "2024-07-17T14:29:46",
"upload_time_iso_8601": "2024-07-17T14:29:46.020769Z",
"url": "https://files.pythonhosted.org/packages/87/24/86565c43455678246692af56c5e5d0699691e1004fab40dbd1b0755c279c/pytest_failure_tracker-0.2.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d1c3fb29102156736bf03dea4d6e2264fc42a762a0c6f1f28e97c513da25f0c6",
"md5": "4e4c7bf704d1d8b314f1b1ae8a11b8d8",
"sha256": "d8b22f29094d1e7d3fc8cb2a21c31146fba065acc1358e888d452ac5c3c9bd0d"
},
"downloads": -1,
"filename": "pytest_failure_tracker-0.2.3.tar.gz",
"has_sig": false,
"md5_digest": "4e4c7bf704d1d8b314f1b1ae8a11b8d8",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 10661,
"upload_time": "2024-07-17T14:29:47",
"upload_time_iso_8601": "2024-07-17T14:29:47.007955Z",
"url": "https://files.pythonhosted.org/packages/d1/c3/fb29102156736bf03dea4d6e2264fc42a762a0c6f1f28e97c513da25f0c6/pytest_failure_tracker-0.2.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-17 14:29:47",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "izikeros",
"github_project": "pytest-failure-tracker",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "pytest-failure-tracker"
}
JSON
