# pytest-ranking
[](https://github.com/softwareTestingResearch/pytest-ranking/actions?workflow=CI)
[](https://pypi.org/project/pytest-ranking)
[](https://results.pre-commit.ci/latest/github/softwareTestingResearch/pytest-ranking/main)
A Pytest plugin for reducing failure detection time of your Python test suite with automated test prioritization/ranking.
This [pytest](https://github.com/pytest-dev/pytest) plugin allows you to find failures faster and receive sooner debugging feedback from CI. It speed up the failure detection of your test suite by prioritizing the execution of tests that are faster, recently failed and/or more textually related to code change.
## Installation
To install `pytest-ranking` via [pip](https://pypi.org/project/pip/) from [PyPI](https://pypi.org/project):
```bash
pip install pytest-ranking
```
## Usage
Pytest will automatically find the plugin and use it when you run ``pytest``. You can use the default prioritization heuristic, which runs tests that have shorter execution time first by passing the ``--rank`` option:
```bash
pytest --rank
```
The terminal output will tell you the current configurations and runtime overhead of this plugin. For example:
```text
[pytest-ranking] Weights: 1-1-0
[pytest-ranking] History length: 30
[pytest-ranking] Number of files with new hashes: 0
[pytest-ranking] Change relatedness computation time (s): 0.0007872581481933594
[pytest-ranking] Test order computation time(s): 0.00020933151245117188
```
You can configure the weights of different prioritization heuristics by passing the optional `--rank-weight` flag with formatted values:
```bash
pytest --rank --rank-weight=0-1-0
```
Weights are separated by hyphens ``-``. The 1st weight is for running faster tests, the 2nd weight is for running recently failed tests, and the 3rd weight is for running tests more textually similar to the changed `*.py` files in the codebase since the last run.
All weights must be integers or floats, and their sum will be normalized to 1.
A higher weight means that a corresponding heuristic is favored. The default value is ``1-0-0``, only prioritizes faster tests.
You can also configure the maximum window size for looking into previous test runs, which is used to compute the number of runs since a test had failed, by passing the optional `--rank-hist-len` flag (the default value is 50):
```bash
pytest --rank --rank-hist-len=30
```
You can always apply these options by adding them to the ``addopts`` setting in your [pytest.ini](https://docs.pytest.org/en/latest/reference/customize.html#configuration).
For example, create `pytest.ini` in your codebase root folder as such:
```ini
[pytest]
addopts = --rank --rank-weight=0-1-0 --rank-hist-len=30
```
and run `pytest` on command line.
Alternatively, you can also create `pytest.ini` in your codebase root folder as such:
```ini
[pytest]
rank_weight=0-1-0
rank_hist_len=30
```
and run `pytest --rank` on command line.
### Warning
Because `pytest-ranking` re-orders tests to speed up failure detection time, please disable other pytest plugins that enforeces other test orders, e.g., [pytest-randomly](https://github.com/pytest-dev/pytest-randomly), [pytest-random-order](https://github.com/pytest-dev/pytest-random-order), [pytest-reverse](https://github.com/adamchainz/pytest-reverse).
## Contributing
Contributions are very welcome. Tests can be run with [tox](https://tox.readthedocs.io/en/latest/).
## License
Distributed under the terms of the [MIT](http://opensource.org/licenses/MIT) license, `pytest-ranking` is free and open source software.
## Issues
If you encounter any problems, please [file an issue](https://github.com/softwareTestingResearch/pytest-ranking/issues) or [pull request](https://github.com/softwareTestingResearch/pytest-ranking/pulls) along with a detailed description.
Raw data
{
"_id": null,
"home_page": "https://github.com/softwareTestingResearch/pytest-ranking",
"name": "pytest-ranking",
"maintainer": "softwareTestingResearch",
"docs_url": null,
"requires_python": ">=3.5",
"maintainer_email": "testingresearch4all@gmail.com",
"keywords": "pytest,py.test,test prioritization,test ordering,test ranking",
"author": "softwareTestingResearch",
"author_email": "testingresearch4all@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/fa/d6/46c0468eec48fde0a62788411980c770820a10947caf0a86374ed2e41c50/pytest-ranking-0.2.6.tar.gz",
"platform": null,
"description": "\n# pytest-ranking\n\n\n\n[](https://github.com/softwareTestingResearch/pytest-ranking/actions?workflow=CI)\n[](https://pypi.org/project/pytest-ranking)\n[](https://results.pre-commit.ci/latest/github/softwareTestingResearch/pytest-ranking/main)\n\n\nA Pytest plugin for reducing failure detection time of your Python test suite with automated test prioritization/ranking.\n\nThis [pytest](https://github.com/pytest-dev/pytest) plugin allows you to find failures faster and receive sooner debugging feedback from CI. It speed up the failure detection of your test suite by prioritizing the execution of tests that are faster, recently failed and/or more textually related to code change.\n\n## Installation\n\nTo install `pytest-ranking` via [pip](https://pypi.org/project/pip/) from [PyPI](https://pypi.org/project):\n\n```bash\npip install pytest-ranking\n```\n\n\n## Usage\n\nPytest will automatically find the plugin and use it when you run ``pytest``. You can use the default prioritization heuristic, which runs tests that have shorter execution time first by passing the ``--rank`` option:\n\n```bash\npytest --rank\n```\n\nThe terminal output will tell you the current configurations and runtime overhead of this plugin. For example:\n\n ```text\n[pytest-ranking] Weights: 1-1-0\n[pytest-ranking] History length: 30\n[pytest-ranking] Number of files with new hashes: 0\n[pytest-ranking] Change relatedness computation time (s): 0.0007872581481933594\n[pytest-ranking] Test order computation time(s): 0.00020933151245117188\n```\n\nYou can configure the weights of different prioritization heuristics by passing the optional `--rank-weight` flag with formatted values:\n\n```bash\npytest --rank --rank-weight=0-1-0\n```\n\nWeights are separated by hyphens ``-``. The 1st weight is for running faster tests, the 2nd weight is for running recently failed tests, and the 3rd weight is for running tests more textually similar to the changed `*.py` files in the codebase since the last run.\nAll weights must be integers or floats, and their sum will be normalized to 1.\nA higher weight means that a corresponding heuristic is favored. The default value is ``1-0-0``, only prioritizes faster tests.\n\n\nYou can also configure the maximum window size for looking into previous test runs, which is used to compute the number of runs since a test had failed, by passing the optional `--rank-hist-len` flag (the default value is 50):\n\n```bash\npytest --rank --rank-hist-len=30\n```\n\n\nYou can always apply these options by adding them to the ``addopts`` setting in your [pytest.ini](https://docs.pytest.org/en/latest/reference/customize.html#configuration).\n\nFor example, create `pytest.ini` in your codebase root folder as such:\n```ini\n[pytest]\naddopts = --rank --rank-weight=0-1-0 --rank-hist-len=30\n```\nand run `pytest` on command line.\n\nAlternatively, you can also create `pytest.ini` in your codebase root folder as such:\n```ini\n[pytest]\nrank_weight=0-1-0\nrank_hist_len=30\n```\n\nand run `pytest --rank` on command line.\n\n\n### Warning\n\nBecause `pytest-ranking` re-orders tests to speed up failure detection time, please disable other pytest plugins that enforeces other test orders, e.g., [pytest-randomly](https://github.com/pytest-dev/pytest-randomly), [pytest-random-order](https://github.com/pytest-dev/pytest-random-order), [pytest-reverse](https://github.com/adamchainz/pytest-reverse).\n\n\n## Contributing\n\nContributions are very welcome. Tests can be run with [tox](https://tox.readthedocs.io/en/latest/).\n\n\n\n## License\n\nDistributed under the terms of the [MIT](http://opensource.org/licenses/MIT) license, `pytest-ranking` is free and open source software.\n\n## Issues\n\nIf you encounter any problems, please [file an issue](https://github.com/softwareTestingResearch/pytest-ranking/issues) or [pull request](https://github.com/softwareTestingResearch/pytest-ranking/pulls) along with a detailed description.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A Pytest plugin for automatically prioritizing/ranking tests to speed up failure detection",
"version": "0.2.6",
"project_urls": {
"Homepage": "https://github.com/softwareTestingResearch/pytest-ranking"
},
"split_keywords": [
"pytest",
"py.test",
"test prioritization",
"test ordering",
"test ranking"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "89d4ef7a8a1f2ff0d90485a8a6f461d57f045f58216e4f0b1d24b516d176affa",
"md5": "bdca54484969aff54496376b2c03b671",
"sha256": "487381f60d3fe82b3e6ffbc391252be95ca70bb91f088f981f822f8cc7bce2f3"
},
"downloads": -1,
"filename": "pytest_ranking-0.2.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "bdca54484969aff54496376b2c03b671",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.5",
"size": 8856,
"upload_time": "2024-03-01T18:04:57",
"upload_time_iso_8601": "2024-03-01T18:04:57.706118Z",
"url": "https://files.pythonhosted.org/packages/89/d4/ef7a8a1f2ff0d90485a8a6f461d57f045f58216e4f0b1d24b516d176affa/pytest_ranking-0.2.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "fad646c0468eec48fde0a62788411980c770820a10947caf0a86374ed2e41c50",
"md5": "25044583d7f080acf0942eca756bf73f",
"sha256": "559b214247a4a408ccc45e6a778f8ea0cd7eae2e464c1b492216e27e42183a9e"
},
"downloads": -1,
"filename": "pytest-ranking-0.2.6.tar.gz",
"has_sig": false,
"md5_digest": "25044583d7f080acf0942eca756bf73f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.5",
"size": 10744,
"upload_time": "2024-03-01T18:04:59",
"upload_time_iso_8601": "2024-03-01T18:04:59.479397Z",
"url": "https://files.pythonhosted.org/packages/fa/d6/46c0468eec48fde0a62788411980c770820a10947caf0a86374ed2e41c50/pytest-ranking-0.2.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-01 18:04:59",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "softwareTestingResearch",
"github_project": "pytest-ranking",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "pytest-ranking"
}
JSON
