| Name | pytest-runtime-xfail JSON |
| Version |
1.1.1
JSON |
| download |
| home_page | None |
| Summary | Call runtime_xfail() to mark running test as xfail. |
| upload_time | 2025-10-10 19:48:45 |
| maintainer | None |
| docs_url | None |
| author | Brian Okken |
| requires_python | >=3.8 |
| license | MIT License
Copyright (c) 2021 Brian Okken
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE. |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# pytest-runtime-xfail
pytest plugin, providing a `runtime_xfail` fixture, which is callable as `runtime_xfail()`, to allow runtime decisions to mark a test as `xfail`.
## Installation
Install with pip:
pip install pytest-runtime-xfail
## Usage
Include the fixture, then call it if you want to mark a test as `xfail` during runtime.
```python
def test_something(runtime_xfail):
if (runtime_condition):
runtime_xfail()
# ... the rest of your test
```
Can also be used in a fixture, of course.
```python
@pytest.fixture()
def foo(runtime_xfail):
if (runtime_condition):
runtime_xfail()
# ... the rest of your fixture
def test_something(foo):
# ... the rest of your test
```
## Reason this plugin is needed
pytest allows you to mark tests as expected to fail, or xfail, in two ways.
1. `@pytest.mark.xfail`. This allows you to mark tests or test parametrizations as `xfail` during test collection time.
* pytest runs tests marked with `xfail` just like any other test.
* If the test fails, it will result in `XFAIL`.
* If it passes, `XPASS`. Unless you have `xfail_strict=true` or `@pytest.mark.xfail(strict=True)`, in which case, passing xfail-marked tests will result in `FAIL`.
* This is useful to be alerted when an expected failing test starts to pass.
2. `pytest.xfail()`. If you need information only known at runtime to decide if `xfail` is appropriate, you can call `pytest.xfail()` during a test or fixture.
* pytest runs the test as normal UNTIL `pytest.xfail()` is called.
* When `pytest.xfail()` is called, the test execution stops and the test results in `XFAIL`.
* The rest of the test is not run.
* There is no way to get `XPASS` from `pytest.xfail()`.
* `xfail_strict` has no effect.
There are times when we want a combination of these behaviors.
* We don't know until runtime if we should mark a test as `xfail`.
* We want the test run.
* We want the possibility of both `XFAIL` and `XPASS` results.
* We want to be able to use `xfail_strict=true` to alert us when the test starts passing.
This plugin fills that gap.
## Alternatives
You can get around the same limitation yourself by adding the marker through the `requests` object:
```python
def test_something(request):
if (runtime_condition):
request.node.add_marker(pytest.mark.xfail(reason='some reason'))
# ... rest of test
```
That's basically what this plugin does, just in a fixture.
## Example found in example/test_xfail.py
```python
"""
Run this with
* pytest -v
* pytest -v -o xfail_strict=true
"""
import pytest
@pytest.mark.xfail()
def test_marker_pass():
'Can be XPASS or FAIL (if xfail_strict)'
assert True
@pytest.mark.xfail()
def test_marker_fail():
'Will always be XFAIL'
assert False # this statememt will be run
def test_old_xfail_pass():
'Will always be XFAIL'
pytest.xfail()
assert True # this statememt will NOT be run
def test_old_xfail_fail():
'Will always be XFAIL'
pytest.xfail()
assert False # this statememt will NOT be run
def test_runtime_xfail_pass(runtime_xfail):
runtime_xfail()
assert True # this statement will be run
def test_runtime_xfail_fail(runtime_xfail):
runtime_xfail()
assert False # this statement will be run
def test_runtime_xfail_reason(runtime_xfail):
runtime_xfail(reason="for demo")
assert False # this statement will be run
```
**Output:**
```
(venv) $ pytest -v test_xfail.py
========================= test session starts ==========================
collected 7 items
test_xfail.py::test_marker_pass XPASS [ 14%]
test_xfail.py::test_marker_fail XFAIL [ 28%]
test_xfail.py::test_old_xfail_pass XFAIL [ 42%]
test_xfail.py::test_old_xfail_fail XFAIL [ 57%]
test_xfail.py::test_runtime_xfail_pass XPASS [ 71%]
test_xfail.py::test_runtime_xfail_fail XFAIL [ 85%]
test_xfail.py::test_runtime_xfail_reason XFAIL (for demo) [100%]
==================== 5 xfailed, 2 xpassed in 0.05s =====================
(venv) $ pytest -v test_xfail.py -o xfail_strict=true
========================= test session starts ==========================
collected 7 items
test_xfail.py::test_marker_pass FAILED [ 14%]
test_xfail.py::test_marker_fail XFAIL [ 28%]
test_xfail.py::test_old_xfail_pass XFAIL [ 42%]
test_xfail.py::test_old_xfail_fail XFAIL [ 57%]
test_xfail.py::test_runtime_xfail_pass FAILED [ 71%]
test_xfail.py::test_runtime_xfail_fail XFAIL [ 85%]
test_xfail.py::test_runtime_xfail_reason XFAIL (for demo) [100%]
===================== 2 failed, 5 xfailed in 0.04s =====================
```
Raw data
{
"_id": null,
"home_page": null,
"name": "pytest-runtime-xfail",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": null,
"author": "Brian Okken",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/dc/12/ffffc8aa869d6bd0b6219e63b7936e29fc97607e7c7cdb010886ef6f69a3/pytest_runtime_xfail-1.1.1.tar.gz",
"platform": null,
"description": "# pytest-runtime-xfail\n\npytest plugin, providing a `runtime_xfail` fixture, which is callable as `runtime_xfail()`, to allow runtime decisions to mark a test as `xfail`. \n\n## Installation\n\nInstall with pip:\n\n pip install pytest-runtime-xfail\n\n## Usage\n\nInclude the fixture, then call it if you want to mark a test as `xfail` during runtime.\n\n```python\n def test_something(runtime_xfail):\n if (runtime_condition):\n runtime_xfail()\n # ... the rest of your test\n```\n\nCan also be used in a fixture, of course.\n\n```python\n@pytest.fixture()\ndef foo(runtime_xfail):\n if (runtime_condition):\n runtime_xfail()\n # ... the rest of your fixture\n\ndef test_something(foo):\n # ... the rest of your test\n```\n\n## Reason this plugin is needed\n\npytest allows you to mark tests as expected to fail, or xfail, in two ways.\n\n1. `@pytest.mark.xfail`. This allows you to mark tests or test parametrizations as `xfail` during test collection time.\n * pytest runs tests marked with `xfail` just like any other test.\n * If the test fails, it will result in `XFAIL`.\n * If it passes, `XPASS`. Unless you have `xfail_strict=true` or `@pytest.mark.xfail(strict=True)`, in which case, passing xfail-marked tests will result in `FAIL`.\n * This is useful to be alerted when an expected failing test starts to pass.\n\n2. `pytest.xfail()`. If you need information only known at runtime to decide if `xfail` is appropriate, you can call `pytest.xfail()` during a test or fixture.\n * pytest runs the test as normal UNTIL `pytest.xfail()` is called.\n * When `pytest.xfail()` is called, the test execution stops and the test results in `XFAIL`.\n * The rest of the test is not run.\n * There is no way to get `XPASS` from `pytest.xfail()`.\n * `xfail_strict` has no effect.\n\n\nThere are times when we want a combination of these behaviors.\n\n* We don't know until runtime if we should mark a test as `xfail`.\n* We want the test run.\n* We want the possibility of both `XFAIL` and `XPASS` results.\n* We want to be able to use `xfail_strict=true` to alert us when the test starts passing.\n\nThis plugin fills that gap.\n\n## Alternatives\n\nYou can get around the same limitation yourself by adding the marker through the `requests` object:\n\n```python\n\ndef test_something(request):\n if (runtime_condition): \n request.node.add_marker(pytest.mark.xfail(reason='some reason'))\n # ... rest of test\n```\n\nThat's basically what this plugin does, just in a fixture.\n\n\n## Example found in example/test_xfail.py\n\n```python\n\"\"\"\nRun this with\n* pytest -v\n* pytest -v -o xfail_strict=true\n\"\"\"\n\nimport pytest\n\n@pytest.mark.xfail()\ndef test_marker_pass():\n 'Can be XPASS or FAIL (if xfail_strict)'\n assert True\n\n@pytest.mark.xfail()\ndef test_marker_fail():\n 'Will always be XFAIL'\n assert False # this statememt will be run\n\ndef test_old_xfail_pass():\n 'Will always be XFAIL'\n pytest.xfail()\n assert True # this statememt will NOT be run\n\ndef test_old_xfail_fail():\n 'Will always be XFAIL'\n pytest.xfail()\n assert False # this statememt will NOT be run\n\ndef test_runtime_xfail_pass(runtime_xfail):\n runtime_xfail()\n assert True # this statement will be run\n\ndef test_runtime_xfail_fail(runtime_xfail):\n runtime_xfail()\n assert False # this statement will be run\n\ndef test_runtime_xfail_reason(runtime_xfail):\n runtime_xfail(reason=\"for demo\")\n assert False # this statement will be run\n```\n\n**Output:**\n\n```\n(venv) $ pytest -v test_xfail.py \n========================= test session starts ==========================\ncollected 7 items \n\ntest_xfail.py::test_marker_pass XPASS [ 14%]\ntest_xfail.py::test_marker_fail XFAIL [ 28%]\ntest_xfail.py::test_old_xfail_pass XFAIL [ 42%]\ntest_xfail.py::test_old_xfail_fail XFAIL [ 57%]\ntest_xfail.py::test_runtime_xfail_pass XPASS [ 71%]\ntest_xfail.py::test_runtime_xfail_fail XFAIL [ 85%]\ntest_xfail.py::test_runtime_xfail_reason XFAIL (for demo) [100%]\n\n==================== 5 xfailed, 2 xpassed in 0.05s =====================\n(venv) $ pytest -v test_xfail.py -o xfail_strict=true\n========================= test session starts ==========================\ncollected 7 items \n\ntest_xfail.py::test_marker_pass FAILED [ 14%]\ntest_xfail.py::test_marker_fail XFAIL [ 28%]\ntest_xfail.py::test_old_xfail_pass XFAIL [ 42%]\ntest_xfail.py::test_old_xfail_fail XFAIL [ 57%]\ntest_xfail.py::test_runtime_xfail_pass FAILED [ 71%]\ntest_xfail.py::test_runtime_xfail_fail XFAIL [ 85%]\ntest_xfail.py::test_runtime_xfail_reason XFAIL (for demo) [100%]\n\n===================== 2 failed, 5 xfailed in 0.04s =====================\n```",
"bugtrack_url": null,
"license": "MIT License\n \n Copyright (c) 2021 Brian Okken\n \n Permission is hereby granted, free of charge, to any person obtaining a copy\n of this software and associated documentation files (the \"Software\"), to deal\n in the Software without restriction, including without limitation the rights\n to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n copies of the Software, and to permit persons to whom the Software is\n furnished to do so, subject to the following conditions:\n \n The above copyright notice and this permission notice shall be included in all\n copies or substantial portions of the Software.\n \n THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n SOFTWARE.",
"summary": "Call runtime_xfail() to mark running test as xfail.",
"version": "1.1.1",
"project_urls": {
"Home": "https://github.com/okken/pytest-runtime-xfail"
},
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "2e0bec56d9c87a653f00bcdcf73160e4ccda3e8dacdf4c91cbc21d116a0392f0",
"md5": "73c5fc103b07738b271113b5f541a3e6",
"sha256": "1927a7505367668bbf0a85ceb9224167bc6335bf24bb2b78b40cca311c6aeb6a"
},
"downloads": -1,
"filename": "pytest_runtime_xfail-1.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "73c5fc103b07738b271113b5f541a3e6",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 4629,
"upload_time": "2025-10-10T19:48:44",
"upload_time_iso_8601": "2025-10-10T19:48:44.184756Z",
"url": "https://files.pythonhosted.org/packages/2e/0b/ec56d9c87a653f00bcdcf73160e4ccda3e8dacdf4c91cbc21d116a0392f0/pytest_runtime_xfail-1.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "dc12ffffc8aa869d6bd0b6219e63b7936e29fc97607e7c7cdb010886ef6f69a3",
"md5": "49df5fb97f647c84f37a01206802c2f6",
"sha256": "773c013dea99700c82e71333b966a20254491a255ced3c4c72bc7ce1c339cf1b"
},
"downloads": -1,
"filename": "pytest_runtime_xfail-1.1.1.tar.gz",
"has_sig": false,
"md5_digest": "49df5fb97f647c84f37a01206802c2f6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 5844,
"upload_time": "2025-10-10T19:48:45",
"upload_time_iso_8601": "2025-10-10T19:48:45.065217Z",
"url": "https://files.pythonhosted.org/packages/dc/12/ffffc8aa869d6bd0b6219e63b7936e29fc97607e7c7cdb010886ef6f69a3/pytest_runtime_xfail-1.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-10 19:48:45",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "okken",
"github_project": "pytest-runtime-xfail",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "pytest-runtime-xfail"
}
