| Name | pytest-smoke JSON |
| Version |
0.6.0
JSON |
| download |
| home_page | None |
| Summary | Pytest plugin for smoke testing |
| upload_time | 2025-01-10 18:43:38 |
| maintainer | None |
| docs_url | None |
| author | Yugo Kato |
| requires_python | >=3.9 |
| license | MIT License Copyright (c) 2024 Yugo Kato Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
pytest
smoke
smoke testing
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
pytest-smoke
======================
[](https://pypi.org/project/pytest-smoke/)
[](https://pypi.org/project/pytest-smoke/)
[](https://github.com/yugokato/pytest-smoke/actions/workflows/test.yml?query=branch%3Amain)
[](https://docs.astral.sh/ruff/)
`pytest-smoke` is a `pytest` plugin designed to quickly perform smoke testing on large test suites. It allows you to
scale down test execution by limiting the number of tests run from each test function or specified scope to a smaller
subset, defined by a value of `N`.
## Installation
```bash
pip install pytest-smoke
```
## Quick Start
For a quick smoke test with all default options, simply run:
```bash
pytest --smoke
```
This will run a small subset of tests from your test suite to quickly check basic functionality.
## Usage
This plugin provides the following command options:
```
$ pytest -h
<snip>
Smoke testing:
--smoke=[N] Run only N tests from each test function or specified scope.
N can be a number (e.g. 5) or a percentage (e.g. 10%).
If not provided, the default value is 1.
--smoke-scope=SCOPE Specify the scope at which the value of N from the above options is applied.
The plugin provides the following predefined scopes, as well as custom user-defined scopes via a hook:
- function: Applies to each test function (default)
- class: Applies to each test class
- auto: Applies function scope for test functions, class scope for test methods
- file: Applies to each test file
- directory: Applies to each test directory
- all: Applies to the entire test suite
--smoke-select-mode=MODE
Specify the mode for selecting tests from each scope.
The plugin provides the following predefined values, as well as custom user-defined values via a hook:
- first: The first N tests (default)
- last: The last N tests
- random: N randomly selected tests
```
> [!NOTE]
> - The `--smoke` option is always required to use any `pytest-smoke` plugin functionality
> - The `--smoke-scope` and `--smoke-select-mode` options also support any custom values, as long as they are handled in the hook. See the "Hooks" section below
> - You can override the plugin's default values for `N`, `SCOPE`, and `MODE` using INI options. See the "INI Options" section below
> - When using the [pytest-xdist](https://pypi.org/project/pytest-xdist/) plugin for parallel testing, you can configure the `pytest-smoke` plugin to replace the default scheduler with a custom distribution algorithm that distributes tests based on the smoke scope
## Examples
Given you have the following test code:
```python
import pytest
def test_something1():
pass
@pytest.mark.parametrize("p", range(10))
def test_something2(p):
pass
@pytest.mark.parametrize("p", range(20))
def test_something3(p):
pass
```
You can run smoke tests on subsets of different sizes with the `--smoke` option.
Here are some basic examples:
- Run only the first test from each test function
```
$ pytest -v --smoke
======================== test session starts ==========================
<snip>
collected 31 items / 28 deselected / 3 selected
tests/test_something.py::test_something1 PASSED [ 33%]
tests/test_something.py::test_something2[0] PASSED [ 66%]
tests/test_something.py::test_something3[0] PASSED [100%]
=================== 3 passed, 28 deselected in 0.02s ===================
```
- Run up to 3 tests from each test function (`N`=3)
```
$ pytest -v --smoke 3
======================== test session starts ==========================
<snip>
collected 31 items / 24 deselected / 7 selected
tests/test_something.py::test_something1 PASSED [ 14%]
tests/test_something.py::test_something2[0] PASSED [ 28%]
tests/test_something.py::test_something2[1] PASSED [ 42%]
tests/test_something.py::test_something2[2] PASSED [ 57%]
tests/test_something.py::test_something3[0] PASSED [ 71%]
tests/test_something.py::test_something3[1] PASSED [ 85%]
tests/test_something.py::test_something3[2] PASSED [100%]
=================== 7 passed, 24 deselected in 0.02s ===================
```
- Run 20% of tests from each test function (`N`=20%)
```
$ pytest -v --smoke 20%
======================== test session starts ==========================
<snip>
collected 31 items / 24 deselected / 7 selected
tests/test_something.py::test_something1 PASSED [ 14%]
tests/test_something.py::test_something2[0] PASSED [ 28%]
tests/test_something.py::test_something2[1] PASSED [ 42%]
tests/test_something.py::test_something3[0] PASSED [ 57%]
tests/test_something.py::test_something3[1] PASSED [ 71%]
tests/test_something.py::test_something3[2] PASSED [ 85%]
tests/test_something.py::test_something3[3] PASSED [100%]
=================== 7 passed, 24 deselected in 0.02s ===================
```
## Markers
### `@pytest.mark.smoke(*, mustpass=False, runif=True)`
When the feature is explicitly enabled via the `smoke_marked_tests_as_critical` INI option, collected tests marked with
`@pytest.mark.smoke` are considered "critical" smoke tests while ones without this marker are considered "regular"
smoke tests. Additionally, if the optional `mustpass` keyword argument is set to `True` in the marker, the test is
considered a "must-pass" critical smoke test.
The plugin will apply the following behavior:
- All collected critical tests with `runif=True` are automatically included, in addition to the regular tests selected as part of `N` (Ones with `runif=False` will be deselected)
- Execute critical smoke tests first, before any regular smoke tests
- If any "must-pass" test fails, all subsequent regular smoke tests will be skipped
> [!NOTE]
> - The marker will have no effect on the plugin until the feature has been enabled
> - When enabled, the plugin assumes that tests will run sequentially. It will not work when running tests in parallel using a plugin like `pytest-xdist`
## Hooks
The plugin provides the following hooks to customize or extend the plugin's capabilities:
### `pytest_smoke_generate_group_id(item, scope)`
This hook allows you to implement your own custom scopes for the `--smoke-scope` option, or override the logic of the
predefined scopes. Items with the same group ID are grouped together and are considered to be in the same scope,
at which `N` is applied.
Any custom values passed to the `--smoke-scope` option must be handled in this hook.
### `pytest_smoke_include(item, scope)`
Return `True` for tests that should be included as "additional" tests. These tests will not be counted towards the
calculation of `N`.
### `pytest_smoke_exclude(item, scope)`
Return `True` for tests that should not be selected. These items will not be included in the total number of tests to
which `N`% is applied. An example use case is to prevent tests that are marked with `skip` and/or `xfail` from being
selected.
Note that this hook takes precedence over any other options provided by the plugin.
### `pytest_smoke_sort_by_select_mode(items, scope, select_mode)`
This hook allows you to implement your own custom select modes for the `--smoke-select-mode` option. Return sorted items
to implement a test selection logic for the custom select mode. The plugin will pick `N` tests from each scope group
based on the sorted items, meaning that an item appearing earlier in the same scope group has a higher chance of being
selected.
Any custom values passed to the `--smoke-select-mode` option must be handled in this hook.
Note that the hook does not affect the test execution order.
## INI Options
You can override the plugin's default values by setting the following options in a configuration
file (pytest.ini, pyproject.toml, etc.).
### `smoke_default_n`
The default `N` value to be applied when not provided to the `--smoke` option.
Plugin default: `1`
### `smoke_default_scope`
The default smoke scope to be applied when not explicitly specified with the `--smoke-scope` option.
Plugin default: `function`
### `smoke_default_select_mode`
The default smoke select mode to be applied when not explicitly specified with the `--smoke-select-mode`
option.
Plugin default: `first`
### `smoke_default_xdist_dist_by_scope`
When using the [pytest-xdist](https://pypi.org/project/pytest-xdist/) plugin (>=2.3.0) for parallel testing, this
option replaces the default scheduler with a custom distribution algorithm that distributes tests based on the smoke
scope. When enabled, the custom scheduler will be automatically used when the `-n`/`--numprocesses` option is used
without a dist option (`--dist` or `-d`).
Plugin default: `false`
### `smoke_marked_tests_as_critical`
Treat tests marked with `@pytest.mark.smoke` as "critical" smoke tests.
Plugin default: `false`
Raw data
{
"_id": null,
"home_page": null,
"name": "pytest-smoke",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "pytest, smoke, smoke testing",
"author": "Yugo Kato",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/24/a9/4092023e16424d3829be72cc05e63c123bf836bb02725d6adfe3433c023f/pytest_smoke-0.6.0.tar.gz",
"platform": null,
"description": "pytest-smoke\n======================\n\n[](https://pypi.org/project/pytest-smoke/)\n[](https://pypi.org/project/pytest-smoke/)\n[](https://github.com/yugokato/pytest-smoke/actions/workflows/test.yml?query=branch%3Amain)\n[](https://docs.astral.sh/ruff/)\n\n`pytest-smoke` is a `pytest` plugin designed to quickly perform smoke testing on large test suites. It allows you to \nscale down test execution by limiting the number of tests run from each test function or specified scope to a smaller \nsubset, defined by a value of `N`.\n\n\n## Installation\n\n```bash\npip install pytest-smoke\n```\n\n\n## Quick Start\n\nFor a quick smoke test with all default options, simply run:\n```bash\npytest --smoke\n```\nThis will run a small subset of tests from your test suite to quickly check basic functionality.\n\n## Usage\n\nThis plugin provides the following command options:\n```\n$ pytest -h\n\n<snip>\n\nSmoke testing:\n --smoke=[N] Run only N tests from each test function or specified scope.\n N can be a number (e.g. 5) or a percentage (e.g. 10%).\n If not provided, the default value is 1.\n --smoke-scope=SCOPE Specify the scope at which the value of N from the above options is applied.\n The plugin provides the following predefined scopes, as well as custom user-defined scopes via a hook:\n - function: Applies to each test function (default)\n - class: Applies to each test class\n - auto: Applies function scope for test functions, class scope for test methods\n - file: Applies to each test file\n - directory: Applies to each test directory\n - all: Applies to the entire test suite\n --smoke-select-mode=MODE\n Specify the mode for selecting tests from each scope.\n The plugin provides the following predefined values, as well as custom user-defined values via a hook:\n - first: The first N tests (default)\n - last: The last N tests\n - random: N randomly selected tests\n```\n\n> [!NOTE]\n> - The `--smoke` option is always required to use any `pytest-smoke` plugin functionality\n> - The `--smoke-scope` and `--smoke-select-mode` options also support any custom values, as long as they are handled in the hook. See the \"Hooks\" section below\n> - You can override the plugin's default values for `N`, `SCOPE`, and `MODE` using INI options. See the \"INI Options\" section below\n> - When using the [pytest-xdist](https://pypi.org/project/pytest-xdist/) plugin for parallel testing, you can configure the `pytest-smoke` plugin to replace the default scheduler with a custom distribution algorithm that distributes tests based on the smoke scope\n\n\n## Examples\n\nGiven you have the following test code:\n\n```python\nimport pytest\n\n\ndef test_something1():\n pass\n \n@pytest.mark.parametrize(\"p\", range(10))\ndef test_something2(p):\n pass\n \n@pytest.mark.parametrize(\"p\", range(20))\ndef test_something3(p):\n pass\n```\n\nYou can run smoke tests on subsets of different sizes with the `--smoke` option. \nHere are some basic examples:\n\n- Run only the first test from each test function\n```\n$ pytest -v --smoke\n======================== test session starts ==========================\n<snip>\ncollected 31 items / 28 deselected / 3 selected\n\ntests/test_something.py::test_something1 PASSED [ 33%]\ntests/test_something.py::test_something2[0] PASSED [ 66%]\ntests/test_something.py::test_something3[0] PASSED [100%]\n\n=================== 3 passed, 28 deselected in 0.02s ===================\n```\n\n- Run up to 3 tests from each test function (`N`=3)\n```\n$ pytest -v --smoke 3\n======================== test session starts ==========================\n<snip>\ncollected 31 items / 24 deselected / 7 selected\n\ntests/test_something.py::test_something1 PASSED [ 14%]\ntests/test_something.py::test_something2[0] PASSED [ 28%]\ntests/test_something.py::test_something2[1] PASSED [ 42%]\ntests/test_something.py::test_something2[2] PASSED [ 57%]\ntests/test_something.py::test_something3[0] PASSED [ 71%]\ntests/test_something.py::test_something3[1] PASSED [ 85%]\ntests/test_something.py::test_something3[2] PASSED [100%]\n\n=================== 7 passed, 24 deselected in 0.02s ===================\n```\n\n- Run 20% of tests from each test function (`N`=20%)\n```\n$ pytest -v --smoke 20%\n======================== test session starts ==========================\n<snip>\ncollected 31 items / 24 deselected / 7 selected\n\ntests/test_something.py::test_something1 PASSED [ 14%]\ntests/test_something.py::test_something2[0] PASSED [ 28%]\ntests/test_something.py::test_something2[1] PASSED [ 42%]\ntests/test_something.py::test_something3[0] PASSED [ 57%]\ntests/test_something.py::test_something3[1] PASSED [ 71%]\ntests/test_something.py::test_something3[2] PASSED [ 85%]\ntests/test_something.py::test_something3[3] PASSED [100%]\n\n=================== 7 passed, 24 deselected in 0.02s ===================\n```\n\n\n## Markers\n\n### `@pytest.mark.smoke(*, mustpass=False, runif=True)`\nWhen the feature is explicitly enabled via the `smoke_marked_tests_as_critical` INI option, collected tests marked with \n`@pytest.mark.smoke` are considered \"critical\" smoke tests while ones without this marker are considered \"regular\" \nsmoke tests. Additionally, if the optional `mustpass` keyword argument is set to `True` in the marker, the test is \nconsidered a \"must-pass\" critical smoke test. \n\nThe plugin will apply the following behavior:\n- All collected critical tests with `runif=True` are automatically included, in addition to the regular tests selected as part of `N` (Ones with `runif=False` will be deselected)\n- Execute critical smoke tests first, before any regular smoke tests\n- If any \"must-pass\" test fails, all subsequent regular smoke tests will be skipped\n\n> [!NOTE]\n> - The marker will have no effect on the plugin until the feature has been enabled\n> - When enabled, the plugin assumes that tests will run sequentially. It will not work when running tests in parallel using a plugin like `pytest-xdist`\n\n\n## Hooks\n\nThe plugin provides the following hooks to customize or extend the plugin's capabilities: \n\n### `pytest_smoke_generate_group_id(item, scope)`\nThis hook allows you to implement your own custom scopes for the `--smoke-scope` option, or override the logic of the \npredefined scopes. Items with the same group ID are grouped together and are considered to be in the same scope, \nat which `N` is applied. \nAny custom values passed to the `--smoke-scope` option must be handled in this hook.\n\n### `pytest_smoke_include(item, scope)`\nReturn `True` for tests that should be included as \"additional\" tests. These tests will not be counted towards the \ncalculation of `N`.\n\n### `pytest_smoke_exclude(item, scope)`\nReturn `True` for tests that should not be selected. These items will not be included in the total number of tests to \nwhich `N`% is applied. An example use case is to prevent tests that are marked with `skip` and/or `xfail` from being \nselected. \nNote that this hook takes precedence over any other options provided by the plugin.\n\n### `pytest_smoke_sort_by_select_mode(items, scope, select_mode)`\nThis hook allows you to implement your own custom select modes for the `--smoke-select-mode` option. Return sorted items \nto implement a test selection logic for the custom select mode. The plugin will pick `N` tests from each scope group \nbased on the sorted items, meaning that an item appearing earlier in the same scope group has a higher chance of being \nselected. \nAny custom values passed to the `--smoke-select-mode` option must be handled in this hook. \nNote that the hook does not affect the test execution order.\n\n\n## INI Options\n\nYou can override the plugin's default values by setting the following options in a configuration \nfile (pytest.ini, pyproject.toml, etc.). \n\n### `smoke_default_n`\nThe default `N` value to be applied when not provided to the `--smoke` option. \nPlugin default: `1`\n\n### `smoke_default_scope`\nThe default smoke scope to be applied when not explicitly specified with the `--smoke-scope` option. \nPlugin default: `function`\n\n### `smoke_default_select_mode`\nThe default smoke select mode to be applied when not explicitly specified with the `--smoke-select-mode` \noption. \nPlugin default: `first`\n\n### `smoke_default_xdist_dist_by_scope`\nWhen using the [pytest-xdist](https://pypi.org/project/pytest-xdist/) plugin (>=2.3.0) for parallel testing, this \noption replaces the default scheduler with a custom distribution algorithm that distributes tests based on the smoke \nscope. When enabled, the custom scheduler will be automatically used when the `-n`/`--numprocesses` option is used \nwithout a dist option (`--dist` or `-d`). \nPlugin default: `false`\n\n### `smoke_marked_tests_as_critical`\nTreat tests marked with `@pytest.mark.smoke` as \"critical\" smoke tests. \nPlugin default: `false`\n",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2024 Yugo Kato Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ",
"summary": "Pytest plugin for smoke testing",
"version": "0.6.0",
"project_urls": {
"Homepage": "https://github.com/yugokato/pytest-smoke"
},
"split_keywords": [
"pytest",
" smoke",
" smoke testing"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6050052961f35b296d844ffe030f0fad763cc17baf593b78b7ab4131176a7fce",
"md5": "796fa93234679aa3c12ee07268c9d887",
"sha256": "bc7ec163821619d98962bab36b6530043aa27156df708af89c30933e170e8fed"
},
"downloads": -1,
"filename": "pytest_smoke-0.6.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "796fa93234679aa3c12ee07268c9d887",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 15133,
"upload_time": "2025-01-10T18:43:36",
"upload_time_iso_8601": "2025-01-10T18:43:36.800461Z",
"url": "https://files.pythonhosted.org/packages/60/50/052961f35b296d844ffe030f0fad763cc17baf593b78b7ab4131176a7fce/pytest_smoke-0.6.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "24a94092023e16424d3829be72cc05e63c123bf836bb02725d6adfe3433c023f",
"md5": "2ee412738b392bb3e55d1243f3f1b545",
"sha256": "cd5b77513d4e942a12bce7fa6add2b698f29827512320b26efc4a4682b24b003"
},
"downloads": -1,
"filename": "pytest_smoke-0.6.0.tar.gz",
"has_sig": false,
"md5_digest": "2ee412738b392bb3e55d1243f3f1b545",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 26422,
"upload_time": "2025-01-10T18:43:38",
"upload_time_iso_8601": "2025-01-10T18:43:38.259210Z",
"url": "https://files.pythonhosted.org/packages/24/a9/4092023e16424d3829be72cc05e63c123bf836bb02725d6adfe3433c023f/pytest_smoke-0.6.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-10 18:43:38",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "yugokato",
"github_project": "pytest-smoke",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "pytest-smoke"
}
