# coveo-testing
A set of test/pytest helpers to facilitate common routines.
Content in a nutshell:
- Reusable pytest markers (UnitTest, IntegrationTest)
- Unique ID generation for tests
- Multiline logging assertions with includes, excludes, levels and comprehensive assertion output
- Refactorable `unittest.mock.patch('this.module')` module references
- Human-readable (but still customizable) display for parametrized tests
This project is used as the test base for all other projects in this repository.
Therefore, it cannot depend on any of them.
More complex use cases may be implemented in the `coveo-testing-extras` project.
That's also where you can depend on projects that depend on `coveo-testing`.
# pytest markers and auto-registration
This enables code completion on markers.
Three markers are already provided: `[UnitTest, Integration, Interactive]`
Here's how you can create additional markers:
```python
# /test_some_module/markers.py
import pytest
DockerTest = pytest.mark.docker_test
CloudTest = pytest.mark.cloud_test
ALL_MARKERS = [DockerTest, CloudTest]
```
You can then import these markers and decorate your test functions accordingly:
```python
# /test_some_module/test_something.py
from coveo_testing.markers import UnitTest, Integration, Interactive
from test_some_module.markers import CloudTest, DockerTest
@UnitTest
def test_unit() -> None:
... # designed to be fast and lightweight, most likely parametrized
@Integration
def test_integration() -> None:
... # combines multiple features to achieve a test
@CloudTest
def test_in_the_cloud() -> None:
... # this could be a post-deployment test, for instance.
@DockerTest
@Integration
def test_through_docker() -> None:
... # will run whenever docker tests or integration tests are requested
@Interactive
def test_interactive() -> None:
... # these tests rely on eye-validations, special developer setups, etc
```
Pytest will issue a warning when markers are not registered.
To register coveo-testing's markers along with your custom markers, use the provided `register_markers` method:
```python
# /test_some_module/conftest.py
from _pytest.config import Config
from coveo_testing.markers import register_markers
from test_some_module.markers import ALL_MARKERS
def pytest_configure(config: Config) -> None:
"""This pytest hook is ran once, before collecting tests."""
register_markers(config, *ALL_MARKERS)
```
# Human-readable unique ID generation
The generated ID has this format:
`friendly-name.timestamp.pid.host.executor.sequence`
- friendly-name:
- provided by you, for your own benefit
- timestamp:
- format "%m%d%H%M%S" (month, day, hour, minutes, seconds)
- computed once, when TestId is imported
- pid:
- the pid of the python process
- host:
- the network name of the machine
- executor:
- the content of the `EXECUTOR_NUMBER` environment variable
- returns 'default' when not defined
- historically, this variable comes from jenkins
- conceptually, it can be used to help distribute (and identify) tests and executors
- sequence:
- Thread-safe
- Each `friendly-name` has an isolated `sequence` that starts at 0
- Incremented on each new instance
- Enables support for parallel parametrized tests
```python
from coveo_testing.temporary_resource.unique_id import TestId, unique_test_id
# the friendly name is the only thing you need to specify
test_id = TestId('friendly-name')
str(test_id)
'friendly-name.0202152243.18836.WORKSTATION.default.0'
# you can pass the instance around to share the ID
str(test_id)
'friendly-name.0202152243.18836.WORKSTATION.default.0'
# create new instances to increment the sequence number
test_id = TestId('friendly-name')
str(test_id)
'friendly-name.0202152243.18836.WORKSTATION.default.1'
# use it in parallel parameterized tests
import pytest
@pytest.mark.parametrize('param', (True, False))
def test_param(param: bool, unique_test_id: TestId) -> None:
# in this case, the friendly name is the function name and
# the sequence will increase on each parameter
# test_param.0202152243.18836.WORKSTATION.default.0
# test_param.0202152243.18836.WORKSTATION.default.1
...
```
# multiline logging assertions
Maybe pytest's `caplog` is enough for your needs, or maybe you need more options.
This tool uses `in` and `not in` to match strings in a case-sensitive way.
```python
import logging
from coveo_testing.logging import assert_logging
with assert_logging(
logging.getLogger('logger-name'),
present=['evidence1', 'evidence2'],
absent=[...],
level=logging.WARN):
...
```
# Human-readable (but still customizable) display for parametrized tests
If you're like me, you typed `@pytest.mark.parametrize` wrong a couple of times!
Enable IDE completion by using this one instead:
```python
from coveo_testing.parametrize import parametrize
@parametrize('var', (True, False))
def test_var(var: bool) -> None:
...
```
It has one difference vs the pytest one, and it's the way it formats the "parameter name" for each iteration of the test.
Pytest will skip a lot of types and will simply name your test "var0", "var1" and so on.
Using this `@parametrize` instead, the variable's content will be inspected:
```python
from typing import Any
from coveo_testing.parametrize import parametrize
import pytest
class StrMe:
def __init__(self, var: Any) -> None:
self.var = var
def __str__(self) -> str:
return f"Value: {self.var}"
@parametrize('var', [['list', 'display'], [StrMe('hello')]])
def test_param(var: bool) -> None:
...
@pytest.mark.parametrize('var', [['list', 'display'], [StrMe('hello')]])
def test_param_from_pytest(var: bool) -> None:
...
```
If you run `pytest --collect-only` you will obtain the following:
```
<Function test_param[list-display]>
<Function test_param[Value: hello]>
<Function test_param_from_pytest[var0]>
<Function test_param_from_pytest[var1]>
```
# Refactorable mock targets
The `ref` tool has moved to its own package called [coveo-ref](https://github.com/coveooss/coveo-python-oss/tree/main/coveo-ref).
## Backward Compatibility
You can still continue using `ref` from `coveo-testing`: the backward compatibility patch will not be deprecated.
## Migration Guide
If you'd rather use the new package directly:
- Import the `coveo-ref` dependency into your project
- Replace `from coveo_testing.mocks import ref` by `from coveo_ref import ref`
- Exceptions have been moved to `coveo_ref.exceptions`
Raw data
{
"_id": null,
"home_page": "https://github.com/coveooss/coveo-python-oss/tree/main/coveo-testing",
"name": "coveo-testing",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": null,
"author": "Jonathan Pich\u00e9",
"author_email": "tools@coveo.com",
"download_url": "https://files.pythonhosted.org/packages/d6/6e/7647bbb58f9377e2d6888bee241ce6e768c0dee145199b9b255106e1e295/coveo_testing-2.0.14.tar.gz",
"platform": null,
"description": "# coveo-testing\n\nA set of test/pytest helpers to facilitate common routines.\n\n\nContent in a nutshell:\n\n- Reusable pytest markers (UnitTest, IntegrationTest)\n- Unique ID generation for tests\n- Multiline logging assertions with includes, excludes, levels and comprehensive assertion output\n- Refactorable `unittest.mock.patch('this.module')` module references\n\n- Human-readable (but still customizable) display for parametrized tests\n\n\nThis project is used as the test base for all other projects in this repository.\n\nTherefore, it cannot depend on any of them.\n\nMore complex use cases may be implemented in the `coveo-testing-extras` project. \nThat's also where you can depend on projects that depend on `coveo-testing`. \n\n\n# pytest markers and auto-registration\n\nThis enables code completion on markers.\n\nThree markers are already provided: `[UnitTest, Integration, Interactive]`\n\nHere's how you can create additional markers:\n\n```python\n# /test_some_module/markers.py\nimport pytest\n\nDockerTest = pytest.mark.docker_test\nCloudTest = pytest.mark.cloud_test\n\nALL_MARKERS = [DockerTest, CloudTest]\n```\n\nYou can then import these markers and decorate your test functions accordingly:\n\n```python\n# /test_some_module/test_something.py\nfrom coveo_testing.markers import UnitTest, Integration, Interactive\nfrom test_some_module.markers import CloudTest, DockerTest\n\n@UnitTest\ndef test_unit() -> None:\n ... # designed to be fast and lightweight, most likely parametrized\n\n\n@Integration\ndef test_integration() -> None:\n ... # combines multiple features to achieve a test\n\n\n@CloudTest\ndef test_in_the_cloud() -> None:\n ... # this could be a post-deployment test, for instance.\n\n\n@DockerTest\n@Integration\ndef test_through_docker() -> None:\n ... # will run whenever docker tests or integration tests are requested\n\n\n@Interactive\ndef test_interactive() -> None:\n ... # these tests rely on eye-validations, special developer setups, etc \n\n```\n\nPytest will issue a warning when markers are not registered.\n\nTo register coveo-testing's markers along with your custom markers, use the provided `register_markers` method:\n\n```python\n# /test_some_module/conftest.py\nfrom _pytest.config import Config\nfrom coveo_testing.markers import register_markers\nfrom test_some_module.markers import ALL_MARKERS\n\ndef pytest_configure(config: Config) -> None:\n \"\"\"This pytest hook is ran once, before collecting tests.\"\"\"\n register_markers(config, *ALL_MARKERS)\n```\n\n\n# Human-readable unique ID generation\n\nThe generated ID has this format:\n\n`friendly-name.timestamp.pid.host.executor.sequence`\n\n- friendly-name:\n - provided by you, for your own benefit\n \n- timestamp: \n - format \"%m%d%H%M%S\" (month, day, hour, minutes, seconds)\n - computed once, when TestId is imported\n \n- pid:\n - the pid of the python process\n \n- host:\n - the network name of the machine\n\n- executor:\n - the content of the `EXECUTOR_NUMBER` environment variable\n - returns 'default' when not defined \n - historically, this variable comes from jenkins\n - conceptually, it can be used to help distribute (and identify) tests and executors\n\n- sequence:\n - Thread-safe\n - Each `friendly-name` has an isolated `sequence` that starts at 0\n - Incremented on each new instance\n - Enables support for parallel parametrized tests\n\n```python\nfrom coveo_testing.temporary_resource.unique_id import TestId, unique_test_id\n\n\n# the friendly name is the only thing you need to specify\ntest_id = TestId('friendly-name')\nstr(test_id)\n'friendly-name.0202152243.18836.WORKSTATION.default.0'\n\n\n# you can pass the instance around to share the ID\nstr(test_id)\n'friendly-name.0202152243.18836.WORKSTATION.default.0'\n\n\n# create new instances to increment the sequence number\ntest_id = TestId('friendly-name')\nstr(test_id)\n'friendly-name.0202152243.18836.WORKSTATION.default.1'\n\n\n# use it in parallel parameterized tests\nimport pytest\n\n@pytest.mark.parametrize('param', (True, False))\ndef test_param(param: bool, unique_test_id: TestId) -> None:\n # in this case, the friendly name is the function name and\n # the sequence will increase on each parameter\n # test_param.0202152243.18836.WORKSTATION.default.0\n # test_param.0202152243.18836.WORKSTATION.default.1\n ...\n```\n\n\n# multiline logging assertions\n\nMaybe pytest's `caplog` is enough for your needs, or maybe you need more options.\nThis tool uses `in` and `not in` to match strings in a case-sensitive way.\n\n```python\nimport logging\nfrom coveo_testing.logging import assert_logging\n\nwith assert_logging(\n logging.getLogger('logger-name'),\n present=['evidence1', 'evidence2'], \n absent=[...], \n level=logging.WARN):\n ...\n```\n\n\n# Human-readable (but still customizable) display for parametrized tests\n\nIf you're like me, you typed `@pytest.mark.parametrize` wrong a couple of times!\n\nEnable IDE completion by using this one instead:\n\n```python\nfrom coveo_testing.parametrize import parametrize\n\n@parametrize('var', (True, False))\ndef test_var(var: bool) -> None:\n ...\n```\n\nIt has one difference vs the pytest one, and it's the way it formats the \"parameter name\" for each iteration of the test.\n\nPytest will skip a lot of types and will simply name your test \"var0\", \"var1\" and so on.\nUsing this `@parametrize` instead, the variable's content will be inspected:\n\n```python\nfrom typing import Any\nfrom coveo_testing.parametrize import parametrize\nimport pytest\n\n\nclass StrMe:\n def __init__(self, var: Any) -> None:\n self.var = var\n \n def __str__(self) -> str:\n return f\"Value: {self.var}\"\n\n\n@parametrize('var', [['list', 'display'], [StrMe('hello')]])\ndef test_param(var: bool) -> None:\n ...\n\n@pytest.mark.parametrize('var', [['list', 'display'], [StrMe('hello')]])\ndef test_param_from_pytest(var: bool) -> None:\n ...\n```\n\nIf you run `pytest --collect-only` you will obtain the following:\n```\n <Function test_param[list-display]>\n <Function test_param[Value: hello]>\n <Function test_param_from_pytest[var0]>\n <Function test_param_from_pytest[var1]>\n```\n\n\n# Refactorable mock targets\n\nThe `ref` tool has moved to its own package called [coveo-ref](https://github.com/coveooss/coveo-python-oss/tree/main/coveo-ref).\n\n## Backward Compatibility\n\nYou can still continue using `ref` from `coveo-testing`: the backward compatibility patch will not be deprecated.\n\n## Migration Guide\n\nIf you'd rather use the new package directly:\n\n- Import the `coveo-ref` dependency into your project\n- Replace `from coveo_testing.mocks import ref` by `from coveo_ref import ref`\n- Exceptions have been moved to `coveo_ref.exceptions`\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Lightweight testing helpers",
"version": "2.0.14",
"project_urls": {
"Homepage": "https://github.com/coveooss/coveo-python-oss/tree/main/coveo-testing",
"Repository": "https://github.com/coveooss/coveo-python-oss/tree/main/coveo-testing"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a1646cee01f574fcf82b16b796e6ebf42acc405186078df170b2c9f5d87bde43",
"md5": "55200d84ca9a97b7bb49e1177d59af38",
"sha256": "ed690afcf398d16dd7fa9b936444eb1e9ac836cc0b891b5fc73665bef6c5b418"
},
"downloads": -1,
"filename": "coveo_testing-2.0.14-py3-none-any.whl",
"has_sig": false,
"md5_digest": "55200d84ca9a97b7bb49e1177d59af38",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 9648,
"upload_time": "2024-03-24T12:46:10",
"upload_time_iso_8601": "2024-03-24T12:46:10.851973Z",
"url": "https://files.pythonhosted.org/packages/a1/64/6cee01f574fcf82b16b796e6ebf42acc405186078df170b2c9f5d87bde43/coveo_testing-2.0.14-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d66e7647bbb58f9377e2d6888bee241ce6e768c0dee145199b9b255106e1e295",
"md5": "be2022530be25cc4e8b977e795cc7339",
"sha256": "cb8d449214a51277048d5ce22f1022fe6276c39667eaa3f0185a9be8360c9f1b"
},
"downloads": -1,
"filename": "coveo_testing-2.0.14.tar.gz",
"has_sig": false,
"md5_digest": "be2022530be25cc4e8b977e795cc7339",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 9618,
"upload_time": "2024-03-24T12:46:12",
"upload_time_iso_8601": "2024-03-24T12:46:12.800438Z",
"url": "https://files.pythonhosted.org/packages/d6/6e/7647bbb58f9377e2d6888bee241ce6e768c0dee145199b9b255106e1e295/coveo_testing-2.0.14.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-24 12:46:12",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "coveooss",
"github_project": "coveo-python-oss",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "coveo-testing"
}
JSON
