| Name | pytest-jira-xray JSON |
| Version |
0.9.1
JSON |
| download |
| home_page | None |
| Summary | pytest plugin to integrate tests with JIRA XRAY |
| upload_time | 2024-10-27 11:26:31 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.7 |
| license | None |
| keywords |
pytest
jira
xray
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
|
================
pytest-jira-xray
================
.. image:: https://img.shields.io/pypi/v/pytest-jira-xray.png
:target: https://pypi.python.org/pypi/pytest-jira-xray
:alt: PyPi
.. image:: https://github.com/fundakol/pytest-jira-xray/actions/workflows/main.yml/badge.svg?branch=master
:target: https://github.com/fundakol/pytest-jira-xray/actions?query=workflow?master
:alt: Build status
.. image:: https://codecov.io/gh/fundakol/pytest-jira-xray/branch/master/graph/badge.svg
:target: https://codecov.io/gh/fundakol/pytest-jira-xray
:alt: Code coverage
pytest-jira-xray is a plugin for pytest that uploads test results to JIRA XRAY.
Installation
------------
Installing from pypi repository:
.. code-block::
pip install -U pytest-jira-xray
Installing from local source:
.. code-block::
pip install <path>
Installing from local source in development mode:
.. code-block::
pip install -e <path>
Usage
-----
Mark a test with JIRA XRAY test ID or list of IDs
.. code-block:: python
# -- FILE: test_example.py
import pytest
@pytest.mark.xray('JIRA-1')
def test_foo():
assert True
@pytest.mark.xray(['JIRA-2', 'JIRA-3'])
def test_bar():
assert True
Jira Xray configuration can be provided via Environment Variables:
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
* Jira base URL:
.. code-block:: bash
$ export XRAY_API_BASE_URL=<Jira base URL>
* Basic authentication with username and password (default)
.. code-block:: bash
$ export XRAY_API_USER=<jira username>
$ export XRAY_API_PASSWORD=<user password>
* Personal Access Token authentication (``--api-key-auth`` option)
.. code-block:: bash
$ export XRAY_API_KEY=<api key>
* SSL Client Certificate
To disable SSL certificate verification, at the client side (no case-sensitive), default is True:
.. code-block:: bash
$ export XRAY_API_VERIFY_SSL=False
Or you can provide path to certificate file
.. code-block:: bash
$ export XRAY_API_VERIFY_SSL=</path/to/PEM file>
* Authentication with client ID and client secret (``--client-secret-auth`` option):
.. code-block:: bash
$ export XRAY_CLIENT_ID=<client id>
$ export XRAY_CLIENT_SECRET=<client secret>
* Test Execution parameters:
.. code-block:: bash
$ export XRAY_EXECUTION_TEST_ENVIRONMENTS="Env1 Env2 Env3"
$ export XRAY_EXECUTION_FIX_VERSION="1.0"
$ export XRAY_EXECUTION_REVISION=`git rev-parse HEAD`
$ export XRAY_EXECUTION_SUMMARY="Smoke tests" # New execution only
$ export XRAY_EXECUTION_DESC="This is an automated test execution of the smoke tests" # New execution only
Upload results
++++++++++++++
* Upload results to new test execution:
.. code-block:: bash
$ pytest --jira-xray
* Upload results to existing test execution:
.. code-block:: bash
$ pytest --jira-xray --execution TestExecutionId
* Upload results to existing test plan (new test execution will be created):
.. code-block:: bash
$ pytest --jira-xray --testplan TestPlanId
* Store results in a file instead of exporting directly to a XRAY server
.. code-block:: bash
$ pytest --jira-xray --xraypath=xray.json
* Use with Jira cloud:
The Xray REST API may use two different endpoints: Server+DC or Cloud.
There is a difference between formats of test results (test status can be ``PASS`` or ``PASSED``),
so if you want to use this plugin with Xray Cloud endpoint you should run pytest with additional
argument ``--cloud``. This will generate test results with the format accepted by the Cloud service.
.. code-block:: bash
$ pytest --jira-xray --cloud
Jira authentication
+++++++++++++++++++
* Jira `basic authentication <https://developer.atlassian.com/server/jira/platform/basic-authentication/>`_:
It is default authentication.
* Jira authentication with `Client ID and a Client Secret <https://docs.getxray.app/display/XRAYCLOUD/Authentication+-+REST+v2>`_:
.. code-block:: bash
$ pytest --jira-xray --client-secret-auth
* Jira `Personal access tokens <https://confluence.atlassian.com/enterprise/using-personal-access-tokens-1026032365.html>`_ (API KEY) authentication:
.. code-block:: bash
$ pytest --jira-xray --api-key-auth
Multiple ids support
++++++++++++++++++++
Tests can be marked to handle multiple Jira tests by adding a list, rather than a string. Example:
.. code-block:: python
# -- FILE: test_example.py
import pytest
@pytest.mark.xray([
'JIRA-1',
'JIRA-2'
])
def test_my_process():
assert True
If the test fails, both JIRA-1 and JIRA-2 tests will be marked as fail. The
failure comment will contain the same message for both tests.
This situation can be useful for validation tests or tests that probe multiple
functionalities in a single run, to reduce execution time.
Duplicated ids support
++++++++++++++++++++++
By default, the jira-xray plugin does not allow to have multiple tests marked with
the same identifier, like in this case:
.. code-block:: python
# -- FILE: test_example.py
import pytest
@pytest.mark.xray('JIRA-1')
def test_my_process_1():
assert True
@pytest.mark.xray('JIRA-1')
def test_my_process_2():
assert True
However, depending how the user story and the associated test are formulated,
this scenario may be useful. The option ``--allow-duplicate-ids`` will perform the tests
even when duplicate ids are present. The JIRA-1 test result will be created according to
the following rules:
- The comment will be the comment from each of the test, separated by a horizontal divider.
- The status will be the intuitive combination of the individual results: if ``test_my_process_1``
is a ``PASS`` but ``test_my_process_2`` is a ``FAIL``, ``JIRA-1`` will be marked as ``FAIL``.
Defects support
+++++++++++++++
Tests can be marked with defects, which will be part of the data uploaded to Xray:
.. code-block:: python
# -- FILE: test_example.py
import pytest
@pytest.mark.xray(
'JIRA-1',
defects=['BUG-1', 'BUG-2']
)
def test_with_defects():
assert True
Note that defects are always present in the data uploaded to Xray, regardless of the test outcome.
Attach test evidences
+++++++++++++++++++++
The following example adds the test evidences to the Xray report
using a ``pytest_runtest_makereport`` hook.
.. code-block:: python
# -- FILE: conftest.py
import pytest
from pytest_xray import evidence
@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
outcome = yield
report = outcome.get_result()
evidences = getattr(report, "evidences", [])
if report.when == "call":
xfail = hasattr(report, "wasxfail")
if (report.skipped and xfail) or (report.failed and not xfail):
data = open("screenshot.jpeg", "rb").read()
evidences.append(evidence.jpeg(data=data, filename="screenshot.jpeg"))
report.evidences = evidences
Hooks
+++++
There is possibility to modify a XRAY report before it is send to a server by ``pytest_xray_results`` hook.
.. code-block:: python
def pytest_xray_results(results, session):
results['info']['user'] = 'pytest'
IntelliJ integration
++++++++++++++++++++
When you want to synchronize your test results via. Pytest integration in IntelliJ, you need to configure the following:
1. Use the *pytest* test configuration template and add ``--jira-xray -o log_cli=true`` to *Additional Arguments*
.. image:: https://user-images.githubusercontent.com/22340156/145638520-c6bf56d2-089e-430c-94ae-ac8122a3adea.png
:target: https://user-images.githubusercontent.com/22340156/145638520-c6bf56d2-089e-430c-94ae-ac8122a3adea.png
2. Disable `--no-summary` in *Settings*
.. image:: https://user-images.githubusercontent.com/22340156/145638538-71590ec8-86c6-4b93-9a99-460b4e38e153.png
:target: https://user-images.githubusercontent.com/22340156/145638538-71590ec8-86c6-4b93-9a99-460b4e38e153.png
Troubleshooting
+++++++++++++++
This section holds information about common issues.
`The Test XXX is in a non-executable status`
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* Problem: The test is not executable by the user.
* Solution: Make sure, that your test is not deactivated, approved and ready to use in Jira.
`Error message from server: fixVersions: fixVersions`
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* Problem: The version is malformed or doesn't exist.
* Solution: Make sure the version exists and the name matches the existing version and that only one version is used.
References
----------
- XRay import execution endpoint: `<https://docs.getxray.app/display/XRAY/Import+Execution+Results>`_
Raw data
{
"_id": null,
"home_page": null,
"name": "pytest-jira-xray",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "pytest, JIRA, XRAY",
"author": null,
"author_email": "Lukasz Fundakowski <fundakol@yahoo.com>",
"download_url": null,
"platform": null,
"description": "================\npytest-jira-xray\n================\n\n.. image:: https://img.shields.io/pypi/v/pytest-jira-xray.png\n :target: https://pypi.python.org/pypi/pytest-jira-xray\n :alt: PyPi\n.. image:: https://github.com/fundakol/pytest-jira-xray/actions/workflows/main.yml/badge.svg?branch=master\n :target: https://github.com/fundakol/pytest-jira-xray/actions?query=workflow?master\n :alt: Build status\n.. image:: https://codecov.io/gh/fundakol/pytest-jira-xray/branch/master/graph/badge.svg\n :target: https://codecov.io/gh/fundakol/pytest-jira-xray\n :alt: Code coverage\n\n\npytest-jira-xray is a plugin for pytest that uploads test results to JIRA XRAY.\n\n\nInstallation\n------------\n\nInstalling from pypi repository:\n\n.. code-block::\n\n pip install -U pytest-jira-xray\n\nInstalling from local source:\n\n.. code-block::\n\n pip install <path>\n\nInstalling from local source in development mode:\n\n.. code-block::\n\n pip install -e <path>\n\n\nUsage\n-----\n\nMark a test with JIRA XRAY test ID or list of IDs\n\n.. code-block:: python\n\n # -- FILE: test_example.py\n\n import pytest\n\n @pytest.mark.xray('JIRA-1')\n def test_foo():\n assert True\n\n @pytest.mark.xray(['JIRA-2', 'JIRA-3'])\n def test_bar():\n assert True\n\n\nJira Xray configuration can be provided via Environment Variables:\n++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n\n* Jira base URL:\n\n.. code-block:: bash\n\n $ export XRAY_API_BASE_URL=<Jira base URL>\n\n\n* Basic authentication with username and password (default)\n\n.. code-block:: bash\n\n $ export XRAY_API_USER=<jira username>\n $ export XRAY_API_PASSWORD=<user password>\n\n* Personal Access Token authentication (``--api-key-auth`` option)\n\n.. code-block:: bash\n\n $ export XRAY_API_KEY=<api key>\n\n* SSL Client Certificate\n\nTo disable SSL certificate verification, at the client side (no case-sensitive), default is True:\n\n.. code-block:: bash\n\n $ export XRAY_API_VERIFY_SSL=False\n\n\nOr you can provide path to certificate file\n\n.. code-block:: bash\n\n $ export XRAY_API_VERIFY_SSL=</path/to/PEM file>\n\n\n* Authentication with client ID and client secret (``--client-secret-auth`` option):\n\n.. code-block:: bash\n\n $ export XRAY_CLIENT_ID=<client id>\n $ export XRAY_CLIENT_SECRET=<client secret>\n\n\n* Test Execution parameters:\n\n.. code-block:: bash\n\n $ export XRAY_EXECUTION_TEST_ENVIRONMENTS=\"Env1 Env2 Env3\"\n $ export XRAY_EXECUTION_FIX_VERSION=\"1.0\"\n $ export XRAY_EXECUTION_REVISION=`git rev-parse HEAD`\n\n $ export XRAY_EXECUTION_SUMMARY=\"Smoke tests\" # New execution only\n $ export XRAY_EXECUTION_DESC=\"This is an automated test execution of the smoke tests\" # New execution only\n\n\nUpload results\n++++++++++++++\n\n* Upload results to new test execution:\n\n.. code-block:: bash\n\n $ pytest --jira-xray\n\n\n* Upload results to existing test execution:\n\n.. code-block:: bash\n\n $ pytest --jira-xray --execution TestExecutionId\n\n\n* Upload results to existing test plan (new test execution will be created):\n\n.. code-block:: bash\n\n $ pytest --jira-xray --testplan TestPlanId\n\n\n* Store results in a file instead of exporting directly to a XRAY server\n\n.. code-block:: bash\n\n $ pytest --jira-xray --xraypath=xray.json\n\n\n* Use with Jira cloud:\n\nThe Xray REST API may use two different endpoints: Server+DC or Cloud.\nThere is a difference between formats of test results (test status can be ``PASS`` or ``PASSED``),\nso if you want to use this plugin with Xray Cloud endpoint you should run pytest with additional\nargument ``--cloud``. This will generate test results with the format accepted by the Cloud service.\n\n.. code-block:: bash\n\n $ pytest --jira-xray --cloud\n\n\nJira authentication\n+++++++++++++++++++\n\n* Jira `basic authentication <https://developer.atlassian.com/server/jira/platform/basic-authentication/>`_:\n\nIt is default authentication.\n\n\n* Jira authentication with `Client ID and a Client Secret <https://docs.getxray.app/display/XRAYCLOUD/Authentication+-+REST+v2>`_:\n\n.. code-block:: bash\n\n $ pytest --jira-xray --client-secret-auth\n\n\n* Jira `Personal access tokens <https://confluence.atlassian.com/enterprise/using-personal-access-tokens-1026032365.html>`_ (API KEY) authentication:\n\n.. code-block:: bash\n\n $ pytest --jira-xray --api-key-auth\n\n\nMultiple ids support\n++++++++++++++++++++\n\nTests can be marked to handle multiple Jira tests by adding a list, rather than a string. Example:\n\n.. code-block:: python\n\n # -- FILE: test_example.py\n\n import pytest\n\n @pytest.mark.xray([\n 'JIRA-1',\n 'JIRA-2'\n ])\n def test_my_process():\n assert True\n\nIf the test fails, both JIRA-1 and JIRA-2 tests will be marked as fail. The\nfailure comment will contain the same message for both tests.\n\nThis situation can be useful for validation tests or tests that probe multiple\nfunctionalities in a single run, to reduce execution time.\n\nDuplicated ids support\n++++++++++++++++++++++\n\nBy default, the jira-xray plugin does not allow to have multiple tests marked with\nthe same identifier, like in this case:\n\n.. code-block:: python\n\n # -- FILE: test_example.py\n\n import pytest\n\n @pytest.mark.xray('JIRA-1')\n def test_my_process_1():\n assert True\n\n @pytest.mark.xray('JIRA-1')\n def test_my_process_2():\n assert True\n\nHowever, depending how the user story and the associated test are formulated,\nthis scenario may be useful. The option ``--allow-duplicate-ids`` will perform the tests\neven when duplicate ids are present. The JIRA-1 test result will be created according to\nthe following rules:\n\n- The comment will be the comment from each of the test, separated by a horizontal divider.\n- The status will be the intuitive combination of the individual results: if ``test_my_process_1``\n is a ``PASS`` but ``test_my_process_2`` is a ``FAIL``, ``JIRA-1`` will be marked as ``FAIL``.\n\nDefects support\n+++++++++++++++\n\nTests can be marked with defects, which will be part of the data uploaded to Xray:\n\n.. code-block:: python\n\n # -- FILE: test_example.py\n\n import pytest\n\n @pytest.mark.xray(\n 'JIRA-1',\n defects=['BUG-1', 'BUG-2']\n )\n def test_with_defects():\n assert True\n\nNote that defects are always present in the data uploaded to Xray, regardless of the test outcome.\n\nAttach test evidences\n+++++++++++++++++++++\n\nThe following example adds the test evidences to the Xray report\nusing a ``pytest_runtest_makereport`` hook.\n\n.. code-block:: python\n\n # -- FILE: conftest.py\n\n import pytest\n from pytest_xray import evidence\n\n @pytest.hookimpl(hookwrapper=True)\n def pytest_runtest_makereport(item, call):\n outcome = yield\n report = outcome.get_result()\n evidences = getattr(report, \"evidences\", [])\n if report.when == \"call\":\n xfail = hasattr(report, \"wasxfail\")\n if (report.skipped and xfail) or (report.failed and not xfail):\n data = open(\"screenshot.jpeg\", \"rb\").read()\n evidences.append(evidence.jpeg(data=data, filename=\"screenshot.jpeg\"))\n report.evidences = evidences\n\n\nHooks\n+++++\n\nThere is possibility to modify a XRAY report before it is send to a server by ``pytest_xray_results`` hook.\n\n.. code-block:: python\n\n def pytest_xray_results(results, session):\n results['info']['user'] = 'pytest'\n\n\nIntelliJ integration\n++++++++++++++++++++\n\nWhen you want to synchronize your test results via. Pytest integration in IntelliJ, you need to configure the following:\n\n1. Use the *pytest* test configuration template and add ``--jira-xray -o log_cli=true`` to *Additional Arguments*\n\n.. image:: https://user-images.githubusercontent.com/22340156/145638520-c6bf56d2-089e-430c-94ae-ac8122a3adea.png\n :target: https://user-images.githubusercontent.com/22340156/145638520-c6bf56d2-089e-430c-94ae-ac8122a3adea.png\n\n2. Disable `--no-summary` in *Settings*\n\n.. image:: https://user-images.githubusercontent.com/22340156/145638538-71590ec8-86c6-4b93-9a99-460b4e38e153.png\n :target: https://user-images.githubusercontent.com/22340156/145638538-71590ec8-86c6-4b93-9a99-460b4e38e153.png\n\n\nTroubleshooting\n+++++++++++++++\n\nThis section holds information about common issues.\n\n`The Test XXX is in a non-executable status`\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n* Problem: The test is not executable by the user.\n\n* Solution: Make sure, that your test is not deactivated, approved and ready to use in Jira.\n\n`Error message from server: fixVersions: fixVersions`\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n* Problem: The version is malformed or doesn't exist.\n\n* Solution: Make sure the version exists and the name matches the existing version and that only one version is used.\n\n\nReferences\n----------\n\n- XRay import execution endpoint: `<https://docs.getxray.app/display/XRAY/Import+Execution+Results>`_\n",
"bugtrack_url": null,
"license": null,
"summary": "pytest plugin to integrate tests with JIRA XRAY",
"version": "0.9.1",
"project_urls": {
"homepage": "https://github.com/fundakol/pytest-jira-xray"
},
"split_keywords": [
"pytest",
" jira",
" xray"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "4de320da6674127e76af299aa4b0974e36ab1026ea5a3068977ea01dc7d4c4a6",
"md5": "7117c69c293b696db237bcf1e9e85e77",
"sha256": "e487b3dfa228571655741f00f671079f86e1e1a5689465a381e8b0b8892a8c4e"
},
"downloads": -1,
"filename": "pytest_jira_xray-0.9.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7117c69c293b696db237bcf1e9e85e77",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 20203,
"upload_time": "2024-10-27T11:26:31",
"upload_time_iso_8601": "2024-10-27T11:26:31.239439Z",
"url": "https://files.pythonhosted.org/packages/4d/e3/20da6674127e76af299aa4b0974e36ab1026ea5a3068977ea01dc7d4c4a6/pytest_jira_xray-0.9.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-27 11:26:31",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "fundakol",
"github_project": "pytest-jira-xray",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"tox": true,
"lcname": "pytest-jira-xray"
}
