| Name | testrail-api-reporter JSON |
| Version |
1.4.10
JSON |
| download |
| home_page | |
| Summary | TestRail API reporter tools |
| upload_time | 2024-01-30 14:33:48 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.7 |
| license | MIT License Copyright (c) 2023 Ilya Vereshchagin Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
api
automation
automation-testing
chart
confluence
email
google-drive
pytest
pytest-report
qa
reporter
reports
slack
testing
testing-tools
testrail
testrail-api
testrail-python
testrail-reporting
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# This is Testrail API reporter tools.
This package contains several tools to interact with [TestRail](https://www.testrail.com/) via [API](https://support.testrail.com/hc/en-us/categories/7076541806228-API-Manual).
General part is *TestRailResultsReporter*, which is designed to report test results via api. This part is close to
[trcli](https://github.com/gurock/trcli), but without nasty bugs.
## Quick start
Firstly, you need to obtain test results in xml format. You can do it via running your testsuite, i.e. using pytest:
```bash
pytest --junitxml "junit-report.xml" "./tests"
```
Also, you need to add custom field (string type) to TestRails with name `automation_id`.
Now, you ready to upload results to TestRails.
To it, use:
```python
url='https://your_tr.testrail.io'
email='your@email.com'
password='your_password'
project_number=42
test_suite_number=66
api=TestRailResultsReporter(url=url, email=email, password=password, project_id=project_id,
suite_id=test_suite_id, xml_report='junit-report.xml')
# then just call:
api.send_results()
```
After this new testcases, test run and test results will be created. Testrun will have a name like
```AT run 2022-09-01T20:25:51```
If you fill `automation_id` for existing testcases using correct format
`path.to.testfile.filename.test_class.test_step`, then in such case results will be added to existing testcases.
Also, you can customize test run by passing:
- `title` param to send_results function - it will replace whole test run title.
- `environment` - it will be added to end of string like `AT run 2022-09-01T20:25:51 on Dev`
- `timestamp` - it will replace timestamp, obtained from XML file)
- `run_id` - if specified, results will be added to test run, no new testrun will be created
- `close_run` - may be `True` (by default) or `False` - if `True`, then every testrun will be closed
- `run_name` - you may use run name (title) instead of it's ID, in this case run id will be ignored even if it's filled
- `delete_old_run` - may be `True` or `False` (by default) - if `True`, then specified testrun will be deleted and new one will be created
Also, you can set up other params separately without need to re-initialize api object:
```set_project_id(project_id)``` - change project id
```def set_suite_id(suite_id)``` - change suite id
```def set_xml_filename(xml_filename)``` - change path/filename of xml report
```set_at_report_section(section_name)``` - change default folder name where non-linked testcases will be created
```set_timestamp(new_timestamp)``` - change default timestamp to custom
---
# TestRail coverage / distribution reporter
Second part of reporter it's visualization and reporting for test cases of TestRails.
Currently, TestRails machine does not contain functionality to track history data of some test cases attributes to make
reports of dynamics of automation coverage of existing testcases, or distribution by custom areas.
Also, it would be great to pass results as charts and report it to, for example, to Confluence.
So, I have a solution, you can use `ConfluenceReporter`!
```python
# Create reporter
confluence_reporter = ConfluenceReporter(username='Liberator', password='NoWar', url="https://my.confluence.com", confluence_page="1234")
# Now, generate several report at once!
confluence_reporter.generate_report(reports=automation_distribution, cases=area_distribution, values=priority_distribution, type_platforms=my_platforms,
automation_platforms=my_automation_platforms)
# for detailed info refer to code itself, you can use also separate calls:
confluence_reporter.history_type_chart(type_platforms=my_platforms) # history report of coverage by sections
confluence_reporter.history_state_chart(automation_platforms=my_automation_platforms) # history report of coverage by some attribute
confluence_reporter.contest_case_area_distribution(cases=area_distribution) # bar chart of area distribution
confluence_reporter.test_case_priority_distribution(values=priority_distribution) # bar chart of priority distribution
confluence_reporter.automation_state(reports=automation_distribution) # stacked bar chart using specific field as input
```
Ok, most likely, you wonder when you can obtain these distributions? You can do it by using `ATCoverageReporter`!
```python
testrails_adapter = ATCoverageReporter(url=tr_url, email=tr_client_email, password=tr_client_password,
project=tr_default_project, priority=4, type_platforms=my_platforms,
automation_platforms=automation_platforms)
# now get the values for charts!
values = tr_reporter.test_case_by_priority()
cases = tr_reporter.test_case_by_type()
reports = tr_reporter.automation_state_report()
```
So, I guess you still confused, what is the "platforms"? It's the settings, where and which data needs to be collected
```python
# You need specify where (at which top section) test cases for specific platform (or test type, whatever) is stored
# Also you need to specify by which field is used as criteria for automation, default 'internal_name' is 'type_id' and
# it is by default "Automated", "Functional", "Other", etc.
automation_platforms = (
{'name': 'Desktop Chrome', 'internal_name': 'type_id', 'sections': [4242]},
{'name': 'Desktop Firefox', 'internal_name': 'custom_firefox', 'sections': [2424]})
# Also, when you don't need to obtain automation state for cases, you can simply use just passing of section:
type_platforms = (
{'name': 'UI', 'sections': [6969]},
{'name': 'API', 'sections': [9696]})
```
I hope now it's clear. But what if you do not use Confluence? Ok, well, you can draw charts directly:
```python
plotly_reporter = PlotlyReporter(type_platforms=type_platforms)
plotly_reporter.draw_test_case_by_priority(filename='stacked_bar_chart.png', values=values)
plotly_reporter.draw_test_case_by_area(filename='pie_chart1.png', cases=cases)
plotly_reporter.draw_automation_state_report(filename="pie_chart2.png", reports=reports)
plotly_reporter.draw_history_type_chart(filename="line_stacked_chart.png")
for item in automation_platforms:
plotly_reporter.draw_history_state_chart(chart_name=item['name'])
```
# More ways to share data
If you still want to share reports, you can do it via email using `EmailSender`:
```python
chart_drawings = ['report_chart.png', 'path/to/more_graphics.png']
chart_captions = ['Priority distribution', 'AT coverage']
emailer = EmailSender(email="my_personal@email.com",
password="my_secure_password",
server_smtp="smtp.email_server.com",
server_port=587)
emailer.send_message(files=chart_drawings, captions=chart_captions, recipients=['buddy@email.com', 'boss@email.com'])
```
Alternatively, you can use GMail API with OAuth token instead of less secure auth:
```python
emailer = EmailSender(email="my_personal@gmail.com",
gmail_token="token.json")
```
For setup GMail OAuth credentials see the [Google API Reference](https://developers.google.com/identity/gsi/web/guides/get-google-api-clientid).
Or you can send as Slack message using `SlackSender`
```python
slack_sender = SlackSender(hook_url='https://hooks.slack.com/services/{your}/{api}/{key}')
slack_sender.send_message(files=chart_drawings, captions=chart_captions)
```
# Backup Test Cases
Sometimes you need backup you progress. Minimum what you can do - it's backup test cases. So, you can do it using `TCBackup`
```python
tc_backup = TCBackup(tr_url, tr_email, tr_password, test_rails_suite=3)
tc_backup.backup() # this will produce backup.xml file
tc_backup.get_archive_backup(suffix='') # this will produce backup.zip file
```
You still need to save it? Let's use Google Drive and `GoogleDriveUploader`
```python
# Google token needs to be configured firstly, to do it, you have to visit:
# https://console.developers.google.com/apis/credentials?pli=1
# Create Credentials => OAuth client ID => TV and limited Input Devices and get client_id and a client_secret
# Then pass it as google_id = client_id and google_secret = client_secret
gdrive = GoogleDriveUploader(google_id=client_id, google_secret=client_secret)
# first run prompts you to enter user_code to your user account and activate API token,
# but if you already have access tokens, do following:
gdrive = GoogleDriveUploader(google_id=client_id, google_secret=client_secret, google_api_refresh_token=refresh_token)
# now you may upload any file, but by default I assume you will use backup.zip
gdrive.upload(filename='backup.zip', mime_type='application/zip')
```
## Troubleshooting
To make plotly works, you need to set up Orca independently:
```bash
npm install -g electron orca
```
Please note, that Slack expecting urls instead of filenames, so, you must upload images to some hosting.
As option, you can do it using https://freeimage.host via function:
```python
image_uploaded = upload_image(filename=chart_drawings[0], api_token=YOUR_SECRET_TOKEN)
# now you can extract URL
image_url = image_uploaded['image']
# or its thumbnail
image_thumb = image_uploaded['thumb']
```
## Donations
If you like this project, you can support it by donating via [DonationAlerts](https://www.donationalerts.com/r/rocketsciencegeek).
Raw data
{
"_id": null,
"home_page": "",
"name": "testrail-api-reporter",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "api,automation,automation-testing,chart,confluence,email,google-drive,pytest,pytest-report,qa,reporter,reports,slack,testing,testing-tools,testrail,testrail-api,testrail-python,testrail-reporting",
"author": "",
"author_email": "Iliya Vereshchagin <i.vereshchagin@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/99/60/b383430b6651ced49e6bea6a483936b17058b51c93a095318bdd90ddd642/testrail_api_reporter-1.4.10.tar.gz",
"platform": null,
"description": "# This is Testrail API reporter tools.\n\nThis package contains several tools to interact with [TestRail](https://www.testrail.com/) via [API](https://support.testrail.com/hc/en-us/categories/7076541806228-API-Manual).\n\nGeneral part is *TestRailResultsReporter*, which is designed to report test results via api. This part is close to \n[trcli](https://github.com/gurock/trcli), but without nasty bugs.\n\n## Quick start\n\nFirstly, you need to obtain test results in xml format. You can do it via running your testsuite, i.e. using pytest:\n\n```bash\npytest --junitxml \"junit-report.xml\" \"./tests\"\n```\n\nAlso, you need to add custom field (string type) to TestRails with name `automation_id`.\n\nNow, you ready to upload results to TestRails.\nTo it, use:\n\n```python\nurl='https://your_tr.testrail.io'\nemail='your@email.com'\npassword='your_password'\nproject_number=42\ntest_suite_number=66\napi=TestRailResultsReporter(url=url, email=email, password=password, project_id=project_id,\n suite_id=test_suite_id, xml_report='junit-report.xml')\n# then just call:\napi.send_results()\n```\nAfter this new testcases, test run and test results will be created. Testrun will have a name like \n```AT run 2022-09-01T20:25:51```\n\n\n\n\nIf you fill `automation_id` for existing testcases using correct format \n`path.to.testfile.filename.test_class.test_step`, then in such case results will be added to existing testcases.\n\nAlso, you can customize test run by passing:\n- `title` param to send_results function - it will replace whole test run title.\n- `environment` - it will be added to end of string like `AT run 2022-09-01T20:25:51 on Dev`\n- `timestamp` - it will replace timestamp, obtained from XML file)\n- `run_id` - if specified, results will be added to test run, no new testrun will be created\n- `close_run` - may be `True` (by default) or `False` - if `True`, then every testrun will be closed\n- `run_name` - you may use run name (title) instead of it's ID, in this case run id will be ignored even if it's filled\n- `delete_old_run` - may be `True` or `False` (by default) - if `True`, then specified testrun will be deleted and new one will be created\n\n\n\nAlso, you can set up other params separately without need to re-initialize api object:\n\n```set_project_id(project_id)``` - change project id\n```def set_suite_id(suite_id)``` - change suite id\n```def set_xml_filename(xml_filename)``` - change path/filename of xml report\n```set_at_report_section(section_name)``` - change default folder name where non-linked testcases will be created\n```set_timestamp(new_timestamp)``` - change default timestamp to custom\n\n---\n\n# TestRail coverage / distribution reporter\n\nSecond part of reporter it's visualization and reporting for test cases of TestRails.\nCurrently, TestRails machine does not contain functionality to track history data of some test cases attributes to make\nreports of dynamics of automation coverage of existing testcases, or distribution by custom areas. \nAlso, it would be great to pass results as charts and report it to, for example, to Confluence.\n\nSo, I have a solution, you can use `ConfluenceReporter`!\n\n```python\n# Create reporter\nconfluence_reporter = ConfluenceReporter(username='Liberator', password='NoWar', url=\"https://my.confluence.com\", confluence_page=\"1234\")\n# Now, generate several report at once!\nconfluence_reporter.generate_report(reports=automation_distribution, cases=area_distribution, values=priority_distribution, type_platforms=my_platforms,\n automation_platforms=my_automation_platforms)\n# for detailed info refer to code itself, you can use also separate calls:\nconfluence_reporter.history_type_chart(type_platforms=my_platforms) # history report of coverage by sections\nconfluence_reporter.history_state_chart(automation_platforms=my_automation_platforms) # history report of coverage by some attribute\nconfluence_reporter.contest_case_area_distribution(cases=area_distribution) # bar chart of area distribution\nconfluence_reporter.test_case_priority_distribution(values=priority_distribution) # bar chart of priority distribution\nconfluence_reporter.automation_state(reports=automation_distribution) # stacked bar chart using specific field as input\n```\n\n\n\nOk, most likely, you wonder when you can obtain these distributions? You can do it by using `ATCoverageReporter`!\n\n```python\ntestrails_adapter = ATCoverageReporter(url=tr_url, email=tr_client_email, password=tr_client_password,\n project=tr_default_project, priority=4, type_platforms=my_platforms,\n automation_platforms=automation_platforms)\n# now get the values for charts!\nvalues = tr_reporter.test_case_by_priority()\ncases = tr_reporter.test_case_by_type()\nreports = tr_reporter.automation_state_report() \n```\n\nSo, I guess you still confused, what is the \"platforms\"? It's the settings, where and which data needs to be collected\n\n```python\n# You need specify where (at which top section) test cases for specific platform (or test type, whatever) is stored\n# Also you need to specify by which field is used as criteria for automation, default 'internal_name' is 'type_id' and\n# it is by default \"Automated\", \"Functional\", \"Other\", etc.\nautomation_platforms = (\n {'name': 'Desktop Chrome', 'internal_name': 'type_id', 'sections': [4242]},\n {'name': 'Desktop Firefox', 'internal_name': 'custom_firefox', 'sections': [2424]})\n# Also, when you don't need to obtain automation state for cases, you can simply use just passing of section:\ntype_platforms = (\n {'name': 'UI', 'sections': [6969]},\n {'name': 'API', 'sections': [9696]})\n```\n\nI hope now it's clear. But what if you do not use Confluence? Ok, well, you can draw charts directly:\n\n```python\nplotly_reporter = PlotlyReporter(type_platforms=type_platforms)\nplotly_reporter.draw_test_case_by_priority(filename='stacked_bar_chart.png', values=values)\nplotly_reporter.draw_test_case_by_area(filename='pie_chart1.png', cases=cases)\nplotly_reporter.draw_automation_state_report(filename=\"pie_chart2.png\", reports=reports)\nplotly_reporter.draw_history_type_chart(filename=\"line_stacked_chart.png\")\nfor item in automation_platforms:\n plotly_reporter.draw_history_state_chart(chart_name=item['name'])\n```\n\n# More ways to share data\n\nIf you still want to share reports, you can do it via email using `EmailSender`:\n\n```python\nchart_drawings = ['report_chart.png', 'path/to/more_graphics.png']\nchart_captions = ['Priority distribution', 'AT coverage']\nemailer = EmailSender(email=\"my_personal@email.com\",\n password=\"my_secure_password\",\n server_smtp=\"smtp.email_server.com\",\n server_port=587)\nemailer.send_message(files=chart_drawings, captions=chart_captions, recipients=['buddy@email.com', 'boss@email.com'])\n```\n\nAlternatively, you can use GMail API with OAuth token instead of less secure auth:\n```python\nemailer = EmailSender(email=\"my_personal@gmail.com\",\n gmail_token=\"token.json\")\n```\n\nFor setup GMail OAuth credentials see the [Google API Reference](https://developers.google.com/identity/gsi/web/guides/get-google-api-clientid).\n\n\n\n\nOr you can send as Slack message using `SlackSender`\n\n```python\nslack_sender = SlackSender(hook_url='https://hooks.slack.com/services/{your}/{api}/{key}')\nslack_sender.send_message(files=chart_drawings, captions=chart_captions)\n```\n\n\n\n\n# Backup Test Cases\n\nSometimes you need backup you progress. Minimum what you can do - it's backup test cases. So, you can do it using `TCBackup`\n\n```python\ntc_backup = TCBackup(tr_url, tr_email, tr_password, test_rails_suite=3)\ntc_backup.backup() # this will produce backup.xml file\ntc_backup.get_archive_backup(suffix='') # this will produce backup.zip file\n```\n\nYou still need to save it? Let's use Google Drive and `GoogleDriveUploader`\n\n```python\n# Google token needs to be configured firstly, to do it, you have to visit:\n# https://console.developers.google.com/apis/credentials?pli=1\n# Create Credentials => OAuth client ID => TV and limited Input Devices and get client_id and a client_secret\n# Then pass it as google_id = client_id and google_secret = client_secret\ngdrive = GoogleDriveUploader(google_id=client_id, google_secret=client_secret)\n# first run prompts you to enter user_code to your user account and activate API token, \n# but if you already have access tokens, do following:\ngdrive = GoogleDriveUploader(google_id=client_id, google_secret=client_secret, google_api_refresh_token=refresh_token)\n# now you may upload any file, but by default I assume you will use backup.zip\ngdrive.upload(filename='backup.zip', mime_type='application/zip')\n```\n\n\n\n\n\n\n## Troubleshooting\n\nTo make plotly works, you need to set up Orca independently:\n```bash\nnpm install -g electron orca\n```\n\nPlease note, that Slack expecting urls instead of filenames, so, you must upload images to some hosting.\nAs option, you can do it using https://freeimage.host via function:\n\n```python\nimage_uploaded = upload_image(filename=chart_drawings[0], api_token=YOUR_SECRET_TOKEN)\n# now you can extract URL\nimage_url = image_uploaded['image']\n# or its thumbnail\nimage_thumb = image_uploaded['thumb']\n```\n\n## Donations\n\nIf you like this project, you can support it by donating via [DonationAlerts](https://www.donationalerts.com/r/rocketsciencegeek).",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2023 Ilya Vereshchagin Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.",
"summary": "TestRail API reporter tools",
"version": "1.4.10",
"project_urls": {
"Bug Tracker": "https://github.com/wwakabobik/testrail_api_reporter/issues",
"Contribution Guide": "https://github.com/wwakabobik/testrail_api_reporter/blob/main/CONTRIBUTING.md",
"Donations": "https://www.donationalerts.com/r/rocketsciencegeek",
"Homepage": "https://github.com/wwakabobik/testrail_api_reporter"
},
"split_keywords": [
"api",
"automation",
"automation-testing",
"chart",
"confluence",
"email",
"google-drive",
"pytest",
"pytest-report",
"qa",
"reporter",
"reports",
"slack",
"testing",
"testing-tools",
"testrail",
"testrail-api",
"testrail-python",
"testrail-reporting"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "88ef9155a21ab29dd25eb3f41475313bb3ccdafed4d3fbdf247e7d4f72751d93",
"md5": "2cfcd6f7b0fecd6c9c7976e6c313b486",
"sha256": "1ffa57d2c2e73c3a1324873e6a7e71491ea3a8df84debca022d29eca4c96f2b3"
},
"downloads": -1,
"filename": "testrail_api_reporter-1.4.10-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2cfcd6f7b0fecd6c9c7976e6c313b486",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 1019434,
"upload_time": "2024-01-30T14:33:42",
"upload_time_iso_8601": "2024-01-30T14:33:42.811036Z",
"url": "https://files.pythonhosted.org/packages/88/ef/9155a21ab29dd25eb3f41475313bb3ccdafed4d3fbdf247e7d4f72751d93/testrail_api_reporter-1.4.10-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9960b383430b6651ced49e6bea6a483936b17058b51c93a095318bdd90ddd642",
"md5": "df21f3fe093598832319163209e7c69a",
"sha256": "01149e56aa8de96a1d5c2583d38add1535c23ab26267cacc49c7d579a06b8b81"
},
"downloads": -1,
"filename": "testrail_api_reporter-1.4.10.tar.gz",
"has_sig": false,
"md5_digest": "df21f3fe093598832319163209e7c69a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 1011622,
"upload_time": "2024-01-30T14:33:48",
"upload_time_iso_8601": "2024-01-30T14:33:48.498426Z",
"url": "https://files.pythonhosted.org/packages/99/60/b383430b6651ced49e6bea6a483936b17058b51c93a095318bdd90ddd642/testrail_api_reporter-1.4.10.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-30 14:33:48",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "wwakabobik",
"github_project": "testrail_api_reporter",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "testrail-api-reporter"
}
