## TestiPy - Python Test Tool
[](https://pypi.python.org/pypi/TestiPy)
[](https://pypi.org/project/TestiPy)
This documentation provides an overview of TestiPy, a Python tool designed for test selection, execution, and reporting. TestiPy offers features like test management, parallel execution, and integration with various reporting tools. It aims to streamline the testing process, making it easier to organize, execute, and analyze test results.
# 1. Installation
- ## From GitHub
- Clone this repo:
```bash
git clone https://github.com/pjn2work/TestiPy.git
```
- Change any setting you'd like inside this file [default_config.py](https://github.com/pjn2work/TestiPy/blob/main/testipy/configs/default_config.py) \(this step is optional\)
- Install:
```bash
# Windows
install.bat
```
```bash
# Linux or Mac
./install.sh
```
- ## From PyPi
```bash
pip install testipy
```
# 2. Running TestiPy
- ## if you didn't install it, just cloned
```bash
# goto your folder where you've cloned this repo
cd /where_you_have_cloned_this_repo
# run demo tests indicating where the tests are and using the web reporter
python testipy/run.py -tf testipy/tests -r web
```
- ## if you install
```bash
# run demo tests indicating where the tests are and using the web reporter
testipy -tf /where_you_have_cloned_this_repo/testipy/tests -r web
```
# 2.1. Test selection options:
Select Tests (by tags coded in suites and tests with @TAG, @NAME, @LEVEL, @FEATURES)
#### options (excludes have higher importance over includes):
-ip Include Package (ex: -is qa.regression -es qa.regression.dev), can be several
-ep Exclude Package
-is Include Suite (ex: -is suiteCertificates -is MICROSERVICES -es DEV), can be several
-es Exclude Suite
-it Include Test (ex: -it REST), can be several
-et Exclude Test
-sb filename of the StoryBoard to run (ex: -sb storyboard_QA_rest.json -sb "/qa/tests/sb/sb01.json"), can be several
-ilv Include tests of level y (ex: -ilv 5 -ilv 8)
-elv Exclude tests of level y
-alv Include tests above level y (ex: -alv 5)
-blv Include tests below level y
-if Include tests by @FEATURES tag (ex: -if 850222)
-ef Exclude tests by @FEATURES tag
-itn Include tests by @TN tag (beginsWith) (ex: -itn 1.3.1.10)
-etn Exclude tests by @TN tag (beginsWith)
# 2.2. Select Reporters
- ### options:
* **-reporter** or **-r** add Reporter (ex: -reporter echo -reporter log -reporter web)
* **echo:** shows test execution on stdout, and errors in stderr
* **excel:** creates an Excel file with test execution summary and test execution details
* **log:** shows test execution on .log file, with the same name as the project_name, errors are shown in stderr
* **portalio:** ReportPortalIO Web REST DB and reporter:
* **slack:** tests results are sent to Slack channel
* **web:** tests results can be seen in realtime on a browser. use **-r-web-port PORT** to specify other than 9204
* **xml:** test results will be saved on report.xml file
# 2.3. Run:
- ### options:
* `-rid`: Run ID (e.g., `-rid 17`). If not provided, the current hour and minute will be used (e.g., `2359`).
* `-pn`: Project Name (e.g., `-pn my_project`).
* `-env`: Environment Name for testing (e.g., `-env qa`).
* `-rf`: Results Folder (e.g., `-rf /path/to/results/`), where test results will be stored.
* `-tf`: Tests Folder (e.g., `-tf /path/to/tests/`), the directory containing test scripts.
* `-r` or `-reporter`: Add a reporter (e.g., `-r echo -r log -r web`).
* Available reporters include: `echo`, `excel`, `log`, `portalio`, `slack`, `web`, `xml`.
* `-repeat`: Number of times to repeat the test execution (e.g., `-repeat 3`).
* `-st`: Suite Threads (1 to 8), controls the number of suites that can run in parallel (e.g., `-st 4`).
- ### flags
* `--dryrun`: Runs tests without actual execution (all tests are marked as 'SKIPPED').
* `--debugcode`: Disables try/except blocks in tests, showing detailed error messages.
* `--debug-testipy`: Shows stack traces for TestiPy classes (useful for debugging the tool itself).
* `--1`: Overrides the default number of test cycles (ncycles) defined in test suites, forcing all tests to run only once.
* `--prof`: Generates a `.prof` file with profiling data.
# 3. Example of usage:
- #### Example of usage:
```
python3 run.py -env dev -reporter log -reporter web -rid 1 -tf "/home/testipy/my_test_scripts" -et NO_RUN -it DEV
```
- #### Storyboard:
- If storyboard passed, tests will run by the order defined on json file
- If no storyboard is passed, then tests will run ordered (DESC) by package name, @PRIO defined on suite, then by @PRIO defined on test itself
- #### Results Folder:
- A folder will be created under the (specified -rf option) composed by: projectName_currentDate_RID (ex: testipy_20201231_00525)
- Under the folder defined above, subfolders can be created with package_name/suite_name containing the tests results (created by each reporter)
- #### Tests not ended:
- If a test ends without being formally ended (by a testFailed, testSkipped or testPassed), it will be passed by the executor
In addition to the reporter-specific outputs, TestiPy creates a `results.yaml` file in the results folder, containing a summary of the overall test execution results.
This tool provides a flexible and customizable framework for Python testing, offering features for test organization, execution control, and comprehensive reporting through various channels.
For integrations with Behave framework, [check the example here](https://github.com/pjn2work/testipy-tests)
# 4. Suite Example
``` python
from typing import Dict
from testipy.helpers.handle_assertions import ExpectedError
from testipy.reporter import ReportManager
from pet_store_toolbox import Toolbox
_new_pet = {
"id": 1,
"name": "Sissi",
"category": {
"id": 1,
"name": "Dogs"
},
"photoUrls": [""],
"tags": [
{
"id": 0,
"name": "Schnauzer"
},
{
"id": 0,
"name": "mini"
}
],
"status": "available"
}
class SuitePetStore:
"""
@LEVEL 1
@TAG PETSTORE
@PRIO 2
"""
def __init__(self):
self.toolbox = Toolbox()
# Create a new pet
def test_create_pet_valid(self, sd: SuiteDetails, rm: ReportManager, ncycles=1, param=None):
"""
@LEVEL 3
@PRIO 5
"""
current_test = rm.startTest(sd)
data = {
"control": {"expected_status_code": 200},
"param": _new_pet,
"expected_response": _new_pet
}
try:
self.toolbox.post_pet(rm, current_test, data, "create_pet")
except Exception as ex:
rm.testFailed(current_test, reason_of_state=str(ex), exc_value=ex)
else:
rm.testPassed(current_test, reason_of_state="pet created")
# Get the pet created before
def test_get_pet_valid(self, sd: SuiteDetails, rm: ReportManager, ncycles=1, param=None):
"""
@LEVEL 3
@PRIO 10
@ON_SUCCESS 5
"""
current_test = rm.startTest(sd)
data = {
"control": {"expected_status_code": 200},
"param": _new_pet["id"],
"expected_response": _new_pet
}
try:
self.toolbox.get_pet(rm, current_test, data, "get_pet")
except Exception as ex:
rm.testFailed(current_test, reason_of_state=str(ex), exc_value=ex)
else:
rm.testPassed(current_test, reason_of_state="pet fetched")
```
Raw data
{
"_id": null,
"home_page": "https://github.com/pjn2work/testipy",
"name": "TestiPy",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "testing, reporting, behave",
"author": "Pedro Nunes",
"author_email": "pjn2work@google.com",
"download_url": "https://files.pythonhosted.org/packages/ef/85/7935a16f78e5fb2578ed15fa7288861b111ed4ee3bf350b3d917dea62703/testipy-0.10.5.tar.gz",
"platform": null,
"description": "## TestiPy - Python Test Tool\n\n[](https://pypi.python.org/pypi/TestiPy)\n[](https://pypi.org/project/TestiPy)\n\nThis documentation provides an overview of TestiPy, a Python tool designed for test selection, execution, and reporting. TestiPy offers features like test management, parallel execution, and integration with various reporting tools. It aims to streamline the testing process, making it easier to organize, execute, and analyze test results.\n\n\n# 1. Installation\n - ## From GitHub\n - Clone this repo:\n ```bash\n git clone https://github.com/pjn2work/TestiPy.git\n ```\n - Change any setting you'd like inside this file [default_config.py](https://github.com/pjn2work/TestiPy/blob/main/testipy/configs/default_config.py) \\(this step is optional\\)\n - Install:\n ```bash\n # Windows\n install.bat\n ```\n ```bash\n # Linux or Mac\n ./install.sh\n ```\n - ## From PyPi\n ```bash\n pip install testipy\n ```\n\n\n# 2. Running TestiPy\n - ## if you didn't install it, just cloned\n ```bash\n # goto your folder where you've cloned this repo\n cd /where_you_have_cloned_this_repo\n\n # run demo tests indicating where the tests are and using the web reporter\n python testipy/run.py -tf testipy/tests -r web\n ```\n - ## if you install\n ```bash\n # run demo tests indicating where the tests are and using the web reporter\n testipy -tf /where_you_have_cloned_this_repo/testipy/tests -r web\n ```\n\n\n# 2.1. Test selection options:\n Select Tests (by tags coded in suites and tests with @TAG, @NAME, @LEVEL, @FEATURES)\n #### options (excludes have higher importance over includes):\n -ip Include Package (ex: -is qa.regression -es qa.regression.dev), can be several\n -ep Exclude Package\n -is Include Suite (ex: -is suiteCertificates -is MICROSERVICES -es DEV), can be several\n -es Exclude Suite\n -it Include Test (ex: -it REST), can be several\n -et Exclude Test\n -sb filename of the StoryBoard to run (ex: -sb storyboard_QA_rest.json -sb \"/qa/tests/sb/sb01.json\"), can be several\n -ilv Include tests of level y (ex: -ilv 5 -ilv 8)\n -elv Exclude tests of level y\n -alv Include tests above level y (ex: -alv 5)\n -blv Include tests below level y\n -if Include tests by @FEATURES tag (ex: -if 850222)\n -ef Exclude tests by @FEATURES tag\n -itn Include tests by @TN tag (beginsWith) (ex: -itn 1.3.1.10)\n -etn Exclude tests by @TN tag (beginsWith)\n\n\n# 2.2. Select Reporters\n - ### options:\n * **-reporter** or **-r** add Reporter (ex: -reporter echo -reporter log -reporter web)\n\n * **echo:** shows test execution on stdout, and errors in stderr\n * **excel:** creates an Excel file with test execution summary and test execution details\n * **log:** shows test execution on .log file, with the same name as the project_name, errors are shown in stderr\n * **portalio:** ReportPortalIO Web REST DB and reporter:\n * **slack:** tests results are sent to Slack channel\n * **web:** tests results can be seen in realtime on a browser. use **-r-web-port PORT** to specify other than 9204\n * **xml:** test results will be saved on report.xml file\n\n\n# 2.3. Run:\n - ### options:\n * `-rid`: Run ID (e.g., `-rid 17`). If not provided, the current hour and minute will be used (e.g., `2359`).\n * `-pn`: Project Name (e.g., `-pn my_project`).\n * `-env`: Environment Name for testing (e.g., `-env qa`).\n * `-rf`: Results Folder (e.g., `-rf /path/to/results/`), where test results will be stored.\n * `-tf`: Tests Folder (e.g., `-tf /path/to/tests/`), the directory containing test scripts.\n * `-r` or `-reporter`: Add a reporter (e.g., `-r echo -r log -r web`).\n * Available reporters include: `echo`, `excel`, `log`, `portalio`, `slack`, `web`, `xml`.\n * `-repeat`: Number of times to repeat the test execution (e.g., `-repeat 3`).\n * `-st`: Suite Threads (1 to 8), controls the number of suites that can run in parallel (e.g., `-st 4`).\n\n - ### flags\n * `--dryrun`: Runs tests without actual execution (all tests are marked as 'SKIPPED').\n * `--debugcode`: Disables try/except blocks in tests, showing detailed error messages.\n * `--debug-testipy`: Shows stack traces for TestiPy classes (useful for debugging the tool itself).\n * `--1`: Overrides the default number of test cycles (ncycles) defined in test suites, forcing all tests to run only once.\n * `--prof`: Generates a `.prof` file with profiling data.\n\n\n# 3. Example of usage:\n - #### Example of usage:\n ```\n python3 run.py -env dev -reporter log -reporter web -rid 1 -tf \"/home/testipy/my_test_scripts\" -et NO_RUN -it DEV\n ```\n - #### Storyboard:\n - If storyboard passed, tests will run by the order defined on json file\n - If no storyboard is passed, then tests will run ordered (DESC) by package name, @PRIO defined on suite, then by @PRIO defined on test itself\n - #### Results Folder:\n - A folder will be created under the (specified -rf option) composed by: projectName_currentDate_RID (ex: testipy_20201231_00525)\n - Under the folder defined above, subfolders can be created with package_name/suite_name containing the tests results (created by each reporter)\n - #### Tests not ended:\n - If a test ends without being formally ended (by a testFailed, testSkipped or testPassed), it will be passed by the executor\n\n\nIn addition to the reporter-specific outputs, TestiPy creates a `results.yaml` file in the results folder, containing a summary of the overall test execution results.\n\nThis tool provides a flexible and customizable framework for Python testing, offering features for test organization, execution control, and comprehensive reporting through various channels.\n\n\nFor integrations with Behave framework, [check the example here](https://github.com/pjn2work/testipy-tests)\n\n\n# 4. Suite Example\n``` python\nfrom typing import Dict\n\nfrom testipy.helpers.handle_assertions import ExpectedError\nfrom testipy.reporter import ReportManager\n\nfrom pet_store_toolbox import Toolbox\n\n\n_new_pet = {\n \"id\": 1,\n \"name\": \"Sissi\",\n \"category\": {\n \"id\": 1,\n \"name\": \"Dogs\"\n },\n \"photoUrls\": [\"\"],\n \"tags\": [\n {\n \"id\": 0,\n \"name\": \"Schnauzer\"\n },\n {\n \"id\": 0,\n \"name\": \"mini\"\n }\n ],\n \"status\": \"available\"\n }\n\n\nclass SuitePetStore:\n \"\"\"\n @LEVEL 1\n @TAG PETSTORE\n @PRIO 2\n \"\"\"\n\n def __init__(self):\n self.toolbox = Toolbox()\n\n # Create a new pet\n def test_create_pet_valid(self, sd: SuiteDetails, rm: ReportManager, ncycles=1, param=None):\n \"\"\"\n @LEVEL 3\n @PRIO 5\n \"\"\"\n current_test = rm.startTest(sd)\n\n data = {\n \"control\": {\"expected_status_code\": 200},\n \"param\": _new_pet,\n \"expected_response\": _new_pet\n }\n\n try:\n self.toolbox.post_pet(rm, current_test, data, \"create_pet\")\n except Exception as ex:\n rm.testFailed(current_test, reason_of_state=str(ex), exc_value=ex)\n else:\n rm.testPassed(current_test, reason_of_state=\"pet created\")\n\n # Get the pet created before\n def test_get_pet_valid(self, sd: SuiteDetails, rm: ReportManager, ncycles=1, param=None):\n \"\"\"\n @LEVEL 3\n @PRIO 10\n @ON_SUCCESS 5\n \"\"\"\n current_test = rm.startTest(sd)\n\n data = {\n \"control\": {\"expected_status_code\": 200},\n \"param\": _new_pet[\"id\"],\n \"expected_response\": _new_pet\n }\n\n try:\n self.toolbox.get_pet(rm, current_test, data, \"get_pet\")\n except Exception as ex:\n rm.testFailed(current_test, reason_of_state=str(ex), exc_value=ex)\n else:\n rm.testPassed(current_test, reason_of_state=\"pet fetched\")\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "Python test selection/execution/reporting tool",
"version": "0.10.5",
"project_urls": {
"Homepage": "https://github.com/pjn2work/testipy"
},
"split_keywords": [
"testing",
" reporting",
" behave"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "974b9f747c6be597df0adc48f3b5fc9fc264d09bcb72534b212a10b10a7805d9",
"md5": "327223a5bb3a0c00ad868c4a1d37fcf3",
"sha256": "2de60aaee23a5b57e4eccb70cb508f40efaaeba495438709518b9f4c05d63d0b"
},
"downloads": -1,
"filename": "TestiPy-0.10.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "327223a5bb3a0c00ad868c4a1d37fcf3",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 171559,
"upload_time": "2024-11-09T11:38:25",
"upload_time_iso_8601": "2024-11-09T11:38:25.349373Z",
"url": "https://files.pythonhosted.org/packages/97/4b/9f747c6be597df0adc48f3b5fc9fc264d09bcb72534b212a10b10a7805d9/TestiPy-0.10.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ef857935a16f78e5fb2578ed15fa7288861b111ed4ee3bf350b3d917dea62703",
"md5": "d071676b01a68f0c44de51ead74bc9a8",
"sha256": "ef9796092b144695e64da3390d6436e29e6dcf7e38279bd225adf08f3cdc11df"
},
"downloads": -1,
"filename": "testipy-0.10.5.tar.gz",
"has_sig": false,
"md5_digest": "d071676b01a68f0c44de51ead74bc9a8",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 147975,
"upload_time": "2024-11-09T11:38:27",
"upload_time_iso_8601": "2024-11-09T11:38:27.311088Z",
"url": "https://files.pythonhosted.org/packages/ef/85/7935a16f78e5fb2578ed15fa7288861b111ed4ee3bf350b3d917dea62703/testipy-0.10.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-09 11:38:27",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "pjn2work",
"github_project": "testipy",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "testipy"
}
JSON
