qgate-perf


Nameqgate-perf JSON
Version 0.4.6 PyPI version JSON
download
home_page
SummaryPerformance test generator, part of Quality Gate
upload_time2023-12-22 11:46:48
maintainer
docs_urlNone
author
requires_python>=3.6
licenseMIT
keywords performancetest performance qualitygate
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage 
            [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![PyPI version fury.io](https://badge.fury.io/py/qgate-perf.svg)](https://pypi.python.org/pypi/qgate-perf/)
![coverage](https://github.com/george0st/qgate-perf/blob/main/coverage.svg?raw=true)
![GitHub commit activity](https://img.shields.io/github/commit-activity/w/george0st/qgate-perf)
![GitHub release](https://img.shields.io/github/v/release/george0st/qgate-perf)

# QGate-Perf

The QGate Performance is enabler for performance test execution. Key benefits:
 - **easy performance testing** your python code (key parts - init, start, stop, return)
 - **measure only specific part** of your code 
 - scalability **without limits** (e.g. from 1 to 1k executors)
 - scalability **in level of processes and threads** (easy way, how to avoid GIL in python)
 - **sequences for execution and data bulk**
 - relation to graph generator

## Usage

```python
from qgate_perf.parallel_executor import ParallelExecutor
from qgate_perf.parallel_probe import ParallelProbe
from qgate_perf.run_setup import RunSetup
import time

def prf_GIL_impact(run_setup: RunSetup):
    """ Your own function for performance testing, you have to add
    only part INIT, START, STOP and RETURN"""
    
    # INIT - contain executor synchonization, if needed
    probe=ParallelProbe(run_setup)

    while (True):
        # START - probe, only for this specific code part
        probe.start()

        for r in range(run_setup.bulk_row * run_setup.bulk_col):
            time.sleep(0)

        # STOP - probe
        if probe.stop():
            break

    # RETURN - data from probe
    return probe

# Execution setting
generator = ParallelExecutor(prf_GIL_impact,
                             label="GIL_impact",
                             detail_output=True,
                             output_file="prf_gil_impact_01.txt")

# Run setup, with test execution 20 seconds and zero delay before start 
# (without waiting to other executors)
setup=RunSetup(duration_second=20,start_delay=0)

# Run performance test with: 
#  data bulk_list with two data sets 
#    - first has 10 rows and 5 columns as [10, 5]
#    - second has 1000 rows and 50 columns as [1000, 50]
#  executor_list with six executor sets
#    - first line has three executors with 2, 4 and 8 processes each with 2 threads 
#    - second line has three executors with 2, 4 and 8 processes each with 4 threads
generator.run_bulk_executor(bulk_list=[[10, 5], [1000, 50]],
                            executor_list=[[2, 2, '2x thread'], [4, 2, '2x thread'],[8, 2,'2x thread'],
                                           [2, 4, '4x thread'], [4, 4, '4x thread'],[8, 4,'4x thread']],
                            run_setup=setup)

# Note: We made 12 performance tests (two bulk_list x six executor_list) and write 
# outputs to the file 'prf_gil_impact_01.txt'

# We generate performance graph based on performance tests to the 
# directory './output/graph-perf/*' (two files each for different bundle) 
generator.create_graph_perf()
```

## Outputs in text file 
```
############### 2023-05-05 06:30:36.194849 ###############
{"type": "headr", "label": "GIL_impact", "bulk": [1, 1], "available_cpu": 12, "now": "2023-05-05 06:30:36.194849"}
  {"type": "core", "plan_executors": 4, "plan_executors_detail": [4, 1], "real_executors": 4, "group": "1x thread", "total_calls": 7590439, "avrg_time": 1.4127372338382197e-06, "std_deviation": 3.699171006877347e-05, "total_call_per_sec": 2831382.8673804617, "endexec": "2023-05-05 06:30:44.544829"}
  {"type": "core", "plan_executors": 8, "plan_executors_detail": [8, 1], "real_executors": 8, "group": "1x thread", "total_calls": 11081697, "avrg_time": 1.789265660825848e-06, "std_deviation": 4.164309967620533e-05, "total_call_per_sec": 4471107.994274894, "endexec": "2023-05-05 06:30:52.623666"}
  {"type": "core", "plan_executors": 16, "plan_executors_detail": [16, 1], "real_executors": 16, "group": "1x thread", "total_calls": 8677305, "avrg_time": 6.2560950624827455e-06, "std_deviation": 8.629422798757681e-05, "total_call_per_sec": 2557505.8946835063, "endexec": "2023-05-05 06:31:02.875799"}
  {"type": "core", "plan_executors": 8, "plan_executors_detail": [4, 2], "real_executors": 8, "group": "2x threads", "total_calls": 2761851, "avrg_time": 1.1906723084757647e-05, "std_deviation": 0.00010741937495211329, "total_call_per_sec": 671889.3135459893, "endexec": "2023-05-05 06:31:10.283786"}
  {"type": "core", "plan_executors": 16, "plan_executors_detail": [8, 2], "real_executors": 16, "group": "2x threads", "total_calls": 3605920, "avrg_time": 1.858694254439209e-05, "std_deviation": 0.00013301637613377212, "total_call_per_sec": 860819.3607844017, "endexec": "2023-05-05 06:31:18.740831"}
  {"type": "core", "plan_executors": 16, "plan_executors_detail": [4, 4], "real_executors": 16, "group": "4x threads", "total_calls": 1647508, "avrg_time": 4.475957498576462e-05, "std_deviation": 0.00020608402170105327, "total_call_per_sec": 357465.41393855185, "endexec": "2023-05-05 06:31:26.008649"}
############### Duration: 49.9 seconds ###############
```

## Outputs in text file with detail
```
############### 2023-05-05 07:01:18.571700 ###############
{"type": "headr", "label": "GIL_impact", "bulk": [1, 1], "available_cpu": 12, "now": "2023-05-05 07:01:18.571700"}
     {"type": "detail", "processid": 12252, "calls": 1896412, "total": 2.6009109020233154, "avrg": 1.371490426143325e-06, "min": 0.0, "max": 0.0012514591217041016, "st-dev": 3.6488665183545995e-05, "initexec": "2023-05-05 07:01:21.370528", "startexec": "2023-05-05 07:01:21.370528", "endexec": "2023-05-05 07:01:26.371062"}
     {"type": "detail", "processid": 8944, "calls": 1855611, "total": 2.5979537963867188, "avrg": 1.4000530264084008e-06, "min": 0.0, "max": 0.001207590103149414, "st-dev": 3.6889275786419565e-05, "initexec": "2023-05-05 07:01:21.466496", "startexec": "2023-05-05 07:01:21.466496", "endexec": "2023-05-05 07:01:26.466551"}
     {"type": "detail", "processid": 2108, "calls": 1943549, "total": 2.6283881664276123, "avrg": 1.3523652691172758e-06, "min": 0.0, "max": 0.0012514591217041016, "st-dev": 3.624462003401045e-05, "initexec": "2023-05-05 07:01:21.709203", "startexec": "2023-05-05 07:01:21.709203", "endexec": "2023-05-05 07:01:26.709298"}
     {"type": "detail", "processid": 19292, "calls": 1973664, "total": 2.6392557621002197, "avrg": 1.3372366127670262e-06, "min": 0.0, "max": 0.0041027069091796875, "st-dev": 3.620965943471147e-05, "initexec": "2023-05-05 07:01:21.840541", "startexec": "2023-05-05 07:01:21.840541", "endexec": "2023-05-05 07:01:26.841266"}
  {"type": "core", "plan_executors": 4, "plan_executors_detail": [4, 1], "real_executors": 4, "group": "1x thread", "total_calls": 7669236, "avrg_time": 1.3652863336090071e-06, "std_deviation": 3.645805510967187e-05, "total_call_per_sec": 2929788.3539391863, "endexec": "2023-05-05 07:01:26.891144"}
  ...
```

## Graphs generated from qgate-graph based on outputs from qgate-perf
#### 512 executors (128 processes x 4 threads)
![graph](https://github.com/george0st/qgate-perf/blob/main/assets/PRF-Calc-2023-05-06_18-22-19-bulk-1x10.png?raw=true)
![graph](https://github.com/george0st/qgate-perf/blob/main/assets/EXE-Calc-2023-05-06_18-22-19-bulk-1x10-plan-128x4.png?raw=true)

#### 32 executors (8 processes x 4 threads)
![graph](https://github.com/george0st/qgate-perf/blob/main/assets/PRF-NoSQL_igz_nonprod-2023-04-23_14-41-18-bulk-100x50.png?raw=true)
![graph](https://github.com/george0st/qgate-perf/blob/main/assets/EXE-NoSQL-2023-05-04_19-33-30-bulk-1x50-plan-8x2.png?raw=true)

Raw data

            {
    "_id": null,
    "home_page": "",
    "name": "qgate-perf",
    "maintainer": "",
    "docs_url": null,
    "requires_python": ">=3.6",
    "maintainer_email": "Jiri Steuer <steuer.jiri@gmail.com>",
    "keywords": "PerformanceTest,Performance,QualityGate",
    "author": "",
    "author_email": "Jiri Steuer <steuer.jiri@gmail.com>",
    "download_url": "",
    "platform": null,
    "description": "[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\r\n[![PyPI version fury.io](https://badge.fury.io/py/qgate-perf.svg)](https://pypi.python.org/pypi/qgate-perf/)\r\n![coverage](https://github.com/george0st/qgate-perf/blob/main/coverage.svg?raw=true)\r\n![GitHub commit activity](https://img.shields.io/github/commit-activity/w/george0st/qgate-perf)\r\n![GitHub release](https://img.shields.io/github/v/release/george0st/qgate-perf)\r\n\r\n# QGate-Perf\r\n\r\nThe QGate Performance is enabler for performance test execution. Key benefits:\r\n - **easy performance testing** your python code (key parts - init, start, stop, return)\r\n - **measure only specific part** of your code \r\n - scalability **without limits** (e.g. from 1 to 1k executors)\r\n - scalability **in level of processes and threads** (easy way, how to avoid GIL in python)\r\n - **sequences for execution and data bulk**\r\n - relation to graph generator\r\n\r\n## Usage\r\n\r\n```python\r\nfrom qgate_perf.parallel_executor import ParallelExecutor\r\nfrom qgate_perf.parallel_probe import ParallelProbe\r\nfrom qgate_perf.run_setup import RunSetup\r\nimport time\r\n\r\ndef prf_GIL_impact(run_setup: RunSetup):\r\n    \"\"\" Your own function for performance testing, you have to add\r\n    only part INIT, START, STOP and RETURN\"\"\"\r\n    \r\n    # INIT - contain executor synchonization, if needed\r\n    probe=ParallelProbe(run_setup)\r\n\r\n    while (True):\r\n        # START - probe, only for this specific code part\r\n        probe.start()\r\n\r\n        for r in range(run_setup.bulk_row * run_setup.bulk_col):\r\n            time.sleep(0)\r\n\r\n        # STOP - probe\r\n        if probe.stop():\r\n            break\r\n\r\n    # RETURN - data from probe\r\n    return probe\r\n\r\n# Execution setting\r\ngenerator = ParallelExecutor(prf_GIL_impact,\r\n                             label=\"GIL_impact\",\r\n                             detail_output=True,\r\n                             output_file=\"prf_gil_impact_01.txt\")\r\n\r\n# Run setup, with test execution 20 seconds and zero delay before start \r\n# (without waiting to other executors)\r\nsetup=RunSetup(duration_second=20,start_delay=0)\r\n\r\n# Run performance test with: \r\n#  data bulk_list with two data sets \r\n#    - first has 10 rows and 5 columns as [10, 5]\r\n#    - second has 1000 rows and 50 columns as [1000, 50]\r\n#  executor_list with six executor sets\r\n#    - first line has three executors with 2, 4 and 8 processes each with 2 threads \r\n#    - second line has three executors with 2, 4 and 8 processes each with 4 threads\r\ngenerator.run_bulk_executor(bulk_list=[[10, 5], [1000, 50]],\r\n                            executor_list=[[2, 2, '2x thread'], [4, 2, '2x thread'],[8, 2,'2x thread'],\r\n                                           [2, 4, '4x thread'], [4, 4, '4x thread'],[8, 4,'4x thread']],\r\n                            run_setup=setup)\r\n\r\n# Note: We made 12 performance tests (two bulk_list x six executor_list) and write \r\n# outputs to the file 'prf_gil_impact_01.txt'\r\n\r\n# We generate performance graph based on performance tests to the \r\n# directory './output/graph-perf/*' (two files each for different bundle) \r\ngenerator.create_graph_perf()\r\n```\r\n\r\n## Outputs in text file \r\n```\r\n############### 2023-05-05 06:30:36.194849 ###############\r\n{\"type\": \"headr\", \"label\": \"GIL_impact\", \"bulk\": [1, 1], \"available_cpu\": 12, \"now\": \"2023-05-05 06:30:36.194849\"}\r\n  {\"type\": \"core\", \"plan_executors\": 4, \"plan_executors_detail\": [4, 1], \"real_executors\": 4, \"group\": \"1x thread\", \"total_calls\": 7590439, \"avrg_time\": 1.4127372338382197e-06, \"std_deviation\": 3.699171006877347e-05, \"total_call_per_sec\": 2831382.8673804617, \"endexec\": \"2023-05-05 06:30:44.544829\"}\r\n  {\"type\": \"core\", \"plan_executors\": 8, \"plan_executors_detail\": [8, 1], \"real_executors\": 8, \"group\": \"1x thread\", \"total_calls\": 11081697, \"avrg_time\": 1.789265660825848e-06, \"std_deviation\": 4.164309967620533e-05, \"total_call_per_sec\": 4471107.994274894, \"endexec\": \"2023-05-05 06:30:52.623666\"}\r\n  {\"type\": \"core\", \"plan_executors\": 16, \"plan_executors_detail\": [16, 1], \"real_executors\": 16, \"group\": \"1x thread\", \"total_calls\": 8677305, \"avrg_time\": 6.2560950624827455e-06, \"std_deviation\": 8.629422798757681e-05, \"total_call_per_sec\": 2557505.8946835063, \"endexec\": \"2023-05-05 06:31:02.875799\"}\r\n  {\"type\": \"core\", \"plan_executors\": 8, \"plan_executors_detail\": [4, 2], \"real_executors\": 8, \"group\": \"2x threads\", \"total_calls\": 2761851, \"avrg_time\": 1.1906723084757647e-05, \"std_deviation\": 0.00010741937495211329, \"total_call_per_sec\": 671889.3135459893, \"endexec\": \"2023-05-05 06:31:10.283786\"}\r\n  {\"type\": \"core\", \"plan_executors\": 16, \"plan_executors_detail\": [8, 2], \"real_executors\": 16, \"group\": \"2x threads\", \"total_calls\": 3605920, \"avrg_time\": 1.858694254439209e-05, \"std_deviation\": 0.00013301637613377212, \"total_call_per_sec\": 860819.3607844017, \"endexec\": \"2023-05-05 06:31:18.740831\"}\r\n  {\"type\": \"core\", \"plan_executors\": 16, \"plan_executors_detail\": [4, 4], \"real_executors\": 16, \"group\": \"4x threads\", \"total_calls\": 1647508, \"avrg_time\": 4.475957498576462e-05, \"std_deviation\": 0.00020608402170105327, \"total_call_per_sec\": 357465.41393855185, \"endexec\": \"2023-05-05 06:31:26.008649\"}\r\n############### Duration: 49.9 seconds ###############\r\n```\r\n\r\n## Outputs in text file with detail\r\n```\r\n############### 2023-05-05 07:01:18.571700 ###############\r\n{\"type\": \"headr\", \"label\": \"GIL_impact\", \"bulk\": [1, 1], \"available_cpu\": 12, \"now\": \"2023-05-05 07:01:18.571700\"}\r\n     {\"type\": \"detail\", \"processid\": 12252, \"calls\": 1896412, \"total\": 2.6009109020233154, \"avrg\": 1.371490426143325e-06, \"min\": 0.0, \"max\": 0.0012514591217041016, \"st-dev\": 3.6488665183545995e-05, \"initexec\": \"2023-05-05 07:01:21.370528\", \"startexec\": \"2023-05-05 07:01:21.370528\", \"endexec\": \"2023-05-05 07:01:26.371062\"}\r\n     {\"type\": \"detail\", \"processid\": 8944, \"calls\": 1855611, \"total\": 2.5979537963867188, \"avrg\": 1.4000530264084008e-06, \"min\": 0.0, \"max\": 0.001207590103149414, \"st-dev\": 3.6889275786419565e-05, \"initexec\": \"2023-05-05 07:01:21.466496\", \"startexec\": \"2023-05-05 07:01:21.466496\", \"endexec\": \"2023-05-05 07:01:26.466551\"}\r\n     {\"type\": \"detail\", \"processid\": 2108, \"calls\": 1943549, \"total\": 2.6283881664276123, \"avrg\": 1.3523652691172758e-06, \"min\": 0.0, \"max\": 0.0012514591217041016, \"st-dev\": 3.624462003401045e-05, \"initexec\": \"2023-05-05 07:01:21.709203\", \"startexec\": \"2023-05-05 07:01:21.709203\", \"endexec\": \"2023-05-05 07:01:26.709298\"}\r\n     {\"type\": \"detail\", \"processid\": 19292, \"calls\": 1973664, \"total\": 2.6392557621002197, \"avrg\": 1.3372366127670262e-06, \"min\": 0.0, \"max\": 0.0041027069091796875, \"st-dev\": 3.620965943471147e-05, \"initexec\": \"2023-05-05 07:01:21.840541\", \"startexec\": \"2023-05-05 07:01:21.840541\", \"endexec\": \"2023-05-05 07:01:26.841266\"}\r\n  {\"type\": \"core\", \"plan_executors\": 4, \"plan_executors_detail\": [4, 1], \"real_executors\": 4, \"group\": \"1x thread\", \"total_calls\": 7669236, \"avrg_time\": 1.3652863336090071e-06, \"std_deviation\": 3.645805510967187e-05, \"total_call_per_sec\": 2929788.3539391863, \"endexec\": \"2023-05-05 07:01:26.891144\"}\r\n  ...\r\n```\r\n\r\n## Graphs generated from qgate-graph based on outputs from qgate-perf\r\n#### 512 executors (128 processes x 4 threads)\r\n![graph](https://github.com/george0st/qgate-perf/blob/main/assets/PRF-Calc-2023-05-06_18-22-19-bulk-1x10.png?raw=true)\r\n![graph](https://github.com/george0st/qgate-perf/blob/main/assets/EXE-Calc-2023-05-06_18-22-19-bulk-1x10-plan-128x4.png?raw=true)\r\n\r\n#### 32 executors (8 processes x 4 threads)\r\n![graph](https://github.com/george0st/qgate-perf/blob/main/assets/PRF-NoSQL_igz_nonprod-2023-04-23_14-41-18-bulk-100x50.png?raw=true)\r\n![graph](https://github.com/george0st/qgate-perf/blob/main/assets/EXE-NoSQL-2023-05-04_19-33-30-bulk-1x50-plan-8x2.png?raw=true)\r\n\r\n",
    "bugtrack_url": null,
    "license": "MIT",
    "summary": "Performance test generator, part of Quality Gate",
    "version": "0.4.6",
    "project_urls": {
        "homepage": "https://github.com/george0st/qgate-perf/",
        "repository": "https://pypi.org/project/qgate_perf/"
    },
    "split_keywords": [
        "performancetest",
        "performance",
        "qualitygate"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "294c130b51a2ff21b176d4c6332099bd8affd5f130f57862ed9effc0b95e2ed1",
                "md5": "a589b32a14824ac7d5df8571beb04d76",
                "sha256": "11872a73b837a8286fe08babfd7ee5f640cf7da014bc13f243e0b2f54280db76"
            },
            "downloads": -1,
            "filename": "qgate_perf-0.4.6-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "a589b32a14824ac7d5df8571beb04d76",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.6",
            "size": 18656,
            "upload_time": "2023-12-22T11:46:48",
            "upload_time_iso_8601": "2023-12-22T11:46:48.545745Z",
            "url": "https://files.pythonhosted.org/packages/29/4c/130b51a2ff21b176d4c6332099bd8affd5f130f57862ed9effc0b95e2ed1/qgate_perf-0.4.6-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-12-22 11:46:48",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "george0st",
    "github_project": "qgate-perf",
    "travis_ci": false,
    "coveralls": true,
    "github_actions": false,
    "requirements": [],
    "lcname": "qgate-perf"
}
Elapsed time: 0.17324s