========================
Regression Testing Utils
========================
Collection of Python tools to support execution of regression testing.
Exported Console Scripts
------------------------
The following exported console scripts are available:
- regression-testing-runner
Usage
-----
Install the package:
.. code-block:: shell
pip install regression-testing-utils
Prepare your regression configuration file.
Contents of example.yaml:
.. code-block:: yaml
---
# Acceptable values for file_format are csv or tsv.
# Meaning, this software can only compare files
# where both files are tab-delimited (tsv) or both
# files are comma-separated (csv).
file_format: tsv
# The line number that the column headers occur on.
# Column headers are necessary for reporting which
# which columns the differences were detected in.
header_line_number: 1
# The line number that the records start on.
# Beginning on this line, records will be compared
# between the two files.
records_start_line_number: 2
# The maximum number of columns. If specified, this
# will inform the software up to which line should
# be consider. Otherwise, all lines beginning at-
# and including records_start_line_number (above).
max_column_count:
# If the following is True, then the evaluator software will ignore
# the columns listed in the "ignore_columns_str".
ignore_columns: false
# The following will list the columns to be ignored by the evaluator
# software. Note: if the evaluator software should ignore columns
# specified in this list, then the "ignore_columns" boolean above
# should be set to true.
ignore_columns_str: null
workflow:
# If running in test mode, the actual commands will not be executed.
test_mode: false
stages:
# Each workflow stage is optional.
# The stage_name and commands are all required.
# The desc (description) is optional.
- stage_name: evaluation
desc: "Evaluate the results of the software being tested/evaluated"
commands:
- evaluator:
# Set-up for testing:
# cp regression-testing-utils/examples/example_actual.tsv /tmp/demo-regression-testing-utils/actual.tsv
# cp regression-testing-utils/examples/example_expected.tsv /tmp/demo-regression-testing-utils/expected.tsv
args:
# file1 is required. It must indicate the first file that will be compared against file2.
file1: /tmp/demo-regression-testing-utils/actual.tsv
# file2 is required. It must indicate the first file that will be compared against file1.
file2: /tmp/demo-regression-testing-utils/expected.tsv
# file_format is optional. It indicates what the file formats are.
file_format: tsv
# outfile is required. It indicates where the comparison report will be written to.
outfile: /tmp/demo-regression-testing-utils/comparison_report.txt
# If the following is set to true,
# the software will use the Rich Table
# API for displaying which stage is
# being executed.
# If set to false, will use a simple
# banner to display the same information.
use_rich_table: true
Contents of actual.tsv
.. code-block:: text
cat -n /tmp/demo-regression-testing-utils/actual.tsv
1 #CHROM POS ID REF ALT QUAL FILTER INFO
2 1 12345 rs567 A G 50 PASS DP=30;AF=0.2;AN=1000;CSQ=missense_variant|HIGH|GeneA|ENSG00000112345|transcriptA|ENST00000234567|protein_coding|1/10|c.123C>T|p.Arg41Trp|123/1000|ensembl
3 2 56789 rs555 T C 44 PASS DP=25;AF=0.1;AN=1200;CSQ=synonymous_variant|MEDIUM|GeneB|ENSG00000123456|transcriptB|ENST00000345678|protein_coding|5/20|c.567A>G|p.Ala189Ala|567/1200|ensembl
4 3 98765 rs123 G T 61 PASS DP=40;AF=0.3;AN=800;CSQ=splice_acceptor_variant|HIGH|GeneC|ENSG00000134567|transcriptC|ENST00000456789|protein_coding|2/15|c.987+1G>T|p.?|987/800|ensembl
5 1 34567 rs456 C A 55 FAIL DP=35;AF=0.15;AN=900;CSQ=frameshift_variant|HIGH|GeneX|ENSG00000145678|transcriptX|ENST00000567890|protein_coding|8/25|c.345_346insT|p.Leu116Phefs*12|345/900|ensembl
Contents of expected.tsv
.. code-block:: text
cat -n /tmp/demo-regression-testing-utils/expected.tsv
1 #CHROM POS ID REF ALT QUAL FILTER INFO
2 1 12345 rs567 A G 50 PASS DP=30;AF=0.2;AN=1000;CSQ=missense_variant|HIGH|GeneA|ENSG00000112345|transcriptA|ENST00000234567|protein_coding|1/10|c.123C>T|p.Arg41Trp|123/1000|ensembl
3 2 56789 rs890 T C 44 PASS DP=25;AF=0.1;AN=1200;CSQ=synonymous_variant|MEDIUM|GeneB|ENSG00000123456|transcriptB|ENST00000345678|protein_coding|5/20|c.567A>G|p.Ala189Ala|567/1200|ensembl
4 3 98765 rs123 G T 60 PASS DP=40;AF=0.3;AN=800;CSQ=splice_acceptor_variant|HIGH|GeneC|ENSG00000134567|transcriptC|ENST00000456789|protein_coding|2/15|c.987+1G>T|p.?|987/800|ensembl
5 1 34567 rs456 C A 55 PASS DP=35;AF=0.15;AN=900;CSQ=frameshift_variant|HIGH|GeneX|ENSG00000145678|transcriptX|ENST00000567890|protein_coding|8/25|c.345_346insT|p.Leu116Phefs*12|345/900|ensembl
Execution of exported console script:
.. code-block:: shell
regression-testing-runner --config_file /tmp/demo-regression-testing-utils/simple.yaml --outdir /tmp/demo-regression-testing-utils
--logfile was not specified and therefore was set to '/tmp/demo-regression-testing-utils/runner.log'
Setting file format to 'tsv'
3 differences found
Contents of comparison report:
.. code-block:: text
cat -n /tmp/demo-regression-testing-utils/comparison_report.txt
1 ## method-created: /tmp/regression-testing-utils/venv/lib/python3.10/site-packages/regression_testing_utils/evaluator.py
2 ## date-created: 2025-02-15-121022
3 ## created-by: sundaram
4 ## file 1: /tmp/demo-regression-testing-utils/actual.tsv
5 ## file 2: /tmp/demo-regression-testing-utils/expected.tsv
6 ## file-format: tsv
7 ## logfile: /tmp/demo-regression-testing-utils/runner.log
8 ## Number of differences: 3
9 Line # Column Name Column # Column Letter Value in File 1 Value in File 2
10 1 rs567 3 C rs555 rs890
11 2 50 6 F 61 60
12 3 PASS 7 G FAIL PASS
=======
History
=======
0.1.0 (2024-02-24)
------------------
* First release on PyPI.
Raw data
{
"_id": null,
"home_page": "https://github.com/jai-python3/regression-testing-utils",
"name": "regression-testing-utils",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "regression_testing_utils",
"author": "Jaideep Sundaram",
"author_email": "jai.python3@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/e4/f3/bf1b21a41a455c1cb370deee039003591b9bfb8951653421b3029da1f48e/regression_testing_utils-0.1.2.tar.gz",
"platform": null,
"description": "========================\nRegression Testing Utils\n========================\n\nCollection of Python tools to support execution of regression testing.\n\n\nExported Console Scripts\n------------------------\n\nThe following exported console scripts are available:\n\n- regression-testing-runner\n\n\nUsage\n-----\n\nInstall the package:\n\n.. code-block:: shell\n\n pip install regression-testing-utils\n\n\nPrepare your regression configuration file.\n\nContents of example.yaml:\n\n.. code-block:: yaml\n\n ---\n # Acceptable values for file_format are csv or tsv.\n # Meaning, this software can only compare files\n # where both files are tab-delimited (tsv) or both\n # files are comma-separated (csv).\n file_format: tsv\n\n # The line number that the column headers occur on.\n # Column headers are necessary for reporting which\n # which columns the differences were detected in.\n header_line_number: 1\n\n # The line number that the records start on.\n # Beginning on this line, records will be compared\n # between the two files.\n records_start_line_number: 2\n\n # The maximum number of columns. If specified, this\n # will inform the software up to which line should\n # be consider. Otherwise, all lines beginning at-\n # and including records_start_line_number (above).\n max_column_count:\n\n # If the following is True, then the evaluator software will ignore\n # the columns listed in the \"ignore_columns_str\".\n ignore_columns: false\n\n # The following will list the columns to be ignored by the evaluator\n # software. Note: if the evaluator software should ignore columns\n # specified in this list, then the \"ignore_columns\" boolean above\n # should be set to true.\n ignore_columns_str: null\n\n workflow:\n # If running in test mode, the actual commands will not be executed.\n test_mode: false\n stages:\n # Each workflow stage is optional.\n # The stage_name and commands are all required.\n # The desc (description) is optional.\n - stage_name: evaluation\n desc: \"Evaluate the results of the software being tested/evaluated\"\n commands:\n - evaluator:\n # Set-up for testing:\n # cp regression-testing-utils/examples/example_actual.tsv /tmp/demo-regression-testing-utils/actual.tsv\n # cp regression-testing-utils/examples/example_expected.tsv /tmp/demo-regression-testing-utils/expected.tsv\n args:\n # file1 is required. It must indicate the first file that will be compared against file2.\n file1: /tmp/demo-regression-testing-utils/actual.tsv\n # file2 is required. It must indicate the first file that will be compared against file1.\n file2: /tmp/demo-regression-testing-utils/expected.tsv\n # file_format is optional. It indicates what the file formats are.\n file_format: tsv\n # outfile is required. It indicates where the comparison report will be written to.\n outfile: /tmp/demo-regression-testing-utils/comparison_report.txt\n # If the following is set to true,\n # the software will use the Rich Table\n # API for displaying which stage is\n # being executed.\n # If set to false, will use a simple\n # banner to display the same information.\n use_rich_table: true\n\nContents of actual.tsv\n\n.. code-block:: text\n\n cat -n /tmp/demo-regression-testing-utils/actual.tsv \n 1 #CHROM POS ID REF ALT QUAL FILTER INFO\n 2 1 12345 rs567 A G 50 PASS DP=30;AF=0.2;AN=1000;CSQ=missense_variant|HIGH|GeneA|ENSG00000112345|transcriptA|ENST00000234567|protein_coding|1/10|c.123C>T|p.Arg41Trp|123/1000|ensembl\n 3 2 56789 rs555 T C 44 PASS DP=25;AF=0.1;AN=1200;CSQ=synonymous_variant|MEDIUM|GeneB|ENSG00000123456|transcriptB|ENST00000345678|protein_coding|5/20|c.567A>G|p.Ala189Ala|567/1200|ensembl\n 4 3 98765 rs123 G T 61 PASS DP=40;AF=0.3;AN=800;CSQ=splice_acceptor_variant|HIGH|GeneC|ENSG00000134567|transcriptC|ENST00000456789|protein_coding|2/15|c.987+1G>T|p.?|987/800|ensembl\n 5 1 34567 rs456 C A 55 FAIL DP=35;AF=0.15;AN=900;CSQ=frameshift_variant|HIGH|GeneX|ENSG00000145678|transcriptX|ENST00000567890|protein_coding|8/25|c.345_346insT|p.Leu116Phefs*12|345/900|ensembl\n\nContents of expected.tsv\n\n.. code-block:: text\n\n cat -n /tmp/demo-regression-testing-utils/expected.tsv \n 1 #CHROM POS ID REF ALT QUAL FILTER INFO\n 2 1 12345 rs567 A G 50 PASS DP=30;AF=0.2;AN=1000;CSQ=missense_variant|HIGH|GeneA|ENSG00000112345|transcriptA|ENST00000234567|protein_coding|1/10|c.123C>T|p.Arg41Trp|123/1000|ensembl\n 3 2 56789 rs890 T C 44 PASS DP=25;AF=0.1;AN=1200;CSQ=synonymous_variant|MEDIUM|GeneB|ENSG00000123456|transcriptB|ENST00000345678|protein_coding|5/20|c.567A>G|p.Ala189Ala|567/1200|ensembl\n 4 3 98765 rs123 G T 60 PASS DP=40;AF=0.3;AN=800;CSQ=splice_acceptor_variant|HIGH|GeneC|ENSG00000134567|transcriptC|ENST00000456789|protein_coding|2/15|c.987+1G>T|p.?|987/800|ensembl\n 5 1 34567 rs456 C A 55 PASS DP=35;AF=0.15;AN=900;CSQ=frameshift_variant|HIGH|GeneX|ENSG00000145678|transcriptX|ENST00000567890|protein_coding|8/25|c.345_346insT|p.Leu116Phefs*12|345/900|ensembl\n\nExecution of exported console script:\n\n.. code-block:: shell\n\n regression-testing-runner --config_file /tmp/demo-regression-testing-utils/simple.yaml --outdir /tmp/demo-regression-testing-utils\n --logfile was not specified and therefore was set to '/tmp/demo-regression-testing-utils/runner.log'\n Setting file format to 'tsv'\n 3 differences found\n\nContents of comparison report:\n\n.. code-block:: text\n\n cat -n /tmp/demo-regression-testing-utils/comparison_report.txt \n 1 ## method-created: /tmp/regression-testing-utils/venv/lib/python3.10/site-packages/regression_testing_utils/evaluator.py\n 2 ## date-created: 2025-02-15-121022\n 3 ## created-by: sundaram\n 4 ## file 1: /tmp/demo-regression-testing-utils/actual.tsv\n 5 ## file 2: /tmp/demo-regression-testing-utils/expected.tsv\n 6 ## file-format: tsv\n 7 ## logfile: /tmp/demo-regression-testing-utils/runner.log\n 8 ## Number of differences: 3\n 9 Line # Column Name Column # Column Letter Value in File 1 Value in File 2\n 10 1 rs567 3 C rs555 rs890\n 11 2 50 6 F 61 60\n 12 3 PASS 7 G FAIL PASS\n\n\n=======\nHistory\n=======\n\n0.1.0 (2024-02-24)\n------------------\n\n* First release on PyPI.\n",
"bugtrack_url": null,
"license": null,
"summary": "Collection of Python tools to support execution of regression testing.",
"version": "0.1.2",
"project_urls": {
"Homepage": "https://github.com/jai-python3/regression-testing-utils"
},
"split_keywords": [
"regression_testing_utils"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b4ff85ffed7590f40f2530faab46639f489af7a82443b9dda1a689ac9f844f06",
"md5": "1b41b8886a4a9be63503ab0100d360fa",
"sha256": "f43d6f5539061c7441c250df18167690f3cd8724dcdc266896a3d90a70d153ad"
},
"downloads": -1,
"filename": "regression_testing_utils-0.1.2-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "1b41b8886a4a9be63503ab0100d360fa",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.10",
"size": 26349,
"upload_time": "2025-02-15T17:20:36",
"upload_time_iso_8601": "2025-02-15T17:20:36.357052Z",
"url": "https://files.pythonhosted.org/packages/b4/ff/85ffed7590f40f2530faab46639f489af7a82443b9dda1a689ac9f844f06/regression_testing_utils-0.1.2-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e4f3bf1b21a41a455c1cb370deee039003591b9bfb8951653421b3029da1f48e",
"md5": "b5aae24fbf22e6fcdae69811d3f0f405",
"sha256": "b62a8a7331b3ed8b634681cd385e1eae1535d2d0d2744e6c2b2b3b3e500f93fa"
},
"downloads": -1,
"filename": "regression_testing_utils-0.1.2.tar.gz",
"has_sig": false,
"md5_digest": "b5aae24fbf22e6fcdae69811d3f0f405",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 25146,
"upload_time": "2025-02-15T17:20:38",
"upload_time_iso_8601": "2025-02-15T17:20:38.210695Z",
"url": "https://files.pythonhosted.org/packages/e4/f3/bf1b21a41a455c1cb370deee039003591b9bfb8951653421b3029da1f48e/regression_testing_utils-0.1.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-15 17:20:38",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "jai-python3",
"github_project": "regression-testing-utils",
"github_not_found": true,
"lcname": "regression-testing-utils"
}
JSON
