Package **PairedCompCalc** implements probabilistic Bayesian analysis
of Paired-Comparison data collected from psycho-physical experiments.
Paired comparisons are used, for example,
to evaluate the subjective sound quality of two or more different hearing aids
or any other quality aspects of any multimedia presentations.
The analysis model estimates quality parameters numerically on an objective *interval scale*,
although the raw input data are *subjective*
and indicate only an *ordinal* judgment for each presented pair.
## Paired-comparison Experiments
In a paired-comparison test procedure, each presentation includes exactly two test items,
and the test participant indicates a binary or graded judgment
about which of the two items is better in terms of the *perceptual attribute*
being investigated.
The perceptual attribute might be, for example, *speech clarity*, *sound quality*,
or whatever is specified by the instructions to participants.
The paired-comparison procedure can be applied
even if the difference between tested objects is very subtle,
even so small that the difference is just barely noticeable.
Data can be collected either in controlled *laboratory sessions*,
or by *Ecological Momentary Assessment (EMA)*, where each subject
evaluates the tested objects, e.g., hearing-aid programs,
while using them normally in real life.
This package does *not* include functions to administer the actual experiment;
it can only use existing results from an earlier experiment.
The package can analyze data from simple or quite complex experimental designs,
including the following features:
1. The experiment must include two or more **Test Objects** (A, B, C, etc.).
The analysis results will show quality parameters for all objects,
but the first object will always be used as a reference with quality = 0,
because the zero point on the scale is arbitrary.
2. One or more distinct participant **Groups** may be included.
The analysis will show systematic differences between the populations
from which the subjects were recruited.
3. One or more perceptual **Attributes** may be evaluated.
The analysis will show correlations between different attributes.
4. Response data may be collected in one or more distinct **Test Conditions**.
Each test condition may be a combination of categories from one or more *Test Factors*.
For example, one test factor may be *Stimulus Type*,
with categories *Speech*, or *Music*.
Another test factor may be, e.g.,
*Background*, with categories *Quiet*, or *Weak Noise*, or *Loud Noise*.
The analysis will show quality differences between categories within each test factor.
5. The subjects' **Responses** may be either *binary*, e.g.,
*B better than A*, or include an *ordinal grading* of the perceived difference, e.g.,
*no difference*, *slightly better*, or *much better*.
6. The analysis model does not require anything about the number of
replicated judgments for each pair, and the numbers may differ among subjects.
Of course, the reliability is improved if there are
many test participants, each giving many judgments
for each pair of objects.
The analysis estimates the *statistical credibility*
of any observed quality differences,
given the amount of collected data.
The Bayesian model is hierarchical.
The package can estimate predictive distributions of results for
* a random individual in the population from which the participants were recruited,
* the mean quality parameters in the population,
* a random individual in the group of test participants,
* each individual participant.
## Package Documentation
General information and version history is given in the package doc-string that may be accessed by command
`help(PairedCompCalc)`.
Input data can be accessed from files in several
of the formats that package Pandas can handle, e.g., .csv, .xlsx.
Specific information about the organization of input data files
is presented in the doc-string of module pc_data, accessible via `help(PairedCompCalc.pc_data)`.
After running an analysis, the logging output briefly explains
the analysis results presented in figures and tables.
Figures can be saved in any file format
allowed by Matplotlib.
Result tables can be saved in many of the
file formats that Pandas can handle,
e.g., .csv, .txt, .tex, as defined in module `pc_file`.
Thus, the results can easily be imported to a word-processing document
or to other statistical packages.
## Usage
1. Install the most recent package version:
`python3 -m pip install --upgrade PairedCompCalc`
2. For an introduction to the analysis results
and the xlsx input format,
run the simulation script: `python3 run_sim.py`
3. Copy the template script `run_pc.py`, rename it, and
edit the copy as suggested in the template, to specify
- your experimental layout,
- the top directory containing input data files,
- a directory where all output results will be stored.
4. Run your edited script: `python3 run_my_pc.py`
5. When planning an experiment, the statistical power can be crudely predicted,
to estimate the number of test participants needed to get a desired statistical strength.
Copy, edit, and run the script template `run_plan.py`
6. In the planning phase, complete analysis results may also be calculated
for synthetic data generated from simulated experiments.
Simulated experiments allow the same design variants as real experiments.
Copy, edit, and run the script template `run_sim.py`.
## Requirements
This package requires Python 3.9 or newer, with Numpy, Scipy, and Matplotlib,
as well as a support package samppy.
Package Pandas is used to handle input data and result tables
and requires openpyxl to access Excel (.xlsx) files.
The pip installer will check and install the required packages if needed.
Pandas can also access several other file formats,
but may then need other support packages that must be installed manually.
## References
A. Leijon, M. Dahlquist, and K. Smeds (2019).
Bayesian analysis of paired-comparison sound quality ratings.
*J Acoust Soc Amer, 146(5): 3174-3183*. [download](https://asa.scitation.org/doi/10.1121/1.5131024)
K. Smeds, F. Wolters, J. Larsson, P. Herrlin, and M. Dahlquist (2018).
Ecological momentary assessments for evaluation of hearing-aid preference.
*J Acoust Soc Amer* 143(3):1742–1742. [download](https://asa.scitation.org/doi/10.1121/1.5035685)
M. Dahlquist and A. Leijon (2003).
Paired-comparison rating of sound quality using MAP parameter estimation for data analysis.
In *First ISCA Tutorial and Research Workshop on Auditory Quality of Systems*,
Mont-Cenis, Germany. [download](https://www.isca-speech.org/archive_open/aqs2003/aqs3_079.html)
This Python package is a re-implementation and generalization of a similar MatLab package,
developed by Arne Leijon for *ORCA Europe, Widex A/S, Stockholm, Sweden*.
The MatLab development was financially supported by *Widex A/S, Denmark*.
Raw data
{
"_id": null,
"home_page": "",
"name": "PairedCompCalc",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "",
"keywords": "paired-comparison,Bayesian,psycho-physics",
"author": "Arne Leijon",
"author_email": "leijon@kth.se",
"download_url": "https://files.pythonhosted.org/packages/cd/ba/ffe3dd6e675e7b868bd8fe3bbc0c45d9e006ee717dfbb363f5ea573b923f/PairedCompCalc-2.1.1.tar.gz",
"platform": null,
"description": "Package **PairedCompCalc** implements probabilistic Bayesian analysis\nof Paired-Comparison data collected from psycho-physical experiments.\nPaired comparisons are used, for example,\nto evaluate the subjective sound quality of two or more different hearing aids\nor any other quality aspects of any multimedia presentations.\n\nThe analysis model estimates quality parameters numerically on an objective *interval scale*,\nalthough the raw input data are *subjective*\nand indicate only an *ordinal* judgment for each presented pair.\n\n## Paired-comparison Experiments\nIn a paired-comparison test procedure, each presentation includes exactly two test items,\nand the test participant indicates a binary or graded judgment\nabout which of the two items is better in terms of the *perceptual attribute*\nbeing investigated.\nThe perceptual attribute might be, for example, *speech clarity*, *sound quality*,\nor whatever is specified by the instructions to participants.\nThe paired-comparison procedure can be applied\neven if the difference between tested objects is very subtle,\neven so small that the difference is just barely noticeable.\n\nData can be collected either in controlled *laboratory sessions*,\nor by *Ecological Momentary Assessment (EMA)*, where each subject\nevaluates the tested objects, e.g., hearing-aid programs,\nwhile using them normally in real life.\n\nThis package does *not* include functions to administer the actual experiment;\nit can only use existing results from an earlier experiment.\nThe package can analyze data from simple or quite complex experimental designs,\nincluding the following features:\n\n1. The experiment must include two or more **Test Objects** (A, B, C, etc.).\n The analysis results will show quality parameters for all objects,\n but the first object will always be used as a reference with quality = 0,\n because the zero point on the scale is arbitrary.\n\n2. One or more distinct participant **Groups** may be included.\n The analysis will show systematic differences between the populations \n from which the subjects were recruited.\n\n3. One or more perceptual **Attributes** may be evaluated.\n The analysis will show correlations between different attributes.\n\n4. Response data may be collected in one or more distinct **Test Conditions**.\n Each test condition may be a combination of categories from one or more *Test Factors*.\n For example, one test factor may be *Stimulus Type*,\n with categories *Speech*, or *Music*.\n Another test factor may be, e.g.,\n *Background*, with categories *Quiet*, or *Weak Noise*, or *Loud Noise*.\n The analysis will show quality differences between categories within each test factor.\n\n5. The subjects' **Responses** may be either *binary*, e.g.,\n *B better than A*, or include an *ordinal grading* of the perceived difference, e.g.,\n *no difference*, *slightly better*, or *much better*.\n\n6. The analysis model does not require anything about the number of\n replicated judgments for each pair, and the numbers may differ among subjects.\n Of course, the reliability is improved if there are\n many test participants, each giving many judgments\n for each pair of objects.\n The analysis estimates the *statistical credibility*\n of any observed quality differences,\n given the amount of collected data.\n\n\nThe Bayesian model is hierarchical.\nThe package can estimate predictive distributions of results for\n* a random individual in the population from which the participants were recruited,\n* the mean quality parameters in the population,\n* a random individual in the group of test participants,\n* each individual participant.\n\n## Package Documentation\nGeneral information and version history is given in the package doc-string that may be accessed by command\n`help(PairedCompCalc)`.\n\nInput data can be accessed from files in several \nof the formats that package Pandas can handle, e.g., .csv, .xlsx.\nSpecific information about the organization of input data files\nis presented in the doc-string of module pc_data, accessible via `help(PairedCompCalc.pc_data)`.\n\nAfter running an analysis, the logging output briefly explains\nthe analysis results presented in figures and tables.\nFigures can be saved in any file format \nallowed by Matplotlib.\nResult tables can be saved in many of the \nfile formats that Pandas can handle,\ne.g., .csv, .txt, .tex, as defined in module `pc_file`.\nThus, the results can easily be imported to a word-processing document \nor to other statistical packages.\n\n## Usage\n1. Install the most recent package version:\n `python3 -m pip install --upgrade PairedCompCalc`\n\n2. For an introduction to the analysis results\n and the xlsx input format, \n run the simulation script: `python3 run_sim.py`\n\n3. Copy the template script `run_pc.py`, rename it, and\n edit the copy as suggested in the template, to specify\n - your experimental layout,\n - the top directory containing input data files,\n - a directory where all output results will be stored.\n\n4. Run your edited script: `python3 run_my_pc.py`\n\n5. When planning an experiment, the statistical power can be crudely predicted,\n to estimate the number of test participants needed to get a desired statistical strength.\n Copy, edit, and run the script template `run_plan.py`\n\n6. In the planning phase, complete analysis results may also be calculated\n for synthetic data generated from simulated experiments.\n Simulated experiments allow the same design variants as real experiments.\n Copy, edit, and run the script template `run_sim.py`.\n\n## Requirements\nThis package requires Python 3.9 or newer, with Numpy, Scipy, and Matplotlib,\nas well as a support package samppy. \nPackage Pandas is used to handle input data and result tables \nand requires openpyxl to access Excel (.xlsx) files.\nThe pip installer will check and install the required packages if needed.\n\nPandas can also access several other file formats, \nbut may then need other support packages that must be installed manually.\n\n## References\nA. Leijon, M. Dahlquist, and K. Smeds (2019).\nBayesian analysis of paired-comparison sound quality ratings.\n*J Acoust Soc Amer, 146(5): 3174-3183*. [download](https://asa.scitation.org/doi/10.1121/1.5131024)\n\nK. Smeds, F. Wolters, J. Larsson, P. Herrlin, and M. Dahlquist (2018).\nEcological momentary assessments for evaluation of hearing-aid preference.\n*J Acoust Soc Amer* 143(3):1742\u20131742. [download](https://asa.scitation.org/doi/10.1121/1.5035685)\n\nM. Dahlquist and A. Leijon (2003).\nPaired-comparison rating of sound quality using MAP parameter estimation for data analysis.\nIn *First ISCA Tutorial and Research Workshop on Auditory Quality of Systems*,\nMont-Cenis, Germany. [download](https://www.isca-speech.org/archive_open/aqs2003/aqs3_079.html)\n\nThis Python package is a re-implementation and generalization of a similar MatLab package,\ndeveloped by Arne Leijon for *ORCA Europe, Widex A/S, Stockholm, Sweden*.\nThe MatLab development was financially supported by *Widex A/S, Denmark*.\n\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "Statistical Analysis of Paired-Comparison Data",
"version": "2.1.1",
"project_urls": null,
"split_keywords": [
"paired-comparison",
"bayesian",
"psycho-physics"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5415dd406d237ee7cc33fab9cacb8b0d07fdbc9fb4905dd97ed4665deeebf3ab",
"md5": "04663c891bda153f873fd6a2186d187b",
"sha256": "97175326e87251025c7ec7912fa1733d96df0cdaff35de7d6dab39c2f44846a0"
},
"downloads": -1,
"filename": "PairedCompCalc-2.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "04663c891bda153f873fd6a2186d187b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 99445,
"upload_time": "2023-06-12T13:11:48",
"upload_time_iso_8601": "2023-06-12T13:11:48.815310Z",
"url": "https://files.pythonhosted.org/packages/54/15/dd406d237ee7cc33fab9cacb8b0d07fdbc9fb4905dd97ed4665deeebf3ab/PairedCompCalc-2.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "cdbaffe3dd6e675e7b868bd8fe3bbc0c45d9e006ee717dfbb363f5ea573b923f",
"md5": "2b63ac87388fba39631495b2833f5240",
"sha256": "ffbd3f5a845d2bc42dc428ab759c29382166145b216d6b9a40e75ac4d1311404"
},
"downloads": -1,
"filename": "PairedCompCalc-2.1.1.tar.gz",
"has_sig": false,
"md5_digest": "2b63ac87388fba39631495b2833f5240",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 90851,
"upload_time": "2023-06-12T13:11:50",
"upload_time_iso_8601": "2023-06-12T13:11:50.310615Z",
"url": "https://files.pythonhosted.org/packages/cd/ba/ffe3dd6e675e7b868bd8fe3bbc0c45d9e006ee717dfbb363f5ea573b923f/PairedCompCalc-2.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-12 13:11:50",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "pairedcompcalc"
}
JSON