# ROC Face
This is a Python package to compute basic signal detection theory measures and to fit theoretical recognition memory models to data using ROC curves.
Currently-supported models are those most frequently seen in the literature:
- High Threshold
- Equal-Variance Signal Detection
- Unequal-Variance Signal Detection
- Dual-Process Signal Detection
These models are fit to observed data by minimising the $G$, $\chi^2$, log-likelihood, or sum of squared error statistics between observed and model-predicted data.
# Setting Up
Install with `pip`:
```$ pip install roc-face```
It is generally recommend to use a virtual environment when installing python packages ([see here](https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/)).
# Usage
## Example 1: Basic signal detection theory measures
With a single true positive and false positive rate, compute the common measures of sensitivity and bias:
```python
>>> from roc_face import measures
>>> measures.d_prime(0.75, 0.21)
1.480910997214322
>>> measures.c_bias(0.75, 0.21)
0.06596574841107933
```
You can also return a dictionary of measures in a similar way:
```python
>>> measures.compute_performance(tpr=0.75, fpr=0.21)
{
'FPR': 0.21,
'TPR': 0.75,
'dprime': 1.480910997214322,
'cbias': 0.06596574841107933,
'aprime': 0.850886075949367,
'beta': 1.1026202605581668,
}
```
## Example 2: Receiver operating characteristic (ROC) modelling
Given a set of responses to a set of signal and noise trials, the ROC and z-ROC plots of the observed frequencies can be viewed as follows.
```python
>>> import matplotlib.pyplot as plt
>>> from roc_face import utils
# Strongest "signal" <---> Strongest "noise"
# All responses to signal-present trials
>>> signal = [505,248,226,172,144,93]
# All responses to signal-absent (i.e. noise) trials
>>> noise = [115,185,304,523,551,397]
>>> fig, ax = plt.subplots(1, 2)
>>> utils.plot_roc(signal, noise, ax=ax[0])
>>> utils.plot_zroc(signal, noise, poly=1, ax=ax[1])
>>> ax[1].legend()
>>> plt.show()
```
<img src="https://github.com/lcdunne/signal-detection/raw/develop/example/example_ROC_zROC.png" alt="" width="620">
The `utils.plot_roc` and `utils.plot_zroc` functions are for convenience as
they carry out some minor plotting customisations (square axes, chance-line,
etc). The fitted line on the *z*-ROC in this example is a simple linear model,
which is useful for interpreting the ROC data.
With the signal and noise data, the different models can be fitted.
```python
>>> from roc_face.models import SignalDetection
# Create an equal- and unequal-variance signal detection models
>>> evsd = SignalDetection(signal, noise)
>>> uvsd = SignalDetection(signal, noise, equal_variance=False)
```
After creation, the models are fit as follows:
```python
# Fit the models using the G-test fit function
>>> evsd.fit(verbose=True)
(
{
'model': 'Equal Variance Signal Detection',
'success': True,
'method': 'G',
'statistic': 86.65649645461215,
'log_likelihood': -13560.04271197083,
'AIC': 27132.08542394166,
'BIC': 27171.729820697474,
'SSE': 0.0038210383584672968
},
{
'd': 1.3706692903039621,
'criteria': array([ 0.87497843, 0.40584089, -0.01393461, -0.48919362, -1.0589949 ])
}
)
```
Using `verbose=True` in the `fit` method prints out the results of the fitting procedure when it ends, along with the parameter estimates. After calling `fit`, they can also be accessed via `.results` and `.parameter_estimates`:
```python
>>> uvsd.fit()
>>> uvsd.results
{
'model': 'Unequal Variance Signal Detection',
'success': True,
'method': 'G',
'statistic': 0.9957013337195149,
'log_likelihood': -13517.212314410383,
'AIC': 27048.424628820765,
'BIC': 27094.676425035883,
'SSE': 0.00014501577579512347
}
>>> uvsd.parameter_estimates
{
'd': 1.6023922394537697,
'scale': 1.32284210876227,
'criteria': array([ 0.95984187, 0.36806527, -0.12572417, -0.65421799, -1.2496735 ])
}
```
These models can also be compared to one another. Although it is common practice to compare with the AIC or BIC values (see the results), it can also be done with the `.compare` method:
```python
>>> evsd.compare(uvsd)
('G(EVSD - UVSD)', 85.66079512089179, 1, 2.1360619666273588e-20)
```
This shows that the UVSD provides a significantly better fit than the EVSD.
Finally, we can just view the ROC data and the two fitted models as follows:
```python
>>> fig, ax = plt.subplots(dpi=120)
>>> utils.plot_roc(signal, noise, c='k', ax=ax, label='data')
>>> ax.plot(*evsd.curve, label='EVSD')
>>> ax.plot(*uvsd.curve, label='UVSD')
>>> ax.legend(loc='lower right')
>>> plt.show()
```
<img src="https://github.com/lcdunne/signal-detection/raw/develop/example/example_EVSD_UVSD.png" alt="" width="480">
Raw data
{
"_id": null,
"home_page": "",
"name": "roc-face",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "signal detection theory,receiver operating characteristics,ROC,psychology",
"author": "Lewis Dunne",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/96/65/8a38064bce7d2296d9299861d4f97bdd491fddb586ee8f580645e0ba1b05/roc_face-0.1.1.tar.gz",
"platform": null,
"description": "# ROC Face\n\nThis is a Python package to compute basic signal detection theory measures and to fit theoretical recognition memory models to data using ROC curves.\n\nCurrently-supported models are those most frequently seen in the literature:\n- High Threshold\n- Equal-Variance Signal Detection\n- Unequal-Variance Signal Detection\n- Dual-Process Signal Detection\n\nThese models are fit to observed data by minimising the $G$, $\\chi^2$, log-likelihood, or sum of squared error statistics between observed and model-predicted data.\n\n# Setting Up\nInstall with `pip`:\n\n```$ pip install roc-face```\n\nIt is generally recommend to use a virtual environment when installing python packages ([see here](https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/)).\n\n# Usage\n\n## Example 1: Basic signal detection theory measures\nWith a single true positive and false positive rate, compute the common measures of sensitivity and bias:\n\n```python\n>>> from roc_face import measures\n\n>>> measures.d_prime(0.75, 0.21)\n1.480910997214322\n\n>>> measures.c_bias(0.75, 0.21)\n0.06596574841107933\n```\nYou can also return a dictionary of measures in a similar way:\n```python\n>>> measures.compute_performance(tpr=0.75, fpr=0.21)\n{\n 'FPR': 0.21,\n 'TPR': 0.75,\n 'dprime': 1.480910997214322,\n 'cbias': 0.06596574841107933,\n 'aprime': 0.850886075949367,\n 'beta': 1.1026202605581668,\n}\n```\n\n## Example 2: Receiver operating characteristic (ROC) modelling\nGiven a set of responses to a set of signal and noise trials, the ROC and z-ROC plots of the observed frequencies can be viewed as follows.\n\n```python\n>>> import matplotlib.pyplot as plt\n>>> from roc_face import utils\n\n# Strongest \"signal\" <---> Strongest \"noise\"\n# All responses to signal-present trials\n>>> signal = [505,248,226,172,144,93]\n\n# All responses to signal-absent (i.e. noise) trials\n>>> noise = [115,185,304,523,551,397]\n\n>>> fig, ax = plt.subplots(1, 2)\n>>> utils.plot_roc(signal, noise, ax=ax[0])\n>>> utils.plot_zroc(signal, noise, poly=1, ax=ax[1])\n>>> ax[1].legend()\n>>> plt.show()\n```\n<img src=\"https://github.com/lcdunne/signal-detection/raw/develop/example/example_ROC_zROC.png\" alt=\"\" width=\"620\">\n\nThe `utils.plot_roc` and `utils.plot_zroc` functions are for convenience as \nthey carry out some minor plotting customisations (square axes, chance-line, \netc). The fitted line on the *z*-ROC in this example is a simple linear model, \nwhich is useful for interpreting the ROC data.\n\nWith the signal and noise data, the different models can be fitted.\n\n```python\n>>> from roc_face.models import SignalDetection\n\n# Create an equal- and unequal-variance signal detection models\n>>> evsd = SignalDetection(signal, noise)\n>>> uvsd = SignalDetection(signal, noise, equal_variance=False)\n```\n\nAfter creation, the models are fit as follows:\n\n```python\n# Fit the models using the G-test fit function\n>>> evsd.fit(verbose=True)\n(\n {\n 'model': 'Equal Variance Signal Detection',\n 'success': True,\n 'method': 'G',\n 'statistic': 86.65649645461215,\n 'log_likelihood': -13560.04271197083,\n 'AIC': 27132.08542394166,\n 'BIC': 27171.729820697474,\n 'SSE': 0.0038210383584672968\n },\n {\n 'd': 1.3706692903039621,\n 'criteria': array([ 0.87497843, 0.40584089, -0.01393461, -0.48919362, -1.0589949 ])\n }\n)\n```\nUsing `verbose=True` in the `fit` method prints out the results of the fitting procedure when it ends, along with the parameter estimates. After calling `fit`, they can also be accessed via `.results` and `.parameter_estimates`:\n```python\n>>> uvsd.fit()\n>>> uvsd.results\n{\n 'model': 'Unequal Variance Signal Detection',\n 'success': True,\n 'method': 'G',\n 'statistic': 0.9957013337195149,\n 'log_likelihood': -13517.212314410383,\n 'AIC': 27048.424628820765,\n 'BIC': 27094.676425035883,\n 'SSE': 0.00014501577579512347\n}\n>>> uvsd.parameter_estimates\n{\n 'd': 1.6023922394537697,\n 'scale': 1.32284210876227,\n 'criteria': array([ 0.95984187, 0.36806527, -0.12572417, -0.65421799, -1.2496735 ])\n}\n```\n\nThese models can also be compared to one another. Although it is common practice to compare with the AIC or BIC values (see the results), it can also be done with the `.compare` method:\n```python\n>>> evsd.compare(uvsd)\n('G(EVSD - UVSD)', 85.66079512089179, 1, 2.1360619666273588e-20)\n```\nThis shows that the UVSD provides a significantly better fit than the EVSD.\n\nFinally, we can just view the ROC data and the two fitted models as follows:\n\n```python\n>>> fig, ax = plt.subplots(dpi=120)\n>>> utils.plot_roc(signal, noise, c='k', ax=ax, label='data')\n>>> ax.plot(*evsd.curve, label='EVSD')\n>>> ax.plot(*uvsd.curve, label='UVSD')\n>>> ax.legend(loc='lower right')\n>>> plt.show()\n```\n<img src=\"https://github.com/lcdunne/signal-detection/raw/develop/example/example_EVSD_UVSD.png\" alt=\"\" width=\"480\">\n",
"bugtrack_url": null,
"license": "",
"summary": "A package to fit models to recognition memory ROC data and compute signal detection measures.",
"version": "0.1.1",
"split_keywords": [
"signal detection theory",
"receiver operating characteristics",
"roc",
"psychology"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8309723a051b79a995f12c2ee89f0408e67bf6ffd2b3d2c440bc40a5f13b92e4",
"md5": "e576a82f2e2089053eb9b75732051bc2",
"sha256": "064572023875990eb51e82f38c94850cc7ebf33da31d8a8017fed891875f0def"
},
"downloads": -1,
"filename": "roc_face-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e576a82f2e2089053eb9b75732051bc2",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 19457,
"upload_time": "2023-01-31T08:59:44",
"upload_time_iso_8601": "2023-01-31T08:59:44.806674Z",
"url": "https://files.pythonhosted.org/packages/83/09/723a051b79a995f12c2ee89f0408e67bf6ffd2b3d2c440bc40a5f13b92e4/roc_face-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "96658a38064bce7d2296d9299861d4f97bdd491fddb586ee8f580645e0ba1b05",
"md5": "68727fe198e35d123a1d32e6e54cc07e",
"sha256": "76a7d4828809f13ff62a073ae4cd0e41a1927d192a10c06acc8b614ba029700a"
},
"downloads": -1,
"filename": "roc_face-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "68727fe198e35d123a1d32e6e54cc07e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 19150,
"upload_time": "2023-01-31T08:59:46",
"upload_time_iso_8601": "2023-01-31T08:59:46.873010Z",
"url": "https://files.pythonhosted.org/packages/96/65/8a38064bce7d2296d9299861d4f97bdd491fddb586ee8f580645e0ba1b05/roc_face-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-01-31 08:59:46",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "roc-face"
}
JSON