# iQual
[](https://github.com/worldbank/iQual/blob/main/LICENSE.md)
[](https://badge.fury.io/py/iQual)
[](https://worldbank.github.io/iQual/)
This repository contains the code and resources necessary to implement the techniques described in the paper [A Method to Scale-Up Interpretative Qualitative Analysis, with an Application to Aspirations in Cox's Bazaar, Bangladesh](https://documents.worldbank.org/en/publication/documents-reports/documentdetail/099759305162210822/idu0a357362e00b6004c580966006b1c2f2e3996). The `iQual` package is designed for qualitative analysis of open-ended interviews and aims to extend a small set of interpretative human-codes to a much larger set of documents using natural language processing. The package provides a method for assessing the robustness and reliability of this approach. The `iQual` package has been applied to analyze 2,200 open-ended interviews on parent's aspirations for children from Rohingya refugees and their Bangladeshi hosts in Cox's Bazaar, Bangladesh.
With `iQual`, researchers can efficiently analyze large amounts of qualitative data while maintaining the nuance and accuracy of human interpretation.
---
## Installation
- To install `iQual` using pip, use the following command:
```sh
pip install -U iQual
```
---
## Getting Started
For a quick introduction to using iQual, check out our [Getting Started notebook](notebooks/Getting%20Started.ipynb). This tutorial provides:
- A complete overview of the basic workflow
- Step-by-step examples using real-world data
- Clear explanations of key concepts
- Code you can run immediately
This notebook is perfect for new users who want to understand iQual's core functionality without diving into the technical details.
---
## Model Training
The [Model Training](notebooks/Model%20Training.ipynb) notebook demonstrates advanced training techniques including cross-validation and hyperparameter optimization using the same politeness dataset from [Getting Started](notebooks/Getting%20Started.ipynb).
---
## Features
`iQual` is a package designed for qualitative analysis of open-ended interviews. It allows researchers to efficiently analyze large amounts of qualitative data while maintaining the nuance and accuracy of human interpretation.
- Customizable pipelines using [scikit-learn](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.pipeline) pipelines
- Text-vectorization using:
- Any of the [scikit-learn](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.feature_extraction.text) text feature extraction method.
- Any [sentence-transformers](https://sbert.net/) compatible model.
- Any [spaCy](https://spacy.io/) model with a `doc.vector` attribute.
- Classification using any [scikit-learn](https://scikit-learn.org/stable/modules/) classification method
- Feature Transformation:
- Dimensionality reduction using any [scikit-learn](https://scikit-learn.org/stable/modules/) `decomposition` method, or UMAP using [umap-learn](https://umap-learn.readthedocs.io/en/latest/).
- Feature scaling using any [scikit-learn](https://scikit-learn.org/stable/modules/) `preprocessing` method.
- Model selection and performance evaluation using [scikit-learn](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection) methods.
- Tests for bias and interpretability, with [statsmodels](https://www.statsmodels.org/stable/index.html).
---
## Basic Usage
The following code demonstrates the basic usage of the `iQual` package:
```python
from iqual import iqualnlp # Import `iqualnlp` from the `iqual` package
iqual_model = iqualnlp.Model() # Initiate the model class
# Add text features (using TF-IDF vectorization by default)
iqual_model.add_text_features('question', 'answer')
# Add a classifier (Logistic Regression by default)
iqual_model.add_classifier()
# Add a threshold layer for improved performance on imbalanced data
iqual_model.add_threshold()
# Compile the model
iqual_model.compile()
# Fit the model to your data
iqual_model.fit(X_train, y_train)
# Make predictions
y_pred = iqual_model.predict(X_test)
```
For a more detailed introduction, check out our [Getting Started notebook](notebooks/Getting%20Started.ipynb).
---
## Notebooks
The [notebooks](https://github.com/worldbank/iQual/tree/main/notebooks) folder contains detailed examples on using `iQual`:
* [Getting Started](https://github.com/worldbank/iQual/tree/main/notebooks/Getting%20Started.ipynb)
A complete introduction to iQual with a self-contained example for new users.
* [Basic Modelling](https://github.com/worldbank/iQual/tree/main/notebooks/basic)
These notebooks demonstrate the basic usage of the package, the pipeline construction, and the vectorization and classification options.
* [Advanced Modelling](https://github.com/worldbank/iQual/tree/main/notebooks/advanced)
These notebooks demonstrate advanced pipeline construction, mixing and matching of feature extraction and classification methods, and model selection.
* [Interpretability](https://github.com/worldbank/iQual/tree/main/notebooks/interpretability)
These notebooks demonstrate the interpretability and related tests for measurement and comparison of interpretability across human and enhanced (machine + human) codes.
* [Bias](https://github.com/worldbank/iQual/tree/main/notebooks/bias) and [Efficiency](https://github.com/worldbank/iQual/tree/main/notebooks/efficiency)
These notebooks demonstrate the bias and efficiency tests for determining the value and validity of enhanced codes.
---
## Citation & Authors
If you use this package, please cite the following paper:
[A Method to Scale-Up Interpretative Qualitative Analysis, with an Application to Aspirations in Cox’s Bazaar, Bangladesh](https://documents.worldbank.org/en/publication/documents-reports/documentdetail/099759305162210822/idu0a357362e00b6004c580966006b1c2f2e3996)
```
Ashwin,Julian; Rao,Vijayendra; Biradavolu,Monica Rao; Chhabra,Aditya; Haque,Arshia; Khan,Afsana Iffat; Krishnan,Nandini.
A Method to Scale-Up Interpretative Qualitative Analysis, with an Application to Aspirations in Cox’s Bazaar, Bangladesh (English). (Policy Research Working Paper No. WPS 10046)
Paper is funded by the Knowledge for Change Program (KCP) Washington, D.C. : World Bank Group.
http://documents.worldbank.org/curated/en/099759305162210822/IDU0a357362e00b6004c580966006b1c2f2e3996
```
---
## Maintainers
### Please contact the following people for any queries regarding the package:
- [Aditya Karan Chhabra](mailto:aditya0chhabra@gmail.com)
- [Julian Ashwin](mailto:julianashwin@googlemail.com)
Raw data
{
"_id": null,
"home_page": "https://github.com/worldbank/iQual",
"name": "iQual",
"maintainer": "Aditya Karan Chhabra",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "aditya0chhabra@gmail.com",
"keywords": "nlp natural-language-processing qualitative-analysis human-coding qualitative-research",
"author": "Aditya Karan Chhabra",
"author_email": "aditya0chhabra@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/c3/31/bda0f4613a5f60a76ae0efd403a479a4c06987db0d450278cac6cc3da386/iqual-0.1.3.tar.gz",
"platform": null,
"description": "# iQual\n\n[](https://github.com/worldbank/iQual/blob/main/LICENSE.md)\n[](https://badge.fury.io/py/iQual)\n[](https://worldbank.github.io/iQual/)\n\nThis repository contains the code and resources necessary to implement the techniques described in the paper [A Method to Scale-Up Interpretative Qualitative Analysis, with an Application to Aspirations in Cox's Bazaar, Bangladesh](https://documents.worldbank.org/en/publication/documents-reports/documentdetail/099759305162210822/idu0a357362e00b6004c580966006b1c2f2e3996). The `iQual` package is designed for qualitative analysis of open-ended interviews and aims to extend a small set of interpretative human-codes to a much larger set of documents using natural language processing. The package provides a method for assessing the robustness and reliability of this approach. The `iQual` package has been applied to analyze 2,200 open-ended interviews on parent's aspirations for children from Rohingya refugees and their Bangladeshi hosts in Cox's Bazaar, Bangladesh.\n\nWith `iQual`, researchers can efficiently analyze large amounts of qualitative data while maintaining the nuance and accuracy of human interpretation.\n\n---\n\n## Installation\n- To install `iQual` using pip, use the following command:\n```sh\npip install -U iQual\n```\n\n---\n\n## Getting Started\n\nFor a quick introduction to using iQual, check out our [Getting Started notebook](notebooks/Getting%20Started.ipynb). This tutorial provides:\n- A complete overview of the basic workflow\n- Step-by-step examples using real-world data\n- Clear explanations of key concepts\n- Code you can run immediately\n\nThis notebook is perfect for new users who want to understand iQual's core functionality without diving into the technical details.\n\n---\n\n## Model Training\n\nThe [Model Training](notebooks/Model%20Training.ipynb) notebook demonstrates advanced training techniques including cross-validation and hyperparameter optimization using the same politeness dataset from [Getting Started](notebooks/Getting%20Started.ipynb).\n\n---\n\n## Features\n\n`iQual` is a package designed for qualitative analysis of open-ended interviews. It allows researchers to efficiently analyze large amounts of qualitative data while maintaining the nuance and accuracy of human interpretation.\n\n- Customizable pipelines using [scikit-learn](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.pipeline) pipelines\n\n- Text-vectorization using:\n - Any of the [scikit-learn](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.feature_extraction.text) text feature extraction method.\n - Any [sentence-transformers](https://sbert.net/) compatible model.\n - Any [spaCy](https://spacy.io/) model with a `doc.vector` attribute.\n\n- Classification using any [scikit-learn](https://scikit-learn.org/stable/modules/) classification method\n\n- Feature Transformation:\n - Dimensionality reduction using any [scikit-learn](https://scikit-learn.org/stable/modules/) `decomposition` method, or UMAP using [umap-learn](https://umap-learn.readthedocs.io/en/latest/).\n - Feature scaling using any [scikit-learn](https://scikit-learn.org/stable/modules/) `preprocessing` method.\n \n- Model selection and performance evaluation using [scikit-learn](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection) methods.\n- Tests for bias and interpretability, with [statsmodels](https://www.statsmodels.org/stable/index.html).\n\n---\n\n## Basic Usage\n\nThe following code demonstrates the basic usage of the `iQual` package:\n\n```python\nfrom iqual import iqualnlp # Import `iqualnlp` from the `iqual` package\n\niqual_model = iqualnlp.Model() # Initiate the model class\n\n# Add text features (using TF-IDF vectorization by default)\niqual_model.add_text_features('question', 'answer')\n\n# Add a classifier (Logistic Regression by default)\niqual_model.add_classifier()\n\n# Add a threshold layer for improved performance on imbalanced data\niqual_model.add_threshold()\n\n# Compile the model\niqual_model.compile()\n\n# Fit the model to your data\niqual_model.fit(X_train, y_train)\n\n# Make predictions\ny_pred = iqual_model.predict(X_test)\n```\n\nFor a more detailed introduction, check out our [Getting Started notebook](notebooks/Getting%20Started.ipynb).\n\n---\n\n## Notebooks\n\nThe [notebooks](https://github.com/worldbank/iQual/tree/main/notebooks) folder contains detailed examples on using `iQual`:\n\n* [Getting Started](https://github.com/worldbank/iQual/tree/main/notebooks/Getting%20Started.ipynb)\nA complete introduction to iQual with a self-contained example for new users.\n\n* [Basic Modelling](https://github.com/worldbank/iQual/tree/main/notebooks/basic)\nThese notebooks demonstrate the basic usage of the package, the pipeline construction, and the vectorization and classification options.\n\n* [Advanced Modelling](https://github.com/worldbank/iQual/tree/main/notebooks/advanced) \nThese notebooks demonstrate advanced pipeline construction, mixing and matching of feature extraction and classification methods, and model selection.\n\n* [Interpretability](https://github.com/worldbank/iQual/tree/main/notebooks/interpretability)\nThese notebooks demonstrate the interpretability and related tests for measurement and comparison of interpretability across human and enhanced (machine + human) codes.\n\n* [Bias](https://github.com/worldbank/iQual/tree/main/notebooks/bias) and [Efficiency](https://github.com/worldbank/iQual/tree/main/notebooks/efficiency)\nThese notebooks demonstrate the bias and efficiency tests for determining the value and validity of enhanced codes.\n\n---\n\n## Citation & Authors\nIf you use this package, please cite the following paper:\n\n[A Method to Scale-Up Interpretative Qualitative Analysis, with an Application to Aspirations in Cox\u2019s Bazaar, Bangladesh](https://documents.worldbank.org/en/publication/documents-reports/documentdetail/099759305162210822/idu0a357362e00b6004c580966006b1c2f2e3996)\n\n```\nAshwin,Julian; Rao,Vijayendra; Biradavolu,Monica Rao; Chhabra,Aditya; Haque,Arshia; Khan,Afsana Iffat; Krishnan,Nandini.\nA Method to Scale-Up Interpretative Qualitative Analysis, with an Application to Aspirations in Cox\u2019s Bazaar, Bangladesh (English). (Policy Research Working Paper No. WPS 10046)\nPaper is funded by the Knowledge for Change Program (KCP) Washington, D.C. : World Bank Group.\nhttp://documents.worldbank.org/curated/en/099759305162210822/IDU0a357362e00b6004c580966006b1c2f2e3996\n```\n\n---\n\n## Maintainers\n### Please contact the following people for any queries regarding the package:\n\n- [Aditya Karan Chhabra](mailto:aditya0chhabra@gmail.com)\n- [Julian Ashwin](mailto:julianashwin@googlemail.com)\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "iQual is a package that leverages natural language processing to scale up interpretative qualitative analysis. It also provides methods to assess the bias, interpretability and efficiency of the machine-enhanced codes.",
"version": "0.1.3",
"project_urls": {
"Homepage": "https://github.com/worldbank/iQual"
},
"split_keywords": [
"nlp",
"natural-language-processing",
"qualitative-analysis",
"human-coding",
"qualitative-research"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "f1823a151cc66a36b519a654731e3b7c6e92c79d200e34f60b60063f1d496e68",
"md5": "0b5792ded447e28b830c4d4effe453ed",
"sha256": "776eb6c4e14c73f30dd8fdfdd485784a7b26bf7c92c5a0aec239bc34aa5cbc9b"
},
"downloads": -1,
"filename": "iqual-0.1.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "0b5792ded447e28b830c4d4effe453ed",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 22619,
"upload_time": "2025-10-25T17:18:12",
"upload_time_iso_8601": "2025-10-25T17:18:12.229993Z",
"url": "https://files.pythonhosted.org/packages/f1/82/3a151cc66a36b519a654731e3b7c6e92c79d200e34f60b60063f1d496e68/iqual-0.1.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "c331bda0f4613a5f60a76ae0efd403a479a4c06987db0d450278cac6cc3da386",
"md5": "354c5ee519bfa70a03e974a073f18791",
"sha256": "753304e9ea0377a2f9484cefa51ef502282dbf67a08fd37f197c5f83b8193cb8"
},
"downloads": -1,
"filename": "iqual-0.1.3.tar.gz",
"has_sig": false,
"md5_digest": "354c5ee519bfa70a03e974a073f18791",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 20034,
"upload_time": "2025-10-25T17:18:13",
"upload_time_iso_8601": "2025-10-25T17:18:13.971109Z",
"url": "https://files.pythonhosted.org/packages/c3/31/bda0f4613a5f60a76ae0efd403a479a4c06987db0d450278cac6cc3da386/iqual-0.1.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-25 17:18:13",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "worldbank",
"github_project": "iQual",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "scikit-learn",
"specs": [
[
">=",
"1.1.0"
]
]
},
{
"name": "sentence-transformers",
"specs": []
},
{
"name": "umap-learn",
"specs": []
},
{
"name": "pandas",
"specs": []
},
{
"name": "numpy",
"specs": []
},
{
"name": "scipy",
"specs": []
},
{
"name": "simplejson",
"specs": []
},
{
"name": "statsmodels",
"specs": []
},
{
"name": "spacy",
"specs": []
},
{
"name": "matplotlib",
"specs": []
},
{
"name": "seaborn",
"specs": []
}
],
"lcname": "iqual"
}
JSON
