# qualtrics-utils
Utilities for qualtrics surveys. Get and sync survey responses, generate codebooks, & c.
## Quickstart 🚀
This project requires Python `^3.10` to run.
### via [`poetry`](https://python-poetry.org/docs/)
Install poetry, then run
> poetry install
And you're done.
## [`surveys`](qualtrics_utils/surveys.py)
### Exporting
Example (get a survey's responses, convert to a pandas DataFrame):
```python
from qualtrics_utils import Surveys
surveys = Surveys(api_token=QUALTRICS_API_TOKEN)
exported_file = surveys.get_responses_df(
survey_id=SURVEY_ID
)
df = exported_file.data
```
Survey's can be exported to a variety of formats, including:
- `.csv`
- `.xlsx`
- `.json`
And with a variety of parameters. Please see the `ExportCreationRequest` documentation [herein](./qualtrics_utils/surveys_response_import_export_api_client/models/export_creation_request.py)
## [`sync`](qualtrics_utils/sync.py)
Perhaps one of the more useful features hereof is the ability to sync survey responses to the following services:
- Google Sheets
- MySQL
Future services will include:
- PostgreSQL
- MongoDB
- AWS S3
Syncing can either be leveraged as a standard python module, or as a CLI tool.
### CLI
Execute the `help` command to see the available options:
```bash
python -m qualtrics_utils.sync --help
```
See also the [config.example.toml](config.example.toml) for an example configuration file.
### Module
Simply import `sync_*` from the `qualtrics_utils.sync` module, and execute the function with the appropriate arguments.
### Syncing information
The process is fairly straightforward:
1. Ensure that the service has two "tables", or datastores for:
- Survey responses: This defaults to the input base name (defaults to the survey ID) + `_responses`
- Survey export statuses: This defaults to the input base name (defaults to the survey ID) + `_status`
2. Export the survey responses to the service
3. Update the survey export statuses to reflect the export
This will allow for a sync to pick up where it left off, only exporting newly found responses. Please note, if a first time sync contains enough survey responses to exceed the service's limits (~1.8 GB), the sync will fail. Please see the [Qualtrics documentation](https://api.qualtrics.com/docs/response-exports) for more information.
For example, in google sheets:
```python
from qualtrics_utils import Surveys, sync_sheets
import pandas as pd
# Survey ID or URL
survey_id = ...
# Sheet URL
responses_url = ...
# dict of extra survey arguments for the export creation request
survey_args = ...
sheets = Sheets()
def post_processing_func(df: pd.DataFrame) -> pd.DataFrame:
# Do some post processing of the responses before syncing here
return df
sync_sheets(
survey_id=survey_id,
surveys=surveys,
response_post_processing_func=post_processing_func,
sheet_name=table_name,
sheet_url=responses_url,
sheets=sheets,
**survey_args,
)
```
## Codebook mapping
### [`generate.py`](qualtrics_utils/codebook/generate_codebook.py)
Takes the exported `.qsf` file from Qualtrics and generates a codebook mapping question
IDs to question text and answer choices. The output is a JSON file containing a list of
dictionaries.
Example row:
```json
{
"question_number": "Q5.10",
"question_string": "What is your role at this school?",
"answer_choices": "..."
},
```
### [`map_columns.py`](qualtrics_utils/codebook/map_codebook_columns.py)
Takes a codebook mapping (generated by the above function) and creates conditional
statements to map the question columns into valid Tableau or SQL code. Used to create a
singular question column in the above formats when there are multiple questions in a
single question block (e.g. multiple Likert scale questions).
## OpenAPI client
To handle the majority of the qualtrics API calls, we use the publicly available OpenAPI spec, found on the qualtrics API docs [website]("https://stoplight.io/api/v1/projects/qualtricsv2/publicapidocs/nodes/reference/")
The OpenAPI spec is fed to [openapi-python-client](https://github.com/openapi-generators/openapi-python-client), where it's used to generate a python client(s) for the qualtrics API. These clients (for each of the utilized APIs) come bundled and pre-generated as-is, but to create them anew, one can execute the [create_api_client.py](create_api_client.py) script. This will re-generate the clients and place them in the [qualtrics_utils](qualtrics_utils) directory. Occasionally, the OpenAPI spec is broken and mis-validated from Qualtrics; regenerate at your own risk.
Raw data
{
"_id": null,
"home_page": "https://github.com/mkbabb/qualtrics-utils",
"name": "qualtrics-utils",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.12,<4.0",
"maintainer_email": "",
"keywords": "qualtrics,qualtrics-api,qualtrics-util",
"author": "Mike Babb",
"author_email": "mike7400@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/db/6b/4e1f20881cfccf7b033441d3038adbf0df9b367e640040af9f88636d6764/qualtrics_utils-1.0.4.tar.gz",
"platform": null,
"description": "# qualtrics-utils\n\nUtilities for qualtrics surveys. Get and sync survey responses, generate codebooks, & c.\n\n## Quickstart \ud83d\ude80\n\nThis project requires Python `^3.10` to run.\n\n### via [`poetry`](https://python-poetry.org/docs/)\n\nInstall poetry, then run\n\n> poetry install\n\nAnd you're done.\n\n## [`surveys`](qualtrics_utils/surveys.py)\n\n### Exporting\n\nExample (get a survey's responses, convert to a pandas DataFrame):\n\n```python\nfrom qualtrics_utils import Surveys\n\nsurveys = Surveys(api_token=QUALTRICS_API_TOKEN)\n\nexported_file = surveys.get_responses_df(\n survey_id=SURVEY_ID\n)\ndf = exported_file.data\n```\n\nSurvey's can be exported to a variety of formats, including:\n\n- `.csv`\n- `.xlsx`\n- `.json`\n\nAnd with a variety of parameters. Please see the `ExportCreationRequest` documentation [herein](./qualtrics_utils/surveys_response_import_export_api_client/models/export_creation_request.py)\n\n## [`sync`](qualtrics_utils/sync.py)\n\nPerhaps one of the more useful features hereof is the ability to sync survey responses to the following services:\n\n- Google Sheets\n- MySQL\n\nFuture services will include:\n\n- PostgreSQL\n- MongoDB\n- AWS S3\n\nSyncing can either be leveraged as a standard python module, or as a CLI tool.\n\n### CLI\n\nExecute the `help` command to see the available options:\n\n```bash\npython -m qualtrics_utils.sync --help\n```\n\nSee also the [config.example.toml](config.example.toml) for an example configuration file.\n\n### Module\n\nSimply import `sync_*` from the `qualtrics_utils.sync` module, and execute the function with the appropriate arguments.\n\n### Syncing information\n\nThe process is fairly straightforward:\n\n1. Ensure that the service has two \"tables\", or datastores for:\n - Survey responses: This defaults to the input base name (defaults to the survey ID) + `_responses`\n - Survey export statuses: This defaults to the input base name (defaults to the survey ID) + `_status`\n2. Export the survey responses to the service\n3. Update the survey export statuses to reflect the export\n\nThis will allow for a sync to pick up where it left off, only exporting newly found responses. Please note, if a first time sync contains enough survey responses to exceed the service's limits (~1.8 GB), the sync will fail. Please see the [Qualtrics documentation](https://api.qualtrics.com/docs/response-exports) for more information.\n\nFor example, in google sheets:\n\n```python\nfrom qualtrics_utils import Surveys, sync_sheets\nimport pandas as pd\n\n# Survey ID or URL\nsurvey_id = ...\n# Sheet URL\nresponses_url = ...\n\n# dict of extra survey arguments for the export creation request\nsurvey_args = ...\n\nsheets = Sheets()\n\ndef post_processing_func(df: pd.DataFrame) -> pd.DataFrame:\n # Do some post processing of the responses before syncing here\n return df\n\nsync_sheets(\n survey_id=survey_id,\n surveys=surveys,\n response_post_processing_func=post_processing_func,\n sheet_name=table_name,\n sheet_url=responses_url,\n sheets=sheets,\n **survey_args,\n)\n\n```\n\n## Codebook mapping\n\n### [`generate.py`](qualtrics_utils/codebook/generate_codebook.py)\n\nTakes the exported `.qsf` file from Qualtrics and generates a codebook mapping question\nIDs to question text and answer choices. The output is a JSON file containing a list of\ndictionaries.\n\nExample row:\n\n```json\n{\n \"question_number\": \"Q5.10\",\n \"question_string\": \"What is your role at this school?\",\n \"answer_choices\": \"...\"\n},\n```\n\n### [`map_columns.py`](qualtrics_utils/codebook/map_codebook_columns.py)\n\nTakes a codebook mapping (generated by the above function) and creates conditional\nstatements to map the question columns into valid Tableau or SQL code. Used to create a\nsingular question column in the above formats when there are multiple questions in a\nsingle question block (e.g. multiple Likert scale questions).\n\n## OpenAPI client\n\nTo handle the majority of the qualtrics API calls, we use the publicly available OpenAPI spec, found on the qualtrics API docs [website](\"https://stoplight.io/api/v1/projects/qualtricsv2/publicapidocs/nodes/reference/\")\n\nThe OpenAPI spec is fed to [openapi-python-client](https://github.com/openapi-generators/openapi-python-client), where it's used to generate a python client(s) for the qualtrics API. These clients (for each of the utilized APIs) come bundled and pre-generated as-is, but to create them anew, one can execute the [create_api_client.py](create_api_client.py) script. This will re-generate the clients and place them in the [qualtrics_utils](qualtrics_utils) directory. Occasionally, the OpenAPI spec is broken and mis-validated from Qualtrics; regenerate at your own risk.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Utilities for qualtrics surveys.",
"version": "1.0.4",
"project_urls": {
"Homepage": "https://github.com/mkbabb/qualtrics-utils",
"Repository": "https://github.com/mkbabb/qualtrics-utils"
},
"split_keywords": [
"qualtrics",
"qualtrics-api",
"qualtrics-util"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "25d52babdd1880593f37e76075e1a78a4a88ac7fe7407317d40d28dce6bee971",
"md5": "8e36a383cd7c38e2f2526473ebae5f9d",
"sha256": "24701dd2d2e0aa365c0a6c8378f2b6cdccd24dbc0040262770beb10a6fd7faa7"
},
"downloads": -1,
"filename": "qualtrics_utils-1.0.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8e36a383cd7c38e2f2526473ebae5f9d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.12,<4.0",
"size": 128657,
"upload_time": "2024-03-14T06:46:56",
"upload_time_iso_8601": "2024-03-14T06:46:56.358929Z",
"url": "https://files.pythonhosted.org/packages/25/d5/2babdd1880593f37e76075e1a78a4a88ac7fe7407317d40d28dce6bee971/qualtrics_utils-1.0.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "db6b4e1f20881cfccf7b033441d3038adbf0df9b367e640040af9f88636d6764",
"md5": "9924c16317b2a5180f7dd4b92131d139",
"sha256": "3aa1534407eeda938f3cec7a88ecb7797d740944a231a999a20531709da93ee4"
},
"downloads": -1,
"filename": "qualtrics_utils-1.0.4.tar.gz",
"has_sig": false,
"md5_digest": "9924c16317b2a5180f7dd4b92131d139",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.12,<4.0",
"size": 61072,
"upload_time": "2024-03-14T06:46:58",
"upload_time_iso_8601": "2024-03-14T06:46:58.086896Z",
"url": "https://files.pythonhosted.org/packages/db/6b/4e1f20881cfccf7b033441d3038adbf0df9b367e640040af9f88636d6764/qualtrics_utils-1.0.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-14 06:46:58",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "mkbabb",
"github_project": "qualtrics-utils",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "qualtrics-utils"
}
JSON
