# PassivePy: A Tool to Automatically Identify Passive Voice in Big Text Data
**_This repository is the code of the following paper, so if you use this package, please cite the work as:
Sepehri, A., Mirshafiee, M. S., & Markowitz, D. M. (2022). PassivePy: A tool to automatically identify passive voice in big text data. Journal of Consumer Psychology. [Preprint available](https://doi.org/10.1002/jcpy.1377)**
Our aim with this work is to create a reliable (e.g., passive voice judgments are consistent), valid (e.g., passive voice judgments are accurate), flexible (e.g., texts can be assessed at different units of analysis), replicable (e.g., the approach can be performed by a range of research teams with varying levels of computational expertise), and scalable way (e.g., small and large collections of texts can be analyzed) to capture passive voice from different corpora for social and psychological evaluations of text. To achieve these aims, we introduce PassivePy, a fully transparent and documented Python library.
For accessing the datasets in our paper, please click on [this link](https://osf.io/j2b6u/?view_only=0e78d7f4028041b693d6b64547b514ca).
If you haven't used Python before or need detailed instructions about how to use this package please visit [our website](https://passivepy.streamlit.app/).
First we have to install the requirements in the following way (all requirements are needed for spaCy or other libraries we use.):
```
!pip install -r https://raw.githubusercontent.com/mitramir55/PassivePy/main/PassivePyCode/PassivePySrc/requirements_lg.txt
!pip install PassivePy==0.2.23
```
Then, import PassivePy and initiate the analyzer:
```
from PassivePySrc import PassivePy
passivepy = PassivePy.PassivePyAnalyzer(spacy_model = "en_core_web_lg")
```
Use passivepy for single sentences:
```
# Try changing the sentence below:
sample_text = "The painting has been drawn."
passivepy.match_text(sample_text, full_passive=True, truncated_passive=True)
```
The output will be:
```
sentence : the input sentence
binary : Whether any passive voice is detected
passive_match(es) : The span of passive form in text
raw_passive_count : Number of passive voices
```
You can set the full_passive or truncated_passive to true if you want the same sort of output for these two types of passive. (truncated is a passive without an object of preposition, while a full passive is one with the object of preposition - e.g., this was broken by him.)
For processing datasets, we have can either analyze the records sentence- or corpus-level. Your dataset can be in any format (e.g., CSV, XLSX or XLS).; however, make sure to that it has at least one column with the text that needs to be analyzed.
In case of large datasets, you can also add `batch_size = ...` and `n_process=...` to speed up the analysis (the default for both is 1).
```
# sentence level:
df_detected_s = passivepy.match_sentence_level(df, column_name='documents', n_process = 1,
batch_size = 1000, add_other_columns=True,
truncated_passive=False, full_passive=False)
# corpus level
df_detected_c = passivepy.match_corpus_level(df, column_name='sentences', n_process = 1,
batch_size = 1000, add_other_columns=True,
truncated_passive=False, full_passive=False)
```
In the output you will have a data frame with the following columns:
```
# corpus level
document : Records in the input data frame
binary : Whether a passive was detected in that document
passive_match(es) : Parts of the document detected as passive
raw_passive_count : Number of passive voices detected in the sentence
raw_passive_sents_count : Number of sentences with passive voice
raw_sentence_count : Number of sentences detected in the document
passive_sents_percentage : Proportion of passive sentences to total number of sentences
# Sentence level
docId : Initial index of the record in the input file
sentenceId : The ith sentence in one specific record
sentence : The detected sentence
binary : Whether a passive was detected in that sentence
passive_match(es) : The part of the record detected as passive voice
raw_passive_count : Number of passive forms detected in the sentence
```
If you needed to analyze each token of a sentence, i.g., print out the `DEP` (dependency), `POS` (coarse-grained part of speech tags), `TAG` (fine-grained part of speech tags), `LEMMA` (canonical form) of a word, you can use the `parse_sentence` method of passivepy in the following way:
```
sample_sentence = "She has been killed"
passivepy.parse_sentence(sample_sentence)
```
And the output will be like the sample below:
```
word: She
pos: PRON
dependency: nsubjpass
tag: PRP
lemma: she
...
```
If you do not need any columns to be appended to the main dataset, simply add `add_other_columns = False`, or if you don't what the percentages to show up add `percentage_of_passive_sentences = False` in any of the following functions.
Accuracy on the CoLA dataset: 0.97
Accuracy on the CrowdSource Dataset: 0.98
Raw data
{
"_id": null,
"home_page": "https://github.com/mitramir55/PassivePy",
"name": "PassivePy",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "passive voice,text analysis,spacy,dependency parsing,part of speech tagging",
"author": "Mitra Mirshafiee",
"author_email": "mitra.mirshafiee@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/09/71/aa6fb625e041c9af319f59c790ca0bcaee3beda206772806c169cb27e4d0/PassivePy-0.2.23.tar.gz",
"platform": null,
"description": "# PassivePy: A Tool to Automatically Identify Passive Voice in Big Text Data\r\n\r\n**_This repository is the code of the following paper, so if you use this package, please cite the work as:\r\nSepehri, A., Mirshafiee, M. S., & Markowitz, D. M. (2022). PassivePy: A tool to automatically identify passive voice in big text data. Journal of Consumer Psychology. [Preprint available](https://doi.org/10.1002/jcpy.1377)**\r\n\r\n\r\nOur aim with this work is to create a reliable (e.g., passive voice judgments are consistent), valid (e.g., passive voice judgments are accurate), flexible (e.g., texts can be assessed at different units of analysis), replicable (e.g., the approach can be performed by a range of research teams with varying levels of computational expertise), and scalable way (e.g., small and large collections of texts can be analyzed) to capture passive voice from different corpora for social and psychological evaluations of text. To achieve these aims, we introduce PassivePy, a fully transparent and documented Python library.\r\n\r\nFor accessing the datasets in our paper, please click on [this link](https://osf.io/j2b6u/?view_only=0e78d7f4028041b693d6b64547b514ca). \r\n\r\nIf you haven't used Python before or need detailed instructions about how to use this package please visit [our website](https://passivepy.streamlit.app/).\r\n\r\n\r\nFirst we have to install the requirements in the following way (all requirements are needed for spaCy or other libraries we use.):\r\n```\r\n!pip install -r https://raw.githubusercontent.com/mitramir55/PassivePy/main/PassivePyCode/PassivePySrc/requirements_lg.txt\r\n!pip install PassivePy==0.2.23\r\n\r\n```\r\nThen, import PassivePy and initiate the analyzer:\r\n\r\n```\r\nfrom PassivePySrc import PassivePy\r\n\r\npassivepy = PassivePy.PassivePyAnalyzer(spacy_model = \"en_core_web_lg\")\r\n```\r\nUse passivepy for single sentences:\r\n```\r\n# Try changing the sentence below:\r\nsample_text = \"The painting has been drawn.\"\r\npassivepy.match_text(sample_text, full_passive=True, truncated_passive=True)\r\n```\r\nThe output will be:\r\n```\r\nsentence : the input sentence\r\nbinary : Whether any passive voice is detected \r\npassive_match(es) : The span of passive form in text\r\nraw_passive_count : Number of passive voices\r\n```\r\nYou can set the full_passive or truncated_passive to true if you want the same sort of output for these two types of passive. (truncated is a passive without an object of preposition, while a full passive is one with the object of preposition - e.g., this was broken by him.)\r\n\r\n\r\nFor processing datasets, we have can either analyze the records sentence- or corpus-level. Your dataset can be in any format (e.g., CSV, XLSX or XLS).; however, make sure to that it has at least one column with the text that needs to be analyzed.\r\n\r\nIn case of large datasets, you can also add `batch_size = ...` and `n_process=...` to speed up the analysis (the default for both is 1).\r\n\r\n\r\n``` \r\n# sentence level:\r\ndf_detected_s = passivepy.match_sentence_level(df, column_name='documents', n_process = 1,\r\n batch_size = 1000, add_other_columns=True,\r\n truncated_passive=False, full_passive=False)\r\n\r\n# corpus level\r\ndf_detected_c = passivepy.match_corpus_level(df, column_name='sentences', n_process = 1,\r\n batch_size = 1000, add_other_columns=True,\r\n truncated_passive=False, full_passive=False)\r\n```\r\nIn the output you will have a data frame with the following columns:\r\n\r\n```\r\n# corpus level\r\ndocument : Records in the input data frame\r\nbinary : Whether a passive was detected in that document\r\npassive_match(es) : Parts of the document detected as passive\r\nraw_passive_count : Number of passive voices detected in the sentence\r\nraw_passive_sents_count : Number of sentences with passive voice\r\nraw_sentence_count : Number of sentences detected in the document\r\npassive_sents_percentage : Proportion of passive sentences to total number of sentences\r\n\r\n# Sentence level\r\ndocId : Initial index of the record in the input file\r\nsentenceId : The ith sentence in one specific record\r\nsentence : The detected sentence\r\nbinary : Whether a passive was detected in that sentence\r\npassive_match(es) : The part of the record detected as passive voice\r\nraw_passive_count : Number of passive forms detected in the sentence\r\n\r\n```\r\n\r\nIf you needed to analyze each token of a sentence, i.g., print out the `DEP` (dependency), `POS` (coarse-grained part of speech tags), `TAG` (fine-grained part of speech tags), `LEMMA` (canonical form) of a word, you can use the `parse_sentence` method of passivepy in the following way:\r\n\r\n```\r\nsample_sentence = \"She has been killed\"\r\npassivepy.parse_sentence(sample_sentence)\r\n```\r\nAnd the output will be like the sample below:\r\n```\r\nword: She \r\npos: PRON \r\ndependency: nsubjpass \r\ntag: PRP \r\nlemma: she\r\n...\r\n```\r\n\r\n\r\n\r\nIf you do not need any columns to be appended to the main dataset, simply add `add_other_columns = False`, or if you don't what the percentages to show up add `percentage_of_passive_sentences = False` in any of the following functions.\r\n\r\n\r\nAccuracy on the CoLA dataset: 0.97\r\nAccuracy on the CrowdSource Dataset: 0.98\r\n\r\n\r\n\r\n\r\n\r\n",
"bugtrack_url": null,
"license": "",
"summary": "A package for processing large corpora and detecting passive voice.",
"version": "0.2.23",
"project_urls": {
"Bug Tracker": "https://github.com/pypa/sampleproject/issues",
"Homepage": "https://github.com/mitramir55/PassivePy"
},
"split_keywords": [
"passive voice",
"text analysis",
"spacy",
"dependency parsing",
"part of speech tagging"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0971aa6fb625e041c9af319f59c790ca0bcaee3beda206772806c169cb27e4d0",
"md5": "0d37b41a2fcb0986ae89f93b1b8dec86",
"sha256": "3fae03a5055d7a0f7b992648d6a167c7425dddc1e92d7a82937f4581e3a5c88d"
},
"downloads": -1,
"filename": "PassivePy-0.2.23.tar.gz",
"has_sig": false,
"md5_digest": "0d37b41a2fcb0986ae89f93b1b8dec86",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 12900,
"upload_time": "2023-07-19T15:34:43",
"upload_time_iso_8601": "2023-07-19T15:34:43.848201Z",
"url": "https://files.pythonhosted.org/packages/09/71/aa6fb625e041c9af319f59c790ca0bcaee3beda206772806c169cb27e4d0/PassivePy-0.2.23.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-19 15:34:43",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "mitramir55",
"github_project": "PassivePy",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "passivepy"
}
JSON
