# General Class Balancer
This program finds a subset of your dataset with balanced confounding factors (also known as "data matching"), though it can work with any combination of categorical and continuous variables. Given a labeled dataset with any number of classes and a number of confounding factors for each datapoint, this matches data in each class to one another, such that the distributions of each confounding factor are the same in each class. This may be used to sample a training set on which a given deep learning model will not take confounding factors into account during its classification.
To install:
```
pip3 install general-class-balancer
```
To use with a Pandas dataframe:
~~~python
import pandas as pd
df = pd.read_pickle('dementia_diagnostic_records.pkl')
# Match the label "dementia" by age and sex
selection = class_balance(df,class_col="Dementia",confounds=["Age","Sex"],plim=0.1)
# selection is an array of booleans. This takes a matched subset of the dataframe
df = df[selection]
~~~
This will match patients in the column "Dementia" by age and sex. Note that continuous labels are not supported, only class labels.
A version of this method was originally introduced in https://arxiv.org/abs/2002.07874 to ensure that deep learning classifications of sex based on brain activity did not take into account head motion or intracranial volume, both of which are statistically different between sexes and which affect measurements of brain activity, but which we did not want the machine learning model to consider. A more detailed explanation of the method may be found in that paper. Below is a pictural description of the algorithm (assuming plim = 0.10)
Everything in the presented code uses numpy arrays. The code, as well as a script that simulates data from random variables, is given. Simply run
```
python random_example_test.py
```
Raw data
{
"_id": null,
"home_page": null,
"name": "general-class-balancer",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "Matt Leming <mleming@mgh.harvard.edu>",
"keywords": "biomedical, deep learning, ehr, machine learning, mri, pytorch",
"author": null,
"author_email": "Matt Leming <mleming@mgh.harvard.edu>",
"download_url": "https://files.pythonhosted.org/packages/19/3c/2ec3f87d7c016cb3f2f725c264897c4f778871531e9f82e69ae0274089df/general_class_balancer-0.0.7.tar.gz",
"platform": null,
"description": "# General Class Balancer\n\nThis program finds a subset of your dataset with balanced confounding factors (also known as \"data matching\"), though it can work with any combination of categorical and continuous variables. Given a labeled dataset with any number of classes and a number of confounding factors for each datapoint, this matches data in each class to one another, such that the distributions of each confounding factor are the same in each class. This may be used to sample a training set on which a given deep learning model will not take confounding factors into account during its classification.\n\nTo install:\n\n```\npip3 install general-class-balancer\n```\n\nTo use with a Pandas dataframe:\n\n~~~python\nimport pandas as pd\n\ndf = pd.read_pickle('dementia_diagnostic_records.pkl')\n\n# Match the label \"dementia\" by age and sex\nselection = class_balance(df,class_col=\"Dementia\",confounds=[\"Age\",\"Sex\"],plim=0.1)\n\n# selection is an array of booleans. This takes a matched subset of the dataframe\ndf = df[selection]\n~~~\n\nThis will match patients in the column \"Dementia\" by age and sex. Note that continuous labels are not supported, only class labels.\n\nA version of this method was originally introduced in https://arxiv.org/abs/2002.07874 to ensure that deep learning classifications of sex based on brain activity did not take into account head motion or intracranial volume, both of which are statistically different between sexes and which affect measurements of brain activity, but which we did not want the machine learning model to consider. A more detailed explanation of the method may be found in that paper. Below is a pictural description of the algorithm (assuming plim = 0.10)\n\n![alt text](https://github.com/mleming/general_class_balancer/blob/master/description.png \"A description of the general class balancer algorithm\")\n\nEverything in the presented code uses numpy arrays. The code, as well as a script that simulates data from random variables, is given. Simply run\n\n\n```\npython random_example_test.py\n```\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "Matches datapoints from an imbalanced class to a balanced one.",
"version": "0.0.7",
"project_urls": {
"Homepage": "https://github.com/mleming/general_class_balancer"
},
"split_keywords": [
"biomedical",
" deep learning",
" ehr",
" machine learning",
" mri",
" pytorch"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "006355ecee0409b573bdf0725be8617ebb3ea496ebaeeea01e080e9ec1141fb1",
"md5": "e1fac5a24fd853b8c470b8eaeef17539",
"sha256": "da8d321bbb02d93450b630645604dfee60f774965827f710dbf135a465195581"
},
"downloads": -1,
"filename": "general_class_balancer-0.0.7-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e1fac5a24fd853b8c470b8eaeef17539",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 7078,
"upload_time": "2024-12-16T18:40:36",
"upload_time_iso_8601": "2024-12-16T18:40:36.597908Z",
"url": "https://files.pythonhosted.org/packages/00/63/55ecee0409b573bdf0725be8617ebb3ea496ebaeeea01e080e9ec1141fb1/general_class_balancer-0.0.7-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "193c2ec3f87d7c016cb3f2f725c264897c4f778871531e9f82e69ae0274089df",
"md5": "937567f3673edc3c8ae2c7665a7c812e",
"sha256": "a7fc6cd88f7a63dfc32974c27b91fbea0b657c9ecd617956ea1d37f0ec5b0778"
},
"downloads": -1,
"filename": "general_class_balancer-0.0.7.tar.gz",
"has_sig": false,
"md5_digest": "937567f3673edc3c8ae2c7665a7c812e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 159418,
"upload_time": "2024-12-16T18:40:37",
"upload_time_iso_8601": "2024-12-16T18:40:37.824636Z",
"url": "https://files.pythonhosted.org/packages/19/3c/2ec3f87d7c016cb3f2f725c264897c4f778871531e9f82e69ae0274089df/general_class_balancer-0.0.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-16 18:40:37",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "mleming",
"github_project": "general_class_balancer",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "general-class-balancer"
}
JSON
