[](https://github.com/MilkaLichtblau/faim/actions/workflows/python-build-test.yaml)
[](https://opensource.org/licenses/Apache-2.0)
# FAIM
FAIM (FAir Interpolation Method), described in
[Beyond Incompatibility: Interpolation between Mutually
Exclusive Fairness Criteria in Classification Problems](https://arxiv.org/abs/2212.00469),
is a post-processing algorithm for achieving a combination of group-fairness criteria
(equalized false positive rates, equalized false negative rates, group calibration).
**This README.md is under construction!**
## Installation
### Environment
Ensure you have a environment with Python>=3.7 and pip>=2.21, preferably by creating a virtual environment.
One way to do this is using [miniconda](https://docs.conda.io/en/latest/miniconda.html). Install miniconda following
the instructions on [this page](https://docs.conda.io/en/latest/miniconda.html)
and create a python 3.10 environment:
```bash
conda create -n faim python=3.10
```
Activate the environment
```bash
conda activate faim
```
Check that versions of python are >=3.7 and >=2.21, respectively:
```bash
python --version
pip --version
```
### Python Package
If you intend to develop the package and/or contribute, follow the install instructions in the
[Development Environment](#development-environment) section below instead. Otherwise, follow these instructions.
The package and experiment CLI can be installed with pip:
```bash
pip install "faim[experiment]"
```
Note the `[experiment]` notation is required for now since, for the moment, the algorithm can only be run in experiment
mode for recreating experimental results in the [paper](https://arxiv.org/abs/2212.00469).
In the future, `faim` will be made available for post-processing classifier scores
(given ground truth and group information), going beyond reproducing paper experiments.
### Removal
From the environment where you installed the package, run
```bash
pip uninstall faim
```
## Usage
Installing faim also (currently) installs one command line interface (CLI) tool, `faim-experiment` which can be
used to reproduce the work in the paper.
[A general API will be added soon]
### Experiments
This section contains information for reproducing experiments in our [paper](https://arxiv.org/abs/2212.00469).
Ensure the package has been installed with `[experiment]` extra requirements before continuing
(see [Installation | Python Package](#python-package))!
#### Prepare Data
The CLI can be used to prepare any of the three datasets used in the [paper](https://arxiv.org/abs/2212.00469):
```bash
faim-experiment --prepare-data DATASET
```
where `DATASET` is one of:
* `synthetic-from-paper`
* `compas`
* `zalando` [waiting for permission to release, contact us for more information]
The dataset will be downloaded, and prepared to a folder called `prepared-data`.
The following sections include info about each dataset:
###### Synthetic data
The raw dataset in the GitHub repo corresponds to synthetic prediction and ground truth scores for two groups,
for each group sampling from a corresponding binormal distribution.
###### COMPAS data
The raw data was obtained from [ProPublica's COMPAS Analysis repository](https://github.com/propublica/compas-analysis).
###### Zalando data
**Under construction, more information to follow!**
#### Run Experiment
Having prepared data following the instruction above, you are ready to run a FAIM experiment:
```bash
faim-experiment --run PREPARED-DATASET LOW_SCORE_VAL,HIGH_SCORE_VAL THETAS PREPARED_DATA_FILEPATH
```
`PREPARED-DATASET` is now one of the following options (depending on what has been prepared):
* `syntheticTwoGroups` (prepared using `--prepare-data synthetic`)
* `compasGender` (prepared using `--prepare-data compas`)
* `compasRace` (prepared using `--prepare-data compas`)
* `compasAge` (prepared using `--prepare-data compas`)
* `zalando` (prepared using `--prepare-data zalando`) [waiting for permission to release, contact us for more information]
`LOW_SCORE_VAL,HIGH_SCORE_VAL` are two numbers that define the score range.
`THETAS` correspond to the fairness compromise you want. There are three thetas per group corresponding to the
desired amount of the three fairness criteria that the system should achieve:
1. group calibration
2. equalized false negative rates
3. equalized false positive rates
Note, as discussed in the paper, thetas = 1,1,1 does not indicate that the system will simultaneously achieve all
three (mutually incompatible) fairness criteria, but rather the result will be a compromise between all three.
See the [paper](https://arxiv.org/abs/2212.00469) for more details.
Finally, `PREPARED_DATA_FILEPATH` corresponds to the filepath of the prepared data.
###### Examples
Run all of the following from the same folder where `faim-experiment --prepare-data` was run.
In each example, a FAIM post-processor is trained and evaluated with results saved under the `results` folder:
* Train FAIM model on synthetic dataset with callibration as fairness correction
```bash
faim-experiment --run syntheticTwoGroups 0.1 1,0,0,1,0,0 prepared-data/synthetic/2groups/2022-01-12/dataset.csv
```
* Train FAIM model on synthetic dataset to achieve a combination of all three fairness criteria.
```bash
faim-experiment --run syntheticTwoGroups 0.1 1,1,1,1,1,1 prepared-data/synthetic/2groups/2022-01-12/dataset.csv
```
#### Visualize Results
**Needs documentation!**
### Development Environment
To develop and/or contribute, clone the repository
```bash
git clone <this repo URL>
```
From the root directory of the git repository, install the package with pip in editable mode (`-e`)
with extra requirements for experiments (experiment) and development (dev):
```bash
pip install -e ".[experiment,dev]"
```
Don't confuse the `[]` to mean optional. The `".[experiment, dev]"` notation tells pip to install extra
"experiment" and "dev" requirements including things like `pytest` and `pre-commit`.
When contributing, be sure to install (and use) our [pre-commit](https://pre-commit.com/) hooks:
```bash
pre-commit install -t pre-commit -t pre-push
```
Raw data
{
"_id": null,
"home_page": "",
"name": "faim",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "fairness,optimal transport",
"author": "Meike Zehlike, Alex Loosley",
"author_email": "meike.zehlike@zalando.de, alex.loosley@zalando.de",
"download_url": "https://files.pythonhosted.org/packages/64/59/dffb5559d170cdd70cac7a30e1fbdf6175c4ed5435d086785fb96d4d0efc/faim-0.0.3.tar.gz",
"platform": null,
"description": "[](https://github.com/MilkaLichtblau/faim/actions/workflows/python-build-test.yaml)\n[](https://opensource.org/licenses/Apache-2.0)\n\n# FAIM\n\nFAIM (FAir Interpolation Method), described in\n[Beyond Incompatibility: Interpolation between Mutually\nExclusive Fairness Criteria in Classification Problems](https://arxiv.org/abs/2212.00469),\nis a post-processing algorithm for achieving a combination of group-fairness criteria\n(equalized false positive rates, equalized false negative rates, group calibration).\n\n**This README.md is under construction!**\n\n## Installation\n\n### Environment\nEnsure you have a environment with Python>=3.7 and pip>=2.21, preferably by creating a virtual environment.\n\nOne way to do this is using [miniconda](https://docs.conda.io/en/latest/miniconda.html). Install miniconda following\nthe instructions on [this page](https://docs.conda.io/en/latest/miniconda.html)\nand create a python 3.10 environment:\n\n```bash\nconda create -n faim python=3.10\n```\n\nActivate the environment\n```bash\nconda activate faim\n```\n\nCheck that versions of python are >=3.7 and >=2.21, respectively:\n```bash\npython --version\npip --version\n```\n\n### Python Package\nIf you intend to develop the package and/or contribute, follow the install instructions in the\n[Development Environment](#development-environment) section below instead. Otherwise, follow these instructions.\n\nThe package and experiment CLI can be installed with pip:\n```bash\npip install \"faim[experiment]\"\n```\n\nNote the `[experiment]` notation is required for now since, for the moment, the algorithm can only be run in experiment\nmode for recreating experimental results in the [paper](https://arxiv.org/abs/2212.00469).\nIn the future, `faim` will be made available for post-processing classifier scores\n(given ground truth and group information), going beyond reproducing paper experiments.\n\n\n\n### Removal\nFrom the environment where you installed the package, run\n```bash\npip uninstall faim\n```\n\n## Usage\nInstalling faim also (currently) installs one command line interface (CLI) tool, `faim-experiment` which can be\nused to reproduce the work in the paper.\n\n[A general API will be added soon]\n\n### Experiments\nThis section contains information for reproducing experiments in our [paper](https://arxiv.org/abs/2212.00469).\n\nEnsure the package has been installed with `[experiment]` extra requirements before continuing\n(see [Installation | Python Package](#python-package))!\n\n#### Prepare Data\nThe CLI can be used to prepare any of the three datasets used in the [paper](https://arxiv.org/abs/2212.00469):\n```bash\nfaim-experiment --prepare-data DATASET\n```\nwhere `DATASET` is one of:\n* `synthetic-from-paper`\n* `compas`\n* `zalando` [waiting for permission to release, contact us for more information]\n\nThe dataset will be downloaded, and prepared to a folder called `prepared-data`.\n\nThe following sections include info about each dataset:\n\n###### Synthetic data\nThe raw dataset in the GitHub repo corresponds to synthetic prediction and ground truth scores for two groups,\nfor each group sampling from a corresponding binormal distribution.\n\n###### COMPAS data\nThe raw data was obtained from [ProPublica's COMPAS Analysis repository](https://github.com/propublica/compas-analysis).\n\n###### Zalando data\n**Under construction, more information to follow!**\n\n#### Run Experiment\n\nHaving prepared data following the instruction above, you are ready to run a FAIM experiment:\n```bash\nfaim-experiment --run PREPARED-DATASET LOW_SCORE_VAL,HIGH_SCORE_VAL THETAS PREPARED_DATA_FILEPATH\n```\n\n`PREPARED-DATASET` is now one of the following options (depending on what has been prepared):\n* `syntheticTwoGroups` (prepared using `--prepare-data synthetic`)\n* `compasGender` (prepared using `--prepare-data compas`)\n* `compasRace` (prepared using `--prepare-data compas`)\n* `compasAge` (prepared using `--prepare-data compas`)\n* `zalando` (prepared using `--prepare-data zalando`) [waiting for permission to release, contact us for more information]\n\n`LOW_SCORE_VAL,HIGH_SCORE_VAL` are two numbers that define the score range.\n\n`THETAS` correspond to the fairness compromise you want. There are three thetas per group corresponding to the\ndesired amount of the three fairness criteria that the system should achieve:\n1. group calibration\n2. equalized false negative rates\n3. equalized false positive rates\n\nNote, as discussed in the paper, thetas = 1,1,1 does not indicate that the system will simultaneously achieve all\nthree (mutually incompatible) fairness criteria, but rather the result will be a compromise between all three.\n\nSee the [paper](https://arxiv.org/abs/2212.00469) for more details.\n\nFinally, `PREPARED_DATA_FILEPATH` corresponds to the filepath of the prepared data.\n\n###### Examples\nRun all of the following from the same folder where `faim-experiment --prepare-data` was run.\n\nIn each example, a FAIM post-processor is trained and evaluated with results saved under the `results` folder:\n* Train FAIM model on synthetic dataset with callibration as fairness correction\n ```bash\n faim-experiment --run syntheticTwoGroups 0.1 1,0,0,1,0,0 prepared-data/synthetic/2groups/2022-01-12/dataset.csv\n ```\n* Train FAIM model on synthetic dataset to achieve a combination of all three fairness criteria.\n ```bash\n faim-experiment --run syntheticTwoGroups 0.1 1,1,1,1,1,1 prepared-data/synthetic/2groups/2022-01-12/dataset.csv\n ```\n\n#### Visualize Results\n**Needs documentation!**\n\n### Development Environment\nTo develop and/or contribute, clone the repository\n```bash\ngit clone <this repo URL>\n```\n\nFrom the root directory of the git repository, install the package with pip in editable mode (`-e`)\nwith extra requirements for experiments (experiment) and development (dev):\n```bash\npip install -e \".[experiment,dev]\"\n```\n\nDon't confuse the `[]` to mean optional. The `\".[experiment, dev]\"` notation tells pip to install extra\n\"experiment\" and \"dev\" requirements including things like `pytest` and `pre-commit`.\n\nWhen contributing, be sure to install (and use) our [pre-commit](https://pre-commit.com/) hooks:\n```bash\npre-commit install -t pre-commit -t pre-push\n```\n",
"bugtrack_url": null,
"license": "Apache License Version 2.0",
"summary": "FAIM (FAir Interpolation Method), described in \"Beyond Incompatibility: Interpolation between Mutually",
"version": "0.0.3",
"split_keywords": [
"fairness",
"optimal transport"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "4346a5414b37891109a115372c607bd1cfd874da77c2d9ae3d0eb24cb9e37800",
"md5": "047acccad17069e26bbcc17be294cd8d",
"sha256": "1e17f2bb4087a72fbd6317639fff775c06dcd55aa0732aa8bc86ea7a87bbfb65"
},
"downloads": -1,
"filename": "faim-0.0.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "047acccad17069e26bbcc17be294cd8d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 33298,
"upload_time": "2023-01-04T22:27:21",
"upload_time_iso_8601": "2023-01-04T22:27:21.113898Z",
"url": "https://files.pythonhosted.org/packages/43/46/a5414b37891109a115372c607bd1cfd874da77c2d9ae3d0eb24cb9e37800/faim-0.0.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "6459dffb5559d170cdd70cac7a30e1fbdf6175c4ed5435d086785fb96d4d0efc",
"md5": "30a70abd778d31efc5d8ccdf90dc8046",
"sha256": "fda5e15aabfb152711f52cfdb4b0ccd22bdc5946725ac1a623b3998a368ee427"
},
"downloads": -1,
"filename": "faim-0.0.3.tar.gz",
"has_sig": false,
"md5_digest": "30a70abd778d31efc5d8ccdf90dc8046",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 29696,
"upload_time": "2023-01-04T22:27:22",
"upload_time_iso_8601": "2023-01-04T22:27:22.923804Z",
"url": "https://files.pythonhosted.org/packages/64/59/dffb5559d170cdd70cac7a30e1fbdf6175c4ed5435d086785fb96d4d0efc/faim-0.0.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-01-04 22:27:22",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "faim"
}
JSON