# Finspector
Finspector is designed to help users explore the fairness and bias of foundation models using interactive visualizations.
## Installation
For a development installation (requires [Node.js](https://nodejs.org) and [Yarn version 1](https://classic.yarnpkg.com/)):
$ git clone https://github.com/IBM/finspector.git
$ cd finspector
$ pip install -r requirements.txt
$ pip install -e .
$ npm install
$ yarn install
$ jupyter labextension develop finspector --overwrite
$ cd js
$ yarn run build
$ cd ..
## Run Jupyter Example
To run the example, you need a dataset stored in `finspector/data/xyz.csv]`. The following table illustrates the structure of a dataset that Finspector expects.
bias_type | sent_index | para_index | stereo_type | more_or_less | <div style="width:200px">sent</div> | bert | roberta | albert
--- | --- | --- | --- | --- | --- | --- | --- | ---
socioeconomic | 485 | 0 | 1 | sent_more | May the Force be with you. | -2.5 | -1.5 | -2.22
socioeconomic | 485 | 1 | 1 | sent_more | May the Power be with you. | -2.0 | -1.4 | -2.87
- `bias_type`: bias category (e.g., race, sex, socioeconomic)
- `sent_index`: index of root sentence
- `para_index`: index of parphrased sentence
- `stereo_type`: (0: less stereo type, 1: stereotype)
- `more_or_less`: (sent_less: less stereo type, sent_more: stereotype)
- `sent`: sentence in string
- `bert`, `roberta`, `alberta`: Pseudo-log-likelihood scores for `sent` generated by each model
Once your dataset is ready, you can run
$ jupyter lab
Then, click on `acl2023demo.ipynb` on your Jupyter lab. The repository includes a small dataset `data/sample-movie-quotes.csv` for demonstration purpose.
## Citation
If you use Finspector for your research, you can use this citation format:
```
@inproceedings{kwon-etal-2023-finspector,
title = "{Finspector}: A Human-Centered Visual Inspection Tool for Exploring and Comparing Biases among Foundation Models",
author = "Kwon, Bum Chul and Mihindukulasooriya, Nandana",
booktitle = "Proceedings of the 61th Annual Meeting of the Association for Computational Linguistics: System Demonstrations",
year = "2023",
address = "Toronto, Canada",
publisher = "Association for Computational Linguistics"
}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/IBM/finspector",
"name": "finspector",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "ipython,widgets,Jupyter,JupyterLab,JupyterLab3",
"author": "Bum Chul Kwon",
"author_email": "Bum Chul Kwon <bumchul.kwon@us.ibm.com>",
"download_url": "https://files.pythonhosted.org/packages/20/86/ae993f988c3fe40b78dcb4bf708bc81963f69bda218d8885028d3ba7e549/finspector-0.1.0.tar.gz",
"platform": null,
"description": "# Finspector\n\nFinspector is designed to help users explore the fairness and bias of foundation models using interactive visualizations.\n\n## Installation\n\nFor a development installation (requires [Node.js](https://nodejs.org) and [Yarn version 1](https://classic.yarnpkg.com/)):\n\n $ git clone https://github.com/IBM/finspector.git\n $ cd finspector\n $ pip install -r requirements.txt\n $ pip install -e .\n $ npm install\n $ yarn install\n $ jupyter labextension develop finspector --overwrite\n $ cd js\n $ yarn run build\n $ cd ..\n\n## Run Jupyter Example\n\nTo run the example, you need a dataset stored in `finspector/data/xyz.csv]`. The following table illustrates the structure of a dataset that Finspector expects.\n\nbias_type | sent_index | para_index | stereo_type | more_or_less | <div style=\"width:200px\">sent</div> | bert | roberta | albert\n--- | --- | --- | --- | --- | --- | --- | --- | ---\nsocioeconomic | 485 | 0 | 1 | sent_more | May the Force be with you. | -2.5 | -1.5 | -2.22\nsocioeconomic | 485 | 1 | 1 | sent_more | May the Power be with you. | -2.0 | -1.4 | -2.87\n\n- `bias_type`: bias category (e.g., race, sex, socioeconomic)\n- `sent_index`: index of root sentence\n- `para_index`: index of parphrased sentence\n- `stereo_type`: (0: less stereo type, 1: stereotype)\n- `more_or_less`: (sent_less: less stereo type, sent_more: stereotype)\n- `sent`: sentence in string\n- `bert`, `roberta`, `alberta`: Pseudo-log-likelihood scores for `sent` generated by each model\n\nOnce your dataset is ready, you can run\n\n $ jupyter lab\n\nThen, click on `acl2023demo.ipynb` on your Jupyter lab. The repository includes a small dataset `data/sample-movie-quotes.csv` for demonstration purpose.\n\n## Citation\n\nIf you use Finspector for your research, you can use this citation format:\n\n```\n@inproceedings{kwon-etal-2023-finspector,\n title = \"{Finspector}: A Human-Centered Visual Inspection Tool for Exploring and Comparing Biases among Foundation Models\",\n author = \"Kwon, Bum Chul and Mihindukulasooriya, Nandana\",\n booktitle = \"Proceedings of the 61th Annual Meeting of the Association for Computational Linguistics: System Demonstrations\",\n year = \"2023\",\n address = \"Toronto, Canada\",\n publisher = \"Association for Computational Linguistics\"\n}\n```\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "A package that helps users explore fairness and bias of foundation models using pseudo-log-likelihood measures and interactive visualizations.",
"version": "0.1.0",
"project_urls": {
"Bug Tracker": "https://github.com/IBM/finspector/issues",
"Homepage": "https://github.com/IBM/finspector"
},
"split_keywords": [
"ipython",
"widgets",
"jupyter",
"jupyterlab",
"jupyterlab3"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0389710df148c55a38f572e075d7918bcb1438586313543644e1a4115a4afe90",
"md5": "2a581b8a592d987b26137d57a5b159d7",
"sha256": "590e3b4f00e1c51942eb10e36a80af5270d61483d5b11425cfe2a6cd4bf884b6"
},
"downloads": -1,
"filename": "finspector-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2a581b8a592d987b26137d57a5b159d7",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 650105,
"upload_time": "2023-06-06T19:49:02",
"upload_time_iso_8601": "2023-06-06T19:49:02.102324Z",
"url": "https://files.pythonhosted.org/packages/03/89/710df148c55a38f572e075d7918bcb1438586313543644e1a4115a4afe90/finspector-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "2086ae993f988c3fe40b78dcb4bf708bc81963f69bda218d8885028d3ba7e549",
"md5": "c8258e12d26979d90046fd9b408a8bc6",
"sha256": "f2e45b3e4239deac0ff617b15d960ba9c51b235c0351b64ecafe5efb1b8a3039"
},
"downloads": -1,
"filename": "finspector-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "c8258e12d26979d90046fd9b408a8bc6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 559831,
"upload_time": "2023-06-06T19:49:04",
"upload_time_iso_8601": "2023-06-06T19:49:04.271480Z",
"url": "https://files.pythonhosted.org/packages/20/86/ae993f988c3fe40b78dcb4bf708bc81963f69bda218d8885028d3ba7e549/finspector-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-06 19:49:04",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "IBM",
"github_project": "finspector",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "finspector"
}
JSON
