| Name | conformist JSON |
| Version |
1.1.0
JSON |
| download |
| home_page | None |
| Summary | Conformal prediction for machine learning classifiers |
| upload_time | 2024-12-09 09:10:37 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.8 |
| license | MIT License Copyright (c) 2024 Mariya Lysenkova Wiklander Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
machinelearning
statistics
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
<!-- Link to Google Font -->
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=League+Script&display=swap" rel="stylesheet">
<link href="readme.css" rel="stylesheet">
[](https://github.com/Molmed/conformist/actions/workflows/python-package.yml)
[](https://github.com/Molmed/conformist/actions/workflows/python-publish.yml)
<h1 class="custom-font miami-neon-text">Conformist</h1>
Conformist v1.1.0 is an implementation of conformal prediction, specifically conformal risk control. It was written using Python 3.8.
*BaseCoP* contains utility functions common to all conformal predictors, such as splitting data into calibration and validation sets, and setting up runs. It is extended by *FNRCoP* that implements conformal risk control.
The *ValidationRun* class contains the data from a single run, which entails shuffling the data randomly, splitting it into calibration and validation datasets, calibrating the conformal predictor on the calibration data and creating prediction sets for the validation data.
The *ValidationTrial* class contains a list of runs and calculates statistics across these runs.
## Installation
`pip install conformist`
## Input file format
The input to Conformist is a CSV file with the following columns:
`id, known_class, predicted_class, [proba_columns]`
The proba_columns should contain class-specific probability scores and correspond to the names used in the `known_class` and `predicted_class` columns.
Example:
| id | known_class | predicted_class | ClassA | ClassB | ClassC |
| --------| ----------- | --------------- | ------ | ------ | ------ |
| Sample1 | ClassB | ClassA | 0.70 | 0.25 | 0.05 |
| Sample2 | ClassA | ClassA | 0.98 | 0.02 | 0.0 |
| Sample3 | ClassC | ClassC | 0.01 | 0.01 | 0.98 |
## Example implementation
```
from conformist import FNRCoP, PredictionDataset, ValidationTrial
CALIB_DATA_CSV = 'path/to/calib.csv'
TEST_DATA_CSV = 'path/to/test.csv'
OUTPUT_DIR = 'path/to/output'
ALPHA = 0.05 # Select a reasonable False Negative Rate
# Read in formatted predictions
calpd = PredictionDataset(predictions_csv=CALIB_DATA_CSV,
dataset_name='my_cal_data')
testpd = PredictionDataset(predictions_csv=TEST_DATA_CSV,
dataset_name='my_test_data')
# Get class counts and prediction heatmap
calpd.run_reports(OUTPUT_DIR)
# Validation trial and reports
mcp = FNRCoP(calpd, alpha=ALPHA)
trial = mcp.do_validation_trial(n_runs=10000)
trial.run_reports(OUTPUT_DIR)
# Recalibrate conformal predictor
mcp = FNRCoP(calpd, alpha=ALPHA)
mcp.calibrate()
# Predict
formatted_predictions = mcp.predict(testpd,
OUTPUT_DIR)
```
Raw data
{
"_id": null,
"home_page": null,
"name": "conformist",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "machinelearning, statistics",
"author": null,
"author_email": "Mariya Lysenkova Wiklander <mariya.lysenkova@medsci.uu.se>",
"download_url": "https://files.pythonhosted.org/packages/77/52/b702a69b52e8ef983181f8256595c5e0d069b14b08d90394fe695e2284e4/conformist-1.1.0.tar.gz",
"platform": null,
"description": "<!-- Link to Google Font -->\n<link rel=\"preconnect\" href=\"https://fonts.googleapis.com\">\n<link rel=\"preconnect\" href=\"https://fonts.gstatic.com\" crossorigin>\n<link href=\"https://fonts.googleapis.com/css2?family=League+Script&display=swap\" rel=\"stylesheet\">\n<link href=\"readme.css\" rel=\"stylesheet\">\n\n\n[](https://github.com/Molmed/conformist/actions/workflows/python-package.yml)\n[](https://github.com/Molmed/conformist/actions/workflows/python-publish.yml)\n\n\n<h1 class=\"custom-font miami-neon-text\">Conformist</h1>\n\nConformist v1.1.0 is an implementation of conformal prediction, specifically conformal risk control. It was written using Python 3.8.\n\n*BaseCoP* contains utility functions common to all conformal predictors, such as splitting data into calibration and validation sets, and setting up runs. It is extended by *FNRCoP* that implements conformal risk control.\n\nThe *ValidationRun* class contains the data from a single run, which entails shuffling the data randomly, splitting it into calibration and validation datasets, calibrating the conformal predictor on the calibration data and creating prediction sets for the validation data.\n\nThe *ValidationTrial* class contains a list of runs and calculates statistics across these runs.\n\n## Installation\n`pip install conformist`\n\n## Input file format\n\nThe input to Conformist is a CSV file with the following columns:\n`id, known_class, predicted_class, [proba_columns]`\n\nThe proba_columns should contain class-specific probability scores and correspond to the names used in the `known_class` and `predicted_class` columns.\n\nExample:\n| id | known_class | predicted_class | ClassA | ClassB | ClassC |\n| --------| ----------- | --------------- | ------ | ------ | ------ |\n| Sample1 | ClassB | ClassA | 0.70 | 0.25 | 0.05 |\n| Sample2 | ClassA | ClassA | 0.98 | 0.02 | 0.0 |\n| Sample3 | ClassC | ClassC | 0.01 | 0.01 | 0.98 |\n\n## Example implementation\n```\nfrom conformist import FNRCoP, PredictionDataset, ValidationTrial\n\nCALIB_DATA_CSV = 'path/to/calib.csv'\nTEST_DATA_CSV = 'path/to/test.csv'\nOUTPUT_DIR = 'path/to/output'\nALPHA = 0.05 # Select a reasonable False Negative Rate\n\n# Read in formatted predictions\ncalpd = PredictionDataset(predictions_csv=CALIB_DATA_CSV,\n dataset_name='my_cal_data')\ntestpd = PredictionDataset(predictions_csv=TEST_DATA_CSV,\n dataset_name='my_test_data')\n\n# Get class counts and prediction heatmap\ncalpd.run_reports(OUTPUT_DIR)\n\n# Validation trial and reports\nmcp = FNRCoP(calpd, alpha=ALPHA)\ntrial = mcp.do_validation_trial(n_runs=10000)\ntrial.run_reports(OUTPUT_DIR)\n\n# Recalibrate conformal predictor\nmcp = FNRCoP(calpd, alpha=ALPHA)\nmcp.calibrate()\n\n# Predict\nformatted_predictions = mcp.predict(testpd,\n OUTPUT_DIR)\n```\n",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2024 Mariya Lysenkova Wiklander Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ",
"summary": "Conformal prediction for machine learning classifiers",
"version": "1.1.0",
"project_urls": {
"Homepage": "https://github.com/Molmed/conformist"
},
"split_keywords": [
"machinelearning",
" statistics"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1f1897dde6ff74a9dfb0c7f1372399a4ab4f6dcc936b5ba841f6ddc448bf0363",
"md5": "9f61cbb22bf6d4be72fce13ac00c06f8",
"sha256": "85d5b0266592495a087d383be3c44ab706890cbf3464bd543568b1d8e364799c"
},
"downloads": -1,
"filename": "conformist-1.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9f61cbb22bf6d4be72fce13ac00c06f8",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 23968,
"upload_time": "2024-12-09T09:10:34",
"upload_time_iso_8601": "2024-12-09T09:10:34.377272Z",
"url": "https://files.pythonhosted.org/packages/1f/18/97dde6ff74a9dfb0c7f1372399a4ab4f6dcc936b5ba841f6ddc448bf0363/conformist-1.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7752b702a69b52e8ef983181f8256595c5e0d069b14b08d90394fe695e2284e4",
"md5": "7328dbb3c1f77ce7c3fbf65f9b469a39",
"sha256": "fdae46d80b5071ef4ab47e47c0cf9ee3926852aed565dedc1cb54c8a0948034c"
},
"downloads": -1,
"filename": "conformist-1.1.0.tar.gz",
"has_sig": false,
"md5_digest": "7328dbb3c1f77ce7c3fbf65f9b469a39",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 21623,
"upload_time": "2024-12-09T09:10:37",
"upload_time_iso_8601": "2024-12-09T09:10:37.422308Z",
"url": "https://files.pythonhosted.org/packages/77/52/b702a69b52e8ef983181f8256595c5e0d069b14b08d90394fe695e2284e4/conformist-1.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-09 09:10:37",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Molmed",
"github_project": "conformist",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "conformist"
}
