## Rank Preserving Calibration of multiclass probabilities via Dykstra's alternating projections
[](https://pypi.org/project/rank_preserving_calibration/)
[](https://pepy.tech/projects/rank_preserving_calibration)
[](https://www.python.org/downloads/)
[](https://opensource.org/licenses/MIT)
Survey statisticians and machine learning practitioners often need to adjust the predicted class probabilities from a classifier so that they match known population totals (column marginals). Simple post-hoc methods that apply separate logit shifts or raking to each class can scramble the ranking of individuals within a class when there are three or more classes. This package implements a rank-preserving calibration procedure that projects probabilities onto the intersection of two convex sets:
1. **Row-simplex**: each row sums to one and all entries are non-negative.
2. **Isotonic column marginals**: within each class, values are non-decreasing when instances are sorted by their original scores for that class, and the sum of each column equals a user-supplied target.
The algorithm uses Dykstra's alternating projection method in Euclidean geometry. When the specified column totals are feasible, the procedure returns a matrix that preserves cross-person discrimination within each class, matches the desired totals, and remains a valid probability distribution for each instance. If no such matrix exists, the algorithm converges to the closest point (in L2 sense) satisfying both sets of constraints.
An **ADMM optimization** implementation is also provided as an alternative solver that minimizes `||Q - P||²` subject to the same constraints.
## Installation
```bash
pip install -e .
```
The only runtime dependency is `numpy`. Optional dependencies include `scipy` (for enhanced test case generation) and `matplotlib` (for examples).
## Usage
```python
import numpy as np
from rank_preserving_calibration import calibrate_dykstra
P = np.array([
[0.6, 0.3, 0.1],
[0.2, 0.5, 0.3],
[0.1, 0.2, 0.7],
])
# Target column sums, e.g. population class frequencies. Must sum to the
# number of rows (3 in this example) for perfect feasibility.
M = np.array([1.0, 1.0, 1.0])
result = calibrate_dykstra(P, M)
print("Adjusted probabilities:\n", result.Q)
print("Converged:", result.converged)
print("Iterations:", result.iterations)
print("Max row error:", result.max_row_error)
print("Max column error:", result.max_col_error)
print("Rank violations:", result.max_rank_violation)
```
The returned `CalibrationResult` contains the calibrated matrix `Q` with the same shape as `P`. Each row of `Q` sums to one, the column sums match `M`, and within each column the entries are sorted in non-decreasing order according to the order implied by the original `P`.
## Functions
### `calibrate_dykstra(P, M, **kwargs)`
Calibrate using Dykstra's alternating projections (recommended).
### `calibrate_admm(P, M, **kwargs)`
Calibrate using ADMM optimization with penalty parameter `rho`.
### `create_test_case(case_type, N, J, **kwargs)`
Generate synthetic test data for various scenarios.
## Arguments
| Parameter | Type | Description |
| --- | --- | --- |
| `P` | `ndarray` of shape `[N, J]` | Base multiclass probabilities or non-negative scores. Rows will be projected to the simplex. |
| `M` | `ndarray` of shape `[J]` | Target column totals (e.g. population class frequencies). The sum of `M` should equal the number of rows `N` for exact feasibility. |
| `max_iters` | `int` | Maximum number of projection iterations (default `3000` for Dykstra, `1000` for ADMM). |
| `tol` | `float` | Relative convergence tolerance (default `1e-7` for Dykstra, `1e-6` for ADMM). |
| `verbose` | `bool` | If `True`, prints convergence diagnostics. |
| `rho` | `float` | ADMM penalty parameter (default `1.0`, ADMM only). |
## Returns
### CalibrationResult
Both functions return a `CalibrationResult` object with the following attributes:
* `Q`: NumPy array of shape `[N, J]` containing the calibrated probabilities. Each row sums to one, each column approximately sums to the corresponding entry of `M`, and within each column the values are non-decreasing according to the ordering induced by `P`.
* `converged`: boolean indicating whether the solver met the tolerance criteria.
* `iterations`: number of iterations performed.
* `max_row_error`: maximum absolute deviation of row sums from 1.
* `max_col_error`: maximum absolute deviation of column sums from `M`.
* `max_rank_violation`: maximum violation of monotonicity (should be 0 up to numerical tolerance).
* `final_change`: final relative change between iterations.
### ADMMResult
The ADMM function returns an `ADMMResult` object with additional convergence history:
* All `CalibrationResult` attributes plus:
* `objective_values`: objective function values over iterations.
* `primal_residuals`: primal residual norms over iterations.
* `dual_residuals`: dual residual norms over iterations.
## Algorithm Notes
* **Dykstra's Method**: Uses alternating projections with memory terms to ensure convergence to the intersection of constraint sets. Rows are projected onto the simplex via the algorithm of Duchi et al., and columns are projected via the pool-adjacent-violators algorithm followed by an additive shift to match column totals. This is the recommended method for most applications.
* **ADMM**: Solves the constrained optimization problem using the Alternating Direction Method of Multipliers. May converge faster for some problems but requires tuning the penalty parameter `rho`. The algorithm minimizes the sum of squared differences `0.5 * ||Q - P||²_F` subject to the calibration constraints.
## Examples
See `examples.ipynb` for comprehensive examples including:
- Basic usage and visualization
- Real-world classifier calibration scenarios
- Survey reweighting applications
- Algorithm comparison and performance analysis
## Testing
```bash
python -m pytest tests/ -v
```
## Legacy Compatibility
For backward compatibility, the following aliases are available:
- `calibrate_rank_preserving` → `calibrate_dykstra`
- `admm_rank_preserving_simplex_marginals` → `calibrate_dykstra`
## License
This software is released under the terms of the MIT license.
## Author
Gaurav Sood `<gsood07@gmail.com>`
Survey statisticians and machine learning practitioners often need to adjust
the predicted class probabilities from a classifier so that they match known
population totals (column marginals). Simple post‑hoc methods that apply
separate logit shifts or raking to each class can scramble the ranking of
individuals within a class when there are three or more classes. This
package implements a rank‑preserving calibration procedure that projects
probabilities onto the intersection of two convex sets:
1. **Row‑simplex**: each row sums to one and all entries are non‑negative.
2. **Isotonic column marginals**: within each class, values are
non‑decreasing when instances are sorted by their original scores for
that class, and the sum of each column equals a user‑supplied target.
The algorithm uses Dykstra's alternating projection method in Euclidean
geometry. When the specified column totals are feasible, the procedure
returns a matrix that preserves cross‑person discrimination within each
class, matches the desired totals, and remains a valid probability
distribution for each instance. If no such matrix exists, the algorithm
converges to the closest point (in L2 sense) satisfying both sets of
constraints.
Experimental support for **KL (I‑divergence) geometry** is also provided.
In that mode, row projections normalise each row by its sum and column
projections are based on KL‑isotonic regression (PAV in log space) followed
by multiplicative scaling. Both geometries enforce the same row and
column constraints and preserve within‑class ranking.
## Installation
To install the package from source, clone the repository and run:
```sh
pip install .
```
The only runtime dependency is `numpy`.
## Usage
```python
import numpy as np
from rank_preserving_calibration import admm_rank_preserving_simplex_marginals
P = np.array([
[0.6, 0.3, 0.1],
[0.2, 0.5, 0.3],
[0.1, 0.2, 0.7],
])
# Target column sums, e.g. population class frequencies. Must sum to the
# number of rows (3 in this example) for perfect feasibility.
M = np.array([1.0, 1.0, 1.0])
Q, info = admm_rank_preserving_simplex_marginals(P, M)
print("Adjusted probabilities:\n", Q)
print("Diagnostics:\n", info)
```
The returned matrix `Q` has the same shape as `P`. Each row of `Q` sums
to one, the column sums match `M`, and within each column the entries are
sorted in non‑decreasing order according to the order implied by the
original `P`. The `info` dictionary reports the number of iterations
used, the maximum row and column errors, and any residual rank
violations (at numerical precision).
## Arguments
| Parameter | Type | Description |
| --- | --- | --- |
| `P` | `ndarray` of shape `[N, J]` | Base multiclass probabilities or non‑negative scores. Rows will be projected to the simplex. |
| `M` | `ndarray` of shape `[J]` | Target column totals (e.g. population class frequencies). The sum of `M` should equal the number of rows `N` for exact feasibility. |
| `geometry` | `str` | Either `'euclidean'` (default) or `'kl'` (experimental). Determines the geometry used for projections. |
| `max_iters` | `int` | Maximum number of projection iterations (default `3000`). |
| `tol` | `float` | Relative convergence tolerance (default `1e‑7`). |
| `verbose` | `bool` | If `True`, prints convergence diagnostics. |
## Returns
The function returns a tuple `(Q, info)` where:
* `Q` is a NumPy array of shape `[N, J]` containing the calibrated probabilities. Each row sums to one, each column approximately sums to the corresponding entry of `M`, and within each column the values are non‑decreasing according to the ordering induced by `P`.
* `info` is a dictionary with diagnostics:
- `iterations`: number of iterations performed.
- `max_row_error`: maximum absolute deviation of row sums from 1.
- `max_col_error`: maximum absolute deviation of column sums from `M`.
- `max_rank_violation`: maximum violation of monotonicity (should be 0 up to numerical tolerance).
- `converged`: boolean indicating whether the solver met the tolerance criteria.
- `geometry`: which geometry was used (`'euclidean'` or `'kl'`).
## Geometry Notes
* **Euclidean (L2)**: The default solver uses Euclidean projections. Rows are projected onto the simplex via the algorithm of Duchi et al., and columns are projected via the pool‑adjacent‑violators algorithm followed by an additive shift to match the column totals. This minimises the sum of squared differences `0.5 * ||Q - P||_F^2`.
* **KL (I‑divergence)**: Setting `geometry='kl'` switches to KL‑style projections. Each row is normalised by its sum (multiplicative projection), and each column is projected by applying isotonic regression to the logarithms of the values (PAV in log space) followed by multiplicative scaling to match the column sum. This mode is experimental but preserves within‑class ranking and approximately minimises the I‑divergence from `P`.
## License
This software is released under the terms of the MIT license. See the
`LICENSE` file for details.
## Author
Gaurav Sood `<gsood07@gmail.com>`
Raw data
{
"_id": null,
"home_page": null,
"name": "rank-preserving-calibration",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "calibration, machine learning, multiclass, isotonic, dykstra, admm",
"author": null,
"author_email": "Gaurav Sood <gsood07@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/18/0a/b11fdfceabce726b1367f0219a863b490ee0d693f39f9fa716f757a59b33/rank_preserving_calibration-0.3.0.tar.gz",
"platform": null,
"description": "## Rank Preserving Calibration of multiclass probabilities via Dykstra's alternating projections\n\n[](https://pypi.org/project/rank_preserving_calibration/)\n[](https://pepy.tech/projects/rank_preserving_calibration)\n[](https://www.python.org/downloads/)\n[](https://opensource.org/licenses/MIT)\n\nSurvey statisticians and machine learning practitioners often need to adjust the predicted class probabilities from a classifier so that they match known population totals (column marginals). Simple post-hoc methods that apply separate logit shifts or raking to each class can scramble the ranking of individuals within a class when there are three or more classes. This package implements a rank-preserving calibration procedure that projects probabilities onto the intersection of two convex sets:\n\n1. **Row-simplex**: each row sums to one and all entries are non-negative.\n2. **Isotonic column marginals**: within each class, values are non-decreasing when instances are sorted by their original scores for that class, and the sum of each column equals a user-supplied target.\n\nThe algorithm uses Dykstra's alternating projection method in Euclidean geometry. When the specified column totals are feasible, the procedure returns a matrix that preserves cross-person discrimination within each class, matches the desired totals, and remains a valid probability distribution for each instance. If no such matrix exists, the algorithm converges to the closest point (in L2 sense) satisfying both sets of constraints.\n\nAn **ADMM optimization** implementation is also provided as an alternative solver that minimizes `||Q - P||\u00b2` subject to the same constraints.\n\n## Installation\n\n```bash\npip install -e .\n```\n\nThe only runtime dependency is `numpy`. Optional dependencies include `scipy` (for enhanced test case generation) and `matplotlib` (for examples).\n\n## Usage\n\n```python\nimport numpy as np\nfrom rank_preserving_calibration import calibrate_dykstra\n\nP = np.array([\n [0.6, 0.3, 0.1],\n [0.2, 0.5, 0.3],\n [0.1, 0.2, 0.7],\n])\n\n# Target column sums, e.g. population class frequencies. Must sum to the\n# number of rows (3 in this example) for perfect feasibility.\nM = np.array([1.0, 1.0, 1.0])\n\nresult = calibrate_dykstra(P, M)\n\nprint(\"Adjusted probabilities:\\n\", result.Q)\nprint(\"Converged:\", result.converged)\nprint(\"Iterations:\", result.iterations)\nprint(\"Max row error:\", result.max_row_error)\nprint(\"Max column error:\", result.max_col_error)\nprint(\"Rank violations:\", result.max_rank_violation)\n```\n\nThe returned `CalibrationResult` contains the calibrated matrix `Q` with the same shape as `P`. Each row of `Q` sums to one, the column sums match `M`, and within each column the entries are sorted in non-decreasing order according to the order implied by the original `P`.\n\n## Functions\n\n### `calibrate_dykstra(P, M, **kwargs)`\n\nCalibrate using Dykstra's alternating projections (recommended).\n\n### `calibrate_admm(P, M, **kwargs)` \n\nCalibrate using ADMM optimization with penalty parameter `rho`.\n\n### `create_test_case(case_type, N, J, **kwargs)`\n\nGenerate synthetic test data for various scenarios.\n\n## Arguments\n\n| Parameter | Type | Description |\n| --- | --- | --- |\n| `P` | `ndarray` of shape `[N, J]` | Base multiclass probabilities or non-negative scores. Rows will be projected to the simplex. |\n| `M` | `ndarray` of shape `[J]` | Target column totals (e.g. population class frequencies). The sum of `M` should equal the number of rows `N` for exact feasibility. |\n| `max_iters` | `int` | Maximum number of projection iterations (default `3000` for Dykstra, `1000` for ADMM). |\n| `tol` | `float` | Relative convergence tolerance (default `1e-7` for Dykstra, `1e-6` for ADMM). |\n| `verbose` | `bool` | If `True`, prints convergence diagnostics. |\n| `rho` | `float` | ADMM penalty parameter (default `1.0`, ADMM only). |\n\n## Returns\n\n### CalibrationResult\n\nBoth functions return a `CalibrationResult` object with the following attributes:\n\n* `Q`: NumPy array of shape `[N, J]` containing the calibrated probabilities. Each row sums to one, each column approximately sums to the corresponding entry of `M`, and within each column the values are non-decreasing according to the ordering induced by `P`.\n* `converged`: boolean indicating whether the solver met the tolerance criteria.\n* `iterations`: number of iterations performed.\n* `max_row_error`: maximum absolute deviation of row sums from 1.\n* `max_col_error`: maximum absolute deviation of column sums from `M`.\n* `max_rank_violation`: maximum violation of monotonicity (should be 0 up to numerical tolerance).\n* `final_change`: final relative change between iterations.\n\n### ADMMResult\n\nThe ADMM function returns an `ADMMResult` object with additional convergence history:\n\n* All `CalibrationResult` attributes plus:\n* `objective_values`: objective function values over iterations.\n* `primal_residuals`: primal residual norms over iterations.\n* `dual_residuals`: dual residual norms over iterations.\n\n## Algorithm Notes\n\n* **Dykstra's Method**: Uses alternating projections with memory terms to ensure convergence to the intersection of constraint sets. Rows are projected onto the simplex via the algorithm of Duchi et al., and columns are projected via the pool-adjacent-violators algorithm followed by an additive shift to match column totals. This is the recommended method for most applications.\n\n* **ADMM**: Solves the constrained optimization problem using the Alternating Direction Method of Multipliers. May converge faster for some problems but requires tuning the penalty parameter `rho`. The algorithm minimizes the sum of squared differences `0.5 * ||Q - P||\u00b2_F` subject to the calibration constraints.\n\n## Examples\n\nSee `examples.ipynb` for comprehensive examples including:\n- Basic usage and visualization\n- Real-world classifier calibration scenarios \n- Survey reweighting applications\n- Algorithm comparison and performance analysis\n\n## Testing\n\n```bash\npython -m pytest tests/ -v\n```\n\n## Legacy Compatibility\n\nFor backward compatibility, the following aliases are available:\n- `calibrate_rank_preserving` \u2192 `calibrate_dykstra`\n- `admm_rank_preserving_simplex_marginals` \u2192 `calibrate_dykstra`\n\n## License\n\nThis software is released under the terms of the MIT license.\n\n## Author\n\nGaurav Sood `<gsood07@gmail.com>`\nSurvey statisticians and machine learning practitioners often need to adjust\nthe predicted class probabilities from a classifier so that they match known\npopulation totals (column marginals). Simple post\u2011hoc methods that apply\nseparate logit shifts or raking to each class can scramble the ranking of\nindividuals within a class when there are three or more classes. This\npackage implements a rank\u2011preserving calibration procedure that projects\nprobabilities onto the intersection of two convex sets:\n\n1. **Row\u2011simplex**: each row sums to one and all entries are non\u2011negative.\n2. **Isotonic column marginals**: within each class, values are\n non\u2011decreasing when instances are sorted by their original scores for\n that class, and the sum of each column equals a user\u2011supplied target.\n\nThe algorithm uses Dykstra's alternating projection method in Euclidean\ngeometry. When the specified column totals are feasible, the procedure\nreturns a matrix that preserves cross\u2011person discrimination within each\nclass, matches the desired totals, and remains a valid probability\ndistribution for each instance. If no such matrix exists, the algorithm\nconverges to the closest point (in L2 sense) satisfying both sets of\nconstraints.\n\nExperimental support for **KL (I\u2011divergence) geometry** is also provided.\nIn that mode, row projections normalise each row by its sum and column\nprojections are based on KL\u2011isotonic regression (PAV in log space) followed\nby multiplicative scaling. Both geometries enforce the same row and\ncolumn constraints and preserve within\u2011class ranking.\n\n## Installation\n\nTo install the package from source, clone the repository and run:\n\n```sh\npip install .\n```\n\nThe only runtime dependency is `numpy`.\n\n## Usage\n\n```python\nimport numpy as np\nfrom rank_preserving_calibration import admm_rank_preserving_simplex_marginals\n\nP = np.array([\n [0.6, 0.3, 0.1],\n [0.2, 0.5, 0.3],\n [0.1, 0.2, 0.7],\n])\n\n# Target column sums, e.g. population class frequencies. Must sum to the\n# number of rows (3 in this example) for perfect feasibility.\nM = np.array([1.0, 1.0, 1.0])\n\nQ, info = admm_rank_preserving_simplex_marginals(P, M)\n\nprint(\"Adjusted probabilities:\\n\", Q)\nprint(\"Diagnostics:\\n\", info)\n```\n\nThe returned matrix `Q` has the same shape as `P`. Each row of `Q` sums\nto one, the column sums match `M`, and within each column the entries are\nsorted in non\u2011decreasing order according to the order implied by the\noriginal `P`. The `info` dictionary reports the number of iterations\nused, the maximum row and column errors, and any residual rank\nviolations (at numerical precision).\n\n## Arguments\n\n| Parameter | Type | Description |\n| --- | --- | --- |\n| `P` | `ndarray` of shape `[N, J]` | Base multiclass probabilities or non\u2011negative scores. Rows will be projected to the simplex. |\n| `M` | `ndarray` of shape `[J]` | Target column totals (e.g. population class frequencies). The sum of `M` should equal the number of rows `N` for exact feasibility. |\n| `geometry` | `str` | Either `'euclidean'` (default) or `'kl'` (experimental). Determines the geometry used for projections. |\n| `max_iters` | `int` | Maximum number of projection iterations (default `3000`). |\n| `tol` | `float` | Relative convergence tolerance (default `1e\u20117`). |\n| `verbose` | `bool` | If `True`, prints convergence diagnostics. |\n\n## Returns\n\nThe function returns a tuple `(Q, info)` where:\n\n* `Q` is a NumPy array of shape `[N, J]` containing the calibrated probabilities. Each row sums to one, each column approximately sums to the corresponding entry of `M`, and within each column the values are non\u2011decreasing according to the ordering induced by `P`.\n* `info` is a dictionary with diagnostics:\n - `iterations`: number of iterations performed.\n - `max_row_error`: maximum absolute deviation of row sums from 1.\n - `max_col_error`: maximum absolute deviation of column sums from `M`.\n - `max_rank_violation`: maximum violation of monotonicity (should be 0 up to numerical tolerance).\n - `converged`: boolean indicating whether the solver met the tolerance criteria.\n - `geometry`: which geometry was used (`'euclidean'` or `'kl'`).\n\n## Geometry Notes\n\n* **Euclidean (L2)**: The default solver uses Euclidean projections. Rows are projected onto the simplex via the algorithm of Duchi et al., and columns are projected via the pool\u2011adjacent\u2011violators algorithm followed by an additive shift to match the column totals. This minimises the sum of squared differences `0.5 * ||Q - P||_F^2`.\n* **KL (I\u2011divergence)**: Setting `geometry='kl'` switches to KL\u2011style projections. Each row is normalised by its sum (multiplicative projection), and each column is projected by applying isotonic regression to the logarithms of the values (PAV in log space) followed by multiplicative scaling to match the column sum. This mode is experimental but preserves within\u2011class ranking and approximately minimises the I\u2011divergence from `P`.\n\n\n## License\n\nThis software is released under the terms of the MIT license. See the\n`LICENSE` file for details.\n\n## Author\n\nGaurav Sood `<gsood07@gmail.com>`\n",
"bugtrack_url": null,
"license": null,
"summary": "Rank-preserving calibration of multiclass probabilities via Dykstra's projections and ADMM.",
"version": "0.3.0",
"project_urls": {
"Documentation": "https://github.com/finite_sample/rank-preserving-calibration#readme",
"Homepage": "https://github.com/finite_sample/rank-preserving-calibration",
"Issues": "https://github.com/finite_sample/rank-preserving-calibration/issues",
"Source": "https://github.com/finite_sample/rank-preserving-calibration"
},
"split_keywords": [
"calibration",
" machine learning",
" multiclass",
" isotonic",
" dykstra",
" admm"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "2c84f5196689ff4349fba8592ca473c4ec1c36ad992b20ce6f1c0a8f6c8c0636",
"md5": "e2cda943ac1ec2195df252014d727a9e",
"sha256": "1eda42454871da14e8ad73633f176dd0b9c1976a7cf7f581daa1de527d1fb5e8"
},
"downloads": -1,
"filename": "rank_preserving_calibration-0.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e2cda943ac1ec2195df252014d727a9e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 15185,
"upload_time": "2025-08-17T18:09:30",
"upload_time_iso_8601": "2025-08-17T18:09:30.478872Z",
"url": "https://files.pythonhosted.org/packages/2c/84/f5196689ff4349fba8592ca473c4ec1c36ad992b20ce6f1c0a8f6c8c0636/rank_preserving_calibration-0.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "180ab11fdfceabce726b1367f0219a863b490ee0d693f39f9fa716f757a59b33",
"md5": "17ba506f8cf03777de3a3b231cc85bb6",
"sha256": "5b34c2472a1459affbdbd8b034e05a432661095dd289db09db1f7d51dc00c618"
},
"downloads": -1,
"filename": "rank_preserving_calibration-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "17ba506f8cf03777de3a3b231cc85bb6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 18696,
"upload_time": "2025-08-17T18:09:31",
"upload_time_iso_8601": "2025-08-17T18:09:31.554140Z",
"url": "https://files.pythonhosted.org/packages/18/0a/b11fdfceabce726b1367f0219a863b490ee0d693f39f9fa716f757a59b33/rank_preserving_calibration-0.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-17 18:09:31",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "finite_sample",
"github_project": "rank-preserving-calibration#readme",
"github_not_found": true,
"lcname": "rank-preserving-calibration"
}
JSON
