# Density Ratio Estimation with PyTorch
**torchdensityratio** is a package that provides an implementation of the relative
unconstrained least squares importance fitting (RuLSIF) algorithm for the estimation of
ratio of probability densities (or density ratios) [[1](#1),[2](#2),[3](#3)]. The implementation is in PyTorch and
follows a functional paradigm.
[](https://badge.fury.io/py/torchdensityratio)
[](https://github.com/FilippoAiraldi/torch-density-ratio/blob/master/LICENSE)
[](https://github.com/FilippoAiraldi/torch-density-ratio/actions/workflows/tests.yml)
[](https://www.pepy.tech/projects/torchdensityratio)
[](https://qlty.sh/gh/FilippoAiraldi/projects/torch-density-ratio)
[](https://qlty.sh/gh/FilippoAiraldi/projects/torch-density-ratio)
[](https://docs.astral.sh/ruff/)
---
## Installation
### Using `torchdensityratio`
You can use `pip` to install **torchdensityratio** with the command
```bash
pip install torchdensityratio
```
**torchdensityratio** has the following dependencies
- Python 3.10 or higher
- [PyTorch](https://pytorch.org/)
### Using source code
If you'd like to play around with the source code instead, run
```bash
git clone https://github.com/FilippoAiraldi/torch-density-ratio.git
```
You can then install the package to edit it as you wish as
```bash
pip install -e /path/to/torch-density-ratio
```
---
## Getting started
Here we provide a compact example on how **torchdensityratio** can be employed to estimate the ratio of two probability density functions (PDFs) without the need to explicitly estimate the PDFs themselves. This can be done with different methods, but this package
implements the relative unconstrained least squares importance fitting (RuLSIF) algorithm [[1](#1),[2](#2),[3](#3)].
We start by generating some data from two different
multivariate Gaussian distributions:
```python
import numpy as np
import torch
from scipy.stats import multivariate_normal as mvnorm
# define the two multivariate normal distributions
n_dim = 2
mean = np.ones(n_dim)
cov_x = np.eye(n_dim) / 8
cov_y = np.eye(n_dim) / 2
# generate samples from the two distributions
n_samples = 3000
rng = np.random.default_rng(0)
x = torch.from_numpy(mvnorm.rvs(size=n_samples, mean=mean, cov=cov_x, random_state=rng))
y = torch.from_numpy(mvnorm.rvs(size=n_samples, mean=mean, cov=cov_y, random_state=rng))
```
We can then use the `torchdensityratio` package to estimate the ratio of the two PDFs using the RuLSIF algorithm as follows:
```python
from torchdensityratio import rulsif_fit, rulsif_predict
alpha = 0.1
sigmas = torch.as_tensor([0.1, 0.3, 0.5, 0.7, 1.0], dtype=x.dtype)
lambdas = torch.as_tensor([0.01, 0.02, 0.03, 0.04, 0.05], dtype=x.dtype)
mdl = rulsif_fit(x, y, alpha, sigmas, lambdas, seed=int(rng.integers(0, 2**32)))
```
For reproducibility, we can set the `seed` argument to a fixed value. In this way, results are consistent across runs. Then, we evaluate the predicted density ratio on a grid of points, and compute also the true one for comparison:
```python
n_vals = 200
vals = torch.linspace(0, 2, n_vals, dtype=x.dtype)
grid = torch.dstack(torch.meshgrid(vals, vals, indexing="xy")).reshape(-1, 2)
predicted = rulsif_predict(mdl, grid).reshape(n_vals, n_vals)
grid_np = grid.numpy()
pdf_x = mvnorm.pdf(grid_np, mean, cov_x)
target = pdf_x / (alpha * pdf_x + (1 - alpha) * mvnorm.pdf(grid_np, mean, cov_y))
target = target.reshape(n_vals, n_vals)
```
Finally, we can visualize the results using `matplotlib`:
```python
from matplotlib import pyplot as plt
vals_np = vals.numpy()
levels = [0, 0.5, 1, 1.5, 2, 2.5, 3, 3.5, 4.5]
_, axs = plt.subplots(1, 2, constrained_layout=True)
axs[0].contourf(vals_np, vals_np, target, levels=levels)
axs[0].set_title("True density ratio")
axs[1].contourf(vals_np, vals_np, predicted.numpy(), levels=levels)
axs[1].set_title("RuLSIF density ratio")
plt.show()
```
This code produces the following image:
<div align="center">
<img src="https://raw.githubusercontent.com/FilippoAiraldi/torch-density-ratio/master/resources/demo.png" alt="torchdensityratio-demo" height="300">
</div>
---
## Examples
Our [examples](https://github.com/FilippoAiraldi/torch-density-ratio/tree/master/examples)
subdirectory contains other example applications of this library.
---
## License
The repository is provided under the MIT License. See the LICENSE file included with
this repository.
---
## Author
[Filippo Airaldi](https://www.tudelft.nl/staff/f.airaldi/), PhD Candidate
[f.airaldi@tudelft.nl | filippoairaldi@gmail.com]
> [Delft Center for Systems and Control](https://www.tudelft.nl/en/me/about/departments/delft-center-for-systems-and-control/)
in [Delft University of Technology](https://www.tudelft.nl/en/)
Copyright (c) 2025 Filippo Airaldi.
Copyright notice: Technische Universiteit Delft hereby disclaims all copyright interest
in the program “csnlp” (Nonlinear Progamming with CasADi) written by the Author(s).
Prof. Dr. Ir. Fred van Keulen, Dean of ME.
---
## Thanks
This repository is heavily inspired by [hoxo-m/densratio_py](https://github.com/hoxo-m/densratio_py) and
[JohnYKiyo/density_ratio_estimation](https://github.com/JohnYKiyo/density_ratio_estimation), please visit their repositories and star them if you find them useful!
---
## References
<a id="1">[1]</a>
Yamada, M., Suzuki, T., Kanamori, T., Hachiya, H. and Sugiyama, M., 2013.
[Relative density-ratio estimation for robust distribution comparison](https://ieeexplore.ieee.org/abstract/document/6797650).
Neural computation, 25(5), pp.1324-1370.
<a id="2">[2]</a>
Liu, S., Yamada, M., Collier, N. and Sugiyama, M., 2013.
[Change-point detection in time-series data by relative density-ratio estimation](https://www.sciencedirect.com/science/article/abs/pii/S0893608013000270).
Neural Networks, 43, pp.72-83.
<a id="3">[3]</a>
Kanamori, T., Hido, S. and Sugiyama, M., 2009.
[A least-squares approach to direct importance estimation](https://jmlr.csail.mit.edu/papers/volume10/kanamori09a/kanamori09a.pdf).
The Journal of Machine Learning Research, 10, pp.1391-1445
Raw data
{
"_id": null,
"home_page": null,
"name": "torchdensityratio",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "density-ratio-estimation, machine-learning, pytorch",
"author": null,
"author_email": "Filippo Airaldi <filippoairaldi@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/47/3f/15213e5626171737d8ce2df7ac3543541b45c97972a13cfb86ddb3500e5e/torchdensityratio-1.0.1.tar.gz",
"platform": null,
"description": "# Density Ratio Estimation with PyTorch\r\n\r\n**torchdensityratio** is a package that provides an implementation of the relative\r\nunconstrained least squares importance fitting (RuLSIF) algorithm for the estimation of\r\nratio of probability densities (or density ratios) [[1](#1),[2](#2),[3](#3)]. The implementation is in PyTorch and\r\nfollows a functional paradigm.\r\n\r\n[](https://badge.fury.io/py/torchdensityratio)\r\n[](https://github.com/FilippoAiraldi/torch-density-ratio/blob/master/LICENSE)\r\n\r\n\r\n[](https://github.com/FilippoAiraldi/torch-density-ratio/actions/workflows/tests.yml)\r\n[](https://www.pepy.tech/projects/torchdensityratio)\r\n[](https://qlty.sh/gh/FilippoAiraldi/projects/torch-density-ratio)\r\n[](https://qlty.sh/gh/FilippoAiraldi/projects/torch-density-ratio)\r\n[](https://docs.astral.sh/ruff/)\r\n\r\n---\r\n\r\n## Installation\r\n\r\n### Using `torchdensityratio`\r\n\r\nYou can use `pip` to install **torchdensityratio** with the command\r\n\r\n```bash\r\npip install torchdensityratio\r\n```\r\n\r\n**torchdensityratio** has the following dependencies\r\n\r\n- Python 3.10 or higher\r\n- [PyTorch](https://pytorch.org/)\r\n\r\n### Using source code\r\n\r\nIf you'd like to play around with the source code instead, run\r\n\r\n```bash\r\ngit clone https://github.com/FilippoAiraldi/torch-density-ratio.git\r\n```\r\n\r\nYou can then install the package to edit it as you wish as\r\n\r\n```bash\r\npip install -e /path/to/torch-density-ratio\r\n```\r\n\r\n---\r\n\r\n## Getting started\r\n\r\nHere we provide a compact example on how **torchdensityratio** can be employed to estimate the ratio of two probability density functions (PDFs) without the need to explicitly estimate the PDFs themselves. This can be done with different methods, but this package\r\nimplements the relative unconstrained least squares importance fitting (RuLSIF) algorithm [[1](#1),[2](#2),[3](#3)].\r\n\r\n\r\nWe start by generating some data from two different\r\nmultivariate Gaussian distributions:\r\n\r\n```python\r\nimport numpy as np\r\nimport torch\r\nfrom scipy.stats import multivariate_normal as mvnorm\r\n\r\n# define the two multivariate normal distributions\r\nn_dim = 2\r\nmean = np.ones(n_dim)\r\ncov_x = np.eye(n_dim) / 8\r\ncov_y = np.eye(n_dim) / 2\r\n\r\n# generate samples from the two distributions\r\nn_samples = 3000\r\nrng = np.random.default_rng(0)\r\nx = torch.from_numpy(mvnorm.rvs(size=n_samples, mean=mean, cov=cov_x, random_state=rng))\r\ny = torch.from_numpy(mvnorm.rvs(size=n_samples, mean=mean, cov=cov_y, random_state=rng))\r\n```\r\n\r\nWe can then use the `torchdensityratio` package to estimate the ratio of the two PDFs using the RuLSIF algorithm as follows:\r\n\r\n```python\r\nfrom torchdensityratio import rulsif_fit, rulsif_predict\r\n\r\nalpha = 0.1\r\nsigmas = torch.as_tensor([0.1, 0.3, 0.5, 0.7, 1.0], dtype=x.dtype)\r\nlambdas = torch.as_tensor([0.01, 0.02, 0.03, 0.04, 0.05], dtype=x.dtype)\r\nmdl = rulsif_fit(x, y, alpha, sigmas, lambdas, seed=int(rng.integers(0, 2**32)))\r\n```\r\n\r\nFor reproducibility, we can set the `seed` argument to a fixed value. In this way, results are consistent across runs. Then, we evaluate the predicted density ratio on a grid of points, and compute also the true one for comparison:\r\n\r\n```python\r\nn_vals = 200\r\nvals = torch.linspace(0, 2, n_vals, dtype=x.dtype)\r\ngrid = torch.dstack(torch.meshgrid(vals, vals, indexing=\"xy\")).reshape(-1, 2)\r\npredicted = rulsif_predict(mdl, grid).reshape(n_vals, n_vals)\r\n\r\ngrid_np = grid.numpy()\r\npdf_x = mvnorm.pdf(grid_np, mean, cov_x)\r\ntarget = pdf_x / (alpha * pdf_x + (1 - alpha) * mvnorm.pdf(grid_np, mean, cov_y))\r\ntarget = target.reshape(n_vals, n_vals)\r\n```\r\n\r\nFinally, we can visualize the results using `matplotlib`:\r\n\r\n```python\r\nfrom matplotlib import pyplot as plt\r\n\r\nvals_np = vals.numpy()\r\nlevels = [0, 0.5, 1, 1.5, 2, 2.5, 3, 3.5, 4.5]\r\n_, axs = plt.subplots(1, 2, constrained_layout=True)\r\naxs[0].contourf(vals_np, vals_np, target, levels=levels)\r\naxs[0].set_title(\"True density ratio\")\r\naxs[1].contourf(vals_np, vals_np, predicted.numpy(), levels=levels)\r\naxs[1].set_title(\"RuLSIF density ratio\")\r\nplt.show()\r\n```\r\n\r\nThis code produces the following image:\r\n\r\n<div align=\"center\">\r\n <img src=\"https://raw.githubusercontent.com/FilippoAiraldi/torch-density-ratio/master/resources/demo.png\" alt=\"torchdensityratio-demo\" height=\"300\">\r\n</div>\r\n\r\n---\r\n\r\n## Examples\r\n\r\nOur [examples](https://github.com/FilippoAiraldi/torch-density-ratio/tree/master/examples)\r\nsubdirectory contains other example applications of this library.\r\n\r\n---\r\n\r\n## License\r\n\r\nThe repository is provided under the MIT License. See the LICENSE file included with\r\nthis repository.\r\n\r\n---\r\n\r\n## Author\r\n\r\n[Filippo Airaldi](https://www.tudelft.nl/staff/f.airaldi/), PhD Candidate\r\n[f.airaldi@tudelft.nl | filippoairaldi@gmail.com]\r\n\r\n> [Delft Center for Systems and Control](https://www.tudelft.nl/en/me/about/departments/delft-center-for-systems-and-control/)\r\nin [Delft University of Technology](https://www.tudelft.nl/en/)\r\n\r\nCopyright (c) 2025 Filippo Airaldi.\r\n\r\nCopyright notice: Technische Universiteit Delft hereby disclaims all copyright interest\r\nin the program \u201ccsnlp\u201d (Nonlinear Progamming with CasADi) written by the Author(s).\r\nProf. Dr. Ir. Fred van Keulen, Dean of ME.\r\n\r\n---\r\n\r\n## Thanks\r\n\r\nThis repository is heavily inspired by [hoxo-m/densratio_py](https://github.com/hoxo-m/densratio_py) and\r\n[JohnYKiyo/density_ratio_estimation](https://github.com/JohnYKiyo/density_ratio_estimation), please visit their repositories and star them if you find them useful!\r\n\r\n---\r\n\r\n## References\r\n\r\n<a id=\"1\">[1]</a>\r\nYamada, M., Suzuki, T., Kanamori, T., Hachiya, H. and Sugiyama, M., 2013.\r\n[Relative density-ratio estimation for robust distribution comparison](https://ieeexplore.ieee.org/abstract/document/6797650).\r\nNeural computation, 25(5), pp.1324-1370.\r\n\r\n<a id=\"2\">[2]</a>\r\nLiu, S., Yamada, M., Collier, N. and Sugiyama, M., 2013.\r\n[Change-point detection in time-series data by relative density-ratio estimation](https://www.sciencedirect.com/science/article/abs/pii/S0893608013000270).\r\nNeural Networks, 43, pp.72-83.\r\n\r\n<a id=\"3\">[3]</a>\r\nKanamori, T., Hido, S. and Sugiyama, M., 2009.\r\n[A least-squares approach to direct importance estimation](https://jmlr.csail.mit.edu/papers/volume10/kanamori09a/kanamori09a.pdf).\r\nThe Journal of Machine Learning Research, 10, pp.1391-1445\r\n",
"bugtrack_url": null,
"license": null,
"summary": "Density Ratio Estimation with PyTorch",
"version": "1.0.1",
"project_urls": {
"Bug Tracker": "https://github.com/FilippoAiraldi/torch-density-ratio/issues",
"Homepage": "https://github.com/FilippoAiraldi/torch-density-ratio"
},
"split_keywords": [
"density-ratio-estimation",
" machine-learning",
" pytorch"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "4da66cb7e9cf8902d6c30b4a76031d8fcca44dc98a3fb96aee6066908a926c92",
"md5": "172d8d418dc269cef59f115f460f25b8",
"sha256": "78cec1224a3d2a1b2d5951be07235b846ac1d2a1f3b8d1f1cbb63e65191358cb"
},
"downloads": -1,
"filename": "torchdensityratio-1.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "172d8d418dc269cef59f115f460f25b8",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 8382,
"upload_time": "2025-07-11T20:54:17",
"upload_time_iso_8601": "2025-07-11T20:54:17.630979Z",
"url": "https://files.pythonhosted.org/packages/4d/a6/6cb7e9cf8902d6c30b4a76031d8fcca44dc98a3fb96aee6066908a926c92/torchdensityratio-1.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "473f15213e5626171737d8ce2df7ac3543541b45c97972a13cfb86ddb3500e5e",
"md5": "df70109fbf956b82d35d0f6ac40ced43",
"sha256": "89217861b1244c96138273349dd5e49d52ec356f4306e8a52ee781566009dc43"
},
"downloads": -1,
"filename": "torchdensityratio-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "df70109fbf956b82d35d0f6ac40ced43",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 8665,
"upload_time": "2025-07-11T20:54:19",
"upload_time_iso_8601": "2025-07-11T20:54:19.102906Z",
"url": "https://files.pythonhosted.org/packages/47/3f/15213e5626171737d8ce2df7ac3543541b45c97972a13cfb86ddb3500e5e/torchdensityratio-1.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-11 20:54:19",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "FilippoAiraldi",
"github_project": "torch-density-ratio",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "torch",
"specs": [
[
">=",
"2.7.0"
]
]
}
],
"lcname": "torchdensityratio"
}
JSON
