# Pytorch Principal Component Analysis (PCA)
Principal Component Anlaysis (PCA) in PyTorch. The intention is to provide a
simple and easy to use implementation of PCA in PyTorch, the most similar to
the `sklearn`'s PCA as possible (in terms of API and, of course, output).
Plus, this implementation is **fully differentiable and faster** (thanks to GPU parallelization)!
[](https://pypi.org/project/torch_pca/)
[](https://github.com/valentingol)
[](https://github.com/valentingol)
[](https://github.com/charliermarsh/ruff)
[](https://github.com/psf/black)
[](https://github.com/valentingol/Dinosor/actions/workflows/ruff.yaml)
[](https://github.com/valentingol/Dinosor/actions/workflows/flake.yaml)
[](https://github.com/valentingol/Dinosor/actions/workflows/mypy.yaml)
[](https://github.com/valentingol/torch_pca/actions/workflows/pylint.yaml)
[](https://github.com/valentingol/torch_pca/actions/workflows/tests.yaml)
[](https://github.com/valentingol/torch_pca/actions/workflows/tests.yaml)
[](https://torch-pca.readthedocs.io/en/latest/?badge=latest)
## Links
Github repository: https://github.com/valentingol/torch_pca
Pypi project: https://pypi.org/project/torch_pca/
Documentation: https://torch-pca.readthedocs.io/en/latest/
## Installation
Simply install it with pip:
```bash
pip install torch-pca
```
## How to use
Exactly like `sklearn.decomposition.PCA` but it uses PyTorch tensors as input and output!
```python
from torch_cpa import PCA
# Create like sklearn.decomposition.PCA, e.g.:
pca_model = PCA(n_components=None, svd_solver='full')
# Use like sklearn.decomposition.PCA, e.g.:
>>> new_train_data = pca_model.fit_transform(train_data)
>>> new_test_data = pca_model.transform(test_data)
>>> print(pca.explained_variance_ratio_)
[0.756, 0.142, 0.062, ...]
```
More details and features in the [API documentation](https://torch-pca.readthedocs.io/en/latest/api.html#torch_pca.pca_main.PCA).
## Gradient backward pass
Use the pytorch framework allows the automatic differentiation of the PCA!
The PCA transform method is always differentiable so it is always possible to
compute gradient like that:
```python
pca = PCA()
for ep in range(n_epochs):
optimizer.zero_grad()
out = neural_net(inputs)
with torch.no_grad():
pca.fit(out)
out = pca.transform(out)
loss = loss_fn(out, targets)
loss.backward()
```
If you want to compute the gradient over the full PCA model (including the
fitted `pca.n_components`), you can do it by using the "full" SVD solver
and removing the part of the `fit` method that enforce the deterministic
output by passing `determinist=False` in `fit` or `fit_transform` method.
This part sort the components using the singular values and change their sign
accordingly so it is not differentiable by nature but may be not necessary if
you don't care about the determinism of the output:
```python
pca = PCA(svd_solver="full")
for ep in range(n_epochs):
optimizer.zero_grad()
out = neural_net(inputs)
out = pca.fit_transform(out, determinist=False)
loss = loss_fn(out, targets)
loss.backward()
```
## Comparison of execution time with sklearn's PCA
As we can see below the PyTorch PCA is faster than sklearn's PCA, in all the
configs tested with the parameter by default (for each PCA model):
## Implemented features
- [x] `fit`, `transform`, `fit_transform` methods.
- [x] All attributes from sklean's PCA are available: `explained_variance_(ratio_)`,
`singular_values_`, `components_`, `mean_`, `noise_variance_`, ...
- [x] Full SVD solver
- [x] SVD by covariance matrix solver
- [x] Randomized SVD solver
- [x] (absent from sklearn) Decide how to center the input data in `transform` method
(default is like sklearn's PCA)
- [x] Find number of components with explained variance proportion
- [x] Automatically find number of components with MLE
- [x] `inverse_transform` method
- [x] Whitening option
- [x] `get_covariance` method
- [x] `get_precision` method and `score`/`score_samples` methods
## To be implemented
- [ ] Support sparse matrices with ARPACK solver
## Contributing
Feel free to contribute to this project! Just fork it and make an issue or a pull request.
See the [CONTRIBUTING.md](CONTRIBUTING.md) file for more information.
Raw data
{
"_id": null,
"home_page": null,
"name": "torch-pca",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "principal component analysis, PCA, pytorch, torch",
"author": null,
"author_email": "Valentin Goldite <valentin.goldite@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/4f/e7/3ac60c5d2d8249782aca2bd2433683377d1038ce99b132380ef4af86ed59/torch_pca-0.1.1.tar.gz",
"platform": null,
"description": "# Pytorch Principal Component Analysis (PCA)\n\nPrincipal Component Anlaysis (PCA) in PyTorch. The intention is to provide a\nsimple and easy to use implementation of PCA in PyTorch, the most similar to\nthe `sklearn`'s PCA as possible (in terms of API and, of course, output).\nPlus, this implementation is **fully differentiable and faster** (thanks to GPU parallelization)!\n\n[](https://pypi.org/project/torch_pca/)\n\n\n\n[](https://github.com/valentingol)\n[](https://github.com/valentingol)\n\n[](https://github.com/charliermarsh/ruff)\n[](https://github.com/psf/black)\n\n[](https://github.com/valentingol/Dinosor/actions/workflows/ruff.yaml)\n[](https://github.com/valentingol/Dinosor/actions/workflows/flake.yaml)\n[](https://github.com/valentingol/Dinosor/actions/workflows/mypy.yaml)\n[](https://github.com/valentingol/torch_pca/actions/workflows/pylint.yaml)\n\n[](https://github.com/valentingol/torch_pca/actions/workflows/tests.yaml)\n[](https://github.com/valentingol/torch_pca/actions/workflows/tests.yaml)\n[](https://torch-pca.readthedocs.io/en/latest/?badge=latest)\n\n## Links\n\nGithub repository: https://github.com/valentingol/torch_pca\n\nPypi project: https://pypi.org/project/torch_pca/\n\nDocumentation: https://torch-pca.readthedocs.io/en/latest/\n\n## Installation\n\nSimply install it with pip:\n\n```bash\npip install torch-pca\n```\n\n## How to use\n\nExactly like `sklearn.decomposition.PCA` but it uses PyTorch tensors as input and output!\n\n```python\nfrom torch_cpa import PCA\n\n# Create like sklearn.decomposition.PCA, e.g.:\npca_model = PCA(n_components=None, svd_solver='full')\n\n# Use like sklearn.decomposition.PCA, e.g.:\n>>> new_train_data = pca_model.fit_transform(train_data)\n>>> new_test_data = pca_model.transform(test_data)\n>>> print(pca.explained_variance_ratio_)\n[0.756, 0.142, 0.062, ...]\n```\n\nMore details and features in the [API documentation](https://torch-pca.readthedocs.io/en/latest/api.html#torch_pca.pca_main.PCA).\n\n## Gradient backward pass\n\nUse the pytorch framework allows the automatic differentiation of the PCA!\n\nThe PCA transform method is always differentiable so it is always possible to\ncompute gradient like that:\n\n```python\npca = PCA()\nfor ep in range(n_epochs):\n optimizer.zero_grad()\n out = neural_net(inputs)\n with torch.no_grad():\n pca.fit(out)\n out = pca.transform(out)\n loss = loss_fn(out, targets)\n loss.backward()\n```\n\nIf you want to compute the gradient over the full PCA model (including the\nfitted `pca.n_components`), you can do it by using the \"full\" SVD solver\nand removing the part of the `fit` method that enforce the deterministic\noutput by passing `determinist=False` in `fit` or `fit_transform` method.\nThis part sort the components using the singular values and change their sign\naccordingly so it is not differentiable by nature but may be not necessary if\nyou don't care about the determinism of the output:\n\n```python\npca = PCA(svd_solver=\"full\")\nfor ep in range(n_epochs):\n optimizer.zero_grad()\n out = neural_net(inputs)\n out = pca.fit_transform(out, determinist=False)\n loss = loss_fn(out, targets)\n loss.backward()\n```\n\n## Comparison of execution time with sklearn's PCA\n\nAs we can see below the PyTorch PCA is faster than sklearn's PCA, in all the\nconfigs tested with the parameter by default (for each PCA model):\n\n\n\n## Implemented features\n\n- [x] `fit`, `transform`, `fit_transform` methods.\n- [x] All attributes from sklean's PCA are available: `explained_variance_(ratio_)`,\n `singular_values_`, `components_`, `mean_`, `noise_variance_`, ...\n- [x] Full SVD solver\n- [x] SVD by covariance matrix solver\n- [x] Randomized SVD solver\n- [x] (absent from sklearn) Decide how to center the input data in `transform` method\n (default is like sklearn's PCA)\n- [x] Find number of components with explained variance proportion\n- [x] Automatically find number of components with MLE\n- [x] `inverse_transform` method\n- [x] Whitening option\n- [x] `get_covariance` method\n- [x] `get_precision` method and `score`/`score_samples` methods\n\n## To be implemented\n\n- [ ] Support sparse matrices with ARPACK solver\n\n## Contributing\n\nFeel free to contribute to this project! Just fork it and make an issue or a pull request.\n\nSee the [CONTRIBUTING.md](CONTRIBUTING.md) file for more information.\n",
"bugtrack_url": null,
"license": null,
"summary": "PyTorch implementation of PCA (similar to sklearn PCA).",
"version": "0.1.1",
"project_urls": {
"Source": "https://github.com/valentingol/torch_pca"
},
"split_keywords": [
"principal component analysis",
" pca",
" pytorch",
" torch"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "de5d25c6ffc6fa525b1d43d44d99b649fecc595e0ba984d568c16ad82f6560c3",
"md5": "6246bb604c755fc28f8197cb9c32e709",
"sha256": "b605ece2d88fcf70e877b8b33d83b47d13dab9a0adb7a0ce57516350c907bfff"
},
"downloads": -1,
"filename": "torch_pca-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6246bb604c755fc28f8197cb9c32e709",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 12940,
"upload_time": "2024-06-24T08:52:22",
"upload_time_iso_8601": "2024-06-24T08:52:22.706387Z",
"url": "https://files.pythonhosted.org/packages/de/5d/25c6ffc6fa525b1d43d44d99b649fecc595e0ba984d568c16ad82f6560c3/torch_pca-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4fe73ac60c5d2d8249782aca2bd2433683377d1038ce99b132380ef4af86ed59",
"md5": "a87e064fa929dc2c8bda68d61ffa4557",
"sha256": "04362e49d56a767875b4a8aeb7d513976c64163b3729f26cb9cdc948bdc504fb"
},
"downloads": -1,
"filename": "torch_pca-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "a87e064fa929dc2c8bda68d61ffa4557",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 91592,
"upload_time": "2024-06-24T08:52:24",
"upload_time_iso_8601": "2024-06-24T08:52:24.527566Z",
"url": "https://files.pythonhosted.org/packages/4f/e7/3ac60c5d2d8249782aca2bd2433683377d1038ce99b132380ef4af86ed59/torch_pca-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-06-24 08:52:24",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "valentingol",
"github_project": "torch_pca",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"requirements": [],
"lcname": "torch-pca"
}
JSON
