# plenoptic
[](https://pypi.org/project/plenoptic/)
[](https://anaconda.org/conda-forge/plenoptic)
[](https://github.com/plenoptic-org/plenoptic/blob/main/LICENSE)
[](https://github.com/plenoptic-org/plenoptic/actions?query=workflow%3Abuild)
[](https://doi.org/10.5281/zenodo.10151131)
[](https://codecov.io/gh/plenoptic-org/plenoptic)
[](https://mybinder.org/v2/gh/plenoptic-org/plenoptic/1.2.0?filepath=examples)
[](https://www.repostatus.org/#active)
[](https://github.com/astral-sh/ruff)
<picture>
<source media="(prefers-color-scheme: dark)" srcset="Plenoptic_Logo_CMYK_Full_DarkMode_Wide.svg">
<source media="(prefers-color-scheme: light)" srcset="Plenoptic_Logo_CMYK_Full_Wide.svg">
<img alt="plenoptic logo" src="Plenoptic_Logo_CMYK_Full_Wide.svg">
</picture>
`plenoptic` is a python library for model-based synthesis of perceptual stimuli.
For `plenoptic`, models are those of visual[^1] information processing: they
accept an image[^2] as input, perform some computations, and return some output,
which can be mapped to neuronal firing rate, fMRI BOLD response, behavior on
some task, image category, etc. The intended audience is researchers in
neuroscience, psychology, and machine learning. The generated stimuli enable
interpretation of model properties through examination of features that are
enhanced, suppressed, or discarded. More importantly, they can facilitate the
scientific process, through use in further perceptual or neural experiments
aimed at validating or falsifying model predictions.
See our [documentation site](https://docs.plenoptic.org) for more details,
including how to get started!
### Installation
The best way to install `plenoptic` is via `pip`:
``` bash
$ pip install plenoptic
```
or `conda`:
``` bash
$ conda install plenoptic -c conda-forge
```
Our dependencies include [pytorch](https://pytorch.org/) and
[pyrtools](https://pyrtools.readthedocs.io/en/latest/). Installation should take
care of them (along with our other dependencies) automatically, but if you have
an installation problem (especially on a non-Linux operating system), it is
likely that the problem lies with one of those packages. [Open an
issue](https://github.com/plenoptic-org/plenoptic/issues) and we'll
try to help you figure out the problem!
See the [installation
page](https://docs.plenoptic.org/docs/branch/main/install.html) for more details,
including how to set up a virtual environment and jupyter.
### ffmpeg and videos
Several methods in this package generate videos. There are several backends
possible for saving the animations to file, see [matplotlib
documentation](https://matplotlib.org/stable/api/animation_api.html#writer-classes)
for more details. In order convert them to HTML5 for viewing (and thus, to view
in a jupyter notebook), you'll need [ffmpeg](https://ffmpeg.org/download.html)
installed and on your path as well. Depending on your system, this might already
be installed, but if not, the easiest way is probably through [conda]
(https://anaconda.org/conda-forge/ffmpeg): `conda install -c conda-forge
ffmpeg`.
To change the backend, run `matplotlib.rcParams['animation.writer'] = writer`
before calling any of the animate functions. If you try to set that `rcParam`
with a random string, `matplotlib` will tell you the available choices.
## Contents
### Synthesis methods
- Metamers: given a model and a reference image, stochastically generate a new image whose model representation is identical to that of the reference image (a "metamer", as originally defined in the literature on Trichromacy). This method investigates what image features the model disregards entirely.
- Eigendistortions: given a model and a reference image, compute the image perturbation that produces the smallest and largest changes in the model response space. This method investigates the image features the model considers the least and most important.
- Maximal differentiation (MAD) competition: given two metrics that measure distance between images and a reference image, generate pairs of images that optimally differentiate the models. Specifically, synthesize a pair of images that the first model says are equi-distant from the reference while the second model says they are maximally/minimally distant from the reference. Then synthesize a second pair with the roles of the two models reversed. This method allows for efficient comparison of two metrics, highlighting the aspects in which their sensitivities differ.
### Models, Metrics, and Model Components
- Portilla-Simoncelli texture model, which measures the statistical properties of visual textures, here defined as "repeating visual patterns."
- Steerable pyramid, a multi-scale oriented image decomposition. The basis are oriented (steerable) filters, localized in space and frequency. Among other uses, the steerable pyramid serves as a good representation from which to build a primary visual cortex model. See the [pyrtools documentation](https://pyrtools.readthedocs.io/en/latest/index.html) for more details on image pyramids in general and the steerable pyramid in particular.
- Structural Similarity Index (SSIM), is a perceptual similarity metric, returning a number between -1 (totally different) and 1 (identical) reflecting how similar two images are. This is based on the images' luminance, contrast, and structure, which are computed convolutionally across the images.
- Multiscale Structural Similarity Index (MS-SSIM), is a perceptual similarity metric similar to SSIM, except it operates at multiple scales (i.e., spatial frequencies).
- Normalized Laplacian distance, is a perceptual distance metric based on transformations associated with the early visual system: local luminance subtraction and local contrast gain control, at six scales.
## Getting help
We communicate via several channels on Github:
- [Discussions](https://github.com/plenoptic-org/plenoptic/discussions)
is the place to ask usage questions, discuss issues too broad for a
single issue, or show off what you've made with plenoptic.
- If you've come across a bug, open an
[issue](https://github.com/plenoptic-org/plenoptic/issues).
- If you have an idea for an extension or enhancement, please post in the
[ideas
section](https://github.com/plenoptic-org/plenoptic/discussions/categories/ideas)
of discussions first. We'll discuss it there and, if we decide to pursue it,
open an issue to track progress.
- See the [contributing guide](CONTRIBUTING.md) for how to get involved.
In all cases, please follow our [code of conduct](CODE_OF_CONDUCT.md).
## Citing us
If you use `plenoptic` in a published academic article or presentation, please
cite both the code by the DOI as well the JOV paper. If you are not using the
code, but just discussing the project, please cite the paper. You can click on
`Cite this repository` on the right side of the GitHub page to get a copyable
citation for the code, or use the following:
- Code: [](https://doi.org/10.5281/zenodo.10151131)
- Paper:
``` bibtex
@article{duong2023plenoptic,
title={Plenoptic: A platform for synthesizing model-optimized visual stimuli},
author={Duong, Lyndon and Bonnen, Kathryn and Broderick, William and Fiquet, Pierre-{\'E}tienne and Parthasarathy, Nikhil and Yerxa, Thomas and Zhao, Xinyuan and Simoncelli, Eero},
journal={Journal of Vision},
volume={23},
number={9},
pages={5822--5822},
year={2023},
publisher={The Association for Research in Vision and Ophthalmology}
}
```
See the [citation
guide](https://docs.plenoptic.org/docs/branch/main/citation.html) for more
details, including citations for the different synthesis methods and
computational moels included in plenoptic.
## Support
This package is supported by the [Simons Foundation Flatiron Institute's Center
for Computational
Neuroscience](https://www.simonsfoundation.org/flatiron/center-for-computational-neuroscience/).
[^1]: These methods also work with auditory models, such as in [Feather et al., 2019](https://proceedings.neurips.cc/paper_files/paper/2019/hash/ac27b77292582bc293a51055bfc994ee-Abstract.html), though we haven't yet implemented examples. If you're interested, please post in [Discussions](https://github.com/plenoptic-org/plenoptic/discussions)!
[^2]: Here and throughout the documentation, we use "image" to describe the input. The models and metrics that are included in plenoptic are intended to work on images, represented as 4d tensors. However, the synthesis methods should also work on videos (5d tensors), audio (3d tensors) and more! If you have a problem using a tensor with different dimensionality, please [open an issue](https://github.com/plenoptic-org/plenoptic/issues/new?template=bug_report.md)!
Raw data
{
"_id": null,
"home_page": null,
"name": "plenoptic",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "neuroscience, pytorch, visual information processing, machine learning, explainability, computational models",
"author": "Plenoptic authors",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/9c/68/d358ec4bb3ae6967e7fd8b15677df565070f2dc570205cf4991972ed0a9d/plenoptic-1.3.1.tar.gz",
"platform": null,
"description": "# plenoptic\n\n[](https://pypi.org/project/plenoptic/)\n[](https://anaconda.org/conda-forge/plenoptic)\n[](https://github.com/plenoptic-org/plenoptic/blob/main/LICENSE)\n\n[](https://github.com/plenoptic-org/plenoptic/actions?query=workflow%3Abuild)\n[](https://doi.org/10.5281/zenodo.10151131)\n[](https://codecov.io/gh/plenoptic-org/plenoptic)\n[](https://mybinder.org/v2/gh/plenoptic-org/plenoptic/1.2.0?filepath=examples)\n[](https://www.repostatus.org/#active)\n[](https://github.com/astral-sh/ruff)\n\n<picture>\n <source media=\"(prefers-color-scheme: dark)\" srcset=\"Plenoptic_Logo_CMYK_Full_DarkMode_Wide.svg\">\n <source media=\"(prefers-color-scheme: light)\" srcset=\"Plenoptic_Logo_CMYK_Full_Wide.svg\">\n <img alt=\"plenoptic logo\" src=\"Plenoptic_Logo_CMYK_Full_Wide.svg\">\n</picture>\n\n\n`plenoptic` is a python library for model-based synthesis of perceptual stimuli.\nFor `plenoptic`, models are those of visual[^1] information processing: they\naccept an image[^2] as input, perform some computations, and return some output,\nwhich can be mapped to neuronal firing rate, fMRI BOLD response, behavior on\nsome task, image category, etc. The intended audience is researchers in\nneuroscience, psychology, and machine learning. The generated stimuli enable\ninterpretation of model properties through examination of features that are\nenhanced, suppressed, or discarded. More importantly, they can facilitate the\nscientific process, through use in further perceptual or neural experiments\naimed at validating or falsifying model predictions.\n\nSee our [documentation site](https://docs.plenoptic.org) for more details,\nincluding how to get started!\n\n### Installation\n\nThe best way to install `plenoptic` is via `pip`:\n\n``` bash\n$ pip install plenoptic\n```\n\nor `conda`:\n\n``` bash\n$ conda install plenoptic -c conda-forge\n```\n\nOur dependencies include [pytorch](https://pytorch.org/) and\n[pyrtools](https://pyrtools.readthedocs.io/en/latest/). Installation should take\ncare of them (along with our other dependencies) automatically, but if you have\nan installation problem (especially on a non-Linux operating system), it is\nlikely that the problem lies with one of those packages. [Open an\nissue](https://github.com/plenoptic-org/plenoptic/issues) and we'll\ntry to help you figure out the problem!\n\nSee the [installation\npage](https://docs.plenoptic.org/docs/branch/main/install.html) for more details,\nincluding how to set up a virtual environment and jupyter.\n\n### ffmpeg and videos\n\nSeveral methods in this package generate videos. There are several backends\npossible for saving the animations to file, see [matplotlib\ndocumentation](https://matplotlib.org/stable/api/animation_api.html#writer-classes)\nfor more details. In order convert them to HTML5 for viewing (and thus, to view\nin a jupyter notebook), you'll need [ffmpeg](https://ffmpeg.org/download.html)\ninstalled and on your path as well. Depending on your system, this might already\nbe installed, but if not, the easiest way is probably through [conda]\n(https://anaconda.org/conda-forge/ffmpeg): `conda install -c conda-forge\nffmpeg`.\n\nTo change the backend, run `matplotlib.rcParams['animation.writer'] = writer`\nbefore calling any of the animate functions. If you try to set that `rcParam`\nwith a random string, `matplotlib` will tell you the available choices.\n\n## Contents\n\n### Synthesis methods\n\n\n\n- Metamers: given a model and a reference image, stochastically generate a new image whose model representation is identical to that of the reference image (a \"metamer\", as originally defined in the literature on Trichromacy). This method investigates what image features the model disregards entirely.\n- Eigendistortions: given a model and a reference image, compute the image perturbation that produces the smallest and largest changes in the model response space. This method investigates the image features the model considers the least and most important.\n- Maximal differentiation (MAD) competition: given two metrics that measure distance between images and a reference image, generate pairs of images that optimally differentiate the models. Specifically, synthesize a pair of images that the first model says are equi-distant from the reference while the second model says they are maximally/minimally distant from the reference. Then synthesize a second pair with the roles of the two models reversed. This method allows for efficient comparison of two metrics, highlighting the aspects in which their sensitivities differ.\n\n\n### Models, Metrics, and Model Components\n\n- Portilla-Simoncelli texture model, which measures the statistical properties of visual textures, here defined as \"repeating visual patterns.\"\n- Steerable pyramid, a multi-scale oriented image decomposition. The basis are oriented (steerable) filters, localized in space and frequency. Among other uses, the steerable pyramid serves as a good representation from which to build a primary visual cortex model. See the [pyrtools documentation](https://pyrtools.readthedocs.io/en/latest/index.html) for more details on image pyramids in general and the steerable pyramid in particular.\n- Structural Similarity Index (SSIM), is a perceptual similarity metric, returning a number between -1 (totally different) and 1 (identical) reflecting how similar two images are. This is based on the images' luminance, contrast, and structure, which are computed convolutionally across the images.\n- Multiscale Structural Similarity Index (MS-SSIM), is a perceptual similarity metric similar to SSIM, except it operates at multiple scales (i.e., spatial frequencies).\n- Normalized Laplacian distance, is a perceptual distance metric based on transformations associated with the early visual system: local luminance subtraction and local contrast gain control, at six scales.\n\n\n## Getting help\n\nWe communicate via several channels on Github:\n\n- [Discussions](https://github.com/plenoptic-org/plenoptic/discussions)\n is the place to ask usage questions, discuss issues too broad for a\n single issue, or show off what you've made with plenoptic.\n- If you've come across a bug, open an\n [issue](https://github.com/plenoptic-org/plenoptic/issues).\n- If you have an idea for an extension or enhancement, please post in the\n [ideas\n section](https://github.com/plenoptic-org/plenoptic/discussions/categories/ideas)\n of discussions first. We'll discuss it there and, if we decide to pursue it,\n open an issue to track progress.\n- See the [contributing guide](CONTRIBUTING.md) for how to get involved.\n\nIn all cases, please follow our [code of conduct](CODE_OF_CONDUCT.md).\n\n## Citing us\n\nIf you use `plenoptic` in a published academic article or presentation, please\ncite both the code by the DOI as well the JOV paper. If you are not using the\ncode, but just discussing the project, please cite the paper. You can click on\n`Cite this repository` on the right side of the GitHub page to get a copyable\ncitation for the code, or use the following:\n\n- Code: [](https://doi.org/10.5281/zenodo.10151131)\n- Paper:\n ``` bibtex\n @article{duong2023plenoptic,\n title={Plenoptic: A platform for synthesizing model-optimized visual stimuli},\n author={Duong, Lyndon and Bonnen, Kathryn and Broderick, William and Fiquet, Pierre-{\\'E}tienne and Parthasarathy, Nikhil and Yerxa, Thomas and Zhao, Xinyuan and Simoncelli, Eero},\n journal={Journal of Vision},\n volume={23},\n number={9},\n pages={5822--5822},\n year={2023},\n publisher={The Association for Research in Vision and Ophthalmology}\n }\n ```\n\nSee the [citation\nguide](https://docs.plenoptic.org/docs/branch/main/citation.html) for more\ndetails, including citations for the different synthesis methods and\ncomputational moels included in plenoptic.\n\n## Support\n\nThis package is supported by the [Simons Foundation Flatiron Institute's Center\nfor Computational\nNeuroscience](https://www.simonsfoundation.org/flatiron/center-for-computational-neuroscience/).\n\n\n\n[^1]: These methods also work with auditory models, such as in [Feather et al., 2019](https://proceedings.neurips.cc/paper_files/paper/2019/hash/ac27b77292582bc293a51055bfc994ee-Abstract.html), though we haven't yet implemented examples. If you're interested, please post in [Discussions](https://github.com/plenoptic-org/plenoptic/discussions)!\n\n[^2]: Here and throughout the documentation, we use \"image\" to describe the input. The models and metrics that are included in plenoptic are intended to work on images, represented as 4d tensors. However, the synthesis methods should also work on videos (5d tensors), audio (3d tensors) and more! If you have a problem using a tensor with different dimensionality, please [open an issue](https://github.com/plenoptic-org/plenoptic/issues/new?template=bug_report.md)!\n",
"bugtrack_url": null,
"license": null,
"summary": "Python library for model-based stimulus synthesis.",
"version": "1.3.1",
"project_urls": {
"Documentation": "https://docs.plenoptic.org",
"Download": "https://zenodo.org/doi/10.5281/zenodo.10151130",
"Homepage": "https://github.com/plenoptic-org/plenoptic"
},
"split_keywords": [
"neuroscience",
" pytorch",
" visual information processing",
" machine learning",
" explainability",
" computational models"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "4f34bc6873755f07d8d057d2dcb941d66f767c254fa3a35b7cc1cd55b5513bdd",
"md5": "29b47365dcd6dee7c623bb13dce1f8e4",
"sha256": "86cabbf59964578432dd259a566e802897758c306d5f5171be739182f03db85e"
},
"downloads": -1,
"filename": "plenoptic-1.3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "29b47365dcd6dee7c623bb13dce1f8e4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 420061,
"upload_time": "2025-10-15T19:38:09",
"upload_time_iso_8601": "2025-10-15T19:38:09.219968Z",
"url": "https://files.pythonhosted.org/packages/4f/34/bc6873755f07d8d057d2dcb941d66f767c254fa3a35b7cc1cd55b5513bdd/plenoptic-1.3.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "9c68d358ec4bb3ae6967e7fd8b15677df565070f2dc570205cf4991972ed0a9d",
"md5": "1d37dd00969fd5468b1ec47eeb071359",
"sha256": "6e123a21cfeb92a71fdabb97ed1af5117fac1442ed9cda2bd3016e9ebab9255a"
},
"downloads": -1,
"filename": "plenoptic-1.3.1.tar.gz",
"has_sig": false,
"md5_digest": "1d37dd00969fd5468b1ec47eeb071359",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 4937046,
"upload_time": "2025-10-15T19:38:10",
"upload_time_iso_8601": "2025-10-15T19:38:10.871601Z",
"url": "https://files.pythonhosted.org/packages/9c/68/d358ec4bb3ae6967e7fd8b15677df565070f2dc570205cf4991972ed0a9d/plenoptic-1.3.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-15 19:38:10",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "plenoptic-org",
"github_project": "plenoptic",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "plenoptic"
}
JSON
