[](https://www.repostatus.org/#active)
[](https://github.com/ArashAkbarinia/osculari)
[](https://pypi.org/project/osculari/)
[](https://pypi.org/project/osculari/)
[](https://osculari.readthedocs.io/en/latest/?badge=latest)
[](https://github.com/ArashAkbarinia/osculari)
[](https://app.codecov.io/gh/ArashAkbarinia/osculari)
[](https://pytorch.org/get-started/locally/)
[](LICENSE)
[](https://zenodo.org/doi/10.5281/zenodo.10214005)
Exploring artificial neural networks with psychophysical experiments.
## Overview
The `osculari` package provides an easy interface for different techniques to explore and interpret
the internal presentation of deep neural networks.
- Supporting the following pretrained models:
* All classification and segmentation networks
from [PyTorch's official website](https://pytorch.org/vision/stable/models.html).
* All OpenAI [CLIP](https://github.com/openai/CLIP) language-vision models.
* All [Taskonomy](http://taskonomy.stanford.edu/) networks.
- Managing convolution and transformer architectures.
- Allowing to readout the network at any given depth.
- Training a linear classifier on top of the extract features from any network/layer.
- Experimenting with 2AFC and 4AFC paradigms.
At a granular level, Kornia is a library that consists of the following components:
| **Module** | **Description** |
|-----------------------------------------------------------------------------------------|----------------------------------------------------------------------------------|
| [osculari](https://osculari.readthedocs.io/en/latest/index.html) | Open source library to explore and interpret pretrained deep neural networks. |
| [osculari.datasets](https://osculari.readthedocs.io/en/latest/osculari.datasets.html) | A module to create datasets and dataloaders to train and test linear probes. |
| [osculari.models](https://osculari.readthedocs.io/en/latest/osculari.models.html) | A module to readout pretrained networks and add linear layers on top of them. |
| [osculari.paradigms](https://osculari.readthedocs.io/en/latest/osculari.paradigms.html) | A module to implement psychophysical paradigms to experiment with deep networks. |
## Installation
### From pip
```bash
pip install osculari
```
<details>
<summary>Alternative installation options</summary>
### From source with symbolic links:
```bash
pip install -e .
```
### From source using pip:
```bash
pip install git+https://github.com/ArashAkbarinia/osculari
```
</details>
## Examples
Please check the [example page](https://osculari.readthedocs.io/en/latest/examples.html) of our
documentation with many notebooks that can also be executed on Google Colab.
<details>
<summary>Quick start</summary>
### Pretrained features
Let's create a linear classifier on top of the extracted features from a pretrained network to
perform a binary classification task (i.e., 2AFC – two-alternative-force-choice). This is easily
achieved by calling the `paradigm_2afc_merge_concatenate` from the `osculari.models` module.
``` python
architecture = 'resnet50' # network's architecture
weights = 'resnet50' # the pretrained weights
img_size = 224 # network's input size
layer = 'block0' # the readout layer
readout_kwargs = {
'architecture': architecture,
'weights': weights,
'layers': layer,
'img_size': img_size,
}
net_2afc = osculari.models.paradigm_2afc_merge_concatenate(**readout_kwargs)
```
### Datasets
The `osculari.datasets` module provides datasets that are generated randomly on the fly with
flexible properties that can be dynamically changed based on the experiment of interest.
For instance, by passing a `appearance_fun` to the `ShapeAppearanceDataset` class, we can
dynamically merge foreground masks with background images to generate stimuli of interest.
```python
def appearance_fun(foregrounds, backgrounds):
# implementing the required appearance (colour, texture, etc.) on foreground and merging
# to background.
return merged_imgs, ground_truth
num_samples = 1000 # the number of random samples generated in the dataset
num_imgs = net_2afc.input_nodes # the number of images in each sample
background = 128 # the background type
dataset = osculari.datasets.geometrical_shapes.ShapeAppearanceDataset(
num_samples, num_imgs, img_size, background, appearance_fun,
unique_bg=True, transform=net_2afc.preprocess_transform()
)
```
### Linear probe
The `osculari.paradigms` module implements a set of psychophysical paradigms. The `train_linear_probe`
function trains the network on a dataset following the paradigm passed to the function.
```python
# experiment-dependent function to train on an epoch of data
epoch_fun = osculari.paradigms.forced_choice.epoch_loop
# calling the generic train_linear_probe function
training_log = osculari.paradigms.paradigm_utils.train_linear_probe(
net_2afc, dataset, epoch_fun, './osculari_test/'
)
```
### Psychophysical experiment
The `osculari.paradigms` module also implements a set of psychophysical experiments similar to the
experiments conducted with human participants. In this example, we use the `staircase` function
to gradually measure the network's sensitivity.
```python
# experiment-dependent function to test an epoch of data
test_epoch_fun = osculari.paradigms.forced_choice.test_dataset
# the test dataset implementing desired stimuli.
class TestDataset(TorchDataset):
def __getitem__(self, idx):
return stimuli
test_log = osculari.paradigms.staircase(
net_2afc, test_epoch_fun, TestDataset(), low_val=0, high_val=1
)
```
</details>
## Contribution
We welcome all contributions to the project that extend or improve code and/or documentation!
Please read the [CONTRIBUTING](CONTRIBUTING.md) page and follow the [Code of Conduct](CODE_OF_CONDUCT.md).
Raw data
{
"_id": null,
"home_page": "",
"name": "osculari",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "deep-neural-networks,psychophysics,cognitive-neuroscience,linear-probing,explainable-ai,interpreting-models",
"author": "",
"author_email": "Arash Akbarinia <akbarinia.arash@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/1f/73/4db359d60f147db220e2d43782dacb23219f241f60040b40985a14a12d23/osculari-0.0.4.tar.gz",
"platform": null,
"description": "\n\n[](https://www.repostatus.org/#active)\n[](https://github.com/ArashAkbarinia/osculari)\n[](https://pypi.org/project/osculari/)\n[](https://pypi.org/project/osculari/)\n[](https://osculari.readthedocs.io/en/latest/?badge=latest)\n[](https://github.com/ArashAkbarinia/osculari)\n[](https://app.codecov.io/gh/ArashAkbarinia/osculari)\n[](https://pytorch.org/get-started/locally/)\n[](LICENSE)\n[](https://zenodo.org/doi/10.5281/zenodo.10214005)\n\nExploring artificial neural networks with psychophysical experiments.\n\n## Overview\n\nThe `osculari` package provides an easy interface for different techniques to explore and interpret\nthe internal presentation of deep neural networks.\n\n- Supporting the following pretrained models:\n * All classification and segmentation networks\n from [PyTorch's official website](https://pytorch.org/vision/stable/models.html).\n * All OpenAI [CLIP](https://github.com/openai/CLIP) language-vision models.\n * All [Taskonomy](http://taskonomy.stanford.edu/) networks.\n- Managing convolution and transformer architectures.\n- Allowing to readout the network at any given depth.\n- Training a linear classifier on top of the extract features from any network/layer.\n- Experimenting with 2AFC and 4AFC paradigms.\n\nAt a granular level, Kornia is a library that consists of the following components:\n\n| **Module** | **Description** |\n|-----------------------------------------------------------------------------------------|----------------------------------------------------------------------------------|\n| [osculari](https://osculari.readthedocs.io/en/latest/index.html) | Open source library to explore and interpret pretrained deep neural networks. |\n| [osculari.datasets](https://osculari.readthedocs.io/en/latest/osculari.datasets.html) | A module to create datasets and dataloaders to train and test linear probes. |\n| [osculari.models](https://osculari.readthedocs.io/en/latest/osculari.models.html) | A module to readout pretrained networks and add linear layers on top of them. |\n| [osculari.paradigms](https://osculari.readthedocs.io/en/latest/osculari.paradigms.html) | A module to implement psychophysical paradigms to experiment with deep networks. |\n\n## Installation\n\n### From pip\n\n```bash\npip install osculari\n```\n\n<details>\n <summary>Alternative installation options</summary>\n\n### From source with symbolic links:\n\n```bash\npip install -e .\n```\n\n### From source using pip:\n\n```bash\npip install git+https://github.com/ArashAkbarinia/osculari\n```\n\n</details>\n\n## Examples\n\nPlease check the [example page](https://osculari.readthedocs.io/en/latest/examples.html) of our\ndocumentation with many notebooks that can also be executed on Google Colab.\n\n<details>\n <summary>Quick start</summary>\n\n### Pretrained features\n\nLet's create a linear classifier on top of the extracted features from a pretrained network to \nperform a binary classification task (i.e., 2AFC \u2013 two-alternative-force-choice). This is easily \nachieved by calling the `paradigm_2afc_merge_concatenate` from the `osculari.models` module.\n\n``` python\n\narchitecture = 'resnet50' # network's architecture\nweights = 'resnet50' # the pretrained weights\nimg_size = 224 # network's input size\nlayer = 'block0' # the readout layer\nreadout_kwargs = {\n 'architecture': architecture, \n 'weights': weights,\n 'layers': layer,\n 'img_size': img_size,\n}\nnet_2afc = osculari.models.paradigm_2afc_merge_concatenate(**readout_kwargs)\n\n```\n\n### Datasets\n\nThe `osculari.datasets` module provides datasets that are generated randomly on the fly with\nflexible properties that can be dynamically changed based on the experiment of interest.\nFor instance, by passing a `appearance_fun` to the `ShapeAppearanceDataset` class, we can \ndynamically merge foreground masks with background images to generate stimuli of interest.\n\n```python\n\ndef appearance_fun(foregrounds, backgrounds):\n # implementing the required appearance (colour, texture, etc.) on foreground and merging\n # to background.\n return merged_imgs, ground_truth\n\nnum_samples = 1000 # the number of random samples generated in the dataset\nnum_imgs = net_2afc.input_nodes # the number of images in each sample\nbackground = 128 # the background type\ndataset = osculari.datasets.geometrical_shapes.ShapeAppearanceDataset(\n num_samples, num_imgs, img_size, background, appearance_fun,\n unique_bg=True, transform=net_2afc.preprocess_transform()\n)\n\n```\n\n### Linear probe\n\nThe `osculari.paradigms` module implements a set of psychophysical paradigms. The `train_linear_probe`\nfunction trains the network on a dataset following the paradigm passed to the function.\n\n```python\n\n# experiment-dependent function to train on an epoch of data\nepoch_fun = osculari.paradigms.forced_choice.epoch_loop\n# calling the generic train_linear_probe function\ntraining_log = osculari.paradigms.paradigm_utils.train_linear_probe(\n net_2afc, dataset, epoch_fun, './osculari_test/'\n)\n\n```\n\n### Psychophysical experiment\n\nThe `osculari.paradigms` module also implements a set of psychophysical experiments similar to the\nexperiments conducted with human participants. In this example, we use the `staircase` function\nto gradually measure the network's sensitivity.\n\n```python\n\n# experiment-dependent function to test an epoch of data\ntest_epoch_fun = osculari.paradigms.forced_choice.test_dataset\n# the test dataset implementing desired stimuli.\nclass TestDataset(TorchDataset):\n def __getitem__(self, idx):\n return stimuli\n\ntest_log = osculari.paradigms.staircase(\n net_2afc, test_epoch_fun, TestDataset(), low_val=0, high_val=1\n)\n\n```\n\n</details>\n\n\n## Contribution\n\nWe welcome all contributions to the project that extend or improve code and/or documentation!\nPlease read the [CONTRIBUTING](CONTRIBUTING.md) page and follow the [Code of Conduct](CODE_OF_CONDUCT.md).\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "Open source library to explore artificial neural networks with psychophysical experiments.",
"version": "0.0.4",
"project_urls": {
"Bug Tracker": "https://github.com/ArashAkbarinia/osculari/issues",
"Documentation": "https://osculari.readthedocs.io/en/latest",
"Download": "https://github.com/ArashAkbarinia/osculari/releases",
"Homepage": "https://github.com/ArashAkbarinia/osculari",
"Source Code": "https://github.com/ArashAkbarinia/osculari"
},
"split_keywords": [
"deep-neural-networks",
"psychophysics",
"cognitive-neuroscience",
"linear-probing",
"explainable-ai",
"interpreting-models"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "67334301158aec82f9d153f7cd530bdac110dae26f58d2871303e2f9df917167",
"md5": "ee6ba1f62a1c3941507f2655432b93f6",
"sha256": "1d815cf032b845bffc94e561f0558c8ce86116bba47adb8f4ae7a6fa599b87d0"
},
"downloads": -1,
"filename": "osculari-0.0.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ee6ba1f62a1c3941507f2655432b93f6",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 34483,
"upload_time": "2023-12-21T20:31:40",
"upload_time_iso_8601": "2023-12-21T20:31:40.039842Z",
"url": "https://files.pythonhosted.org/packages/67/33/4301158aec82f9d153f7cd530bdac110dae26f58d2871303e2f9df917167/osculari-0.0.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "1f734db359d60f147db220e2d43782dacb23219f241f60040b40985a14a12d23",
"md5": "de5b18a5efe1840e2d8fe0535066219f",
"sha256": "8a32dc30fb5560d98fe9469eccfb3e593a83348b08469b04cb4f21c65b09243a"
},
"downloads": -1,
"filename": "osculari-0.0.4.tar.gz",
"has_sig": false,
"md5_digest": "de5b18a5efe1840e2d8fe0535066219f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 31614,
"upload_time": "2023-12-21T20:31:42",
"upload_time_iso_8601": "2023-12-21T20:31:42.976111Z",
"url": "https://files.pythonhosted.org/packages/1f/73/4db359d60f147db220e2d43782dacb23219f241f60040b40985a14a12d23/osculari-0.0.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-12-21 20:31:42",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ArashAkbarinia",
"github_project": "osculari",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "osculari"
}
JSON
