# APalysis
## Introduction
This project aims to:
1. Visualize the raw activations and latent space activations of a single or a group of images for any neural networks.
2. Compare individual and class activation pathways for different datasets.
3. Apply visual analytics on the classification ambiguousness of the classes in a dataset.
4. Visual analytics to identify opportunities for pruning of neural networks.
## Quickstart (w/ Docker)
You can run the following the start the whole dockerized application quickly in localhost.
```bash
docker run -p 8000:8000/tcp rahatzamancse/apalysis
```
And open `http://localhost:8000` in your browser.
## Installation
The project is available on PYPI. So you can install using
```bash
pip install apalysis
```
You will need a running redis server for the project to work. You can install redis using
```bash
# Install redis
sudo apt install redis-server # For debian-based distros
# sudo pacman -S redis # For arch
# Run redis
redis-server --daemonize yes
```
You can also install redis using docker. You can find the docker image [here](https://hub.docker.com/_/redis).
## Usage
### Tensorflow
```python
# Import everything
from activation_pathway_analysis_backend.apalysis_tf import APAnalysisTensorflowModel
import tensorflow as tf
import tensorflow_datasets as tfds
import numpy as np
from nltk.corpus import wordnet as wn
# Load the tensorflow model
model = tf.keras.applications.vgg16.VGG16(weights='imagenet')
model.compile(loss="categorical_crossentropy", optimizer="adam")
# Load the dataset
# The dataset needs to be in tensorflow_datasets format
ds, info = tfds.load(
'imagenette/320px-v2',
shuffle_files=False,
with_info=True,
as_supervised=True,
batch_size=None,
)
# For classification, the labels is just a list of names for each encoded class
labels = names=list(map(lambda l: wn.synset_from_pos_and_offset(
l[0], int(l[1:])).name(), info.features['label'].names))
dataset = ds['train']
# To feed the dataset into the model, we need to provide a preprocessing function
vgg16_input_shape = tf.keras.applications.vgg16.VGG16().input.shape[1:3].as_list()
@tf.function
def preprocess(x, y):
x = tf.image.resize(x, vgg16_input_shape, method=tf.image.ResizeMethod.BILINEAR)
x = tf.keras.applications.vgg16.preprocess_input(x)
return x, y
# We also need to provide a preprocessing inverse function to convert the preprocessed image back to the original image
# This is used to display the original image in the frontend
# This will be automated in the future
def preprocess_inv(x, y):
x = x.squeeze(0)
# Again adding the mean pixel values
x[:, :, 0] += 103.939
x[:, :, 1] += 116.779
x[:, :, 2] += 123.68
x = x[:, :, ::-1]
x = np.clip(x, 0, 255).astype('uint8')
return x, y
# Create the server and run it
server = APAnalysisTensorflowModel(
model=model,
dataset=dataset,
label_names=labels,
preprocess=preprocess,
preprocess_inverse=preprocess_inv,
log_level="info",
)
server.run_server(host="localhost", port=8000)
```
### PyTorch
Very similar to the above example.
```python
# Imports
from activation_pathway_analysis_backend.apalysis_torch import APAnalysisTorchModel
import torch
import torchvision.models as models
import torchvision.datasets as datasets
import torchvision.transforms as transforms
from torch.utils.data import DataLoader
import numpy as np
from nltk.corpus import wordnet as wn
# Load the model
model = models.vgg16(pretrained=True)
loss_fn = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters())
# We do not need any preprocessing function to provide to the server for pytorch. we can provide it to the dataset instead.
transform = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize((0.1307,), (0.3081,)) # MNIST mean and std
])
dataset = datasets.MNIST('/run/media/insane/SSD Games/Pytorch', train=True, download=True, transform=transform)
# Start the server
server = APAnalysisTorchModel(
model=model,
input_shape=(1, 3, 224, 224),
dataset=dataset,
label_names=labels,
log_level="info",
)
server.run_server(host="localhost", port=8000)
```
Raw data
{
"_id": null,
"home_page": "https://github.com/rahatzamancse/APalysis",
"name": "APalysis",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10,<3.11",
"maintainer_email": "",
"keywords": "APalysis",
"author": "Rahat Zaman",
"author_email": "rahatzamancse@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/cf/a7/614dd437b7abfc793615c44214793eab3ca0995184f5eacce6b777f88cd1/apalysis-0.0.2.tar.gz",
"platform": null,
"description": "# APalysis\n\n## Introduction\nThis project aims to:\n\n1. Visualize the raw activations and latent space activations of a single or a group of images for any neural networks.\n2. Compare individual and class activation pathways for different datasets.\n3. Apply visual analytics on the classification ambiguousness of the classes in a dataset.\n4. Visual analytics to identify opportunities for pruning of neural networks.\n\n## Quickstart (w/ Docker)\n\nYou can run the following the start the whole dockerized application quickly in localhost.\n\n```bash\ndocker run -p 8000:8000/tcp rahatzamancse/apalysis\n```\n\nAnd open `http://localhost:8000` in your browser.\n\n## Installation\n\nThe project is available on PYPI. So you can install using \n\n```bash\npip install apalysis\n```\n\nYou will need a running redis server for the project to work. You can install redis using\n\n```bash\n# Install redis\nsudo apt install redis-server # For debian-based distros\n# sudo pacman -S redis # For arch\n\n# Run redis\nredis-server --daemonize yes\n```\n\nYou can also install redis using docker. You can find the docker image [here](https://hub.docker.com/_/redis).\n\n## Usage\n### Tensorflow\n```python\n# Import everything\nfrom activation_pathway_analysis_backend.apalysis_tf import APAnalysisTensorflowModel\nimport tensorflow as tf\nimport tensorflow_datasets as tfds\nimport numpy as np\nfrom nltk.corpus import wordnet as wn\n\n# Load the tensorflow model\nmodel = tf.keras.applications.vgg16.VGG16(weights='imagenet')\nmodel.compile(loss=\"categorical_crossentropy\", optimizer=\"adam\")\n\n# Load the dataset\n# The dataset needs to be in tensorflow_datasets format\nds, info = tfds.load(\n 'imagenette/320px-v2',\n shuffle_files=False,\n with_info=True,\n as_supervised=True,\n batch_size=None,\n)\n# For classification, the labels is just a list of names for each encoded class\nlabels = names=list(map(lambda l: wn.synset_from_pos_and_offset(\n l[0], int(l[1:])).name(), info.features['label'].names))\ndataset = ds['train']\n\n# To feed the dataset into the model, we need to provide a preprocessing function\nvgg16_input_shape = tf.keras.applications.vgg16.VGG16().input.shape[1:3].as_list()\n@tf.function\ndef preprocess(x, y):\n x = tf.image.resize(x, vgg16_input_shape, method=tf.image.ResizeMethod.BILINEAR)\n x = tf.keras.applications.vgg16.preprocess_input(x)\n return x, y\n\n# We also need to provide a preprocessing inverse function to convert the preprocessed image back to the original image\n# This is used to display the original image in the frontend\n# This will be automated in the future\ndef preprocess_inv(x, y):\n x = x.squeeze(0)\n # Again adding the mean pixel values\n x[:, :, 0] += 103.939\n x[:, :, 1] += 116.779\n x[:, :, 2] += 123.68\n x = x[:, :, ::-1]\n x = np.clip(x, 0, 255).astype('uint8')\n return x, y\n\n# Create the server and run it\nserver = APAnalysisTensorflowModel(\n model=model,\n dataset=dataset,\n label_names=labels,\n preprocess=preprocess,\n preprocess_inverse=preprocess_inv,\n log_level=\"info\",\n)\nserver.run_server(host=\"localhost\", port=8000)\n```\n\n### PyTorch\nVery similar to the above example.\n```python\n# Imports\nfrom activation_pathway_analysis_backend.apalysis_torch import APAnalysisTorchModel\nimport torch\nimport torchvision.models as models\nimport torchvision.datasets as datasets\nimport torchvision.transforms as transforms\nfrom torch.utils.data import DataLoader\nimport numpy as np\nfrom nltk.corpus import wordnet as wn\n\n# Load the model\nmodel = models.vgg16(pretrained=True)\nloss_fn = torch.nn.CrossEntropyLoss()\noptimizer = torch.optim.Adam(model.parameters())\n\n# We do not need any preprocessing function to provide to the server for pytorch. we can provide it to the dataset instead.\ntransform = transforms.Compose([\n transforms.ToTensor(),\n transforms.Normalize((0.1307,), (0.3081,)) # MNIST mean and std\n])\ndataset = datasets.MNIST('/run/media/insane/SSD Games/Pytorch', train=True, download=True, transform=transform)\n\n# Start the server\nserver = APAnalysisTorchModel(\n model=model,\n input_shape=(1, 3, 224, 224),\n dataset=dataset,\n label_names=labels,\n log_level=\"info\",\n)\n\nserver.run_server(host=\"localhost\", port=8000)\n```",
"bugtrack_url": null,
"license": "",
"summary": "",
"version": "0.0.2",
"project_urls": {
"Homepage": "https://github.com/rahatzamancse/APalysis",
"Repository": "https://github.com/rahatzamancse/APalysis"
},
"split_keywords": [
"apalysis"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "04e8399e5608bef790317ba9b329bd2f4d37ee0ba280db9453ebaa8510ba8b72",
"md5": "201e7a32ed4f7ebf16d5dad64ed8ce66",
"sha256": "d83a757fd8610a399309f80335f63db48d37e80915305f0c740501ec5830ec28"
},
"downloads": -1,
"filename": "apalysis-0.0.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "201e7a32ed4f7ebf16d5dad64ed8ce66",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10,<3.11",
"size": 22124,
"upload_time": "2024-01-13T18:26:27",
"upload_time_iso_8601": "2024-01-13T18:26:27.625562Z",
"url": "https://files.pythonhosted.org/packages/04/e8/399e5608bef790317ba9b329bd2f4d37ee0ba280db9453ebaa8510ba8b72/apalysis-0.0.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "cfa7614dd437b7abfc793615c44214793eab3ca0995184f5eacce6b777f88cd1",
"md5": "881dad17ee649363aab4479f8493674b",
"sha256": "b6229c63b647fa160a25ac67f4346fca45ea452c363a6db25ba1f5a8b56fbe6b"
},
"downloads": -1,
"filename": "apalysis-0.0.2.tar.gz",
"has_sig": false,
"md5_digest": "881dad17ee649363aab4479f8493674b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10,<3.11",
"size": 16433,
"upload_time": "2024-01-13T18:26:29",
"upload_time_iso_8601": "2024-01-13T18:26:29.859481Z",
"url": "https://files.pythonhosted.org/packages/cf/a7/614dd437b7abfc793615c44214793eab3ca0995184f5eacce6b777f88cd1/apalysis-0.0.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-13 18:26:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "rahatzamancse",
"github_project": "APalysis",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "apalysis"
}
JSON
