| Name | llm-mri JSON |
| Version |
0.1.2
JSON |
| download |
| home_page | None |
| Summary | Package to visualize LLM's Neural Networks activation regions |
| upload_time | 2024-07-26 18:50:12 |
| maintainer | None |
| docs_url | None |
| author | lipecorradini |
| requires_python | <4.0,>=3.10 |
| license | None |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# LLM-MRI: a brain scanner for LLMs
As the everyday use of large language models (LLMs) expands, so does the necessity of understanding how these models achieve their designated outputs. While many approaches focus on the interpretability of LLMs through visualizing different attention mechanisms and methods that explain the model's architecture, `LLM-MRI` focuses on the activations of the feed-forward layers in a transformer-based LLM.
By adopting this approach, the library examines the neuron activations produced by the model for each distinct label. Through a series of steps, such as dimensionality reduction and representing each layer as a grid, the tool provides various visualization methods for the activation patterns in the feed-forward layers. Accordingly, the objective of this library is to contribute to LLM interpretability research, enabling users to explore visualization methods, such as heatmaps and graph representations of the hidden layers' activations in transformer-based LLMs.
This model allows users to explore questions such as:
- How do different categories of text in the corpus activate different neural regions?
- What are the differences between the properties of graphs formed by activations from two distinct categories?
- Are there regions of activation in the model more related to specific aspects of a category?
We encourage you to not only use this toolkit but also to extend it as you see fit.
## Index
- [Online Example](#online-example)
- [Installation](#installation)
- [Execution](#execution)
- [Usage](#usage)
- [Functions](#functions)
- [Activation Extraction](#activation-extraction)
- [Heatmap Representation of Activations](#heatmap-representation-of-activations)
- [Graph Representation of Activations](#graph-representation-of-activations)
- [Composed Graph Visualization](#composed-graph-visualization)
## Online Example
The link below runs an online example of our library, in the Jupyter platform running over the Binder server:
[](https://mybinder.org/v2/gh/luizcelsojr/LLM-MRI/v01.2?labpath=examples%2FEmotions.ipynb)
## Instalation
To see LLM-MRI in action on your own data:
**Clone this repository on your machine**
```
git clone https://github.com/luizcelsojr/LLM-MRI
cd LLM-MRI
```
**Install Poetry**
```
pip install poetry
```
**Install Graphviz**
```
sudo apt install graphviz
sudo apt install libgraphviz-dev
```
**Install other dependencies**
```
poetry install --no-root
```
## Execution
**Enable poetry's shell:**
```
poetry shell
```
To run your python file:
```
python3 file.py
```
To run your jupyter notebook:
```
poetry run jupyter notebook
```
## Usage
Firstly, the user needs to import the `LLM-MRI` and `matplotlib,pyplot` packages:
```
import LLM_MRI
import matplotlib.pyplot as plt
```
The user also needs to specify the Hugging Face Dataset that will be used to process the model's activations. There are two ways to do this:
- Load the Dataset from Hugging Face Hub:
```
dataset_url = "https://huggingface.co/datasets/dataset_link"
dataset = load_dataset("csv", data_files=dataset_url)
```
- If you already has the dataset loaded on your machine, you can use the _load_from_disk_ function:
```
dataset = load_from_disk(dataset_path) # Specify the Dataset's path
```
Next, the user selects the model to be used as a string:
```
model_ckpt = "distilbert/distilbert-base-multilingual-cased"
```
Then, the user instantiates `LLM-MRI`, to apply the methods defined on Functions:
```
llm_mri = LLM_MRI(model=model_ckpt, device="cpu", dataset=dataset)
```
## Functions
The library's functionality is divided into the following sections:
### Activation Extraction:
As the user inputs the model and corpus to be analyzed, the dimensionality of the model's hidden layers is reduced, enabling visualization as an NxN grid.
```
llm_mri.process_activation_areas(map_dimension)
```
### Heatmap representation of activations:
This includes the _get_layer_image_ function, which transforms the NxN grid for a selected layer into a heatmap. In this heatmap, each cell represents the number of activations that different regions received for the provided corpus. Additionally, users can visualize activations for a specific label.
```
fig = llm_mri.get_layer_image(layer, category)
```
### Graph Representation of Activations:
Using the _get_graph_ function, the module connects regions from neighboring layers based on co-activations to form a graph representing the entire network. The graph's edges can also be colored according to different labels, allowing the user to identify the specific category that activated each neighboring node.
```
graph = llm_mri.get_graph(category)
graph_image = llm_mri.get_graph_image(graph)
```
The user is also able to obtain a composed visualization of two different categories using the _get_composed_graph_ function. By setting a category, each edge is colored based on the designated label, so the user is able to see which document label activated each region.
```
g_composed = llm_mri.get_composed_graph("true", "fake")
g_composed_img = llm_mri.get_graph_image(g_composed)
```
Raw data
{
"_id": null,
"home_page": null,
"name": "llm-mri",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.10",
"maintainer_email": null,
"keywords": null,
"author": "lipecorradini",
"author_email": "luizfelipecorradini@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/01/ef/08cb1d68b29502d04444cfe717a5e3c06522597604347fa06b980f6b095f/llm_mri-0.1.2.tar.gz",
"platform": null,
"description": "# LLM-MRI: a brain scanner for LLMs\n\nAs the everyday use of large language models (LLMs) expands, so does the necessity of understanding how these models achieve their designated outputs. While many approaches focus on the interpretability of LLMs through visualizing different attention mechanisms and methods that explain the model's architecture, `LLM-MRI` focuses on the activations of the feed-forward layers in a transformer-based LLM.\n\nBy adopting this approach, the library examines the neuron activations produced by the model for each distinct label. Through a series of steps, such as dimensionality reduction and representing each layer as a grid, the tool provides various visualization methods for the activation patterns in the feed-forward layers. Accordingly, the objective of this library is to contribute to LLM interpretability research, enabling users to explore visualization methods, such as heatmaps and graph representations of the hidden layers' activations in transformer-based LLMs.\n\nThis model allows users to explore questions such as:\n\n- How do different categories of text in the corpus activate different neural regions?\n- What are the differences between the properties of graphs formed by activations from two distinct categories?\n- Are there regions of activation in the model more related to specific aspects of a category?\n\nWe encourage you to not only use this toolkit but also to extend it as you see fit.\n\n## Index\n- [Online Example](#online-example)\n- [Installation](#installation)\n- [Execution](#execution)\n- [Usage](#usage)\n- [Functions](#functions)\n - [Activation Extraction](#activation-extraction)\n - [Heatmap Representation of Activations](#heatmap-representation-of-activations)\n - [Graph Representation of Activations](#graph-representation-of-activations)\n - [Composed Graph Visualization](#composed-graph-visualization)\n\n\n## Online Example\n\nThe link below runs an online example of our library, in the Jupyter platform running over the Binder server:\n\n[](https://mybinder.org/v2/gh/luizcelsojr/LLM-MRI/v01.2?labpath=examples%2FEmotions.ipynb)\n\n## Instalation\n\nTo see LLM-MRI in action on your own data:\n\n**Clone this repository on your machine**\n\n```\ngit clone https://github.com/luizcelsojr/LLM-MRI\n\ncd LLM-MRI\n```\n\n**Install Poetry**\n\n```\npip install poetry\n```\n\n\n**Install Graphviz**\n\n```\nsudo apt install graphviz\n\nsudo apt install libgraphviz-dev\n```\n\n**Install other dependencies**\n\n```\npoetry install --no-root\n```\n\n## Execution\n\n**Enable poetry's shell:**\n```\npoetry shell\n```\n\nTo run your python file:\n```\npython3 file.py\n```\n\nTo run your jupyter notebook:\n```\npoetry run jupyter notebook\n```\n## Usage\n\nFirstly, the user needs to import the `LLM-MRI` and `matplotlib,pyplot` packages:\n\n```\nimport LLM_MRI\nimport matplotlib.pyplot as plt\n```\nThe user also needs to specify the Hugging Face Dataset that will be used to process the model's activations. There are two ways to do this:\n\n\n- Load the Dataset from Hugging Face Hub: \n ```\n dataset_url = \"https://huggingface.co/datasets/dataset_link\"\n dataset = load_dataset(\"csv\", data_files=dataset_url)\n ```\n- If you already has the dataset loaded on your machine, you can use the _load_from_disk_ function:\n ```\n dataset = load_from_disk(dataset_path) # Specify the Dataset's path\n ```\n\nNext, the user selects the model to be used as a string:\n```\nmodel_ckpt = \"distilbert/distilbert-base-multilingual-cased\"\n```\nThen, the user instantiates `LLM-MRI`, to apply the methods defined on Functions:\n```\nllm_mri = LLM_MRI(model=model_ckpt, device=\"cpu\", dataset=dataset)\n```\n## Functions\nThe library's functionality is divided into the following sections:\n\n### Activation Extraction: \nAs the user inputs the model and corpus to be analyzed, the dimensionality of the model's hidden layers is reduced, enabling visualization as an NxN grid.\n ```\n llm_mri.process_activation_areas(map_dimension)\n ```\n\n\n \n### Heatmap representation of activations:\nThis includes the _get_layer_image_ function, which transforms the NxN grid for a selected layer into a heatmap. In this heatmap, each cell represents the number of activations that different regions received for the provided corpus. Additionally, users can visualize activations for a specific label.\n ```\n fig = llm_mri.get_layer_image(layer, category)\n ```\n\n\n\n\n \n### Graph Representation of Activations:\nUsing the _get_graph_ function, the module connects regions from neighboring layers based on co-activations to form a graph representing the entire network. The graph's edges can also be colored according to different labels, allowing the user to identify the specific category that activated each neighboring node.\n ```\n graph = llm_mri.get_graph(category)\n graph_image = llm_mri.get_graph_image(graph)\n ```\n\n\n\nThe user is also able to obtain a composed visualization of two different categories using the _get_composed_graph_ function. By setting a category, each edge is colored based on the designated label, so the user is able to see which document label activated each region.\n```\ng_composed = llm_mri.get_composed_graph(\"true\", \"fake\")\ng_composed_img = llm_mri.get_graph_image(g_composed)\n```\n\n\n\n",
"bugtrack_url": null,
"license": null,
"summary": "Package to visualize LLM's Neural Networks activation regions",
"version": "0.1.2",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "bb4aafd664b60405a232b2b011f2cfd6a11b0b0c67279858aca00647cb3698ae",
"md5": "0085db6011abbe780acc22c38194ce18",
"sha256": "ad81df85b3a2ee53f26780a86e3c19fe6113e0b3e31939ac272f88aea5fc76b2"
},
"downloads": -1,
"filename": "llm_mri-0.1.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "0085db6011abbe780acc22c38194ce18",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.10",
"size": 10868,
"upload_time": "2024-07-26T18:50:11",
"upload_time_iso_8601": "2024-07-26T18:50:11.571960Z",
"url": "https://files.pythonhosted.org/packages/bb/4a/afd664b60405a232b2b011f2cfd6a11b0b0c67279858aca00647cb3698ae/llm_mri-0.1.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "01ef08cb1d68b29502d04444cfe717a5e3c06522597604347fa06b980f6b095f",
"md5": "10d723f4f6e5cbb0be0d7b32b11c9dee",
"sha256": "a8db3b9b7fd484c4624ad6c4b03e3785e3736f0ced85142cb803b713bf67d8db"
},
"downloads": -1,
"filename": "llm_mri-0.1.2.tar.gz",
"has_sig": false,
"md5_digest": "10d723f4f6e5cbb0be0d7b32b11c9dee",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.10",
"size": 9645,
"upload_time": "2024-07-26T18:50:12",
"upload_time_iso_8601": "2024-07-26T18:50:12.699889Z",
"url": "https://files.pythonhosted.org/packages/01/ef/08cb1d68b29502d04444cfe717a5e3c06522597604347fa06b980f6b095f/llm_mri-0.1.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-26 18:50:12",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "llm-mri"
}
