# Rastermap
[](https://codecov.io/gh/MouseLand/rastermap)
[](https://badge.fury.io/py/rastermap)
[](https://pepy.tech/project/rastermap)
[](https://pepy.tech/project/rastermap)
[](https://pypistats.org/packages/rastermap)
[](https://github.com/MouseLand/rastermap/blob/main/LICENSE)
[](https://github.com/MouseLand/rastermap/graphs/contributors)
[](https://github.com/MouseLand/rastermap/)
[](https://github.com/MouseLand/rastermap/)
[](https://github.com/MouseLand/rastermap/)
Rastermap is a discovery algorithm for neural data. The algorithm was written by Carsen Stringer and Marius Pachitariu. For support, please open an [issue](https://github.com/MouseLand/rastermap/issues). Please see install instructions [below](README.md/#Installation). Check out the [**paper**](https://www.nature.com/articles/s41593-024-01783-4) and the [**tutorial video**](https://youtu.be/oQHq7yUWn2k) for more info. Rastermap runs in python 3.8+ and has a [**graphical user interface (GUI)**](#gui) for running it easily.
**If you use Rastermap or analysis code in this repo in your work, please cite the paper:**
Stringer C., Zhong L., Syeda A., Du F., Kesa M., & Pachitariu M. (2024). Rastermap: a discovery method for neural population recordings. *Nature Neuroscience*. https://doi.org/10.1038/s41593-024-01783-4.
Table of Contents
=================
* [Rastermap](#rastermap)
* [Example notebooks](#example-notebooks)
* [Installation](#installation)
* [Local installation (< 2 minutes)](#local-installation--2-minutes)
* [System requirements](#system-requirements)
* [Instructions](#instructions)
* [Dependencies](#dependencies)
* [Using rastermap](#using-rastermap)
* [GUI](#gui)
* [In a notebook](#in-a-notebook)
* [From the command line](#from-the-command-line)
* [From MATLAB](#from-matlab)
* [Inputs](#inputs)
* [Settings](#settings)
* [Outputs](#outputs)
* [License](#license)
## Example notebooks
* [rastermap_largescale.ipynb](notebooks/rastermap_largescale.ipynb) [](https://colab.research.google.com/github/MouseLand/rastermap/blob/main/notebooks/rastermap_largescale.ipynb) shows how to use it with large-scale data from mouse cortex (> 200 neurons)
* [rastermap_singleneurons.ipynb](notebooks/rastermap_singleneurons.ipynb) [](https://colab.research.google.com/github/MouseLand/rastermap/blob/main/notebooks/rastermap_singleneurons.ipynb) shows how to use it with small to medium sized data (< 200 neurons), in this case recorded from rat hippocampus
* [rastermap_zebrafish.ipynb](notebooks/rastermap_zebrafish.ipynb) [](https://colab.research.google.com/github/MouseLand/rastermap/blob/main/notebooks/rastermap_zebrafish.ipynb) shows how to use it with large-scale data from zebrafish
* [rastermap_widefield.ipynb](notebooks/rastermap_widefield.ipynb) [](https://colab.research.google.com/github/MouseLand/rastermap/blob/main/notebooks/rastermap_widefield.ipynb) shows how to use it with widefield imaging data, or other types of datasets that are too large to fit into memory
* [rastermap_interactive.ipynb](notebooks/rastermap_interactive.ipynb) [](https://colab.research.google.com/github/MouseLand/rastermap/blob/main/notebooks/rastermap_interactive.ipynb) allows running Rastermap in an interactive way without a local installation
* [tutorial.ipynb](notebooks/tutorial.ipynb) [](https://colab.research.google.com/github/MouseLand/rastermap/blob/main/notebooks/tutorial.ipynb) is a guided tutorial for integrating rastermap and facemap to visualize behavioral representations. See the student/teacher versions [here](https://github.com/MouseLand/course-materials/tree/main/behavior_encoding).
Also all notebooks to analyze the data and create the figures in the paper are [here](paper/). All data available [here](https://osf.io/xn4cm/).
Here is what the output looks like for a segment of a mesoscope recording in a mouse during spontaneous activity (3.2Hz sampling rate), compared to random neural sorting:
<img src="https://www.suite2p.org/static/images/rastermap_spont.png" width="800" alt="random sorting and rastermap sorting of spontaneous activity"/>
# Installation
## Local installation (< 2 minutes)
### System requirements
Linux, Windows and Mac OS are supported for running the code. For running the graphical interface in Mac, you will need a Mac OS later than Yosemite. At least 8GB of RAM is recommended to run the software. 16GB-32GB may be required for larger datasets. The software has been heavily tested on Windows 10 and Ubuntu 20.04 and less well-tested on Mac OS. Please open an [issue](https://github.com/MouseLand/rastermap/issues) if you have problems with installation.
### Instructions
We recommend to install a [miniforge](https://github.com/conda-forge/miniforge) (conda-based) distribution of Python. Note you might need to use an anaconda prompt (windows) if you did not add anaconda to the path. Open an anaconda prompt / command prompt with **python 3** in the path, then:
~~~sh
pip install rastermap
~~~
For the GUI
~~~sh
pip install rastermap[gui]
~~~
Rastermap has only a few dependencies so you may not need to make a special environment for it
(e.g. it should work in a `suite2p` or `facemap` environment), but if the pip install above does not work,
please follow these instructions:
1. Open an anaconda prompt / command prompt which has `conda` for **python 3** in the path
3. Create a new environment with `conda create --name rastermap python=3.9`. We recommend python 3.9, but python 3.8, 3.9 and 3.11 will also work.
4. To activate this new environment, run `conda activate rastermap`
5. (option 1) To install rastermap with the GUI, run `python -m pip install rastermap[gui]`. If you're on a zsh server, you may need to use ' ': `python -m pip install 'rastermap[gui]'`.
6. (option 2) To install rastermap without the GUI, run `python -m pip install rastermap`.
To upgrade rastermap (package [here](https://pypi.org/project/rastermap/)), run the following in the environment:
~~~sh
pip install rastermap --upgrade
~~~
If you have an older `rastermap` environment you can remove it with `conda env remove -n rastermap` before creating a new one.
Note you will always have to run **conda activate rastermap** before you run rastermap. If you want to run jupyter notebooks in this environment, then also `pip install notebook`.
### Dependencies
This package relies on the awesomeness of **numpy**, **scipy**, **numba**, **scikit-learn**, **PyQt6**, **PyQt6.sip** and **pyqtgraph**. You can pip install or conda install all of these packages. If having issues with **PyQt6**, then make an Anaconda environment and try to install within it `conda install pyqt`. On **Ubuntu** you may need to `sudo apt-get install libegl1` to support PyQt6. Alternatively, you can use PyQt5 by running `pip uninstall PyQt6` and `pip install PyQt5`. If you already have a PyQt version installed, Rastermap will not install a new one.
# Using rastermap
## GUI
The quickest way to start is to open the GUI from a command line terminal. You might need to open an anaconda prompt if you did not add anaconda to the path. Then run:
~~~sh
python -m rastermap
~~~
To start using the GUI, save your data into an npy file that is just a matrix that is neurons x timepoints. Then "File > Load data matrix" and choose this file (or drag and drop your file). You can also try it out with our demo data available [here](https://osf.io/xn4cm/).
Next click "Run > Run rastermap" and click run. See the parameters section to learn about the parameters.
The GUI will start with a highlighted region that you can drag to visualize the average activity of neurons in a given part of the plot. To draw more regions, you right-click to start a region, then right-click to end it. The neurons' activity traces then show up on the botton of the GUI, and if the neuron positions are loaded, you will see them colored by the region color. You can delete a region by holding CTRL and clicking on it. You can save the ROIs you've drawn with the "Save > Save processed data" button. They will save along with the embedding so you can reload the file with the "Load processed data" option.
NOTE: If you are using suite2p "spks.npy", then the GUI will automatically use the "iscell.npy" file in the same folder to subsample your recording with the chosen neurons, and will automatically load
the neuron positions from the "stat.npy" file.
GUI examples:
zebrafish:
<img src="https://www.suite2p.org/static/images/fish.gif" width="600" alt="wholebrain neural activity from a zebrafish sorted by rastermap"/>
mouse sensorimotor activity:
<img src="https://www.suite2p.org/static/images/spont.gif" width="600" alt="sensorimotor neural activity from a mouse sorted by rastermap"/>
rat hippocampus:
<img src="https://www.suite2p.org/static/images/hippocampus.gif" width="600" alt="hippocampal neural activity from a rat sorted by rastermap"/>
mouse widefield:
<img src="https://www.suite2p.org/static/images/widefield.gif" width="600" alt="widefield neural activity from a mouse sorted by rastermap"/>
## In a notebook
For this, `pip install notebook` and `pip install matplotlib`. See example [notebooks](notebooks/) for full examples.
Short example code snippet for running rastermap:
```
import numpy as np
import matplotlib.pyplot as plt
from rastermap import Rastermap
# spks is neurons by time
spks = np.load("spks.npy").astype("float32")
# fit rastermap
model = Rastermap(n_PCs=200, n_clusters=100,
locality=0.75, time_lag_window=5).fit(spks)
y = model.embedding # neurons x 1
isort = model.isort
# visualize binning over neurons
X_embedding = model.X_embedding
# plot
fig = plt.figure(figsize=(12,5))
ax = fig.add_subplot(111)
ax.imshow(X_embedding, vmin=0, vmax=1.5, cmap="gray_r", aspect="auto")
```
If you are using google colab, you can mount your google drive and use your data from there with the following command, you will then see your files in the left bar under `drive`:
```
from google.colab import drive
drive.mount('/content/drive')
```
## From the command line
Save an "ops.npy" file with the parameters and a "spks.npy" file with a matrix of neurons by time, and run
~~~sh
python -m rastermap --S spks.npy --ops ops.npy
~~~
## From MATLAB
If you have an existing MATLAB analysis pipeline (and are using MATLAB version R2021b or newer), you can use MATLAB's Python interface to call Rastermap. First you need to tell MATLAB where your Python enviroment with Rastermap is (more details: https://www.mathworks.com/help/matlab/matlab_external/install-supported-python-implementation.html). Create or modify the "PYTHONHOME" environmental variable in your OS to point to the Rastermap environment root folder. Then in MATLAB run the following statement, modified for the specific path to your Rastermap environment pythonw executable:
```
pyenv('Version','C:\Users\admin\.conda\envs\rastermap\pythonw.exe', 'ExecutionMode', 'OutOfProcess')
```
Then you should be able to see your environment details in MATLAB after typing "pyenv" (the intepreter will not actually load until you try to run a Python statement). An example function to call Rastermap and return the sort order back as a MATLAB array is:
```
function [sortIdx] = rastermapSort(shankDataToSort)
% wrapper to convert to numpy array, call Rastermap, and then convert back to MATLAB array
data = shankDataToSort.zScoredFiringRates;
dataNdArray = py.numpy.array(data);
pyrun("from rastermap import Rastermap") %load interpreter, import main function
rmModel = pyrun("model = Rastermap(locality=0.5, time_lag_window=50).fit(spks)", "model", spks=dataNdArray);
sortIdx = int16(py.memoryview(rmModel.isort.data)) + 1; %back to MATLAB array, 1-indexing
end
```
# Inputs
Most of the time you will input to `Rastermap().fit` a matrix of neurons by time. For more details, these are all the inputs to the function:
* **data** : array, shape (n_samples, n_features) (optional, default None)
this matrix is usually neurons/voxels by time, or None if using decomposition,
e.g. as in widefield imaging
* **Usv** : array, shape (n_samples, n_PCs) (optional, default None)
singular vectors U times singular values sv
* **Vsv** : array, shape (n_features, n_PCs) (optional, default None)
singular vectors U times singular values sv
* **U_nodes** : array, shape (n_clusters, n_PCs) (optional, default None)
cluster centers in PC space, if you have precomputed them
* **itrain** : array, shape (n_features,) (optional, default None)
fit embedding on timepoints itrain only
If you have a `spike_times.npy` and `spike_clusters.npy`, create your time-binned data
matrix with, where the bin size `st_bin` is in milliseconds (assuming your spike times are in seconds):
```
from rastermap import io
# bin spike times into neurons by time matrix
data = io.load_spike_times("spike_times.npy", "spike_clusters.npy", st_bin=100)
```
You can also load these matrices into the GUI with the `File > Load spike_times...` option.
# Settings
These are inputs to the `Rastermap` class initialization, the settings are sorted in order of importance
(you will probably never need to change any other than the first few):
* **n_clusters** : int, optional (default: 100)
number of clusters created from data before upsampling and creating embedding
(any number above 150 will be slow due to NP-hard sorting problem, max is 200)
* **n_PCs** : int, optional (default: 200)
number of PCs to use during optimization
* **time_lag_window** : int, optional (default: 0)
number of time points into the future to compute cross-correlation,
useful to set to several timepoints for sequence finding
* **locality** : float, optional (default: 0.0)
how local should the algorithm be -- set to 1.0 for highly local +
sequence finding, and 0.0 for global sorting
* **grid_upsample** : int, optional (default: 10)
how much to upsample clusters, if set to 0.0 then no upsampling
* **time_bin** : int, optional (default: 0)
binning of data in time before PCA is computed, if set to 0 or 1 no binning occurs
* **mean_time** : bool, optional (default: True)
whether to project out the mean over data samples at each timepoint,
usually good to keep on to find structure
* **n_splits** : int, optional (default: 0)
split, recluster and sort n_splits times
(increases local neighborhood preservation for high-dim data);
results in (n_clusters * 2**n_splits) clusters
* **run_scaled_kmeans** : bool, optional (default: True)
run scaled_kmeans as clustering algorithm; if False, run kmeans
* **verbose** : bool (default: True)
whether to output progress during optimization
* **verbose_sorting** : bool (default: False)
output progress in travelling salesman
* **keep_norm_X** : bool, optional (default: True)
keep normalized version of X saved as member of class
* **bin_size** : int, optional (default: 0)
binning of data across n_samples to return embedding figure, X_embedding;
if 0, then binning based on data size, if 1 then no binning
* **symmetric** : bool, optional (default: False)
if False, use only positive time lag cross-correlations for sorting
(only makes a difference if time_lag_window > 0);
recommended to keep False for sequence finding
* **sticky** : bool, optional (default: True)
if n_splits>0, sticky=True keeps neurons in same place as initial sorting before splitting;
otherwise neurons can move each split (which generally does not work as well)
* **nc_splits** : int, optional (default: None)
if n_splits > 0, size to split n_clusters into;
if None, nc_splits = min(50, n_clusters // 4)
* **smoothness** : int, optional (default: 1)
how much to smooth over clusters when upsampling, number from 1 to number of
clusters (recommended to not change, instead use locality to change sorting)
# Outputs
The main output you want is the sorting, `isort`, which is assigned to the `Rastermap` class, e.g.
```
model = Rastermap().fit(spks)
isort = model.isort
```
You may also want to color the neurons by their positions which are in `embedding`, e.g.
```
y = model.embedding[:,0]
plt.scatter(xpos, ypos, cmap="gist_rainbow", c=y, s=1)
```
Here is the list of all variables assigned from `fit`:
* **embedding** : array, shape (n_samples, 1)
embedding of each neuron / voxel
* **isort** : array, shape (n_samples,)
sorting along first dimension of input matrix - use this to get neuron / voxel sorting
* **igood** : array, shape (n_samples, 1)
neurons/voxels which had non-zero activity and were used for sorting
* **Usv** : array, shape (n_samples, n_PCs)
singular vectors U times singular values sv
* **Vsv** : array, shape (n_features, n_PCs)
singular vectors U times singular values sv
* **U_nodes** : array, shape (n_clusters, n_PCs)
cluster centers in PC space
* **Y_nodes** : array, shape (n_clusters, 1)
np.arange(0, n_clusters)
* **X_nodes** : array, shape (n_clusters, n_features)
cluster activity traces in time
* **cc** : array, shape (n_clusters, n_clusters)
sorted asymmetric similarity matrix
* **embedding_clust** : array, shape (n_samples, 1)
assignment of each neuron/voxel to each cluster (before upsampling)
* **X** : array, shape (n_samples, n_features)
normalized data stored (if keep_norm_X is True)
* **X_embedding** : array, shape (n_samples//bin_size, n_features)
normalized data binned across samples (if compute_X_embedding is True)
The output from the GUI and the command line is a file that ends with `_embedding.npy`. This file contains:
* **filename**: str,
path to file that rastermap was run on
* **save_path**: str,
folder with filename
* **embedding** : array, shape (n_samples, 1)
embedding of each neuron / voxel
* **isort** : array, shape (n_samples,)
sorting along first dimension of input matrix - use this to get neuron / voxel sorting
* **user_clusters**: list,
list of user drawn clusters in GUI
* **ops**: dict,
dictionary of options used to run rastermap
# License
Copyright (C) 2023 Howard Hughes Medical Institute Janelia Research Campus, the labs of Carsen Stringer and Marius Pachitariu.
**This code is licensed under GPL v3 (no redistribution without credit, and no redistribution in private repos, see the [license](LICENSE) for more details).**
Raw data
{
"_id": null,
"home_page": "https://github.com/MouseLand/rastermap",
"name": "rastermap",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "Marius Pachitariu and Carsen Stringer",
"author_email": "carsen.stringer@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/31/db/e0ed0068934fb588a8e934ebeafb39adbcf9d709a56cb3cb83b3e72be959/rastermap-1.0.tar.gz",
"platform": null,
"description": "# Rastermap\n\n\n[](https://codecov.io/gh/MouseLand/rastermap)\n[](https://badge.fury.io/py/rastermap)\n[](https://pepy.tech/project/rastermap)\n[](https://pepy.tech/project/rastermap)\n[](https://pypistats.org/packages/rastermap)\n[](https://github.com/MouseLand/rastermap/blob/main/LICENSE)\n[](https://github.com/MouseLand/rastermap/graphs/contributors)\n[](https://github.com/MouseLand/rastermap/)\n[](https://github.com/MouseLand/rastermap/)\n[](https://github.com/MouseLand/rastermap/)\n\n\nRastermap is a discovery algorithm for neural data. The algorithm was written by Carsen Stringer and Marius Pachitariu. For support, please open an [issue](https://github.com/MouseLand/rastermap/issues). Please see install instructions [below](README.md/#Installation). Check out the [**paper**](https://www.nature.com/articles/s41593-024-01783-4) and the [**tutorial video**](https://youtu.be/oQHq7yUWn2k) for more info. Rastermap runs in python 3.8+ and has a [**graphical user interface (GUI)**](#gui) for running it easily.\n\n**If you use Rastermap or analysis code in this repo in your work, please cite the paper:**\nStringer C., Zhong L., Syeda A., Du F., Kesa M., & Pachitariu M. (2024). Rastermap: a discovery method for neural population recordings. *Nature Neuroscience*. https://doi.org/10.1038/s41593-024-01783-4.\n\nTable of Contents\n=================\n\n* [Rastermap](#rastermap)\n * [Example notebooks](#example-notebooks)\n* [Installation](#installation)\n * [Local installation (< 2 minutes)](#local-installation--2-minutes)\n * [System requirements](#system-requirements)\n * [Instructions](#instructions)\n * [Dependencies](#dependencies)\n* [Using rastermap](#using-rastermap)\n * [GUI](#gui)\n * [In a notebook](#in-a-notebook)\n * [From the command line](#from-the-command-line)\n * [From MATLAB](#from-matlab)\n* [Inputs](#inputs)\n* [Settings](#settings)\n* [Outputs](#outputs)\n* [License](#license)\n\n## Example notebooks\n\n* [rastermap_largescale.ipynb](notebooks/rastermap_largescale.ipynb) [](https://colab.research.google.com/github/MouseLand/rastermap/blob/main/notebooks/rastermap_largescale.ipynb) shows how to use it with large-scale data from mouse cortex (> 200 neurons) \n* [rastermap_singleneurons.ipynb](notebooks/rastermap_singleneurons.ipynb) [](https://colab.research.google.com/github/MouseLand/rastermap/blob/main/notebooks/rastermap_singleneurons.ipynb) shows how to use it with small to medium sized data (< 200 neurons), in this case recorded from rat hippocampus \n* [rastermap_zebrafish.ipynb](notebooks/rastermap_zebrafish.ipynb) [](https://colab.research.google.com/github/MouseLand/rastermap/blob/main/notebooks/rastermap_zebrafish.ipynb) shows how to use it with large-scale data from zebrafish \n* [rastermap_widefield.ipynb](notebooks/rastermap_widefield.ipynb) [](https://colab.research.google.com/github/MouseLand/rastermap/blob/main/notebooks/rastermap_widefield.ipynb) shows how to use it with widefield imaging data, or other types of datasets that are too large to fit into memory \n* [rastermap_interactive.ipynb](notebooks/rastermap_interactive.ipynb) [](https://colab.research.google.com/github/MouseLand/rastermap/blob/main/notebooks/rastermap_interactive.ipynb) allows running Rastermap in an interactive way without a local installation\n* [tutorial.ipynb](notebooks/tutorial.ipynb) [](https://colab.research.google.com/github/MouseLand/rastermap/blob/main/notebooks/tutorial.ipynb) is a guided tutorial for integrating rastermap and facemap to visualize behavioral representations. See the student/teacher versions [here](https://github.com/MouseLand/course-materials/tree/main/behavior_encoding).\n\nAlso all notebooks to analyze the data and create the figures in the paper are [here](paper/). All data available [here](https://osf.io/xn4cm/). \n\nHere is what the output looks like for a segment of a mesoscope recording in a mouse during spontaneous activity (3.2Hz sampling rate), compared to random neural sorting:\n\n<img src=\"https://www.suite2p.org/static/images/rastermap_spont.png\" width=\"800\" alt=\"random sorting and rastermap sorting of spontaneous activity\"/>\n\n# Installation\n\n## Local installation (< 2 minutes)\n\n### System requirements\n\nLinux, Windows and Mac OS are supported for running the code. For running the graphical interface in Mac, you will need a Mac OS later than Yosemite. At least 8GB of RAM is recommended to run the software. 16GB-32GB may be required for larger datasets. The software has been heavily tested on Windows 10 and Ubuntu 20.04 and less well-tested on Mac OS. Please open an [issue](https://github.com/MouseLand/rastermap/issues) if you have problems with installation.\n\n### Instructions\n\nWe recommend to install a [miniforge](https://github.com/conda-forge/miniforge) (conda-based) distribution of Python. Note you might need to use an anaconda prompt (windows) if you did not add anaconda to the path. Open an anaconda prompt / command prompt with **python 3** in the path, then:\n\n~~~sh\npip install rastermap\n~~~\n\nFor the GUI \n~~~sh\npip install rastermap[gui]\n~~~\n\nRastermap has only a few dependencies so you may not need to make a special environment for it \n(e.g. it should work in a `suite2p` or `facemap` environment), but if the pip install above does not work,\n please follow these instructions:\n \n1. Open an anaconda prompt / command prompt which has `conda` for **python 3** in the path\n3. Create a new environment with `conda create --name rastermap python=3.9`. We recommend python 3.9, but python 3.8, 3.9 and 3.11 will also work.\n4. To activate this new environment, run `conda activate rastermap`\n5. (option 1) To install rastermap with the GUI, run `python -m pip install rastermap[gui]`. If you're on a zsh server, you may need to use ' ': `python -m pip install 'rastermap[gui]'`.\n6. (option 2) To install rastermap without the GUI, run `python -m pip install rastermap`. \nTo upgrade rastermap (package [here](https://pypi.org/project/rastermap/)), run the following in the environment:\n\n~~~sh\npip install rastermap --upgrade\n~~~\n\nIf you have an older `rastermap` environment you can remove it with `conda env remove -n rastermap` before creating a new one.\n\nNote you will always have to run **conda activate rastermap** before you run rastermap. If you want to run jupyter notebooks in this environment, then also `pip install notebook`.\n\n### Dependencies\n\nThis package relies on the awesomeness of **numpy**, **scipy**, **numba**, **scikit-learn**, **PyQt6**, **PyQt6.sip** and **pyqtgraph**. You can pip install or conda install all of these packages. If having issues with **PyQt6**, then make an Anaconda environment and try to install within it `conda install pyqt`. On **Ubuntu** you may need to `sudo apt-get install libegl1` to support PyQt6. Alternatively, you can use PyQt5 by running `pip uninstall PyQt6` and `pip install PyQt5`. If you already have a PyQt version installed, Rastermap will not install a new one.\n\n# Using rastermap\n\n## GUI\n\nThe quickest way to start is to open the GUI from a command line terminal. You might need to open an anaconda prompt if you did not add anaconda to the path. Then run:\n\n~~~sh\npython -m rastermap\n~~~\n\nTo start using the GUI, save your data into an npy file that is just a matrix that is neurons x timepoints. Then \"File > Load data matrix\" and choose this file (or drag and drop your file). You can also try it out with our demo data available [here](https://osf.io/xn4cm/).\n\nNext click \"Run > Run rastermap\" and click run. See the parameters section to learn about the parameters.\n\nThe GUI will start with a highlighted region that you can drag to visualize the average activity of neurons in a given part of the plot. To draw more regions, you right-click to start a region, then right-click to end it. The neurons' activity traces then show up on the botton of the GUI, and if the neuron positions are loaded, you will see them colored by the region color. You can delete a region by holding CTRL and clicking on it. You can save the ROIs you've drawn with the \"Save > Save processed data\" button. They will save along with the embedding so you can reload the file with the \"Load processed data\" option.\n\nNOTE: If you are using suite2p \"spks.npy\", then the GUI will automatically use the \"iscell.npy\" file in the same folder to subsample your recording with the chosen neurons, and will automatically load \nthe neuron positions from the \"stat.npy\" file.\n\nGUI examples:\n\nzebrafish:\n\n<img src=\"https://www.suite2p.org/static/images/fish.gif\" width=\"600\" alt=\"wholebrain neural activity from a zebrafish sorted by rastermap\"/>\n\nmouse sensorimotor activity:\n\n<img src=\"https://www.suite2p.org/static/images/spont.gif\" width=\"600\" alt=\"sensorimotor neural activity from a mouse sorted by rastermap\"/>\n\nrat hippocampus:\n\n<img src=\"https://www.suite2p.org/static/images/hippocampus.gif\" width=\"600\" alt=\"hippocampal neural activity from a rat sorted by rastermap\"/>\n\nmouse widefield:\n\n<img src=\"https://www.suite2p.org/static/images/widefield.gif\" width=\"600\" alt=\"widefield neural activity from a mouse sorted by rastermap\"/>\n\n\n## In a notebook\n\nFor this, `pip install notebook` and `pip install matplotlib`. See example [notebooks](notebooks/) for full examples.\n\nShort example code snippet for running rastermap:\n\n```\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom rastermap import Rastermap\n\n# spks is neurons by time\nspks = np.load(\"spks.npy\").astype(\"float32\")\n\n# fit rastermap\nmodel = Rastermap(n_PCs=200, n_clusters=100, \n locality=0.75, time_lag_window=5).fit(spks)\ny = model.embedding # neurons x 1\nisort = model.isort\n\n# visualize binning over neurons\nX_embedding = model.X_embedding\n\n# plot\nfig = plt.figure(figsize=(12,5))\nax = fig.add_subplot(111)\nax.imshow(X_embedding, vmin=0, vmax=1.5, cmap=\"gray_r\", aspect=\"auto\")\n```\n\nIf you are using google colab, you can mount your google drive and use your data from there with the following command, you will then see your files in the left bar under `drive`:\n\n```\nfrom google.colab import drive\ndrive.mount('/content/drive')\n```\n\n\n## From the command line\n\nSave an \"ops.npy\" file with the parameters and a \"spks.npy\" file with a matrix of neurons by time, and run\n\n~~~sh\npython -m rastermap --S spks.npy --ops ops.npy\n~~~\n\n## From MATLAB\nIf you have an existing MATLAB analysis pipeline (and are using MATLAB version R2021b or newer), you can use MATLAB's Python interface to call Rastermap. First you need to tell MATLAB where your Python enviroment with Rastermap is (more details: https://www.mathworks.com/help/matlab/matlab_external/install-supported-python-implementation.html). Create or modify the \"PYTHONHOME\" environmental variable in your OS to point to the Rastermap environment root folder. Then in MATLAB run the following statement, modified for the specific path to your Rastermap environment pythonw executable:\n\n```\npyenv('Version','C:\\Users\\admin\\.conda\\envs\\rastermap\\pythonw.exe', 'ExecutionMode', 'OutOfProcess')\n```\nThen you should be able to see your environment details in MATLAB after typing \"pyenv\" (the intepreter will not actually load until you try to run a Python statement). An example function to call Rastermap and return the sort order back as a MATLAB array is:\n```\nfunction [sortIdx] = rastermapSort(shankDataToSort)\n% wrapper to convert to numpy array, call Rastermap, and then convert back to MATLAB array\n\n data = shankDataToSort.zScoredFiringRates; \n dataNdArray = py.numpy.array(data);\n pyrun(\"from rastermap import Rastermap\") %load interpreter, import main function\n rmModel = pyrun(\"model = Rastermap(locality=0.5, time_lag_window=50).fit(spks)\", \"model\", spks=dataNdArray);\n sortIdx = int16(py.memoryview(rmModel.isort.data)) + 1; %back to MATLAB array, 1-indexing\n\nend\n```\n\n# Inputs\n\nMost of the time you will input to `Rastermap().fit` a matrix of neurons by time. For more details, these are all the inputs to the function:\n\n* **data** : array, shape (n_samples, n_features) (optional, default None) \n this matrix is usually neurons/voxels by time, or None if using decomposition, \n e.g. as in widefield imaging\n* **Usv** : array, shape (n_samples, n_PCs) (optional, default None)\n singular vectors U times singular values sv\n* **Vsv** : array, shape (n_features, n_PCs) (optional, default None)\n singular vectors U times singular values sv\n* **U_nodes** : array, shape (n_clusters, n_PCs) (optional, default None)\n cluster centers in PC space, if you have precomputed them\n* **itrain** : array, shape (n_features,) (optional, default None)\n fit embedding on timepoints itrain only\n\nIf you have a `spike_times.npy` and `spike_clusters.npy`, create your time-binned data \nmatrix with, where the bin size `st_bin` is in milliseconds (assuming your spike times are in seconds):\n\n```\nfrom rastermap import io \n\n# bin spike times into neurons by time matrix\ndata = io.load_spike_times(\"spike_times.npy\", \"spike_clusters.npy\", st_bin=100)\n```\n\nYou can also load these matrices into the GUI with the `File > Load spike_times...` option.\n\n# Settings\n\nThese are inputs to the `Rastermap` class initialization, the settings are sorted in order of importance \n(you will probably never need to change any other than the first few):\n\n* **n_clusters** : int, optional (default: 100)\n number of clusters created from data before upsampling and creating embedding\n (any number above 150 will be slow due to NP-hard sorting problem, max is 200)\n* **n_PCs** : int, optional (default: 200)\n number of PCs to use during optimization\n* **time_lag_window** : int, optional (default: 0)\n number of time points into the future to compute cross-correlation, \n useful to set to several timepoints for sequence finding\n* **locality** : float, optional (default: 0.0)\n how local should the algorithm be -- set to 1.0 for highly local + \n sequence finding, and 0.0 for global sorting\n* **grid_upsample** : int, optional (default: 10)\n how much to upsample clusters, if set to 0.0 then no upsampling\n* **time_bin** : int, optional (default: 0)\n binning of data in time before PCA is computed, if set to 0 or 1 no binning occurs\n* **mean_time** : bool, optional (default: True)\n whether to project out the mean over data samples at each timepoint,\n usually good to keep on to find structure\n* **n_splits** : int, optional (default: 0)\n split, recluster and sort n_splits times \n (increases local neighborhood preservation for high-dim data); \n results in (n_clusters * 2**n_splits) clusters\n* **run_scaled_kmeans** : bool, optional (default: True)\n run scaled_kmeans as clustering algorithm; if False, run kmeans\n* **verbose** : bool (default: True)\n whether to output progress during optimization\n* **verbose_sorting** : bool (default: False)\n output progress in travelling salesman \n* **keep_norm_X** : bool, optional (default: True)\n keep normalized version of X saved as member of class\n* **bin_size** : int, optional (default: 0)\n binning of data across n_samples to return embedding figure, X_embedding; \n if 0, then binning based on data size, if 1 then no binning\n* **symmetric** : bool, optional (default: False)\n if False, use only positive time lag cross-correlations for sorting \n (only makes a difference if time_lag_window > 0); \n recommended to keep False for sequence finding\n* **sticky** : bool, optional (default: True)\n if n_splits>0, sticky=True keeps neurons in same place as initial sorting before splitting; \n otherwise neurons can move each split (which generally does not work as well)\n* **nc_splits** : int, optional (default: None)\n if n_splits > 0, size to split n_clusters into; \n if None, nc_splits = min(50, n_clusters // 4)\n* **smoothness** : int, optional (default: 1)\n how much to smooth over clusters when upsampling, number from 1 to number of \n clusters (recommended to not change, instead use locality to change sorting)\n \n\n# Outputs\n\nThe main output you want is the sorting, `isort`, which is assigned to the `Rastermap` class, e.g.\n\n```\nmodel = Rastermap().fit(spks)\nisort = model.isort\n```\n\nYou may also want to color the neurons by their positions which are in `embedding`, e.g.\n```\ny = model.embedding[:,0]\nplt.scatter(xpos, ypos, cmap=\"gist_rainbow\", c=y, s=1)\n```\n\nHere is the list of all variables assigned from `fit`:\n\n* **embedding** : array, shape (n_samples, 1)\n embedding of each neuron / voxel\n* **isort** : array, shape (n_samples,)\n sorting along first dimension of input matrix - use this to get neuron / voxel sorting\n* **igood** : array, shape (n_samples, 1)\n neurons/voxels which had non-zero activity and were used for sorting\n* **Usv** : array, shape (n_samples, n_PCs) \n singular vectors U times singular values sv\n* **Vsv** : array, shape (n_features, n_PCs)\n singular vectors U times singular values sv\n* **U_nodes** : array, shape (n_clusters, n_PCs) \n cluster centers in PC space\n* **Y_nodes** : array, shape (n_clusters, 1) \n np.arange(0, n_clusters)\n* **X_nodes** : array, shape (n_clusters, n_features)\n cluster activity traces in time\n* **cc** : array, shape (n_clusters, n_clusters)\n sorted asymmetric similarity matrix\n* **embedding_clust** : array, shape (n_samples, 1)\n assignment of each neuron/voxel to each cluster (before upsampling)\n* **X** : array, shape (n_samples, n_features)\n normalized data stored (if keep_norm_X is True)\n* **X_embedding** : array, shape (n_samples//bin_size, n_features)\n normalized data binned across samples (if compute_X_embedding is True)\n\nThe output from the GUI and the command line is a file that ends with `_embedding.npy`. This file contains:\n* **filename**: str,\n path to file that rastermap was run on\n* **save_path**: str,\n folder with filename\n* **embedding** : array, shape (n_samples, 1)\n embedding of each neuron / voxel\n* **isort** : array, shape (n_samples,)\n sorting along first dimension of input matrix - use this to get neuron / voxel sorting\n* **user_clusters**: list, \n list of user drawn clusters in GUI\n* **ops**: dict,\n dictionary of options used to run rastermap\n\n\n# License\n\nCopyright (C) 2023 Howard Hughes Medical Institute Janelia Research Campus, the labs of Carsen Stringer and Marius Pachitariu.\n\n**This code is licensed under GPL v3 (no redistribution without credit, and no redistribution in private repos, see the [license](LICENSE) for more details).**\n",
"bugtrack_url": null,
"license": null,
"summary": "Unsupervised clustering algorithm for 2D data (neurons by time)",
"version": "1.0",
"project_urls": {
"Homepage": "https://github.com/MouseLand/rastermap"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c0c801a9eafcc8e2c074c494824f33cc688f6ad4ac98964b6ca5d8c2368246b0",
"md5": "5f765bf73314add3909de1d5c3190eed",
"sha256": "c7ec55d625bc88cf97df50d5a2e068869e5790091ea595120f497cc7fab9370a"
},
"downloads": -1,
"filename": "rastermap-1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "5f765bf73314add3909de1d5c3190eed",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 90535,
"upload_time": "2024-10-18T20:05:11",
"upload_time_iso_8601": "2024-10-18T20:05:11.694999Z",
"url": "https://files.pythonhosted.org/packages/c0/c8/01a9eafcc8e2c074c494824f33cc688f6ad4ac98964b6ca5d8c2368246b0/rastermap-1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "31dbe0ed0068934fb588a8e934ebeafb39adbcf9d709a56cb3cb83b3e72be959",
"md5": "d466d4f3d80a5d925ca51c8553c29188",
"sha256": "e71fb30346737ecde596cae27ab8be066988d1bf8d1b54601bbb99e72021e2e8"
},
"downloads": -1,
"filename": "rastermap-1.0.tar.gz",
"has_sig": false,
"md5_digest": "d466d4f3d80a5d925ca51c8553c29188",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 169907,
"upload_time": "2024-10-18T20:05:13",
"upload_time_iso_8601": "2024-10-18T20:05:13.127393Z",
"url": "https://files.pythonhosted.org/packages/31/db/e0ed0068934fb588a8e934ebeafb39adbcf9d709a56cb3cb83b3e72be959/rastermap-1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-18 20:05:13",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "MouseLand",
"github_project": "rastermap",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "rastermap"
}
JSON
