# ndx-miniscope Extension for NWB
This is a Neurodata Extension (NDX) for Neurodata Without Borders (NWB) 2.0 for Miniscope acquisition data.
[](https://badge.fury.io/py/ndx-miniscope)
`Miniscope` extends the `Device` core NWB neurodata_type by including additional metadata for the Miniscope.
Depending on the version of the acquisition software the data structure can be quite different.
## Miniscope V4 format
The data recorded by the software is saved in a folder structure similar to this:
C6-J588_Disc5/ (main folder)
├── 15_03_28/ (subfolder corresponding to the recording time)
│ ├── Miniscope/ (subfolder containing the microscope video stream)
│ │ ├── 0.avi (microscope video)
│ │ ├── metaData.json (metadata for the microscope device)
│ │ └── timeStamps.csv (timing of this video stream)
│ ├── BehavCam_2/ (subfolder containing the behavioral video stream)
│ │ ├── 0.avi (bevavioral video)
│ │ ├── metaData.json (metadata for the behavioral camera)
│ │ └── timeStamps.csv (timing of this video stream)
│ └── metaData.json (metadata for the recording, such as the start time)
├── 15_06_28/
│ ├── Miniscope/
│ ├── BehavCam_2/
│ └── metaData.json
└── 15_12_28/
## Miniscope V3 format
The Miniscope V3 acquisition software generally outputs the following files:
* msCam[##].avi
* behavCam[##].avi
* timestamp.dat
* settings_and_notes.dat
## python
### Installation
Get most recent release:
```bash
pip install ndx-miniscope
```
Install latest:
```bash
git clone https://github.com/catalystneuro/ndx-miniscope.git
cd ndx-miniscope
pip install -e .
```
The following code demonstrates the usage of this extension to convert Miniscope acquisition data into NWB.
### Usage
```python
from datetime import datetime
from dateutil.tz import tzlocal
import glob
import os
from pynwb import NWBFile, NWBHDF5IO
from pynwb.image import ImageSeries
from natsort import natsorted
from ndx_miniscope.utils import (
add_miniscope_device,
get_starting_frames,
get_timestamps,
read_miniscope_config,
read_notes,
)
# The main folder that contains subfolders with the Miniscope data
folder_path = "C6-J588_Disc5/"
# Create the NWBFile
session_start_time = datetime(2017, 4, 15, 12, tzinfo=tzlocal())
nwbfile = NWBFile(
session_description="session_description",
identifier="identifier",
session_start_time=session_start_time,
)
# Load the miscroscope settings
miniscope_folder_path = "C6-J588_Disc5/15_03_28/Miniscope/"
miniscope_metadata = read_miniscope_config(folder_path=miniscope_folder_path)
# Create the Miniscope device with the microscope metadata and add it to NWB
add_miniscope_device(nwbfile=nwbfile, device_metadata=miniscope_metadata)
# Load the behavioral camera settings
behavcam_folder_path = "C6-J588_Disc5/15_03_28/BehavCam_2/"
behavcam_metadata = read_miniscope_config(folder_path=behavcam_folder_path)
# Create the Miniscope device with the behavioral camera metadata and add it to NWB
add_miniscope_device(nwbfile=nwbfile, device_metadata=behavcam_metadata)
# Loading the timestamps
behavcam_timestamps = get_timestamps(folder_path=folder_path, file_pattern="BehavCam*/timeStamps.csv")
# Load the starting frames of the video files
# Note this function requires to have `cv2` installed
starting_frames = get_starting_frames(folder_path=folder_path, video_file_pattern="*/BehavCam*/*.avi")
# Legacy usage for Miniscope V3
ms_files = natsorted(glob(os.path.join(folder_path, 'msCam*.avi')))
nwbfile.add_acquisition(
ImageSeries(
name='OnePhotonSeries', # this is not recommended since pynwb has native OnePhotonSeries
format='external',
external_file=[os.path.split(x)[1] for x in ms_files],
timestamps=get_timestamps(folder_path=folder_path, cam_num=1),
starting_frame=get_starting_frames(folder_path=folder_path, video_file_pattern="msCam*.avi"),
)
)
behav_files = natsorted(glob(os.path.join(folder_path, 'behavCam*.avi')))
nwbfile.add_acquisition(
ImageSeries(
name='behaviorCam',
format='external',
external_file=[os.path.split(x)[1] for x in behav_files],
timestamps=get_timestamps(folder_path=folder_path, cam_num=2),
starting_frame=get_starting_frames(folder_path=folder_path, video_file_pattern="behavCam*.avi"),
)
)
annotations = read_notes(folder_path=folder_path)
if annotations is not None:
nwbfile.add_acquisition(annotations)
save_path = os.path.join(folder_path, "test_out.nwb")
with NWBHDF5IO(save_path, "w") as io:
io.write(nwbfile)
# test read
with NWBHDF5IO(save_path, "r") as io:
nwbfile_in = io.read()
```
## MATLAB:
### Installation
```bash
git clone https://github.com/bendichter/ndx-miniscope.git
```
```matlab
generateExtension('path/to/ndx-miniscope/spec');
```
### Usage
under construction...
Raw data
{
"_id": null,
"home_page": "https://github.com/catalystneuro/ndx-miniscope",
"name": "ndx-miniscope",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "NeurodataWithoutBorders,NWB,nwb-extension,ndx-extension",
"author": "Ben Dichter",
"author_email": "ben.dichter@catalystneuro.com",
"download_url": "https://files.pythonhosted.org/packages/fa/25/338cb8fa6f444e5a15035fa845a9be0f40e7b9f46085b7d7ff3db39200f6/ndx-miniscope-0.5.1.tar.gz",
"platform": null,
"description": "# ndx-miniscope Extension for NWB\n\nThis is a Neurodata Extension (NDX) for Neurodata Without Borders (NWB) 2.0 for Miniscope acquisition data.\n\n[](https://badge.fury.io/py/ndx-miniscope)\n\n`Miniscope` extends the `Device` core NWB neurodata_type by including additional metadata for the Miniscope.\nDepending on the version of the acquisition software the data structure can be quite different.\n\n## Miniscope V4 format\nThe data recorded by the software is saved in a folder structure similar to this:\n\n C6-J588_Disc5/ (main folder)\n \u251c\u2500\u2500 15_03_28/ (subfolder corresponding to the recording time)\n \u2502 \u251c\u2500\u2500 Miniscope/ (subfolder containing the microscope video stream)\n \u2502 \u2502 \u251c\u2500\u2500 0.avi (microscope video)\n \u2502 \u2502 \u251c\u2500\u2500 metaData.json (metadata for the microscope device)\n \u2502 \u2502 \u2514\u2500\u2500 timeStamps.csv (timing of this video stream)\n \u2502 \u251c\u2500\u2500 BehavCam_2/ (subfolder containing the behavioral video stream)\n \u2502 \u2502 \u251c\u2500\u2500 0.avi (bevavioral video)\n \u2502 \u2502 \u251c\u2500\u2500 metaData.json (metadata for the behavioral camera)\n \u2502 \u2502 \u2514\u2500\u2500 timeStamps.csv (timing of this video stream)\n \u2502 \u2514\u2500\u2500 metaData.json (metadata for the recording, such as the start time)\n \u251c\u2500\u2500 15_06_28/\n \u2502 \u251c\u2500\u2500 Miniscope/\n \u2502 \u251c\u2500\u2500 BehavCam_2/\n \u2502 \u2514\u2500\u2500 metaData.json\n \u2514\u2500\u2500 15_12_28/\n\n## Miniscope V3 format\nThe Miniscope V3 acquisition software generally outputs the following files:\n\n* msCam[##].avi\n* behavCam[##].avi\n* timestamp.dat\n* settings_and_notes.dat\n\n\n## python\n### Installation\n\nGet most recent release:\n```bash\npip install ndx-miniscope\n```\n\nInstall latest:\n```bash\ngit clone https://github.com/catalystneuro/ndx-miniscope.git\ncd ndx-miniscope\npip install -e .\n```\n\nThe following code demonstrates the usage of this extension to convert Miniscope acquisition data into NWB.\n\n### Usage\n\n```python\nfrom datetime import datetime\nfrom dateutil.tz import tzlocal\nimport glob\nimport os\nfrom pynwb import NWBFile, NWBHDF5IO\nfrom pynwb.image import ImageSeries\nfrom natsort import natsorted\n\nfrom ndx_miniscope.utils import (\n add_miniscope_device,\n get_starting_frames,\n get_timestamps,\n read_miniscope_config,\n read_notes,\n)\n\n# The main folder that contains subfolders with the Miniscope data\nfolder_path = \"C6-J588_Disc5/\"\n\n# Create the NWBFile\nsession_start_time = datetime(2017, 4, 15, 12, tzinfo=tzlocal())\nnwbfile = NWBFile(\n session_description=\"session_description\",\n identifier=\"identifier\",\n session_start_time=session_start_time,\n)\n\n# Load the miscroscope settings\nminiscope_folder_path = \"C6-J588_Disc5/15_03_28/Miniscope/\"\nminiscope_metadata = read_miniscope_config(folder_path=miniscope_folder_path)\n# Create the Miniscope device with the microscope metadata and add it to NWB\nadd_miniscope_device(nwbfile=nwbfile, device_metadata=miniscope_metadata)\n\n# Load the behavioral camera settings\nbehavcam_folder_path = \"C6-J588_Disc5/15_03_28/BehavCam_2/\"\nbehavcam_metadata = read_miniscope_config(folder_path=behavcam_folder_path)\n# Create the Miniscope device with the behavioral camera metadata and add it to NWB\nadd_miniscope_device(nwbfile=nwbfile, device_metadata=behavcam_metadata)\n\n# Loading the timestamps\nbehavcam_timestamps = get_timestamps(folder_path=folder_path, file_pattern=\"BehavCam*/timeStamps.csv\")\n# Load the starting frames of the video files\n# Note this function requires to have `cv2` installed\nstarting_frames = get_starting_frames(folder_path=folder_path, video_file_pattern=\"*/BehavCam*/*.avi\")\n\n\n# Legacy usage for Miniscope V3\n\nms_files = natsorted(glob(os.path.join(folder_path, 'msCam*.avi')))\nnwbfile.add_acquisition(\n ImageSeries(\n name='OnePhotonSeries', # this is not recommended since pynwb has native OnePhotonSeries\n format='external',\n external_file=[os.path.split(x)[1] for x in ms_files],\n timestamps=get_timestamps(folder_path=folder_path, cam_num=1),\n starting_frame=get_starting_frames(folder_path=folder_path, video_file_pattern=\"msCam*.avi\"),\n )\n)\n\nbehav_files = natsorted(glob(os.path.join(folder_path, 'behavCam*.avi')))\nnwbfile.add_acquisition(\n ImageSeries(\n name='behaviorCam',\n format='external',\n external_file=[os.path.split(x)[1] for x in behav_files],\n timestamps=get_timestamps(folder_path=folder_path, cam_num=2),\n starting_frame=get_starting_frames(folder_path=folder_path, video_file_pattern=\"behavCam*.avi\"),\n )\n)\n\nannotations = read_notes(folder_path=folder_path)\nif annotations is not None:\n nwbfile.add_acquisition(annotations)\n\n\nsave_path = os.path.join(folder_path, \"test_out.nwb\")\nwith NWBHDF5IO(save_path, \"w\") as io:\n io.write(nwbfile)\n\n# test read\nwith NWBHDF5IO(save_path, \"r\") as io:\n nwbfile_in = io.read()\n\n```\n\n\n## MATLAB:\n### Installation\n```bash\ngit clone https://github.com/bendichter/ndx-miniscope.git\n```\n```matlab\ngenerateExtension('path/to/ndx-miniscope/spec');\n```\n\n### Usage\nunder construction...\n",
"bugtrack_url": null,
"license": "BSD-3",
"summary": "Represent metadata for Miniscope acquisition system.",
"version": "0.5.1",
"project_urls": {
"Homepage": "https://github.com/catalystneuro/ndx-miniscope"
},
"split_keywords": [
"neurodatawithoutborders",
"nwb",
"nwb-extension",
"ndx-extension"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "627a8ee433b95653c43bee59f7d98e2796aedfa502736a17297f28e3de7a0fbb",
"md5": "e2ef4956a5627edfea7ec6fc1c7d72b1",
"sha256": "00b82fa2501e5ba100893abfb26c48e02bc23866258c9462c61eb6f1fdf821fa"
},
"downloads": -1,
"filename": "ndx_miniscope-0.5.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e2ef4956a5627edfea7ec6fc1c7d72b1",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 11115,
"upload_time": "2023-06-25T17:56:14",
"upload_time_iso_8601": "2023-06-25T17:56:14.578874Z",
"url": "https://files.pythonhosted.org/packages/62/7a/8ee433b95653c43bee59f7d98e2796aedfa502736a17297f28e3de7a0fbb/ndx_miniscope-0.5.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "fa25338cb8fa6f444e5a15035fa845a9be0f40e7b9f46085b7d7ff3db39200f6",
"md5": "f0d135b61496518b70791faf2dda48ff",
"sha256": "5136d4ffdbe2c2456f7f9595aa4ceef098b280b9e4b52c5efc51713376f1d7f5"
},
"downloads": -1,
"filename": "ndx-miniscope-0.5.1.tar.gz",
"has_sig": false,
"md5_digest": "f0d135b61496518b70791faf2dda48ff",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 19175,
"upload_time": "2023-06-25T17:56:16",
"upload_time_iso_8601": "2023-06-25T17:56:16.466085Z",
"url": "https://files.pythonhosted.org/packages/fa/25/338cb8fa6f444e5a15035fa845a9be0f40e7b9f46085b7d7ff3db39200f6/ndx-miniscope-0.5.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-25 17:56:16",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "catalystneuro",
"github_project": "ndx-miniscope",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "pynwb",
"specs": []
},
{
"name": "hdmf-docutils",
"specs": []
},
{
"name": "natsort",
"specs": [
[
">=",
"8.3.1"
]
]
}
],
"lcname": "ndx-miniscope"
}
JSON
