| Name | Microns-DataCleaner JSON |
| Version |
0.1.5
JSON |
| download |
| home_page | None |
| Summary | Provides common functions to download and process data from the mm3 data. |
| upload_time | 2025-09-03 16:41:58 |
| maintainer | All contributors |
| docs_url | None |
| author | Victor Buendia |
| requires_python | >=3.9 |
| license | None |
| keywords |
neuroscience
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# MICrONS-datacleaner
[](https://opensource.org/licenses/MIT)
This project contains tools to work with the [MICrONS Cortical MM3 dataset](https://www.microns-explorer.org/cortical-mm3), providing a **robust interface** to interact with the nucleus data.
## Key features
- **Simple interface** to download and keep organized anatomical data via CAVEClient.
- **Allows to query the synapse table in chunks** avoiding common pitfalls.
- **Easily process nucleus annotation tables**.
- **Automatically segment** the brain volume into cortical **layers.**
- **Tools for filtering** and constructing connectome subsets.
- Basic interface to add functional properties, including **tuning curves and selectivity**.
## Install đź“Ą
```bash
pip install microns-datacleaner
```
## Using the package ⏩
- **Few lines of code** to get a full table with neurons' brain area, layer, cell_type, proofreading information, and nucleus position:
```python
#Import the lib
import microns_datacleaner as mic
#Target version and download folder
cleaner = mic.MicronsDataCleaner(datadir = "data", version=1300)
#Download the data
cleaner.download_nucleus_data()
#Process the downloaded data and segment into layers
units, segments = cleaner.process_nucleus_data()
```
- **Filter easily!** How can we get all neurons in V1, layers L2/3 and L4 with proofread axons?
```python
units_filter = fl.filter_neurons(units, layer=['L2/3', 'L4'], proofread='ax_clean', brain_area='V1')
```
- **Robustly download synapses** between a subset of pre and post-synaptic neurons in chunks.
```python
preids = units_filter['pt_root_id']
postids = units_filter['pt_root_id']
cleaner.download_synapse_data(preids, postids)
#Connection problems at chunk number 23? Just restart from there
cleanerdownload_synapse_data(preids, postids, start_index=23)
```
Check the docs and our tutorial notebook just below to get started!
## Docs & Tutorials đź“ś
If it is the first time working with the MICrONS data, we recommend you read our basic tutorial (also available as a Python Notebook), as well as the official documentation of the MICrONS project.
If you want to contribute, please read our guidelines first. Feel free to open an issue if you find any problem.
You can find a full documentation of the API and functions in the [docs](https://margheritapremi.github.io/MICrONS-datacleaner).
## Requirements
### Dependencies
- CaveCLIENT
- Pandas
- Numpy
- Scipy
- TQDM
- Standard transform for coordinate change (MICrONS ecosystem)
### Dev-dependencies
- pdoc (to generate the docs)
- ruff (to keep contributions in a consistent format)
## Citation Policy đź“š
If you use our code, **please consider to cite the associated repository,** as well as the [IARPA MICrONS Minnie Project](https://doi.org/10.60533/BOSS-2021-T0SY) and the [Microns Phase 3 NDA](https://github.com/cajal/microns_phase3_nda) repository.
Our code serves as an interface for the MICrONS data. Please cite appropiate the literature for the data used following their [Citation Policy](https://www.microns-explorer.org/citation-policy). The papers may depend on the annotation tables used.
Our unit table is constructed by integrating information from the following papers:
1. [Functional connectomics spanning multiple areas of mouse visual cortex](https://doi.org/10.1038/s41586-025-08790-w). The Microns Consortium. 2025
2. [Foundation model of neural activity predicts response to new stimulus types](https://doi.org/10.1038/s41586-025-08829-y)
3. [Perisomatic ultrastructure efficiently classifies cells in mouse cortex](http://doi.org/10.1038/s41586-024-07765-7)
4. [NEURD offers automated proofreading and feature extraction for connectomics](https://doi.org/)
5. [CAVE: Connectome Annotation Versioning Engine](https://doi.org/10.1038/s41592-024-02426-z)
## Acknowledgements
We acknowledge funding by the NextGenerationEU, in the framework of the FAIR—Future Artificial Intelligence Research project (FAIR PE00000013—CUP B43C22000800006).
## Generating the Docs
## Generating the docs
Go to the main folder of the repository, and run
```
pdoc -t docs/template source/mic_datacleaner.py -o docs/html
```
The docs will be generated in the `docs/html` folder in HTML format, which can be checked with the browser. If you need the docs for all the files, and not only the class, use `source/*.py` instead of `source/mic_datacleaner.py` above.
Raw data
{
"_id": null,
"home_page": null,
"name": "Microns-DataCleaner",
"maintainer": "All contributors",
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "neuroscience",
"author": "Victor Buendia",
"author_email": "victor.buendia@unibocconi.it",
"download_url": "https://files.pythonhosted.org/packages/c9/e6/1f7830bd0bdc12c15d8bf346ace2c042b669abc9ca85dd0d1e80413a4ffb/microns_datacleaner-0.1.5.tar.gz",
"platform": null,
"description": "# MICrONS-datacleaner\n\n[](https://opensource.org/licenses/MIT)\n\nThis project contains tools to work with the [MICrONS Cortical MM3 dataset](https://www.microns-explorer.org/cortical-mm3), providing a **robust interface** to interact with the nucleus data. \n\n## Key features \n\n- **Simple interface** to download and keep organized anatomical data via CAVEClient. \n- **Allows to query the synapse table in chunks** avoiding common pitfalls. \n- **Easily process nucleus annotation tables**. \n- **Automatically segment** the brain volume into cortical **layers.**\n- **Tools for filtering** and constructing connectome subsets. \n- Basic interface to add functional properties, including **tuning curves and selectivity**. \n\n## Install \ud83d\udce5\n\n```bash\npip install microns-datacleaner\n```\n\n## Using the package \u23e9\n\n- **Few lines of code** to get a full table with neurons' brain area, layer, cell_type, proofreading information, and nucleus position:\n\n```python\n#Import the lib\nimport microns_datacleaner as mic\n\n#Target version and download folder\ncleaner = mic.MicronsDataCleaner(datadir = \"data\", version=1300) \n\n#Download the data\ncleaner.download_nucleus_data()\n\n#Process the downloaded data and segment into layers\nunits, segments = cleaner.process_nucleus_data()\n```\n\n- **Filter easily!** How can we get all neurons in V1, layers L2/3 and L4 with proofread axons?\n\n```python\nunits_filter = fl.filter_neurons(units, layer=['L2/3', 'L4'], proofread='ax_clean', brain_area='V1')\n```\n\n- **Robustly download synapses** between a subset of pre and post-synaptic neurons in chunks. \n\n```python\npreids = units_filter['pt_root_id']\npostids = units_filter['pt_root_id']\ncleaner.download_synapse_data(preids, postids)\n\n#Connection problems at chunk number 23? Just restart from there\ncleanerdownload_synapse_data(preids, postids, start_index=23)\n```\n\nCheck the docs and our tutorial notebook just below to get started!\n\n\n## Docs & Tutorials \ud83d\udcdc \n\nIf it is the first time working with the MICrONS data, we recommend you read our basic tutorial (also available as a Python Notebook), as well as the official documentation of the MICrONS project. \n\nIf you want to contribute, please read our guidelines first. Feel free to open an issue if you find any problem. \n\nYou can find a full documentation of the API and functions in the [docs](https://margheritapremi.github.io/MICrONS-datacleaner). \n\n\n## Requirements \n\n### Dependencies \n\n- CaveCLIENT\n- Pandas\n- Numpy\n- Scipy\n- TQDM\n- Standard transform for coordinate change (MICrONS ecosystem)\n\n### Dev-dependencies \n\n- pdoc (to generate the docs)\n- ruff (to keep contributions in a consistent format)\n\n\n\n## Citation Policy \ud83d\udcda\n\nIf you use our code, **please consider to cite the associated repository,** as well as the [IARPA MICrONS Minnie Project](https://doi.org/10.60533/BOSS-2021-T0SY) and the [Microns Phase 3 NDA](https://github.com/cajal/microns_phase3_nda) repository. \n\nOur code serves as an interface for the MICrONS data. Please cite appropiate the literature for the data used following their [Citation Policy](https://www.microns-explorer.org/citation-policy). The papers may depend on the annotation tables used.\n\nOur unit table is constructed by integrating information from the following papers:\n\n1. [Functional connectomics spanning multiple areas of mouse visual cortex](https://doi.org/10.1038/s41586-025-08790-w). The Microns Consortium. 2025 \n2. [Foundation model of neural activity predicts response to new stimulus types](https://doi.org/10.1038/s41586-025-08829-y)\n3. [Perisomatic ultrastructure efficiently classifies cells in mouse cortex](http://doi.org/10.1038/s41586-024-07765-7)\n4. [NEURD offers automated proofreading and feature extraction for connectomics](https://doi.org/)\n5. [CAVE: Connectome Annotation Versioning Engine](https://doi.org/10.1038/s41592-024-02426-z)\n\n## Acknowledgements\n\nWe acknowledge funding by the NextGenerationEU, in the framework of the FAIR\u2014Future Artificial Intelligence Research project (FAIR PE00000013\u2014CUP B43C22000800006). \n\n\n## Generating the Docs \n\n## Generating the docs\n\nGo to the main folder of the repository, and run\n\n```\npdoc -t docs/template source/mic_datacleaner.py -o docs/html\n```\n\nThe docs will be generated in the `docs/html` folder in HTML format, which can be checked with the browser. If you need the docs for all the files, and not only the class, use `source/*.py` instead of `source/mic_datacleaner.py` above.\n\n\n",
"bugtrack_url": null,
"license": null,
"summary": "Provides common functions to download and process data from the mm3 data.",
"version": "0.1.5",
"project_urls": null,
"split_keywords": [
"neuroscience"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e7990084f3ffabf1d2aeebf27ab3fbf394ec92f9649ba7801d6c3da06f0cc591",
"md5": "2cb09d0a0c91429cb7f830a915163027",
"sha256": "811dadf31ccb1e6720f3dd09b725896bef1b4cba3265fc91662c7e957b544bed"
},
"downloads": -1,
"filename": "microns_datacleaner-0.1.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2cb09d0a0c91429cb7f830a915163027",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 25214,
"upload_time": "2025-09-03T16:41:56",
"upload_time_iso_8601": "2025-09-03T16:41:56.841080Z",
"url": "https://files.pythonhosted.org/packages/e7/99/0084f3ffabf1d2aeebf27ab3fbf394ec92f9649ba7801d6c3da06f0cc591/microns_datacleaner-0.1.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c9e61f7830bd0bdc12c15d8bf346ace2c042b669abc9ca85dd0d1e80413a4ffb",
"md5": "5c4bf23a721644473f61b067c75543d8",
"sha256": "9b5abcb4b841fb4f7fd32f0e79f9ce0dfbb4af0623f0a097264c29910803e825"
},
"downloads": -1,
"filename": "microns_datacleaner-0.1.5.tar.gz",
"has_sig": false,
"md5_digest": "5c4bf23a721644473f61b067c75543d8",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 23034,
"upload_time": "2025-09-03T16:41:58",
"upload_time_iso_8601": "2025-09-03T16:41:58.056490Z",
"url": "https://files.pythonhosted.org/packages/c9/e6/1f7830bd0bdc12c15d8bf346ace2c042b669abc9ca85dd0d1e80413a4ffb/microns_datacleaner-0.1.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-09-03 16:41:58",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "microns-datacleaner"
}
