[](https://github.com/guiwitz/napari-convpaint/raw/main/LICENSE)
[](https://pypi.org/project/napari-convpaint)
[](https://python.org)
[](https://github.com/guiwitz/napari-convpaint/actions)
[](https://codecov.io/gh/guiwitz/napari-convpaint)
[](https://napari-hub.org/plugins/napari-convpaint)
This napari plugin can be used to segment objects or structures in images based on a few brush strokes providing examples of the classes. Based on the same idea as other tools like ilastik, its main strength is that it can use features from pretrained neural networks like VGG16 or DINOV2, enabling the segmentation of more complex images.
**Find more information and tutorials in the [docs](https://guiwitz.github.io/napari-convpaint/) or read the [preprint](https://doi.org/10.1101/2024.09.12.610926).**
## Installation
You can install `napari-convpaint` via [pip]
pip install napari-convpaint
To install latest development version :
pip install git+https://github.com/guiwitz/napari-convpaint.git
## Example use case: Tracking shark body parts in a movie
These are the scribble annotations provided for training:
And this is the resulting Convpaint segmentation:
<video src="https://github.com/user-attachments/assets/6a2be1fe-25cc-4af1-9f50-aab9bc4123d9"></video>
Check out the documentation or the paper for more usecases!
## API
You can now use the API in a fashion very similar to the napari plugin. The ConvpaintModel class combines a feature extractor and a classifier model, and holds all the parameters defining the model. Initialize a ConvpaintModel object, train its classifier and use it to segment an image:
```Python
cp_model = ConvpaintModel("dino") # alternatively use vgg, cellpose or gaussian
cp_model.train(image, annotations)
segmentation = cp_model.segment(image)
```
There are many other options, such as predicting all class probabilities (see below) and we will update the documentation and notebook examples soon. In the meantime feel free to test it yourself.
```Python
probas = cp_model.predict_probas(image)
```
## License
Distributed under the terms of the [BSD-3] license,
"napari-convpaint" is free and open source software
## Contributing
Contributions are very welcome. Tests can be run with [tox], please ensure
the coverage at least stays the same before you submit a pull request.
## Issues
If you encounter any problems, please [file an issue] along with a detailed description.
[napari]: https://github.com/napari/napari
[Cookiecutter]: https://github.com/audreyr/cookiecutter
[@napari]: https://github.com/napari
[MIT]: http://opensource.org/licenses/MIT
[BSD-3]: http://opensource.org/licenses/BSD-3-Clause
[GNU GPL v3.0]: http://www.gnu.org/licenses/gpl-3.0.txt
[GNU LGPL v3.0]: http://www.gnu.org/licenses/lgpl-3.0.txt
[Apache Software License 2.0]: http://www.apache.org/licenses/LICENSE-2.0
[Mozilla Public License 2.0]: https://www.mozilla.org/media/MPL/2.0/index.txt
[cookiecutter-napari-plugin]: https://github.com/napari/cookiecutter-napari-plugin
[file an issue]: https://github.com/guiwitz/napari-convpaint/issues
[napari]: https://github.com/napari/napari
[tox]: https://tox.readthedocs.io/en/latest/
[pip]: https://pypi.org/project/pip/
[PyPI]: https://pypi.org/
## Authors
The idea behind this napari plugin was first developed by [Lucien Hinderling](https://hinderling.github.io) in the group of [Olivier Pertz](https://www.pertzlab.net/), at the Institute of Cell Biology, University of Bern. Pertz lab obtained a CZI napari plugin development grant with the title ["Democratizing Image Analysis with an Easy-to-Train Classifier"](https://chanzuckerberg.com/science/programs-resources/imaging/napari/democratizing-image-analysis-with-an-easy-to-train-classifier/) which supported the adaptation of the initial concept as a napari plugin called napari-convpaint. The plugin has been developed by [Guillaume Witz<sup>1</sup>](https://guiwitz.github.io/blog/about/), [Roman Schwob<sup>1,2</sup>](https://github.com/quasar1357) and Lucien Hinderling<sup>2</sup> with much appreciated assistance of [Benjamin Grädel<sup>2</sup>](https://x.com/benigraedel), [Maciej Dobrzyński<sup>2</sup>](https://macdobry.net), Mykhailo Vladymyrov<sup>1</sup> and Ana Stojiljković<sup>1</sup>.
<sup>1</sup>[Data Science Lab](https://www.dsl.unibe.ch/), University of Bern \
<sup>2</sup>[Pertz Lab](https://www.pertzlab.net/), Institute of Cell Biology, University of Bern
## Cite Convpaint
If you find Convpaint useful in your research, please consider citing our work. Please also cite any Feature Extractor you have used in Convpaint, such as [ilastik](https://github.com/ilastik/ilastik-napari), [cellpose](https://cellpose.readthedocs.io/en/latest/) or [DINOv2](https://github.com/facebookresearch/dinov2).
Convpaint:
```
@article {Hinderling2024,
author = {Hinderling, Lucien and Witz, Guillaume and Schwob, Roman and Stojiljković, Ana and Dobrzyński, Maciej and Vladymyrov, Mykhailo and Frei, Joël and Grädel, Benjamin and Frismantiene, Agne and Pertz, Olivier},
title = {Convpaint - Interactive pixel classification using pretrained neural networks},
elocation-id = {2024.09.12.610926},
doi = {10.1101/2024.09.12.610926},
journal = {bioRxiv},
publisher = {Cold Spring Harbor Laboratory},
year = {2024},
}
```
Suggested citations for feature extractors:
```
@article {Berg2019,
author = {Berg, Stuart and Kutra, Dominik and Kroeger, Thorben and Straehle, Christoph N. and Kausler, Bernhard X. and Haubold, Carsten and Schiegg, Martin and Ales, Janez and Beier, Thorsten and Rudy, Markus and Eren, Kemal and Cervantes, Jaime I. and Xu, Buote and Beuttenmueller, Fynn and Wolny, Adrian and Zhang, Chong and Koethe, Ullrich and Hamprecht, Fred A. and Kreshuk, Anna},
title = {ilastik: interactive machine learning for (bio)image analysis.},
issn = {1548-7105},
url = {https://doi.org/10.1038/s41592-019-0582-9},
doi = {10.1038/s41592-019-0582-9},
journal = {Nature Methods},
publisher = {Springer Nature},
year = {2019},
journal = {Nature Methods},
}
```
```
@article {Stringer2021,
author = {Stringer, Carsen and Wang, Tim and Michaelos, Michalis and Pachitariu Marius},
title = {Cellpose: a generalist algorithm for cellular segmentation.},
elocation-id = {s41592-020-01018-x},
doi = {10.1038/s41592-020-01018-x},
journal = {Nature Methods},
publisher = {Springer Nature},
year = {2021},
}
```
```
@article {oquab2024dinov2learningrobustvisual,
title={DINOv2: Learning Robust Visual Features without Supervision},
author={Maxime Oquab and Timothée Darcet and Théo Moutakanni and Huy Vo and Marc Szafraniec and Vasil Khalidov and Pierre Fernandez and Daniel Haziza and Francisco Massa and Alaaeldin El-Nouby and Mahmoud Assran and Nicolas Ballas and Wojciech Galuba and Russell Howes and Po-Yao Huang and Shang-Wen Li and Ishan Misra and Michael Rabbat and Vasu Sharma and Gabriel Synnaeve and Hu Xu and Hervé Jegou and Julien Mairal and Patrick Labatut and Armand Joulin and Piotr Bojanowski},
year={2024},
eprint={2304.07193},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2304.07193}
}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/guiwitz/napari-convpaint",
"name": "napari-convpaint",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "napari, plugin, segmentation, machine learning, deep learning, image analysis, neural networks",
"author": "Guillaume Witz",
"author_email": "guillaume.witz@unibe.ch",
"download_url": "https://files.pythonhosted.org/packages/22/74/d61e45416506fbb9ca7f8a6e7574df6d2a212f72a4b19394c0fac69c6420/napari_convpaint-0.7.1.tar.gz",
"platform": null,
"description": "\n[](https://github.com/guiwitz/napari-convpaint/raw/main/LICENSE)\n[](https://pypi.org/project/napari-convpaint)\n[](https://python.org)\n[](https://github.com/guiwitz/napari-convpaint/actions)\n[](https://codecov.io/gh/guiwitz/napari-convpaint)\n[](https://napari-hub.org/plugins/napari-convpaint)\n\n\n\n\nThis napari plugin can be used to segment objects or structures in images based on a few brush strokes providing examples of the classes. Based on the same idea as other tools like ilastik, its main strength is that it can use features from pretrained neural networks like VGG16 or DINOV2, enabling the segmentation of more complex images.\n\n**Find more information and tutorials in the [docs](https://guiwitz.github.io/napari-convpaint/) or read the [preprint](https://doi.org/10.1101/2024.09.12.610926).**\n\n\n\n\n## Installation\n\nYou can install `napari-convpaint` via [pip]\n\n pip install napari-convpaint\n\nTo install latest development version :\n\n pip install git+https://github.com/guiwitz/napari-convpaint.git\n\n\n## Example use case: Tracking shark body parts in a movie\nThese are the scribble annotations provided for training:\n\n\nAnd this is the resulting Convpaint segmentation:\n<video src=\"https://github.com/user-attachments/assets/6a2be1fe-25cc-4af1-9f50-aab9bc4123d9\"></video>\n\nCheck out the documentation or the paper for more usecases!\n\n## API\n\nYou can now use the API in a fashion very similar to the napari plugin. The ConvpaintModel class combines a feature extractor and a classifier model, and holds all the parameters defining the model. Initialize a ConvpaintModel object, train its classifier and use it to segment an image:\n\n```Python\ncp_model = ConvpaintModel(\"dino\") # alternatively use vgg, cellpose or gaussian\ncp_model.train(image, annotations)\nsegmentation = cp_model.segment(image)\n```\n\nThere are many other options, such as predicting all class probabilities (see below) and we will update the documentation and notebook examples soon. In the meantime feel free to test it yourself.\n\n```Python\nprobas = cp_model.predict_probas(image)\n```\n\n## License\n\nDistributed under the terms of the [BSD-3] license,\n\"napari-convpaint\" is free and open source software\n\n## Contributing\n\nContributions are very welcome. Tests can be run with [tox], please ensure\nthe coverage at least stays the same before you submit a pull request.\n\n## Issues\n\nIf you encounter any problems, please [file an issue] along with a detailed description.\n\n[napari]: https://github.com/napari/napari\n[Cookiecutter]: https://github.com/audreyr/cookiecutter\n[@napari]: https://github.com/napari\n[MIT]: http://opensource.org/licenses/MIT\n[BSD-3]: http://opensource.org/licenses/BSD-3-Clause\n[GNU GPL v3.0]: http://www.gnu.org/licenses/gpl-3.0.txt\n[GNU LGPL v3.0]: http://www.gnu.org/licenses/lgpl-3.0.txt\n[Apache Software License 2.0]: http://www.apache.org/licenses/LICENSE-2.0\n[Mozilla Public License 2.0]: https://www.mozilla.org/media/MPL/2.0/index.txt\n[cookiecutter-napari-plugin]: https://github.com/napari/cookiecutter-napari-plugin\n\n[file an issue]: https://github.com/guiwitz/napari-convpaint/issues\n\n[napari]: https://github.com/napari/napari\n[tox]: https://tox.readthedocs.io/en/latest/\n[pip]: https://pypi.org/project/pip/\n[PyPI]: https://pypi.org/\n\n## Authors\n\nThe idea behind this napari plugin was first developed by [Lucien Hinderling](https://hinderling.github.io) in the group of [Olivier Pertz](https://www.pertzlab.net/), at the Institute of Cell Biology, University of Bern. Pertz lab obtained a CZI napari plugin development grant with the title [\"Democratizing Image Analysis with an Easy-to-Train Classifier\"](https://chanzuckerberg.com/science/programs-resources/imaging/napari/democratizing-image-analysis-with-an-easy-to-train-classifier/) which supported the adaptation of the initial concept as a napari plugin called napari-convpaint. The plugin has been developed by [Guillaume Witz<sup>1</sup>](https://guiwitz.github.io/blog/about/), [Roman Schwob<sup>1,2</sup>](https://github.com/quasar1357) and Lucien Hinderling<sup>2</sup> with much appreciated assistance of [Benjamin Gr\u00e4del<sup>2</sup>](https://x.com/benigraedel), [Maciej Dobrzy\u0144ski<sup>2</sup>](https://macdobry.net), Mykhailo Vladymyrov<sup>1</sup> and Ana Stojiljkovi\u0107<sup>1</sup>.\n\n<sup>1</sup>[Data Science Lab](https://www.dsl.unibe.ch/), University of Bern \\\n<sup>2</sup>[Pertz Lab](https://www.pertzlab.net/), Institute of Cell Biology, University of Bern \n\n## Cite Convpaint\n\nIf you find Convpaint useful in your research, please consider citing our work. Please also cite any Feature Extractor you have used in Convpaint, such as [ilastik](https://github.com/ilastik/ilastik-napari), [cellpose](https://cellpose.readthedocs.io/en/latest/) or [DINOv2](https://github.com/facebookresearch/dinov2).\n\nConvpaint:\n```\n@article {Hinderling2024,\n\tauthor = {Hinderling, Lucien and Witz, Guillaume and Schwob, Roman and Stojiljkovi\u0107, Ana and Dobrzy\u0144ski, Maciej and Vladymyrov, Mykhailo and Frei, Jo\u00ebl and Gr\u00e4del, Benjamin and Frismantiene, Agne and Pertz, Olivier},\n\ttitle = {Convpaint - Interactive pixel classification using pretrained neural networks},\n\telocation-id = {2024.09.12.610926},\n\tdoi = {10.1101/2024.09.12.610926},\n\tjournal = {bioRxiv},\n\tpublisher = {Cold Spring Harbor Laboratory},\n\tyear = {2024},\n}\n```\nSuggested citations for feature extractors:\n```\n@article {Berg2019,\n\tauthor = {Berg, Stuart and Kutra, Dominik and Kroeger, Thorben and Straehle, Christoph N. and Kausler, Bernhard X. and Haubold, Carsten and Schiegg, Martin and Ales, Janez and Beier, Thorsten and Rudy, Markus and Eren, Kemal and Cervantes, Jaime I. and Xu, Buote and Beuttenmueller, Fynn and Wolny, Adrian and Zhang, Chong and Koethe, Ullrich and Hamprecht, Fred A. and Kreshuk, Anna},\n\ttitle = {ilastik: interactive machine learning for (bio)image analysis.},\n\tissn = {1548-7105},\n\turl = {https://doi.org/10.1038/s41592-019-0582-9},\n\tdoi = {10.1038/s41592-019-0582-9},\n\tjournal = {Nature Methods},\n\tpublisher = {Springer Nature},\n\tyear = {2019},\n\tjournal = {Nature Methods},\n}\n```\n```\n@article {Stringer2021,\n\tauthor = {Stringer, Carsen and Wang, Tim and Michaelos, Michalis and Pachitariu Marius},\n\ttitle = {Cellpose: a generalist algorithm for cellular segmentation.},\n\telocation-id = {s41592-020-01018-x},\n\tdoi = {10.1038/s41592-020-01018-x},\n\tjournal = {Nature Methods},\n\tpublisher = {Springer Nature},\n\tyear = {2021},\n}\n```\n```\n@article {oquab2024dinov2learningrobustvisual,\n title={DINOv2: Learning Robust Visual Features without Supervision}, \n author={Maxime Oquab and Timoth\u00e9e Darcet and Th\u00e9o Moutakanni and Huy Vo and Marc Szafraniec and Vasil Khalidov and Pierre Fernandez and Daniel Haziza and Francisco Massa and Alaaeldin El-Nouby and Mahmoud Assran and Nicolas Ballas and Wojciech Galuba and Russell Howes and Po-Yao Huang and Shang-Wen Li and Ishan Misra and Michael Rabbat and Vasu Sharma and Gabriel Synnaeve and Hu Xu and Herv\u00e9 Jegou and Julien Mairal and Patrick Labatut and Armand Joulin and Piotr Bojanowski},\n year={2024},\n eprint={2304.07193},\n archivePrefix={arXiv},\n primaryClass={cs.CV},\n url={https://arxiv.org/abs/2304.07193}\n}\n\n```\n",
"bugtrack_url": null,
"license": "BSD-3-Clause",
"summary": "A plugin for segmentation by pixel classification using pre-trained neural networks for feature extraction",
"version": "0.7.1",
"project_urls": {
"Bug Tracker": "https://github.com/guiwitz/napari-convpaint/issues",
"Documentation": "https://github.com/guiwitz/napari-convpaint#README.md",
"Homepage": "https://github.com/guiwitz/napari-convpaint",
"Source Code": "https://github.com/guiwitz/napari-convpaint",
"User Support": "https://github.com/guiwitz/napari-convpaint/issues"
},
"split_keywords": [
"napari",
" plugin",
" segmentation",
" machine learning",
" deep learning",
" image analysis",
" neural networks"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "942c7dc5f26a71aefee7ead76c40a9c72dacadb6442aa9916715feec39fae476",
"md5": "c2e36d544cbbf3a8e7f9686ac4fd4bac",
"sha256": "5d2a1beecfe897f1b83efd360a61a767b1f734d1dab294c1fcdd77a1e969bebe"
},
"downloads": -1,
"filename": "napari_convpaint-0.7.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c2e36d544cbbf3a8e7f9686ac4fd4bac",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 372074,
"upload_time": "2025-09-01T20:57:45",
"upload_time_iso_8601": "2025-09-01T20:57:45.225957Z",
"url": "https://files.pythonhosted.org/packages/94/2c/7dc5f26a71aefee7ead76c40a9c72dacadb6442aa9916715feec39fae476/napari_convpaint-0.7.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "2274d61e45416506fbb9ca7f8a6e7574df6d2a212f72a4b19394c0fac69c6420",
"md5": "662804fb963b60abd1c70bf2dad704fa",
"sha256": "53dcc8eb2178839a66e6295e8f455592ab9d1dca95e8eeb8e3f379093847697b"
},
"downloads": -1,
"filename": "napari_convpaint-0.7.1.tar.gz",
"has_sig": false,
"md5_digest": "662804fb963b60abd1c70bf2dad704fa",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 17711573,
"upload_time": "2025-09-01T20:57:46",
"upload_time_iso_8601": "2025-09-01T20:57:46.597131Z",
"url": "https://files.pythonhosted.org/packages/22/74/d61e45416506fbb9ca7f8a6e7574df6d2a212f72a4b19394c0fac69c6420/napari_convpaint-0.7.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-09-01 20:57:46",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "guiwitz",
"github_project": "napari-convpaint",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "napari-convpaint"
}
JSON
