# napari-workflow-optimizer
[](https://github.com/haesleinhuepf/napari-workflow-optimizer/raw/main/LICENSE)
[](https://pypi.org/project/napari-workflow-optimizer)
[](https://python.org)
[](https://github.com/haesleinhuepf/napari-workflow-optimizer/actions)
[](https://codecov.io/gh/haesleinhuepf/napari-workflow-optimizer)
[](https://en.wikipedia.org/wiki/Software_release_life_cycle#Alpha)
[](https://napari-hub.org/plugins/napari-workflow-optimizer)
Optimize image processing workflows in napari for segmentation quality
## Usage
The starting point for workflow optimization is a workflow and some reference ("ground truth") labels image.
The label image can be a sparse annotation, which means only some objects and also parts of objets are annotated (see [hints](https://github.com/haesleinhuepf/napari-workflow-optimizer#optimization-hints)).
These datasets should be ready. You can reproduce the following procedure by downloading an
[examle raw image](https://github.com/haesleinhuepf/napari-workflow-optimizer/blob/main/demo/membranes_2d.tif) (derived from the
[scikit-image cells3d example data set](https://scikit-image.org/docs/dev/api/skimage.data.html#skimage.data.cells3d)) and a corresponding
[sparse annotation label image](https://github.com/haesleinhuepf/napari-workflow-optimizer/blob/main/demo/membranes_2d_sparse_labels.tif).
For reproducing the following procedure, also follow the [installation instructions](https://github.com/haesleinhuepf/napari-workflow-optimizer#optimization-hints) below.
The whole procedure is [also shown in this video](https://github.com/haesleinhuepf/napari-workflow-optimizer/raw/main/docs/napari-workflow-optimizer.mp4), an extended version of the trailer above.
### Step 0: Loading data and setting up the workflow
Load the "membranes_2d.tif" data set, e.g. by drag&drop on napari and start the Assistant from the `Tools > Utilities > Assistant (clEsperanto)` menu.
Click the `Label` button and select as operation "Seeded watershed using local minima as seeds and an intensity threshold (nsbatwm)".
Draw an annotation in a new labels layer or load the example spare annotation "membranes_2d_sparse_labels.tif".
In case the image is not displayed as label image, convert it to a label image by right-clicking on the entry in the layers list:
### Step 1: The Workflow Optimizer
Start the Workflow Optimizer from the `Tools > Utilities > Workflow optimizer (Labels)` menu.
Configure the target layer, showing the label image that should be optimized.
Select the manual annotation as reference layer for the optimization.
Consider increasing the number of iterations. This number depends on your segmenation problem.
In the present example, 100 iterations should be enough.
The optimizer will plot quality over the number of iterations to show the progress of optimization.
To determine the quality, the optimizer will measure the maximum overlap ([Jaccard index](https://en.wikipedia.org/wiki/Jaccard_index))
of any labeled object over the manually annotated objects and calculate the mean of this value over all annotated objects.
After a moment, optimization will finish and update the labeled image.
If your starting point for the optimization was already good, the result may now look better than before.
### Step 2: Manual parameter space plotting
In case the result is not perfect yet (as the fringed segmentation above suggests), consider manual plotting of the
individual parameters and their relation to segmentation quality to get an idea about the surrounding parameter space.
Therefore, click the `Plot` button next to one of the workflow parameters.
Select the range in which the labeling quality should be determined (green arrows). In our example, the optimizer was setting the parameter to 2.34.
Thus, to demonstrate the procedure we plot the parameter space beween 0 and 10.
The quality plotted over this parameter obviously has a local maxium at 2.34, which was detected by the optimizer.
However, it also has another local maxium at 8 and actually a plateau in the quality plot (orange arrows).
For further optimization, we re-configure the algorithm and set a new starting point for optimization of the parameter to 8.
Afterwards, we restart the optimization. It will then optimize the settings again from the new starting point.
After another moment, optimization will finish again, potentially leading to an even better result.
### Step 3: Visualization of results
Make sure the segmentation has high quality by inspecting the result visually. Use the `contour` setting of the labels layer
and hide/show the outlines of the labeled layer:
### Optimization Hints
The Workflow Optimizer uses the [Nelder-Mead simplex method](https://en.wikipedia.org/wiki/Nelder%E2%80%93Mead_method)
for optimizing parameters. This algorithm varies individual parameters and makes steps in the parameter space ideally following a gradient
to a local optimum. Hence, this algorithm may not be capable of determining a global optimum in parameter space.
Parameter optimization is no magic. If it does not immediately work on your data, plot the parameters as introduced in Step 2
and identify parameters with a clear gradient and those with many local maxima.
Consider optimizing the parameters with many local maxima manually and de-selecting their checkboxes for the optimization.
The optimizer will then only optimize the parameters showing the clear gradient.
Repeat these steps a couple of times to get a feeling for your parameter space.
Furthermore, parameter optimization works well if
* the initial settings are close to a good segmentation,
* a small number of parameters (a short workflow) are optimized,
* the reference annotation is prepared carefully and
* the dataset is small. Consider using a small representative crop in case of bigger datasets.
### Workflow optimization scripting
For optimizing workflows from within a jupyter notebook, check out our [example notebook for optimization using spare labels](https://github.com/haesleinhuepf/napari-workflow-optimizer/blob/main/demo/sparse_label_image_optimizer.ipynb).
The examples are more flexible than the graphical user interface and allow for example [optimizing intensity images](https://github.com/haesleinhuepf/napari-workflow-optimizer/blob/main/demo/intensity_image_optimizer.ipynb)
and [binary images](https://github.com/haesleinhuepf/napari-workflow-optimizer/blob/main/demo/binary_image_optimizer.ipynb).
The membrane segmentation workflow optimization similar to the one shown above is also available as [jupyter notebook](https://github.com/haesleinhuepf/napari-workflow-optimizer/blob/main/demo/membrane_segmentation.ipynb).
### Known issues
If you change the workflow architecture after the optimizer window was opened, please re-open it
to select the parameters that should be optimized. Changing parameters is ok and re-opening is not necessary.
----------------------------------
This [napari] plugin was generated with [Cookiecutter] using [@napari]'s [cookiecutter-napari-plugin] template.
## Installation
Furthermore, to reproduce the procedure above, please download and install
[napari](https://napari.org/),
[pyopencl](https://documen.tician.de/pyopencl/),
the [napari-pyclesperanto-assistant](https://www.napari-hub.org/plugins/napari-pyclesperanto-assistant) and
the [napari-segment-blobs-and-things-with-membranes](https://www.napari-hub.org/plugins/napari-segment-blobs-and-things-with-membranes) plugin. E.g. using
[conda](https://docs.conda.io/en/latest/) and [pip](https://pypi.org/project/pip/):
```
conda create --name napari-opti python=3.8
conda activate napari-opti
conda install pyopencl napari
pip install napari-pyclesperanto-assistant napari-segment-blobs-and-things-with-membranes
pip install napari-workflow-optimizer
```
## Contributing
Contributions are very welcome. Tests can be run with [tox], please ensure
the coverage at least stays the same before you submit a pull request.
## License
Distributed under the terms of the [BSD-3] license,
"napari-workflow-optimizer" is free and open source software
## Issues
If you encounter any problems, please [file an issue] along with a detailed description.
[napari]: https://github.com/napari/napari
[Cookiecutter]: https://github.com/audreyr/cookiecutter
[@napari]: https://github.com/napari
[MIT]: http://opensource.org/licenses/MIT
[BSD-3]: http://opensource.org/licenses/BSD-3-Clause
[GNU GPL v3.0]: http://www.gnu.org/licenses/gpl-3.0.txt
[GNU LGPL v3.0]: http://www.gnu.org/licenses/lgpl-3.0.txt
[Apache Software License 2.0]: http://www.apache.org/licenses/LICENSE-2.0
[Mozilla Public License 2.0]: https://www.mozilla.org/media/MPL/2.0/index.txt
[cookiecutter-napari-plugin]: https://github.com/napari/cookiecutter-napari-plugin
[file an issue]: https://github.com/haesleinhuepf/napari-workflow-optimizer/issues
[napari]: https://github.com/napari/napari
[tox]: https://tox.readthedocs.io/en/latest/
[pip]: https://pypi.org/project/pip/
[PyPI]: https://pypi.org/
Raw data
{
"_id": null,
"home_page": "https://github.com/haesleinhuepf/napari-workflow-optimizer",
"name": "napari-workflow-optimizer",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "",
"author": "Robert Haase",
"author_email": "robert.haase@tu-dresden.de",
"download_url": "https://files.pythonhosted.org/packages/64/c0/593f5cc3ce82960e7b3ad1ef7569ec7c31a4718eac0156ab49e99acf0125/napari-workflow-optimizer-0.1.4.tar.gz",
"platform": null,
"description": "# napari-workflow-optimizer\n\n[](https://github.com/haesleinhuepf/napari-workflow-optimizer/raw/main/LICENSE)\n[](https://pypi.org/project/napari-workflow-optimizer)\n[](https://python.org)\n[](https://github.com/haesleinhuepf/napari-workflow-optimizer/actions)\n[](https://codecov.io/gh/haesleinhuepf/napari-workflow-optimizer)\n[](https://en.wikipedia.org/wiki/Software_release_life_cycle#Alpha)\n[](https://napari-hub.org/plugins/napari-workflow-optimizer)\n\nOptimize image processing workflows in napari for segmentation quality\n\n\n\n## Usage\n\nThe starting point for workflow optimization is a workflow and some reference (\"ground truth\") labels image. \nThe label image can be a sparse annotation, which means only some objects and also parts of objets are annotated (see [hints](https://github.com/haesleinhuepf/napari-workflow-optimizer#optimization-hints)). \nThese datasets should be ready. You can reproduce the following procedure by downloading an \n[examle raw image](https://github.com/haesleinhuepf/napari-workflow-optimizer/blob/main/demo/membranes_2d.tif) (derived from the \n[scikit-image cells3d example data set](https://scikit-image.org/docs/dev/api/skimage.data.html#skimage.data.cells3d)) and a corresponding \n[sparse annotation label image](https://github.com/haesleinhuepf/napari-workflow-optimizer/blob/main/demo/membranes_2d_sparse_labels.tif).\nFor reproducing the following procedure, also follow the [installation instructions](https://github.com/haesleinhuepf/napari-workflow-optimizer#optimization-hints) below.\nThe whole procedure is [also shown in this video](https://github.com/haesleinhuepf/napari-workflow-optimizer/raw/main/docs/napari-workflow-optimizer.mp4), an extended version of the trailer above.\n\n### Step 0: Loading data and setting up the workflow\n\nLoad the \"membranes_2d.tif\" data set, e.g. by drag&drop on napari and start the Assistant from the `Tools > Utilities > Assistant (clEsperanto)` menu.\n\n\n\nClick the `Label` button and select as operation \"Seeded watershed using local minima as seeds and an intensity threshold (nsbatwm)\".\n\n\n\nDraw an annotation in a new labels layer or load the example spare annotation \"membranes_2d_sparse_labels.tif\". \n\n\n\nIn case the image is not displayed as label image, convert it to a label image by right-clicking on the entry in the layers list:\n\n\n\n### Step 1: The Workflow Optimizer\n\nStart the Workflow Optimizer from the `Tools > Utilities > Workflow optimizer (Labels)` menu. \nConfigure the target layer, showing the label image that should be optimized.\nSelect the manual annotation as reference layer for the optimization. \nConsider increasing the number of iterations. This number depends on your segmenation problem. \nIn the present example, 100 iterations should be enough.\n\n\n\nThe optimizer will plot quality over the number of iterations to show the progress of optimization. \nTo determine the quality, the optimizer will measure the maximum overlap ([Jaccard index](https://en.wikipedia.org/wiki/Jaccard_index)) \nof any labeled object over the manually annotated objects and calculate the mean of this value over all annotated objects.\nAfter a moment, optimization will finish and update the labeled image. \nIf your starting point for the optimization was already good, the result may now look better than before.\n\n\n\n### Step 2: Manual parameter space plotting\n\nIn case the result is not perfect yet (as the fringed segmentation above suggests), consider manual plotting of the \nindividual parameters and their relation to segmentation quality to get an idea about the surrounding parameter space.\nTherefore, click the `Plot` button next to one of the workflow parameters.\nSelect the range in which the labeling quality should be determined (green arrows). In our example, the optimizer was setting the parameter to 2.34. \nThus, to demonstrate the procedure we plot the parameter space beween 0 and 10. \nThe quality plotted over this parameter obviously has a local maxium at 2.34, which was detected by the optimizer.\nHowever, it also has another local maxium at 8 and actually a plateau in the quality plot (orange arrows).\n\n\n\nFor further optimization, we re-configure the algorithm and set a new starting point for optimization of the parameter to 8.\nAfterwards, we restart the optimization. It will then optimize the settings again from the new starting point.\n\n\n\nAfter another moment, optimization will finish again, potentially leading to an even better result.\n\n\n\n### Step 3: Visualization of results\n\nMake sure the segmentation has high quality by inspecting the result visually. Use the `contour` setting of the labels layer\nand hide/show the outlines of the labeled layer:\n\n\n\n\n\n### Optimization Hints\n\nThe Workflow Optimizer uses the [Nelder-Mead simplex method](https://en.wikipedia.org/wiki/Nelder%E2%80%93Mead_method)\nfor optimizing parameters. This algorithm varies individual parameters and makes steps in the parameter space ideally following a gradient \nto a local optimum. Hence, this algorithm may not be capable of determining a global optimum in parameter space. \nParameter optimization is no magic. If it does not immediately work on your data, plot the parameters as introduced in Step 2 \nand identify parameters with a clear gradient and those with many local maxima. \nConsider optimizing the parameters with many local maxima manually and de-selecting their checkboxes for the optimization.\nThe optimizer will then only optimize the parameters showing the clear gradient. \nRepeat these steps a couple of times to get a feeling for your parameter space. \n\nFurthermore, parameter optimization works well if\n* the initial settings are close to a good segmentation,\n* a small number of parameters (a short workflow) are optimized,\n* the reference annotation is prepared carefully and\n* the dataset is small. Consider using a small representative crop in case of bigger datasets.\n\n### Workflow optimization scripting\n\nFor optimizing workflows from within a jupyter notebook, check out our [example notebook for optimization using spare labels](https://github.com/haesleinhuepf/napari-workflow-optimizer/blob/main/demo/sparse_label_image_optimizer.ipynb). \nThe examples are more flexible than the graphical user interface and allow for example [optimizing intensity images](https://github.com/haesleinhuepf/napari-workflow-optimizer/blob/main/demo/intensity_image_optimizer.ipynb)\nand [binary images](https://github.com/haesleinhuepf/napari-workflow-optimizer/blob/main/demo/binary_image_optimizer.ipynb).\nThe membrane segmentation workflow optimization similar to the one shown above is also available as [jupyter notebook](https://github.com/haesleinhuepf/napari-workflow-optimizer/blob/main/demo/membrane_segmentation.ipynb).\n\n### Known issues\n\nIf you change the workflow architecture after the optimizer window was opened, please re-open it\nto select the parameters that should be optimized. Changing parameters is ok and re-opening is not necessary.\n\n----------------------------------\n\nThis [napari] plugin was generated with [Cookiecutter] using [@napari]'s [cookiecutter-napari-plugin] template.\n\n## Installation\n\nFurthermore, to reproduce the procedure above, please download and install \n[napari](https://napari.org/),\n[pyopencl](https://documen.tician.de/pyopencl/),\nthe [napari-pyclesperanto-assistant](https://www.napari-hub.org/plugins/napari-pyclesperanto-assistant) and\nthe [napari-segment-blobs-and-things-with-membranes](https://www.napari-hub.org/plugins/napari-segment-blobs-and-things-with-membranes) plugin. E.g. using \n[conda](https://docs.conda.io/en/latest/) and [pip](https://pypi.org/project/pip/):\n\n```\nconda create --name napari-opti python=3.8\nconda activate napari-opti\n\nconda install pyopencl napari\npip install napari-pyclesperanto-assistant napari-segment-blobs-and-things-with-membranes\npip install napari-workflow-optimizer\n```\n\n## Contributing\n\nContributions are very welcome. Tests can be run with [tox], please ensure\nthe coverage at least stays the same before you submit a pull request.\n\n## License\n\nDistributed under the terms of the [BSD-3] license,\n\"napari-workflow-optimizer\" is free and open source software\n\n## Issues\n\nIf you encounter any problems, please [file an issue] along with a detailed description.\n\n[napari]: https://github.com/napari/napari\n[Cookiecutter]: https://github.com/audreyr/cookiecutter\n[@napari]: https://github.com/napari\n[MIT]: http://opensource.org/licenses/MIT\n[BSD-3]: http://opensource.org/licenses/BSD-3-Clause\n[GNU GPL v3.0]: http://www.gnu.org/licenses/gpl-3.0.txt\n[GNU LGPL v3.0]: http://www.gnu.org/licenses/lgpl-3.0.txt\n[Apache Software License 2.0]: http://www.apache.org/licenses/LICENSE-2.0\n[Mozilla Public License 2.0]: https://www.mozilla.org/media/MPL/2.0/index.txt\n[cookiecutter-napari-plugin]: https://github.com/napari/cookiecutter-napari-plugin\n\n[file an issue]: https://github.com/haesleinhuepf/napari-workflow-optimizer/issues\n\n[napari]: https://github.com/napari/napari\n[tox]: https://tox.readthedocs.io/en/latest/\n[pip]: https://pypi.org/project/pip/\n[PyPI]: https://pypi.org/\n\n\n",
"bugtrack_url": null,
"license": "BSD-3-Clause",
"summary": "Optimize image processing workflows in napari for segmentation quality",
"version": "0.1.4",
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"md5": "6427e72fe7c2428502ef97f351d707ec",
"sha256": "0aaf249c14acbdaba8587fff1a74eea53907e94434fa629dc85dd4bd7c829896"
},
"downloads": -1,
"filename": "napari_workflow_optimizer-0.1.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6427e72fe7c2428502ef97f351d707ec",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 17293,
"upload_time": "2022-04-15T16:35:08",
"upload_time_iso_8601": "2022-04-15T16:35:08.144452Z",
"url": "https://files.pythonhosted.org/packages/bf/fa/b83ed8b8a081e35dbca2c7c07fa2de3ad0a48b5b46264ee547ddfa292684/napari_workflow_optimizer-0.1.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"md5": "227ba4d209a2e7d322aa154b39b8e256",
"sha256": "e756ab013740cdc89fbb2626291386579ec8ce30ec30153a1b79517d7790f9c2"
},
"downloads": -1,
"filename": "napari-workflow-optimizer-0.1.4.tar.gz",
"has_sig": false,
"md5_digest": "227ba4d209a2e7d322aa154b39b8e256",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 18604,
"upload_time": "2022-04-15T16:35:09",
"upload_time_iso_8601": "2022-04-15T16:35:09.559205Z",
"url": "https://files.pythonhosted.org/packages/64/c0/593f5cc3ce82960e7b3ad1ef7569ec7c31a4718eac0156ab49e99acf0125/napari-workflow-optimizer-0.1.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-04-15 16:35:09",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "haesleinhuepf",
"github_project": "napari-workflow-optimizer",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "napari-workflows",
"specs": []
},
{
"name": "napari-assistant",
"specs": [
[
">=",
"0.1.9"
]
]
},
{
"name": null,
"specs": []
}
],
"tox": true,
"lcname": "napari-workflow-optimizer"
}
JSON
