## Development is moving
Development of this package is moving to the Rosalind Franklin Institute. A fork is now available at https://github.com/rosalindfranklininstitute/volume-segmantics
# Volume Segmantics
A toolkit for semantic segmentation of volumetric data using PyTorch deep learning models.
[](https://doi.org/10.21105/joss.04691)  
Volume Segmantics provides a simple command-line interface and API that allows researchers to quickly train a variety of 2D PyTorch segmentation models (e.g. U-Net, U-Net++, FPN, DeepLabV3+) on their 3D datasets. These models use pre-trained encoders, enabling fast training on small datasets. Subsequently, the library enables using these trained models to segment larger 3D datasets, automatically merging predictions made in orthogonal planes and rotations to reduce artifacts that may result from predicting 3D segmentation using a 2D network.
Given a 3d image volume and corresponding dense labels (the segmentation), a 2d model is trained on image slices taken along the x, y, and z axes. The method is optimised for small training datasets, e.g a single dataset in between $128^3$ and $512^3$ pixels. To achieve this, all models use pre-trained encoders and image augmentations are used to expand the size of the training dataset.
This work utilises the abilities afforded by the excellent [segmentation-models-pytorch](https://github.com/qubvel/segmentation_models.pytorch) library in combination with augmentations made available via [Albumentations](https://albumentations.ai/). Also the metrics and loss functions used make use of the hard work done by Adrian Wolny in his [pytorch-3dunet](https://github.com/wolny/pytorch-3dunet) repository.
## Requirements
A machine capable of running CUDA enabled PyTorch version 1.7.1 or greater is required. This generally means a reasonably modern NVIDIA GPU. The exact requirements differ according to operating system. For example on Windows you will need Visual Studio Build Tools as well as CUDA Toolkit installed see [the CUDA docs](https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html) for more details.
## Installation
The easiest way to install the package is to first create a new conda environment or virtualenv with python (ideally >= version 3.8) and also pip, then activate the environment and `pip install volume-segmantics`. If a CUDA-enabled build of PyTorch is not being installed by pip, you can try `pip install volume-segmantics --extra-index-url https://download.pytorch.org/whl` this particularity seems to be an issue on Windows.
## Configuration and command line use
After installation, two new commands will be available from your terminal whilst your environment is activated, `model-train-2d` and `model-predict-2d`.
These commands require access to some settings stored in YAML files. These need to be located in a directory named `volseg-settings` within the directory where you are running the commands. The settings files can be copied from [here](https://github.com/DiamondLightSource/volume-segmantics/releases/download/v0.3.2/volseg-settings.zip).
The file `2d_model_train_settings.yaml` can be edited in order to change training parameters such as number of epochs, loss functions, evaluation metrics and also model and encoder architectures. The file `2d_model_predict_settings.yaml` can be edited to change parameters such as the prediction "quality" e.g "low" quality refers to prediction of the volume segmentation by taking images along a single axis (images in the (x,y) plane). For "medium" and "high" quality, predictions are done along 3 axes and in 12 directions (3 axes, 4 rotations) respectively, before being combined by maximum probability.
### For training a 2d model on a 3d image volume and corresponding labels
Run the following command. Input files can be in HDF5 or multi-page TIFF format.
```shell
model-train-2d --data path/to/image/data.h5 --labels path/to/corresponding/segmentation/labels.h5
```
Paths to multiple data and label volumes can be added after the `--data` and `--labels` flags respectively. A model will be trained according to the settings defined in `/volseg-settings/2d_model_train_settings.yaml` and saved to your working directory. In addition, a figure showing "ground truth" segmentation vs model segmentation for some images in the validation set will be saved.
### For 3d volume segmentation prediction using a 2d model
Run the following command. Input image files can be in HDF5 or multi-page TIFF format.
```shell
model-predict-2d path/to/model_file.pytorch path/to/data_for_prediction.h5
```
The input data will be segmented using the input model following the settings specified in `volseg-settings/2d_model_predict_settings.yaml`. An HDF5 file containing the segmented volume will be saved to your working directory.
### Tutorial using example data
A tutorial is available [here](training_data/README.md) that provides a walk-through of how to segment blood vessels from synchrotron X-ray micro-CT data collected on a sample of human placental tissue.
## Currently supported model architectures and encoders
The model architectures which are currently available and tested are:
- U-Net
- U-Net++
- FPN
- DeepLabV3
- DeepLabV3+
- MA-Net
- LinkNet
- PAN
The pre-trained encoders that can be used with these architectures are:
- ResNet-34
- ResNet50
- ResNeXt-50_32x4d
- Efficientnet-b3
- Efficientnet-b4
- Resnest50d\*
- Resnest101e\*
\* Encoders with asterisk not compatible with PAN.
## Using the API
You can use the functionality of the package in your own program via the API, this is [documented here](https://diamondlightsource.github.io/volume-segmantics/). This interface is the one used by [SuRVoS2](https://github.com/DiamondLightSource/SuRVoS2), a client/server GUI application that allows fast annotation and segmentation of volumetric data.
## Contributing
We welcome contributions from the community. Please take a look at out [contribution guidelines](https://github.com/DiamondLightSource/volume-segmantics/blob/main/CONTRIBUTING.md) for more information.
## Citation
If you use this package for you research, please cite:
[King O.N.F, Bellos, D. and Basham, M. (2022). Volume Segmantics: A Python Package for Semantic Segmentation of Volumetric Data Using Pre-trained PyTorch Deep Learning Models. Journal of Open Source Software, 7(78), 4691. doi: 10.21105/joss.04691](https://doi.org/10.21105/joss.04691)
```bibtex
@article{King2022,
doi = {10.21105/joss.04691},
url = {https://doi.org/10.21105/joss.04691},
year = {2022},
publisher = {The Open Journal},
volume = {7},
number = {78},
pages = {4691},
author = {Oliver N. F. King and Dimitrios Bellos and Mark Basham},
title = {Volume Segmantics: A Python Package for Semantic Segmentation of Volumetric Data Using Pre-trained PyTorch Deep Learning Models},
journal = {Journal of Open Source Software} }
```
## References
**Albumentations**
Buslaev, A., Iglovikov, V.I., Khvedchenya, E., Parinov, A., Druzhinin, M., and Kalinin, A.A. (2020). Albumentations: Fast and Flexible Image Augmentations. Information 11. [https://doi.org/10.3390/info11020125](https://doi.org/10.3390/info11020125)
**Segmentation Models PyTorch**
Yakubovskiy, P. (2020). Segmentation Models Pytorch. [GitHub](https://github.com/qubvel/segmentation_models.pytorch)
**PyTorch-3dUnet**
Wolny, A., Cerrone, L., Vijayan, A., Tofanelli, R., Barro, A.V., Louveaux, M., Wenzl, C., Strauss, S., Wilson-Sánchez, D., Lymbouridou, R., et al. (2020). Accurate and versatile 3D segmentation of plant tissues at cellular resolution. ELife 9, e57613. [https://doi.org/10.7554/eLife.57613](https://doi.org/10.7554/eLife.57613)
Raw data
{
"_id": null,
"home_page": "https://github.com/DiamondLightSource/volume-segmantics",
"name": "volume-segmantics",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7,<4.0",
"maintainer_email": "",
"keywords": "segmentation,deep-learning,volumetric,3d",
"author": "Oliver King",
"author_email": "olly.king@diamond.ac.uk",
"download_url": "https://files.pythonhosted.org/packages/e4/22/801d5715c97320a2b0e4d0397b6a12e0eae547655211606130dac90005db/volume_segmantics-0.3.2.tar.gz",
"platform": null,
"description": "## Development is moving\n\nDevelopment of this package is moving to the Rosalind Franklin Institute. A fork is now available at https://github.com/rosalindfranklininstitute/volume-segmantics\n\n# Volume Segmantics\n\nA toolkit for semantic segmentation of volumetric data using PyTorch deep learning models.\n\n[](https://doi.org/10.21105/joss.04691)  \n\nVolume Segmantics provides a simple command-line interface and API that allows researchers to quickly train a variety of 2D PyTorch segmentation models (e.g. U-Net, U-Net++, FPN, DeepLabV3+) on their 3D datasets. These models use pre-trained encoders, enabling fast training on small datasets. Subsequently, the library enables using these trained models to segment larger 3D datasets, automatically merging predictions made in orthogonal planes and rotations to reduce artifacts that may result from predicting 3D segmentation using a 2D network. \n\nGiven a 3d image volume and corresponding dense labels (the segmentation), a 2d model is trained on image slices taken along the x, y, and z axes. The method is optimised for small training datasets, e.g a single dataset in between $128^3$ and $512^3$ pixels. To achieve this, all models use pre-trained encoders and image augmentations are used to expand the size of the training dataset.\n\nThis work utilises the abilities afforded by the excellent [segmentation-models-pytorch](https://github.com/qubvel/segmentation_models.pytorch) library in combination with augmentations made available via [Albumentations](https://albumentations.ai/). Also the metrics and loss functions used make use of the hard work done by Adrian Wolny in his [pytorch-3dunet](https://github.com/wolny/pytorch-3dunet) repository. \n\n## Requirements\n\nA machine capable of running CUDA enabled PyTorch version 1.7.1 or greater is required. This generally means a reasonably modern NVIDIA GPU. The exact requirements differ according to operating system. For example on Windows you will need Visual Studio Build Tools as well as CUDA Toolkit installed see [the CUDA docs](https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html) for more details. \n\n## Installation\n\nThe easiest way to install the package is to first create a new conda environment or virtualenv with python (ideally >= version 3.8) and also pip, then activate the environment and `pip install volume-segmantics`. If a CUDA-enabled build of PyTorch is not being installed by pip, you can try `pip install volume-segmantics --extra-index-url https://download.pytorch.org/whl` this particularity seems to be an issue on Windows. \n\n## Configuration and command line use\n\nAfter installation, two new commands will be available from your terminal whilst your environment is activated, `model-train-2d` and `model-predict-2d`.\n\nThese commands require access to some settings stored in YAML files. These need to be located in a directory named `volseg-settings` within the directory where you are running the commands. The settings files can be copied from [here](https://github.com/DiamondLightSource/volume-segmantics/releases/download/v0.3.2/volseg-settings.zip). \n\nThe file `2d_model_train_settings.yaml` can be edited in order to change training parameters such as number of epochs, loss functions, evaluation metrics and also model and encoder architectures. The file `2d_model_predict_settings.yaml` can be edited to change parameters such as the prediction \"quality\" e.g \"low\" quality refers to prediction of the volume segmentation by taking images along a single axis (images in the (x,y) plane). For \"medium\" and \"high\" quality, predictions are done along 3 axes and in 12 directions (3 axes, 4 rotations) respectively, before being combined by maximum probability. \n\n### For training a 2d model on a 3d image volume and corresponding labels\nRun the following command. Input files can be in HDF5 or multi-page TIFF format.\n\n```shell\nmodel-train-2d --data path/to/image/data.h5 --labels path/to/corresponding/segmentation/labels.h5\n```\n\nPaths to multiple data and label volumes can be added after the `--data` and `--labels` flags respectively. A model will be trained according to the settings defined in `/volseg-settings/2d_model_train_settings.yaml` and saved to your working directory. In addition, a figure showing \"ground truth\" segmentation vs model segmentation for some images in the validation set will be saved. \n\n### For 3d volume segmentation prediction using a 2d model\nRun the following command. Input image files can be in HDF5 or multi-page TIFF format.\n\n```shell\nmodel-predict-2d path/to/model_file.pytorch path/to/data_for_prediction.h5\n```\n\nThe input data will be segmented using the input model following the settings specified in `volseg-settings/2d_model_predict_settings.yaml`. An HDF5 file containing the segmented volume will be saved to your working directory.\n\n### Tutorial using example data\n\nA tutorial is available [here](training_data/README.md) that provides a walk-through of how to segment blood vessels from synchrotron X-ray micro-CT data collected on a sample of human placental tissue.\n\n## Currently supported model architectures and encoders\n\nThe model architectures which are currently available and tested are: \n- U-Net\n- U-Net++\n- FPN\n- DeepLabV3\n- DeepLabV3+\n- MA-Net\n- LinkNet\n- PAN\n\nThe pre-trained encoders that can be used with these architectures are: \n- ResNet-34\n- ResNet50\n- ResNeXt-50_32x4d\n- Efficientnet-b3\n- Efficientnet-b4\n- Resnest50d\\*\n- Resnest101e\\*\n\n\\* Encoders with asterisk not compatible with PAN.\n\n## Using the API\n\nYou can use the functionality of the package in your own program via the API, this is [documented here](https://diamondlightsource.github.io/volume-segmantics/). This interface is the one used by [SuRVoS2](https://github.com/DiamondLightSource/SuRVoS2), a client/server GUI application that allows fast annotation and segmentation of volumetric data. \n\n## Contributing\n\nWe welcome contributions from the community. Please take a look at out [contribution guidelines](https://github.com/DiamondLightSource/volume-segmantics/blob/main/CONTRIBUTING.md) for more information.\n\n## Citation\n\nIf you use this package for you research, please cite:\n\n[King O.N.F, Bellos, D. and Basham, M. (2022). Volume Segmantics: A Python Package for Semantic Segmentation of Volumetric Data Using Pre-trained PyTorch Deep Learning Models. Journal of Open Source Software, 7(78), 4691. doi: 10.21105/joss.04691](https://doi.org/10.21105/joss.04691)\n\n```bibtex\n@article{King2022,\n doi = {10.21105/joss.04691},\n url = {https://doi.org/10.21105/joss.04691},\n year = {2022},\n publisher = {The Open Journal},\n volume = {7},\n number = {78},\n pages = {4691},\n author = {Oliver N. F. King and Dimitrios Bellos and Mark Basham},\n title = {Volume Segmantics: A Python Package for Semantic Segmentation of Volumetric Data Using Pre-trained PyTorch Deep Learning Models},\n journal = {Journal of Open Source Software} }\n```\n\n## References\n\n**Albumentations**\n\nBuslaev, A., Iglovikov, V.I., Khvedchenya, E., Parinov, A., Druzhinin, M., and Kalinin, A.A. (2020). Albumentations: Fast and Flexible Image Augmentations. Information 11. [https://doi.org/10.3390/info11020125](https://doi.org/10.3390/info11020125)\n\n**Segmentation Models PyTorch**\n\nYakubovskiy, P. (2020). Segmentation Models Pytorch. [GitHub](https://github.com/qubvel/segmentation_models.pytorch)\n\n**PyTorch-3dUnet**\n\nWolny, A., Cerrone, L., Vijayan, A., Tofanelli, R., Barro, A.V., Louveaux, M., Wenzl, C., Strauss, S., Wilson-S\u00e1nchez, D., Lymbouridou, R., et al. (2020). Accurate and versatile 3D segmentation of plant tissues at cellular resolution. ELife 9, e57613. [https://doi.org/10.7554/eLife.57613](https://doi.org/10.7554/eLife.57613)\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "A toolkit for semantic segmentation of volumetric data using pyTorch deep learning models",
"version": "0.3.2",
"split_keywords": [
"segmentation",
"deep-learning",
"volumetric",
"3d"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "48a2c1bca219cb96bc991823a79a2b52aae8a5c71a4f7a57af2c0a32a02862c5",
"md5": "132e2bca27431b3802fb04df3ea3e916",
"sha256": "aeb2c3d0db0b55cc22c650588349a4f369dfaad4d99e89e5a4fcd5682c3ba558"
},
"downloads": -1,
"filename": "volume_segmantics-0.3.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "132e2bca27431b3802fb04df3ea3e916",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7,<4.0",
"size": 44090,
"upload_time": "2023-04-19T17:41:02",
"upload_time_iso_8601": "2023-04-19T17:41:02.352858Z",
"url": "https://files.pythonhosted.org/packages/48/a2/c1bca219cb96bc991823a79a2b52aae8a5c71a4f7a57af2c0a32a02862c5/volume_segmantics-0.3.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e422801d5715c97320a2b0e4d0397b6a12e0eae547655211606130dac90005db",
"md5": "a118f1f4e3d2a67701eb5092858ce708",
"sha256": "9a9e4eb43924dd3f5d5f3f3c51dd303f231c093d062ee9b04aca5e50e1705b5a"
},
"downloads": -1,
"filename": "volume_segmantics-0.3.2.tar.gz",
"has_sig": false,
"md5_digest": "a118f1f4e3d2a67701eb5092858ce708",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7,<4.0",
"size": 37749,
"upload_time": "2023-04-19T17:41:04",
"upload_time_iso_8601": "2023-04-19T17:41:04.254804Z",
"url": "https://files.pythonhosted.org/packages/e4/22/801d5715c97320a2b0e4d0397b6a12e0eae547655211606130dac90005db/volume_segmantics-0.3.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-04-19 17:41:04",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "DiamondLightSource",
"github_project": "volume-segmantics",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "volume-segmantics"
}
JSON
