# MMV Im2Im Transformation
[](https://github.com/MMV-Lab/mmv_im2im/actions)
A generic python package for deep learning based image-to-image transformation in biomedical applications
(We are actively working on the documentation and tutorials. Submit a feature request if there is anything you need.)
---
## Overview
The overall package is designed with a generic image-to-image transformation framework, which could be directly used for semantic segmentation, instance segmentation, image restoration, image generation, labelfree prediction, staining transformation, etc.. The implementation takes advantage of the state-of-the-art ML engineering techniques for users to focus on researches without worrying about the engineering details. In our pre-print [arxiv link](https://arxiv.org/abs/2209.02498), we demonstrated the effectiveness of *MMV_Im2Im* in more than ten different biomedical problems/datasets.
* For computational biomedical researchers (e.g., AI algorithm development or bioimage analysis workflow development), we hope this package could serve as the starting point for their specific problems, since the image-to-image "boilerplates" can be easily extended further development or adapted for users' specific problems.
* For experimental biomedical researchers, we hope this work provides a comprehensive view of the image-to-image transformation concept through diversified examples and use cases, so that deep learning based image-to-image transformation could be integrated into the assay development process and permit new biomedical studies that can hardly be done only with traditional experimental methods
## Installation
Before starting, we recommend to [create a new conda environment](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#creating-an-environment-with-commands) or [a virtual environment](https://docs.python.org/3/library/venv.html) with Python 3.9+.
Please note that the proper setup of hardware is beyond the scope of this pacakge. This package was tested with GPU/CPU on Linux/Windows and CPU on MacOS. [Special note for MacOS users: Directly pip install in MacOS may need [additional setup of xcode](https://developer.apple.com/forums/thread/673827).]
### Install the development head of MONAI
Due to this bug in MONAI: https://github.com/Project-MONAI/MONAI/pull/6523, which has been fixed in the dev branch but not in the stable release yet, we need to install the current dev head to avoid this bug. To do this:
```
git clone https://github.com/Project-MONAI/MONAI.git
cd ./MONAI
# note: the dev branch is the default branch after cloning MONAI
pip install .
```
### Install MMV_Im2Im for basic usage:
(For users only using this package, not planning to change any code or make any extension):
**Option 1: core functionality only** `pip install mmv_im2im`<br>
**Option 2: advanced functionality (core + logger)** `pip install mmv_im2im[advance]`<br>
**Option 3: to reproduce paper:** `pip install mmv_im2im[paper]`<br>
**Option 4: install everything:** `pip install mmv_im2im[all]`<br>
For MacOS users, additional ' ' marks are need when using installation tags in zsh. For example, `pip install mmv_im2im[paper]` should be `pip install mmv_im2im'[paper]'` in MacOS.
### Install MMV_Im2Im for customization or extension:
```
git clone https://github.com/MMV-Lab/mmv_im2im.git
cd mmv_im2im
pip install -e .[all]
```
Note: The `-e` option is the so-called "editable" mode. This will allow code changes taking effect immediately. The installation tags, `embedseg`, `advance`, `paper`, `all`, are be selected based on your needs.
### (Optional) Install using Docker
It is also possible to use our package through [docker](https://www.docker.com/). The installation tutorial is [here](docker/tutorial.md).
### (Optional) Use MMV_Im2Im with Google Colab
We provide a web-based demo, if cloud computing is preferred. you can [](https://colab.research.google.com/github/MMV-Lab/mmv_im2im/blob/main/tutorials/colab/labelfree_2d.ipynb). The same demo can de adapted for different applications.
## Quick start
You can try out on a simple example following [the quick start guide](tutorials/quick_start.md)
Basically, you can specify your training configuration in a yaml file and run training with `run_im2im --config /path/to/train_config.yaml`. Then, you can specify the inference configuration in another yaml file and run inference with `run_im2im --config /path/to/inference_config.yaml`. You can also run the inference as a function with the provided API. This will be useful if you want to run the inference within another python script or workflow. Here is an example:
```
from pathlib import Path
from aicsimageio import AICSImage
from aicsimageio.writers import OmeTiffWriter
from mmv_im2im.configs.config_base import ProgramConfig, parse_adaptor, configuration_validation
from mmv_im2im import ProjectTester
# load the inference configuration
cfg = parse_adaptor(config_class=ProgramConfig, config="./paper_configs/semantic_seg_2d_inference.yaml")
cfg = configuration_validation(cfg)
# define the executor for inference
executor = ProjectTester(cfg)
executor.setup_model()
executor.setup_data_processing()
# get the data, run inference, and save the result
fn = Path("./data/img_00_IM.tiff")
img = AICSImage(fn).get_image_data("YX", Z=0, C=0, T=0)
# or using delayed loading if the data is large
# img = AICSImage(fn).get_image_dask_data("YX", Z=0, C=0, T=0)
seg = executor.process_one_image(img)
OmeTiffWriter.save(seg, "output.tiff", dim_orders="YX")
```
## Tutorials, examples, demonstrations and documentations
The overall package aims to achieve both simplicty and flexibilty with the modularized image-to-image boilerplates. To help different users to best use this package, we provide documentations from four different aspects:
* [Examples (i.e., scripts and config files)](tutorials/example_by_use_case.md) for reproducing all the experiments in our [pre-print](https://arxiv.org/abs/2209.02498)
* A bottom-up tutorials on [how to understand the modularized image-to-image boilerplates](tutorials/how_to_understand_boilerplates.md) (for extending or adapting the package) and [how to understand the configuration system in details](tutorials/how_to_understand_config.md) (for advance usage to make specific customization).
* A top-down tutorials as [FAQ](tutorials/FAQ.md), which will continuously grow as we receive more questions.
### Contribute models to [BioImage Model Zoo](https://bioimage.io/#/)
We highly appreciate the BioImage Model Zoo's initiative to provide a comprehensive collection of pre-trained models for a wide range of applications. To make MMV_Im2Im trained models available as well, the first step involves extracting the state_dict from the PyTorch Lightning checkpoint.
This can be done via:
```python
import torch
ckpt_path = "./lightning_logs/version_0/checkpoints/last.ckpt"
checkpoint = torch.load(ckpt_path, map_location=torch.device('cpu'))
state_dict = checkpoint['state_dict']
torch.save(state_dict, "./state_dict.pt")
```
All further steps to provide models can be found in the [official documentation](https://bioimage.io/docs/#/contribute_models/README).
## Development
See [CONTRIBUTING.md](CONTRIBUTING.md) for information related to developing the code.
**MIT license**
Raw data
{
"_id": null,
"home_page": "https://github.com/MMV-Lab/mmv_im2im",
"name": "mmv-im2im",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "",
"keywords": "deep learning,microscopy image analysis,biomedical image analysis",
"author": "Jianxu Chen",
"author_email": "jianxuchen.ai@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/ae/9e/72df805358db16842293f2bce3e470ea68c7fb2f6ea1fdefa0d3207507f5/mmv_im2im-0.5.1.tar.gz",
"platform": null,
"description": "# MMV Im2Im Transformation\n\n[](https://github.com/MMV-Lab/mmv_im2im/actions)\n\nA generic python package for deep learning based image-to-image transformation in biomedical applications\n\n(We are actively working on the documentation and tutorials. Submit a feature request if there is anything you need.)\n\n---\n\n## Overview\n\nThe overall package is designed with a generic image-to-image transformation framework, which could be directly used for semantic segmentation, instance segmentation, image restoration, image generation, labelfree prediction, staining transformation, etc.. The implementation takes advantage of the state-of-the-art ML engineering techniques for users to focus on researches without worrying about the engineering details. In our pre-print [arxiv link](https://arxiv.org/abs/2209.02498), we demonstrated the effectiveness of *MMV_Im2Im* in more than ten different biomedical problems/datasets. \n\n* For computational biomedical researchers (e.g., AI algorithm development or bioimage analysis workflow development), we hope this package could serve as the starting point for their specific problems, since the image-to-image \"boilerplates\" can be easily extended further development or adapted for users' specific problems.\n* For experimental biomedical researchers, we hope this work provides a comprehensive view of the image-to-image transformation concept through diversified examples and use cases, so that deep learning based image-to-image transformation could be integrated into the assay development process and permit new biomedical studies that can hardly be done only with traditional experimental methods\n\n\n## Installation\n\nBefore starting, we recommend to [create a new conda environment](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#creating-an-environment-with-commands) or [a virtual environment](https://docs.python.org/3/library/venv.html) with Python 3.9+.\n\nPlease note that the proper setup of hardware is beyond the scope of this pacakge. This package was tested with GPU/CPU on Linux/Windows and CPU on MacOS. [Special note for MacOS users: Directly pip install in MacOS may need [additional setup of xcode](https://developer.apple.com/forums/thread/673827).]\n\n### Install the development head of MONAI\n\nDue to this bug in MONAI: https://github.com/Project-MONAI/MONAI/pull/6523, which has been fixed in the dev branch but not in the stable release yet, we need to install the current dev head to avoid this bug. To do this:\n```\ngit clone https://github.com/Project-MONAI/MONAI.git\ncd ./MONAI\n# note: the dev branch is the default branch after cloning MONAI\npip install .\n```\n\n### Install MMV_Im2Im for basic usage:\n\n(For users only using this package, not planning to change any code or make any extension):\n\n**Option 1: core functionality only** `pip install mmv_im2im`<br>\n**Option 2: advanced functionality (core + logger)** `pip install mmv_im2im[advance]`<br>\n**Option 3: to reproduce paper:** `pip install mmv_im2im[paper]`<br>\n**Option 4: install everything:** `pip install mmv_im2im[all]`<br>\n\nFor MacOS users, additional ' ' marks are need when using installation tags in zsh. For example, `pip install mmv_im2im[paper]` should be `pip install mmv_im2im'[paper]'` in MacOS.\n\n### Install MMV_Im2Im for customization or extension:\n\n\n```\ngit clone https://github.com/MMV-Lab/mmv_im2im.git\ncd mmv_im2im\npip install -e .[all]\n```\n\nNote: The `-e` option is the so-called \"editable\" mode. This will allow code changes taking effect immediately. The installation tags, `embedseg`, `advance`, `paper`, `all`, are be selected based on your needs.\n\n### (Optional) Install using Docker\n\nIt is also possible to use our package through [docker](https://www.docker.com/). The installation tutorial is [here](docker/tutorial.md).\n\n### (Optional) Use MMV_Im2Im with Google Colab\n\nWe provide a web-based demo, if cloud computing is preferred. you can [](https://colab.research.google.com/github/MMV-Lab/mmv_im2im/blob/main/tutorials/colab/labelfree_2d.ipynb). The same demo can de adapted for different applications.\n\n## Quick start\n\nYou can try out on a simple example following [the quick start guide](tutorials/quick_start.md)\n\nBasically, you can specify your training configuration in a yaml file and run training with `run_im2im --config /path/to/train_config.yaml`. Then, you can specify the inference configuration in another yaml file and run inference with `run_im2im --config /path/to/inference_config.yaml`. You can also run the inference as a function with the provided API. This will be useful if you want to run the inference within another python script or workflow. Here is an example:\n\n```\nfrom pathlib import Path\nfrom aicsimageio import AICSImage\nfrom aicsimageio.writers import OmeTiffWriter\nfrom mmv_im2im.configs.config_base import ProgramConfig, parse_adaptor, configuration_validation\nfrom mmv_im2im import ProjectTester\n\n# load the inference configuration\ncfg = parse_adaptor(config_class=ProgramConfig, config=\"./paper_configs/semantic_seg_2d_inference.yaml\")\ncfg = configuration_validation(cfg)\n\n# define the executor for inference\nexecutor = ProjectTester(cfg)\nexecutor.setup_model()\nexecutor.setup_data_processing()\n\n# get the data, run inference, and save the result\nfn = Path(\"./data/img_00_IM.tiff\")\nimg = AICSImage(fn).get_image_data(\"YX\", Z=0, C=0, T=0)\n# or using delayed loading if the data is large\n# img = AICSImage(fn).get_image_dask_data(\"YX\", Z=0, C=0, T=0)\nseg = executor.process_one_image(img)\nOmeTiffWriter.save(seg, \"output.tiff\", dim_orders=\"YX\")\n```\n\n\n## Tutorials, examples, demonstrations and documentations\n\nThe overall package aims to achieve both simplicty and flexibilty with the modularized image-to-image boilerplates. To help different users to best use this package, we provide documentations from four different aspects:\n\n* [Examples (i.e., scripts and config files)](tutorials/example_by_use_case.md) for reproducing all the experiments in our [pre-print](https://arxiv.org/abs/2209.02498)\n* A bottom-up tutorials on [how to understand the modularized image-to-image boilerplates](tutorials/how_to_understand_boilerplates.md) (for extending or adapting the package) and [how to understand the configuration system in details](tutorials/how_to_understand_config.md) (for advance usage to make specific customization).\n* A top-down tutorials as [FAQ](tutorials/FAQ.md), which will continuously grow as we receive more questions.\n\n\n### Contribute models to [BioImage Model Zoo](https://bioimage.io/#/)\n\nWe highly appreciate the BioImage Model Zoo's initiative to provide a comprehensive collection of pre-trained models for a wide range of applications. To make MMV_Im2Im trained models available as well, the first step involves extracting the state_dict from the PyTorch Lightning checkpoint.\nThis can be done via:\n\n```python\nimport torch\n\nckpt_path = \"./lightning_logs/version_0/checkpoints/last.ckpt\"\ncheckpoint = torch.load(ckpt_path, map_location=torch.device('cpu'))\nstate_dict = checkpoint['state_dict']\ntorch.save(state_dict, \"./state_dict.pt\")\n```\n\nAll further steps to provide models can be found in the [official documentation](https://bioimage.io/docs/#/contribute_models/README).\n\n## Development\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for information related to developing the code.\n\n\n**MIT license**\n\n\n",
"bugtrack_url": null,
"license": "MIT license",
"summary": "A python package for deep learing based image to image transformation",
"version": "0.5.1",
"project_urls": {
"Homepage": "https://github.com/MMV-Lab/mmv_im2im"
},
"split_keywords": [
"deep learning",
"microscopy image analysis",
"biomedical image analysis"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0b7c2f6efb8ac571eabbf949013e63552672a54ede8af9d07dd68962842af733",
"md5": "a5bebd918eba57a2e0717bdb469996a1",
"sha256": "6c3a9601f1402eecdf285cd91735d327827d1cafa53d60bb398596242e7ea982"
},
"downloads": -1,
"filename": "mmv_im2im-0.5.1-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "a5bebd918eba57a2e0717bdb469996a1",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.9",
"size": 76211,
"upload_time": "2023-10-05T11:31:04",
"upload_time_iso_8601": "2023-10-05T11:31:04.305621Z",
"url": "https://files.pythonhosted.org/packages/0b/7c/2f6efb8ac571eabbf949013e63552672a54ede8af9d07dd68962842af733/mmv_im2im-0.5.1-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ae9e72df805358db16842293f2bce3e470ea68c7fb2f6ea1fdefa0d3207507f5",
"md5": "56964e3598f91d7c1c23d3501c606aeb",
"sha256": "b96b5ab81f70768940ed276e4094a2527054fcb2785a8c9d82b58c822c70e1a1"
},
"downloads": -1,
"filename": "mmv_im2im-0.5.1.tar.gz",
"has_sig": false,
"md5_digest": "56964e3598f91d7c1c23d3501c606aeb",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 60706,
"upload_time": "2023-10-05T11:31:06",
"upload_time_iso_8601": "2023-10-05T11:31:06.407474Z",
"url": "https://files.pythonhosted.org/packages/ae/9e/72df805358db16842293f2bce3e470ea68c7fb2f6ea1fdefa0d3207507f5/mmv_im2im-0.5.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-10-05 11:31:06",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "MMV-Lab",
"github_project": "mmv_im2im",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "mmv-im2im"
}
JSON
