| Name | pathopatch JSON |
| Version |
1.0.3
JSON |
| download |
| home_page | None |
| Summary | PathoPatch - Accelerating Artificial Intelligence Based Whole Slide Image Analysis with an Optimized Preprocessing Pipeline |
| upload_time | 2024-10-23 14:24:56 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.9 |
| license | CC BY-NC-SA 4.0 |
| keywords |
python
pathopatch
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
[](https://www.python.org/downloads/release/python-360/)
[](https://github.com/psf/black)
<img src="https://img.shields.io/badge/PyTorch-EE4C2C?style=flat-square&logo=Pytorch&logoColor=white"/></a>
___
# PathoPatch:
## Accelerating Artificial Intelligence Based Whole Slide Image Analysis with an Optimized Preprocessing Pipeline
---
## Installation
### Prerequisite
1. Openslide (>= 3.4.1) needs to be installed (either directly https://openslide.org/download/ or via conda)<details>
<summary>OpenSlide conda</summary>
- Recommended: `conda install conda-forge::openslide=4.0.0` for DICOM support
- Generic/minimum version: `conda-forge::openslide>=3.4.1`
</details>
2. Openslide python: `pip install openslide-python`
3. Optional for speedup: [cuCIM installation instructions](https://github.com/rapidsai/cucim?tab=readme-ov-file#install-cucim)
### PIP-Package
The package can be found here: https://pypi.org/project/pathopatch/
Installation: `pip install pathopatch`
### Development
1. Install pre-commit with `pre-commit install`
## Usage
We provide different use cases - Offline-Dataset (Store on Disk :floppy_disk:) and Inference-Dataset for :zap: PyTorch :zap:
In our Pre-Processing pipeline, we are able to extract quadratic patches from detected tissue areas, load annotation files (`.json`) and apply color normlizations. We make use of the popular [OpenSlide](https://openslide.org/) library, but extended it with the [RAPIDS cuCIM](https://github.com/rapidsai/cucim) framework for a speedup in patch-extraction.
> We support all OpenSlide file formats + .dcm-File format (DICOM), by utilizing [`wsidicom`](https://github.com/imi-bigpicture/wsidicom) and [`wsidicomizer`](https://github.com/imi-bigpicture/wsidicomizer).
### Offline-Dataset
In general, our framework has the following commands registered in your shell:
> **wsi_extraction**: Extract patches with specific configuration and store them on the disk
> **annotation_conversion**: Can be used to convert annotations
> **macenko_vector_generation**: To generate new macenko vectors for a new dataset, if custom vectors are tend to be used
### Parameter handover and CLI
#### Option 1: Config.yaml
Arguments are passed via CLIs. In addition to the CLI, also a configuration file can be passed via
```bash
wsi_extraction --config path/to/config.yaml
```
Exemplary configuration file: [patch_extraction.yaml](examples/patch_extraction.yaml).
#### Option 2: CLI
The CLI of the main script for patch extraction ([wsi_extraction](pathopatch/wsi_extraction.py)) is as follows:
```bash
wsi_extraction [-h]
[--wsi_paths WSI_PATHS]
[--wsi_filelist WSI_FILELIST]
[--output_path OUTPUT_PATH]
[--wsi_extension {svs}]
[--config CONFIG]
[--patch_size PATCH_SIZE]
[--patch_overlap PATCH_OVERLAP]
[--target_mpp TARGET_MPP]
[--target_mag TARGET_MAG]
[--downsample DOWNSAMPLE]
[--level LEVEL]
[--context_scales [CONTEXT_SCALES ...]]
[--check_resolution CHECK_RESOLUTION]
[--processes PROCESSES]
[--overwrite]
[--annotation_paths ANNOTATION_PATHS]
[--annotation_extension {json,xml}]
[--incomplete_annotations]
[--label_map_file LABEL_MAP_FILE]
[--save_only_annotated_patches]
[--save_context_without_mask]
[--exclude_classes EXCLUDE_CLASSES]
[--store_masks]
[--overlapping_labels]
[--normalize_stains]
[--normalization_vector_json NORMALIZATION_VECTOR_JSON]
[--min_intersection_ratio MIN_INTERSECTION_RATIO]
[--tissue_annotation TISSUE_ANNOTATION]
[--tissue_annotation_intersection_ratio TISSUE_ANNOTATION_INTERSECTION_RATIO]
[--masked_otsu]
[--otsu_annotation OTSU_ANNOTATION]
[--filter_patches FILTER_PATCHES]
[--apply_prefilter APPLY_PREFILTER]
[--log_path LOG_PATH]
[--log_level {critical,error,warning,info,debug}]
[--hardware_selection {cucim,openslide,wsidicom}]
[--wsi_magnification WSI_MAGNIFICATION]
[--wsi_mpp WSI_MPP]
options:
-h, --help show this help message and exit
--wsi_paths WSI_PATHS
Path to the folder where all WSI are stored or path to a
single WSI-file. (default: None)
--wsi_filelist WSI_FILELIST
Path to a csv-filelist with WSI files (separator: `,`), if
provided just these files are used.Must include full paths
to WSIs, including suffixes.Can be used as an replacement
for the wsi_paths option.If both are provided, yields an
error. (default: None)
--output_path OUTPUT_PATH
Path to the folder where the resulting dataset should be
stored. (default: None)
--wsi_extension {svs,tiff,tif,bif,scn,ndpi,vms,vmu}
The extension types used for the WSI files, the options
are: ['svs', 'tiff', 'tif', 'bif', 'scn', 'ndpi', 'vms',
'vmu'] (default: None)
--config CONFIG Path to a config file. The config file can hold the same
parameters as the CLI. Parameters provided with the CLI are
always having precedence over the parameters in the config
file. (default: None)
--patch_size PATCH_SIZE
The size of the patches in pixel that will be retrieved
from the WSI, e.g. 256 for 256px (default: None)
--patch_overlap PATCH_OVERLAP
The percentage amount pixels that should overlap between
two different patches. Please Provide as integer between 0
and 100, indicating overlap in percentage. (default: None)
--target_mpp TARGET_MPP
If this parameter is provided, the output level of the WSI
corresponds to the level that is at the target microns per
pixel of the WSI. Alternative to target_mag, downsaple and
level. Highest priority, overwrites all other setups for
magnifcation, downsample, or level. (default: None)
--target_mag TARGET_MAG
If this parameter is provided, the output level of the WSI
corresponds to the level that is at the target
magnification of the WSI. Alternative to target_mpp,
downsaple and level. High priority, just target_mpp has a
higher priority, overwrites downsample and level if
provided. (default: None)
--downsample DOWNSAMPLE
Each WSI level is downsampled by a factor of 2, downsample
expresses which kind of downsampling should be used with
respect to the highest possible resolution. Medium
priority, gets overwritten by target_mag and target_mpp if
provided, but overwrites level. (default: None)
--level LEVEL The tile level for sampling, alternative to downsample.
Lowest priority, gets overwritten by target_mag and
downsample if they are provided. (default: None)
--context_scales [CONTEXT_SCALES ...]
Define context scales for context patches. Context patches
are centered around a central patch. The context-patch size
is equal to the patch-size, but downsampling is different
(default: None)
--check_resolution CHECK_RESOLUTION
If a float value is supplies, the program checks whether
the resolution of all images corresponds to the given value
(default: None)
--processes PROCESSES
The number of processes to use. (default: None)
--overwrite Overwrite the patches that have already been created in
case they already exist. Removes dataset. Handle with care!
(default: None)
--annotation_paths ANNOTATION_PATHS
Path to the subfolder where the XML/JSON annotations are
stored or path to a file (default: None)
--annotation_extension {json}
The extension types used for the annotation files, the
options are: ['json'] (default: None)
--incomplete_annotations
Set to allow WSI without annotation file (default: None)
--label_map_file LABEL_MAP_FILE
The path to a json file that contains the mapping between
the annotation labels and some integers; an example can be
found in examples (default: None)
--save_only_annotated_patches
If true only patches containing annotations will be stored
(default: None)
--save_context_without_mask
This is helpful for extracting patches, that are not within
a mask, but needed for the Valuing Vicinity Segmentation
Algorithms. This flag is specifically helpful if only fully
annotated patches should be extracted from a region of
interest (ROI) and their masks are stored, but also
sourrounding neighbourhood patches (without mask) are
needed. (default: None)
--exclude_classes EXCLUDE_CLASSES
Can be used to exclude annotation classes (default: None)
--store_masks Set to store masks per patch. Defaults to false (default:
None)
--overlapping_labels Per default, labels (annotations) are mutually exclusive.
If labels overlap, they are overwritten according to the
label_map.json ordering (highest number = highest priority)
(default: None)
--normalize_stains Uses Macenko normalization on a portion of the whole slide
image (default: None)
--normalization_vector_json NORMALIZATION_VECTOR_JSON
The path to a JSON file where the normalization vectors are
stored (default: None)
--adjust_brightness Normalize brightness in a batch by clipping to 90 percent.
Not recommended, but kept for legacy reasons (default:
None)
--min_intersection_ratio MIN_INTERSECTION_RATIO
The minimum intersection between the tissue mask and the
patch. Must be between 0 and 1. 0 means that all patches
are extracted. (default: None)
--tissue_annotation TISSUE_ANNOTATION
Can be used to name a polygon annotation to determine the
tissue area. If a tissue annotation is provided, no Otsu-
thresholding is performed (default: None)
--tissue_annotation_intersection_ratio TISSUE_ANNOTATION_INTERSECTION_RATIO
Intersection ratio with tissue annotation. Helpful, if ROI
annotation is passed, which should not interfere with
background ratio. If not provided, the default
min_intersection_ratio with the background is used.
(default: None)
--masked_otsu Use annotation to mask the thumbnail before otsu-
thresholding is used (default: None)
--otsu_annotation OTSU_ANNOTATION
Can be used to name a polygon annotation to determine the
area for masked otsu thresholding. Seperate multiple labels
with ' ' (whitespace) (default: None)
--filter_patches Post-extraction patch filtering to sort out artefacts,
marker and other non-tissue patches with a DL model. Time
consuming. Defaults to False. (default: None)
--apply_prefilter Pre-extraction mask filtering to remove marker from mask
before applying otsu. Defaults to False. (default: None)
--log_path LOG_PATH Path where log files should be stored. Otherwise, log files
are stored in the output folder (default: None)
--log_level {critical,error,warning,info,debug}
Set the logging level. Options are ['critical', 'error',
'warning', 'info', 'debug'] (default: None)
--hardware_selection {cucim,openslide,wsidicom}
Select hardware device (just if available, otherwise always
cucim). Defaults to None. (default: None)
--wsi_magnification WSI_MAGNIFICATION
Manual WSI magnification, but just applies if metadata
cannot be derived from OpenSlide (e.g., for .tiff files).
(default: None)
--wsi_mpp WSI_MPP Manual WSI MPP, but just applies if metadata cannot be
derived from OpenSlide (e.g., for .tiff files). (default:
None)
```
#### Option 3: CLI + Config
Both can be combined, but arguments in the CLI have precedence!
### Inference-Dataset (PyTorch)
TBD, Elements: LivePatchWSIConfig, LivePatchWSIDataset, LivePatchWSIDataloader [Link](pathopatch/patch_extracton/dataset.py)
Usage:
```python
patch_config = LivePatchWSIConfig(
wsi_path="/Users/fhoerst/Fabian-Projekte/Selocan/RicardoScans/266819.svs",
patch_size=256,
patch_overlap=0,
target_mpp=0.3,
target_mpp_tolerance=0.1,
)
patch_dataset = LivePatchWSIDataset(patch_config, logger)
patch_dataloader = LivePatchWSIDataloader(patch_dataset, batch_size=8)
for batch in patch_dataloader:
...
```
### Resulting Dataset Structure
In general, the folder structure for a preprocessed dataset looks like this:
The aim of pre-processing is to create one dataset per WSI in the following structure:
```bash
WSI_Name
├── annotation_masks # thumbnails of extracted annotation masks
│ ├── all_overlaid.png # all with same dimension as the thumbnail
│ ├── tumor.png
│ └── ...
├── context # context patches, if extracted
│ ├── 2 # subfolder for each scale
│ │ ├── WSI_Name_row1_col1_context_2.png
│ │ ├── WSI_Name_row2_col1_context_2.png
│ │ └── ...
│ └── 4
│ │ ├── WSI_Name_row1_col1_context_2.png
│ │ ├── WSI_Name_row2_col1_context_2.png
│ │ └── ...
├── masks # Mask (numpy) files for each patch -> optional folder for segmentation
│ ├── WSI_Name_row1_col1.npy
│ ├── WSI_Name_row2_col1.npy
│ └── ...
├── metadata # Metadata files for each patch
│ ├── WSI_Name_row1_col1.yaml
│ ├── WSI_Name_row2_col1.yaml
│ └── ...
├── patches # Patches as .png files
│ ├── WSI_Name_row1_col1.png
│ ├── WSI_Name_row2_col1.png
│ └── ...
├── thumbnails # Different kind of thumbnails
│ ├── thumbnail_mpp_5.png
│ ├── thumbnail_downsample_32.png
│ └── ...
├── tissue_masks # Tissue mask images for checking
│ ├── mask.png # all with same dimension as the thumbnail
│ ├── mask_nogrid.png
│ └── tissue_grid.png
├── mask.png # tissue mask with green grid
├── metadata.yaml # WSI metdata for patch extraction
├── patch_metadata.json # Patch metadata of WSI merged in one file
└── thumbnail.png # WSI thumbnail
```
## Further information
For more information, check out the git.
## License
<p xmlns:cc="http://creativecommons.org/ns#" xmlns:dct="http://purl.org/dc/terms/"><a property="dct:title" rel="cc:attributionURL" href="https://github.com/TIO-IKIM/PathoPatcher">PathoPatcher</a> by <a rel="cc:attributionURL dct:creator" property="cc:attributionName" href="https://github.com/FabianHoerst">Fabian Hörst, University Hospital Essen,</a> is licensed under <a href="http://creativecommons.org/licenses/by-nc-sa/4.0/?ref=chooser-v1" target="_blank" rel="license noopener noreferrer" style="display:inline-block;">CC BY-NC-SA 4.0<img style="height:22px!important;margin-left:3px;vertical-align:text-bottom;" src="https://mirrors.creativecommons.org/presskit/icons/cc.svg?ref=chooser-v1"><img style="height:22px!important;margin-left:3px;vertical-align:text-bottom;" src="https://mirrors.creativecommons.org/presskit/icons/by.svg?ref=chooser-v1"><img style="height:22px!important;margin-left:3px;vertical-align:text-bottom;" src="https://mirrors.creativecommons.org/presskit/icons/nc.svg?ref=chooser-v1"><img style="height:22px!important;margin-left:3px;vertical-align:text-bottom;" src="https://mirrors.creativecommons.org/presskit/icons/sa.svg?ref=chooser-v1"></a></p>
Raw data
{
"_id": null,
"home_page": null,
"name": "pathopatch",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "python, pathopatch",
"author": null,
"author_email": "Fabian H\u00f6rst <fabian.hoerst@uk-essen.de>",
"download_url": "https://files.pythonhosted.org/packages/47/37/30e76eb7be3ff15664bce2549aca82b8085304125b00735c7163daadac81/pathopatch-1.0.3.tar.gz",
"platform": null,
"description": "[](https://www.python.org/downloads/release/python-360/)\n[](https://github.com/psf/black)\n\n<img src=\"https://img.shields.io/badge/PyTorch-EE4C2C?style=flat-square&logo=Pytorch&logoColor=white\"/></a>\n\n___\n\n# PathoPatch:\n## Accelerating Artificial Intelligence Based Whole Slide Image Analysis with an Optimized Preprocessing Pipeline\n---\n\n## Installation\n\n### Prerequisite\n1. Openslide (>= 3.4.1) needs to be installed (either directly https://openslide.org/download/ or via conda)<details>\n <summary>OpenSlide conda</summary>\n - Recommended: `conda install conda-forge::openslide=4.0.0` for DICOM support\n - Generic/minimum version: `conda-forge::openslide>=3.4.1`\n </details>\n2. Openslide python: `pip install openslide-python`\n3. Optional for speedup: [cuCIM installation instructions](https://github.com/rapidsai/cucim?tab=readme-ov-file#install-cucim)\n\n### PIP-Package\nThe package can be found here: https://pypi.org/project/pathopatch/\nInstallation: `pip install pathopatch`\n\n### Development\n1. Install pre-commit with `pre-commit install`\n\n\n## Usage\nWe provide different use cases - Offline-Dataset (Store on Disk :floppy_disk:) and Inference-Dataset for :zap: PyTorch :zap:\n\nIn our Pre-Processing pipeline, we are able to extract quadratic patches from detected tissue areas, load annotation files (`.json`) and apply color normlizations. We make use of the popular [OpenSlide](https://openslide.org/) library, but extended it with the [RAPIDS cuCIM](https://github.com/rapidsai/cucim) framework for a speedup in patch-extraction.\n\n> We support all OpenSlide file formats + .dcm-File format (DICOM), by utilizing [`wsidicom`](https://github.com/imi-bigpicture/wsidicom) and [`wsidicomizer`](https://github.com/imi-bigpicture/wsidicomizer).\n\n### Offline-Dataset\n\n\nIn general, our framework has the following commands registered in your shell:\n> **wsi_extraction**: Extract patches with specific configuration and store them on the disk\n> **annotation_conversion**: Can be used to convert annotations\n> **macenko_vector_generation**: To generate new macenko vectors for a new dataset, if custom vectors are tend to be used\n\n### Parameter handover and CLI\n#### Option 1: Config.yaml\nArguments are passed via CLIs. In addition to the CLI, also a configuration file can be passed via\n```bash\nwsi_extraction --config path/to/config.yaml\n```\nExemplary configuration file: [patch_extraction.yaml](examples/patch_extraction.yaml).\n\n#### Option 2: CLI\n\nThe CLI of the main script for patch extraction ([wsi_extraction](pathopatch/wsi_extraction.py)) is as follows:\n\n```bash\nwsi_extraction [-h]\n [--wsi_paths WSI_PATHS]\n [--wsi_filelist WSI_FILELIST]\n [--output_path OUTPUT_PATH]\n [--wsi_extension {svs}]\n [--config CONFIG]\n [--patch_size PATCH_SIZE]\n [--patch_overlap PATCH_OVERLAP]\n [--target_mpp TARGET_MPP]\n [--target_mag TARGET_MAG]\n [--downsample DOWNSAMPLE]\n [--level LEVEL]\n [--context_scales [CONTEXT_SCALES ...]]\n [--check_resolution CHECK_RESOLUTION]\n [--processes PROCESSES]\n [--overwrite]\n [--annotation_paths ANNOTATION_PATHS]\n [--annotation_extension {json,xml}]\n [--incomplete_annotations]\n [--label_map_file LABEL_MAP_FILE]\n [--save_only_annotated_patches]\n [--save_context_without_mask]\n [--exclude_classes EXCLUDE_CLASSES]\n [--store_masks]\n [--overlapping_labels]\n [--normalize_stains]\n [--normalization_vector_json NORMALIZATION_VECTOR_JSON]\n [--min_intersection_ratio MIN_INTERSECTION_RATIO]\n [--tissue_annotation TISSUE_ANNOTATION]\n [--tissue_annotation_intersection_ratio TISSUE_ANNOTATION_INTERSECTION_RATIO]\n [--masked_otsu]\n [--otsu_annotation OTSU_ANNOTATION]\n [--filter_patches FILTER_PATCHES]\n [--apply_prefilter APPLY_PREFILTER]\n [--log_path LOG_PATH]\n [--log_level {critical,error,warning,info,debug}]\n [--hardware_selection {cucim,openslide,wsidicom}]\n [--wsi_magnification WSI_MAGNIFICATION]\n [--wsi_mpp WSI_MPP]\n\noptions:\n -h, --help show this help message and exit\n --wsi_paths WSI_PATHS\n Path to the folder where all WSI are stored or path to a\n single WSI-file. (default: None)\n --wsi_filelist WSI_FILELIST\n Path to a csv-filelist with WSI files (separator: `,`), if\n provided just these files are used.Must include full paths\n to WSIs, including suffixes.Can be used as an replacement\n for the wsi_paths option.If both are provided, yields an\n error. (default: None)\n --output_path OUTPUT_PATH\n Path to the folder where the resulting dataset should be\n stored. (default: None)\n --wsi_extension {svs,tiff,tif,bif,scn,ndpi,vms,vmu}\n The extension types used for the WSI files, the options\n are: ['svs', 'tiff', 'tif', 'bif', 'scn', 'ndpi', 'vms',\n 'vmu'] (default: None)\n --config CONFIG Path to a config file. The config file can hold the same\n parameters as the CLI. Parameters provided with the CLI are\n always having precedence over the parameters in the config\n file. (default: None)\n --patch_size PATCH_SIZE\n The size of the patches in pixel that will be retrieved\n from the WSI, e.g. 256 for 256px (default: None)\n --patch_overlap PATCH_OVERLAP\n The percentage amount pixels that should overlap between\n two different patches. Please Provide as integer between 0\n and 100, indicating overlap in percentage. (default: None)\n --target_mpp TARGET_MPP\n If this parameter is provided, the output level of the WSI\n corresponds to the level that is at the target microns per\n pixel of the WSI. Alternative to target_mag, downsaple and\n level. Highest priority, overwrites all other setups for\n magnifcation, downsample, or level. (default: None)\n --target_mag TARGET_MAG\n If this parameter is provided, the output level of the WSI\n corresponds to the level that is at the target\n magnification of the WSI. Alternative to target_mpp,\n downsaple and level. High priority, just target_mpp has a\n higher priority, overwrites downsample and level if\n provided. (default: None)\n --downsample DOWNSAMPLE\n Each WSI level is downsampled by a factor of 2, downsample\n expresses which kind of downsampling should be used with\n respect to the highest possible resolution. Medium\n priority, gets overwritten by target_mag and target_mpp if\n provided, but overwrites level. (default: None)\n --level LEVEL The tile level for sampling, alternative to downsample.\n Lowest priority, gets overwritten by target_mag and\n downsample if they are provided. (default: None)\n --context_scales [CONTEXT_SCALES ...]\n Define context scales for context patches. Context patches\n are centered around a central patch. The context-patch size\n is equal to the patch-size, but downsampling is different\n (default: None)\n --check_resolution CHECK_RESOLUTION\n If a float value is supplies, the program checks whether\n the resolution of all images corresponds to the given value\n (default: None)\n --processes PROCESSES\n The number of processes to use. (default: None)\n --overwrite Overwrite the patches that have already been created in\n case they already exist. Removes dataset. Handle with care!\n (default: None)\n --annotation_paths ANNOTATION_PATHS\n Path to the subfolder where the XML/JSON annotations are\n stored or path to a file (default: None)\n --annotation_extension {json}\n The extension types used for the annotation files, the\n options are: ['json'] (default: None)\n --incomplete_annotations\n Set to allow WSI without annotation file (default: None)\n --label_map_file LABEL_MAP_FILE\n The path to a json file that contains the mapping between\n the annotation labels and some integers; an example can be\n found in examples (default: None)\n --save_only_annotated_patches\n If true only patches containing annotations will be stored\n (default: None)\n --save_context_without_mask\n This is helpful for extracting patches, that are not within\n a mask, but needed for the Valuing Vicinity Segmentation\n Algorithms. This flag is specifically helpful if only fully\n annotated patches should be extracted from a region of\n interest (ROI) and their masks are stored, but also\n sourrounding neighbourhood patches (without mask) are\n needed. (default: None)\n --exclude_classes EXCLUDE_CLASSES\n Can be used to exclude annotation classes (default: None)\n --store_masks Set to store masks per patch. Defaults to false (default:\n None)\n --overlapping_labels Per default, labels (annotations) are mutually exclusive.\n If labels overlap, they are overwritten according to the\n label_map.json ordering (highest number = highest priority)\n (default: None)\n --normalize_stains Uses Macenko normalization on a portion of the whole slide\n image (default: None)\n --normalization_vector_json NORMALIZATION_VECTOR_JSON\n The path to a JSON file where the normalization vectors are\n stored (default: None)\n --adjust_brightness Normalize brightness in a batch by clipping to 90 percent.\n Not recommended, but kept for legacy reasons (default:\n None)\n --min_intersection_ratio MIN_INTERSECTION_RATIO\n The minimum intersection between the tissue mask and the\n patch. Must be between 0 and 1. 0 means that all patches\n are extracted. (default: None)\n --tissue_annotation TISSUE_ANNOTATION\n Can be used to name a polygon annotation to determine the\n tissue area. If a tissue annotation is provided, no Otsu-\n thresholding is performed (default: None)\n --tissue_annotation_intersection_ratio TISSUE_ANNOTATION_INTERSECTION_RATIO\n Intersection ratio with tissue annotation. Helpful, if ROI\n annotation is passed, which should not interfere with\n background ratio. If not provided, the default\n min_intersection_ratio with the background is used.\n (default: None)\n --masked_otsu Use annotation to mask the thumbnail before otsu-\n thresholding is used (default: None)\n --otsu_annotation OTSU_ANNOTATION\n Can be used to name a polygon annotation to determine the\n area for masked otsu thresholding. Seperate multiple labels\n with ' ' (whitespace) (default: None)\n --filter_patches Post-extraction patch filtering to sort out artefacts,\n marker and other non-tissue patches with a DL model. Time\n consuming. Defaults to False. (default: None)\n --apply_prefilter Pre-extraction mask filtering to remove marker from mask\n before applying otsu. Defaults to False. (default: None)\n --log_path LOG_PATH Path where log files should be stored. Otherwise, log files\n are stored in the output folder (default: None)\n --log_level {critical,error,warning,info,debug}\n Set the logging level. Options are ['critical', 'error',\n 'warning', 'info', 'debug'] (default: None)\n --hardware_selection {cucim,openslide,wsidicom}\n Select hardware device (just if available, otherwise always\n cucim). Defaults to None. (default: None)\n --wsi_magnification WSI_MAGNIFICATION\n Manual WSI magnification, but just applies if metadata\n cannot be derived from OpenSlide (e.g., for .tiff files).\n (default: None)\n --wsi_mpp WSI_MPP Manual WSI MPP, but just applies if metadata cannot be\n derived from OpenSlide (e.g., for .tiff files). (default:\n None)\n```\n#### Option 3: CLI + Config\nBoth can be combined, but arguments in the CLI have precedence!\n\n### Inference-Dataset (PyTorch)\nTBD, Elements: LivePatchWSIConfig, LivePatchWSIDataset, LivePatchWSIDataloader [Link](pathopatch/patch_extracton/dataset.py)\n\nUsage:\n```python\npatch_config = LivePatchWSIConfig(\n wsi_path=\"/Users/fhoerst/Fabian-Projekte/Selocan/RicardoScans/266819.svs\",\n patch_size=256,\n patch_overlap=0,\n target_mpp=0.3,\n target_mpp_tolerance=0.1,\n)\npatch_dataset = LivePatchWSIDataset(patch_config, logger)\npatch_dataloader = LivePatchWSIDataloader(patch_dataset, batch_size=8)\nfor batch in patch_dataloader:\n ...\n```\n\n\n### Resulting Dataset Structure\nIn general, the folder structure for a preprocessed dataset looks like this:\nThe aim of pre-processing is to create one dataset per WSI in the following structure:\n```bash\nWSI_Name\n\u251c\u2500\u2500 annotation_masks # thumbnails of extracted annotation masks\n\u2502 \u251c\u2500\u2500 all_overlaid.png # all with same dimension as the thumbnail\n\u2502 \u251c\u2500\u2500 tumor.png\n\u2502 \u2514\u2500\u2500 ... \n\u251c\u2500\u2500 context # context patches, if extracted\n\u2502 \u251c\u2500\u2500 2 # subfolder for each scale\n\u2502 \u2502 \u251c\u2500\u2500 WSI_Name_row1_col1_context_2.png\n\u2502 \u2502 \u251c\u2500\u2500 WSI_Name_row2_col1_context_2.png\n\u2502 \u2502 \u2514\u2500\u2500 ...\n\u2502 \u2514\u2500\u2500 4\n\u2502 \u2502 \u251c\u2500\u2500 WSI_Name_row1_col1_context_2.png\n\u2502 \u2502 \u251c\u2500\u2500 WSI_Name_row2_col1_context_2.png\n\u2502 \u2502 \u2514\u2500\u2500 ...\n\u251c\u2500\u2500 masks # Mask (numpy) files for each patch -> optional folder for segmentation\n\u2502 \u251c\u2500\u2500 WSI_Name_row1_col1.npy\n\u2502 \u251c\u2500\u2500 WSI_Name_row2_col1.npy\n\u2502 \u2514\u2500\u2500 ...\n\u251c\u2500\u2500 metadata # Metadata files for each patch\n\u2502 \u251c\u2500\u2500 WSI_Name_row1_col1.yaml\n\u2502 \u251c\u2500\u2500 WSI_Name_row2_col1.yaml\n\u2502 \u2514\u2500\u2500 ...\n\u251c\u2500\u2500 patches # Patches as .png files\n\u2502 \u251c\u2500\u2500 WSI_Name_row1_col1.png\n\u2502 \u251c\u2500\u2500 WSI_Name_row2_col1.png\n\u2502 \u2514\u2500\u2500 ...\n\u251c\u2500\u2500 thumbnails # Different kind of thumbnails\n\u2502 \u251c\u2500\u2500 thumbnail_mpp_5.png\n\u2502 \u251c\u2500\u2500 thumbnail_downsample_32.png\n\u2502 \u2514\u2500\u2500 ...\n\u251c\u2500\u2500 tissue_masks # Tissue mask images for checking\n\u2502 \u251c\u2500\u2500 mask.png # all with same dimension as the thumbnail\n\u2502 \u251c\u2500\u2500 mask_nogrid.png\n\u2502 \u2514\u2500\u2500 tissue_grid.png\n\u251c\u2500\u2500 mask.png # tissue mask with green grid \n\u251c\u2500\u2500 metadata.yaml # WSI metdata for patch extraction\n\u251c\u2500\u2500 patch_metadata.json # Patch metadata of WSI merged in one file\n\u2514\u2500\u2500 thumbnail.png # WSI thumbnail\n```\n\n## Further information\nFor more information, check out the git.\n\n## License\n<p xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:dct=\"http://purl.org/dc/terms/\"><a property=\"dct:title\" rel=\"cc:attributionURL\" href=\"https://github.com/TIO-IKIM/PathoPatcher\">PathoPatcher</a> by <a rel=\"cc:attributionURL dct:creator\" property=\"cc:attributionName\" href=\"https://github.com/FabianHoerst\">Fabian H\u00f6rst, University Hospital Essen,</a> is licensed under <a href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/?ref=chooser-v1\" target=\"_blank\" rel=\"license noopener noreferrer\" style=\"display:inline-block;\">CC BY-NC-SA 4.0<img style=\"height:22px!important;margin-left:3px;vertical-align:text-bottom;\" src=\"https://mirrors.creativecommons.org/presskit/icons/cc.svg?ref=chooser-v1\"><img style=\"height:22px!important;margin-left:3px;vertical-align:text-bottom;\" src=\"https://mirrors.creativecommons.org/presskit/icons/by.svg?ref=chooser-v1\"><img style=\"height:22px!important;margin-left:3px;vertical-align:text-bottom;\" src=\"https://mirrors.creativecommons.org/presskit/icons/nc.svg?ref=chooser-v1\"><img style=\"height:22px!important;margin-left:3px;vertical-align:text-bottom;\" src=\"https://mirrors.creativecommons.org/presskit/icons/sa.svg?ref=chooser-v1\"></a></p>\n",
"bugtrack_url": null,
"license": "CC BY-NC-SA 4.0",
"summary": "PathoPatch - Accelerating Artificial Intelligence Based Whole Slide Image Analysis with an Optimized Preprocessing Pipeline",
"version": "1.0.3",
"project_urls": {
"Homepage": "https://github.com/TIO-IKIM/PathoPatcher"
},
"split_keywords": [
"python",
" pathopatch"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "422a1943d2bb630c1b4e827d9936e8aba7241dec260528b33255022982f7e4ee",
"md5": "db37f8fa92c1aa91860538c9a974773a",
"sha256": "d81b135fb81778c8dfdf8a7a2fbd3d93c272793a21cdbd740302b79fe5791bc9"
},
"downloads": -1,
"filename": "pathopatch-1.0.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "db37f8fa92c1aa91860538c9a974773a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 5825117,
"upload_time": "2024-10-23T14:24:54",
"upload_time_iso_8601": "2024-10-23T14:24:54.578268Z",
"url": "https://files.pythonhosted.org/packages/42/2a/1943d2bb630c1b4e827d9936e8aba7241dec260528b33255022982f7e4ee/pathopatch-1.0.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "473730e76eb7be3ff15664bce2549aca82b8085304125b00735c7163daadac81",
"md5": "58a2a94004ea75abf47f5b7f322e9ce5",
"sha256": "d2204d305eee79a9cf4eab9a7257962de6ad065c6024ab91b7ecae9d2423bcfb"
},
"downloads": -1,
"filename": "pathopatch-1.0.3.tar.gz",
"has_sig": false,
"md5_digest": "58a2a94004ea75abf47f5b7f322e9ce5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 5818811,
"upload_time": "2024-10-23T14:24:56",
"upload_time_iso_8601": "2024-10-23T14:24:56.832688Z",
"url": "https://files.pythonhosted.org/packages/47/37/30e76eb7be3ff15664bce2549aca82b8085304125b00735c7163daadac81/pathopatch-1.0.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-23 14:24:56",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "TIO-IKIM",
"github_project": "PathoPatcher",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "pathopatch"
}
