# Histopathology Models and Workflows
## Models on public data
This repository contains a set of models built on and for public datasets (PANDA, TCGA). Detailed instructions to
on-board the datasets and run the models are provided [on
readthedocs](https://hi-ml.readthedocs.io/en/latest/histopathology.html).
## Getting started
### Setting up Python
For working on the histopathology folder, please create a separate Conda environment.
```shell
cd hi-ml-cpath
make env
```
You can then activate the environment via `conda activate HimlHisto`. Set VSCode to use this Conda environment, by choosing "Python: Select Interpreter"
from the command palette.
If the dependencies need to be updated, please modify `hi-ml-cpath/primary_deps.yml`, and then run the script
`hi-ml-cpath/create_and_lock_environment.sh`. This will create a full "locked" environment specification with pinned
versions of all depdencies.
### Setting up AzureML
In addition, please download an AzureML workspace configuration file for the workspace that you wish to use:
* In the browser, navigate to the workspace in question
* Click on the drop-down menu on upper right of the page, to the left of your account picture.
* Select "Download config file".
* Save that file into the the repository root.
Once that config file is in place, all Python runs that you start inside the `hi-ml-cpath` folder will automatically use this config file.
## Running histopathology models
To test your setup, please execute in the `hi-ml-cpath` folder:
```shell
conda activate HimlHisto
python ../hi-ml/src/health_ml/runner.py --model health_cpath.TcgaCrckImageNetMIL --cluster=training-nd24
```
This should start an AzureML job in the AzureML workspace that you configured above via `config.json`. You may need to adjust the name of
the compute cluster (`training-nd24` in the above example).
### Conda environment
If you start your jobs in the `hi-ml-cpath` folder, they will automatically pick up the Conda environment file that is present in that folder.
If you start your jobs in a different folder, you need to add the `--conda_env` option to point to the file `<repo_root>/hi-ml-cpath/environment.yml`.
## Running histopathology tests
In the `hi-ml-cpath` folder, run
```shell
make call_pytest
```
Inside of VSCode, all tests in the repository should be picked up automatically. You can exclude the tests for the `hi-ml` and `hi-ml-azure` packages by
modifying `python.testing.pytestArgs` in the VSCode `.vscode/settings.json` file.
## Tests that require a GPU
The test pipeline for the histopathology folder contains a run of `pytest` on a machine with 2 GPUs. Only tests that are
marked with the `pytest` mark `gpu` are executed on that GPU machine. Note that all tests that bear the `gpu` mark will
_also_ be executed when running on a CPU machine. You need to manually add a `skipif` flag for tests that are meant to
exclusively run on GPU machines. This also helps to ensure that the test suite can pass when executed outside of the
build agents.
* Tests that run only on a CPU machine: Provide no `pytest` marks
```python
def test_my_code() -> None:
pass
```
* Tests that run on both on a CPU and on a GPU machine: Add `@pytest.mark.gpu`
```python
@pytest.mark.gpu
def test_my_code() -> None:
pass
```
* Tests that run only on a GPU machine:
```python
from health_ml.utils.common_utils import is_gpu_available
no_gpu = not is_gpu_available()
@pytest.mark.skipif(no_gpu, reason="Test requires GPU")
@pytest.mark.gpu
def test_my_code() -> None:
pass
```
Raw data
{
"_id": null,
"home_page": "https://github.com/microsoft/hi-ml",
"name": "hi-ml-cpath",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "Health Futures,Health Intelligence,Computational Pathology,AzureML",
"author": "Biomedical Imaging Team @ Microsoft Health Futures",
"author_email": "innereyedev@microsoft.com",
"download_url": "https://files.pythonhosted.org/packages/82/13/b924045573c7bc795401feb6913aa6b2234c9a6e3bcdf3732e87951d59cb/hi-ml-cpath-0.4.3.tar.gz",
"platform": null,
"description": "# Histopathology Models and Workflows\n\n## Models on public data\n\nThis repository contains a set of models built on and for public datasets (PANDA, TCGA). Detailed instructions to\non-board the datasets and run the models are provided [on\nreadthedocs](https://hi-ml.readthedocs.io/en/latest/histopathology.html).\n\n## Getting started\n\n### Setting up Python\n\nFor working on the histopathology folder, please create a separate Conda environment.\n\n```shell\ncd hi-ml-cpath\nmake env\n```\n\nYou can then activate the environment via `conda activate HimlHisto`. Set VSCode to use this Conda environment, by choosing \"Python: Select Interpreter\"\nfrom the command palette.\n\nIf the dependencies need to be updated, please modify `hi-ml-cpath/primary_deps.yml`, and then run the script\n`hi-ml-cpath/create_and_lock_environment.sh`. This will create a full \"locked\" environment specification with pinned\nversions of all depdencies.\n\n### Setting up AzureML\n\nIn addition, please download an AzureML workspace configuration file for the workspace that you wish to use:\n\n* In the browser, navigate to the workspace in question\n* Click on the drop-down menu on upper right of the page, to the left of your account picture.\n* Select \"Download config file\".\n* Save that file into the the repository root.\n\nOnce that config file is in place, all Python runs that you start inside the `hi-ml-cpath` folder will automatically use this config file.\n\n## Running histopathology models\n\nTo test your setup, please execute in the `hi-ml-cpath` folder:\n\n```shell\nconda activate HimlHisto\npython ../hi-ml/src/health_ml/runner.py --model health_cpath.TcgaCrckImageNetMIL --cluster=training-nd24\n```\n\nThis should start an AzureML job in the AzureML workspace that you configured above via `config.json`. You may need to adjust the name of\nthe compute cluster (`training-nd24` in the above example).\n\n### Conda environment\n\nIf you start your jobs in the `hi-ml-cpath` folder, they will automatically pick up the Conda environment file that is present in that folder.\nIf you start your jobs in a different folder, you need to add the `--conda_env` option to point to the file `<repo_root>/hi-ml-cpath/environment.yml`.\n\n## Running histopathology tests\n\nIn the `hi-ml-cpath` folder, run\n\n```shell\nmake call_pytest\n```\n\nInside of VSCode, all tests in the repository should be picked up automatically. You can exclude the tests for the `hi-ml` and `hi-ml-azure` packages by\nmodifying `python.testing.pytestArgs` in the VSCode `.vscode/settings.json` file.\n\n## Tests that require a GPU\n\nThe test pipeline for the histopathology folder contains a run of `pytest` on a machine with 2 GPUs. Only tests that are\nmarked with the `pytest` mark `gpu` are executed on that GPU machine. Note that all tests that bear the `gpu` mark will\n_also_ be executed when running on a CPU machine. You need to manually add a `skipif` flag for tests that are meant to\nexclusively run on GPU machines. This also helps to ensure that the test suite can pass when executed outside of the\nbuild agents.\n\n* Tests that run only on a CPU machine: Provide no `pytest` marks\n\n```python\ndef test_my_code() -> None:\n pass\n```\n\n* Tests that run on both on a CPU and on a GPU machine: Add `@pytest.mark.gpu`\n\n```python\n@pytest.mark.gpu\ndef test_my_code() -> None:\n pass\n```\n\n* Tests that run only on a GPU machine:\n\n```python\nfrom health_ml.utils.common_utils import is_gpu_available\nno_gpu = not is_gpu_available()\n\n@pytest.mark.skipif(no_gpu, reason=\"Test requires GPU\")\n@pytest.mark.gpu\ndef test_my_code() -> None:\n pass\n```\n\n\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "Microsoft Health Futures package for deep learning on histopathology images",
"version": "0.4.3",
"project_urls": {
"Homepage": "https://github.com/microsoft/hi-ml"
},
"split_keywords": [
"health futures",
"health intelligence",
"computational pathology",
"azureml"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1eb643abe4c9865d5e42f2a2cf465125900921c7a4c6485f3fd712cdf14a6884",
"md5": "942a1f05da32bb635e8d3ed1e6ce333c",
"sha256": "0dab910a49fecb1440f26861d0459f09972ce4ce4fb1e4106b38d6428c51235c"
},
"downloads": -1,
"filename": "hi_ml_cpath-0.4.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "942a1f05da32bb635e8d3ed1e6ce333c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 141902,
"upload_time": "2024-03-07T13:31:27",
"upload_time_iso_8601": "2024-03-07T13:31:27.965035Z",
"url": "https://files.pythonhosted.org/packages/1e/b6/43abe4c9865d5e42f2a2cf465125900921c7a4c6485f3fd712cdf14a6884/hi_ml_cpath-0.4.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8213b924045573c7bc795401feb6913aa6b2234c9a6e3bcdf3732e87951d59cb",
"md5": "5300f93724fca3a77bbec08ed6c92bb8",
"sha256": "4f6f20345b1921dc03dbe6ffabc74c72c490ce55c52fd5cc98fa11a59ebc80af"
},
"downloads": -1,
"filename": "hi-ml-cpath-0.4.3.tar.gz",
"has_sig": false,
"md5_digest": "5300f93724fca3a77bbec08ed6c92bb8",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 114621,
"upload_time": "2024-03-07T13:31:29",
"upload_time_iso_8601": "2024-03-07T13:31:29.815882Z",
"url": "https://files.pythonhosted.org/packages/82/13/b924045573c7bc795401feb6913aa6b2234c9a6e3bcdf3732e87951d59cb/hi-ml-cpath-0.4.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-07 13:31:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "microsoft",
"github_project": "hi-ml",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"test_requirements": [],
"lcname": "hi-ml-cpath"
}
JSON
