# DiffPose
<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
> Intraoperative 2D/3D registration via differentiable X-ray rendering
[](https://github.com/eigenvivek/DiffPose/actions/workflows/test.yaml)
[](LICENSE)
[](https://vivekg.dev/DiffPose)
[](https://github.com/psf/black)
## Install
To install `DiffPose` and the requirements in
[`environment.yml`](https://github.com/eigenvivek/DiffPose/blob/main/environment.yml),
run:
``` zsh
pip install diffpose
```
The differentiable X-ray renderer that powers the backend of `DiffPose`
is available at [`DiffDRR`](https://github.com/eigenvivek/DiffDRR).
## Datasets
We evaluate `DiffPose` networks on the following open-source datasets:
| **Dataset** | **Anatomy** | **\# of Subjects** | **\# of 2D Images** | **CTs** | **X-rays** | Fiducials |
|----------------------------------------------------------------------------|--------------------|:------------------:|:-------------------:|:-------:|:----------:|:---------:|
| [`DeepFluoro`](https://github.com/rg2/DeepFluoroLabeling-IPCAI2020) | Pelvis | 6 | 366 | ✅ | ✅ | ❌ |
| [`Ljubljana`](https://lit.fe.uni-lj.si/en/research/resources/3D-2D-GS-CA/) | Cerebrovasculature | 10 | 20 | ✅ | ✅ | ✅ |
<!-- | [`2D-3D-GS`](https://lit.fe.uni-lj.si/en/research/resources/2D-3D-GS/) | Lumbar Spine | 1 | 18 | ✅ | ✅ | ✅ |
| [`VerSe`](https://github.com/anjany/verse) | Spine | 355 | N/A | ✅ | ❌ | ❌ | -->
- `DeepFluoro` ([**Grupp et al.,
2020**](https://link.springer.com/article/10.1007/s11548-020-02162-7))
provides paired X-ray fluoroscopy images and CT volume of the pelvis.
The data were collected from six cadaveric subjects at John Hopkins
University. Ground truth camera poses were estimated with an offline
registration process. A visualization of one X-ray / CT pair in the
`DeepFluoro` dataset is available
[here](https://vivekg.dev/DiffPose/experiments/render.html).
``` zsh
mkdir -p data/
wget --no-check-certificate -O data/ipcai_2020_full_res_data.zip "http://archive.data.jhu.edu/api/access/datafile/:persistentId/?persistentId=doi:10.7281/T1/IFSXNV/EAN9GH"
unzip -o data/ipcai_2020_full_res_data.zip -d data
rm data/ipcai_2020_full_res_data.zip
```
- `Ljubljana` ([**Mitrovic et al.,
2013**](https://ieeexplore.ieee.org/abstract/document/6507588))
provides paired 2D/3D digital subtraction angiography (DSA) images.
The data were collected from 10 patients undergoing endovascular
image-guided interventions at the University of Ljubljana. Ground
truth camera poses were estimated by registering surface fiducial
markers.
``` zsh
mkdir -p data/
wget --no-check-certificate -O data/ljubljana.zip "https://drive.google.com/uc?export=download&confirm=yes&id=1x585pGLI8QGk21qZ2oGwwQ9LMJ09Tqrx"
unzip -o data/ljubljana.zip -d data
rm data/ljubljana.zip
```
<!-- - `2D-3D-GS` ([**Tomaževič et al., 2004**](https://pubmed.ncbi.nlm.nih.gov/16192053/)) ...
- `VerSe` ([**Sekuboyina et al., 2020**](https://pubs.rsna.org/doi/10.1148/ryai.2020190074)) ... -->
## Experiments
To run the experiments in `DiffPose`, run the following scripts (ensure
you’ve downloaded the data first):
``` zsh
# DeepFluoro dataset
cd experiments/deepfluoro
srun python train.py # Pretrain pose regression CNN on synthetic X-rays
srun python register.py # Run test-time optimization with the best network per subject
```
``` zsh
# Ljubljana dataset
cd experiments/ljubljana
srun python train.py
srun python register.py
```
The training and test-time optimization scripts use SLURM to run on all
subjects in parallel:
- `experiments/deepfluoro/train.py` is configured to run across six
A6000 GPUs
- `experiments/deepfluoro/register.py` is configured to run across six
2080 Ti GPUs
- `experiments/ljubljana/train.py` is configured to run across twenty
2080 Ti GPUs
- `experiments/ljubljana/register.py` is configured to run on twenty
2080 Ti GPUs
The GPU configurations can be changed at the end of each script using
[`submitit`](https://github.com/facebookincubator/submitit).
## Development
`DiffPose` package, docs, and CI are all built using
[`nbdev`](https://nbdev.fast.ai/). To get set up with`nbdev`, install
the following
``` zsh
conda install jupyterlab nbdev -c fastai -c conda-forge
nbdev_install_quarto # To build docs
nbdev_install_hooks # Make notebooks git-friendly
pip install -e ".[dev]" # Install the development verison of DiffPose
```
Running `nbdev_help` will give you the full list of options. The most
important ones are
``` zsh
nbdev_preview # Render docs locally and inspect in browser
nbdev_clean # NECESSARY BEFORE PUSHING
nbdev_test # tests notebooks
nbdev_export # builds package and builds docs
nbdev_readme # Render the readme
```
For more details, follow this [in-depth
tutorial](https://nbdev.fast.ai/tutorials/tutorial.html).
## Citing `DiffPose`
If you find `DiffPose` or
[`DiffDRR`](https://github.com/eigenvivek/DiffDRR) useful in your work,
please cite the appropriate papers:
@inproceedings{gopalakrishnanDiffDRR2022,
author = {Gopalakrishnan, Vivek and Golland, Polina},
title = {Fast Auto-Differentiable Digitally Reconstructed Radiographs for Solving Inverse Problems in Intraoperative Imaging},
year = {2022},
booktitle = {Clinical Image-based Procedures: 11th International Workshop, CLIP 2022, Held in Conjunction with MICCAI 2022, Singapore, Proceedings},
series = {Lecture Notes in Computer Science},
publisher = {Springer},
doi = {https://doi.org/10.1007/978-3-031-23179-7_1},
}
Raw data
{
"_id": null,
"home_page": "https://github.com/eigenvivek/DiffPose",
"name": "diffpose",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "nbdev jupyter notebook python",
"author": "Vivek Gopalakrihsnan",
"author_email": "vivekg@mit.edu",
"download_url": "https://files.pythonhosted.org/packages/21/24/5aa218c88cc2ccc1c37499c913d5dfe1c49f1894be487cc06148a1351411/diffpose-0.0.1.tar.gz",
"platform": null,
"description": "# DiffPose\n\n<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->\n\n> Intraoperative 2D/3D registration via differentiable X-ray rendering\n\n[](https://github.com/eigenvivek/DiffPose/actions/workflows/test.yaml)\n[](LICENSE)\n[](https://vivekg.dev/DiffPose)\n[](https://github.com/psf/black)\n\n\n\n## Install\n\nTo install `DiffPose` and the requirements in\n[`environment.yml`](https://github.com/eigenvivek/DiffPose/blob/main/environment.yml),\nrun:\n\n``` zsh\npip install diffpose\n```\n\nThe differentiable X-ray renderer that powers the backend of `DiffPose`\nis available at [`DiffDRR`](https://github.com/eigenvivek/DiffDRR).\n\n## Datasets\n\nWe evaluate `DiffPose` networks on the following open-source datasets:\n\n| **Dataset** | **Anatomy** | **\\# of Subjects** | **\\# of 2D Images** | **CTs** | **X-rays** | Fiducials |\n|----------------------------------------------------------------------------|--------------------|:------------------:|:-------------------:|:-------:|:----------:|:---------:|\n| [`DeepFluoro`](https://github.com/rg2/DeepFluoroLabeling-IPCAI2020) | Pelvis | 6 | 366 | \u2705 | \u2705 | \u274c |\n| [`Ljubljana`](https://lit.fe.uni-lj.si/en/research/resources/3D-2D-GS-CA/) | Cerebrovasculature | 10 | 20 | \u2705 | \u2705 | \u2705 |\n\n<!-- | [`2D-3D-GS`](https://lit.fe.uni-lj.si/en/research/resources/2D-3D-GS/) | Lumbar Spine | 1 | 18 | \u2705 | \u2705 | \u2705 |\n| [`VerSe`](https://github.com/anjany/verse) | Spine | 355 | N/A | \u2705 | \u274c | \u274c | -->\n\n- `DeepFluoro` ([**Grupp et al.,\n 2020**](https://link.springer.com/article/10.1007/s11548-020-02162-7))\n provides paired X-ray fluoroscopy images and CT volume of the pelvis.\n The data were collected from six cadaveric subjects at John Hopkins\n University. Ground truth camera poses were estimated with an offline\n registration process. A visualization of one X-ray / CT pair in the\n `DeepFluoro` dataset is available\n [here](https://vivekg.dev/DiffPose/experiments/render.html).\n\n``` zsh\nmkdir -p data/\nwget --no-check-certificate -O data/ipcai_2020_full_res_data.zip \"http://archive.data.jhu.edu/api/access/datafile/:persistentId/?persistentId=doi:10.7281/T1/IFSXNV/EAN9GH\"\nunzip -o data/ipcai_2020_full_res_data.zip -d data\nrm data/ipcai_2020_full_res_data.zip\n```\n\n- `Ljubljana` ([**Mitrovic et al.,\n 2013**](https://ieeexplore.ieee.org/abstract/document/6507588))\n provides paired 2D/3D digital subtraction angiography (DSA) images.\n The data were collected from 10 patients undergoing endovascular\n image-guided interventions at the University of Ljubljana. Ground\n truth camera poses were estimated by registering surface fiducial\n markers.\n\n``` zsh\nmkdir -p data/\nwget --no-check-certificate -O data/ljubljana.zip \"https://drive.google.com/uc?export=download&confirm=yes&id=1x585pGLI8QGk21qZ2oGwwQ9LMJ09Tqrx\"\nunzip -o data/ljubljana.zip -d data\nrm data/ljubljana.zip\n```\n\n<!-- - `2D-3D-GS` ([**Toma\u017eevi\u010d et al., 2004**](https://pubmed.ncbi.nlm.nih.gov/16192053/)) ...\n - `VerSe` ([**Sekuboyina et al., 2020**](https://pubs.rsna.org/doi/10.1148/ryai.2020190074)) ... -->\n\n## Experiments\n\nTo run the experiments in `DiffPose`, run the following scripts (ensure\nyou\u2019ve downloaded the data first):\n\n``` zsh\n# DeepFluoro dataset\ncd experiments/deepfluoro\nsrun python train.py # Pretrain pose regression CNN on synthetic X-rays\nsrun python register.py # Run test-time optimization with the best network per subject\n```\n\n``` zsh\n# Ljubljana dataset\ncd experiments/ljubljana\nsrun python train.py\nsrun python register.py\n```\n\nThe training and test-time optimization scripts use SLURM to run on all\nsubjects in parallel:\n\n- `experiments/deepfluoro/train.py` is configured to run across six\n A6000 GPUs\n- `experiments/deepfluoro/register.py` is configured to run across six\n 2080 Ti GPUs\n- `experiments/ljubljana/train.py` is configured to run across twenty\n 2080 Ti GPUs\n- `experiments/ljubljana/register.py` is configured to run on twenty\n 2080 Ti GPUs\n\nThe GPU configurations can be changed at the end of each script using\n[`submitit`](https://github.com/facebookincubator/submitit).\n\n## Development\n\n`DiffPose` package, docs, and CI are all built using\n[`nbdev`](https://nbdev.fast.ai/). To get set up with`nbdev`, install\nthe following\n\n``` zsh\nconda install jupyterlab nbdev -c fastai -c conda-forge \nnbdev_install_quarto # To build docs\nnbdev_install_hooks # Make notebooks git-friendly\npip install -e \".[dev]\" # Install the development verison of DiffPose\n```\n\nRunning `nbdev_help` will give you the full list of options. The most\nimportant ones are\n\n``` zsh\nnbdev_preview # Render docs locally and inspect in browser\nnbdev_clean # NECESSARY BEFORE PUSHING\nnbdev_test # tests notebooks\nnbdev_export # builds package and builds docs\nnbdev_readme # Render the readme\n```\n\nFor more details, follow this [in-depth\ntutorial](https://nbdev.fast.ai/tutorials/tutorial.html).\n\n## Citing `DiffPose`\n\nIf you find `DiffPose` or\n[`DiffDRR`](https://github.com/eigenvivek/DiffDRR) useful in your work,\nplease cite the appropriate papers:\n\n @inproceedings{gopalakrishnanDiffDRR2022,\n author = {Gopalakrishnan, Vivek and Golland, Polina},\n title = {Fast Auto-Differentiable Digitally Reconstructed Radiographs for Solving Inverse Problems in Intraoperative Imaging},\n year = {2022},\n booktitle = {Clinical Image-based Procedures: 11th International Workshop, CLIP 2022, Held in Conjunction with MICCAI 2022, Singapore, Proceedings},\n series = {Lecture Notes in Computer Science},\n publisher = {Springer},\n doi = {https://doi.org/10.1007/978-3-031-23179-7_1},\n }\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "Patient-specific intraoperative 2D/3D registration via differentiable rendering",
"version": "0.0.1",
"project_urls": {
"Homepage": "https://github.com/eigenvivek/DiffPose"
},
"split_keywords": [
"nbdev",
"jupyter",
"notebook",
"python"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8740ba89f2da4b1e491d835614f7e87ae7985e74c29843aa195e05b5a3cfd0d8",
"md5": "606941040a9bfb144b593b3beae62aee",
"sha256": "28c668db7aa44afc77eb405b63e0d424e54f0ad30bc7a4cb468e98276271224b"
},
"downloads": -1,
"filename": "diffpose-0.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "606941040a9bfb144b593b3beae62aee",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 21251,
"upload_time": "2023-12-07T23:38:50",
"upload_time_iso_8601": "2023-12-07T23:38:50.587369Z",
"url": "https://files.pythonhosted.org/packages/87/40/ba89f2da4b1e491d835614f7e87ae7985e74c29843aa195e05b5a3cfd0d8/diffpose-0.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "21245aa218c88cc2ccc1c37499c913d5dfe1c49f1894be487cc06148a1351411",
"md5": "6c456796ea4432e70e2c78879284711f",
"sha256": "30a5bddad6d291c36e7fc0f5dbf1608c5317ee06671559e38eedc114d61792d0"
},
"downloads": -1,
"filename": "diffpose-0.0.1.tar.gz",
"has_sig": false,
"md5_digest": "6c456796ea4432e70e2c78879284711f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 21426,
"upload_time": "2023-12-07T23:38:52",
"upload_time_iso_8601": "2023-12-07T23:38:52.644090Z",
"url": "https://files.pythonhosted.org/packages/21/24/5aa218c88cc2ccc1c37499c913d5dfe1c49f1894be487cc06148a1351411/diffpose-0.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-12-07 23:38:52",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "eigenvivek",
"github_project": "DiffPose",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "diffpose"
}
JSON
