🆕 *For examples running `DiffDRR` on real data, please check out our latest work, [`DiffPose`](https://github.com/eigenvivek/DiffPose).*
DiffDRR
================
> Auto-differentiable DRR synthesis and optimization in PyTorch
[](https://github.com/eigenvivek/DiffDRR/actions/workflows/test.yaml)
[](https://arxiv.org/abs/2208.12737)
[](LICENSE)
[](https://pepy.tech/project/diffdrr)
[](https://vivekg.dev/DiffDRR/)
[](https://github.com/psf/black)
`DiffDRR` is a PyTorch-based digitally reconstructed radiograph (DRR) generator that provides
1. Auto-differentiable DRR syntheisis
2. GPU-accelerated rendering
3. A pure Python implementation
Most importantly, `DiffDRR` implements DRR synthesis as a PyTorch module, making it interoperable in deep learning pipelines.
Below is a comparison of `DiffDRR` to a real X-ray (X-rays and CTs from the [DeepFluoro dataset](https://github.com/rg2/DeepFluoroLabeling-IPCAI2020)):
## Installation Guide
To install `DiffDRR` from PyPI:
```zsh
pip install diffdrr
```
## Usage
The following minimal example specifies the geometry of the projectional radiograph imaging system and traces rays through a CT volume:
``` python
import matplotlib.pyplot as plt
import torch
from diffdrr.drr import DRR
from diffdrr.data import load_example_ct
from diffdrr.visualization import plot_drr
# Read in the volume and get the isocenter
volume, spacing = load_example_ct()
bx, by, bz = torch.tensor(volume.shape) * torch.tensor(spacing) / 2
# Initialize the DRR module for generating synthetic X-rays
device = "cuda" if torch.cuda.is_available() else "cpu"
drr = DRR(
volume, # The CT volume as a numpy array
spacing, # Voxel dimensions of the CT
sdr=300.0, # Source-to-detector radius (half of the source-to-detector distance)
height=200, # Height of the DRR (if width is not seperately provided, the generated image is square)
delx=4.0, # Pixel spacing (in mm)
).to(device)
# Set the camera pose with rotation (yaw, pitch, roll) and translation (x, y, z)
rotation = torch.tensor([[torch.pi, 0.0, torch.pi / 2]], device=device)
translation = torch.tensor([[bx, by, bz]], device=device)
# 📸 Also note that DiffDRR can take many representations of SO(3) 📸
# For example, quaternions, rotation matrix, axis-angle, etc...
img = drr(rotation, translation, parameterization="euler_angles", convention="ZYX")
plot_drr(img, ticks=False)
plt.show()
```
On a single NVIDIA RTX 2080 Ti GPU, producing such an image takes
33.6 ms ± 27.8 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
The full example is available at
[`introduction.ipynb`](https://vivekg.dev/DiffDRR/tutorials/introduction.html).
## 2D/3D Registration
We demonstrate the utility of our auto-differentiable DRR generator by
solving the 2D/3D registration problem with gradient-based optimization.
Here, we generate two DRRs:
1. A fixed DRR from a set of ground truth parameters
2. A moving DRR from randomly initialized parameters
To solve the registration problem, we use gradient descent to maximize
an image loss similarity metric between the two DRRs. This produces
optimization runs like this:
The full example is available at
[`optimizers.ipynb`](https://vivekg.dev/DiffDRR/tutorials/optimizers.html).
## Development
`DiffDRR` source code, docs, and CI are all built using
[`nbdev`](https://nbdev.fast.ai/). To get set up with `nbdev`, install
the following
``` zsh
mamba install jupyterlab nbdev -c fastai -c conda-forge
nbdev_install_quarto # To build docs
nbdev_install_hooks # Make notebooks git-friendly
```
Running `nbdev_help` will give you the full list of options. The most
important ones are
``` zsh
nbdev_preview # Render docs locally and inspect in browser
nbdev_clean # NECESSARY BEFORE PUSHING
nbdev_test # tests notebooks
nbdev_export # builds package and builds docs
```
For more details, follow this [in-depth
tutorial](https://nbdev.fast.ai/tutorials/tutorial.html).
## How does `DiffDRR` work?
`DiffDRR` reformulates Siddon’s method,[^1] the
canonical algorithm for calculating the radiologic path of an X-ray
through a volume, as a series of vectorized tensor operations. This
version of the algorithm is easily implemented in tensor algebra
libraries like PyTorch to achieve a fast auto-differentiable DRR
generator.
[^1]: [Siddon RL. Fast calculation of
the exact radiological path for a three-dimensional CT array. Medical
Physics, 2(12):252–5, 1985.](https://doi.org/10.1118/1.595715)
## Citing `DiffDRR`
If you find `DiffDRR` useful in your work, please cite our
[paper](https://doi.org/10.1007/978-3-031-23179-7_1) (or the [freely
accessible arXiv version](https://arxiv.org/abs/2208.12737)):
@inproceedings{gopalakrishnanDiffDRR2022,
author = {Gopalakrishnan, Vivek and Golland, Polina},
title = {Fast Auto-Differentiable Digitally Reconstructed Radiographs for Solving Inverse Problems in Intraoperative Imaging},
year = {2022},
booktitle = {Clinical Image-based Procedures: 11th International Workshop, CLIP 2022, Held in Conjunction with MICCAI 2022, Singapore, Proceedings},
series = {Lecture Notes in Computer Science},
publisher = {Springer},
doi = {https://doi.org/10.1007/978-3-031-23179-7_1},
}
Raw data
{
"_id": null,
"home_page": "https://github.com/eigenvivek/DiffDRR",
"name": "diffdrr",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "nbdev jupyter notebook python",
"author": "Vivek Gopalakrishnan",
"author_email": "vivekg@mit.edu",
"download_url": "https://files.pythonhosted.org/packages/35/a7/c2300dc219a6c917589aa45834517ed827c3f8689ca1ae5440f97d358558/diffdrr-0.3.12.tar.gz",
"platform": null,
"description": "\ud83c\udd95 *For examples running `DiffDRR` on real data, please check out our latest work, [`DiffPose`](https://github.com/eigenvivek/DiffPose).*\n\nDiffDRR\n================\n\n> Auto-differentiable DRR synthesis and optimization in PyTorch\n\n[](https://github.com/eigenvivek/DiffDRR/actions/workflows/test.yaml)\n[](https://arxiv.org/abs/2208.12737)\n[](LICENSE)\n[](https://pepy.tech/project/diffdrr)\n[](https://vivekg.dev/DiffDRR/)\n[](https://github.com/psf/black)\n\n`DiffDRR` is a PyTorch-based digitally reconstructed radiograph (DRR) generator that provides\n\n1. Auto-differentiable DRR syntheisis\n2. GPU-accelerated rendering\n3. A pure Python implementation\n\nMost importantly, `DiffDRR` implements DRR synthesis as a PyTorch module, making it interoperable in deep learning pipelines.\n\nBelow is a comparison of `DiffDRR` to a real X-ray (X-rays and CTs from the [DeepFluoro dataset](https://github.com/rg2/DeepFluoroLabeling-IPCAI2020)):\n\n\n\n## Installation Guide\n\nTo install `DiffDRR` from PyPI:\n```zsh\npip install diffdrr\n```\n\n## Usage\n\nThe following minimal example specifies the geometry of the projectional radiograph imaging system and traces rays through a CT volume:\n\n``` python\nimport matplotlib.pyplot as plt\nimport torch\n\nfrom diffdrr.drr import DRR\nfrom diffdrr.data import load_example_ct\nfrom diffdrr.visualization import plot_drr\n\n# Read in the volume and get the isocenter\nvolume, spacing = load_example_ct()\nbx, by, bz = torch.tensor(volume.shape) * torch.tensor(spacing) / 2\n\n# Initialize the DRR module for generating synthetic X-rays\ndevice = \"cuda\" if torch.cuda.is_available() else \"cpu\"\ndrr = DRR(\n volume, # The CT volume as a numpy array\n spacing, # Voxel dimensions of the CT\n sdr=300.0, # Source-to-detector radius (half of the source-to-detector distance)\n height=200, # Height of the DRR (if width is not seperately provided, the generated image is square)\n delx=4.0, # Pixel spacing (in mm)\n).to(device)\n\n# Set the camera pose with rotation (yaw, pitch, roll) and translation (x, y, z)\nrotation = torch.tensor([[torch.pi, 0.0, torch.pi / 2]], device=device)\ntranslation = torch.tensor([[bx, by, bz]], device=device)\n\n# \ud83d\udcf8 Also note that DiffDRR can take many representations of SO(3) \ud83d\udcf8\n# For example, quaternions, rotation matrix, axis-angle, etc...\nimg = drr(rotation, translation, parameterization=\"euler_angles\", convention=\"ZYX\")\nplot_drr(img, ticks=False)\nplt.show()\n```\n\n\n\nOn a single NVIDIA RTX 2080 Ti GPU, producing such an image takes\n\n 33.6 ms \u00b1 27.8 \u00b5s per loop (mean \u00b1 std. dev. of 7 runs, 10 loops each)\n\nThe full example is available at\n[`introduction.ipynb`](https://vivekg.dev/DiffDRR/tutorials/introduction.html).\n\n## 2D/3D Registration\n\nWe demonstrate the utility of our auto-differentiable DRR generator by\nsolving the 2D/3D registration problem with gradient-based optimization.\nHere, we generate two DRRs:\n\n1. A fixed DRR from a set of ground truth parameters\n2. A moving DRR from randomly initialized parameters\n\nTo solve the registration problem, we use gradient descent to maximize\nan image loss similarity metric between the two DRRs. This produces\noptimization runs like this:\n\n\n\nThe full example is available at\n[`optimizers.ipynb`](https://vivekg.dev/DiffDRR/tutorials/optimizers.html).\n\n## Development\n\n`DiffDRR` source code, docs, and CI are all built using\n[`nbdev`](https://nbdev.fast.ai/). To get set up with `nbdev`, install\nthe following\n\n``` zsh\nmamba install jupyterlab nbdev -c fastai -c conda-forge \nnbdev_install_quarto # To build docs\nnbdev_install_hooks # Make notebooks git-friendly\n```\n\nRunning `nbdev_help` will give you the full list of options. The most\nimportant ones are\n\n``` zsh\nnbdev_preview # Render docs locally and inspect in browser\nnbdev_clean # NECESSARY BEFORE PUSHING\nnbdev_test # tests notebooks\nnbdev_export # builds package and builds docs\n```\n\nFor more details, follow this [in-depth\ntutorial](https://nbdev.fast.ai/tutorials/tutorial.html).\n\n## How does `DiffDRR` work?\n\n`DiffDRR` reformulates Siddon\u2019s method,[^1] the\ncanonical algorithm for calculating the radiologic path of an X-ray\nthrough a volume, as a series of vectorized tensor operations. This\nversion of the algorithm is easily implemented in tensor algebra\nlibraries like PyTorch to achieve a fast auto-differentiable DRR\ngenerator.\n\n[^1]: [Siddon RL. Fast calculation of\nthe exact radiological path for a three-dimensional CT array. Medical\nPhysics, 2(12):252\u20135, 1985.](https://doi.org/10.1118/1.595715)\n\n## Citing `DiffDRR`\n\nIf you find `DiffDRR` useful in your work, please cite our\n[paper](https://doi.org/10.1007/978-3-031-23179-7_1) (or the [freely\naccessible arXiv version](https://arxiv.org/abs/2208.12737)):\n\n @inproceedings{gopalakrishnanDiffDRR2022,\n author = {Gopalakrishnan, Vivek and Golland, Polina},\n title = {Fast Auto-Differentiable Digitally Reconstructed Radiographs for Solving Inverse Problems in Intraoperative Imaging},\n year = {2022},\n booktitle = {Clinical Image-based Procedures: 11th International Workshop, CLIP 2022, Held in Conjunction with MICCAI 2022, Singapore, Proceedings},\n series = {Lecture Notes in Computer Science},\n publisher = {Springer},\n doi = {https://doi.org/10.1007/978-3-031-23179-7_1},\n }\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "Auto-differentiable digitally reconstructed radiographs in PyTorch",
"version": "0.3.12",
"project_urls": {
"Homepage": "https://github.com/eigenvivek/DiffDRR"
},
"split_keywords": [
"nbdev",
"jupyter",
"notebook",
"python"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "4c3fe22c78db4ad5739e23dc7b57b9bec62d1b7d0db100ac1456dd1382183e19",
"md5": "d19a6d066154392ceb6c11d9b1542a3c",
"sha256": "451b84ccf3c4eb568abac0609da89ff533c865a1398d1a4d647aee220bb5e445"
},
"downloads": -1,
"filename": "diffdrr-0.3.12-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d19a6d066154392ceb6c11d9b1542a3c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 34658726,
"upload_time": "2024-03-12T17:52:33",
"upload_time_iso_8601": "2024-03-12T17:52:33.805480Z",
"url": "https://files.pythonhosted.org/packages/4c/3f/e22c78db4ad5739e23dc7b57b9bec62d1b7d0db100ac1456dd1382183e19/diffdrr-0.3.12-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "35a7c2300dc219a6c917589aa45834517ed827c3f8689ca1ae5440f97d358558",
"md5": "bc4f8cde905be799e26a266bb0a08375",
"sha256": "710bc337ac7b5b822a164a2c09b24187d7e243c505f8aa83b967428ef2ec53ea"
},
"downloads": -1,
"filename": "diffdrr-0.3.12.tar.gz",
"has_sig": false,
"md5_digest": "bc4f8cde905be799e26a266bb0a08375",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 34580061,
"upload_time": "2024-03-12T17:52:50",
"upload_time_iso_8601": "2024-03-12T17:52:50.299843Z",
"url": "https://files.pythonhosted.org/packages/35/a7/c2300dc219a6c917589aa45834517ed827c3f8689ca1ae5440f97d358558/diffdrr-0.3.12.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-12 17:52:50",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "eigenvivek",
"github_project": "DiffDRR",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "diffdrr"
}
JSON
