# HDRutils
Some utility functions to generate HDR images from a sequence of exposure time or gain modulated images. You can find a separate README describing some functinos for realistic noise simulations [here](HDRutils/noise_modeling).
**Table of contents**
- [Installation](#installation)
- [Additional dependencies](#additional-dependencies)
- [HDR image I/O](#reading-and-writing)
- [Capture](#multi-exposure-capture)
- [HDR merge](#merge-input-images)
- [Demosaicing](#merge-and-demosaic-or-demosaic-and-merge)
- [RAW bayer input](#merge-raw-bayer-frames-from-non-raw-formats)
- [Alignment](#alignment)
- [Exposure estimation](#exposure-estimation)
- [Noise simulation](#noise-simulation)
- [Citation](#citation)
## Installation
To download HDRUtils, use Pypi via pip:
pip install HDRutils
If you prefer cloning this repository, install the dependencies using pip:
pip clone https://github.com/gfxdisp/HDRutils.git
cd HDRutils
pip install -e .
### Additional dependencies
You will need the [FreeImage plugin](https://imageio.readthedocs.io/en/stable/_autosummary/imageio.plugins.freeimage.html) for reading and writing OpenEXR images:
imageio_download_bin freeimage
If you wish to capture HDR stacks using a DSLR, you will need gphoto2:
sudo apt install gphoto2
## Reading and writing
Simple wrapper functions for [imageio's](https://imageio.github.io/) `imread` and `imwrite` are provided to set appropriate flags for HDR data. You can even call `imread` on RAW file formats:
```python
import HDRutils
raw_file = 'example_raw.arw'
img_RGB = HDRutils.imread(raw_file)
hdr_file = 'example.exr'
img = HDRutils.imread(raw_file)
HDRutils.imwrite('rgb.png', img_RGB)
HDRutils.imwrite('output_filename.exr', img)
```
## Multi-exposure capture
Make sure gphoto2 is installed. Additionally, set camera to **manual mode** and **disable autofocus** on the lens. Then, decide valid exposure times (by scrolling on the camera) and run:
```python
from HDRutils.capture import DSLR
camera = DSLR(ext='.arw')
exposures = ['10', '1', '1/10', '1/100']
camera.capture_HDR_stack('image', exposures)
```
## Merge input images
The [rawpy](https://github.com/letmaik/rawpy) wrapper is used to read RAW images. [Noise-aware merging](https://www.cl.cam.ac.uk/research/rainbow/projects/noise-aware-merging/) is performed using the Poisson-noise optimal estimator. The generated HDR image is linearly related to the scene radiance
```python
files = ['`image_0.arw`', '`image_1.arw`', '`image_2.arw`'] # RAW input files
HDR_img = HDRutils.merge(files)[0]
HDRutils.imwrite('merged.exr', HDR_img)
```
Sometimes the shortest exposure may contain saturated pixels. These cause artifacts when manual white-balance/color calibration is performed. Thus, `HDRutils.merge()` returns an unsaturated mask in addition to the merged image. The saturated pixels can be clipped after manual white-balance/color calibration.
This function can also be accessed from the command line. Run `HDRmerge -h` for usage.
### Merge and demosaic or demosaic and merge?
The default function processes each image individually using [libraw](https://www.libraw.org/) and then merges the RGB images. This result relies on the robust camera pipeline (including black-level subtraction, demosaicing, white-balance) provided by libraw, and should be suitable for most projects.
If you need finer control over the exact radiance values, this behaviour can be overriden to merge RAW bayer images by setting the flag `demosaic_first=False`. This mode is useful when the camera is custom-calibrated and you have an exact correspondance between camera pixels with the scene luminance and/or color. Moreover, saturated pixels can be precisely identified before demosaicing. In this mode, a basic camera pipeline is reproduced with the following steps:
Subtract black level -> Merge image stack -> Color transformation -> White-balance
Demosaicing algorithms that are currently supported can be found at [this page](https://colour-demosaicing.readthedocs.io/en/latest/colour_demosaicing.bayer.html). Change the algorithm using `HDRutils.merge(..., demosaic_first=False, demosaic=*algorithm*)`
### Merge RAW bayer frames from non-RAW formats
If your camera provides RAW frames in a non-standard format, you can still merge them in the camera color-space without libraw processing
```python
files = ['file1.png', 'file2.png', 'file3.png'] # PNG bayer input files
HDR_img = HDRutils.merge(files, demosaic_first=False, color_space='raw')[0]
HDRutils.imwrite('merged.exr', HDR_img)
```
### Alignment
While merging, some ghosting artifacts can be removed by setting `HDRutils.merge(..., align=True)`. This attempts homography alignment and corrects camera motion for still scenes. Unfortunately non-rigid motion requiring dense optical flow is not yet implemented.
### Exposure estimation
Exposure metadata from EXIF may be inaccurate and it may be benificial to estimate relative exposures directly from the image stack. Please see [our paper](https://www.cl.cam.ac.uk/research/rainbow/projects/exposure-estimation/) for details.
This feauture is currently disabled, and EXIF values are used by default. To enable exposure estimation, run `HDRutils.merge(..., estimate_exp='mst')`.
### Deglaring via MTF inversion
Camera lens aberrations produce attenuations of spatial frequencies, which have a strong effect on the image sharpness.
The Modulation Transfer Function (MTF) models the response of the camera as a function of the spatial frequency (in cycles/pixel),
and can be used to increase the sharpness of images captured by the camera, by applying a Fourier space deconvolution
[[Chen et al. 2023]](https://stereohdrgloss.mpi-inf.mpg.de/#:~:text=A%20faithful%20reproduction%20of%20gloss,a%20real%2Dworld%20reference%20object.).
The procedure to compute a camera's MTF is described in detail [here](https://github.com/gfxdisp/lf_capture/tree/master/deglare).
The output of this procedure is a JSON file describing the MTF via the coefficients of a Gaussian Mixture Model. This file can be used
in HDRutils to perform the deglaring, after merging RAW files and before demosaicing. To perform merge+deglare+demosaic on a set of RAW images,
run the merge function with the following arguments:
```
HDRutils.merge(..., color_space="raw", demosaic_first=False, mtf_json=<mtf.json>)
```
## Noise simulation
Generating realistic camera noise using calibrated parameters of real-world cameras is described [here](HDRutils/noise_modeling/).
## Citation
If you find this package useful, we would be grateful if you cite
@inproceedings{hanji2020noise,
author = {Hanji, Param and Zhong, Fangcheng and Mantiuk, Rafa{\l} K.},
title = {Noise-Aware Merging of High Dynamic Range Image Stacks without Camera Calibration},
booktitle = {Advances in Image Manipulation (ECCV workshop)},
year = {2020},
publisher = {Springer},
pages = {376--391},
url = {http://www.cl.cam.ac.uk/research/rainbow/projects/noise-aware-merging/},
}
@ARTICLE{hanji2023exposures,
author = {Hanji, Param and and Mantiuk, Rafa{\l} K.},
journal = {IEEE Transactions on Computational Imaging},
title = {Robust estimation of exposure ratios in multi-exposure image stacks},
year = {2023},
volume = {9},
number = {},
pages = {721-731},
doi = {10.1109/TCI.2023.3301338},
url = {https://www.cl.cam.ac.uk/research/rainbow/projects/exposure-estimation/},
}
Raw data
{
"_id": null,
"home_page": "https://github.com/gfxdisp/HDRutils",
"name": "HDRutils",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "HDR, Merging, Deghosting, simulation, exposure",
"author": "Param Hanji",
"author_email": "param.hanji@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/0d/6c/3286d9d21f164e127469ac6fdf2dae0ddd536d396ca86f12f57e76b4450f/hdrutils-1.2.tar.gz",
"platform": null,
"description": "# HDRutils\n\nSome utility functions to generate HDR images from a sequence of exposure time or gain modulated images. You can find a separate README describing some functinos for realistic noise simulations [here](HDRutils/noise_modeling).\n\n**Table of contents**\n- [Installation](#installation)\n - [Additional dependencies](#additional-dependencies)\n- [HDR image I/O](#reading-and-writing)\n- [Capture](#multi-exposure-capture)\n- [HDR merge](#merge-input-images)\n - [Demosaicing](#merge-and-demosaic-or-demosaic-and-merge)\n - [RAW bayer input](#merge-raw-bayer-frames-from-non-raw-formats)\n - [Alignment](#alignment)\n - [Exposure estimation](#exposure-estimation)\n- [Noise simulation](#noise-simulation)\n- [Citation](#citation)\n\n## Installation\nTo download HDRUtils, use Pypi via pip:\n\n pip install HDRutils\n\nIf you prefer cloning this repository, install the dependencies using pip:\n \n pip clone https://github.com/gfxdisp/HDRutils.git\n cd HDRutils\n pip install -e .\n\n### Additional dependencies\nYou will need the [FreeImage plugin](https://imageio.readthedocs.io/en/stable/_autosummary/imageio.plugins.freeimage.html) for reading and writing OpenEXR images:\n\n imageio_download_bin freeimage\n\nIf you wish to capture HDR stacks using a DSLR, you will need gphoto2:\n\n sudo apt install gphoto2\n\n\n## Reading and writing\nSimple wrapper functions for [imageio's](https://imageio.github.io/) `imread` and `imwrite` are provided to set appropriate flags for HDR data. You can even call `imread` on RAW file formats:\n\n```python\nimport HDRutils\n\nraw_file = 'example_raw.arw'\nimg_RGB = HDRutils.imread(raw_file)\n\nhdr_file = 'example.exr'\nimg = HDRutils.imread(raw_file)\n\nHDRutils.imwrite('rgb.png', img_RGB)\nHDRutils.imwrite('output_filename.exr', img)\n```\n\n## Multi-exposure capture\nMake sure gphoto2 is installed. Additionally, set camera to **manual mode** and **disable autofocus** on the lens. Then, decide valid exposure times (by scrolling on the camera) and run:\n\n```python\nfrom HDRutils.capture import DSLR\ncamera = DSLR(ext='.arw')\nexposures = ['10', '1', '1/10', '1/100']\ncamera.capture_HDR_stack('image', exposures)\n```\n\n## Merge input images\nThe [rawpy](https://github.com/letmaik/rawpy) wrapper is used to read RAW images. [Noise-aware merging](https://www.cl.cam.ac.uk/research/rainbow/projects/noise-aware-merging/) is performed using the Poisson-noise optimal estimator. The generated HDR image is linearly related to the scene radiance\n\n```python\nfiles = ['`image_0.arw`', '`image_1.arw`', '`image_2.arw`']\t\t# RAW input files\nHDR_img = HDRutils.merge(files)[0]\nHDRutils.imwrite('merged.exr', HDR_img)\n```\n\nSometimes the shortest exposure may contain saturated pixels. These cause artifacts when manual white-balance/color calibration is performed. Thus, `HDRutils.merge()` returns an unsaturated mask in addition to the merged image. The saturated pixels can be clipped after manual white-balance/color calibration.\n\nThis function can also be accessed from the command line. Run `HDRmerge -h` for usage.\n\n### Merge and demosaic or demosaic and merge?\nThe default function processes each image individually using [libraw](https://www.libraw.org/) and then merges the RGB images. This result relies on the robust camera pipeline (including black-level subtraction, demosaicing, white-balance) provided by libraw, and should be suitable for most projects.\n\nIf you need finer control over the exact radiance values, this behaviour can be overriden to merge RAW bayer images by setting the flag `demosaic_first=False`. This mode is useful when the camera is custom-calibrated and you have an exact correspondance between camera pixels with the scene luminance and/or color. Moreover, saturated pixels can be precisely identified before demosaicing. In this mode, a basic camera pipeline is reproduced with the following steps:\n\nSubtract black level -> Merge image stack -> Color transformation -> White-balance\n\nDemosaicing algorithms that are currently supported can be found at [this page](https://colour-demosaicing.readthedocs.io/en/latest/colour_demosaicing.bayer.html). Change the algorithm using `HDRutils.merge(..., demosaic_first=False, demosaic=*algorithm*)`\n\n\n### Merge RAW bayer frames from non-RAW formats\nIf your camera provides RAW frames in a non-standard format, you can still merge them in the camera color-space without libraw processing\n\n```python\nfiles = ['file1.png', 'file2.png', 'file3.png'] # PNG bayer input files\nHDR_img = HDRutils.merge(files, demosaic_first=False, color_space='raw')[0]\nHDRutils.imwrite('merged.exr', HDR_img)\n```\n\n### Alignment\nWhile merging, some ghosting artifacts can be removed by setting `HDRutils.merge(..., align=True)`. This attempts homography alignment and corrects camera motion for still scenes. Unfortunately non-rigid motion requiring dense optical flow is not yet implemented.\n\n\n### Exposure estimation\nExposure metadata from EXIF may be inaccurate and it may be benificial to estimate relative exposures directly from the image stack. Please see [our paper](https://www.cl.cam.ac.uk/research/rainbow/projects/exposure-estimation/) for details.\n\nThis feauture is currently disabled, and EXIF values are used by default. To enable exposure estimation, run `HDRutils.merge(..., estimate_exp='mst')`.\n\n### Deglaring via MTF inversion\nCamera lens aberrations produce attenuations of spatial frequencies, which have a strong effect on the image sharpness.\nThe Modulation Transfer Function (MTF) models the response of the camera as a function of the spatial frequency (in cycles/pixel),\nand can be used to increase the sharpness of images captured by the camera, by applying a Fourier space deconvolution\n[[Chen et al. 2023]](https://stereohdrgloss.mpi-inf.mpg.de/#:~:text=A%20faithful%20reproduction%20of%20gloss,a%20real%2Dworld%20reference%20object.).\n\nThe procedure to compute a camera's MTF is described in detail [here](https://github.com/gfxdisp/lf_capture/tree/master/deglare).\nThe output of this procedure is a JSON file describing the MTF via the coefficients of a Gaussian Mixture Model. This file can be used\nin HDRutils to perform the deglaring, after merging RAW files and before demosaicing. To perform merge+deglare+demosaic on a set of RAW images,\nrun the merge function with the following arguments:\n```\nHDRutils.merge(..., color_space=\"raw\", demosaic_first=False, mtf_json=<mtf.json>)\n```\n\n## Noise simulation\nGenerating realistic camera noise using calibrated parameters of real-world cameras is described [here](HDRutils/noise_modeling/).\n\n## Citation\nIf you find this package useful, we would be grateful if you cite\n\n @inproceedings{hanji2020noise,\n author = {Hanji, Param and Zhong, Fangcheng and Mantiuk, Rafa{\\l} K.},\n title = {Noise-Aware Merging of High Dynamic Range Image Stacks without Camera Calibration},\n booktitle = {Advances in Image Manipulation (ECCV workshop)},\n year = {2020},\n publisher = {Springer},\n pages = {376--391},\n url = {http://www.cl.cam.ac.uk/research/rainbow/projects/noise-aware-merging/},\n }\n\n @ARTICLE{hanji2023exposures,\n author = {Hanji, Param and and Mantiuk, Rafa{\\l} K.},\n journal = {IEEE Transactions on Computational Imaging},\n title = {Robust estimation of exposure ratios in multi-exposure image stacks},\n year = {2023},\n volume = {9},\n number = {},\n pages = {721-731},\n doi = {10.1109/TCI.2023.3301338},\n url = {https://www.cl.cam.ac.uk/research/rainbow/projects/exposure-estimation/},\n }\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Utility functions for performing basic operations on HDR images, including merging and deghosting",
"version": "1.2",
"project_urls": {
"Homepage": "https://github.com/gfxdisp/HDRutils"
},
"split_keywords": [
"hdr",
" merging",
" deghosting",
" simulation",
" exposure"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0d6c3286d9d21f164e127469ac6fdf2dae0ddd536d396ca86f12f57e76b4450f",
"md5": "226e1eae7d6a06ffec135716fd865f52",
"sha256": "1d4abaeafe2c6bcc446a4a668436b0d97af6d607b7b3a3a3649faa47f387c980"
},
"downloads": -1,
"filename": "hdrutils-1.2.tar.gz",
"has_sig": false,
"md5_digest": "226e1eae7d6a06ffec135716fd865f52",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 350210,
"upload_time": "2024-06-27T02:54:30",
"upload_time_iso_8601": "2024-06-27T02:54:30.027116Z",
"url": "https://files.pythonhosted.org/packages/0d/6c/3286d9d21f164e127469ac6fdf2dae0ddd536d396ca86f12f57e76b4450f/hdrutils-1.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-06-27 02:54:30",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "gfxdisp",
"github_project": "HDRutils",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "opencv_python",
"specs": [
[
"==",
"4.5.1.48"
]
]
},
{
"name": "ExifRead",
"specs": [
[
"==",
"2.3.2"
]
]
},
{
"name": "tqdm",
"specs": [
[
"==",
"4.56.0"
]
]
},
{
"name": "imageio",
"specs": [
[
"==",
"2.21.2"
]
]
},
{
"name": "rawpy",
"specs": [
[
"==",
"0.16.0"
]
]
},
{
"name": "matplotlib",
"specs": [
[
"==",
"3.3.3"
]
]
},
{
"name": "numpy",
"specs": [
[
"==",
"1.19.5"
]
]
},
{
"name": "colour_demosaicing",
"specs": [
[
"==",
"0.1.6"
]
]
},
{
"name": "scipy",
"specs": [
[
"==",
"1.7.1"
]
]
},
{
"name": "scikit-image",
"specs": [
[
"==",
"0.18.3"
]
]
},
{
"name": "pyexr",
"specs": [
[
"==",
"0.3.10"
]
]
}
],
"lcname": "hdrutils"
}
JSON
