The photon detection capability and nonlinear response characteristic of quanta image sensors make them an optimal choice for high dynamic range (HDR) imaging. To suppress ghosting artifacts in dynamic scenes and obtain high-quality HDR reconstructed images using quanta image sensors, an HDR image reconstruction method based on multi-weight factor exposure bracketing is proposed in this paper. Initially, adaptive aligning and merging of bit-planes from the same exposure is performed to suppress noise and blur in the low dynamic range (LDR) reconstructed frame. Subsequently, the exposure-referred signal-to-noise ratio (SNR) and gradient information of the frames are computed, which facilitates the reconstruction of an HDR image using multi-weight factor exposure bracketing. Results indicate that the proposed method has better reconstruction quality and is particularly effective for HDR imaging in non-ideal dynamic scenes. Compared to the single-weight exposure bracketing method and threshold optimization method, the proposed method has the lowest log-scale mean squared errors (LMSE) in reconstructed images and the perceptually uniform peak SNR is improved by 9.7%, while the perceptually uniform structural similarity index (PU-SSIM) is also improved by 3.5%, respectively.
| [1] |
Fossum E R. Some thoughts on future digital still cameras. Image sensors and signal processing for digital still cameras, 2017, Boca Raton, CRC Press: 305-314 M]
|
| [2] |
Fossum E R. What to do with sub-diffraction-limit (SDL) pixels?—a proposal for a gigapixel digital film sensor (DFS). Proceedings of the IEEE International Image Sensor Workshop (IISW), June 10–13, 2005, Snowbird, UT, USA, 2005, Cham, Springer: 1-5 [C]
|
| [3] |
Fossum E R. The quanta image sensor (QIS): concepts and challenges. Proceedings of the Optica Topical Meeting on Computational Optical Sensing and Imaging (COSI), July 10–14, 2011, Toronto, Canada, 2011, Washington DC, Optica Publishing Group [C]
|
| [4] |
Ma J, Zhang D, Elgendy OA, et al. . A 0.19 e-rms read noise 16.7 Mpixel stacked quanta image sensor with 1.1 µm-pitch backside illuminated pixels. IEEE electron device letters, 2021, 42(6): 891-894 J]
|
| [5] |
Ma J, Chan S, Fossum E R. Review of quanta image sensors for ultralow-light imaging. IEEE transactions on electron devices, 2022, 69(6): 2824-2839 J]
|
| [6] |
Li C, Qu X, Gnanasambandam A, et al. . Photon-limited object detection using non-local feature matching and knowledge distillation. Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), October 11–17, 2021, Montreal, BC, Canada, 2021, New York, IEEE: 3565-3574 [C]
|
| [7] |
Gnanasambandam A, Chan S H. HDR imaging with quanta image sensors: theoretical limits and optimal reconstruction. IEEE transactions on computational imaging, 2020, 6: 1571-1585 J]
|
| [8] |
Gyongy I, Dutton N W, Henderson R K. Single-photon tracking for high-speed vision. Sensors, 2018, 18(2): 323 J]
|
| [9] |
Elgendy O A, Chan S H. Color filter arrays for quanta image sensors. IEEE transactions on computational imaging, 2020, 6: 652-665 J]
|
| [10] |
Chen S, Ceballos A, Fossum E R, et al. . Digital integration sensor. Proceedings of the IEEE International Image Sensor Workshop (IISW), June 10–13, 2013, Snowbird, UT, USA, 2013, Cham, Springer: 64-71 [C]
|
| [11] |
Fossum E R, Ma J, Masoodian S. Quanta image sensor: concepts and progress. Proceedings of the SPIE Advanced Photon Counting Techniques X, April 20–21, 2016, Baltimore, MD, USA, 2016, Bellingham, SPIE: 985804 [C]
|
| [12] |
Suharwerdi M, Qazi G. Impact of dark current on pinned photo-diode capacitance of CMOS image sensor in low illumination regime. Optoelectronics letters, 2024, 20(11): 654-657 J]
|
| [13] |
Yang F, Lu Y M, Sbaiz L, et al. . Bits from photons: oversampled image acquisition using binary Poisson statistics. IEEE transactions on image processing, 2011, 21(4): 1421-1436 J]
|
| [14] |
Chan S H, Lu Y M. Efficient image reconstruction for gigapixel quantum image sensors. Proceedings of the IEEE Global Conference on Signal and Information Processing (GlobalSIP), December 3–5, 2014, Atlanta, GA, USA, 2014, New York, IEEE: 312-316 [C]
|
| [15] |
Chan S H, Elgendy O A, Wang X. Images from bits: non-iterative image reconstruction for quanta image sensors. Sensors, 2016, 16(11): 1961 J]
|
| [16] |
Elgendy O A, Chan S H. Optimal threshold design for quanta image sensor. IEEE transactions on computational imaging, 2017, 4(1): 99-111 J]
|
| [17] |
Zhang D, Lian Q, Su Y, et al. . Dual-prior integrated image reconstruction for quanta image sensors using multi-agent consensus equilibrium. IEEE/CAA journal of automatica sinica, 2023, 10(6): 1407-1420 J]
|
| [18] |
Sanghvi Y, Gnanasambandam A, Chan S H. Photon limited non-blind deblurring using algorithm unrolling. IEEE transactions on computational imaging, 2022, 8: 851-864 J]
|
| [19] |
Ma S, Gupta S, Ulku A C, et al. . Quanta burst photography. ACM transactions on graphics, 2020, 39(4): 79 J]
|
| [20] |
Iwabuchi K, Yamazaki T, Hamamoto T, et al. . Iterative image reconstruction for quanta image sensor by using variance-based motion estimation. Proceedings of the International Image Sensor Workshop (IISW), June 3–6, 2019, Kyoto, Japan, 2019, Cham, Springer: 1-4 [C]
|
| [21] |
Hu T, Yan Q, Qi Y, et al. . Generating content for HDR deghosting from frequency view. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 16–22, 2024, Seattle, WA, USA, 2024, New York, IEEE: 25732-25741 [C]
|
| [22] |
Kong L, Li B, Xiong Y, et al. . SAFNet: selective alignment fusion network for efficient HDR imaging. Proceedings of the European Conference on Computer Vision (ECCV), October 14–20, 2024, Glasgow, Scotland, 2024, Cham, Springer: 256-273 [C]
|
| [23] |
Granados M, Ajdin B, Wand M, et al. . Optimal HDR reconstruction with linear digital cameras. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 13–18, 2010, San Francisco, CA, USA, 2010, New York, IEEE: 539-546 [C]
|
| [24] |
Gnanasambandam A, Chan S H. Exposure-referred signal-to-noise ratio for digital image sensors. IEEE transactions on computational imaging, 2022, 8: 561-575 J]
|
| [25] |
Aydin T O, Mantiuk R K, Seidel H P. Extending quality metrics to full luminance range images. Proceedings of the SPIE Conference on Human Vision and Electronic Imaging XIII, January, 2008, San Jose, CA, USA, 2008, Bellingham, SPIE: 6806B-1-6806B-10 [C]
|
RIGHTS & PERMISSIONS
Tianjin University of Technology
PDF
