High dynamic range image reconstruction for multi-bit quanta image sensor

Abstract

An adaptive thresholds algorithm is proposed in this letter, which is used to determine the global optimal thresholds for multi-bit quanta image sensor (MB-QIS). Firstly, the senor model of MB-QIS is set up. Then global optimal thresholds theory is analyzed and a thresholds optimization algorithm based on the binary search is designed to determine the optimal global thresholds. Finally, the high dynamic range (HDR) images are reconstructed by the noniterative maximum likelihood estimation (MLE) image reconstruction method. The results of simulation prove that HDR imaging of MB-QIS is realized by the proposed method effectively.

References

[1] FossumE R. What to do with sub-diffraction-limit (SDL) pixels?- a proposal for a gigapixel digital film sensor (DFS)[C], 2005, New York, IEEE: 214-217

[2] FossumE R. The quanta image sensor (QIS): concepts and challenges[C], 2011, Washington, Optical Society of America: 10-14

[3] MASOODIAN S, MA J, STARKEY D, et al. A 1Mjot 1040fps 0.22e-rms stacked BSI quanta image sensor with cluster-parallel readout[C]//2017 International Image Sensor Workshop, May 30-June 2, 2017, Hiroshima, Japan. International Image Sensor Society, 2017: 230–233.

[4] FossumE R. Modeling the performance of single-bit and multi-bit quanta image sensors[J]. IEEE journal of the electron devices society, 2013, 1(9):166-174

[5] FOSSUM E R. Multi-bit quanta image sen-sors[C]//2015 International Image Sensor Workshop, June 8–11, 2015, Vaals, The Netherlands. International Image Sensor Society, 2015: 292–295.

[6] YinZ Y, WangY B M, FossumE R. Low bit-depth ADCs for multi-bit quanta image sensors[J]. IEEE journal of solid-state circuits, 2021, 56(3):950-960

[7] YangF, LuY M, SbaizL, et al.. Bits from photons: oversampled image acquisition using binary Poisson statistics[J]. IEEE transactions on image processing, 2012, 21(4):1421-1436

[8] ChanS H, ElgendyO A, WangX. Images from bits: non-iterative image reconstruction for quanta image sensors[J]. Sensors, 2016, 16(11):1961-1982

[9] RojasR A, LuoW Y, MurrayV, et al.. Learning optimal parameters for binary sensing image reconstruction algorithms[C], 2017, New York, IEEE: 2791-2795

[10] ChoiJ H, ElgendyO A, ChanS H. Image reconstruction for quanta image sensors using deep neural networks[C], 2018, New York, IEEE: 6543-6547

[11] ChanS H, LuY M. Efficient image reconstruction for gigapixel quantum image sensors[C], 2014, New York, IEEE: 312-316

[12] WongH T, LeungC S, HoD. Theoretical analysis and image reconstruction for multi-bit quanta image sensors[J]. Signal processing, 2021, 185: 108087

[13] ElgendyO A, ChanS H. Optimal threshold design for quanta image sensor[J]. IEEE transactions on computational imaging, 2017, 4(1):99-111

[14] GnanasambandamA, MaJ, ChanS H. HDR imaging with quanta image sensors: theoretical limits and optimal reconstruction[J]. IEEE transactions on computational imaging, 2020, 6: 1571-1585

[15] GaoJ, DuX X, NieK M, et al.. Analysis and elimination method of the non-fixed column noise for quantum image sensor[J]. IEEE sensor journal, 2020, 20(1):318-327

[16] MaJ J, ZhangD X, ElgendyO A, et al.. A 0.19e-rms read noise 16.7Mpixel stacked quanta image sensor with 1.1 µm-pitch backside illuminated pixels[J]. IEEE electron device letters, 2021, 42(6):891-894

[17] MaJ J, ZhangD X, ElgendyO A, et al.. A photon-counting 4Mpixel stacked BSI quanta image sensor with 0.3e-read noise and 100dB single-exposure dynamic range[C], 2021, New York, IEEE

[18] LiuB W, XuJ T. Modeling the photon counting and photoelectron counting characteristics of quanta image sensors[J]. Journal of semiconductors, 2021, 42(6):062301

[19] ZhaoT, GaoJ, XuJ T, et al.. Optimized oversam-pling and readout circuit design for quanta image sen-sor[J]. Laser & optoelectronics progress, 2021, 58(24): 2403001 in Chinese)

[20] FossumE R. Analog read noise and quantizer threshold estimation from quanta image sensor bit density[J]. IEEE journal of the electron devices society, 2022, 10: 269-274

[21] XuJ T, ZhaoX Y, HanL Q, et al.. Effect of the transition points mismatch on quanta image sensors[J]. Sensors, 2018, 18(12):4357-4370