HIET: Hybrid Information Enhancement Transformer Network for Single-Photon Image Reconstruction

Abstract

Single-photon sensors are novel devices with extremely high single-photon sensitivity and temporal resolution. However, these advantages also make them highly susceptible to noise. Moreover, single-photon cameras face severe quantization as low as 1 bit/frame. These factors make it a daunting task to recover high-quality scene information from noisy single-photon data. Most current image reconstruction methods for single-photon data are mathematical approaches, which limits information utilization and algorithm performance. In this work, we propose a hybrid information enhancement model which can significantly enhance the efficiency of information utilization by leveraging attention mechanisms from both spatial and channel branches. Furthermore, we introduce a structural feature enhance module for the FFN of the transformer, which explicitly improves the model’s ability to extract and enhance high-frequency structural information through two symmetric convolution branches. Additionally, we propose a single-photon data simulation pipeline based on RAW images to address the challenge of the lack of single-photon datasets. Experimental results show that the proposed method outperforms state-of-the-art methods in various noise levels and exhibits a more efficient capability for recovering high-frequency structures and extracting information.

Keywords

single-photon images / hybrid information enhancement / structual feature enhancement / data simulation pipeline