MRWS: multi-stage RAW low-light image enhancement with wavelet information and SNR prior

Abstract

For low-light image enhancement tasks, RAW images surpass RGB images due to their high information content, however, their noise and single-channel nature challenge feature extraction. Existing methods using multi-stage convolutional neural network (CNN) frameworks struggle with global feature extraction, while single-stage CNN-transformer fusions often result in residual noise. To overcome these limitations, this paper introduces a multi-stage RAW image enhancement network combining CNN and transformer. Considering the characteristics inherent to the task, we devised a CNN-based denoising block for the denoising stage and incorporated wavelet information to enhance frequency features. A transformer-based correction block has been designed for the color and white balance recovery stage, with the white balance being adjusted dynamically using a signal-to-noise ratio (SNR) map. With this design, our method outperforms other state-of-the-art models in all metrics on the Sony and Fuji datasets of see-in-the-dark (SID), and achieves optimal structural similarity index measurement (SSIM) on the mono-colored raw (MCR) dataset.

Keywords

A

References

[1] Liu W, Qu Z, Gong X, et al. . Mag molten pool edge detection algorithm based on a fusion of dark channel prior dehazing and image enhancement. Optoelectronics letters. 2024, 20(10): 607-613 J]

[2] Ren W, Liu S, Ma L, et al. . Low-light image enhancement via a deep hybrid network. IEEE transactions on image processing. 2019, 28(9): 4364-4375 J]

[3] Zhang J, Liu H, Ying X, et al. . Coordinated underwater dark channel prior for artifact removal of challenging image enhancement. Optoelectronics letters. 2023, 19(7): 416-424 J]

[4] Chen C, Chen Q, Xu J, et al. . Learning to see in the dark. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 18–22, 2018, Salt Lake City, UT, USA. 2018, New York, IEEE: 32913300[C]

[5] Malik S, Soundararajan R. Semi-supervised learning for low-light image restoration through quality assisted pseudo-labeling. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, January 3–7, 2023, Waikoloa, HI, USA. 2023, New York, IEEE: 41054114[C]

[6] Chen C, Chen Q, Do M N, et al. . Seeing motion in the dark. Proceedings of the IEEE/CVF International Conference on Computer Vision, October 27–November 2, 2019, Seoul, Korea. 2019, New York, IEEE: 31853194[C]

[7] Zamir S W, Arora A, Khan S, et al. . Learning digital camera pipeline for extreme low-light imaging. Neurocomputing. 2021, 452: 37-47 J]

[8] Zhu M, Pan P, Chen W, et al. . EEMEFN: low-light image enhancement via edge-enhanced multi-exposure fusion network. Proceedings of the AAAI Conference on Artificial Intelligence, February 7–12, 2020, New York, NY, USA. 2020, Menlo Park, AAAI Press: 1310613113[C]

[9] Schwartz E, Giryes R, Bronstein A M. Deepisp: toward learning an end-to-end image processing pipeline. IEEE transactions on image processing. 2018, 28(2): 912-923 J]

[10] Zamir S W, Arora A, Khan S, et al. . Multi-stage progressive image restoration. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 19–25, 2021, Nashville, TN, USA. 2021, New York, IEEE: 1482114831[C]

[11] Huang H, Yang W, Hu Y, et al. . Towards low light enhancement with raw images. IEEE transactions on image processing. 2022, 31: 1391-1405 J]

[12] Dong X, Xu W, Miao Z, et al. . Abandoning the Baye-filter to see in the dark. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 19–25, 2022, New Orleans, LA, USA. 2022, New York, IEEE: 1743117440[C]

[13] Xu K, Yang X, Yin B, et al. . Learning to restore low-light images via decomposition-and-enhancement. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 14–19, 2020, Seattle, WA, USA. 2020, New York, IEEE: 22812290[C]

[14] Xu W, Dong X, Ma L, et al. . RawFormer: an efficient vision transformer for low-light raw image enhancement. IEEE signal processing letters. 2022, 29: 2677-2681 J]

[15] FINDER S E, AMOYAL R, TREISTER E, et al. Wavelet convolutions for large receptive fields[EB/OL]. (2024-07-08) [2025-06-26]. https://arxiv.org/abs/2407.05848.

[16] Ali A M, Benjdira B, Koubaa A, et al. . TESR: two-stage approach for enhancement and super-resolution of remote sensing images. Remote sensing. 2023, 15(9): 2346 J]

[17] Huang H, Yang W, Hu Y, et al. . Towards low light enhancement with raw images. IEEE transactions on image processing. 2022, 31: 1391-1405 J]

[18] Wan C, Yu H, Li Z, et al. . Swift parameter-free attention network for efficient super-resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18–21, 2024, Nashville, TN, USA. 2024, New York, IEEE: 6246-6256[C]

[19] Hou Q, Zhou D, Feng J. Coordinate attention for efficient mobile network design. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 20–26, 2021, Nashville, TN, USA. 2021, New York, IEEE: 1371313722[C]

[20] Wang J, Zou Y, Wu H. Image super-resolution method based on attention aggregation hierarchy feature. The visual computer. 2024, 40(4): 2655-2666 J]

[21] SONG X, HUA Z, LI J. LHDACT: lightweight hybrid dual attention CNN and transformer network for remote sensing image change detection[J]. IEEE geoscience and remote sensing letters, 2023.

[22] Xu X, Wang R, Fu C W, et al. . SNR-aware low-light image enhancement. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 19–25, 2022, New Orleans, LA, USA. 2022, New York, IEEE: 1771417724[C]

[23] Gu S, Li Y, Gool L V, et al. . Self-guided network for fast image denoising. Proceedings of the IEEE/CVF International Conference on Computer Vision, October 27–November 2, 2019, Seoul, Korea. 2019, New York, IEEE: 25112520[C]

[24] Lamba M, Mitra K. Restoring extremely dark images in real time. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 20–26, 2021, Nashville, TN, USA. 2021, New York, IEEE: 34873497[C]

[25] Meng Z, Xu R, Ho C M. GIA-Net: global information aware network for low-light imaging. European Conference on Computer Vision, August 23–28, 2020, Glasgow, UK. 2020, Cham, Springer International Publishing: 327342[C]

[26] Wang Z, Cun X, Bao J, et al. . Uformer: a general U-shaped transformer for image restoration. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 19–25, 2022, New Orleans, LA, USA. 2022, New York, IEEE: 1768317693[C]

RIGHTS & PERMISSIONS

Tianjin University of Technology