Lightweight defocus deblurring network for curved-tunnel line scanning using wide-angle lenses

Abstract

High-resolution line scan cameras with wide-angle lenses are highly accurate and efficient for tunnel detection. However, due to the curvature of the tunnel, there are variations in object distance that exceed the depth of field of the lens, resulting in uneven defocus blur in the captured images. This can significantly affect the accuracy of defect recognition. While existing deblurring algorithms can improve image quality, they often prioritize results over inference time, which is not ideal for high-speed tunnel image acquisition. To address this issue, we developed a lightweight tunnel structure defect deblurring network (TSDDNet) for curved-tunnel line scanning with wide-angle lenses. Our method employs an innovative progressive structure that balances network depth and feature breadth to simultaneously achieve good performance and short inference time. The proposed depthwise ResBlocks significantly improves the parameter efficiency of the network. Additionally, the proposed feature refinement block captures the structurally similar features to enhance the image details, increasing the peak signal-to-noise ratio (PSNR). A raw dataset containing tunnel blur images was created using a high-resolution line scan camera and used to train and test our model. TSDDNet achieved a PSNR of 26.82 dB and a structural similarity index measure of 0.888, while using one-third of the parameters of comparable alternatives. Moreover, our method exhibited a higher computational speed than that of conventional methods, with inference times of 8.82 ms for a single 512 × 512 pixel image patch and 227.22 ms for completely processing a 2048 × 2560 pixel image. The test results indicated that the structural scalability of the network allows it to accommodate large inputs, making it effective for high-resolution images.

Keywords

Image deblurring / Tunnel defect detection / Defocus deblurring / Convolutional neural networks / Massive image acquisition

CRediT authorship contribution statement

Shaojie Qin: Writing - review & editing, Writing - original draft, Visualization, Validation, Methodology. Taiyue Qi: Validation, Supervision, Funding acquisition. Xiaodong Huang: Visualization, Validation, Resources. Xiao Liang: Writing - review & editing, Supervision.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgement

This work was supported by the National Natural Science Foundation of China (Grant No. 51978582).

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

[1] Abuolaim, A., Afifi, M., & Brown, M. S. (2022). Improving single-image defocus deblurring: How dual-pixel images help through multi-task learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 1231-1239).

[2] Abuolaim, A., & Brown, M. S. (2020). Defocus deblurring using dualpixel data. In Proceedings of 16th European Conference, Glasgow, UK (pp.111-126).

[3] Abuolaim, A., Delbracio, M., Kelly, D., Brown, M. S., & Milanfar, P. (2021). Learning to reduce defocus blur by realistically modeling dualpixel data. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp.2289-2298).

[4] Attard, L., Debono, C. J., Valentino, G., & Di Castro, M. (2018). Visionbased change detection for inspection of tunnel liners. Automation in Construction, 91, 142-154.

[5] Attard, L., Debono, C. J., Valentino, G., & Di Castro, M. (2021). Visionbased tunnel lining health monitoring via bi-temporal image comparison and decision-level fusion of change maps. Sensors, 21(12), 4040.

[6] Chantas, G., Galatsanos, N., Molina, R., & Katsaggelos, A. (2010). Variational Bayesian inference image restoration using a product of total variation-like image priors. In Proceedings of the 2010 2nd International Workshop on Cognitive Information Processing (pp.227-231).

[7] Chantas, G. K., Galatsanos, N. P., & Likas, A. C. (2006). Bayesian restoration using a new nonstationary edge-preserving image prior. IEEE Transactions on Image Processing, 15(10), 2987-2997.

[8] Fan, Y. Q., Hong, C. Q., Wang, X. D., Zeng, Z. Q., & Guo, Z. T. (2021). Multi-input-output Fusion Attention Module for Deblurring Networks. In Proceedings of 2021 IEEE International Conference on Big Data (pp.3176-3182).

[9] Gao, H. Y., Yuan, H., Wang, Z. Y., & Ji, S. W. (2020). Pixel transposed convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(5), 1218-1227.

[10] Gong, Q. M., Zhu, L. Q., Wang, Y. D., & Yu, Z. J. (2021). Automatic subway tunnel crack detection system based on line scan camera, Struct. Structural Control and Health Monitoring, 28(8), e2776.

[11] Glenn Jjocher, G., Stoken, A.,Borovec, J., et al. (2021). yolov5. GitHub. yolov5. https://github.com/ultralytics/yolov5.

[12] Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. H. (2020). Squeeze-and- Excitation excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp.7132-7141).

[13] Huang, H. W., Li, Q. T., & Zhang, D. M. (2018). Deep learning based image recognition for crack and leakage defects of metro shield tunnel. Tunnelling and Underground Space Technology, 77, 166-176.

[14] Huang, Z., Fu, H. L., Fan, X. D., Meng, J. H., Chen, W., Zheng, X. J., Wang, F., & Zhang, J. B. (2021). Rapid surface damage detection equipment for subway tunnels based on machine vision system. Journal of Infrastructure Systems, 27(1), 04020047.

[15] Hyun Kim, T., Ahn, B., & Mu Lee, K. (2013). Dynamic scene deblurring. In Proceedings of 2013 IEEE International Conference on Computer Vision (pp.3160-3167).

[16] Jiang, Y. J., Zhang, X. P., & Taniguchi, T. (2019). Quantitative condition inspection and assessment of tunnel lining. Automation in Construction, 102, 258-269.

[17] Isola, P., Zhu, J. Y., Zhou, T. H., & Efros, A. A. (2017). Image-to-image translation with conditional adversarial networks. In Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (pp.1125-1134).

[18] Lai, W. S., Huang, J. B., Ahuja, N., & Yang, M. H. (2017). Deep Laplacian pyramid networks for fast and accurate super-resolution. In Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (pp.624-632).

[19] Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z. H., & Shi, W. Z. (2017). Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (pp.4681-4690).

[20] Lee, J. Y., Lee, S., Cho, S., & Lee, S. (2019). Deep defocus map estimation using domain adaptation. In Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp.12222-12230).

[21] Lee, J., Son, H., Rim, J., Cho, S., & Lee, S. (2021). Iterative filter adaptive network for single image defocus deblurring. In Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp.2034-2042).

[22] Li, D. W., Xie, Q., Gong, X. X., Yu, Z. H., Xu, J. X., Sun, Y. X., & Wang, J. (2021). Automatic defect detection of metro tunnel surfaces using a vision-based inspection system. Advanced Engineering Informatics, 47, 101206.

[23] Liang, C. H., Chen, Y. A., Liu, Y. C., & Hsu, W. H. (2022). Raw image deblurring. IEEE Transactions on Multimedia, 24, 61-72.

[24] Liao, J. H., Yue, Y. H., Zhang, D. J., Tu, W., Cao, R., Zou, Q., & Li, Q. Q. (2022). Automatic tunnel crack inspection using an efficient mobile imaging module and a lightweight CNN. IEEE Transactions on Intelligent Transportation Systems, 23(9), 15190-15203.

[25] Liu, Y. Q., Yeoh, J. K. W., & Chua, D. K. H. (2020). Deep learning-based enhancement of motion blurred UAV concrete crack images. Journal of Computing in Civil Engineering, 34(5), 04020028.

[26] Luan, F. J., Paris, S., Shechtman, E., & Bala, K. (2017). Deep photo style transfer. In Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (pp.4990-4998).

[27] Ma, M., Lu, W., & Lyu, W. J. (2020). Defocus blur detection via edge pixel DCT feature of local patches. Signal Processing, 176, 107670.

[28] Mao, X. J., Shen, C. H., & Yang, Y. B. (2016). Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In Proceedings of the 30th International Conference on Advances in Neural Information Processing Systems (pp.2810-2818).

[29] Molina, R., Núñez, J., Cortijo, F. J., & Mateos, J. (2001). Image restoration in astronomy: A Bayesian perspective. IEEE Signal Processing Magazine, 18(2), 11-29.

[30] Qin, S. J., Qi, T. Y., Deng, T., & Huang, X. D. (2024). Image segmentation using Vision Transformer for tunnel defect assessment. Computer-Aided Civil and Infrastructure Engineering, 1-26.

[31] Qin, S. J., Qi, T. Y., Lei, B., & Huang, X. D. (2023). Batched-image detection model and deployment method for tunnel lining defects using line-scan cameras based on experiments study. Tunnelling and Underground Space Technology, 142, 105428.

[32] Quan, Y. H., Wu, Z. C., & Ji, H. (2021). Gaussian kernel mixture network for single image defocus deblurring. Advances in Neural Information Processing Systems, 34, 20812-20824.

[33] Quan, Y. H., Wu, Z. C., & Ji, H. (2023). Neumann network with recursive kernels for single image defocus deblurring. In Proceedings of 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp.5754-5763).

[34] Richardson, W. H. (1972). Bayesian-based iterative method of image restoration. Journal of the Optical Society of America, 62(1), 55-59.

[35] Ruan, L. Y., Chen, B., Li, J. Z., & Lam, M. L. (2021). AIFNet: All-infocus image restoration network using a light field-based dataset. IEEE Transactions on Computational Imaging, 7, 675-688.

[36] Shi, J. P., Xu, L., & Jia, J. Y. (2015). Just noticeable defocus blur detection and estimation. In Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. (pp.657-665).

[37] Shi, J. P., Xu, L., & Jia, J. Y. (2014). Discriminative blur detection features. In Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition (pp.2965-2972).

[38] Sobel, I. (1990). An isotropic 3×3 image gradient operator. In Machine vision for three-dimensional scenes (pp. 376-379). Academic Press.

[39] Son, H., Cho, S. Lee, J. Y., Cho, S., & Lee, S. (2021). Single image defocus deblurring using kernel-sharing parallel atrous convolutions. In Proceedings of 2021 IEEE/CVF International Conference on Computer Vision (pp.2642-2650).

[40] Su, J. W., Xu, B. Y., & Yin, H. J. (2022). A survey of deep learning approaches to image restoration. Neurocomputing, 487, 46-65.

[41] Tai, Y., Yang, J., & Liu, X. M. (2017). Image super-resolution via deep recursive residual network. In Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (pp.3147-3155).

[42] Teng, Y. D., & Gao, P. X. (2021). Generative robotic grasping using depthwise separable convolution. Computers & Electrical Engineering, 94, 107318.

[43] Tomasi, C., & Manduchi, R. (1998). Bilateral filtering for gray and color images. In Proceedings of Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271) (pp. 839-846).

[44] Tong, T., Li, G., Liu, X. J., & Gao, Q. Q. (2017). Image super-resolution using dense skip connections. In Proceedings of 2017 IEEE International Conference on Computer Vision (pp.4799-4807).

[45] Wang, L. L., & Li, J. J. (2021). Fast blur detection algorithm for UAV crack image sets. Journal of Computing in Civil Engineering, 35(6).

[46] Wang, M. Z., & Zhang, Q. L. (2020). Optimized data storage algorithm of IoT based on cloud computing in distributed system. Computer Communications, 157(2020), 124-131.

[47] Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600-612.

[48] Xu, G. D., Quan, Y. H., & Ji, H. (2017). Estimating defocus blur via rank of local patches. In Proceedings of 2017 IEEE International Conference on Computer Vision (pp.5371-5379).

[49] Yang, J., & Feng, Q. B. (2013). A new method for measuring subgrade settlement in high-speed railway by using a linear CCD. Measurement, 46(5), 1751-1756.

[50] Yu, J., Tan, M., Zhang, H. Y., Rui, Y., & Tao, D. C. (2022). Hierarchical deep click feature prediction for fine-grained image recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(2), 563-578.

[51] Zamir, S.W., Arora, A., Khan, S. H., Hayat, M., Khan, F., Yang, M. H., & Shao, L. (2020). Learning enriched features for real image restoration and enhancement. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXV 16 (pp. 492-511). Springer International Publishing.

[52] Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F., & Yang, M. H. (2022). Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 5728-5739).

[53] Zhang, S. H., Shen, X. H., Lin, Z., Mech, R., Costeira, J. P., & Moura, J. M. F. (2018). Learning to understand image blur. In Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp.6586-6595).

[54] Zhang, Y. L., Tian, Y. P., Kong, Y., Zhong, B. N., & Fu, Y. (2021). Residual dense network for image restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43, 2480-2495.

[55] Zhao, S., Zhang, D. M., Xue, Y. D., Zhou, M. L., & Huang, H. W. (2021). A deep learning-based approach for refined crack evaluation from shield tunnel lining images. Automation in Construction, 132, 103934.

[56] Zhao, W. D., Zhao, F., Wang, D., & Lu, H. C. (2018). Defocus blur detection via multi-stream bottom-top-bottom fully convolutional network. In Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp.3080-3088).

[57] Zhou, Z., Zhang, J. J., & Gong, C. J. (2022). Automatic detection method of tunnel lining multi-defects via an enhanced You Only Look Once network. Computer-Aided Civil and Infrastructure Engineering, 37(6), 762-780.