Enhancing nighttime vehicle detection with day-to-night style transfer and labeling-free augmentation

i

This article has been updated. A Correction to this article was published on .

i

The Original Article was published on

i

The Original Article was published on

i

This article has been updated

i

This article was retracted on

i

This article has been updated

Abstract

Deep learning-based object detection models perform well under daytime conditions but face significant challenges at night, primarily because they are predominantly trained on daytime images. Additionally, training with nighttime images presents another challenge: Even human annotators struggle to accurately label objects in low-light conditions. This issue is particularly pronounced in transportation applications, such as detecting vehicles and other objects of interest on rural roads at night, where street lighting is often absent, and headlights may introduce undesirable glare. In this study, we addressed these challenges by introducing a novel framework for labeling-free data augmentation, leveraging synthetic data generated by the Car Learning to Act (CARLA) simulator for day-to-night image style transfer. Specifically, the framework incorporated the efficient attention Generative Adversarial Network for realistic day-to-night style transfer and used CARLA-generated synthetic nighttime images to help the model learn the vehicle headlight effect. To evaluate the efficacy of the proposed framework, we fine-tuned the state-of-the-art object detection model with an augmented dataset curated for rural nighttime environments, achieving significant improvements in nighttime vehicle detection. This novel approach was simple yet effective, offering a scalable solution to enhance deep learning-based detection systems in low-visibility environments and extended the applicability of object detection models to broader real-world contexts.

Keywords

nighttime vehicle detection / low visibility / headlight glare / generative adversarial networks / image style transfer / rural environments / CARLA / synthetic data / data augmentation

CHANGE HISTORY

A Correction to this paper has been published:
https://doi.org

The Retraction Note was updated to reflect reception of statements of disagreement from all authors.

This article has been retracted.Please see the Retraction Notice for more detail:
https://doi.org

References

[1] National Highway Traffic Safety Administration, Daytime and nighttime seat belt use by fatally injured passenger vehicle occupants, U.S. Department of Transportation, 2010. Available from: https://www.nhtsa.gov/sites/nhtsa.gov/files/documents/811281.pdf.

[2] National Highway Traffic Safety Administration, Report to congress: nighttime glare and driving performance, U.S. Department of Transportation, 2007. Available from: https://www.nhtsa.gov/sites/nhtsa.gov/files/glare_congressional_report.pdf.

[3] U. Mittal, P. Chawla, R. Tiwari, Ensemblenet: a hybrid approach for vehicle detection and estimation of traffic density based on faster r-cnn and yolo models, Neural Comput. Applic., 35 (2023), 4755-4774. https://doi.org/10.1007/s00521-022-07940-9

[4] M. Bie, Y. Liu, G. Li, J. Hong, J. Li, Real-time vehicle detection algorithm based on a lightweight you-only-look-once (yolov5n-l) approach, Expert Syst. Appl., 213 (2023), 119108. https://doi.org/10.1016/j.eswa.2022.119108

[5] S. Huang, C. Lin, S. Chen, Y. Wu, P. Hsu, S. Lai, Auggan: cross domain adaptation with gan-based data augmentation, Proceedings of the European Conference on Computer Vision (ECCV), 2018, 718-731.

[6] G. Bang, J. Lee, Y. Endo, T. Nishimori, K. Nakao, S. Kamijo, Semantic and geometric-aware day-to-night image translation network, Sensors, 24 (2024), 1339. https://doi.org/10.3390/s24041339

[7] X. Shao, C. Wei, Y. Shen, Z. Wang, Feature enhancement based on cyclegan for nighttime vehicle detection, IEEE Access, 9 (2020), 849-859. https://doi.org/10.1109/ACCESS.2020.3046498

[8] J. Zhu, T. Park, P. Isola, A. Efros, Unpaired image-to-image translation using cycle-consistent adversarial networks, Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, 2223-2232.

[9] H. Xu, S. Lai, X. Li, Y. Yang, Cross-domain car detection model with integrated convolutional block attention mechanism, Image Vision Comput., 140 (2023), 104834. https://doi.org/10.1016/j.imavis.2023.104834

[10] L. Fu, H. Yu, F. Xu, J. Li, Q. Guo, S. Wang, Let there be light: improved traffic surveillance via detail preserving night-to-day transfer, IEEE Trans. Circ. Syst. Vid., 32 (2022), 8217-8226. https://doi.org/10.1109/TCSVT.2021.3081999

[11] F. Guo, J. Liu, Q. Xie, H. Chang, Improved nighttime traffic detection using day-to-night image transfer, Transport. Res. Rec., 2677 (2023), 711-721. https://doi.org/10.1177/03611981231166686

[12] B. Kawar, S. Zada, O. Lang, O. Tov, H. Chang, T. Dekel, et al., Imagic: text-based real image editing with diffusion models, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, 6007-6017.

[13] Q. Zou, H. Ling, S. Luo, Y. Huang, M. Tian, Robust nighttime vehicle detection by tracking and grouping headlights, IEEE Trans. Intell. Transp., 16 (2015), 2838-2849. https://doi.org/10.1109/TITS.2015.2425229

[14] S. Parvin, L. Rozario, M. Islam, Vision-based on-road nighttime vehicle detection and tracking using taillight and headlight features, Journal of Computer and Communications, 9 (2021), 107619. https://doi.org/10.4236/jcc.2021.93003

[15] A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, V. Koltun, Carla: an open urban driving simulator, Proceedings of the 1st Annual Conference on Robot Learning, 2017, 1-16.

[16] R. Khanam, M. Hussain, YOLOv11: an overview of the key architectural enhancements, arXiv: 2410.17725. https://doi.org/10.48550/arXiv.2410.17725

[17] S. Malik, M. Khan, H. El-Sayed, Carla: car learning to act-an inside out, Procedia Computer Science, 198 (2022), 742-749. https://doi.org/10.1016/j.procs.2021.12.316

[18] J. Zhu, T. Park, T. Wang, Efficient-attention-GAN: unsupervised image-to-image translation with shared efficient attention mechanism, GitHub, Inc., 2023. Available from: https://github.com/jccb15/efficient-attention-GAN.

[19] X. Mao, Q. Li, H. Xie, R. Lau, Z. Wang, S. Smolley, Least squares generative adversarial networks, Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, 2794-2802.

[20] Y. Taigman, A. Polyak, L. Wolf, Unsupervised cross-domain image generation, arXiv: 1611.02200. https://doi.org/10.48550/arXiv.1611.02200

[21] T. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, et al., Microsoft coco: common objects in context, In: Computer vision-ECCV 2014, Cham: Springer, 2014, 740-755. https://doi.org/10.1007/978-3-319-10602-148

[22] K. You, M. Long, J. Wang, M. Jordan, How does learning rate decay help modern neural networks? arXiv: 1908.01878. https://doi.org/10.48550/arXiv.1908.01878