i
This article was retracted on
Although semantic segmentation of daytime images has achieved remarkable success, nighttime semantic segmentation still faces significant challenges due to the low brightness and contrast in nighttime images. To address these challenges, we proposed an enhanced Swin Transformer for nighttime semantic segmentation (EST- NSS) for nighttime urban streetscape segmentation. The EST-NSS utilizes Swin Transformer as the backbone network. In the image preprocessing stage, we designed a nighttime light enhancer (NLE), which integrates receptive-field attention convolution (RFAConv), attention mechanisms, and a Transformer to enhance image brightness, contrast, and semantic representation, thereby improving the model’ s perception of key regions. In the feature processing stage, we designed a dual-domain context unit (termed SCBlock), which integrates attention mechanisms and depthwise separable convolution to enhance multi-scale feature representations, thereby improving the model ’ s accuracy in segmenting object boundaries. Extensive experimental results on the NightCity dataset show that our EST-NSS significantly enhances the performance of nighttime semantic segmentation. The mean intersection over union (mIoU) improved by 3. 6 percentage points compared with Swin Transformer. Compared with the state-of-the-art models, the EST-NSS achieved superior performance in 15 out of 19 categories.
CHANGE HISTORY
A Correction to this paper has been published:
https://doi.org
The Retraction Note was updated to reflect reception of statements of disagreement from all authors.
This article has been retracted.Please see the Retraction Notice for more detail:
https://doi.org
| [1] |
Yang L N, Liang X D, Wang T R, et al. Real-to-virtual domain unification for end-to-end autonomous driving[C]// European Conference on Computer Vision. Cham: Springer, 2018: 553-570.
|
| [2] |
Chen X, Williams B M, Vallabhaneni S R, et al. Learning active contour models for medical image segmentation[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 11624-11632.
|
| [3] |
Pang B, Liu G H. SGFNet: an attention- enhanced network for colonic polyp segmentation in endoscopic images[J]. Journal of Donghua University (English Edition), 2026, 43 (3): 101-112.
|
| [4] |
Cordts M, Omran M, Ramos S, et al. The Cityscapes dataset for semantic urban scene understanding[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 3213-3223.
|
| [5] |
Geiger A, Lenz P, Urtasun R. Are we ready for autonomous driving? The KITTI vision benchmark suite[C]// 2012 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2012: 3354-3361.
|
| [6] |
Liu W Y, Li W T, Zhu J K, et al. Improving nighttime driving-scene segmentation via dual image-adaptive learnable filters[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(10): 5855-5867.
|
| [7] |
Wu X Y, Wu Z Y, Ju L L, et al. A one-stage domain adaptation network with image alignment for unsupervised nighttime semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(1): 58-72.
|
| [8] |
Dai D X, Van Gool L. Dark model adaptation: semantic image segmentation from daytime to nighttime[C]// 2018 21st International Conference on Intelligent Transportation Systems. Piscataway: IEEE, 2018: 3819-3824.
|
| [9] |
Shen F Y, Pataki Z, Gurram A, et al. LoopDA: constructing self-loops to adapt nighttime semantic segmentation[C]// 2023 IEEE/CVF Winter Conference on Applications of Computer Vision. Piscataway: IEEE, 2023: 3255-3265.
|
| [10] |
Wu X Y, Wu Z Y, Guo H, et al. DANNet: A one-stage domain adaptation network for unsupervised nighttime semantic segmentation[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 15764-15773.
|
| [11] |
Schölkopf B, Platt J, Hofmann T. Analysis of Representations for domain adaptation[C]// Advances in Neural Information Processing Systems 19: Proceedings of the 2006 Conference. Cambridge: The MIT Press, 2007: 137-144.
|
| [12] |
Ganin Y, Lempitsky V. Unsupervised domain adaptation by backpropagation[C]// Proceedings of the 32nd International Conference on Machine Learning (ICML).[S. l.]: JMLR. org, 2015: 1180-1189.
|
| [13] |
Sakaridis C, Dai D X, Van Gool L. Map-guided curriculum domain adaptation and uncertainty- aware evaluation for semantic nighttime image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44 (6): 3139-3153.
|
| [14] |
Romera E, Bergasa L M, Yang K L, et al. Bridging the day and night domain gap for semantic segmentation[C]// 2019 IEEE Intelligent Vehicles Symposium. (IV). Piscataway: IEEE, 2019: 1312-1318.
|
| [15] |
Sun L, Wang K W, Yang K L, et al. See clearer at night: towards robust nighttime semantic segmentation through day-night image conversion[PP/OL]. arXiv (2019-08-16)[2025-02-01]. https://arxiv.org/abs/1908.05868#1.
|
| [16] |
Xie Z F, Qiu R, Wang S, et al. PIG: prompt images guidance for night-time scene parsing[J]. IEEE Transactions on Image Processing, 2024, 33: 3921-3934.
|
| [17] |
Tan X, Xu K, Cao Y, et al. Night-time scene parsing with a large real dataset[J]. IEEE Transactions on Image Processing, 2021, 30: 9085-9098.
|
| [18] |
Liu Z, Lin Y T, Cao Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]// 2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2021: 9992-10002.
|
| [19] |
Zhang X, Liu C, Yang D G, et al. RFAConv: innovating spatial attention and standard convolutional operation[PP/OL]. arXiv (2024- 04-06)[2025-04-21]. https://arxiv. org/abs/2304. 03198.
|
| [20] |
Woo S, Park J, Lee J Y, et al. CBAM: convolutional block attention module[C]// European Conference on Computer Vision. Cham: Springer, 2018: 3-19.
|
| [21] |
Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]// Advances in Neural Information Processing Systems. Cambridge: The MIT Press, 2017, 30: 5998-6008.
|
| [22] |
Howard A G, Zhu M, Chen B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[PP/OL]. arXiv (2017-04-17)[2025-04-21]. https://arxiv.org/abs/1704.04861.
|
| [23] |
Shrivastava A, Gupta A, Girshick R. Training region-based object detectors with online hard example mining[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2016: 761-769.
|
| [24] |
Chen L C, Papandreou G, Kokkinos I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834-848.
|
| [25] |
Lin G S, Milan A, Shen C H, et al. RefineNet: multi-path refinement networks for high- resolution semantic segmentation[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 5168-5177.
|
| [26] |
Cheng B, Schwing A, Kirillov A. Per-pixel classification is not all you need for semantic segmentation[C]// Advances in Neural Information Processing Systems 34 (NeurIPS 2021). Red Hook: Curran Associates, Inc., 2021, 34: 17864-17875.
|
| [27] |
Cheng B, Misra I, Schwing A G, et al. Masked- attention mask transformer for universal image segmentation[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 1290-1299.
|
| [28] |
Xie E, Wang W, Yu Z, et al. SegFormer: simple and efficient design for semantic segmentation with transformers[C]// Advances in Neural Information Processing Systems 34 (NeurIPS 2021), 2021, 34: 12077-12090.
|
| [29] |
Sakaridis C, Dai D X, Van Gool L. Guided curriculum model adaptation and uncertainty-aware evaluation for semantic nighttime image segmentation[C]// 2019 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2019: 7373-7382.
|
| [30] |
Lee H, Han C, Yoo J S, et al. GPS-GLASS: learning nighttime semantic segmentation using daytime video and GPS data[C]// 2023 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2023: 4003-4012.
|
| [31] |
Xia R H, Zhao C Q, Zheng M, et al. CMDA: cross-modality domain adaptation for nighttime semantic segmentation[C]// 2023 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2023: 21515-21524.
|
| [32] |
Sakaridis C, Dai D X, Van Gool L. ACDC: the adverse conditions dataset with correspondences for semantic driving scene understanding[C]/ 2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2021: 10745-10755.
|
| [33] |
Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16 × 16 words: Transformers for image recognition at scale[PP/OL]. arXiv (2020-10-22)[2025-04-21]. https://arxiv. org/abs/2010. 11929.
|
| [34] |
Chen H T, Wang Y H, Guo T Y, et al. Pre- trained image processing transformer[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 12294-12305.
|
| [35] |
Wolf T, Debut L, Sanh V, et al. Transformers: state-of-the-art natural language processing[C/OL]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Stroudsburg: Association for Computational Linguistics, 2020: 38-45.
|
| [36] |
Touvron H, Cord M, Douze M, et al. Training data-efficient image transformers & distillation through attention[C]// Proceedings of the 38th International Conference on Machine Learning.[S. l.]: PMLR, 2021, 139: 10347-10357.
|
| [37] |
Zhao H S, Shi J P, Qi X J, et al. Pyramid scene parsing network[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 6230-6239.
|
| [38] |
Hoyer L S, Dai D X, Van Gool L. DAFormer: improving network architectures and training strategies for domain-adaptive semantic segmentation[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 9914-9925.
|
| [39] |
Hoyer L S, Dai D X, Van Gool L, et al. HRDA: context-aware high-resolution domain- adaptive semantic segmentation[C]// European Conference on Computer Vision. Cham: Springer, 2022: 372-391.
|
Funding
Fundamental Research Funds for the Central Universities, China(2232025H-03)