Real-Time Transformer for Robust Ship Detection under Adverse Weather

Abstract

Maritime situational awareness relies on accurate vessel detection; however, vision-based systems often face severe performance degradation under adverse weather conditions such as fog, nighttime, and glare. In this study, we proposed an improved real-time detection transformer (RT-DETR) tailored for maritime environments to enhance robust detection across diverse weather scenarios. First, we constructed a novel “Ship Weather” image dataset (2 054 images) with fine-grained annotations of ships (Cargo, Tanker, Unknown, and Crane Barge) captured under four representative weather conditions: overcast, fog, nighttime, and glare. This dataset filled a gap in existing benchmarks by systematically covering close-range camera images in poor visibility conditions. Second, we built on the RT-DETR architecture, which is a recent end-to-end transformer detector that eliminates non-maximum suppression and achieves high speed and accuracy. We integrated several advanced modules to improve feature extraction and localization under weather-induced distortions. Specifically, we incorporated Token statistic self-attention (TSSA) in the transformer encoder-decoder to enable linear-complexity global attention and better distinguish targets in noisy backgrounds, omni-dimensional dynamic convolution (ODConv) to adaptively adjust convolutional weights for weather-degraded features, and the Wise-IoU (WIoU) loss to focus regression on ordinary quality predictions for more stable bounding boxes. The proposed model achieved an mAP@50 of 98.8% on the test set, maintaining high accuracy even under dense fog and low-light scenarios, whereas prior models suffered substantial drops. Ablation studies confirmed the contribution of each proposed component (TSSA, ODConv, WIoU) to enhanced robustness. These results highlighted the effectiveness of optimizing the detector’s architecture and learning mechanism, rather than relying solely on image enhancement or sensor fusion, in achieving reliable ship detection for autonomous navigation in challenging maritime environments.

Keywords

Transformer / Adverse weather / Small-target detection / Feature ambiguity

References

[1] Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A, Zagoruyko S. End-to-end object detection with transformers. ECCV 2020: European Conference on Computer Vision, 2020, Cham: Springer International Publishing, 213-229.

[2] Gallego AJ, Pertusa A, Gil P. Automatic ship clas. sification from optical aerial images with convolutional neural networks. Remote Sensing, 2018, 104511.

[3] GitHub, Inc.Ultralytics YOLO (Version 8.0.0), 2023

[4] Hoehner F, Langenohl V, Akyol S, El Moctar O, Schellin TE. Object detection and tracking in maritime environments in case of person-overboard scenarios: an overview. Journal of Marine Science and Engineering, 2024, 12112038.

[5] Huang L, Liu B, Li B, Guo W, Yu W, Zhang Z, Yu W. OpenSARShip: a dataset dedicated to Sentinel-1 ship interpretation. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 111195208.

[6] Kiefer B, Zust L, Muhovic J, Kristan M, Pers J, Tersek M. 3rd workshop on maritime computer vision (MaCVi) 2025: challenge results. 2025 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops(WACVW), 2025, Piscataway: IEEE, 1457-1484.

[7] Lam D, Kuzma R, McGee K, Dooley S, Laielli M, Klaric M, Bulatov Y, Mccord B (2018) xView: objects in context in overhead imagery.V1. arXiv preprint arXiv:1802.07856. https://doi.org/10.48550/arXiv.1802.07856

[8] Li W, Li A, Li Z, Kong X, Zhang Y. RTS-DETR: efficient real-time DETR for small object detection. 2024 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2024, Piscataway: IEEE, 1211-1216.

[9] Li S, Cao X, Zhou Z. Research on inshore ship detection under nighttime low-visibility environment for maritime surveillance. Computers and Electrical Engineering, 2024, 118: 109310.

[10] Liang S, Liu X, Yang Z, Liu M, Yin Y. Offshore ship detection in foggy weather based on improved YOLOv8. Journal of Marine Science and Engineering, 2024, 1291641.

[11] Liu Z, Wang H, Weng L, Yang Y. Ship rotated bounding box space for ship extraction from high-resolution optical satellite images with complex backgrounds. IEEE Geoscience and Remote Sensing Letters, 2016, 13810741078.

[12] Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC. SSD: single shot multibox detector. Computer Vision-ECCV 2016: 14th European Conference, 2016, Cham: Springer, 21-37. 9905

[13] Liu S, Sun J, Yao P, Zhu Y, Mao T, Wang Z. DTDNet: dynamic target driven network for pedestrian trajectory prediction. Frontiers in Neuroscience, 2024, 18: 1346374.

[14] Öztürk OB, Kartal E, Aydin M. Investigating the influence of human errors in master-pilot information exchange on maritime accident risk during pilotage. Ocean Engineering, 2024, 305: 118051.

[15] Ren S, He K, Girshick R, Sun J. Faster R-CNN: towards realtime object detection with region proposal networks. Advances in Neural Information Processing Systems, 2015, 28: 91-99

[16] Shao Z, Yin Y, Lyu H, Soares CG, Jing Q, Yang Z. An efficient model for small object detection in the maritime environment. Applied Ocean Research, 2024, 152: 104194.

[17] Symes S, Blanco-Davis E, Graham T, Wang J, Shaw E. Cyberattacks on the maritime sector: a literature review. Journal of Marine Science and Application, 2024, 234689706.

[18] Tian Y, Ye Q, Doermann D (2025) YOLOV12: attention-centric realtime object detectors. V1. arXiv preprint arXiv: 2502.12524. (2025-02-18)[2025-07-28].https://doi.org/10.48550/arXiv.2502.12524

[19] Wei S, Zeng X, Qu Q, Wang M, Su H, Shi J. HRSID: A highresolution SAR images dataset for ship detection and instance segmentation. IEEE Access, 2020, 8: 120234-120254.

[20] Wu Z, Ding T, Lu Y, Pai D, Zhang J, Wang W, Yu Y, Ma Y, Haeffele BD (2024) Token statistics transformer: linear-time attention via variational rate reduction. V1. arXiv preprint arXiv: 2412.17810.(2024-12-23)[2025-07-28]. https://doi.org/10.48550/arXiv.2412.17810

[21] Xia GS, Bai X, Ding J, Zhu Z, Belongie S, Luo J, Datcu M, Pelillo M, Zhang L. DOTA: a large-scale dataset for object detection in aerial images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, Piscataway: IEEE, 3974-3983

[22] Yaroshko T. Airbus ship detection challenge, 2018

[23] Yao S, Guan R, Peng Z, Xu C, Xu C, Yue Y, Lim E, Seo H, Man K, Zhu X, Yue Y. Exploring radar data representations in autonomous driving: a comprehensive review. IEEE Transactions on Intelligent Transportation Systems, 2025, 26674017425.

[24] Zhao Y, Lv W, Xu S, Wei J, Wang G, Sang Q, Liu Y, Chen J. DETRs beat YOLOs on real-time object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern, 2024, Piscataway: IEEE, 16965-16974

[25] Zhang Y, Li QZ, Zang FN. Ship detection for visual maritime surveillance from non-stationary platforms. Ocean Engineering, 2017, 141: 53-63.

[26] Zhang T, Zhang X, Li J, Xu X, Wang B, Zhan X, Xu Y, Ke X, Zeng T, Su H, Ahmad I, Pan D, Liu C, Zhou Y, Shi J, Wei S. SAR ship detection dataset (SSDD): official release and comprehensive data analysis. Remote Sensing, 2021, 13183690.

[27] Zhang B, Lv M, Wang Z, Liu X, Wang W. Robust sea-sky line detection in complex maritime environments via semantic segmentation. IEEE Sensors Journal, 2025, 2581445314464.

[28] Zhong F, Shen W, Yu H, Wang G, Hu J. Dehazing & reasoning YOLO: prior knowledge-guided network for object detection in foggy weather. Pattern Recognition, 2024, 156: 110756.

[29] Zhu JY, Park T, Isola P, Efros AA. Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE international conference on computer vision, 2017, Piscataway: IEEE, 2242-2251

Rights & permissions

Harbin Engineering University and Springer-Verlag GmbH Germany, part of Springer Nature