Application of self-supervised learning in steel surface defect detection

Abstract

In scientific research, effective utilization of unlabeled data has become pivotal, as exemplified by AlphaFold2, which won the 2024 Nobel Prize. Pioneering this paradigm shift, we develop a universal self-supervised learning methodology for detecting surface defects in steel materials. By harnessing unlabeled data, our approach significantly reduces the dependence for manual annotation and enhances scalability while training robust models capable of generalizing across defect types. Using a Faster R-CNN framework, we achieved a mean average precision (mAP) of 0.385 and a mAP at IoU = 0.5 (mAP_50) of 0.768 on the NEU-DET steel defects dataset. These results demonstrate both the efficacy of our self-supervised strategy and its potential as a framework for developing image detection systems with minimal labeled data requirements in surface defect identification.

Keywords

Unlabelled data / self-supervised learning / deep learning / steel materials / image detection

References

[1] Ren B,Zhang Y,Xu W.Industrial big data analysis strategy based on automatic data classification and interpretable knowledge graph.J Mater Inf2025;5:2

[2] Zhou C,Lv Z.Metal surface defect detection based on improved YOLOv5.Sci Rep2023;13:20803

[3] Park JK,Park JH.Machine learning-based imaging system for surface defect inspection.Int J Precis Eng Manuf Green Tech2016;3:303-10

[4] Han S,Zhang Y,Di H.Employing deep learning in non-parametric inverse visualization of elastic–plastic mechanisms in dual-phase steels.Mater Genome Eng Adv2024;2:e29

[5] Qiao Z,Yi X,Zhang Y.UEFPN: unified and enhanced feature pyramid networks for small object detection.ACM Trans Multimed Comput Commun Appl2023;19:1-21

[6] Liu Y,Wergeles N.A survey and performance evaluation of deep learning methods for small object detection.Expert Syst Appl2021;172:114602

[7] Zhu X,Zhang B,Yu J.An improved feature enhancement CenterNet model for small object defect detection on metal surfaces.Adv Theory Simul2024;7:2301230

[8] Yu J,Li Q.Surface defect detection of steel strips based on anchor-free network with channel attention and bidirectional feature fusion.IEEE Trans Instrum Meas2022;71,1-10

[9] He Q,Yang W.LMFE-RDD: a road damage detector with a lightweight multi-feature extraction network.Multimed Syst2024;30:176

[10] Sun L,Liang K,Zeng W.An intelligent system for high-density small target pest identification and infestation level determination based on an improved YOLOv5 model.Expert Syst Appl2024;239:122190

[11] Yuan Y,Zhao L,Zhang Y.Multiple object detection and tracking from drone videos based on GM-YOLO and multi-tracker.Image Vis Comput2024;143:104951

[12] Zhu Y,Yan J,Yang G.NATCA YOLO-based small object detection for aerial images.Information2024;15:414

[13] Wang Y,Yuan X,Sun B.Distributed defect recognition on steel surfaces using an improved random forest algorithm with optimal multi-feature-set fusion.Multimed Tools Appl2017;77:16741-70

[14] Wang H,Wan Z.Rail surface defect detection based on improved Mask R-CNN.Comput Electr Eng2022;102:108269

[15] Liu LJ,Karimi HR.Resilient machine learning for steel surface defect detection based on lightweight convolution.Int J Adv Manuf Technol2024;134:4639-50

[16] Shi X,Tai Y,Wu S.An improved faster R-CNN for steel surface defect detection. In 2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP), Shanghai, China. Sep 26-28, 2022. IEEE; 2022. p. 1-5.

[17] Akhyar F,Hsu CY,Lin CY.FDD: a deep learning-based steel defect detectors.Int J Adv Manuf Technol2023;126:1093-107

[18] Hong Y,Wu W.Steel surface defect detection based on denoising diffusion implicit models with data augmentation. In 2024 8th International Conference on Imaging, Signal Processing and Communications (ICISPC), Fukuoka, Japan. Jul 19-21, 2024. IEEE; 2024. pp. 15-9.

[19] Kadam S.Advancements in image detection: a comprehensive approach to object localization and classification using deep learning techniques.Int J Multidiscip Res2024;6:27133

[20] Fang Z,Xu J,Paul B.A novel machine learning method to investigate the web crippling behaviour of perforated roll-formed aluminium alloy unlipped channels under interior-two flange loading.J Build Eng2022;51:104261

[21] Balestriero R,Sobal V. A cookbook of self-supervised learning. arXiv 2023, arXiv:2304.12210. https://doi.org/10.48550/arXiv.2304.12210. (accessed 10 Jul 2025)

[22] Wang Y,Zong H.Self-supervised probabilistic models for exploring shape memory alloys.npj Comput Mater2024;10:185

[23] Magar R,Barati Farimani A.Crystal twins: self-supervised learning for crystalline material property prediction.npj Comput Mater2022;8:231

[24] Fu N,Hu J.Physics-guided dual self-supervised learning for structure-based material property prediction.J Phys Chem Lett2024;15:2841-50

[25] Zhang S,Wang X.Large language models enabled intelligent microstructure optimization and defects classification of welded titanium alloys.J Mater Inf2024;4:34

[26] Kim S.Effect of surface and internal defects on the mechanical properties of metallic glasses.Sci Rep2017;7:13472

[27] Masci J,Ciresan D,Fricout G.Steel defect classification with max-pooling convolutional neural networks. In The 2012 International Joint Conference on Neural Networks (IJCNN), Brisbane, Australia. Jun 10-15, 2012. IEEE; 2012. p. 1-6.

[28] Tian R.DCC-CenterNet: a rapid detection method for steel surface defects.Measurement2022;187:110211

[29] Zabin M,Kabir MK,Uddin J.Contrastive self-supervised representation learning framework for metal surface defect detection.J Big Data2023;10:145

[30] Xu R,Huang B.Efficient surface defect detection using self-supervised learning strategy and segmentation network.Adv Eng Inform2022;52:101566

[31] Zhang S,Gu J,Li K.Visual inspection of steel surface defects based on domain adaptation and adaptive convolutional neural network.Mech Syst Signal Process2021;153:107541

[32] Sirotkin K,Carballeira P.Improved transferability of self-supervised learning models through batch normalization finetuning.Appl Intell2024;54:11281-94

[33] Geng X,Wu HH.Data-driven and artificial intelligence accelerated steel material research and intelligent manufacturing technology.Mater Genome Engi Adv2023;1:e10

[34] Bachman P,Buchwalter W. Learning representations by maximizing mutual information across views. arXiv 2019, arXiv:1906.00910. https://doi.org/10.48550/arXiv.1906.00910. (accessed 10 Jul 2025)

[35] Pöppelbaum J,Schwung A.Contrastive learning based self-supervised time-series analysis.Appl Soft Comput2022;117:108397

[36] He K,Wu Y,Girshick R.Momentum contrast for unsupervised visual representation learning. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, USA. Jun 13-19, 2020. IEEE; 2020. pp. 9726-35.

[37] Chen T,Norouzi M. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning. PMLR; 2020. pp. 1597-607. https://proceedings.mlr.press/v119/chen20j.html. (accessed 10 Jul 2025)

[38] Chen X.Exploring simple siamese representation learning. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashiville, USA. Jun 20-25, 2021. IEEE; 2021. pp. 15745-53.

[39] Song K.A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects.Appl Surf Sci2013;285:858-64

[40] Gui J,Zhang J,Sun Z.A survey on self-supervised learning: algorithms, applications, and future trends.IEEE Trans Pattern Anal Mach Intell2024;46:9052-71

[41] Zhao Z,Zhang J,Gu Y.A comparison review of transfer learning and self-supervised learning: definitions, applications, advantages and limitations.Expert Syst Appl2024;242:122807

[42] Huang J,Sun C. Speed/accuracy trade-offs for modern convolutional object detectors. arXiv 2016, arXiv:1611.10012. https://doi.org/10.48550/arXiv.1611.10012. (accessed 10 Jul 2025)

[43] Lin TY,Girshick R,Hariharan B.Feature pyramid networks for object detection. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, USA. Jul 21-26, 2017. IEEE; 2017. pp. 936-44.

[44] Lin TY,Belongie S. Microsoft COCO: common objects in context. arXiv 2014, arXiv:1405.0312. https://doi.org/10.48550/arXiv.1405.0312. (accessed 10 Jul 2025)

[45] Li Z,Jiang X.SSDD-Net: a lightweight and efficient deep learning model for steel surface defect detection. In Pattern Recognition and Computer Vision: 6th Chinese Conference, PRCV 2023, Xiamen, China. Oct 13-15, 2023. Springer-Verlag; 2023. pp. 237-48.

[46] Li M,Zheng B.Steel surface defect detection based on improved YOLOv7. In 2024 4th International Conference on Computer, Control and Robotics (ICCCR), Shanghai, China. Apr 19-21, 2024. IEEE; 2024. pp. 51-5.