VMMAO-YOLO: an ultra-lightweight and scale-aware detector for real-time defect detection of avionics thermistor wire solder joints

Abstract

The quality of the exposed avionics solder joints has a significant impact on the stable operation of the in-orbit spacecrafts. Nevertheless, the previously reported inspection methods for multi-scale solder joint defects generally suffer low accuracy and slow detection speed. Herein, a novel real-time detector VMMAO-YOLO is demonstrated based on variable multi-scale concurrency and multi-depth aggregation network (VMMANet) backbone and “one-stop” global information gather-distribute (OS-GD) module. Combined with infrared thermography technology, it can achieve fast and high-precision detection of both internal and external solder joint defects. Specifically, VMMANet is designed for efficient multi-scale feature extraction, which mainly comprises variable multi-scale feature concurrency (VMC) and multi-depth feature aggregation-alignment (MAA) modules. VMC can extract multi-scale features via multiple fix-sized and deformable convolutions, while MAA can aggregate and align multi-depth features on the same order for feature inference. This allows the low-level features with more spatial details to be transmitted in depth-wise, enabling the deeper network to selectively utilize the preceding inference information. The VMMANet replaces inefficient high-density deep convolution by increasing the width of intermediate feature levels, leading to a salient decline in parameters. The OS-GD is developed for efficacious feature extraction, aggregation and distribution, further enhancing the global information gather and deployment capability of the network. On a self-made solder joint image data set, the VMMAO-YOLO achieves a mean average precision mAP@0.5 of 91.6%, surpassing all the mainstream YOLO-series models. Moreover, the VMMAO-YOLO has a body size of merely 19.3 MB and a detection speed up to 119 frame per second, far superior to the prevalent YOLO-series detectors.

Graphical abstract

Keywords

defect detection of solder joints / VMMAO-YOLO / ultra-lightweight and high-performance / multi-scale feature extraction / VMC and MAA modules / OS-GD

References

[1] Bian Y C, Fu G H, Hou Q S, Sun B, Liao G L, Han H D. Using improved YOLOv5s for defect detection of thermistor wire solder joints based on infrared thermography. In: Proceedings of the 5th International Conference on Automation, Control and Robots. Nanning: IEEE, 2021, 29–32

[2] FangT YAn J SChenQHeY ZWangH J ZhangX G. Progress and comparison in nondestructive detection, imaging and recognition technology for defects of wafers, chips and solder joints. Nondestructive Testing and Evaluation, 2023 (in press)

[3] Dai W T, Mujeeb A, Erdt M, Sourin A. Soldering defect detection in automatic optical inspection. Advanced Engineering Informatics, 2020, 43: 101004

[4] Zhang X, Huang T, Wu B, Hu Y M, Huang S, Zhou Q, Zhang X. Multi-model ensemble deep learning method for intelligent fault diagnosis with high-dimensional samples. Frontiers of Mechanical Engineering, 2021, 16(2): 340–352

[5] Ren S Q, He K M, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137–1149

[6] HeK MGkioxari GDollárPGirshickR. Mask R-CNN. In: Proceedings of 2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017, 2980–2988

[7] Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016, 779–788

[8] Redmon J, Farhadi A. YOLOv3: an incremental improvement. 2018, arXiv:1804.02767

[9] Zhu X K, Lyu S C, Wang X, Zhao Q. TPH-YOLOv5: improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops. Montreal: IEEE, 2021, 2778–2788

[10] Wang C Y, Bochkovskiy A, Liao H Y M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023, 7464–7475

[11] Aboah A, Wang B, Bagci U, Adu-Gyamfi Y. Real-time multi-class helmet violation detection using few-shot data sampling technique and YOLOv8. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Vancouver: IEEE, 2023, 5350–5358

[12] Fan F L, Wang B Y, Zhu G L, Wu J H. Efficient faster R-CNN: used in PCB solder joint defects and components detection. In: Proceedings of the 4th IEEE International Conference on Computer and Communication Engineering Technology. Beijing: IEEE, 2021, 1–5

[13] Zhang K H, Shen H K. Solder joint defect detection in the connectors using improved faster-RCNN algorithm. Applied Sciences, 2021, 11(2): 576

[14] Kim C, Hwang S, Sohn H. Weld crack detection and quantification using laser thermography, mask R-CNN, and CycleGAN. Automation in Construction, 2022, 143: 104568

[15] Liu W P, Shan S Q, Chen H Y, Wang R, Sun J M, Zhou Z K. X-ray weld defect detection based on AF-RCNN. Welding in the World, 2022, 66(6): 1165–1177

[16] Ji C, Wang H B, Li H H. Defects detection in weld joints based on visual attention and deep learning. NDT & E International, 2023, 133: 102764

[17] Cherkasov N, Ivanov M, Ulanov A. Weld surface defect detection based on a laser scanning system and YOLOv5. In: Proceedings of the International Conference on Industrial Engineering, Applications and Manufacturing. Sochi: IEEE, 2023, 851–855

[18] Yang L, Liu Y H, Peng J Z. An automatic detection and identification method of welded joints based on deep neural network. IEEE Access: Practical Innovations, Open Solutions, 2019, 7: 164952–164961

[19] Zuo Y L, Wang J T, Song J L. Application of YOLO object detection network in weld surface defect detection. In: Proceedings of the 11th IEEE Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems. Jiaxing: IEEE, 2021, 704–710

[20] Liao S D, Huang C Y, Liang Y, Zhang H Q, Liu S F. Solder joint defect inspection method based on ConvNeXt-YOLOX. IEEE Transactions on Components, Packaging, and Manufacturing Technology, 2022, 12(11): 1890–1898

[21] Liu M Y, Chen Y P, Xie J M, He L, Zhang Y. LF-YOLO: A lighter and faster YOLO for weld defect detection of X-ray image. IEEE Sensors Journal, 2023, 23(7): 7430–7439

[22] Klco P, Koniar D, Hargas L, Pociskova Dimova K, Chnapko M. Quality inspection of specific electronic boards by deep neural networks. Scientific Reports, 2023, 13(1): 20657

[23] Wang G Q, Zhang C Z, Chen M S, Lin Y C, Tan X H, Liang P, Kang Y X, Zeng W D, Wang Q. Yolo-MSAPF: Multiscale alignment fusion with parallel feature filtering model for high accuracy Weld Defect Detection. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 1–14

[24] Kwon J E, Park J H, Kim J H, Lee Y H, Cho S I. Context and scale-aware YOLO for welding defect detection. NDT & E International, 2023, 139: 102919

[25] Ajmi C, Zapata J, Elferchichi S, Laabidi K. Advanced faster-RCNN model for automated recognition and detection of weld defects on limited X-ray image dataset. Journal of Nondestructive Evaluation, 2024, 43(1): 14

[26] Zhou X Y, Xue Y, Tian G Y, Liu Z. Thermal analysis of solder joint based on eddy current pulsed thermography. IEEE Transactions on Components, Packaging, and Manufacturing Technology, 2017, 7(7): 1111–1118

[27] BochkovskiyAWangC YLiaoH Y M. YOLOv4: optimal speed and accuracy of object detection. 2020, arXiv:2004.10934

[28] Liu S, Qi L, Qin H F, Shi J P, Jia J Y. Path aggregation network for instance segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018, 8759–8768

[29] GeZLiuS T WangFLi Z MSunJ. YOLOX: exceeding YOLO Series in 2021. 2021, arXiv:2107.08430

[30] JocherGChaurasia AStokenABorovecJNanoCode012 YKwonKMichaelTXieJ C FangY FImyhxy CLornaAZengDWongZ Q VCMontesJ WangVFati PNadarALaughingDUnglvKitDe MSonckMTkianaiYxnong SkalskiHogan NairStrobelJain. Ultralytics/yolov5: v7.0-YOLOv5 SOTA realtime instance segmentation. Zenodo, 2022

[31] LiC YLi LJiangH LWengK HGengY F LiLKeZ D LiQ YCheng MNieW QLiY DZhangB LiangY FZhou L YXuX MChuX XWeiX M WeiX L. YOLOv6: a single-stage object detection framework for industrial applications. 2022, arXiv:2209.02976

[32] Jocher G, Chaurasia A, Qiu J. Ultralytics yolov8. 2023, available at Ultralytics

[33] Ding X H, Zhang X Y, Ma N N, Han J G, Ding G G, Sun J. RepVGG: making VGG-style ConvNets great again. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021, 13728–13737

[34] Hu J, Shen L, Sun G. Squeeze-and-excitation networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018, 7132–7141

[35] WooSParkJ LeeJ YKweon I S. CBAM: convolutional block attention module. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y, eds. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y, eds. Cham: Springer, 2018, 3–19

[36] LiuY CShao Z RHoffmannN. Global Attention Mechanism: retain Information to Enhance Channel-Spatial Interactions. 2021, arXiv:2112.05561

[37] WangC CHe WNieYGuoJ YLiuC J HanKWangY H. Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanism. 2023, arXiv:2309.11331

[38] Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015, 1–9

[39] LvW YXu S LZhaoYWangG ZWeiJ M CuiCDuY N DangQ QLiu Y. DETRs Beat YOLOs on Real-time Object Detection. 2023, arXiv:2304.08069

[40] ZhangXSong Y ZSongT TYangD GYeY C ZhouJZhang L M. AKConv: Convolutional Kernel with Arbitrary Sampled Shapes and Arbitrary Number of Parameters. 2023, arXiv:2311.111587

[41] Li J F, Wen Y, He L H. SCConv: spatial and channel reconstruction convolution for feature redundancy. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023, 6153–6162

[42] He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016, 770–778

[43] Li Z, Lang C Y, Liew J H, Li Y D, Hou Q B, Feng J S. Cross-layer feature pyramid network for salient object detection. IEEE Transactions on Image Processing, 2021, 30: 4587–4598

RIGHTS & PERMISSIONS

Higher Education Press