VMMAO-YOLO: an ultra-lightweight and scale-aware detector for real-time defect detection of avionics thermistor wire solder joints

Abstract

The quality of the exposed avionics solder joints has a significant impact on the stable operation of the in-orbit spacecrafts. Nevertheless, the previously reported inspection methods for multi-scale solder joint defects generally suffer low accuracy and slow detection speed. Herein, a novel real-time detector VMMAO-YOLO is demonstrated based on variable multi-scale concurrency and multi-depth aggregation network (VMMANet) backbone and “one-stop” global information gather-distribute (OS-GD) module. Combined with infrared thermography technology, it can achieve fast and high-precision detection of both internal and external solder joint defects. Specifically, VMMANet is designed for efficient multi-scale feature extraction, which mainly comprises variable multi-scale feature concurrency (VMC) and multi-depth feature aggregation-alignment (MAA) modules. VMC can extract multi-scale features via multiple fix-sized and deformable convolutions, while MAA can aggregate and align multi-depth features on the same order for feature inference. This allows the low-level features with more spatial details to be transmitted in depth-wise, enabling the deeper network to selectively utilize the preceding inference information. The VMMANet replaces inefficient high-density deep convolution by increasing the width of intermediate feature levels, leading to a salient decline in parameters. The OS-GD is developed for efficacious feature extraction, aggregation and distribution, further enhancing the global information gather and deployment capability of the network. On a self-made solder joint image data set, the VMMAO-YOLO achieves a mean average precision mAP@0.5 of 91.6%, surpassing all the mainstream YOLO-series models. Moreover, the VMMAO-YOLO has a body size of merely 19.3 MB and a detection speed up to 119 frame per second, far superior to the prevalent YOLO-series detectors.

Graphical abstract

Keywords

defect detection of solder joints / VMMAO-YOLO / ultra-lightweight and high-performance / multi-scale feature extraction / VMC and MAA modules / OS-GD

References

[1] Bian Y C, Fu G H, Hou Q S, Sun B, Liao G L, Han H D. Using improved YOLOv5s for defect detection of thermistor wire solder joints based on infrared thermography. In: Proceedings of the 5th International Conference on Automation, Control and Robots. Nanning: IEEE, 2021, 29–32

[2] FangT YAn J SChenQHeY ZWangH J ZhangX G. Progress and comparison in nondestructive detection, imaging and recognition technology for defects of wafers, chips and solder joints. Nondestructive Testing and Evaluation, 2023 (in press)

[3] Dai W T, Mujeeb A, Erdt M, Sourin A. Soldering defect detection in automatic optical inspection. Advanced Engineering Informatics, 2020, 43: 101004

[4] Zhang X, Huang T, Wu B, Hu Y M, Huang S, Zhou Q, Zhang X. Multi-model ensemble deep learning method for intelligent fault diagnosis with high-dimensional samples. Frontiers of Mechanical Engineering, 2021, 16(2): 340–352

[5] Ren S Q, He K M, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137–1149

[6] HeK MGkioxari GDollárPGirshickR. Mask R-CNN. In: Proceedings of 2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017, 2980–2988

[7] Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016, 779–788

[8] Redmon J, Farhadi A. YOLOv3: an incremental improvement. 2018, arXiv:1804.02767

[9] Zhu X K, Lyu S C, Wang X, Zhao Q. TPH-YOLOv5: improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops. Montreal: IEEE, 2021, 2778–2788

[10] Wang C Y, Bochkovskiy A, Liao H Y M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023, 7464–7475

[11] Aboah A, Wang B, Bagci U, Adu-Gyamfi Y. Real-time multi-class helmet violation detection using few-shot data sampling technique and YOLOv8. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Vancouver: IEEE, 2023, 5350–5358

[12] Fan F L, Wang B Y, Zhu G L, Wu J H. Efficient faster R-CNN: used in PCB solder joint defects and components detection. In: Proceedings of the 4th IEEE International Conference on Computer and Communication Engineering Technology. Beijing: IEEE, 2021, 1–5

[13] Zhang K H, Shen H K. Solder joint defect detection in the connectors using improved faster-RCNN algorithm. Applied Sciences, 2021, 11(2): 576

[14] Kim C, Hwang S, Sohn H. Weld crack detection and quantification using laser thermography, mask R-CNN, and CycleGAN. Automation in Construction, 2022, 143: 104568

[15] Liu W P, Shan S Q, Chen H Y, Wang R, Sun J M, Zhou Z K. X-ray weld defect detection based on AF-RCNN. Welding in the World, 2022, 66(6): 1165–1177

[16] Ji C, Wang H B, Li H H. Defects detection in weld joints based on visual attention and deep learning. NDT & E International, 2023, 133: 102764

[17] Cherkasov N, Ivanov M, Ulanov A. Weld surface defect detection based on a laser scanning system and YOLOv5. In: Proceedings of the International Conference on Industrial Engineering, Applications and Manufacturing. Sochi: IEEE, 2023, 851–855

[18] Yang L, Liu Y H, Peng J Z. An automatic detection and identification method of welded joints based on deep neural network. IEEE Access: Practical Innovations, Open Solutions, 2019, 7: 164952–164961

[19] Zuo Y L, Wang J T, Song J L. Application of YOLO object detection network in weld surface defect detection. In: Proceedings of the 11th IEEE Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems. Jiaxing: IEEE, 2021, 704–710

[20] Liao S D, Huang C Y, Liang Y, Zhang H Q, Liu S F. Solder joint defect inspection method based on ConvNeXt-YOLOX. IEEE Transactions on Components, Packaging, and Manufacturing Technology, 2022, 12(11): 1890–1898

[21] Liu M Y, Chen Y P, Xie J M, He L, Zhang Y. LF-YOLO: A lighter and faster YOLO for weld defect detection of X-ray image. IEEE Sensors Journal, 2023, 23(7): 7430–7439

[22] Klco P, Koniar D, Hargas L, Pociskova Dimova K, Chnapko M. Quality inspection of specific electronic boards by deep neural networks. Scientific Reports, 2023, 13(1): 20657

[23] Wang G Q, Zhang C Z, Chen M S, Lin Y C, Tan X H, Liang P, Kang Y X, Zeng W D, Wang Q. Yolo-MSAPF: Multiscale alignment fusion with parallel feature filtering model for high accuracy Weld Defect Detection. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 1–14

[24] Kwon J E, Park J H, Kim J H, Lee Y H, Cho S I. Context and scale-aware YOLO for welding defect detection. NDT & E International, 2023, 139: 102919

[25] Ajmi C, Zapata J, Elferchichi S, Laabidi K. Advanced faster-RCNN model for automated recognition and detection of weld defects on limited X-ray image dataset. Journal of Nondestructive Evaluation, 2024, 43(1): 14

[26] Zhou X Y, Xue Y, Tian G Y, Liu Z. Thermal analysis of solder joint based on eddy current pulsed thermography. IEEE Transactions on Components, Packaging, and Manufacturing Technology, 2017, 7(7): 1111–1118

[27] BochkovskiyAWangC YLiaoH Y M. YOLOv4: optimal speed and accuracy of object detection. 2020, arXiv:2004.10934

[28] Liu S, Qi L, Qin H F, Shi J P, Jia J Y. Path aggregation network for instance segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018, 8759–8768

[29] GeZLiuS T WangFLi Z MSunJ. YOLOX: exceeding YOLO Series in 2021. 2021, arXiv:2107.08430

[30] JocherGChaurasia AStokenABorovecJNanoCode012 YKwonKMichaelTXieJ C FangY FImyhxy CLornaAZengDWongZ Q VCMontesJ WangVFati PNadarALaughingDUnglvKitDe MSonckMTkianaiYxnong SkalskiHogan NairStrobelJain. Ultralytics/yolov5: v7.0-YOLOv5 SOTA realtime instance segmentation. Zenodo, 2022

[31] LiC YLi LJiangH LWengK HGengY F LiLKeZ D LiQ YCheng MNieW QLiY DZhangB LiangY FZhou L YXuX MChuX XWeiX M WeiX L. YOLOv6: a single-stage object detection framework for industrial applications. 2022, arXiv:2209.02976

[32] Jocher G, Chaurasia A, Qiu J. Ultralytics yolov8. 2023, available at Ultralytics

[33] Ding X H, Zhang X Y, Ma N N, Han J G, Ding G G, Sun J. RepVGG: making VGG-style ConvNets great again. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021, 13728–13737

[34] Hu J, Shen L, Sun G. Squeeze-and-excitation networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018, 7132–7141

[35] WooSParkJ LeeJ YKweon I S. CBAM: convolutional block attention module. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y, eds. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y, eds. Cham: Springer, 2018, 3–19

[36] LiuY CShao Z RHoffmannN. Global Attention Mechanism: retain Information to Enhance Channel-Spatial Interactions. 2021, arXiv:2112.05561

[37] WangC CHe WNieYGuoJ YLiuC J HanKWangY H. Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanism. 2023, arXiv:2309.11331

[38] Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015, 1–9

[39] LvW YXu S LZhaoYWangG ZWeiJ M CuiCDuY N DangQ QLiu Y. DETRs Beat YOLOs on Real-time Object Detection. 2023, arXiv:2304.08069

[40] ZhangXSong Y ZSongT TYangD GYeY C ZhouJZhang L M. AKConv: Convolutional Kernel with Arbitrary Sampled Shapes and Arbitrary Number of Parameters. 2023, arXiv:2311.111587

[41] Li J F, Wen Y, He L H. SCConv: spatial and channel reconstruction convolution for feature redundancy. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023, 6153–6162

[42] He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016, 770–778

[43] Li Z, Lang C Y, Liew J H, Li Y D, Hou Q B, Feng J S. Cross-layer feature pyramid network for salient object detection. IEEE Transactions on Image Processing, 2021, 30: 4587–4598

Acknowledgements

The authors appreciate the financial support from the projects supported by the National Natural Science Foundation of China (Grant No. 52305623), the Natural Science Foundation of Hubei Province, China (Grant No. 2022CFB589), and the Natural Science Foundation of Chongqing, China (Grant No. CSTB2023NSCQ-MSX0636).

Conflict of Interest

Tielin SHI is an executive editor-in-chief of Frontiers of Mechanical Engineering, who was excluded from the peer-review process and all editorial decisions related to the acceptance and publication of this article. Peer-review was handled independently by the other editors to minimize bias.

RIGHTS & PERMISSIONS

2024 Higher Education Press