Improving accuracy of automatic optical inspection with machine learning

Abstract

Electronic devices require the printed circuit board (PCB) to support the whole structure, but the assembly of PCBs suffers from welding problem of the electronic components such as surface mounted devices (SMDs) resistors. The automated optical inspection (AOI) machine, widely used in industrial production, can take the image of PCBs and examine the welding issue. However, the AOI machine could commit false negative errors and dedicated technicians have to be employed to pick out those misjudged PCBs. This paper proposes a machine learning based method to improve the accuracy of AOI. In particular, we propose an adjacent pixel RGB value based method to pre-process the image from the AOI machine and build a customized deep learning model to classify the image. We present a practical scheme including two machine learning procedures to mitigate AOI errors.We conduct experiments with the real dataset from a production line for three months, the experimental results show that our method can reduce the rate of misjudgment from 0.3%–0.5% to 0.02%–0.03%, which is meaningful for thousands of PCBs each containing thousands of electronic components in practice.

Keywords

automated optical inspection / industrial internet of things / machine learning / image classification

References

[1] Blackwell G R. Electronic Systems Maintenance Handbook. 2nd ed. CRC Press, 2002

[2] Huang X, Zhu S, Huang X, Su B, Ou C, Zhou W. Detection of plated through hole defects in printed circuit board with X-ray. In: Proceedings of the 16th IEEE Intnational Conferenceon on Electronic Packaging Technology. 2015, 1296–1301

[3] Alaoui N E B, Tounsi P, Boyer A, Viard A. Detecting PCB assembly defects using infrared thermal signatures. In: Proceedings of International Conference “Mixed Design of Integrated Circuits and Systems”. 2019, 345–349

[4] Härter S, Klinger T, Franke J, Beer D. Comprehensive correlation of inline inspection data for the evaluation of defects in heterogeneous electronic assemblies. In: Proceedings of Pan Pacific Microelectronics Symposium. 2016, 1–6

[5] Wen K P, Wu W M, Huang C Y. Automatic optical inspection system and operating method thereof. U.S. Patent 10,438,340. 2019

[6] Runji JM, Lin C. Automatic optical inspection aided augmented realitybased PCBA inspection: a development. In: Proceedings of IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology. 2019, 667–671

[7] Wang W, Chen S, Chen L, Chang W. A machine vision based automatic optical inspection system for measuring drilling quality of printed circuit boards. IEEE Access, 2017, 5: 10817–10833

[8] Qiang G, Shanshan Z, Yang Z, Mao C. Detection method of PCB component based on automatic optical stitching algorithm. Circuit World, 2015, 41(4): 133–136

[9] Shorten C, Khoshgoftaar TM. A survey on image data augmentation for deep learning. Journal of Big Data, 2019, 6(1):1–48

[10] Lengerich B, Xing E P, Caruana R. On dropout, overfitting, and interaction effects in deep neural networks. 2020, arXiv preprint arXiv: 2007.00823

[11] Liu X, Zhang J, Jiang S, Yang Y, Li K, Cao J, Liu J. Accurate localization of tagged objects using mobile RFID-augmented robots. IEEE Transactions on Mobile Computing, 2021, 20(4): 1273–1284

[12] Liu X, Chen S, Liu J, Qu W, Xiao F, Liu A X, Liu J. Fast and accurate detection of unknown tags for RFID systems-hash collisions are desirable. IEEE/ACM Transactions on Networking, 2020, 28(1):126–139

[13] Tong X, Liu K, Tian X, Fu L, Wang X. Fineloc: a fine-grained selfcalibrating wireless indoor localization system. IEEE Transactions on Mobile Computing, 2018, 18(9): 2077–2090

[14] Du T B, Shen G H, Huang Z Q, Yu Y S,Wu D X. Automatic traceability link recovery via active learning. Frontiers of Information Technology & Electronic Engineering, 2020, 21(8): 1–9

[15] Huang J H, Di X G, Chen A Y. A novel convolutional neural network method for crowd counting. Frontiers of Information Technology & Electronic Engineering, 2020, 21(8): 1150–1160

[16] Alreshidi E. Smart sustainable agriculture (SSA) solution underpinned by internet of things (IoT) and artificial intelligence (AI). 2019, arXiv preprint arXiv: 1906.03106

[17] Tzafestas S G. Synergy of IoT and AI in modern society: the robotics and automation case. Robotics & Automation Engineering Journal, 2018, 31(5): 1–15

[18] Xiao L, Wan X, Lu X, Zhang Y, Wu D. IoT security techniques based on machine learning: how do IoT devices use AI to enhance security? IEEE Signal Processing Magazine, 2018, 35(5): 41–49

[19] Meidan Y, Bohadana M, Shabtai A, Guarnizo J D, Ochoa M, Tippenhauer N O, Elovici Y. ProfilIoT: a machine learning approach for IoT device identification based on network traffic analysis. In: Proceedings of the Symposium on Applied Computing. 2017, 506–509

[20] Njima W, Ahriz I, Zayani R, Terre M, Bouallegueet R. Deep CNN for indoor localization in IoT-sensor systems. Journal of Sensors, 2019, 19(14): 3127–3132

[21] Canedo J, Skjellum A. Using machine learning to secure IoT systems. In: Proceedings of the 14th Annual Conference on Privacy, Security and Trust. 2016, 219–222

[22] Wang S, Tuor T, Salonidis T, Leung K K, Makaya C, He T, Chan K. When edge meets learning: adaptive control for resource-constrained distributed machine learning. In: Proceedings of the IEEE Conference on Computer Communications. 2018, 63–71

[23] Xiao L, Wan X, Lu X, Zhang Y, Wu D. IoT security techniques based on machine learning: how do IoT devices use AI to enhance security? IEEE Signal Processing Magazine, 2018, 35(5): 41–49

[24] Pramudita R, Hariadi F I. Development of techniques to determine object shifts for PCB board assembly automatic optical inspection. In: Proceedings of the International Symposium on Electronics and Smart Devices. 2018, 1–4

[25] Wu F, Li S, Zhao Y. A self-adaptive study method for multi-parameters thresholds in AOI system. In: Proceedings of the 11th World Congress on Intelligent Control and Automation. 2014, 5256–5259

[26] Jia X,Wang T, Li Y, Liu J, Zhang Y. AOI planning method based on genetic algorithm. In: Proceedings of International Conference on Mechatronics and Automation. 2019, 1801–1805

[27] Takacs T, Vajta L. Novel outlier filtering method for AOI image databases. IEEE Transactions on Components, Packaging and Manufacturing Technology, 2012, 2(4): 700–709

[28] Chaudhary V, Dave I R, Upla K P. Automatic visual inspection of printed circuit board for defect detection and classification. In: Proceedings of the International Conference on Wireless Communications, Signal Processing and Networking. 2017, 732–737

[29] Tsai J, Lin C, Chang C, Chou J. Optimized positional compensation parameters for exposure machine for flexible printed circuit board. IEEE Transactions on Industrial Informatics, 2015, 11(6): 1366–1377

[30] Mohammadi P, Wang Z J. Machine learning for quality prediction in abrasion-resistant material manufacturing process. In: Proceedings of the 2016 IEEE Canadian Conference on Electrical and Computer Engineering. 2016, 1–4

[31] Sartzetakis I, Christodoulopoulos K K, Varvarigos EM. Accurate quality of transmission estimation with machine learning. IEEE/OSA Journal of Optical Communications and Networking, 2019, 11(3): 140–150

[32] Von Enzberg S, Al-Hamadi A. A multiresolution approach to modelbased 3-D surface quality inspection. IEEE Transactions on Industrial Informatics, 2016, 12(4): 1498–1507

[33] Alonzo LM B, Chioson F B, Co H S, Bugtai N T, Baldovino R G. A machine learning approach for coconut sugar quality assessment and prediction. In: Proceedings of the 10th IEEE International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment and Management. 2018, 1–4

[34] Sultanow E, Ullrich A, Konopik S, Vladova G. Machine learning based static code analysis for software quality assurance. In: Proceedings of the 13th International Conference on Digital Information Management. 2018, 156–161

[35] Li X, Zhang W, Ding Q, Li X. Diagnosing rotating machines with weakly supervised data using deep transfer learning. IEEE Transactions on Industrial Informatics, 2019, 16(3): 1688–1697

RIGHTS & PERMISSIONS

Higher Education Press