Attention-relation network for mobile phone screen defect classification via a few samples

Abstract

How to use a few defect samples to complete the defect classification is a key challenge in the production of mobile phone screens. An attention-relation network for the mobile phone screen defect classification is proposed in this paper. The architecture of the attention-relation network contains two modules: a feature extract module and a feature metric module. Different from other few-shot models, an attention mechanism is applied to metric learning in our model to measure the distance between features, so as to pay attention to the correlation between features and suppress unwanted information. Besides, we combine dilated convolution and skip connection to extract more feature information for follow-up processing. We validate attention-relation network on the mobile phone screen defect dataset. The experimental results show that the classification accuracy of the attention-relation network is 0.9486 under the 5-way 1-shot training strategy and 0.9039 under the 5-way 5-shot setting. It achieves the excellent effect of classification for mobile phone screen defects and outperforms with dominant advantages.

Keywords

Mobile phone screen defects / A few samples / Relation network / Attention mechanism / Dilated convolution

References

[1] H.W. Kim, S.I. Yoo, Non referential method for defects inspection of tft-lcd pad, in: Image Processing: Machine Vision Applications, 2008, 68130V-8.

[2] Y. Junhong, Y. Xinhui, L. Zhonghu, Defect detection method of mobile phone lcd screen based on ccd, J. Baotou. Vocat.-Tech. Coll. 18 (1) (2017), 18-20þ59.

[3] D. Weimer, B. Scholz-Reiter, M. Shpitalni, Design of defect inspection algorithm for mobile screen based on mathematical morphology, Electrical Automation 3 (34)(2013) 417-420.

[4] D. Weimer, B. Scholz-Reiter, M. Shpitalni, Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection, CIRP Ann.-Manuf. Technol. 65 (1) (2016) 417-420.

[5] T. Wang, Y. Chen, M. Qiao, H. Snoussi, A fast and robust convolutional neural network-based defect detection model in product quality control, Int. J. Adv. Manuf. Technol. 94 (9) (2017) 3465-3471.

[6] M. Shuang, W. Yudan, W. Guojun, Automatic fabric defect detection with a multi-scale convolutional denoising autoencoder network model, Sensors 18 (4) (2018) 1064.

[7] D. Tabernik, S. Ela, J. Skvar, D. Skoaj, Segmentation-based deep-learning approach for surface-defect detection, J. Intell. Manuf. 31 (3) (2019) 759-776.

[8] L. Qiu, X. Wu, Z. Yu, A high-efficiency fully convolutional networks for pixel-wise surface defect detection, IEEE Access 7 (2019) 15884-15893.

[9] X. Xu, H. Zheng, Z. Guo, X. Wu, Z. Zheng, Sdd-cnn: small data-driven convolution neural networks for subtle roller defect inspection, Appl. Sci. 9 (7) (2019) 1364.

[10] X. Gu, Research on Defect Detection Method of Small Sample Image Based on Deep Learning, Harbin Institute of Technology, 2017.

[11] Z. Ma, G. He, Y. Yuan, Fume hood window state recognition method based on few shot deep learning, J. East China Univ. Sci. Technol. 46 (3) (2019) 428-435.

[12] Y. Liu, F. Meng, H. Li, J. Yang, Q. wu, L. Xu, A few shot segmentation method combining global and local similarity, J. Beijing Univ. Aeronaut. Astronaut. 47 (3)(2020) 665-674.

[13] Y. Dong, H. Pan, Q. Cui, X. Bian, T. Teng, B. Wang, Few-shot segmentation method for multi-modal magnetic resonance images of brain tumor, J. Comput. Appl. 41 (4)(2020) 1049-1054.

[14] S. Luo, C. Li, Few-shot instance segmentation based on double similarity siamese network, J. Wuhan Univ. Sci. Technol. 43 (1) (2020) 59-66.

[15] Y. Wang, Q. Yao, J.T. Kwok, L.M. Ni, Generalizing from a few examples: a survey on few-shot learning, ACM Comput. Surv. 53 (3) (2020) 1-34.

[16] C. Finn, P. Abbeel, S. Levine,Model-agnostic meta-learning for fast adaptation of deep networks, in:International Conference on Machine Learning, 2017, pp. 1126-1135.

[17] A. Nichol, J. Schulman, Reptile: a Scalable Metalearning Algorithm, 2018 arXiv preprint arXiv: 1803.02999.

[18] G. Koch, R. Zemel, R. Salakhutdinov, et al., Siamese neural networks for one-shot image recognition,in: ICML Deep Learning Workshop, 2015, pp. 1-30.

[19] O. Vinyals, C. Blundell, T. Lillicrap, D. Wierstra, et al. Matching networks for one shot learning, Adv. Neural Inf. Process., Syst. 29 (2016) 3630-3638.

[20] J. Snell, K. Swersky, R. Zemel, Prototypical networks for few-shot learning, Adv. Neural Inf. Process. Syst. 30 (2017) 4080-4090.

[21] F. Sung, Y. Yang, L. Zhang, T. Xiang, P.H.S. Torr, T.M. Hospedales, Learning to compare: relation network for few-shot learning, in: IEEE Conference on Computer Vision and Pattern Recognition, CVPR), 2018, pp. 1199-1208.

[22] V. Garcia, J. Bruna, Few-shot Learning with Graph Neural Networks, 2017 arXiv preprint arXiv:1711.04043.

[23] J. Kim, T. Kim, S. Kim, C.D. Yoo, Edge-labeling graph neural network for few-shot learning, in: IEEE Conference on Computer Vision and Pattern Recognition, CVPR), 2019, pp. 11-20.

[24] F. Yu, V. Koltun, Multi-scale Context Aggregation by Dilated Convolutions, 2015 arXiv preprint arXiv:1511.07122.

[25] L.C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, A.L. Yuille, Semantic image segmentation with deep convolutional nets and fully connected crfs, Comput. Sci. 4 (2014) 357-361.

[26] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, A.L. Yuille, Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs, IEEE Trans. Pattern Anal. Mach. Intell. 40 (4) (2017) 834-848.

[27] L.-C. Chen, G. Papandreou, F. Schroff, H. Adam, Rethinking Atrous Convolution for Semantic Image Segmentation, 2017 arXiv preprint arXiv:1706.05587.

[28] L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, H. Adam, Encoder-decoder with atrous separable convolution for semantic image segmentation, in: Proceedings of the European Conference on Computer Vision, ECCV), 2018, pp. 801-818.

[29] S. Mehta, M. Rastegari, A. Caspi, L. Shapiro, H. Hajishirzi, Espnet: efficient spatial pyramid of dilated convolutions for semantic segmentation,in:Proceedings of the European Conference on Computer Vision, ECCV), 2018, pp. 552-568.

[30] S. Liu, d. Huang, Y. Wang, Receptive field block net for accurate and fast object detection, in: Proceedings of the European Conference on Computer Vision, ECCV), 2017, pp. 385-400.

[31] Z. Li, C. Peng, G. Yu, X. Zhang, Y. Deng, J. Sun, Detnet: design backbone for object detection,in:Proceedings of the European Conference on Computer Vision, ECCV), 2018, pp. 334-350.

[32] Y. Li, Y. Chen, N. Wang, Z.X. Zhang, Scale-aware trident networks for object detection, in: International Conference on Computer Vision, ICCV), 2020, pp. 6054-6063.

[33] S. Chaudhari, V. Mithal, G. Polatkan, R. Ramanath, An attentive survey of attention models, ACM.Trans. Intell.Syst. Technol. 12 (5) (2021) 1-32.

[34] A. Newell, K. Yang, J. Deng, Stacked hourglass networks for human pose estimation, in: Proceedings of the European Conference on Computer Vision, ECCV), 2016, pp. 483-499.

[35] F. Wang, M. Jiang, C. Qian, S. Yang, C. Li, H. Zhang, X. Wang, X. Tang, Residual attention network for image classification, in: IEEE Conference on Computer Vision and Pattern Recognition, CVPR), 2017, pp. 3156-3164.

[36] X. Chu, W. Yang, W. Ouyang, C. Ma, X. Wang, Multi-context attention for human pose estimation, in: IEEE Conference on Computer Vision and Pattern Recognition, CVPR), 2017, pp. 1831-1840.

[37] M. Jaderberg, K. Simonyan, A. Zisserman, et al. Spatial transformer networks, Adv. Neural Inf. Process., Syst. 28 (2015) 30-45.

[38] J. Hu, L. Shen, G. Sun, Squeeze-and-excitation networks, in: IEEE Conference on Computer Vision and Pattern Recognition, CVPR), 2018, pp. 7132-7141.

[39] A.G. Roy, N. Navab, C. Wachinger,Recalibrating fully convolutional networks with spatial and channel ’squeeze and excitation’ blocks, IEEE Trans. Med. Imag. 38 (2)(2018) 540-549.

[40] J. Fu, J. Liu, H. Tian, Y. Li, Y. Bao, Z. Fang, H. Lu, Dual attention network for scene segmentation, in: IEEE Conference on Computer Vision and Pattern Recognition, CVPR), 2020, pp. 3146-3154.

[41] B. Lake, R. Salakhutdinov, J. Gross, J. Tenenbaum,One shot learning of simple visual concepts, in:Proceedings of the Annual Meeting of the Cognitive Science Society, 2011.

[42] D.P. Kingma, J. Ba, Adam, A Method for Stochastic Optimization, 2014 arXiv preprint arXiv:1412.6980.