Lightweight Hybrid Wafer Defect Pattern Network Based on Feedforward Efficient Attention

Abstract

With the increase of semiconductor integration density, in order to cope with the increase of wafer defect complexity and types, especially the low recognition accuracy of overlapping mixed defects and unknown wafer defects, this study proposes a lightweight model for wafer defect detection called LightWMNet. First, using a hierarchical attention Encoder-Decoder ar-chitecture, the features of wafer defect pattern (WDP) are channel recalibrated to generate high-resolution fine-grained features and low-resolution coarse-grained features. Secondly, the backbone network incorporates two novel attention modules— feedforward spatial attention (FFSa) and feedforward channel attention (FFCa)—to amplify responses in critical defect re-gions and suppress noise from stochastic discrete pixels. These mechanisms synergistically enhance feature discriminability without introducing significant parametric overhead. Finally, the Dice loss function and the cross entropy loss function are combined to jointly evaluate the segmentation and classification accuracy of the model. Experimental results on the public mixed wafer defect dataset MixedWM38 show that the pixel accuracy (PA), intersection over union (IoU) and Dice coefficient of the proposed network reach 98.26%, 94.83% and 97.22%, respectively. Without significantly increasing the computational complexity and size of the model, compared with the existing state-of-the-art (SOTA) model, the classification accuracy of lightWMNet in single defect, three mixed defects and four mixed defects is improved by 0.5%, 0.25% and 0.89% respectively. Furthermore, we used transfer learning for the first time to evaluate the model's generalisation ability for unseen defect cat-egories. The results showed that LightWMNet still has a certain recognition ability even in untrained wafer defects.

Keywords

artificial intelligence / artificial neural network / feature detection / image classification / neural nets / pattern classification / transforms

Funding

This work was supported by the National Natural Science Foundation of China under Grant 61573183.

Conflicts of Interest

The authors declare no confiicts of interest.

Data Availability Statement

Research data are not shared.

References

[1] C. K. Hansen and P. Thyregod, “Use of Wafer Maps in Integrated Circuit Manufacturing,” Microelectronics Reliability 38, no. 6-8 (June 1998): 1155-1164, https://doi.org/10.1016/s0026-2714(98)00127-9.

[2] G. Capizzi, G. Lo Sciuto, C. Napoli, R. Shikler, and M. Woźniak, “Optimizing the Organic Solar Cell Manufacturing Process by Means of AFM Measurements and Neural Networks,” Energies 11, no. 5 (May 2018): 1221, https://doi.org/10.3390/en11051221.

[3] S. Mittal, S. Srivastava, and J. P. Jayanth, “A Survey of Deep Learning Techniques for Underwater Image Classification,” IEEE Transactions on Neural Networks and Learning Systems 34, no. 10 (October 2023): 6968-6982, https://doi.org/10.1109/tnnls.2022.3143887.

[4] G. L. Sciuto, C. Napoli, G. Capizzi, and R. Shikler, “Organic Solar Cells Defects Detection by Means of an Elliptical Basis Neural Network and a New Feature Extraction Technique,” Optik 194 (October 2019): 163038, https://doi.org/10.1016/j.ijleo.2019.163038.

[5] V. Badrinarayanan, A. Kendall, and R. Cipolla, “SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence 39, no. 12 (January 2017): 2481-2495, https://doi.org/10.1109/tpami.2016.2644615.

[6] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” in Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) (Springer International Pub-lishing, October 2015), 234-241.

[7] K. He, G. Gkioxari, P. Dollár and R. Girshick, “Mask R-CNN ” in International Conference on Computer Vision (ICCV) (2017), 2961-2969.

[8] L. C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Semantic Image Segmentation With Deep Convolutional Nets and Fully Connected CRFs,” arXiv preprint arXiv:1412. 7062 (2014), https://doi.org/10.48550/arXiv.1412.7062.

[9] A. Vaswani, “Attention Is all You Need,” in Conference on Neural Information Processing Systems (NeurIPS) (2017).

[10] D. Song,B. Liu, and Y. Li, “Based on End-to-End Object Detection Algorithm With Transformers for Detecting Wafer Maps,” in Pro-ceedings of International Conference on Computer Network, Electronic and Automation (ICCNEA) (IEEE, September 2022), 297-302.

[11] T. Nakazawa and D. V. Kulkarni, “Anomaly Detection and Segmen-tation for Wafer Defect Patterns Using Deep Convolutional Encoder- Decoder Neural Network Architectures in Semiconductor Manufacturing,” IEEE Transactions on Semiconductor Manufacturing 32, no. 2 (May 2019): 250-256, https://doi.org/10.1109/tsm.2019.2897690.

[12] H. Han, C. Gao, Y. Zhao, S. Liao, L. Tang, and X. Li, “Polycrystalline Silicon Wafer Defect Segmentation Based on Deep Convolutional Neural Networks,” Pattern Recognition Letters 130 (February 2020): 234-241, https://doi.org/10.1016/j.patrec.2018.12.013.

[13] G. Wen, Z. Gao, Q. Cai, Y. Wang, and S. Mei, “A Novel Method Based on Deep Convolutional Neural Networks for Wafer Semi-conductor Surface Defect Inspection,” IEEE Transactions on Instru-mentation and Measurement 69, no. 12 (December 2020): 9668-9680, https://doi.org/10.1109/tim.2020.3007292.

[14] N. Yu, Q. Xu, and H. Wang, “Wafer Defect Pattern Recognition and Analysis Based on Convolutional Neural Network,” IEEE Transactions on Semiconductor Manufacturing 32, no. 4 (November 2019): 566-573, https://doi.org/10.1109/tsm.2019.2937793.

[15] Y. Yuan-Fu and S. Min, “Double Feature Extraction Method for Wafer Map Classification Based on Convolution Neural Network,” in 31st Annual SEMI Advanced Semiconductor Manufacturing Conference (ASMC) (IEEE, May 2020), 1-6.

[16] Y. Xie, S. Li, C. T. Wu, Z. Lai, and M. Su, “A Novel Hypergraph Convolution Network for Wafer Flaw Patterns Identification Based on an Unbalanced Dataset,” Journal of Intelligent Manufacturing 35, no. 2 (December 2024): 633-646, https://doi.org/10.1007/s10845-022-02067-z.

[17] Y. Li and J. Wang, “A Defect Detection Method Based on Improved Mask R-CNN for Wafer Maps,” in Proceedings of the International Conference on Computer Network, Electronic and Automation (ICCNEA) (IEEE, September 2021), 133-137.

[18] X. Wang, X. Jia, C. Jiang, and S. Jiang, “A Wafer Surface Defect Detection Method Built on Generic Object Detection Network,” Digital Signal Processing 130 (October 2022): 103718, https://doi.org/10.1016/j.dsp.2022.103718.

[19] Y. Wei and H. Wang, “Mixed-Type Wafer Defect Recognition With Multi-Scale Information Fusion Transformer,” IEEE Transactions on Semiconductor Manufacturing 35, no. 2 (May 2022): 341-352, https://doi.org/10.1109/tsm.2022.3156583.

[20] N. Shukla, “Efficient Mixed-type Wafer Defect Pattern Recognition Using Compact Deformable Convolutional Transformers,” arXiv: 2303. 13827 2023), https://doi.org/10.48550/arXiv.2303.13827.

[21] J. Cha and J. Jeong, “Improved U-Net With Residual Attention Block for mixed-defect Wafer Maps,” Applied Sciences 12, no. 4 (February 2022): 2209, https://doi.org/10.3390/app12042209.

[22] M. C. Chiu and T. M. Chen, “Applying Data Augmentation and Mask R-CNN-based Instance Segmentation Method for Mixed-Type Wafer Maps Defect Patterns Classification,” IEEE Transactions on Semiconductor Manufacturing 34, no. 4 (November 2021): 455-463, https://doi.org/10.1109/tsm.2021.3118922.

[23] H. Chen, J. Liu, S. Wang, and K. Liu, “Robust Dislocation Defects Region Segmentation for Polysilicon Wafer Image With Random Texture Background,” IEEE Access 7 (September 2019): 134318-134329, https://doi.org/10.1109/access.2019.2942218.

[24] S. Nag, D. Makwana, R. S. C. Teja, S. Mittal, and C. K. Mohan, “WaferSegClassNet-A Light-Weight Network for Classification and Segmentation of Semiconductor Wafer Defects,” Computers in Industry 142 (November 2022): 103720, https://doi.org/10.1016/j.compind.2022.103720.

[25] Y. Kong and D. Ni, “Qualitative and Quantitative Analysis of Multi- Pattern Wafer Bin Maps,” IEEE Transactions on Semiconductor Manufacturing 33, no. 4 (November 2020): 578-586, https://doi.org/10.1109/tsm.2020.3022431.

[26] G. Tello, O. Y. Al-Jarrah, P. D. Yoo, Y. Al-Hammadi, S. Muhaidat, and U. Lee, “Deep-Structured Machine Learning Model for the Recog-nition of Mixed-Defect Patterns in Semiconductor Fabrication Pro-cesses,” IEEE Transactions on Semiconductor Manufacturing 31, no. 2 (May 2018): 315-322, https://doi.org/10.1109/tsm.2018.2825482.

[27] J. Yan, Y. Sheng, and M. Piao, “Semantic Segmentation-Based Wafer Map Mixed-Type Defect Pattern Recognition,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 42, no. 11 (November 2023): 4065-4074, https://doi.org/10.1109/tcad.2023.3274958.

[28] M. J. Wu, J. S. R. Jang, and J. L. Chen, “Wafer Map Failure Pattern Recognition and Similarity Ranking for Large-Scale Data Sets,” IEEE Transactions on Semiconductor Manufacturing 28, no. 1 (February 2015): 1-12, https://doi.org/10.1109/TSM.2014.2364237.

[29] J. Wang, C. Xu, Z. Yang, J. Zhang, and X. Li, “Deformable Con-volutional Networks for Efficient Mixed-Type Wafer Defect Pattern Recognition,” IEEE Transactions on Semiconductor Manufacturing 33, no. 4 (November 2020): 587-596, https://doi.org/10.1109/tsm.2020.3020985.

[30] A. Garcia-Garcia, S. Orts-Escolano, S. Oprea, V. Villena-Martinez, and J. Garcia-Rodriguez, “A Review on Deep Learning Techniques Applied to Semantic Segmentation,” arXiv preprint arXiv:1704. 06857 (2017), https://doi.org/10.48550/arXiv.1704.06857.

[31] A. Chen and C. Asawa, Going Beyond the Bounding Box With Se-mantic Segmentation (Gradient, 2018).

[32] F. Milletari, N. Navab, and S. A. Ahmadi, “V-Net: Fully Convolu-tional Neural Networks for Volumetric Medical Image Segmentation,” in Proceedings of the 4th International Conference on 3D Vision (3DV) (IEEE, March 2016), 565-571.

[33] E. Gibaja and S. Ventura, “A Tutorial on Multilabel Learning,” ACM Computing Surveys 47, no. 3 (April 2015): 1-38, https://doi.org/10.1145/2716262.

[34] C. Yu, C. Gao, J. Wang, G. Yu, C. Shen, and N. Sang, “BiSeNet V2: Bilateral Network With Guided Aggregation for Real-Time Semantic Segmentation,” International Journal of Computer Vision 129, no. 11 (September 2021): 593-602, https://doi.org/10.1007/s11263-021-01515-2.

[35] A. Howard, M. Sandler, G. Chu, et al., “Searching for Mobile-NetV3,” in International Conference on Computer Vision (2019), 1314-1324.

[36] J. Long,E. Shelhamer, and T. Darrell, “Fully Convolutional Net-works for Semantic Segmentation,” in Conference on Computer Vision and Pattern Recognition (CVPR) (2015), 3431-3440.

[37] L. C. Chen, “Rethinking Atrous Convolution for Semantic Image Segmentation,” arXiv:1706. 05587 (2017), https://doi.org/10.48550/arXiv.1706.05587.

[38] L. C. Chen, Y. Zhu, G. Papandreou,F. Schroff, and H. Adam, “Encoder-Decoder With Atrous Separable Convolution for Semantic Image Segmentation,” in Conference on Computer Vision (ECCV) (2018), 801-818.

[39] E. Xie, W. Wang, Z. Yu, A. Anandkumar,J. M. Alvarez, and P. Luo, “SegFormer: Simple and Efficient Design for Semantic Segmentation With Transformers,” in Conference on Neural Information Processing Systems (NeurIPS), Vol. 34, (2021), 12077-12090.

[40] M. Tan and Q. Le, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,” in Proceedings of the International Conference on Machine Learning (ICML) (PMLR, June 2019), 6105-6114.

[41] C. Szegedy, V. Vanhoucke, S. Ioffe,J. Shlens, and Z. Wojna, “Rethinking the Inception Architecture for Computer Vision,” in Con-ference on Computer Vision and Pattern Recognition (CVPR) (2016), 2818-2826.

[42] K. He, X. Zhang,S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in Conference on Computer Vision and Pattern Recognition (CVPR) (2016), 770-778.

[43] K. Simonyan and A. Zisserman, “Very Deep Convolutional Net-works for Large-Scale Image Recognition,” arXiv:1409. 1556 (2014), https://doi.org/10.48550/arXiv.1409.1556.

[44] G. Huang, Z. Liu,L. Van Der Maaten, and K. Q. Weinberger, “Densely Connected Convolutional Networks,” in Conference on Com-puter Vision and Pattern Recognition (CVPR) (2017), 4700-4708.

[45] S. Chen, Z. Huang, T. Wang, X. Hou, and J. Ma, “Mixed-Type Wafer Defect Detection Based on Multi-Branch Feature Enhanced Residual Module,” Expert Systems With Applications 242 (May 2024): 122795, https://doi.org/10.1016/j.eswa.2023.122795.