Implicit multi-teacher distillation for unsupervised industrial anomaly detection

Abstract

Industrial anomaly detection is important for intelligent manufacturing since defects can degrade product reliability. Existing methods based on knowledge distillation typically use a single teacher, limiting feature diversity and robustness. Multi-teacher methods often require explicit feature alignment across different network architectures, which may introduce semantic conflicts and weaken anomaly responses. We propose implicit multi-teacher distillation (IMTD) for industrial anomaly detection. IMTD employs three complementary teacher networks, namely ResNet18, ResNet34, and Wide-ResNet50, to capture texture, structural, and semantic representations from different feature spaces. Each teacher is paired with an independent decoder trained only on normal samples, enabling implicit knowledge transfer without cross-architecture feature alignment. A false-negative-aware maximum response aggregation (FN-MRA) module is further introduced to preserve the strongest anomaly evidence during fusion. Experiments on MVTec anomaly detection (MVTec AD), visual anomaly benchmark (VisA), and metal part defect dataset (MPDD) show that IMTD consistently outperforms recent state-of-the-art methods in both image-level and pixel-level anomaly detection. The results demonstrate the effectiveness of combining complementary teachers, implicit reconstruction, and maximum-response aggregation under anomaly-scarce conditions.

Keywords

Industrial anomaly detection / Knowledge distillation / Multi-teacher ensemble / Implicit reconstruction / False-negative-aware maximum response aggregation (FN-MRA)

References

[1] Ahmad M , Rehman A , 2025. Multi-source information fusion for anomaly detection in smart grids using federated learning. Chin J Inform Fus, 2 (2): 157- 170.

[2] Batzner K , Heckler L , König R , 2024. EfficientAD: accurate visual anomaly detection at millisecond-level latencies. Proc IEEE/CVF Winter Conf on Applications of Computer Vision, p.127- 137.

[3] Bergmann P , Fauser M , Sattlegger D , et al., 2019. MVTec AD-a comprehensive real-world dataset for unsupervised anomaly detection. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.9584- 9592.

[4] Cai YX , Liang DK , Luo DL , et al., 2024. A discrepancy aware framework for robust anomaly detection. IEEE Trans Ind Inform, 20 (3): 3986- 3995.

[5] Cao YK , Zhang JN , Frittoli L , et al., 2024. AdaCLIP:adapting CLIP with hybrid learnable prompts for zero-shot anomaly detection. Proc 18^th European Conf on Computer Vision, p.55- 72.

[6] Chen QY , Luo HY , Lv CK , et al., 2025. A unified anomaly synthesis strategy with gradient ascent for industrial anomaly detection and localization. Proc European Conf on Computer Vision, p.37- 54.

[7] Cheng H , Luo JX , Zhang XY , 2025. Multimodal industrial anomaly detection via UNI-modal and cross-modal fusion. IEEE Trans Ind Inform, 21 (6): 5000- 5010.

[8] Deng HQ , Li XY , 2022. Anomaly detection via reverse distillation from one-class embedding. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.9727- 9736.

[9] Gu ZH , Zhang JN , Liu L , et al., 2024. Rethinking reverse distillation for multi-modal anomaly detection. Proc 38^th AAAI Conf on Artificial Intelligence, 38 (8): 8445- 8453.

[10] Hendria WF , Kim H , Seo D , 2025. Multi-model anomaly detection for industrial inspection with dynamic loss weighting and soft-hard features loss. Neur Comput Appl, 37 (21): 17031- 17054.

[11] Jeong J , Zou Y , Kim T , et al., 2023. WinCLIP:zero-/few-shot anomaly classification and segmentation. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.19606- 19616.

[12] Jezek S , Jonak M , Burget R , et al., 2021. Deep learning-based defect detection of metal parts:evaluating current methods in complex conditions. Proc 13^th Int Congress on Ultra Modern Telecommunications and Control Systems and Workshops, p.66- 71.

[13] Li N , Liu AJ , Zhu ZW , et al., 2025. Knowledge distillation-based anomaly detection via adaptive discrepancy optimization. IEEE Trans Ind Inform, 21 (9): 7176- 7187.

[14] Li XR , Huang ZM , Xue F , et al., 2024. MuSc: zero-shot industrial anomaly classification and segmentation with mutual scoring of the unlabeled images. Proc Int Conf on Learning Representations.

[15] Liu Y , Zhang CS , Dong XJ , et al., 2025. A multimodal industrial anomaly detection method based on mask training and teacher-student joint memory. Eng Appl Artif Intell, 161: 112299.

[16] Luo JZ , Cai Y , Cai J , et al., 2025. STMBAD:spatio-temporal multimodal behavior anomaly detector for industrial control systems. IEEE Trans Ind Inform, 21 (5): 3746- 3755.

[17] Luo W , Yao HM , Yu WY , et al., 2025. AMI-Net:adaptive mask inpainting network for industrial anomaly detection and localization. IEEE Trans Autom Sci Eng, 22: 1591- 1605.

[18] Ma Y , Jiang X , Guan N , et al., 2023. Anomaly detection based on multi-teacher knowledge distillation. J Syst Archit, 138: 102861.

[19] Ma ZQ , Li JX , Wong WK , 2025. Patch distance based auto-encoder for industrial anomaly detection. Expert Syst Appl, 270: 126537.

[20] Sun H , Cao YK , Dong H , et al., 2025. Unseen visual anomaly generation. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.25508- 25517.

[21] Tien TD , Nguyen AT , Tran NH , et al., 2023. Revisiting reverse distillation for anomaly detection. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.24511- 24520.

[22] Wahid JA , Ayoub M , Xu ML , et al., 2025. NN2ViT:neural networks and vision transformers based approach for visual anomaly detection in industrial images. Neurocomputing, 615: 128845.

[23] Wang RX , Li Z , Zheng L , et al., 2025. Deep feature clustering for multi-class industrial image anomaly detection. Knowl-Based Syst, 311: 113134.

[24] Wei CY , Han H , Xia Y , et al., 2025. TDAD:self-supervised industrial anomaly detection with a two-stage diffusion model. Comput Ind, 164: 104192.

[25] Xu MH , Zhu CP , Feng G , et al., 2025. Multitask hybrid knowledge distillation for unsupervised anomaly detection. IEEE Trans Ind Inform, 21 (7): 5666- 5676.

[26] Xu MH , Qu LL , Nie ZH , et al., 2026. Towards reliable medical unsupervised anomaly detection:a benchmark and dataset for PET/CT with cross-modality knowledge distillation network. Inform Fus, 133: 104341.

[27] Zhang H , Wang Z , Zeng D , et al., 2025. DiffusionAD:norm-guided one-step denoising diffusion for anomaly detection. IEEE Trans Patt Anal Mach Intell, 47 (8): 7140- 7152.

[28] Zhang HL , Chen DF , Wang C , 2023. Adaptive multi-teacher knowledge distillation with meta-learning. Proc IEEE Int Conf on Multimedia and Expo, p.1943- 1948.

[29] Zhang X , Li SY , Li X , et al., 2023. DeSTSeg:segmentation guided denoising student-teacher for anomaly detection. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.3914- 3923.

[30] Zhang ZL , Zhao ZB , Zhang XW , et al., 2023. Industrial anomaly detection with domain shift:a real-world dataset and masked multi-scale reconstruction. Comput Ind, 151: 103990.

[31] Zhong YZ , Guo B , Sun ZM , et al., 2026. MSDCIR-AD:unsupervised anomaly detection via multi-criteria semantic distances and constrained image reconstruction. Comput Ind, 178: 104467.

[32] Zou Y , Jeong J , Pemula L , et al., 2022. Spot-the-difference selfsupervised pre-training for anomaly detection and segmentation. Proc 17^th European Conf on Computer Vision, p.392- 408.

Rights & permissions

The Authors. Published by Zhejiang University Press Co., Ltd.