Deviation-Guided Attention for Semi-Supervised Anomaly Detection With Contrastive Regularisation

Abstract

Anomaly detection (AD) aims to identify abnormal patterns that deviate from normal behaviour, playing a critical role in applications such as industrial inspection, medical imaging and autonomous driving. However, AD often faces a scarcity of labelled data. To address this challenge, we propose a novel semi-supervised anomaly detection method, DASAD (Deviation- Guided Attention for Semi-Supervised Anomaly Detection), which integrates deviation-guided attention with contrastive reg-ularisation to reduce the unreliability of pseudo-labels. Specifically, a deviation-guided attention mechanism is designed to combine three types of deviations: latent embeddings, residual direction vectors and hierarchical reconstruction errors to capture anomaly specific cues effectively, thereby enhancing the credibility of pseudo-labels for unlabelled samples. Further-more, a class-asymmetric contrastive loss is constructed to promote compact representations of normal instances while pre-serving the structural diversity of anomalies. Extensive experiments on 8 benchmark datasets demonstrate that DASAD consistently outperforms state-of-the-art methods and exhibits strong generalisation across 6 anomaly detection domains.

Keywords

anomaly detection / deep learning / semi-supervised learning

Conflicts of Interest

Xin Xu is an editorial board member for the journal, and was not involved in peer review process or the decision to publish this article. The authors declare that they have no confiict of interest.

Data Availability Statement

The data that support the findings of this study are publicly available: Arrhythmia and Spambase: http://archive.ics.uci.edu/datasets; Cardio, Mammography, Satellite and Shuttle: http://odds.cs.stonybrook.edu; Fraud: shttp://www.ulb.ac.be/di/map/adalpozz/data/creditcard.Rdata and NSL-KDD: http://nsl.cs.unb.ca/NSL-KDD/.

References

[1] K. G. Mehrotra, C. K. Mohan, H. Huang, K. G. Mehrotra, C. K. Mohan, and H. Huang, Anomaly Detection (Springer, 2017).

[2] Z. Li and M. Van Leeuwen, “Explainable Contextual Anomaly Detection Using Quantile Regression Forests,” Data Mining and Knowledge Discovery 37, no. 6 (2023): 2517-2563, https://doi.org/10.1007/s10618-023-00967-z.

[3] W. Shi, D. Karastoyanova, Y. Ma, Y. Huang, and G. Zhang, “Clustering-Based Granular Representation of Time Series With Application to Collective Anomaly Detection,” IEEE Transactions on Instrumentation and Measurement 72 (2023): 1-12, https://doi.org/10.1109/tim.2023.3325521.

[4] J. Liu, G. Xie, J. Wang, et al., “Deep Industrial Image Anomaly Detection: A Survey,” Machine Intelligence Research 21, no. 1 (2024): 104-135, https://doi.org/10.1007/s11633-023-1459-z.

[5] Y. Wang, J. Peng, J. Zhang, R. Yi,Y. Wang, and C. Wang, “Multi-modal Industrial Anomaly Detection via Hybrid Fusion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023), 8032-8041.

[6] J. Bao, H. Sun, H. Deng, Y. He,Z. Zhang, and X. Li, “BMAD: Benchmarks for Medical Anomaly Detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024), 4042-4053.

[7] Y. Cai, W. Zhang, H. Chen, and K.-T. Cheng, “Medianomaly: A Comparative Study of Anomaly Detection in Medical Images,” Medical Image Analysis 102 (2025): 103500, https://doi.org/10.1016/j.media.2025.103500.

[8] W. Hilal, S. A. Gadsden, and J. Yawney, “Financial Fraud: A Review of Anomaly Detection Techniques and Recent Advances,” Expert Sys-tems with Applications 193 (2022): 116429, https://doi.org/10.1016/j.eswa.2021.116429.

[9] P. Vanini, S. Rossi, E. Zvizdic, and T. Domenig, “Online Payment Fraud: From Anomaly Detection to Risk Management,” Financial Inno-vation 9, no. 1 (2023): 66, https://doi.org/10.1186/s40854-023-00470-w.

[10] D. Bogdoll,M. Nitsche, and J. M. Zöllner, “Anomaly Detection in Autonomous Driving: A Survey,” in Proceedings of the IEEE/CVF Con-ference on Computer Vision and Pattern Recognition (2022), 4488-4499.

[11] Z. Wang, L. Xue, X. Luo, X. Ma, and G. Gu, “Driving State-Aware Anomaly Detection for Autonomous Vehicles,” IEEE Transactions on Information Forensics and Security 20 (2025): 3788-3803, https://doi.org/10.1109/tifs.2025.3553370.

[12] H. Dong, G. Frusque, Y. Zhao, E. Chatzi, and O. Fink, “NNG-Mix: Improving Semi-Supervised Anomaly Detection With Pseudo-Anomaly Generation,” IEEE Transactions on Neural Networks and Learning Sys-tems 36, no. 6 (2024): 10635-10647, https://doi.org/10.1109/tnnls.2024.3497801.

[13] F. Zhou, G. Wang, K. Zhang, S. Liu, and T. Zhong, “Semi- Supervised Anomaly Detection via Neural Process,” IEEE Transactions on Knowledge and Data Engineering 35, no. 10 (2023): 10.423-10.435, https://doi.org/10.1109/tkde.2023.3266755.

[14] D. Gong, L. Liu, V. Le, et al., “Memorizing Normality to Detect Anomaly: Memory-Augmented Deep Autoencoder for Unsupervised Anomaly Detection,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (2019), 1705-1714.

[15] Z. Yang, T. Zhang, I. S. Bozchalooi, and E. Darve, “Memory- Augmented Generative Adversarial Networks for Anomaly Detection,” IEEE Transactions on Neural Networks and Learning Systems 33, no. 6 (2021): 2324-2334, https://doi.org/10.1109/tnnls.2021.3132928.

[16] Y. Zhou, X. Song, Y. Zhang, F. Liu, C. Zhu, and L. Liu, “Feature Encoding With Autoencoders for Weakly Supervised Anomaly Detec-tion,” IEEE Transactions on Neural Networks and Learning Systems 33, no. 6 (2022): 2454-2465, https://doi.org/10.1109/tnnls.2021.3086137.

[17] B. Zong, Q. Song, M. R. Min, et al., “Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection,” in International Conference on Learning Representations (2018), 1-14.

[18] C. Zhou and R. C. Paffenroth, “Anomaly Detection With Robust Deep Autoencoders,” in Proceedings of the 23rd ACM SIGKDD Inter-national Conference on Knowledge Discovery and Data Mining (2017), 665-674.

[19] H. Gan, H. Zheng, Z. Wu, C. Ma, and J. Liu, “TFD-Net: Transformer Deviation Network for Weakly Supervised Anomaly Detection,” IEEE Transactions on Network and Service Management 22, no. 1 (2025): 941-954, https://doi.org/10.1109/tnsm.2024.3485545.

[20] E. Eldele, M. Ragab, Z. Chen, et al., “Self-Supervised Contrastive Representation Learning for Semi-Supervised Time-Series Classifica-tion,” IEEE Transactions on Pattern Analysis and Machine Intelligence 45, no. 12 (2023): 15.604-15.618, https://doi.org/10.1109/tpami.2023.3308189.

[21] Y. Wang, X. Du, Z. Lu, et al., “Improved LSTM-Based Time-Series Anomaly Detection in Rail Transit Operation Environments,” IEEE Transactions on Industrial Informatics 18, no. 12 (2022): 9027-9036, https://doi.org/10.1109/TII.2022.3164087.

[22] J. Liu, X. Song, Y. Zhou, et al., “Deep Anomaly Detection in Packet Payload,” Neurocomputing 485 (2022): 205-218, https://doi.org/10.1016/j.neucom.2021.01.146.

[23] K. Yang, J. Ren, Y. Zhu, et al., “Active Learning for Wireless IoT Intrusion Detection,” IEEE Wireless Communications 25, no. 6 (2018): 19-25, https://doi.org/10.1109/MWC.2017.1800079.

[24] Z. Liu, Y. Zhou,Y. Xu, and Z. Wang, “Simplenet: A Simple Network for Image Anomaly Detection and Localization,” in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023), 20.402-20.411.

[25] G. Guo, H. Wang, D. Bell, Y. Bi, and K. Greer, “Knn Model-Based Approach in Classification,” in On the Move to Meaningful Internet Systems 2003: CoopIS, DOA, and ODBASE: OTM Confederated Interna-tional Conferences, CoopIS, DOA, and ODBASE 2003, Catania, Sicily, Italy, November 3-7, 2003. Proceedings (Springer, 2003), 986-996.

[26] M. X. Ma, H. Y. Ngan, and W. Liu, “Density-Based Outlier Detection by Local Outlier Factor on Largescale Traffic Data,” Electronic Imaging 28, no. 14 (2016): 1-4, https://doi.org/10.2352/issn.2470-1173.2016.14.ipmva-385.

[27] F. T. Liu, K. M. Ting, and Z.-H. Zhou, “Isolation Forest,” in 2008 Eighth IEEE International Conference on Data Mining IEEE, 2008), 413-422.

[28] Z. Chen, C. K. Yeo, B. S. Lee, and C. T. Lau, “Autoencoder-Based Network Anomaly Detection ” in 2018 Wireless Telecommunications Symposium (WTS) IEEE, 2018), 1-5.

[29] Y. Zhou, X. Liang, W. Zhang, L. Zhang, and X. Song, “VAE-Based Deep SVDD for Anomaly Detection,” Neurocomputing 453 (2021): 131-140, https://doi.org/10.1016/j.neucom.2021.04.089.

[30] H. Xu, W. Chen, N. Zhao, et al., “Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal Kpis in Web Applications,” in Proceedings of the 2018 World Wide Web Conference (2018), 187-196.

[31] H. Zenati, M. Romain, C.-S. Foo, B. Lecouat and V. Chandrasekhar, “Adversarially Learned Anomaly Detection ” in 2018 IEEE International Conference on Data Mining (ICDM) IEEE, 2018), 727-736.

[32] L. Ruff, R. Vandermeulen, N. Goernitz, et al. “Deep One-Class Classification ” , in International Conference on Machine Learning (PMLR, 2018), 4393-4402.

[33] G. Pang, C. Shen, and A. Van Den Hengel, “Deep Anomaly Detection With Deviation Networks,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (2019), 353-362.

[34] L. Ruff, R. A. Vandermeulen, G. Montavon, et al. “Deep Semi- Supervised Anomaly Detection,” , in International Conference on Learning Representations (ICLR) (2020), 1-13.

[35] Z. Li, C. Sun, C. Liu, X. Chen, M. Wang, and Y. Liu, “Dual-MGAN: An Efficient Approach for Semi-Supervised Outlier Detection With Few Identified Anomalies,” ACM Transactions on Knowledge Discovery from Data 16, no. 6 (2022): 1-30, https://doi.org/10.1145/3522690.

[36] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A Simple Framework for Contrastive Learning of Visual Representations,” in Proceedings of the 37th International Conference on Machine Learning, Ser. ICML’20(JMLR. org, 2020).

[37] K. He, H. Fan, Y. Wu,S. Xie, and R. Girshick, “Momentum Contrast for Unsupervised Visual Representation Learning,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2020), 9726-9735.

[38] C. Xu, T. Zhang, D. Zhang, D. Zhang, and J. Han, “Deep Generative Adversarial Reinforcement Learning for Semi-Supervised Segmentation of Low-Contrast and Small Objects in Medical Images,” IEEE Trans-actions on Medical Imaging 43, no. 9 (2024): 3072-3084, https://doi.org/10.1109/tmi.2024.3383716.

[39] D. Zhang, H. Li, D. He, et al., “Unsupervised Pre-Training With Language-Vision Prompts for Low-Data Instance Segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence 47, no. 10 (2025): 8642-8657, https://doi.org/10.1109/tpami.2025.3579469.

[40] D. Cheng, Y. Ji, D. Gong, et al., “Continual All-in-One Adverse Weather Removal With Knowledge Replay on a Unified Network Structure,” IEEE Transactions on Multimedia 26 (2024): 8184-8196, https://doi.org/10.1109/tmm.2024.3377136.

[41] M. Lichman, “UCI Machine Learning Repository ” (University of California, 2013): [Online], https://archive.ics.uci.edu/ml.

[42] S. Rayana, “ODDS Library: Outlier Detection DataSets,” (Stony Brook University, 2016): [Online], https://odds.cs.stonybrook.edu/.

[43] A. Dal Pozzolo, G. Boracchi, O. Caelen, C. Alippi, and G. Bontempi, “Credit Card Fraud Detection: A Realistic Modeling and a Novel Learning Strategy,” IEEE Transactions on Neural Networks and Learning Systems 29, no. 8 (August 2018): 3784-3797, https://doi.org/10.1109/tnnls.2017.2736643.

[44] M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani, “A Detailed Analysis of the KDD Cup 99 Data Set,” in 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications (2009), 1-6.

[45] Y. Zhou, P. Yang, Y. Qu, X. Xu, Z. Sun, and A. Cichocki, “AnoOnly: Semi-Supervised Anomaly Detection With the Only Loss on Anoma-lies,” Expert Systems With Applications 262 (2025): 125597, https://doi.org/10.1016/j.eswa.2024.125597.

[46] D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimi-zation,” in International Conference on Learning Representations (ICLR) (2015): [Online],1-11, https://arxiv.org/abs/1412.6980.

Funding

National Natural Science Foundation of China under Grant(U24A20279)