Defense against data poisoning attacks in robot vision systems based on adversarial example detection

Abstract

Robot vision systems are integral to the autonomous functioning of robots, enabling tasks such as object recognition, navigation, and interaction with the environment. Nonetheless, these systems are highly prone to data poisoning and adversarial attacks, which can undermine their effectiveness and reliability. This paper investigates the relationship between these two types of attacks, with a particular focus on their similarities in feature space distribution and sensitivity to mutations in robot vision models. By enhancing existing adversarial example detection methods, we make them more effective at defending against data poisoning attacks in robot vision systems. Experimental results show that our improved defense methods not only protect against various types of data poisoning attacks but often outperform techniques specifically designed for such attacks, significantly enhancing the robustness and security of robot vision systems in real-world scenarios.

Graphical abstract

Keywords

poisoning attacks / adversarial example detection / adversarial attacks / robotic vision systems / artificial intelligence

References

[1] Li X. The good, the bad and the ugly: exploring the robustness and applicability of adversarial machine learning. The Pennsylvania State University, Dissertation, 2022

[2] Krizhevsky A, Sutskever I, Hinton G E . Imagenet classification with deep convolutional neural networks. Communications of the ACM, 2017, 60( 6): 84–90

[3] Sainath T N, Kingsbury B, Mohamed A R, Dahl G E, Saon G, Soltau H, Beran T, Aravkin A Y, Ramabhadran B. Improvements to deep convolutional neural networks for LVCSR. In: Proceedings of 2013 IEEE Workshop on Automatic Speech Recognition and Understanding. 2013, 315–320

[4] Deng L, Hinton G, Kingsbury B. New types of deep neural network learning for speech recognition and related applications: an overview. In: Proceedings of 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. 2013, 8599–8603

[5] Amodei D, Ananthanarayanan S, Anubhai R, Bai J, Battenberg E, , . Deep speech 2: end-to-end speech recognition in English and mandarin. In: Proceedings of the 33nd International Conference on Machine Learning. 2016, 173–182

[6] Luong M T, Pham H, Manning C D. Effective approaches to attention-based neural machine translation. In: Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing. 2015, 1412–1421

[7] Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I, Fergus R. Intriguing properties of neural networks. In: Proceedings of the 2nd International Conference on Learning Representations. 2014

[8] Goodfellow I J, Shlens J, Szegedy C. Explaining and harnessing adversarial examples. In: Proceedings of the 3rd International Conference on Learning Representations. 2015

[9] Metzen J H, Genewein T, Fischer V, Bischoff B. On detecting adversarial perturbations. In: Proceedings of the 5th International Conference on Learning Representations. 2017

[10] Feinman R, Curtin R R, Shintre S, Gardner A B. Detecting adversarial samples from artifacts. 2017, arXiv preprint arXiv: 1703.00410

[11] Kim M H, Nguyen T A, Min D. An efficient personal key recovery in self-sovereign identity environments. In: Proceedings of the 15th International Conference on Advanced Computational Intelligence. 2023, 1–8

[12] Xie C, Wang J, Zhang Z, Zhou Y, Xie L, Yuille A. Adversarial examples for semantic segmentation and object detection. Proceedings of the IEEE International Conference on Computer Vision. 2017, 1369–1378

[13] Feltus C. LogicGAN–based data augmentation approach to improve adversarial attack DNN classifiers. In: Proceedings of 2021 International Conference on Computational Science and Computational Intelligence. 2021, 180–185

[14] Liang T, Hu S, Shi Z. Deep neural network copyright protection method based on fingerprinted adversarial samples. In: Proceedings of 2023 IEEE International Conference on Control, Electronics and Computer Technology. 2023, 1086–1092

[15] Wang J, Dong G, Sun J, Wang X, Zhang P. Adversarial sample detection for deep neural network through model mutation testing. In: Proceedings of the 41st IEEE/ACM International Conference on Software Engineering. 2019, 1245–1256

[16] Xu W, Evans D, Qi Y. Feature squeezing: detecting adversarial examples in deep neural networks. In: Proceedings of the 25th Annual Network and Distributed System Security Symposium. 2018

[17] Han K, Xia B, Li Y . (AD)²: adversarial domain adaptation to defense with adversarial perturbation removal. Pattern Recognition, 2022, 122: 108303

[18] Goldblum M, Tsipras D, Xie C, Chen X, Schwarzschild A, Song D, Madry A, Li B, Goldstein T . Dataset security for machine learning: data poisoning, backdoor attacks, and defenses. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45( 2): 1563–1580

[19] Liu M, Zhang H, Liu Z, Zhao N . Attacking spectrum sensing with adversarial deep learning in cognitive radio-enabled internet of things. IEEE Transactions on Reliability, 2023, 72( 2): 431–444

[20] Chen J, Gao Y, Liu G, Abdelmoniem A M, Wang C . Manipulating pre-trained encoder for targeted poisoning attacks in contrastive learning. IEEE Transactions on Information Forensics and Security, 2024, 19: 2412–2424

[21] Chen Y, Zhu X, Gong X, Yi X, Li S . Data poisoning attacks in internet-of-vehicle networks: taxonomy, state-of-the-art, and future directions. IEEE Transactions on Industrial Informatics, 2023, 19( 1): 20–28

[22] Monkam G F, De Lucia M J, Bastian N D . A topological data analysis approach for detecting data poisoning attacks against machine learning based network intrusion detection systems. Computers & Security, 2024, 144: 103929

[23] Biggio B, Nelson B, Laskov P. Support vector machines under adversarial label noise. In: Proceedings of the 3rd Asian Conference on Machine Learning. 2011, 97–112

[24] Zhu C, Huang R, Li H, Taylor G, Studer C, Goldstein T. Transferable clean-label poisoning attacks on deep neural nets. In: Proceedings of the 36th International Conference on Machine Learning. 2019, 7614–7623

[25] Wu S, Chen S, Xie C, Huang X. One-pixel shortcut: on the learning preference of deep neural networks. In: Proceedings of the 11th International Conference on Learning Representations. 2023

[26] Steinhardt J, Koh P W, Liang P. Certified defenses for data poisoning attacks. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 3520–3532

[27] Zhang X, Zhu X, Wright S. Training set debugging using trusted items. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018, 1–8

[28] Diakonikolas I, Kamath G, Kane D, Li J, Steinhardt J, Stewart A. Sever: a robust meta-algorithm for stochastic optimization. In: Proceedings of the 36th International Conference on Machine Learning. 2019, 1596–1606

[29] Peri N, Gupta N, Huang W R, Fowl L, Zhu C, Feizi S, Goldstein T, Dickerson J P. Deep k-NN defense against clean-label data poisoning attacks. In: Proceedings of the Computer Vision–ECCV 2020 Workshops. 2020, 55–70

[30] Chen J, Zhang X, Zhang R, Wang C, Liu L . De-pois: an attack-agnostic defense against data poisoning attacks. IEEE Transactions on Information Forensics and Security, 2021, 16: 3412–3425

[31] Weng C H, Lee Y T, Wu S H. On the trade-off between adversarial and backdoor robustness. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 1004

[32] Pang R, Shen H, Zhang X, Ji S, Vorobeychik Y, Luo X, Liu A, Wang T. A tale of evil twins: adversarial inputs versus poisoned models. In: Proceedings of 2020 ACM SIGSAC Conference on Computer and Communications Security. 2020, 85–99

[33] Jin K, Zhang T, Shen C, Chen Y, Fan M, Lin C, Liu T . Can we mitigate backdoor attack using adversarial detection methods?. IEEE Transactions on Dependable and Secure Computing, 2023, 20( 20): 2867–2881

[34] Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A. Towards deep learning models resistant to adversarial attacks. In: Proceedings of the 6th International Conference on Learning Representations. 2018

[35] Liu X, Xie L, Wang Y, Zou J, Xiong J, Ying Z, Vasilakos A V . Privacy and security issues in deep learning: a survey. IEEE Access, 2021, 9: 4566–4593

[36] Wang Y, Su H, Zhang B, Hu X . Interpret neural networks by extracting critical subnetworks. IEEE Transactions on Image Processing, 2020, 29: 6707–6720

[37] Zhang P, Cao Y, Zhu C, Zhuang Y, Wang H, Li J . DefenseFea: an input transformation feature searching algorithm based latent space for adversarial defense. Foundations of Computing and Decision Sciences, 2024, 49( 1): 21–36

[38] Zhang Y, Zhang Y, Zhang Z, Bai H, Zhong T, Song M. Evaluation of data poisoning attacks on federated learning-based network intrusion detection system. In: Proceedings of the 24th IEEE International Conference on High Performance Computing & Communications; 8th International Conference on Data Science & Systems; 20th International Conference on Smart City; 8th International Conference on Dependability in Sensor, Cloud & Big Data Systems & Application. 2022, 2235–2242

[39] Shan T J, Wax M, Kailath T . On spatial smoothing for direction-of-arrival estimation of coherent signals. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1985, 33( 4): 806–811

[40] Liu W, Zeng W, Dong L, Yao Q . Efficient compression of encrypted grayscale images. IEEE Transactions on Image Processing, 2010, 19( 4): 1097–1102

[41] Wallace G K . The JPEG still picture compression standard. Communications of the ACM, 1991, 34( 4): 30–44

[42] Zhang Q, Xiao L, Shi Y . Extraction and classification of mouth shape features in oral English teaching based on image processing. Traitement du Signal, 2021, 38( 4): 1013–1021

[43] Farkaš I, Masulli P, Wermter S. Artificial Neural Networks and Machine Learning–ICANN 2020: 29th International Conference on Artificial Neural Networks, Bratislava, Slovakia, September 15–18, 2020, Proceedings, Part II. Berlin, Heidelberg: Springer, 2020

[44] Abdi H, Williams L J . Principal component analysis. WIREs Computational Statistics, 2010, 2( 4): 433–459

[45] Lee T W. Independent component analysis. In: Lee T W, ed. Independent Component Analysis: Theory and Applications. Boston: Springer, 1998, 27–66

[46] Yu H, Kang C, Xiao Y, Yang Y . Network intrusion detection method based on hybrid improved residual network blocks and bidirectional gated recurrent units. IEEE Access, 2023, 11: 68961–68971

[47] Alzate C, Monreale A, Bioglio L, Bitetta V, Bordino I, Caldarelli G, Ferretti A, Guidotti R, Gullo F, Pascolutti S, Pensa R G, Robardet C, Squartini T. ECML PKDD 2018 Workshops. Cham: Springer, 2019

[48] Ma L, Zhang F, Sun J, Xue M, Li B, Juefei-Xu F, Xie C, Li L, Liu Y, Zhao J, Wang Y. DeepMutation: mutation testing of deep learning systems. In: Proceedings of the 29th IEEE International Symposium on Software Reliability Engineering. 2018, 100–111

[49] Shi Y, Yin B, Zheng Z, Li T. An empirical study on test case prioritization metrics for deep neural networks. In: Proceedings of the 21st IEEE International Conference on Software Quality, Reliability and Security. 2021, 157–166

[50] Wang S, Liu W, Chang C H. Detecting adversarial examples for deep neural networks via layer directed discriminative noise injection. In: Proceedings of 2019 Asian Hardware Oriented Security and Trust Symposium. 2019, 1–6

[51] Li H, Zhang X, Duan S, Liang H . Speech emotion recognition based on bi-directional acoustic-articulatory conversion. Knowledge-Based Systems, 2024, 299: 112123

[52] Gu W. Watermark removal scheme based on neural network model pruning. In: Proceedings of the 5th International Conference on Machine Learning and Natural Language Processing. 2022, 377–382

[53] Chicco D, Jurman G . The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics, 2020, 21( 1): 6

[54] Krizhevsky A. Learning multiple layers of features from tiny images. Toronto: University of Toronto, 2009

[55] Yu Y F, Ren C X, Jiang M, Sun M Y, Dai D Q, Guo G . Sparse approximation to discriminant projection learning and application to image classification. Pattern Recognition, 2019, 96: 106963

[56] Wang Z, Yan M, Chen J, Liu S, Zhang D. Deep learning library testing via effective model generation. In: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2020. 788–799

[57] Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg A C, Fei-Fei L. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 2015, 115(3): 211–252

[58] Guo J, Yang Y, Li H, Wang J, Tang A, Shan D, Huang B . A hybrid deep learning model towards fault diagnosis of drilling pump. Applied Energy, 2024, 372: 123773

[59] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016, 770–778

[60] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: Proceedings of the 3rd International Conference on Learning Representations. 2015

[61] Huang G, Liu Z, Van Der Maaten L, Weinberger K Q. Densely connected convolutional networks. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017, 4700–4708

[62] Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L C. MobileNetV2: inverted residuals and linear bottlenecks. In: Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 4510–4520

[63] Butt U J, Hussien O, Hasanaj K, Shaalan K, Hassan B, Al-Khateeb H . Predicting the impact of data poisoning attacks in blockchain-enabled supply chain networks. Algorithms, 2023, 16( 12): 549

[64] Yi S, Zhou L, Ma L, Shao D . MTRA-CNN: a multi-scale transfer learning framework for glaucoma classification in retinal fundus images. IEEE Access, 2023, 11: 142689–142701

[65] Jiang Y, Liang L, Li Q. Black-box speech adversarial attack with genetic algorithm and generic attack ideas. In: Proceedings of 2023 IEEE International Conference on Dependable, Autonomic and Secure Computing, International Conference on Pervasive Intelligence and Computing, International Conference on Cloud and Big Data Computing, International Conference on Cyber Science and Technology Congress. 2023, 0860–0867

RIGHTS & PERMISSIONS

Higher Education Press