An overview of machine unlearning

Abstract

Nowadays, machine learning is widely used in various applications. Training a model requires huge amounts of data, but it can pose a threat to user privacy. With the growing concern for privacy, the “Right to be Forgotten” has been proposed, which means that users have the right to request that their personal information be removed from machine learning models. The emergence of machine unlearning is a response to this need. Implementing machine unlearning is not easy because simply deleting samples from a database does not allow the model to “forget” the data. Therefore, this paper summarises the definition of the machine unlearning formulation, process, deletion requests, design requirements and validation, algorithms, applications, and future perspectives, in the hope that it will help future researchers in machine unlearning.

Keywords

Machine unlearning / Unlearning definition / Unlearning requirements and validation / Unlearning algorithms

CRediT authorship contribution statement

Chunxiao Li: Methodology. Haipeng Jiang: Data curation. Jiankang Chen: Formal analysis. Yu Zhao: Validation. Shuxuan Fu: Software & evalation. Fangming Jing: Project administration. Yu Guo: Supervision.

Declaration of competing interest

We declare that there are no competing interests associated with this manuscript. No financial, personal, or professional conflicts have influenced the research and findings presented in this study. This research was conducted independently, and the results presented are the authors’ honest and unbiased findings.

Acknowledgements

This research was supported by the National Natural Science Foundation of China (62102035) and the National Key Research and Development Program of China (2022ZD0115901).

References

[1] Y. Cao, J. Yang, Towards making systems forget with machine unlearn-ing, in: 2015 IEEE Symposium on Security and Privacy, IEEE, 2015, pp. 463-480.

[2] N.G. Marchant, B.I. Rubinstein, S. Alfeld, Hard to forget: Poisoning attacks on certified machine unlearning,in: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, (no. 7) 2022, pp. 7691-7700.

[3] A. Singh, A. Anand, Data leakage detection using cloud computing, Int. J. Eng. Comput. Sci. 6 (4) (2017).

[4] R. Wang, Y.F. Li, X. Wang, H. Tang, X. Zhou, Learning your identity and disease from research papers: Information leaks in genome wide associa-tion study,in: Proceedings of the 16th ACM Conference on Computer and Communications Security, 2009, pp. 534-544.

[5] L. Bourtoule, V. Chandrasekaran, C.A. Choquette-Choo, H. Jia, A. Travers, B. Zhang, D. Lie, N. Papernot, Machine unlearning, in: 2021 IEEE Symposium on Security and Privacy, SP, IEEE, 2021, pp. 141-159.

[6] Q.P. Nguyen, R. Oikawa, D.M. Divakaran, M.C. Chan, B.K.H. Low, Markov chain Monte Carlo-based machine unlearning: Unlearning what needs to be forgotten,in: Proceedings of the 2022 ACM on Asia Conference on Computer and Communications Security, 2022, pp. 351-363.

[7] Q.P. Nguyen, B.K.H. Low, P. Jaillet, Variational Bayesian unlearning, Adv. Neural Inf. Process. Syst. 33 (2020) 16025-16036.

[8] M. Du, Z. Chen, C. Liu, R. Oak, D. Song, Lifelong anomaly detection through unlearning, in:Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, 2019, pp. 1283-1297.

[9] A. Golatkar, A. Achille, S. Soatto, Eternal sunshine of the spotless net: Selective forgetting in deep networks,in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 9304-9312.

[10] S. Garg, S. Goldwasser, P.N. Vasudevan, Formalizing data deletion in the context of the right to be forgotten, in: Annual International Conference on the Theory and Applications of Cryptographic Techniques, Springer, 2020, pp. 373-402.

[11] A. Ginart, M. Guan, G. Valiant, J.Y. Zou, Making ai forget you: Data dele-tion in machine learning, in: Advances in Neural Information Processing Systems, vol. 32, 2019.

[12] A. Sekhari, J. Acharya, G. Kamath, A.T. Suresh, Remember what you want to forget: Algorithms for machine unlearning, Adv. Neural Inf. Process. Syst. 34 (2021) 18075-18086.

[13] C. Guo, T. Goldstein, A. Hannun, L. Van Der Maaten, Certified data removal from machine learning models, 2019, arXiv preprint arXiv:1911.03030.

[14] S. Neel, A. Roth, S. Sharifi-Malvajerdi, Descent-to-delete: Gradient-based methods for machine unlearning, in: Algorithmic Learning Theory, PMLR, 2021, pp. 931-962.

[15] J. Brophy, D. Lowd, Machine unlearning for random forests, in: International Conference on Machine Learning, PMLR, 2021, pp. 1092-1104.

[16] A. Thudi, G. Deza, V. Chandrasekaran, N. Papernot, Unrolling sgd: Un-derstanding factors influencing machine unlearning, in: 2022 IEEE 7th European Symposium on Security and Privacy, EuroS&P, IEEE, 2022, pp. 303-319.

[17] R. Shokri, M. Stronati, C. Song, V. Shmatikov, Membership inference attacks against machine learning models, in: 2017 IEEE Symposium on Security and Privacy, SP, IEEE, 2017, pp. 3-18.

[18] A. Warnecke, L. Pirch, C. Wressnegger, K. Rieck, Machine unlearning of features and labels, 2021, arXiv preprint arXiv:2108.11577.

[19] T. Guo, S. Guo, J. Zhang, W. Xu, J. Wang, Efficient attribute unlearning: Towards selective removal of input attributes from feature representations, 2022, arXiv preprint arXiv:2202.13295.

[20] J. Tanha, Y. Abdi, N. Samadi, N. Razzaghi, M. Asadpour, Boosting methods for multi-class imbalanced data classification: An experimental review, J. Big Data 7 (2020) 1-47.

[21] A.K. Tarun, V.S. Chundawat, M. Mandal, M. Kankanhalli, Fast yet effective machine unlearning, IEEE Trans. Neural Netw. Learn. Syst. (2023).

[22] I. Masi, Y. Wu, T. Hassner, P. Natarajan, Deep face recognition: A survey, in: 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images, SIBGRAPI, IEEE, 2018, pp. 471-478.

[23] D.M. Sommer, L. Song, S. Wagh, P. Mittal, Towards probabilistic verification of machine unlearning, 2020, arXiv preprint arXiv:2003.04247.

[24] Y. Guo, Y. Zhao, S. Hou, C. Wang, X. Jia, Verifying in the dark: Verifiable machine unlearning by using invisible backdoor triggers, IEEE Trans. Inf. Forensics Secur. (2023).

[25] Y. He, G. Meng, K. Chen, J. He, X. Hu, Deepobliviate: a powerful charm for erasing data residual memory in deep neural networks, 2021, arXiv preprint arXiv:2105.06209.

[26] A. Thudi, H. Jia, I. Shumailov, N. Papernot, On the necessity of auditable algorithmic definitions for machine unlearning, in: 31st USENIX Security Symposium, USENIX Security 22, 2022, pp. 4007-4022.

[27] M. Chen, Z. Zhang, T. Wang, M. Backes, M. Humbert, Y. Zhang,When machine unlearning jeopardizes privacy, in:Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, 2021, pp. 896-911.

[28] A. Thudi, I. Shumailov, F. Boenisch, N. Papernot, Bounding membership inference, 2022, arXiv preprint arXiv:2202.12232.

[29] B. Wang, Y. Yao, S. Shan, H. Li, B. Viswanath, H. Zheng, B.Y. Zhao, Neural cleanse: Identifying and mitigating backdoor attacks in neural networks, in: 2019 IEEE Symposium on Security and Privacy, SP, IEEE, 2019, pp. 707-723.

[30] D.M. Sommer, L. Song, S. Wagh, P. Mittal, Athena: Probabilistic verification of machine unlearning, Proc. Priv. Enhanc. Technol. 3 (2022) 268-290.

[31] K. Chen, Y. Huang, Y. Wang, Machine unlearning via GAN, 2021, arXiv preprint arXiv:2111.11869.

[32] J. Wang, S. Guo, X. Xie, H. Qi, Federated unlearning via class-discriminative pruning, in:Proceedings of the ACM Web Conference 2022, 2022, pp. 622-632.

[33] T. Baumhauer, P. Schöttle, M. Zeppelzauer, Machine unlearning: Lin-ear filtration for logit-based classifiers, Mach. Learn. 111 (9) (2022) 3203-3226.

[34] Z. Izzo, M. Anne Smart, K. Chaudhuri, J. Zou, Approximate data deletion from machine learning models, in: A. Banerjee, K. Fukumizu (Eds.), Pro-ceedings of the 24th International Conference on Artificial Intelligence and Statistics, in: Proceedings of Machine Learning Research, vol. 130, PMLR, 2021, pp. 2008-2016.

[35] P. Geurts, D. Ernst, L. Wehenkel, Extremely randomized trees, Mach. Learn. 63 (2006) 3-42.

[36] S. Schelter, S. Grafberger, T. Dunning, Hedgecut: Maintaining randomised trees for low-latency machine unlearning,in: Proceedings of the 2021 International Conference on Management of Data, 2021, pp. 1545-1557.

[37] Z. Cao, J. Wang, S. Si, Z. Huang, J. Xiao, Machine unlearning method based on projection residual, in: 2022 IEEE 9th International Conference on Data Science and Advanced Analytics, DSAA, IEEE, 2022, pp. 1-8.

[38] A. Golatkar, A. Achille, A. Ravichandran, M. Polito, S. Soatto, Mixed-privacy forgetting in deep networks, in:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 792-801.

[39] H. Huang, X. Ma, S.M. Erfani, J. Bailey, Y. Wang, Unlearnable examples: Making personal data unexploitable, 2021, arXiv preprint arXiv:2101. 04898.

[40] G. Wu, M. Hashemi, C. Srinivasa, Puma: Performance unchanged model augmentation for training data removal,in: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, (no. 8) 2022, pp. 8675-8682.

[41] G. Liu, X. Ma, Y. Yang, C. Wang, J. Liu, Federaser: Enabling efficient client-level data removal from federated learning models, in: 2021 IEEE/ACM 29th International Symposium on Quality of Service, IWQOS, IEEE, 2021, pp. 1-10.

[42] C. Wu, S. Zhu, P. Mitra, Federated unlearning with knowledge distillation, 2022, arXiv, arXiv preprint arXiv:2201.09441.

[43] Y. Liu, L. Xu, X. Yuan, C. Wang, B. Li, The right to be forgotten in federated learning: An efficient realization with rapid retraining, in: IEEE INFOCOM 2022-IEEE Conference on Computer Communications, IEEE, 2022, pp. 1749-1758.

[44] A.S. Berahas, J. Nocedal, M. Takác, A multi-batch L-BFGS method for machine learning, Adv. Neural Inf. Process. Syst. 29 (2016).

[45] R. Bollapragada, J. Nocedal, D. Mudigere, H.-J. Shi, P.T.P. Tang, A progressive batching L-BFGS method for machine learning, in: International Conference on Machine Learning, PMLR, 2018, pp. 620-629.

[46] B. Liu, Q. Liu, P. Stone, Continual learning and private unlearning, in: Conference on Lifelong Learning Agents, PMLR, 2022, pp. 243-254.

[47] J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A.A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, et al., Overcoming catastrophic forgetting in neural networks, Proc. Natl. Acad. Sci. 114 (13)(2017) 3521-3526.

[48] G.I. Parisi, R. Kemker, J.L. Part, C. Kanan, S. Wermter, Continual lifelong learning with neural networks: A review, Neural Netw. 113 (2019) 54-71.

[49] T.T. Nguyen, T.T. Huynh, P.L. Nguyen, A.W.-C. Liew, H. Yin, Q.V.H. Nguyen, A survey of machine unlearning, 2022, arXiv preprint arXiv:2209.02299.

[50] V.S. Chundawat, A.K. Tarun, M. Mandal, M. Kankanhalli, Zero-shot machine unlearning, IEEE Trans. Inf. Forensics Secur. (2023).

[51] X. Gao, X. Ma, J. Wang, Y. Sun, B. Li, S. Ji, P. Cheng, J. Chen, Verifi: Towards verifiable federated unlearning, 2022, arXiv preprint arXiv:2205.12709.

[52] J. Nam, H. Cha, S. Ahn, J. Lee, J. Shin, Learning from failure: De-biasing classifier from biased classifier, Adv. Neural Inf. Process. Syst. 33 (2020) 20673-20684.

[53] S. Schelter, Amnesia-a selection of machine learning models that can forget user data very fast, suicide 8364 (44035) (2020) 46992.

[54] X. Hu, K. Tang, C. Miao, X.-S. Hua, H. Zhang, Distilling causal effect of data in class-incremental learning, in:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 3957-3966.