Data distribution inference attack in federated learning via reinforcement learning support

Abstract

Federated Learning (FL) is currently a widely used collaborative learning framework, and the distinguished feature of FL is that the clients involved in training do not need to share raw data, but only transfer the model parameters to share knowledge, and finally get a global model with improved performance. However, recent studies have found that sharing model parameters may still lead to privacy leakage. From the shared model parameters, local training data can be reconstructed and thus lead to a threat to individual privacy and security. We observed that most of the current attacks are aimed at client-specific data reconstruction, while limited attention is paid to the information leakage of the global model. In our work, we propose a novel FL attack based on shared model parameters that can deduce the data distribution of the global model. Different from other FL attacks that aim to infer individual clients’ raw data, the data distribution inference attack proposed in this work shows that the attackers can have the capability to deduce the data distribution information behind the global model. We argue that such information is valuable since the training data behind a well-trained global model indicates the common knowledge of a specific task, such as social networks and e-commerce applications. To implement such an attack, our key idea is to adopt a deep reinforcement learning approach to guide the attack process, where the RL agent adjusts the pseudo-data distribution automatically until it is similar to the ground truth data distribution. By a carefully designed Markov decision proces (MDP) process, our implementation ensures our attack can have stable performance and experimental results verify the effectiveness of our proposed inference attack.

Keywords

Sharing model parameters / Data distribution attacks / Federated learning / Reinforcement learning

CRediT authorship contribution statement

Dongxiao Yu: Conceptualization, Investigation, Resources, Writing - original draft, Writing - review & editing, Su- pervision. Hengming Zhang: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Writing - original draft, Writing - review & editing, Visual- ization. Yan Huang: Investigation, Visualization. Zhenzhen Xie: Software, Writing - original draft, Writing - review & editing, Supervision, Project administration.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References

[1] Z. He, L. Wang, Z. Cai, Clustered federated learning with adaptive local differential privacy on heterogeneous iot data, IEEE Internet Things J. (2023).

[2] P. Kairouz, H.B. McMahan, B. Avent, A. Bellet, M. Bennis, A.N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings, et al., Advances and open problems in federated learning, Found. Trends Mach. Learn. 14 (1-2) (2021) 1-210.

[3] B. McMahan, E. Moore, D. Ramage, S. Hampson, B.A. y Arcas, Communication-efficient learning of deep networks from decentralized data, in: Artificial Intelligence and Statistics, PMLR, 2017, pp. 1273-1282.

[4] C. Xie, M. Chen, P.-Y. Chen, B. Li, Crfl: Certifiably robust federated learning against backdoor attacks,in:International Conference on Machine Learning, PMLR, 2021, pp. 11372-11382.

[5] S. Wang, X. Jiang, Y. Wu, L. Cui, S. Cheng, L. Ohno-Machado, Expectation propagation logistic regression (explorer): distributed privacy-preserving online model learning, J. Biomed. Inform. 46 (3) (2013) 480-496.

[6] X. Han, Q. Zhang, Z. He, Z. Cai, Confidence-based similarity-aware personalized federated learning for autonomous IoT, IEEE Internet Things J. (2023).

[7] L. Lyu, H. Yu, Q. Yang, Threats to federated learning: A survey, 2020, arXiv preprint arXiv:2003.02133.

[8] Z. Xiong, Z. Cai, D. Takabi, W. Li, Privacy threat and defense for federated learning with non-iid data in AIoT, IEEE Trans. Ind. Inform. 18 (2) (2021) 1310-1321.

[9] L. Zhu, Z. Liu, S. Han, Deep leakage from gradients, Adv. Neural Inf. Process Syst. 32 (2019).

[10] B. Zhao, K.R. Mopuri, H. Bilen, Idlg: Improved deep leakage from gradients, 2020, arXiv preprint arXiv:2001.02610.

[11] M. Li, D.G. Andersen, A.J. Smola, K. Yu, Communication efficient distributed machine learning with the parameter server, Adv. Neural Inf. Process. Syst. 27 (2014).

[12] C. Zhang, Y. Xie, H. Bai, B. Yu, W. Li, Y. Gao, A survey on federated learning, Knowl.-Based Syst. 216 (2021) 106775.

[13] A. Sergeev, M.H. Del Balso, Fast and easy distributed deep learning in TensorFlow, 2018, arXiv preprint arXiv:1802.05799.

[14] P. Patarasuk, X. Yuan, Bandwidth optimal all-reduce algorithms for clusters of workstations, J. Parallel Distrib. Comput. 69 (2) (2009) 117-124.

[15] L. Li, Y. Fan, M. Tse, K.-Y. Lin, A review of applications in federated learning, Comput. Ind. Eng. 149 (2020) 106854.

[16] J. Pang, Y. Huang, Z. Xie, Q. Han, Z. Cai, Realizing the heterogeneity: A self-organized federated learning framework for IoT, IEEE Internet Things J. 8 (5) (2020) 3088-3098.

[17] W. Wei, L. Liu, M. Loper, K.-H. Chow, M.E. Gursoy, S. Truex, Y. Wu, A framework for evaluating client privacy leakages in federated learning, in: European Symposium on Research in Computer Security, Springer, 2020, pp. 545-566.

[18] J. Geiping, H. Bauermeister, H. Dröge, M. Moeller, Inverting gradients-how easy is it to break privacy in federated learning? Adv. Neural Inf. Process. Syst. 33 (2020) 16937-16947.

[19] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial networks, Commun. ACM 63 (11) (2020) 139-144.

[20] J. Gui, Z. Sun, Y. Wen, D. Tao, J. Ye, A review on generative adversarial networks: Algorithms, theory, and applications, IEEE Trans. Knowl. Data Eng. (2021).

[21] M. Fredrikson, S. Jha, T. Ristenpart, Model inversion attacks that exploit confidence information and basic countermeasures, in:Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, 2015, pp. 1322-1333.

[22] B. Hitaj, G. Ateniese, F. Perez-Cruz, Deep models under the GAN: information leakage from collaborative deep learning,in:Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, 2017, pp. 603-618.

[23] L. Melis, C. Song, E. De Cristofaro, V. Shmatikov,Exploiting unintended feature leakage in collaborative learning, in: 2019 IEEE Symposium on Security and Privacy, SP, IEEE, 2019, pp. 691-706.

[24] R. Shokri, M. Stronati, C. Song, V. Shmatikov,Membership inference attacks against machine learning models, in: 2017 IEEE Symposium on Security and Privacy, SP, IEEE, 2017, pp. 3-18.

[25] J. Luketina, N. Nardelli, G. Farquhar, J. Foerster, J. Andreas, E. Grefenstette, S. Whiteson, T. Rocktäschel, A survey of reinforcement learning informed by natural language, 2019, arXiv preprint arXiv:1906.03926.

[26] M.M. Afsar, T. Crump, B. Far, Reinforcement learning based recommender systems: A survey, ACM Comput. Surv. 55 (7) (2022) 1-38.

[27] C. Buck, J. Bulian, M. Ciaramita, W. Gajewski, A. Gesmundo, N. Houlsby, W. Wang, Ask the right questions: Active question reformulation with reinforcement learning, 2017, arXiv preprint arXiv:1705.07830.

[28] S. Sohn, S. Lee, J. Choi, H. van Seijen, M. Fatemi, H. Lee, Shortest-path constrained reinforcement learning for sparse reward tasks, 2021, arXiv preprint arXiv:2107.06405.

[29] R. Shah, V. Kumar, Rrl: Resnet as representation for reinforcement learning, 2021, arXiv preprint arXiv:2107.03380.

[30] H. Xu, X. Zhan, X. Zhu, Constraints penalized q-learning for safe offline reinforcement learning, in:Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36, (8) 2022, pp. 8753-8760.

[31] K. Arulkumaran, M.P. Deisenroth, M. Brundage, A.A. Bharath, A brief survey of deep reinforcement learning, 2017, arXiv preprint arXiv:1708.05866.

[32] T.P. Lillicrap, J.J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, D. Wierstra, Continuous control with deep reinforcement learning, 2015, arXiv preprint arXiv:1509.02971.

[33] S. Li, Y. Wu, X. Cui, H. Dong, F. Fang, S. Russell, Robust multi-agent reinforcement learning via minimax deep deterministic policy gradient, in:Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, (01) 2019, pp. 4213-4220.

[34] T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, V. Kumar, H. Zhu, A. Gupta, P. Abbeel, et al., Soft actor-critic algorithms and applications, 2018, arXiv preprint arXiv:1812.05905.

[35] J. Duan, Y. Guan, S.E. Li, Y. Ren, Q. Sun, B. Cheng, Distributional soft actorcritic: Off-policy reinforcement learning for addressing value estimation errors, IEEE Trans. Neural Netw. Learn. Syst. 33 (11) (2021) 6584-6598.

[36] H. Xiao, K. Rasul, R. Vollgraf, Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017, arXiv preprint arXiv: 1708.07747.

[37] X. Li, K. Huang, W. Yang, S. Wang, Z. Zhang, On the convergence of fedavg on non-iid data, 2019, arXiv preprint arXiv:1907.02189.

[38] L. Collins, H. Hassani, A. Mokhtari, S. Shakkottai, Fedavg with fine tuning: Local updates lead to representation learning, Adv. Neural Inf. Process. Syst. 35 (2022) 10572-10586.

[39] X. Yuan, P. Li, On convergence of FedProx: Local dissimilarity invariant bounds, non-smoothness and beyond, Adv. Neural Inf. Process. Syst. 35 (2022) 10752-10765.

[40] Q. Li, B. He, D. Song, Model-contrastive federated learning, in:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 10713-10722.