Towards privacy-preserving and efficient word vector learning for lightweight IoT devices

Abstract

Nowadays, Internet of Things (IoT) is widely deployed and brings great opportunities to change people's daily life. To realize more effective human-computer interaction in the IoT applications, the Question Answering (QA) systems implanted in the IoT services are supposed to improve the ability to understand natural language. Therefore, the distributed representation of words, which contains more semantic or syntactic information, has been playing a more and more important role in the QA systems. However, learning high-quality distributed word vectors requires lots of storage and computing resources, hence it cannot be deployed on the resource-constrained IoT devices. It is a good choice to outsource the data and computation to the cloud servers. Nevertheless, it could cause privacy risks to directly upload private data to the untrusted cloud. Therefore, realizing the word vector learning process over untrusted cloud servers without privacy leakage is an urgent and challenging task. In this paper, we present a novel efficient word vector learning scheme over encrypted data. We first design a series of arithmetic computation protocols. Then we use two non-colluding cloud servers to implement high-quality word vectors learning over encrypted data. The proposed scheme allows us to perform training word vectors on the remote cloud servers while protecting privacy. Security analysis and experiments over real data sets demonstrate that our scheme is more secure and efficient than existing privacy-preserving word vector learning schemes.

Keywords

Privacy-preserving / Word vector learning / Secret sharing / Internet of things

References

[1] Yuan Xu, Xingliang Yuan, Baochun Li, Cong Wang, Toward secure and scalable computation in internet of things data applications, IEEE Internet Things J. 6 (2)(2019) 3753-3763.

[2] Fadi Al-Turjman, Muhammad Hassan Nawaz, Umit Deniz Ulusar, Intelligence in the internet of medical things era: a systematic review of current and future trends, Comput. Commun. 150 (2019) 644-660.

[3] Muhammad Faizan Ayub, Khalid Mahmood, Saru Kumari, et al.Arun Kumar Sangaiah, Lightweight authentication protocol for e-health clouds in iot based applications through 5g technology, Digit. Commun. Netw. 7(2) (2020) 235-244.

[4] Thomas K. Landauer, Peter W. Foltz, Darrell Laham, An introduction to latent semantic analysis, Discourse Process 25 (2-3) (1998) 259-284.

[5] Jeffrey Pennington, Richard Socher, Christopher Manning, Glove: global vectors for word representation,in: Conference on Empirical Methods in Natural Language Processing, 2014.

[6] Andriy Mnih, Geoffrey Hinton, Three new graphical models for statistical language modelling, in: Proceedings of the 24th International Conference on Machine Learning, ACM, 2007, pp. 641-648.

[7] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, Jeffrey Dean,Distributed representations of words and phrases and their compositionality, Adv. Neural Inf. Process. Syst. 26 (2013) 3111-3119.

[8] Guangquan Xu, Yuyang Zhao, Litao Jiao, Meiqi Feng, Ji Zhong, Emmanouil Panaousis, Si Chen, Xi Zheng, Tt-svd: an efficient sparse decision making model with two-way trust recommendation in the ai enabled iot systems, IEEE Internet Things J. 8(12) (2020) 9559-9567.

[9] Meng Shen, Baoli Ma, Liehuang Zhu, Xiaojiang Du, Ke Xu, Secure phrase search for intelligent processing of encrypted data in cloud-based iot, IEEE Internet Things J. 6 (2) (2018) 1998-2008.

[10] Guangquan Xu, Weizhe Wang, Litao Jiao, Xiaotong Li, Kaitai Liang, Xi Zheng, Wenjuan Lian, Hequn Xian, Honghao Gao, Soprotector: safeguard privacy for native so files in evolving mobile iot applications, IEEE Internet Things J. 7 (4) (2019) 2539-2552.

[11] Yang Xu, Ju Ren, Yan Zhang, Cheng Zhang, Bo Shen, Yaoxue Zhang, Blockchain empowered arbitrable data auditing scheme for network storage as a service, IEEE Trans. Serv. Comput. 13 (2) (2020) 289-300.

[12] Maqbool Ali, Hafiz Syed Muhammad Bilal, Muhammad Asif Razzaq, Jawad Khan, Sungyoung Lee, Muhammad Idris, Mohammad Aazam, Taebong Choi, Soyeon caren han, and byeong Ho kang. Iotflip: iot-based flipped learning platform for medical education, Digit. commun. netw. 3 (3) (2017) 188-194.

[13] Dierckx Paul, Curve and Surface Fitting with Splines, Oxford University Press, 1995.

[14] Pengtao Xie, Misha Bilenko, Tom Finley, Ran Gilad-Bachrach, Kristin Lauter, Michael Naehrig, Crypto-nets: Neural Networks over Encrypted Data, 2014 arXiv preprint arXiv:1412.6181.

[15] Ximeng Liu, Robert Deng,Kim-Kwang Raymond Choo, Yang Yang, Privacy-preserving Outsourced Clinical Decision Support System in the Cloud, IEEE Transactions on Services Computing, 2017.

[16] Payman Mohassel, Yupeng Zhang, Secureml: a system for scalable privacy-preserving machine learning, in: 2017 IEEE Symposium on Security and Privacy (SP), IEEE, 2017, pp. 19-38.

[17] Valeria Nikolaenko, Stratis Ioannidis, Udi Weinsberg, Marc Joye, Nina Taft, Dan Boneh,Privacy-preserving matrix factorization, in:Proceedings of the 2013 ACM SIGSAC Conference on Computer & Communications Security, 2013, pp. 801-812.

[18] Martin Abadi, Andy Chu, Ian Goodfellow, Ilya Mironov, Kunal Talwar, Li Zhang,H Brendan McMahan, Deep learning with differential privacy, in:Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, 2016, pp. 308-318.

[19] NhatHai Phan, Yue Wang, Xintao Wu, Dejing Dou, Differential privacy preservation for deep auto-encoders: an application of human behavior prediction,in: Thirtieth AAAI Conference on Artificial Intelligence, 2016.

[20] Max Friedrich, Arne Köhn, Gregor Wiedemann, Chris Biemann, Adversarial Learning of Privacy-Preserving Text Representations for De-identification of Medical Records, 2019 arXiv preprint arXiv: 1906.05000.

[21] Ghazaleh Beigi, Kai Shu, Ruocheng Guo, Suhang Wang, Huan Liu,Privacy preserving text representation learning, in:Proceedings of the 30th ACM Conference on Hypertext and Social Media, 2019, pp. 275-276.

[22] Qian Wang, Minxin Du, Xiuying Chen, Yanjiao Chen, Zhou Pan, Xiaofeng Chen, Xinyi Huang, Privacy-preserving collaborative model learning: the case of word vector training, IEEE Trans. Knowl. Data Eng. 30 (12) (2018) 2381-2393.

[23] Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean, Efficient Estimation of Word Representations in Vector Space, 2013 arXiv preprint arXiv:1301.3781.

[24] Michael U. Gutmann, Aapo Hyvärinen, Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics, J. Mach. Learn. Res. 13 (Feb) (2012) 307-361.

[25] Andrew Chi-Chih Yao, Protocols for secure computations, FOCS 82 (1982) 160-164.

[26] Donald Beaver, Efficient multiparty protocols using circuit randomization, in:Annual International Cryptology Conference, Springer, 1991, pp. 420-432.

[27] Ivan Damgård, Matthias Fitzi, Eike Kiltz,Jesper Buus Nielsen, Tomas Toft, Unconditionally secure constant-rounds multi-party computation for equality, comparison, bits and exponentiation,in:Theory of Cryptography Conference, Springer, 2006, pp. 285-304.

[28] Kai Huang, Ximeng Liu, Shaojing Fu, Deke Guo, Ming Xu, A lightweight privacy-preserving cnn feature extraction framework for mobile sensing, IEEE Trans. Dependable Secure Comput. 18(3) (2019) 1441-1455.

[29] Milton Abramowitz, Irene A. Stegun, Handbook of Mathematical Functions:with Formulas, Graphs, and Mathematical Tables, vol. 55, Courier Corporation, 1965.

[30] Fevzullah Temurtas, Gulbag Ali, Nejat Yumusak, A study on neural networks using taylor series expansion of sigmoid activation function, in:International Conference on Computational Science and its Applications, Springer, 2004, pp. 389-397.

[31] Canetti Ran, Asaf Cohen, Yehuda Lindell, A simpler variant of universally composable security for standard multiparty computation, in:Annual Cryptology Conference, Springer, 2015, pp. 3-22.