C-privacy: A social relationship-driven image customization sharing method in cyber-physical networks

Abstract

Cyber-Physical Networks (CPN) are comprehensive systems that integrate information and physical domains, and are widely used in various fields such as online social networking, smart grids, and the Internet of Vehicles (IoV). With the increasing popularity of digital photography and Internet technology, more and more users are sharing images on CPN. However, many images are shared without any privacy processing, exposing hidden privacy risks and making sensitive content easily accessible to Artificial Intelligence(AI) algorithms. Existing image sharing methods lack fine-grained image sharing policies and cannot protect user privacy. To address this issue, we propose a social relationship-driven privacy customization protection model for publishers and co-photographers. We construct a heterogeneous social information network centered on social relationships, introduce a user intimacy evaluation method with time decay, and evaluate privacy levels considering user interest similarity. To protect user privacy while maintaining image appreciation, we design a lightweight face-swapping algorithm based on Generative Adversarial Network (GAN) to swap faces that need to be protected. Our proposed method minimizes the loss of image utility while satisfying privac

Keywords

Cyber-physical networks / Customized privacy / Face-swapping / Heterogeneous information network / Deep fakes

CRediT authorship contribution statement

Dapeng Wu: Resources, Supervision. Jian Liu: Data curation, Formal analysis, Funding acquisition, Resources, Software, Supervision, Validation, Visualization, Writing - original draft, Writing - review & editing. Yangliang Wan: Supervision. Zhigang Yang: Data curation, Formal analysis, Investigation, Resources, Supervision. Ruyan Wang: Project administration, Resources, Supervision, Investigation, Methodology. Xinqi Lin: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Software, Supervision, Validation, Visualization.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Dapeng Wu is an editorial board member for Digital Communications and Networks and was not involved in the editorial review or the decision to publish this article. All authors declare that there are no competing interests.

Acknowledgements

This research was supported in part by National Natural Science Foundation of China (62271096, U20A20157), Natural Science Foundation of Chongqing, China (cstc2020jcyj-zdxmX0024, CSTB2022NSCQ-MSX0600), University Innovation Research Group of Chongqing (CXQT20017), Program for Innovation Team Building at Institutions of Higher Education in Chongqing (CXTDX201601020), Science and Technology Research Program of Chongqing Municipal Education Commission (KJQN202000626), Youth Innovation Group Support Program of ICE Discipline of CQUPT (SCIE-QN-2022-04), the Science and Technology Research Program of Chongqing Municipal Education Commission under Grant KJQN202000626 and Chongqing Municipal Technology Innovation and Application Development Special Key Project (cstc2020jscx-dxwtBX0053).

References

[1] F.O. Olowononi, D.B. Rawat, C. Liu, Resilient machine learning for networked cy-ber physical systems: a survey for machine learning security to securing machine learning for cps, IEEE Commun. Surv. Tutor. 23 (1) (2020) 524-552.

[2] Y. Zhou, F.R. Yu, J. Chen, Y. Kuo, Cyber-physical-social systems: a state-of-the-art survey, challenges and opportunities, IEEE Commun. Surv. Tutor. 22 (1) (2019) 389-425.

[3] Q. Zhang, Z. Guo, Y. Zhu, P. Vijayakumar, A. Castiglione, B.B. Gupta, A deep learning-based fast fake news detection model for cyber-physical social services, Pattern Recognit. Lett. 168 (2023) 31-38.

[4] C. Wang, L. Zhao, C. Fan, K. Zhao, N. Kumar, J. Li, S. Wan, Metaverse-inspired cybertwin-based space-air-ground integrated networks, IEEE Netw. 37 (2) (2023) 294-300.

[5] P. Marks, Can the biases in facial recognition be fixed; also, should they?, Commun. ACM 64 (3) (2021) 20-22.

[6] C. Liu, T. Zhu, J. Zhang, W. Zhou, Privacy intelligence: a survey on image privacy in online social networks, ACM Comput. Surv. 55 (8) (2022) 1-35.

[7] J. Xiong, R. Ma, L. Chen, Y. Tian, Q. Li, X. Liu, Z. Yao, A personalized privacy pro-tection framework for mobile crowdsensing in iiot, IEEE Trans. Ind. Inform. 16 (6) (2019) 4231-4241.

[8] P. Ravi, J. Howe, A. Chattopadhyay, S. Bhasin, Lattice-based key-sharing schemes: a survey, ACM Comput. Surv. 54 (1) (2021) 1-39.

[9] A. Wood, K. Najarian, D. Kahrobaei, Homomorphic encryption for machine learning in medicine and bioinformatics, ACM Comput. Surv. 53 (4) (2020) 1-35.

[10] Y. Zhao, J. Chen, A survey on differential privacy for unstructured data content, ACM Comput. Surv. 54 (10) (2022) 1-28.

[11] F. Li, Z. Sun, A. Li, B. Niu, H. Li, G. Cao, Hideme: privacy-preserving photo sharing on social networks, in: IEEE INFOCOM 2019-IEEE Conference on Computer Com-munications, IEEE, 2019, pp. 154-162.

[12] G. Xu, B. Liu, L. Jiao, X. Li, M. Feng, K. Liang, L. Ma, X. Zheng, Trust2privacy: a novel fuzzy trust-to-privacy mechanism for mobile social networks, IEEE Wirel. Commun. 27 (3) (2020) 72-78.

[13] B. Niu, Y. Chen, B. Wang, Z. Wang, F. Li, J. Cao, Adapdp: adaptive personalized differential privacy, in: IEEE INFOCOM 2021-IEEE Conference on Computer Com-munications, IEEE, 2021, pp. 1-10.

[14] X. Zheng, Z. Cai, Privacy-preserved data sharing towards multiple parties in indus-trial iots, IEEE J. Sel. Areas Commun. 38 (5) (2020) 968-979.

[15] J. Zhou, Z. Su, J. Ni, Y. Wang, Y. Pan, R. Xing,Personalized privacy-preserving federated learning: optimized trade-off between utility and privacy, in: GLOBECOM 2022-2022 IEEE Global Communications Conference, IEEE, 2022, pp. 4872-4877.

[16] T. Nguyen Canh, H. Nagahara,Deep compressive sensing for visual privacy protec-tion in flatcam imaging, in:Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, 2019, pp. 3978-3986.

[17] F. Pittaluga, S.J. Koppal, Pre-capture privacy for small vision sensors, IEEE Trans. Pattern Anal. Mach. Intell. 39 (11) (2016) 2215-2226.

[18] C. Hinojosa, J.C. Niebles, H. Arguello,Learning privacy-preserving optics for hu-man pose estimation, in:Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 2573-2582.

[19] B. Liu, M. Ding, S. Shaham, W. Rahayu, F. Farokhi, Z. Lin, When machine learning meets privacy: a survey and outlook, ACM Comput. Surv. 54 (2) (2021) 1-36.

[20] A. Tonge, C. Caragea, Image privacy prediction using deep neural networks, ACM Trans. Web 14 (2) (2020) 1-32.

[21] D. Lin, D. Steiert, J. Morris, A. Squicciarini, J. Fan, Remind: risk estimation mecha-nism for images in network distribution, IEEE Trans. Inf. Forensics Secur. 15 (2019) 539-552.

[22] J. Zhang, J. Sang, X. Zhao, X. Huang, Y. Sun, Y. Hu,Adversarial privacy-preserving filter, in:Proceedings of the 28th ACM International Conference on Multimedia, 2020, pp. 1423-1431.

[23] M. Zhang, Z. Sun, H. Li, B. Niu, F. Li, Z. Zhang, Y. Xie, C. Zheng, Go-sharing: a blockchain-based privacy-preserving framework for cross-social network photo shar-ing, IEEE Trans. Dependable Secure Comput. 20 (5) (2022) 3572-3587.

[24] G. Yang, J. Cao, Z. Chen, J. Guo, J. Li, Graph-based neural networks for explainable image privacy inference, Pattern Recognit. 105 (2020) 107360.

[25] G. Yang, J. Cao, Q. Sheng, P. Qi, X. Li, J. Li, Drag: dynamic region-aware gcn for privacy-leaking image detection,in: Proceedings of the AAAI Conference on Artifi-cial Intelligence, 2022, pp. 12217-12225.

[26] R. Jiao, L. Zhang, A. Li, Ieye: personalized image privacy detection, in: 2020 6th International Conference on Big Data Computing and Communications (BIGCOM), IEEE, 2020, pp. 91-95.

[27] B. Liu, M. Ding, H. Xue, T. Zhu, D. Ye, L. Song, W. Zhou, Dp-image: differential privacy for image data in feature space, arXiv preprint arXiv :2103.07073, 2021.

[28] A. El-Kishky, T. Markovich, S. Park, C. Verma, B. Kim, R. Eskander, Y. Malkov, F. Portman, S. Samaniego, Y. Xiao, et al., Twhin: embedding the Twitter hetero-geneous information network for personalized recommendation,in: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022, pp. 2842-2850.

[29] Y. Lu, C. Shi, L. Hu, Z. Liu,Relation structure-aware heterogeneous information net-work embedding, in:Proceedings of the AAAI Conference on Artificial Intelligence, 2019, pp. 4456-4463.

[30] J. Zhao, X. Wang, C. Shi, Z. Liu, Y. Ye,Network schema preserving heterogeneous information network embedding, in:International Joint Conference on Artificial In-telligence (IJCAI), 2020, pp. 4456-4463.

[31] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial networks, Commun. ACM 63 (11) (2020) 139-144.

[32] T.-C. Wang, M.-Y. Liu, J.-Y. Zhu, A. Tao, J. Kautz, B. Catanzaro, High-resolution image synthesis and semantic manipulation with conditional gans, in: Proceed-ings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 8798-8807.

[33] Y.-H. Kwon, M.-G. Park,Predicting future frames using retrospective cycle gan, in:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni-tion, 2019, pp. 1811-1820.

[34] Y. Qu, S. Yu, W. Zhou, S. Chen, J. Wu, Customizable reliable privacy-preserving data sharing in cyber-physical social networks, IEEE Trans. Netw. Sci. Eng. 8 (1) (2020) 269-281.

[35] T. Park, M.-Y. Liu, T.-C. Wang, J.-Y. Zhu,Semantic image synthesis with spatially-adaptive normalization, in:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 2337-2346.

[36] C.-H. Lee, Z. Liu, L. Wu, P. Luo Maskgan,Towards diverse and interactive facial im-age manipulation, in:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 5549-5558.

[37] C. Huo, D. He, C. Liang, D. Jin, T. Qiu, L. Wu, TrustGNN: graph neural network-based trust evaluation via learnable propagative and composable nature, IEEE Trans. Neural Netw. Learn. Syst. 35 (10) (2023) 1-16.

[38] W. Lin, B. Li, Medley: predicting social trust in time-varying online social net-works, in: IEEE INFOCOM 2021-IEEE Conference on Computer Communications, IEEE, 2021, pp. 1-10.

[39] G. Liu, C. Li, Q. Yang, Neuralwalk: trust assessment in online social networks with neural networks, in: IEEE INFOCOM 2019-IEEE Conference on Computer Commu-nications, IEEE, 2019, pp. 1999-2007.

[40] S.D. Kamvar, M.T. Schlosser, H. Garcia-Molina,The eigentrust algorithm for rep-utation management in p2p networks, in:Proceedings of the 12th International Conference on World Wide Web, 2003, pp. 640-651.