Multi-layer network embedding on scc-based network with motif

Abstract

Interconnection of all things challenges the traditional communication methods, and Semantic Communication and Computing (SCC) will become new solutions. It is a challenging task to accurately detect, extract, and represent semantic information in the research of SCC-based networks. In previous research, researchers usually use convolution to extract the feature information of a graph and perform the corresponding task of node classification. However, the content of semantic information is quite complex. Although graph convolutional neural networks provide an effective solution for node classification tasks, due to their limitations in representing multiple relational patterns and not recognizing and analyzing higher-order local structures, the extracted feature information is subject to varying degrees of loss. Therefore, this paper extends from a single-layer topology network to a multi-layer heterogeneous topology network. The Bidirectional Encoder Representations from Transformers (BERT) training word vector is introduced to extract the semantic features in the network, and the existing graph neural network is improved by combining the higher-order local feature module of the network model representation network. A multi-layer network embedding algorithm on SCC-based networks with motifs is proposed to complete the task of end-to-end node classification. We verify the effectiveness of the algorithm on a real multi-layer heterogeneous network.

Keywords

Semantic communication and computing / Multi-layer network / Graph neural network / Motif

References

[1] E.C. Strinati, S. Barbarossa, 6g networks: beyond Shannon towards semantic and goal-oriented communications, Comput. Netw. 190 (2021) 107930.

[2] H. Xie, Z. Qin, G.Y. Li, B.-H. Juang, Deep learning enabled semantic communication systems, IEEE Trans. Signal Process. 69 (2021) 2663-2675.

[3] S. Boccaletti, V. Latora, Y. Moreno, M. Chavez, D.-U. Hwang, Complex networks: structure and dynamics, Phys. Rep. 424 (4-5) (2006) 175-308.

[4] S. Boccaletti, G. Bianconi, R. Criado, C.I. Del Genio, J. Gómez-Gardenes, M. Ro-mance, I. Sendina-Nadal, Z. Wang, M. Zanin, The structure and dynamics of multi-layer networks, Phys. Rep. 544 (1) (2014) 1-122.

[5] R. Interdonato, M. Atzmueller, S. Gaito, R. Kanawati, C. Largeron, A. Sala, Feature-rich networks: going beyond complex network topologies, Appl. Netw. Sci. 4(1)(2019) 1-13.

[6] M. De Domenico, A. Solé-Ribalta, E. Cozzo, M. Kivelä, Y. Moreno, M.A. Porter, S. Gómez, A. Arenas, Mathematical formulation of multilayer networks, Phys. Rev. X 3(4) (2013) 041022.

[7] F. Battiston, V. Nicosia, V. Latora, Structural measures for multiplex networks, Phys. Rev. E 89 (3) (2014) 032804.

[8] P.A. Traganitis, Y. Shen, G.B. Giannakis, Topology inference of multilayer networks, in: 2017 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), IEEE, 2017, pp. 898-903.

[9] J. Luo, L. Ding, C. Liang, N.H. Tu, An efficient network motif discovery approach for co-regulatory networks, IEEE Access 6 (2018) 14151-14158.

[10] W.-J. Xie, R.-Q. Han, W.-X. Zhou, Tetradic motif profiles of horizontal visibility graphs, Commun. Nonlinear Sci. Numer. Simul. 72 (2019) 544-551.

[11] A.R. Benson, D.F. Gleich, J. Leskovec, Higher-order organization of complex net-works, Science 353 (6295) (2016) 163-166.

[12] M.B.Z. Joveini, J. Sadri, Application of fractal theory on motifs counting in biologi-cal networks, IEEE/ACM Trans. Comput. Biol. Bioinform. 15 (2) (2016) 613-623.

[13] S. Gama-Castro, H. Salgado, A. Santos-Zavaleta, D. Ledezma-Tejeida, L. Muñiz-Rascado, J.S. García-Sotelo, K. Alquicira-Hernández, I. Martínez-Flores, L. Pannier, J.A. Castro-Mondragón, et al., Regulondb version 9.0: high-level integration of gene regulation, coexpression, motif clustering and beyond, Nucleic Acids Res. 44 (D1)(2016) D133-D143.

[14] S. Yu, Y. Feng, D. Zhang, H.D. Bedru, B. Xu, F. Xia, Motif discovery in networks: a survey, Comput. Sci. Rev. 37 (2020) 100267.

[15] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv : 1810.04805.

[16] K. Cho, A. Courville, Y. Bengio, Describing multimedia content using attention-based encoder-decoder networks, IEEE Trans. Multimed. 17 (11) (2015) 1875-1886.

[17] H. Yang, L. Lin, S. Zhong, F. Guo, Z. Cui, Aero engines fault diagnosis method based on convolutional neural network using multiple attention mechanism, in: 2021 IEEE International Conference on Sensing, Diagnostics, Prognostics, and Control (SDPC), IEEE, 2021, pp. 13-18.

[18] Q. Yang, B. Ma, H. Cui, J. Ma, Amf-net: attention-aware multi-scale fusion net-work for retinal vessel segmentation, in: 2021 43rd Annual International Confer-ence of the IEEE Engineering in Medicine & Biology Society (EMBC), IEEE, 2021, pp. 3277-3280.

[19] C. Gu, S. Wang, Y. Zhu, Z. Huang, K. Chen, Weakly supervised attention rectifica-tion for scene text recognition, in: 2020 25th International Conference on Pattern Recognition (ICPR), IEEE, 2021, pp. 779-786.

[20] S. Wasserman, K. Faust, et al., Social network analysis: methods and applications, American Ethnologist (2008).

[21] M. Ghorbani, M.S. Baghshah, H.R. Rabiee, Mgcn: semi-supervised classification in multi-layer graphs with graph convolutional networks,in: Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, 2019, pp. 208-211.

[22] V.N. Ioannidis, A.G. Marques, G.B. Giannakis, Tensor graph convolutional networks for multi-relational and robust learning, IEEE Trans. Signal Process. 68 (2020) 6535-6546.

[23] A. Sankar, X. Zhang, K.C.-C. Chang, Meta-gnn: metagraph neural network for semi-supervised learning in attributed heterogeneous information networks,in: Proceed-ings of the 2019 IEEE/ACM International Conference on Advances in Social Net-works Analysis and Mining, 2019, pp. 137-144.

[24] J. Tang, M. Qu, M. Wang, M. Zhang, J. Yan, Q. Mei, Line: large-scale information network embedding,in: Proceedings of the 24th International Conference on World Wide Web, 2015, pp. 1067-1077.