Unsupervised social network embedding via adaptive specific mappings

Abstract

In this paper, we address the problem of unsuperised social network embedding, which aims to embed network nodes, including node attributes, into a latent low dimensional space. In recent methods, the fusion mechanism of node attributes and network structure has been proposed for the problem and achieved impressive prediction performance. However, the non-linear property of node attributes and network structure is not efficiently fused in existing methods, which is potentially helpful in learning a better network embedding. To this end, in this paper, we propose a novel model called ASM (Adaptive Specific Mapping) based on encoder-decoder framework. In encoder, we use the kernel mapping to capture the non-linear property of both node attributes and network structure. In particular, we adopt two feature mapping functions, namely an untrainable function for node attributes and a trainable function for network structure. By the mapping functions, we obtain the low dimensional feature vectors for node attributes and network structure, respectively. Then, we design an attention layer to combine the learning of both feature vectors and adaptively learn the node embedding. In encoder, we adopt the component of reconstruction for the training process of learning node attributes and network structure. We conducted a set of experiments on seven real-world social network datasets. The experimental results verify the effectiveness and efficiency of our method in comparison with state-of-the-art baselines.

Graphical abstract

Keywords

network embedding / specific kernel mapping / attention mechanism

References

[1] Wang X, Zhu M, Bo D, Cui P, Shi C, Pei J. AM-GCN: adaptive multi-channel graph convolutional networks. In: Proceedings of the 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2020, 1243−1253

[2] Pan G, Yao Y, Tong H, Xu F, Lu J. Unsupervised attributed network embedding via cross fusion. In: Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 2021, 797−805

[3] Rahimi A, Recht B. Random features for large-scale kernel machines. In: Proceedings of the 20th International Conference on Neural Information Processing Systems. 2007, 1177−1184

[4] Zhang D, Yin J, Zhu X, Zhang C. User profile preserving social network embedding. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017, 3378−3384

[5] Mikolov T, Sutskever I, Chen K, Corrado G, Dean J. Distributed representations of words and phrases and their compositionality. In: Proceedings of the 27th Annual Conference on Neural Information Processing Systems. 2013, 3111−3119

[6] McAuley J, Leskovec J. Learning to discover social circles in ego networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems. 2012, 539−547

[7] Tang L, Liu H. Relational learning via latent social dimensions. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2009, 817−826

[8] Bengio Y, Courville A, Vincent P . Representation learning: a review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35( 8): 1798–1828

[9] Perozzi B, Al-Rfou R, Skiena S. DeepWalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2014, 701−710

[10] Grover A, Leskovec J. node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016, 855−864

[11] Zhang Z, Yang H, Bu J, Zhou S, Yu P, Zhang J, Ester M, Wang C. ANRL: attributed network representation learning via deep neural networks. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2018, 3155−3161

[12] Gao H, Huang H. Deep attributed network embedding. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2018, 3364−3370

[13] Tang J, Qu M, Wang M, Zhang M, Yan J, Mei Q. LINE: large-scale information network embedding. In: Proceedings of the 24th International Conference on World Wide Web. 2015, 1067−1077

[14] Wang D, Cui P, Zhu W. Structural deep network embedding. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016, 1225−1234

[15] Qiu J, Dong Y, Ma H, Li J, Wang K, Tang J. Network embedding as matrix factorization: unifying DeepWalk, LINE, PTE, and node2vec. In: Proceedings of the 11th ACM International Conference on Web Search and Data Mining. 2018, 459−467

[16] Zhou Z, Gu Y, Yu G . Adversarial network embedding using structural similarity. Frontiers of Computer Science, 2021, 15( 1): 151603

[17] Chen J, Zhang Q, Huang X. Incorporate group information to enhance network embedding. In: Proceedings of the 25th ACM International Conference on Information and Knowledge Management. 2016, 1901−1904

[18] Long Q, Wang Y, Du L, Song G, Jin Y, Lin W. Hierarchical community structure preserving network embedding: a subspace approach. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 2019, 409−418

[19] Wang X, Cui P, Wang J, Pei J, Zhu W, Yang S. Community preserving network embedding. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence. 2017, 203−209

[20] Cao S, Lu W, Xu Q. GraRep: learning graph representations with global structural information. In: Proceedings of the 24th ACM International Conference on Information and Knowledge Management. 2015, 891−900

[21] Chen H, Perozzi B, Hu Y, Skiena S. HARP: hierarchical representation learning for networks. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018, 2127−2134

[22] Ribeiro L F R, Saverese P H P, Figueiredo D R. struc2vec: learning node representations from structural identity. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2017, 385−394

[23] Levy O, Goldberg Y. Neural word embedding as implicit matrix factorization. In: Proceedings of the 27th International Conference on Neural Information Processing Systems. 2014, 2177−2185

[24] Yang C, Liu Z, Zhao D, Sun M, Chang E Y. Network representation learning with rich text information. In: Proceedings of the 24th International Conference on Artificial Intelligence. 2015, 2111−2117

[25] Huang X, Li J, Hu X. Label informed attributed network embedding. In: Proceedings of the 10th ACM International Conference on Web Search and Data Mining. 2017, 731−739

[26] Huang X, Li J, Hu X. Accelerated attributed network embedding. In: Proceedings of 2017 SIAM International Conference on Data Mining. 2017, 633−641

[27] Pan S, Wu J, Zhu X, Zhang C, Wang Y. Tri-party deep network representation. In: Proceedings of the 25th International Joint Conference on Artificial Intelligence. 2016, 1895−1901

[28] Bandyopadhyay S, Lokesh N, Murty M N. Outlier aware network embedding for attributed networks. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 12−19

[29] Yang H, Pan S, Chen L, Zhou C, Zhang P. Low-bit quantization for attributed network representation learning. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019, 4047−4053

[30] Kipf T N, Welling M. Variational graph auto-encoders. 2016, arXiv preprint arXiv: 1611.07308

[31] Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th International Conference on Learning Representations. 2017

[32] Velickovic P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y. Graph attention networks. In: Proceedings of the 6th International Conference on Learning Representations. 2018

[33] Xu K, Hu W, Leskovec J, Jegelka S. How powerful are graph neural networks? In: Proceedings of the 7th International Conference on Learning Representations. 2019

[34] Cui G, Zhou J, Yang C, Liu Z. Adaptive graph encoder for attributed graph embedding. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2020, 976−985

[35] Yang D, Wang S, Li C, Zhang X, Li Z. From properties to links: deep network embedding on incomplete graphs. In: Proceedings of 2017 ACM on Conference on Information and Knowledge Management. 2017, 367−376

[36] Cao S, Lu W, Xu Q. Deep neural networks for learning graph representations. In: Proceedings of the 30th AAAI Conference on Artificial Intelligence. 2016, 1145−1152

[37] Zhang G, Liu Y, Jin X . A survey of autoencoder-based recommender systems. Frontiers of Computer Science, 2020, 14( 2): 430–450

[38] Li C, Wang S, Yang D, Li Z, Yang Y, Zhang X, Zhou J. PPNE: property preserving network embedding. In: Proceedings of the 22nd International Conference on Database Systems for Advanced Applications. 2017, 163−179

[39] Meng Z, Liang S, Fang J, Xiao T. Semi-supervisedly co-embedding attributed networks. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. 2019, 584

[40] Jian S, Hu L, Cao L, Lu K, Gao H. Evolutionarily learning multi-aspect interactions and influences from network structure and node content. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 598−605

[41] Cen Y, Zou X, Zhang J, Yang H, Zhou J, Tang J. Representation learning for attributed multiplex heterogeneous network. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2019, 1358−1368

[42] Meng Z, Liang S, Fang J, Xiao T. Semi-supervisedly co-embedding attributed networks. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. 2019, 584

[43] Qiao L, Zhang F, Huang X, Li K, Chen E . Heterogeneous-attributes enhancement deep framework for network embedding. Frontiers of Computer Science, 2021, 15( 6): 156616

RIGHTS & PERMISSIONS

Higher Education Press