Simplified multi-view graph neural network for multilingual knowledge graph completion

Abstract

Knowledge graph completion (KGC) aims to fill in missing entities and relations within knowledge graphs (KGs) to address their incompleteness. Most existing KGC models suffer from knowledge coverage as they are designed to operate within a single KG. In contrast, Multilingual KGC (MKGC) leverages seed pairs from different language KGs to facilitate knowledge transfer and enhance the completion of the target KG. Previous studies on MKGC based on graph neural networks (GNNs) have primarily focused on using relation-aware GNNs to capture the combined features of neighboring entities and relations. However, these studies still have some shortcomings, particularly in the context of MKGCs. First, each language’s specific semantics, structures, and expressions contribute to the increased heterogeneity of the KG. Therefore, the completion of MKGCs necessitates a thorough consideration of the heterogeneity of the KG and the effective integration of its heterogeneous features. Second, MKGCs typically have a large graph scale due to the need to store and manage information from multiple languages. However, current relation-aware GNNs often inherit complex GNN operations, resulting in unnecessary complexity. Therefore, it is necessary to simplify GNN operations. To address these limitations, we propose a Simplified Multi-view Graph Neural Network (SM-GNN) for MKGC. SM-GNN incorporates two simplified multi-view GNNs as components. One GNN is utilized for learning multi-view graph features to complete the KG. The other generates new alignment pairs, facilitating knowledge transfer between different views of the KG. We simplify the two multi-view GNNs by retaining feature propagation while discarding linear transformation and nonlinear activation to reduce unnecessary complexity and effectively leverage graph contextual information. Extensive experiments demonstrate that our proposed model outperforms competing baselines. The code and dataset are available at the website of github.com/dbbice/SM-GNN.

Graphical abstract

Keywords

multi-view / knowledge graph / graph neural network / multilingual knowledge graph completion

References

[1] Yan J, Wang C, Cheng W, Gao M, Zhou A . A retrospective of knowledge graphs. Frontiers of Computer Science, 2018, 12( 1): 55–74

[2] Ji S, Pan S, Cambria E, Marttinen P, Yu P S . A survey on knowledge graphs: representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33( 2): 494–514

[3] Liu Y, Zhang K, Huang Z, Wang K, Zhang Y, Liu Q, Chen E. Enhancing hierarchical text classification through knowledge graph integration. In: Proceedings of the Association for Computational Linguistics. 2023, 5797−5810

[4] Wu X, Jiang T, Zhu Y, Bu C. Knowledge graph for China’s genealogy. In: Proceedings of 2020 IEEE International Conference on Knowledge Graph. 2020, 529−535

[5] Bu C, Zhang J, Yu X, Wu L, Wu X. Which companies are likely to invest: knowledge-graph-based recommendation for investment promotion. In: Proceedings of 2022 IEEE International Conference on Data Mining. 2022, 11−20

[6] Lin Q, Mao R, Liu J, Xu F, Cambria E . Fusing topology contexts and logical rules in language models for knowledge graph completion. Information Fusion, 2023, 90: 253–264

[7] Xu D, Xu T, Wu S, Zhou J, Chen E. Relation-enhanced negative sampling for multimodal knowledge graph completion. In: Proceedings of the 30th ACM International Conference on Multimedia. 2022, 3857−3866

[8] Liu F, Shen Y, Zhang T, Gao H . Entity-related paths modeling for knowledge base completion. Frontiers of Computer Science, 2020, 14( 5): 145311

[9] Choudhary S, Luthra T, Mittal A, Singh R. A survey of knowledge graph embedding and their applications. 2021, arXiv preprint arXiv: 2107.07842

[10] Rossi A, Barbosa D, Firmani D, Matinata A, Merialdo P . Knowledge graph embedding for link prediction: a comparative analysis. ACM Transactions on Knowledge Discovery from Data, 2021, 15( 2): 14

[11] Bordes A, Usunier N, Garcia-Durán A, Weston J, Yakhnenko O. Translating embeddings for modeling multi-relational data. In: Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013, 2787−2795

[12] Sun Z, Deng Z-H, Nie J-Y, Tang J. RotatE: knowledge graph embedding by relational rotation in complex space. In: Proceedings of the 7th International Conference on Learning Representations. 2019

[13] Li R, Cao Y, Zhu Q, Bi G, Fang F, Liu Y, Li Q. How does knowledge graph embedding extrapolate to unseen data: a semantic evidence view. In: Proceedings of the 36th AAAI Conference on Artificial Intelligence. 2022, 5781−5791

[14] Wang Q, Mao Z, Wang B, Guo L . Knowledge graph embedding: a survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering, 2017, 29( 12): 2724–2743

[15] Chen X, Chen M, Fan C, Uppunda A, Sun Y, Zaniolo C. Multilingual knowledge graph completion via ensemble knowledge transfer. In: Proceedings of the Association for Computational Linguistics. 2020, 3227−3238

[16] Huang Z, Li Z, Jiang H, Cao T, Lu H, Yin B, Subbian K, Sun Y, Wang W. Multilingual knowledge graph completion with self-supervised adaptive graph alignment. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 2022, 474−485

[17] Tong V, Nguyen D Q, Huynh T T, Nguyen T T, Nguyen Q V H, Niepert M. Joint multilingual knowledge graph completion and alignment. In: Proceedings of the Association for Computational Linguistics. 2022, 4646−4658

[18] Li Z, Zhang Q, Zhu F, Li D, Zheng C, Zhang Y . Knowledge graph representation learning with simplifying hierarchical feature propagation. Information Processing & Management, 2023, 60( 4): 103348

[19] Dettmers T, Minervini P, Stenetorp P, Riedel S. Convolutional 2D knowledge graph embeddings. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018, 1811−1818

[20] Yao Z, Zhang W, Chen M, Huang Y, Yang Y, Chen H. Analogical inference enhanced knowledge graph embedding. In: Proceedings of the 37th AAAI Conference on Artificial Intelligence. 2023, 4801−4808

[21] Le T, Le N, Le B . Knowledge graph embedding by relational rotation and complex convolution for link prediction. Expert Systems with Applications, 2023, 214: 119122

[22] Fanourakis N, Efthymiou V, Kotzinos D, Christophides V . Knowledge graph embedding methods for entity alignment: experimental review. Data Mining and Knowledge Discovery, 2023, 37( 5): 2070–2137

[23] Liu C, Li L, Yao X, Tang L. A survey of recommendation algorithms based on knowledge graph embedding. In: Proceedings of the 2019 IEEE International Conference on Computer Science and Educational Informatization. 2019, 168−171

[24] Shokrzadeh Z, Feizi-Derakhshi M R, Balafar M A, Mohasefi J B . Knowledge graph-based recommendation system enhanced by neural collaborative filtering and knowledge graph embedding. Ain Shams Engineering Journal, 2024, 15( 1): 102263

[25] Wang Z, Zhang J, Feng J, Chen Z. Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence. 2014, 1112−1119

[26] Lin Y, Liu Z, Sun M, Liu Y, Zhu X. Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the 29th AAAI Conference on Artificial Intelligence. 2015, 2181−2187

[27] Yao L, Mao C S, Luo Y. KG-BERT: Bert for knowledge graph completion. 2019, arXiv preprint arXiv:1909.03193

[28] Yao L, Mao C, Luo Y. Graph convolutional networks for text classification. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 7370−7377

[29] He X, Deng K, Wang X, Li Y, Zhang Y D, Wang M. LightGCN: simplifying and powering graph convolution network for recommendation. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2020, 639−648

[30] Mao K L, Zhu J M, Xiao X, Lu B, Wang Z W, He X Q. UltraGCN: Ultra simplification of graph convolutional networks for recommendation. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2021, 1253−1262

[31] Schlichtkrull M, Kipf T N, Bloem P, Van Den Berg R, Titov I, Welling M. Modeling relational data with graph convolutional networks. In: Proceedings of the 15th International Conference on the Semantic Web. 2018, 593−607

[32] Vashishth S, Sanyal S, Nitin V, Talukdar P P. Composition-based multi-relational graph convolutional networks. In: Proceedings of the 8th International Conference on Learning Representations. 2020

[33] Wu F, Souza A H Jr, Zhang T, Fifty C, Yu T, Weinberger K Q. Simplifying graph convolutional networks. In: Proceedings of the 36th International Conference on Machine Learning. 2019, 6861−6871

[34] Wang H, Dai S, Su W, Zhong H, Fang Z, Huang Z, Feng S, Chen Z, Sun Y, Yu D. Simple and effective relation-based embedding propagation for knowledge representation learning. In: Proceedings of the 31st International Joint Conference on Artificial Intelligence. 2022, 2755−2761

[35] Jiang T, Bu C, Zhu Y, Wu X . Integrating symbol similarities with knowledge graph embedding for entity alignment: an unsupervised framework. Intelligent Computing, 2023, 2: 0021

[36] Jiang T T, Bu C Y, Zhu Y, Wu X D . Combining embedding-based and symbol-based methods for entity alignment. Pattern Recognition, 2022, 124: 108433

[37] Jiang T T, Bu C Y, Zhu Y, Wu X D. Two-stage entity alignment: combining hybrid knowledge graph embedding with similarity-based relation alignment. In: Proceedings of the 16th Pacific Rim International Conference on Artificial Intelligence. 2019, 162−175

[38] Zhu H, Xie R, Liu Z, Sun M. Iterative entity alignment via joint knowledge embeddings. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017, 4258−4264

[39] Zhu J, Huang C, De Meo P . DFMKE: a dual fusion multi-modal knowledge graph embedding framework for entity alignment. Information Fusion, 2023, 90: 111–119

[40] Chen M, Tian Y, Yang M, Zaniolo C. Multilingual knowledge graph embeddings for cross-lingual knowledge alignment. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017, 1511−1517

[41] Sun Z, Hu W, Zhang Q, Qu Y. Bootstrapping entity alignment with knowledge graph embedding. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2018, 4396−4402

[42] Wang Z, Lv Q, Lan X, Zhang Y. Cross-lingual knowledge graph alignment via graph convolutional networks. In: Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. 2018, 349−357

[43] Singh H, Chakrabarti S, Jain P, Choudhury S R, Mausam. Multilingual knowledge graph completion with joint relation and entity alignment. In: Proceedings of the 3rd Conference on Automated Knowledge Base Construction. 2021

[44] Akhtar M U, Liu J, Xie Z, Liu X, Ahmed S, Huang B . Entity alignment based on relational semantics augmentation for multilingual knowledge graphs. Knowledge-Based Systems, 2022, 252: 109494

[45] Wu Y, Liu X, Feng Y, Wang Z, Yan R, Zhao D. Relation-aware entity alignment for heterogeneous knowledge graphs. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019, 5278−5284

[46] Yang B, Yih W T, He X, Gao J, Deng L. Embedding entities and relations for learning and inference in knowledge bases. In: Proceedings of the 3rd International Conference on Learning Representations. 2015

[47] Devlin J, Chang M-W, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019, 4171−4186

[48] Demšar J . Statistical comparisons of classifiers over multiple data sets. The Journal of Machine Learning Research, 2006, 7: 1–30

[49] Mao X, Wang W, Wu Y, Lan M. Are negative samples necessary in entity alignment?: an approach with high performance, scalability and robustness. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2021, 1263−1273

RIGHTS & PERMISSIONS

Higher Education Press