RPND: a rule guided link prediction model with specific-path selection

Abstract

Knowledge graphs (KGs) often suffer from incompleteness, which limits their performance in practice where a vast amount of entities may co-exist. To aid, knowledge graph completion (KGC) has been proposed to infer the missing links between entities. Among them, reasoning over relation paths in incomplete KG is a popular research topic. However, there are still some issues remained to be solved, such as path noise, path sparsity of KG, the ambiguity of inferred relation and lack of explanability in path representation. To simultaneously address the aforementioned challenges, we propose a novel rule guided link prediction model with path noise avoidance and disambiguation of inferred relation, termed as RPND. Specifically, we utilize path selection strategy to filter noisy path and reduce the interference of path noise. To alleviate the path sparsity of KG, we leverage path overlapping feature of similar relations and combine them based on the semantic similarity. For the ambiguity of inferred relation, we draw the insight from language model like transformer by introducing position embedding to reflect the order of relation along the path when learning its representation. Meanwhile, we employ logic rules to compose paths in semantic level to enhance the explanability of path representation. Extensive experiments conducted on benchmark datasets demonstrate the superiority of our proposed RPND model compared to its SOTAs.

Graphical abstract

Keywords

link prediction / knowledge graph completion / path noise / ambiguity of inferred relation / path sparsity

References

[1] Bollacker K D, Evans C, Paritosh P K, Sturge T, Taylor J. Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the ACM SIGMOD International Conference on Management of Data. 2008, 1247−1250

[2] Wu X, Jiang T, Zhu Y, Bu C . Knowledge graph for China’s genealogy. IEEE Transactions on Knowledge and Data Engineering, 2023, 35( 1): 634–646

[3] Cao P, Wu J . GraphRevisedIE: multimodal information extraction with graph-revised network. Pattern Recognition, 2023, 140: 109542

[4] Shang D, Shang P, Li A . A novel clustering method for complex signals and feature extraction based on advanced information-based dissimilarity measure. Expert Systems with Applications, 2024, 238: 122011

[5] Zhang M, He T, Dong M . Meta-path reasoning of knowledge graph for commonsense question answering. Frontiers of Computer Science, 2024, 18( 1): 181303

[6] Wang S, Qin B . A novel joint training model for knowledge base question answering. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024, 32: 666–679

[7] Zhu Y, Geng Y, Li Y, Qiang J, Wu X . Representation learning: serial-autoencoder for personalized recommendation. Frontiers of Computer Science, 2024, 18( 4): 184316

[8] Li M, Zhang L, Cui L, Bai L, Li Z, Wu X . BLoG: bootstrapped graph representation learning with local and global regularization for recommendation. Pattern Recognition, 2023, 144: 109874

[9] Gong J, Zhao Y, Zhao J, Zhang J, Ma G, Zheng S, Du S, Tang J . Personalized recommendation via inductive spatiotemporal graph neural network. Pattern Recognition, 2024, 145: 109884

[10] Lee Y, Lee J, Lee D, Kim S . Learning to compensate for lack of information: Extracting latent knowledge for effective temporal knowledge graph completion. Information Sciences, 2024, 654: 119857

[11] Wang J, Wang B, Gao J, Hu S, Hu Y, Yin B . Multi-level interaction based knowledge graph completion. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024, 32: 386–396

[12] Zhang F, Chen H, Shi Y, Cheng J, Lin J . Joint framework for tensor decomposition-based temporal knowledge graph completion. Information Sciences, 2024, 654: 119853

[13] Liang S, Shao J, Zhang D, Zhang J, Cui B . DRGI: deep relational graph infomax for knowledge graph completion. IEEE Transactions on Knowledge and Data Engineering, 2023, 35( 3): 2486–2499

[14] Lin Y, Liu Z, Luan H, Sun M, Rao S, Liu S. Modeling relation paths for representation learning of knowledge bases. In: Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing. 2015,705−714

[15] Shen Y, Ding N, Zheng H T, Li Y, Yang M . Modeling relation paths for knowledge graph completion. IEEE Transactions on Knowledge and Data Engineering, 2021, 33( 11): 3607–3617

[16] Jia N, Cheng X, Su S. Improving knowledge graph embedding using locally and globally attentive relation paths. In: Proceedings of the 42nd European Conference on IR Research. 2020, 17−32

[17] Li X, Han Q, Li L, Wang Y. Link prediction based on the relational path inference of triangular structures. In: Proceedings of the 9th International Conference of Pioneering Computer Scientists, Engineers and Educators. 2023,255−268

[18] Lv X, Han X, Hou L, Li J, Liu Z, Zhang W, Zhang Y, Kong H, Wu S. Dynamic anticipation and completion for multi-hop reasoning over sparse knowledge graph. In: Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing. 2020, 5694−5703

[19] Zhang S, Zhang J, Song X, Adeshina S, Zheng D, Faloutsos C, Sun Y. PaGE-Link: Path-based graph neural network explanation for heterogeneous link prediction. In: Proceedings of the 23rd ACM Web Conference. 2023, 3784−3793

[20] Yang L, Zhao J, Long J, Huang J, Wang Z, Chen T. Path-KGE: Preference-aware knowledge graph embedding with path semantics for link prediction. In: Proceedings of the 24th International Conference on Web Information Systems Engineering. 2023,409−424

[21] Neelakantan A, Roth B, McCallum A. Compositional vector space models for knowledge base completion. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015,156−166

[22] Bordes A, Usunier N, Garcıa-Durán A, Weston J, Yakhnenko O. Translating embeddings for modeling multi-relational data. In: Proceedings of the 27th Annual Conference on Neural Information Processing Systems. 2013, 2787−2795

[23] Wang Z, Zhang J, Feng J, Chen Z. Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence. 2014, 1112−1119

[24] Lin Y, Liu Z, Sun M, Liu Y, Zhu X. Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the 29th AAAI Conference on Artificial Intelligence. 2015, 2181−2187

[25] Ma S, Ding J, Jia W, Wang K, Guo M. TransT: type-based multiple embedding representations for knowledge graph completion. In: Proceedings of the 17th European Conference on Machine Learning and Knowledge Discovery in Databases. 2017,717−733

[26] Zhang Z, Cai J, Zhang Y, Wang J. Learning hierarchy-aware knowledge graph embeddings for link prediction. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020, 3065−3072

[27] Wang Y, Zhang H . HARP: A novel hierarchical attention model for relation prediction. ACM Transactions on Knowledge Discovery from Data, 2021, 15( 2): 17

[28] Wang S, Wei X, dos Santos C N, Wang Z, Nallapati R, Arnold A, Xiang B, Yu P S, Cruz I F. Mixed-curvature multi-relational graph neural network for knowledge graph completion. In: Proceedings of the Web Conference. 2021, 1761−1771

[29] Nickel M, Tresp V, Kriegel H P. A three-way model for collective learning on multi-relational data. In: Proceedings of the 28th International Conference on Machine Learning. 2011,809−816

[30] Yang B, Yih W T, He X, Gao J, Deng L. Embedding entities and relations for learning and inference in knowledge bases. In: Proceedings of the 3rd International Conference on Learning Representations. 2015

[31] Trouillon T, Welbl J, Riedel S, Gaussier É, Bouchard G. Complex embeddings for simple link prediction. In: Proceedings of the 33rd International Conference on Machine Learning. 2016, 2071−2080

[32] Balazevic I, Allen C, Hospedales T. TuckER: tensor factorization for knowledge graph completion. In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 5185−5194

[33] Liu H, Wu Y, Yang Y. Analogical inference for multirelational embeddings. In: Proceedings of the 34th International Conference on Machine Learning. 2017, 2168−2178

[34] Zhang W, Paudel B, Zhang W, Bernstein A, Chen H. Interaction embeddings for prediction and explanation in knowledge graphs. In: Proceedings of the 12th ACM International Conference on Web Search and Data Mining. 2019, 96−104

[35] Chami I, Wolf A, Juan D C, Sala F, Ravi S, Ré C. Low-dimensional hyperbolic knowledge graph embeddings. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 6901−6914

[36] Dettmers T, Minervini P, Stenetorp P, Riedel S. Convolutional 2D knowledge graph embeddings. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018,221

[37] Nguyen D Q, Nguyen T D, Nguyen D Q, Phung D Q. A novel embedding model for knowledge base completion based on convolutional neural network. In: Proceedings of 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2018,327−333

[38] Jiang X, Wang Q, Wang B. Adaptive convolution for multi-relational learning. In: Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019,978−987

[39] Vashishth S, Sanyal S, Nitin V, Agrawal N, Talukdar P. InteractE: Improving convolution-based knowledge graph embeddings by increasing feature interactions. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020, 3009−3016

[40] Schlichtkrull M, Kipf T N, Bloem P, van den Berg R, Titov I, Welling M. Modeling relational data with graph convolutional networks. In: Proceedings of the 15th International Conference on the Semantic Web. 2018,593−607

[41] Shang C, Tang Y, Huang J, Bi J, He X, Zhou B. End-to-end structure-aware convolutional networks for knowledge base completion. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 3060−3067

[42] Ebisu T, Ichise R. Graph pattern entity ranking model for knowledge graph completion. In: Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019,988−997

[43] Zhang D, Yin J, Yu P S . Link prediction with contextualized self-supervision. IEEE Transactions on Knowledge and Data Engineering, 2023, 35( 7): 7138–7151

[44] Song L, Li J, Liu J, Yang Y, Shang X, Sun M . Answering knowledge-based visual questions via the exploration of question purpose. Pattern Recognition, 2023, 133: 109015

[45] Song L, Li H, Tan Y, Li Z, Shang X . Enhancing enterprise credit risk assessment with cascaded multi-level graph representation learning. Neural Networks, 2024, 169: 475–484

[46] Song L, He M, Shang X, Yang C, Liu J, Yu M, Lu Y . A deep cross-modal neural cognitive diagnosis framework for modeling student performance. Expert Systems with Applications, 2023, 230: 120675

[47] Lao N, Mitchell T, Cohen W W. Random walk inference and learning in A large scale knowledge base. In: Proceedings of 2011 Conference on Empirical Methods in Natural Language Processing. 2011,529−539

[48] Gardner M, Mitchell T. Efficient and expressive knowledge base completion using subgraph feature extraction. In: Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing. 2015, 1488−1498

[49] Gardner M, Talukdar P P, Kisiel B, Mitchell T M. Improving learning and inference in a large knowledge-Base using latent syntactic cues. In: Proceedings of 2013 Conference on Empirical Methods in Natural Language Processing. 2013,833−838

[50] Gardner M, Talukdar P, Krishnamurthy J, Mitchell T M. Incorporating vector space similarity in random walk inference over knowledge bases. In: Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing. 2014,397−406

[51] Yin W, Yaghoobzadeh Y, Schütze H. Recurrent one-hop predictions for reasoning over knowledge graphs. In: Proceedings of the 27th International Conference on Computational Linguistics. 2018, 2369−2378

[52] Lv X, Han X, Hou L, Li J, Liu Z, Zhang W, Zhang Y, Kong H, W u S. . Dynamic Anticipation and Completion for Multi-Hop Reasoning over Sparse Knowledge Graph. In: Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing., 2020, –5694−5703

[53] Wang H, Ren H, Leskovec J. Entity context and relational paths for knowledge graph completion. 2020, arXiv preprint arXiv: 2002.06757

[54] Zhang M, Wang Q, Xu W, Li W, Sun S. Discriminative path-based knowledge graph embedding for precise link prediction. In: Proceedings of the 40th European Conference on IR Research. 2018,276−288

[55] Zhuo X, Wu G, Zhang Z, Wu X . Geometric-contextual mutual infomax path aggregation for relation reasoning on knowledge graph. IEEE Transactions on Knowledge and Data Engineering, 2024, 36( 7): 3076–3090

[56] Li W, Peng R, Li Z . Knowledge graph completion by jointly learning structural features and soft logical rules. IEEE Transactions on Knowledge and Data Engineering, 2023, 35( 3): 2724–2735

[57] Guu K, Miller J, Liang P. Traversing knowledge graphs in vector space. In: Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing. 2015,318−327

[58] Peng Z, Yu H, Jia X. Path-based reasoning with K-nearest neighbor and position embedding for knowledge graph completion. Journal of Intelligent Information Systems, 2022, 58(3): 513−533

[59] García-Durán A, Bordes A, Usunier N, Grandvalet Y . Combining two and three-way embedding models for link prediction in knowledge bases. Journal of Artificial Intelligence Research, 2016, 55( 1): 715–742

[60] Chang H, Ye J, Lopez-Avila A, Du J, Li J. Path-based explanation for knowledge graph completion. In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2024,231−242

[61] Galárraga L, Teflioudi C, Hose K, Suchanek F M . Fast rule mining in ontological knowledge bases with AMIE+. The VLDB Journal, 2015, 24( 6): 707–730

[62] Toutanova K, Chen D. Observed versus latent features for knowledge base and text inference. In: Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality. 2015, 57−66

[63] Kazemi S M, Poole D. SimplE embedding for link prediction in knowledge graphs. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018, 4289−4300

[64] Sun Z, Deng Z H, Nie J Y, Tang J. RotatE: Knowledge graph embedding by relational rotation in complex space. In: Proceedings of the 7th International Conference on Learning Representations. 2019

[65] Balažević I, Allen C, Hospedales T M. Hypernetwork knowledge graph embeddings. In: Proceedings of the 28th International Conference on Artificial Neural Networks. 2019,553−565

[66] Wang J, Wang B, Gao J, Hu Y, Yin B . Multi-concept representation learning for knowledge graph completion. ACM Transactions on Knowledge Discovery from Data, 2023, 17( 1): 11

[67] Wang Y, Ouyang X, Guo D, Zhu X . MEGA: meta-graph augmented pre-training model for knowledge graph completion. ACM Transactions on Knowledge Discovery from Data, 2024, 18( 1): 30

[68] Li Z, Chen L, Jian Y, Wang H, Zhao Y, Zhang M, Xiao K, Zhang Y, Deng H, Hou X . Aggregation or separation? Adaptive embedding message passing for knowledge graph completion. Information Sciences, 2025, 691: 121639

[69] Shang B, Zhao Y, Liu J, Wang D. Mixed geometry message and trainable convolutional attention network for knowledge graph completion. In: Proceedings of the 38th AAAI Conference on Artificial Intelligence. 2024, 8966−8974

[70] Zhu Y, Liu H, Wu Z, Song Y, Zhang T. Representation learning with ordered relation paths for knowledge graph completion. In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 2662−2671

[71] Guo S, Wang Q, Wang L, Wang B, Guo L. Knowledge graph embedding with iterative guidance from soft rules. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018,590

[72] Ott S, Betz P, Stepanova D, Gad-Elrab M H, Meilicke C, Stuckenschmidt H. Rule-based knowledge graph completion with canonical models. In: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. 2023, 1971−1981

[73] Lu X, Wang L, Jiang Z, Liu S, Lin J . MRE: a translational knowledge graph completion model based on multiple relation embedding. Mathematical Biosciences and Engineering, 2023, 20( 3): 5881–5900

[74] Niu G, Zhang Y, Li B, Cui P, Liu S, Li J, Zhang X. Rule-guided compositional representation learning on knowledge graphs. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020, 2950−2958

[75] Demšar J . Statistical comparisons of classifiers over multiple data sets. The Journal of Machine Learning Research, 2006, 7: 1–30

RIGHTS & PERMISSIONS

Higher Education Press