Uncertain knowledge graph embedding: an effective method combining multi-relation and multi-path

Abstract

Uncertain Knowledge Graphs (UKGs) are used to characterize the inherent uncertainty of knowledge and have a richer semantic structure than deterministic knowledge graphs. The research on the embedding of UKG has only recently begun, Uncertain Knowledge Graph Embedding (UKGE) model has a certain effect on solving this problem. However, there are still unresolved issues. On the one hand, when reasoning the confidence of unseen relation facts, the introduced probabilistic soft logic cannot be used to combine multi-path and multi-step global information, leading to information loss. On the other hand, the existing UKG embedding model can only model symmetric relation facts, but the embedding problem of asymmetric relation facts has not be addressed. To address the above issues, a Multiplex Uncertain Knowledge Graph Embedding (MUKGE) model is proposed in this paper. First, to combine multiple information and achieve more accurate results in confidence reasoning, the Uncertain ResourceRank (URR) reasoning algorithm is introduced. Second, the asymmetry in the UKG is defined. To embed asymmetric relation facts of UKG, a multi-relation embedding model is proposed. Finally, experiments are carried out on different datasets via 4 tasks to verify the effectiveness of MUKGE. The results of experiments demonstrate that MUKGE can obtain better overall performance than the baselines, and it helps advance the research on UKG embedding.

Graphical abstract

Keywords

knowledge representation / uncertain knowledge graph / multi-relation embedding / uncertain reasoning

Qi Liu received the BE degree from Chongqing University of Posts and Telecommunications, China in 2020. She is currently pursuing the ME degree in Chongqing University of Posts and Telecommunications, China. Her research interests include knowledge graph construction, knowledge representation and uncertain reasoning

Qinghua Zhang received the BS degree from the Sichuan University, China in 1998, MS degree from Chongqing University of Posts and Telecommunications, China in 2003, and the PhD degree from the Southwest Jiaotong University, China in 2010. He was at San Jose State University, USA, as a visiting scholar in 2015. Since 1998, he has been at the Chongqing University of Posts and Telecommunications, China where he is currently a professor, and the director of the Science and Technology Division. His research interests include rough set, fuzzy set, granular computing and uncertain information processing. He is a member of the IEEE

Fan Zhao received the BS and MS degrees from Chongqing University of Posts and Telecommunications, China in 2017 and 2020, respectively. She is currently pursuing the Ph.D. degree with the Chongqing University of Posts and Telecommunications, China. Her research interests include analysis and processing of uncertain data, three-way decisions, fuzzy sets, granular computing, and rough sets

Guoyin Wang received the BS, MS, and PhD degrees from Xi’an Jiaotong University, China in 1992, 1994, and 1996, respectively. Since 1996, he has been with the Chongqing University of Posts and Telecommunications, China, where he is currently a Professor, the Director of the Chongqing Key Laboratory of Computational Intelligence and the National International Scientific and Technological Cooperation Base of Big Data Intelligent Computing, and the Vice President of the University. He has authored 12 books, edited dozens of proceedings of international and national conferences, and has over 300 reviewed research publications. His current research interests include rough sets, granular computing, knowledge technology, data mining, neural network, and cognitive computing. Dr. Wang was the President of the International Rough Sets Society (IRSS) for the period 2014–2017. He is the Vice President of the Chinese Association for Artificial Intelligence

References

[1] Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J. Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of 2008 ACM SIGMOD International Conference on Management of Data. 2008, 1247–1250

[2] Mitchell T, Cohen W, Hruschka E, Talukdar P, Yang B S, Betteridge J, Carlson A, Dalvi B, Gardner M, Kisiel B, Krishnamurthy J, Lao N, Mazaitis K, Mohamed T, Nakashole N, Platanios E, Ritter A, Samadi M, Settles B, Wang R, Wijaya D, Gupta A, Chen X, Saparov A, Greaves M, Welling J. . Never-ending learning. Communications of the ACM, 2018, 61( 5): 103–115

[3] Lehmann J, Isele R, Jakob M, Jentzsch A, Kontokostas D, Mendes P N, Hellmann S, Morsey M, Van Kleef P, Auer S, Bizer C . Dbpedia–a large-scale, multilingual knowledge base extracted from wikipedia. Semantic Web, 2015, 6( 2): 167–195

[4] Wang D . Answering contextual questions based on ontologies and question templates. Frontiers of Computer Science in China, 2011, 5( 4): 405–418

[5] Zhong Z, Lin X, He L . Answering range-based reverse kNN and why-not reverse kNN queries. Frontiers of Computer Science, 2020, 14( 1): 233–235

[6] Zhong Z, Lin X, He L, Yang J . Answering why-not questions on knn queries. Frontiers of Computer Science, 2019, 13( 5): 1062–1071

[7] Wu J, He X, Wang X, Wang Q, Chen W, Lian J, Xie X . Graph convolution machine for context-aware recommender system. Frontiers of Computer Science, 2022, 16( 6): 166614

[8] Zhang Z, Li C, Wu Z, Sun A, Ye D, Luo X. . NEXT: a neural network framework for next poi recommendation. Frontiers of Computer Science, 2020, 14( 2): 314–333

[9] Zheng Z, Liu Y, Li D, Zhang X . Distant supervised relation extraction based on residual attention. Frontiers of Computer Science, 2022, 16( 6): 166336

[10] Cao Y, Chen D, Xu Z, Li H, Luo P . Nested relation extraction with iterative neural network. Frontiers of Computer Science, 2021, 15( 3): 153323

[11] Xie T, Wu B, Jia B, Wang B . Graph-ranking collective Chinese entity linking algorithm. Frontiers of Computer Science, 2020, 14( 2): 291–303

[12] Hu Y, Shen D, Nie T, Kou Y, Yu G . Biomedical entity linking based on less labeled data. Frontiers of Computer Science, 2022, 16( 3): 163343

[13] Li M, Xing Y, Kong F, Zhou G . Towards better entity linking. Frontiers of Computer Science, 2022, 16( 2): 162308

[14] Zeng K, Li C, Hou L, Li J, Feng L . A comprehensive survey of entity alignment for knowledge graphs. AI Open, 2021, 2: 1–13

[15] Lin Y, Han X, Xie R, Liu Z, Sun M. Knowledge representation learning: a quantitative review. 2018, arXiv preprint arXiv: 1812.10901v1

[16] Rebele T, Suchanek F, Hoffart J, Biega J, Kuzey E, Weikum G. Yago: a multilingual knowledge base from wikipedia, wordnet, and geonames. In: Proceedings of the 15th International Semantic Web Conference. 2016, 177–185

[17] Chang D, Chen M, Liu C, Liu L, Li D, Li W, Kong F, Liu B, Luo X, Qi J, Jin Q, Xu B. DiaKG: An annotated diabetes dataset for medical knowledge graph construction. In: Proceedings of the 6th China Conference on Knowledge Graph and Semantic Computing. 2021, 308–314

[18] Speer R, Chin J, Havasi C. ConceptNet 5.5: an open multilingual graph of general knowledge. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence. 2017, 4444–4451

[19] Wu W, Li H, Wang H, Zhu K Q. Probase: a probabilistic taxonomy for text understanding. In: Proceedings of 2012 ACM SIGMOD International Conference on Management of Data. 2012, 481–492

[20] Chiachío M, Chiachío J, Prescott D, Andrews J . A new paradigm for uncertain knowledge representation by plausible petri nets. Information Sciences, 2018, 453: 323–345

[21] Holzinger A, Langs G, Denk H, Zatloukal K, Müller H . Causability and explainability of artificial intelligence in medicine. WIREs Data Mining and Knowledge Discovery, 2019, 9( 4): e1312

[22] Raskolnikov A. Probabilistic compliance. Yale Journal on Regulation, 2017, 34(2): 492−493

[23] Yang S, Zhang W, Tang R, Zhang M, Huang Z . Approximate inferring with confidence predicting based on uncertain knowledge graph embedding. Information Sciences, 2022, 609: 679–690

[24] Chen X, Boratko M, Chen M, Dasgupta S S, Li X L, McCallum A. Probabilistic box embeddings for uncertain knowledge graph reasoning. In: Proceedings of 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021

[25] Dubois D, Prade H . Upper and lower possibilities induced by a multivalued mapping. IFAC Proceedings Volumes, 1983, 16( 13): 147–152

[26] Shafer G A. A Mathematical Theory of Evidence. Princeton, NJ: Princeton University Press, 1976

[27] Pawlak Z . Rough sets. International Journal of Computer & Information Sciences, 1982, 11( 5): 341–356

[28] Xu T H, Wang G Y . Finding strongly connected components of simple digraphs based on generalized rough sets theory. Knowledge-Based Systems, 2018, 149: 88–98

[29] Zhang Z, Yang X . Tolerance-based multigranulation rough sets in incomplete systems. Frontiers of Computer Science, 2014, 8( 5): 753–762

[30] Shortliffe E H, Buchanan B G . A model of inexact reasoning in medicine. Mathematical Biosciences, 1975, 23( 3−4): 351–379

[31] Zadeh L A . Fuzzy sets. Information and Control, 1965, 8( 3): 338–353

[32] Yang J, Wang G Y, Zhang Q H, Chen Y H, Xu T H . Optimal granularity selection based on cost-sensitive sequential three-way decisions with rough fuzzy sets. Knowledge-Based Systems, 2019, 163: 131–144

[33] Zhang Q H, Xia D Y, Liu K X, Wang G Y . A general model of decision-theoretic three-way approximations of fuzzy sets based on a heuristic algorithm. Information Sciences, 2020, 507: 522–539

[34] Zhang Q H, Chen Y H, Yang J, Wang G Y . Fuzzy entropy: A more comprehensible perspective for interval shadowed sets of fuzzy sets. IEEE Transactions on Fuzzy Systems, 2020, 28( 11): 3008–3022

[35] Duda R, Gaschnig J, Hart P. Model design in the PROSPECTOR consultant system for mineral exploration. In: Webber B L, Nilsson N J, eds. Readings in Artificial Intelligence. San Mateo, CA: Elsevier, 1981

[36] Chen F, Wang Y, Wang B, Kuo C C J. Graph representation learning: a survey. 2019, arXiv preprint arXiv: 1909.00958

[37] Ji S, Pan S, Cambria E, Marttinen P, Yu P S . A survey on knowledge graphs: Representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33( 2): 494–514

[38] Wang Q, Mao Z D, Wang B, Guo L . Knowledge graph embedding: a survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering, 2017, 29( 12): 2724–2743

[39] Yan J H, Wang C Y, Cheng W L, Gao M, Zhou A Y . A retrospective of knowledge graphs. Frontiers of Computer Science, 2018, 12( 1): 55–74

[40] Doh R F, Zhou C H, Arthur J K, Tawiah I, Doh B . A systematic review of deep knowledge graph-based recommender systems, with focus on explainable embeddings. Data, 2022, 7( 7): 94

[41] Zhang H, Zheng T, Gao J, Miao C, Su L, Li Y, Ren K. Data poisoning attack against knowledge graph embedding. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019

[42] Bordes A, Usunier N, Garcia-Durán A, Weston J, Yakhnenko O. Translating embeddings for modeling multi-relational data. In: Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013, 2787−2795

[43] Wang Z, Zhang J, Feng J, Chen Z. Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence. 2014, 1112−1119

[44] Ji G, He S, Xu L, Liu K, Zhao J. Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015, 687−696

[45] Lin Y, Liu Z, Sun M, Liu Y, Zhu X. Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the 29th AAAI Conference on Artificial Intelligence. 2015, 2181−2187

[46] Nickel M, Tresp V, Kriegel H P. A three-way model for collective learning on multi-relational data. In: Proceedings of the 28th International Conference on International Conference on Machine Learning. 2011, 809−816

[47] Yang B, Yih W T, He X, Gao J, Deng L. Embedding entities and relations for learning and inference in knowledge bases. In: Proceedings of the 3rd International Conference on Learning Representations. 2015, 1−12

[48] Nickel M, Rosasco L, Poggio T. Holographic embeddings of knowledge graphs. In: Proceedings of the 30th AAAI Conference on Artificial Intelligence. 2016, 1955−1961

[49] Trouillon T, Welbl J, Riedel S, Gaussier É, Bouchard G. Complex embeddings for simple link prediction. In: Proceedings of the 33rd International Conference on International Conference on Machine Learning. 2016, 2071−2080

[50] Sun Z, Deng Z, Nie J, Tang J. Rotate: knowledge graph embedding by relational rotation in complex space. In: Proceedings of International Conference on Learning Representations. 2019

[51] Song T, Luo J, Huang L. Rot-pro: modeling transitivity by projection in knowledge graph embedding. In: Proceedings of the 35th International Conference on Neural Information Processing Systems. 2021, 24695–24706

[52] Chao L, He J, Wang T, Chu W. PairRE: knowledge graph embeddings via paired relation vectors. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 2021

[53] Li Z, Liu H, Zhang Z, Liu T, Xiong N. . Learning knowledge graph embedding with heterogeneous relation attention networks. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33( 8): 3961–3973

[54] Zhang Z, Li Z, Liu H, Xiong N . Multi-scale dynamic convolutional network for knowledge graph embedding. IEEE Transactions on Knowledge and Data Engineering, 2022, 34( 8): 2335–2347

[55] Chen Z, Yeh M Y, Kuo T. PASSLEAF: a pool-based semi-supervised learning framework for uncertain knowledge graph embedding. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence. 2021, 4019−4026

[56] Chen X, Boratko M, Chen M, Dasgupta S S, Li X L, McCallum A. Probabilistic box embeddings for uncertain knowledge graph reasoning. In: Proceedings of 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021

[57] Yang S, Zhang W, Tang R. Fast confidence prediction of uncertainty based on knowledge graph embedding. In: Proceedings of the 3rd International Conference on Algorithms, Computing and Artificial Intelligence. 2020, 54

[58] Liu F, Shen Y, Zhang T, Gao H . Entity-related paths modeling for knowledge base completion. Frontiers of Computer Science, 2020, 14( 5): 145311

[59] Hommersom A, Lucas P J F. An introduction to knowledge representation and reasoning in healthcare. In: Hommersom A, Lucas P J F, eds. Foundations of Biomedical Knowledge Representation. Cham: Springer, 2015, 9−32

[60] Zhang J, Wu T, Qi G. Gaussian metric learning for few-shot uncertain knowledge graph completion. In: Proceedings of the 26th International Conference on Database Systems for Advanced Applications. 2021, 256−271

[61] Chen X, Chen M, Shi W, Sun Y, Zaniolo C. Embedding uncertain knowledge graphs. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence and the 31st Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence. 2019, 413

[62] Galárraga L A, Teflioudi C, Hose K, Suchanek F. Amie: association rule mining under incomplete evidence in ontological knowledge bases. In: Proceedings of the 22nd International Conference on World Wide Web. 2013, 413−422

[63] Kimmig A, Bach S H, Broecheler M, Huang B, Getoor L. A short introduction to probabilistic soft logic. In: Proceedings of the 26th Neural Information Processing Systems. 2012, 1−4

[64] El Halaby M, Abdalla A . New phase transitions for formulas in Łukasiewicz logic. Frontiers of Computer Science, 2020, 14( 6): 146403

[65] Zhang Z, Cai J, Zhang Y, Wang J. Learning hierarchy-aware knowledge graph embeddings for link prediction. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020, 3065−3072

[66] Liu H, Wu Y, Yang Y. Analogical inference for multi-relational embeddings. In: Proceedings of the 34th International Conference on Machine Learning. 2017, 2168−2178

[67] Xu W, Zheng S, He L, Shao B, Yin J, Liu T. SEEK: segmented embedding of knowledge graphs. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020

[68] Kazemi S M, Poole D. Simple embedding for link prediction in knowledge graphs. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018

[69] Fan M, Zhou Q, Zheng T F. Learning embedding representations for knowledge inference on imperfect and incomplete repositories. In: Proceedings of 2016 IEEE/WIC/ACM International Conference on Web Intelligence. 2016, 42−48

[70] Kertkeidkachorn N, Liu X, Ichise R. GTransE: generalizing translation-based model on uncertain knowledge graph embedding. In: Proceedings of the 33rd Annual Conference of the Japanese Society for Artificial Intelligence. 2019, 170−178

[71] Hu J, Cheng R, Huang Z, Fang Y, Luo S. On embedding uncertain graphs. In: Proceedings of the 26th ACM on Conference on Information and Knowledge Management. 2017, 157−166

[72] Zhu YanLi, Yang XiaoPing, Wang Liang, Zang ZhiYu. TransRD: embedding of knowledge graph with asymmetric features (in Chinese). Journal of Chinese Information Processing, 2019, 33(11): 73−82

[73] Jia S, Xiang Y, Chen X, Wang K, Shi J. Triple trustworthiness measurement for knowledge graph. In: Proceedings of 2019 World Wide Web Conference. 2019, 2865−2871

[74] Broder A, Kumar R, Maghoul F, Raghavan P, Rajagopalan S, Stata R, Tomkins A, Wiener J. Graph structure in the web. Computer Networks, 2000, 33(1−6): 309−320

[75] Page L, Brin S, Motwani R, Winograd T. The pagerank citation ranking: bringing order to the Web. Stanford InfoLab. 1999

[76] Wu H, Pei Y, Yu J . Detecting academic experts by topic-sensitive link analysis. Frontiers of Computer Science in China, 2009, 3( 4): 445–456

[77] Balazevic I, Allen C, Hospedales T M. Tucker: Tensor factorization for knowledge graph completion. In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019

[78] Zou Y, Qiu D. Combining tensor decomposition and word embedding for asymmetrical relationship prediction in knowledge graphs. In: Proceedings of the 13th International Symposium on Computational Intelligence and Design. 2020

[79] Chen M, Weinberger K Q, Xu Z, Sha F . Marginalizing stacked linear denoising autoencoders. The Journal of Machine Learning Research, 2015, 16( 1): 3849–3875

[80] Szklarczyk D, Morris J H, Cook H, Kuhn M, Wyder S, Simonovic M, Santos A, Doncheva N T, Roth A, Bork P, Jensen L J, Von Mering C . The STRING database in 2017: quality-controlled protein–protein association networks, made broadly accessible. Nucleic Acids Research, 2017, 45( D1): D362–D368

Acknowledgements

This work was supported in part by the National Key Research and Development Program of China (Nos. 2020YFC2003502, 2021YFF0704101), the National Natural Science Foundation of China (Grant No. 62276038), the Natural Science Foundation of Chongqing (Nos. cstc2019jcyj-cxttX0002, cstc2021ycjh-bgzxm0013) and the Key Cooperation Project of Chongqing Municipal Education Commission (HZ20210-08).

RIGHTS & PERMISSIONS

2024 Higher Education Press