Chinese satellite frequency and orbit entity relation extraction method based on dynamic integrated learning

Abstract

Given the scarcity of Satellite Frequency and Orbit (SFO) resources, it holds paramount importance to establish a comprehensive knowledge graph of SFO field (SFO-KG) and employ knowledge reasoning technology to automatically mine available SFO resources. An essential aspect of constructing SFO-KG is the extraction of Chinese entity relations. Unfortunately, there is currently no publicly available Chinese SFO entity Relation Extraction (RE) dataset. Moreover, publicly available SFO text data contain numerous NA (representing for “No Answer”) relation category sentences that resemble other relation sentences and pose challenges in accurate classification, resulting in low recall and precision for the NA relation category in entity RE. Consequently, this issue adversely affects both the accuracy of constructing the knowledge graph and the efficiency of RE processes. To address these challenges, this paper proposes a method for extracting Chinese SFO text entity relations based on dynamic integrated learning. This method includes the construction of a manually annotated Chinese SFO entity RE dataset and a classifier combining features of SFO resource data. The proposed approach combines integrated learning and pre-training models, specifically utilizing Bidirectional Encoder Representation from Transformers (BERT). In addition, it incorporates one-class classification, attention mechanisms, and dynamic feedback mechanisms to improve the performance of the RE model. Experimental results show that the proposed method outperforms the traditional methods in terms of F1 value when extracting entity relations from both balanced and long-tailed datasets.

Keywords

Knowledge graph / Relation extraction / One-class classification / Satellite frequency and orbit resources / BERT

CRediT authorship contribution statement

Yuanzhi He: Funding acquisition, Supervision, Writing - review & editing. Zhiqiang Li: Investigation, Methodology, Software, Writing - original draft. Zheng Dou: Investigation, Writing - review & editing.

Declaration of Competing Interest

The authors declare that they have no conflicts of interest to this work.

References

[1] S. Ren, X. Yang, R. Wang, S. Liu, X. Sun, The interaction between the LEO satellite constellation and the space debris environment, Appl. Sci. 11 (20) (2021) 9490.

[2] S. Ji, S. Pan, E. Cambria, P. Marttinen, S.Y. Philip, A survey on knowledge graphs: representation, acquisition, and applications, IEEE Trans. Neural Netw. Learn. Syst. 33 (2) (2021) 494-514.

[3] V.F. Milas, D. Vouyioukas, N. Moraitis, P. Constantinou, Spectrum planning and per-formance evaluation between heterogeneous satellite networks, Eur. J. Oper. Res. 191 (3) (2008) 1132-1138.

[4] A. Li, X. Wang, W. Wang, A. Zhang, B. Li, A survey of relation extraction of knowl-edge graphs, in: Web and Big Data: APWeb-WAIM 2019 International Workshops, Springer, 2019, pp. 52-66.

[5] J. Cheng, J. Liu, X. Xu, D. Xia, L. Liu, V.S. Sheng, A review of Chinese named entity recognition, KSII Trans. Int. Inf. Syst. 15 (6) (2021) 2012-2030.

[6] M. Miwa, M. Bansal, End-to-end relation extraction using lstms on sequences and tree structures, arXiv preprint arXiv :1601.00770.

[7] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv : 1810.04805.

[8] B. Deng, X. Fan, L. Yang, Entity relation extraction method using semantic pattern, Comput. Eng. 33 (10) (2007) 212-214.

[9] S. Zhang, S. Zhang, G. Gao, Automatic entity relation extraction based on condi-tional random fields, in: 2008 Fifth International Conference on Fuzzy Systems and Knowledge Discovery, vol. 2, IEEE, 2008, pp. 286-290.

[10] X. Jiang, Q. Wang, P. Li, B. Wang,Relation extraction with multi-instance multi-label convolutional neural networks, in:Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics:Technical Papers, 2016, pp. 1471-1480.

[11] Y. Qin, W. Yang, K. Wang, R. Huang, F. Tian, S. Ao, Y. Chen, Entity relation extrac-tion based on entity indicators, Symmetry 13 (4) (2021) 539.

[12] Y. Liang, F. Ailian, Chinese text relation extraction based on a multi-channel con-volutional neural network, J. East China Norm. Univ. Natur. Sci. 2021 (3) (2021) 96.

[13] W. Wu, Y. Chen, J. Xu, Y. Zhang,Attention-based convolutional neural networks for Chinese relation extraction, in:Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data: 17th China National Conference, CCL 2018, and 6th International Symposium, NLP-NABD 2018, Pro-ceedings 17, Springer, 2018, pp. 147-158.

[14] K. Zhang, C. Xia, G. Liu, W. Wang, Y. Wu, Y. Zhang, Y. Yue,Chinese agricultural en-tity relation extraction via deep learning, in:Intelligent Computing Methodologies: 15th International Conference, ICIC 2019, Proceedings, Part III 15, Springer, 2019, pp. 528-534.

[15] E. Haihong, X. Zhou, M. Song,Distant supervised relation extraction based on recurrent convolutional piecewise neural network, in:Proceedings of the 2019 In-ternational Symposium on Signal Processing Systems, 2019, pp. 169-175.

[16] Z. He, Z. Zhou, L. Gan, J. Huang, Y. Zeng, Chinese entity attributes extraction based on bidirectional lstm networks, Int. J. Comput. Sci. Eng. 18 (1) (2019) 65-71.

[17] K. Ding, S. Liu, Y. Zhang, H. Zhang, X. Zhang, T. Wu, X. Zhou, A knowledge-enriched and span-based network for joint entity and relation extraction, Comput. Mater. Continua 68 (1) (2021) 377-389.

[18] M. Wei, Z. Xu, J. Hu,Entity relationship extraction based on bi-lstm and attention mechanism, in:2021 2nd International Conference on Artificial Intelligence and Information Systems, 2021, pp. 1-5.

[19] M. Mintz, S. Bills, R. Snow, D. Jurafsky,Distant supervision for relation extraction without labeled data, in:Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, 2009, pp. 1003-1011.

[20] K. Bollacker, R. Cook, P. Tufts, Freebase: a shared database of structured general human knowledge in: AAAI,vol. 7, 2007, pp. 1962-1963.

[21] D. Vrandeˇci´c,Wikidata: a new platform for collaborative data collection,in:Proceedings of the 21st International Conference on World Wide Web, 2012, pp. 1063-1064.

[22] Q. Zhao, D. Xu, J. Li, L. Zhao, F.A. Rajput, Knowledge guided distance supervision for biomedical relation extraction in Chinese electronic medical records, Expert Syst. Appl. 204 (2022) 117606.

[23] J. Zhang, K. Hao, X.-s. Tang, X. Cai, Y. Xiao, T. Wang, A multi-feature fusion model for Chinese relation extraction with entity sense, Knowl.-Based Syst. 206 (2020) 106348.

[24] W. Bu, W. Yang, D. Chen, T. Ding,Chinese relation extraction based on characters and words fusion, in:Third International Conference on Artificial Intelligence and Electromechanical Automation (AIEA 2022), vol. 12329, SPIE, 2022, pp. 319-330.

[25] L. Floridi, M. Chiriatti,Gpt-3: its nature, scope, limits, and consequences, Minds Mach. 30 (2020) 681-694.

[26] B. Qiao, Z. Zou, Y. Huang, K. Fang, X. Zhu, Y. Chen, A joint model for entity and relation extraction based on bert, Neural Comput. Appl. 68 (1) (2021) 3471-3481.

[27] H. Yu, Y. Cao, G. Cheng, P. Xie, Y. Yang, P. Yu,Relation extraction with bert-based pre-trained model, in: 2020 International Wireless Communications and Mobile Computing (IWCMC), IEEE, 2020, pp. 1382-1387.

[28] K. Yoon,Convolutional neural networks for sentence classification, in:Proceed-ings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014, pp. 1746-1751.

[29] Y. Cui, W. Che, T. Liu, B. Qin, Z. Yang, Pre-training with whole word masking for Chinese bert, IEEE/ACM Trans. Audio Speech Lang. Process. 29 (2021) 3504-3514.

[30] P. Oza, V.M. Patel, One-class convolutional neural network, IEEE Signal Process. Lett. 26 (2) (2018) 277-281.

[31] H. Zhang, J. Miao, Z. Liu, I.L. Wesson, J. Shang,Nlpir-parser: making Chinese and English semantic analysis easier and complete,in:15th International Conference on the Statistical Analysis of Textual Data, 2020, https://api.semanticscholar.org/CorpusID:232067668.