E²CNN: entity-type-enriched cascaded neural network for Chinese financial relation extraction

Abstract

Knowledge Graphs (KGs) are pivotal for effectively organizing and managing structured information across various applications. Financial KGs have been successfully employed in advancing applications such as audit, anti-fraud, and anti-money laundering. Despite their success, the construction of Chinese financial KGs has seen limited research due to the complex semantics. A significant challenge is the overlap triples problem, where entities feature in multiple relations within a sentence, hampering extraction accuracy – more than 39% of the triples in Chinese datasets exhibit the overlap triples. To address this, we propose the Entity-type-Enriched Cascaded Neural Network (E²CNN), leveraging special tokens for entity boundaries and types. E²CNN ensures consistency in entity types and excludes specific relations, mitigating overlap triple problems and enhancing relation extraction. Besides, we introduce the available Chinese financial dataset FINCORPUS.CN, annotated from annual reports of 2,000 companies, containing 48,389 entities and 23,368 triples. Experimental results on the DUIE dataset and FINCORPUS.CN underscore E²CNN’s superiority over state-of-the-art models.

Graphical abstract

Keywords

financial knowledge graph / overlap triples / cascaded neural network / relation extraction

References

[1] Saxena A, Chakrabarti S, Talukdar P. Question answering over temporal knowledge graphs. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 2021, 6663−6676

[2] Zhang M, He T T, Dong M . Meta-path reasoning of knowledge graph for commonsense question answering. Frontiers of Computer Science, 2024, 18( 1): 181303

[3] Collarana D, Galkin M, Traverso-Ribón I, Lange C, Vidal M E, Auer S. Semantic data integration for knowledge graph construction at query time. In: Proceedings of the 11th IEEE International Conference on Semantic Computing. 2017, 109−116

[4] Kim J, Choi K S. Unsupervised fact checking by counter-weighted positive and negative evidential paths in a knowledge graph. In: Proceedings of the 28th International Conference on Computational Linguistics. 2020, 1677−1686

[5] Sang E F T K, De Meulder F. Introduction to the CoNLL-2003 shared task: language-independent named entity recognition. In: Proceedings of the 7th Conference on Natural Language Learning at HLT-NAACL 2003. 2003, 142−147

[6] Ratinov L, Roth D. Design challenges and misconceptions in named entity recognition. In: Proceedings of the 13th Conference on Computational Natural Language Learning. 2009, 147−155

[7] Zelenko D, Aone C, Richardella A. Kernel methods for relation extraction. In: Proceedings of 2002 Conference on Empirical Methods in Natural Language Processing. 2002, 71−78

[8] Bunescu R C, Mooney R J. A shortest path dependency kernel for relation extraction. In: Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing. 2005, 724−731

[9] Zheng Z Y, Liu Y, Li D, Zhang X J . Distant supervised relation extraction based on residual attention. Frontiers of Computer Science, 2022, 16( 6): 166336

[10] Haussmann S, Seneviratne O, Chen Y, Ne’eman Y, Codella J, Chen C H, McGuinness D L, Zaki M J. FoodKG: a semantics-driven knowledge graph for food recommendation. In: Proceedings of the 18th International Semantic Web Conference. 2019, 146−162

[11] Kumar A, Bharadwaj A G, Starly B, Lynch C. FabKG: a knowledge graph of manufacturing science domain utilizing structured and unconventional unstructured knowledge source. In: Proceedings of the Workshop on Structured and Unstructured Knowledge Integration (SUKI). 2022, 1−8

[12] Kang Y Z, Jia N, Cui R B, Deng J . A graph-based semi-supervised reject inference framework considering imbalanced data distribution for consumer credit scoring. Applied Soft Computing, 2021, 105: 107259

[13] Van Belle R, Mitrović S, De Weerdt J. Representation learning in graphs for credit card fraud detection. In: Proceedings of the 4th ECML PKDD Workshop on Mining Data for Financial Applications. 2020, 32−46

[14] Zhan Q, Yin H. A loan application fraud detection method based on knowledge graph and neural network. In Proceedings of the 2nd International Conference on Innovation in Artificial Intelligence. 2018, 111−115

[15] Iglesias-Molina A, Chaves-Fraga D, Priyatna F, Corcho O. Towards the definition of a language-independent mapping template for knowledge graph creation. In: Proceedings of the 3rd International Workshop on Capturing Scientific Knowledge Co-located with the 10th International Conference on Knowledge Capture. 2019, 33−36

[16] Wei Z, Su J, Wang Y, Tian Y, Chang Y. A novel cascade binary tagging framework for relational triple extraction. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 1476−1488

[17] Wang G, Zeng Y, Li R H, Qin H, Shi X, Xia Y, Shang X, Hong L . Temporal graph cube. IEEE Transactions on Knowledge and Data Engineering, 2023, 35( 12): 13015–13030

[18] Cui Y, Che W, Liu T, Qin B, Wang S, Hu G. Revisiting pre-trained models for Chinese natural language processing. In: Proceedings of Findings of the Association for Computational Linguistics: EMNLP 2020. 2020, 657−668

[19] Xu W, Chen Y, Ouyang J. A streamlined span-based factorization method for few shot named entity recognition. In: Proceedings of 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation. 2024, 1673−1683

[20] Zhong Z, Chen D. A frustratingly easy approach for entity and relation extraction. In: Proceedings of 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021, 50−61

[21] Dixit K, Al-Onaizan Y. Span-level model for relation extraction. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019, 5308−5314

[22] Yu J, Bohnet B, Poesio M. Named entity recognition as dependency parsing. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 6470−6476

[23] Yan H, Sun Y, Li X, Qiu X. An embarrassingly easy but strong baseline for nested named entity recognition. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. 2023, 1442−1452

[24] Li S, He W, Shi Y, Jiang W, Liang H, Jiang Y, Zhang Y, Lyu Y, Zhu Y. DuIE: a large-scale Chinese dataset for information extraction. In: Proceedings of the 8th CCF International Conference on Natural Language Processing and Chinese Computing. 2019, 791−800

[25] Wadden D, Wennberg U, Luan Y, Hajishirzi H. Entity, relation, and event extraction with contextualized span representations. In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 5784−5789

[26] Ye D, Lin Y, Li P, Sun M. Packed levitated marker for entity and relation extraction. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 2022, 4904−4917

[27] Ren F, Zhang L, Zhao X, Yin S, Liu S, Li B. A simple but effective bidirectional framework for relational triple extraction. In: Proceedings of the 15th ACM International Conference on Web Search and Data Mining. 2022, 824−832

[28] Ning J, Yang Z, Sun Y, Wang Z, Lin H. OD-RTE: a one-stage object detection framework for relational triple extraction. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. 2023, 11120−11135

[29] Dai H, Peng X, Shi X, He L, Xiong Q, Jin H . Reveal training performance mystery between TensorFlow and PyTorch in the single GPU environment. Science China Information Sciences, 2022, 65: 112103

[30] Lei Z, Ul Haq A, Zeb A, Suzauddola M, Zhang D . Is the suggested food your desired? Multi-modal recipe recommendation with demand-based knowledge graph. Expert Systems with Applications, 2021, 186: 115708

[31] Zehra S, Mohsin S F M, Wasi S, Jami S I, Siddiqui M S, Syed M K U R R . Financial knowledge graph based financial report query system. IEEE Access, 2021, 9: 69766–69782

[32] Rony M R A H, Chaudhuri D, Usbeck R, Lehmann J . Tree-KGQA: an unsupervised approach for question answering over knowledge graphs. IEEE Access, 2022, 10: 50467–50478

[33] Shang C, Wang G, Qi P, J H. Improving time sensitivity for question answering over temporal knowledge graphs. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 2022, 8017−8026

[34] Lin P, Song Q, Wu Y . Fact checking in knowledge graphs with ontological subgraph patterns. Data Science and Engineering, 2018, 3( 4): 341–358

[35] Lin P, Song Q, Wu Y, Pi J . Discovering patterns for fact checking in knowledge graphs. Journal of Data and Information Quality, 2019, 11( 3): 13

[36] Cudré-Mauroux P . Leveraging knowledge graphs for big data integration: the XI pipeline. Semantic Web, 2020, 11( 1): 13–17

[37] Dandan R, Despres S. DIKG2: a semantic data integration approach for knowledge graphs generation from Web forms. In: Proceedings of the 34th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems. 2021, 255−260

[38] Lehmann J, Isele R, Jakob M, Jentzsch A, Kontokostas D, Mendes P N, Hellmann S, Morsey M, van Kleef P, Auer S, Bizer C . DBpedia – a large-scale, multilingual knowledge base extracted from Wikipedia. Semantic Web, 2015, 6( 2): 167–195

[39] Suchanek F M, Kasneci G, Weikum G. Yago: a core of semantic knowledge. In: Proceedings of the 16th International Conference on World Wide Web. 2007, 697−706

[40] Mitchell T, Cohen W, Hruschka E, Talukdar P, Yang B, Betteridge J, Carlson A, Dalvi B, Gardner M, Kisiel B, Krishnamurthy J, Lao N, Mazaitis K, Mohamed T, Nakashole N, Platanios E, Ritter A, Samadi M, Settles B, Wang R, Wijaya D, Gupta A, Chen X, Saparov A, Greaves M, Welling J . Never-ending learning. Communications of the ACM, 2018, 61( 5): 103–115

[41] Elhammadi S, Lakshmanan L V S, Ng R, Simpson M, Huai B, Wang Z, Wang L. A high precision pipeline for financial knowledge graph construction. In: Proceedings of the 28th International Conference on Computational Linguistics. 2020, 967−977

[42] Kim K, Hur Y, Kim G, Lim H . GREG: a global level relation extraction with knowledge graph embedding. Applied Sciences, 2020, 10( 3): 1181

[43] McCallum A, Freitag D, Pereira F C N. Maximum entropy Markov models for information extraction and segmentation. In: Proceedings of the 17th International Conference on Machine Learning. 2000, 591−598

[44] Brin S. Extracting patterns and relations from the world wide web. In: Proceedings of International Workshop on the World Wide Web and Databases. 1998, 172−183

[45] Hasegawa T, Sekine S, Grishman R. Discovering relations among named entities from large corpora. In: Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics. 2004, 415−422

[46] Gong C, Li Z, Xia Q, Chen W, Zhang M . Hierarchical LSTM with char-subword-word tree-structure representation for Chinese named entity recognition. Science China Information Sciences, 2020, 63( 10): 202102

[47] Mintz M, Bills S, Snow R, Jurafsky D. Distant supervision for relation extraction without labeled data. In: Proceedings of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. 2009, 1003−1011

[48] Gormley M R, Yu M, Dredze M. Improved relation extraction with feature-rich compositional embedding models. In: Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing. 2015, 1774−1784

[49] Florian R, Hassan H, Ittycheriah A, Jing H, Kambhatla N, Luo X, Nicolov N, Roukos S. A statistical model for multilingual entity detection and tracking. In: Proceedings of Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics. 2004, 1−8

[50] Florian R, Jing H, Kambhatla N, Zitouni I. Factorizing complex models: a case study in mention detection. In: Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics. 2006, 473−480

[51] Zeng D, Liu K, Lai S, Zhou G, Zhao J. Relation classification via convolutional deep neural network. In: Proceedings of the 25th International Conference on Computational Linguistics. 2014, 2335−2344

[52] Zhou P, Shi W, Tian J, Qi Z, Li B, Hao H, Xu B. Attention-based bidirectional long short-term memory networks for relation classification. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 207−212

[53] Qiao B, Zou Z, Huang Y, Fang K, Zhu X, Chen Y . A joint model for entity and relation extraction based on BERT. Neural Computing and Applications, 2022, 34( 5): 3471–3481

[54] Zheng S, Wang F, Bao H, Hao Y, Zhou P, Xu B. Joint extraction of entities and relations based on a novel tagging scheme. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017, 1227−1236

[55] Bekoulis G, Deleu J, Demeester T, Develder C . Joint entity recognition and relation extraction as a multi-head selection problem. Expert Systems with Applications, 2018, 114: 34–45

[56] Yu X, Lam W. Jointly identifying entities and extracting relations in encyclopedia text via a graphical model approach. In: Proceedings of the 23rd International Conference on Computational Linguistics. 2010, 1399−1407

[57] Li Q, Ji H. Incremental joint extraction of entity mentions and relations. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. 2014, 402−412

[58] Gupta P, Schütze H, Andrassy B. Table filling multi-task recurrent neural network for joint entity and relation extraction. In: Proceedings of the 26th International Conference on Computational Linguistics. 2016, 2537−2547

[59] Katiyar A, Cardie C. Going out on a limb: joint extraction of entity mentions and relations without dependency trees. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017, 917−928

[60] Zeng X, Zeng D, He S, Liu K, Zhao J. Extracting relational facts by an end-to-end neural model with copy mechanism. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018, 506−514

[61] Miwa M, Bansal M. End-to-end relation extraction using LSTMs on sequences and tree structures. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 1105−1116

[62] Nie Y, Tian Y, Song Y, Ao X, Wan X. Improving named entity recognition with attentive ensemble of syntactic information. In: Proceedings of Findings of the Association for Computational Linguistics. 2020, 4231−4245

[63] Li Z, Ding N, Liu Z, Zheng H, Shen Y. Chinese relation extraction with multi-grained information and external linguistic knowledge. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019, 4377−4386

[64] Xuan Z, Bao R, Jiang S. FGN: fusion glyph network for Chinese named entity recognition. In: Proceedings of the 5th China Conference on Knowledge Graph and Semantic Computing: Knowledge Graph and Cognitive Intelligence. 2020, 28−40

[65] Fu T J, Li P H, Ma W Y. GraphRel: modeling text as relational graphs for joint entity and relation extraction. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019, 1409−1418

[66] Shang Y M, Huang H, Mao X. OneRel: joint entity and relation extraction with one module in one step. In: Proceedings of the 36th AAAI Conference on Artificial Intelligence. 2022, 11285−11293

[67] Wang Y, Yu B, Zhang Y, Liu T, Zhu H, Sun L. TPLinker: single-stage joint extraction of entities and relations through token pair linking. In: Proceedings of the 28th International Conference on Computational Linguistics. 2020, 1572−1582

RIGHTS & PERMISSIONS

The Author(s) 2024. This article is published with open access at link.springer.com and journal.hep.com.cn