End-to-end multi-granulation causality extraction model

Abstract

Causality extraction has become a crucial task in natural language processing and knowledge graph. However, most existing methods divide causality extraction into two subtasks: extraction of candidate causal pairs and classification of causality. These methods result in cascading errors and the loss of associated contextual information. Therefore, in this study, based on graph theory, an End-to-end Multi-Granulation Causality Extraction model (EMGCE) is proposed to extract explicit causality and directly mine implicit causality. First, the sentences are represented on different granulation layers, that contain character, word, and contextual string layers. The word layer is fine-grained into three layers: word-index, word-embedding and word-position-embedding layers. Then, a granular causality tree of dataset is built based on the word-index layer. Next, an improved tagREtriplet algorithm is designed to obtain the labeled causality based on the granular causality tree. It can transform the task into a sequence labeling task. Subsequently, the multi-granulation semantic representation is fed into the neural network model to extract causality. Finally, based on the extended public SemEval 2010 Task 8 dataset, the experimental results demonstrate that EMGCE is effective.

Keywords

Causality extraction / Granular computing / Granular causality tree / Semantic representation / Sequence labeling

References

[1] H. Wang, K. Qin, G. Lu, J. Yin, R.Y. Zakari, J.W. Owusu, Document-level relation extraction using evidence reasoning on RST-GRAPH, Knowl. Base Syst. 228 (2021) 107274.

[2] A. Sobrino, C. Puente, J.A. Olivas, Extracting answers from causal mechanisms in a medical document, Neurocomputing 135 (2014) 53-60.

[3] Y. Lan, S. Wang, J. Jiang, Knowledge base question answering with a matching-aggregation model and question-specific contextual relations, IEEE ACM Trans. Audio Speech Lang. Process. 27 (10) (2019) 1629-1638.

[4] Z. Yang, H. Yu, J. Tang, H. Liu, Toward keyword extraction in constrained information retrieval in vehicle social network, IEEE Trans. Veh. Technol. 68 (5)(2019) 4285-4294.

[5] J. Cheon, Y. Ko, Parallel sentence extraction to improve cross-language information retrieval from wikipedia, J. Inf. Sci. 47 (2) (2021) 281-293.

[6] Q. Zhang, Z. Ai, J. Zhang, G. Wang, A novel fast constructing neighborhood covering algorithm for efficient classification, Knowl. Base Syst. 225 (2021) 107104.

[7] Q. Zhang, M. Gao, F. Zhao, G. Wang, Fuzzy-entropy-based game theoretic shadowed sets: a novel game perspective from uncertainty, IEEE Trans. Fuzzy Syst. 30 (3)(2022) 597-609.

[8] B. Liang, H. Su, L. Gui, E. Cambria, R. Xu, Aspect-based sentiment analysis via affective knowledge enhanced graph convolutional networks, Knowl. Base Syst. 235 (2022) 107643.

[9] Y. Chen, C. Wu, Y. Huang, Enhancing structure modeling for relation extraction with fine-grained gating and co-attention, Neurocomputing 467 (2022) 282-291.

[10] M. Zhou, D. Ji, F. Li, Relation extraction in dialogues: a deep learning model based on the generality and specialty of dialogue text, IEEE ACM Trans. Audio Speech Lang. Process. 29 (2021) 2015-2026.

[11] S. Zhao, T. Liu, S. Zhao, Y. Chen, J.-Y. Nie, Event causality extraction based on connectives analysis, Neurocomputing 173 (2016) 1943-1950.

[12] P. Vrablecova, M. Simko, Supporting semantic annotation of educational content by automatic extraction of hierarchical domain relationships, IEEE Trans. Learn. Technol. 9 (3) (2016) 285-298.

[13] R. Li, D. Li, J. Yang, F. Xiang, H. Ren, S. Jiang, L. Zhang, Joint extraction of entities and relations via an entity correlated attention neural model, Inf. Sci. 581 (2021) 179-193.

[14] Z. Li, Q. Li, X. Zou, J. Ren, Causality extraction based on self-attentive bilstm-crf with transferred embeddings, Neurocomputing 423 (2021) 207-219.

[15] J.T. Zhou, H. Zhang, D. Jin, X. Peng, Dual adversarial transfer for sequence labeling, IEEE Trans. Pattern Anal. Mach. Intell. 43 (2) (2021) 434-446.

[16] L.A. Zadeh, Some reflections on soft computing, granular computing and their roles in the conception, design and utilization of information/intelligent systems, Soft Comput. 2 (1) (1998) 23-25.

[17] W. Ding, C. Lin, A.W. Liew, I. Triguero, W. Luo, Current trends of granular data mining for biomedical data analysis, Inf. Sci. 510 (2020) 341-343.

[18] C. Lu, P. Xu, L. Cong, Fault diagnosis model based on granular computing and echo state network, Eng. Appl. Artif. Intell. 94 (2020) 103694.

[19] Q. Zhang, Y. Cheng, F. Zhao, G. Wang, S. Xia, Optimal scale combination selection integrating three-way decision with hasse diagram, IEEE Transact. Neural Networks Learn. Syst. 33 (8) (2022) 3675-3689.

[20] Q. Zhang, C. Yuhong, Y. Jie, G. Wang, Fuzzy entropy: a more comprehensible perspective for interval shadowed sets of fuzzy sets, IEEE Trans. Fuzzy Syst. 28 (11)(2020) 3008-3022.

[21] J. Yang, G. Wang, Q. Zhang, Knowledge distance measure in multigranulation spaces of fuzzy equivalence relations, Inf. Sci. 448-449 (2018) 18-35.

[22] S. Fu, G. Wang, J. Xu, hier2vec: interpretable multi-granular representation learning for hierarchy in social networks, Int. J. Mach. Learn. Cybern. 12 (9) (2021) 2543-2557.

[23] Y. Yao, Three-way decision and granular computing, Int. J. Approx. Reason. 103 (2018) 107-123.

[24] M. PejiABach, A. KrstiA, S. Seljan, L. Turulja, Text mining for big data analysis in financial sector: a literature review, z, Sustainability 11 (2019) 1277.

[25] N.C. Dang, M.N. Moreno-GarcA-a, F. De la Prieta, Sentiment analysis based on deep learning: a comparative study, Electronics 9 (2020) 483.

[26] E. Stai, S. Kafetzoglou, E. Tsiropoulou, S. Papavassiliou, A holistic approach for personalization, relevance feedback & recommendation in enriched multimedia content, Multimed. Tool. Appl. 77 (1) (2018) 283-326.

[27] S. Zheng, F. Wang, H. Bao, Y. Hao, P. Zhou, B. Xu,Joint extraction of entities and relations based on a novel tagging scheme, in: R. Barzilay, M. Kan (Eds.), Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics,vol. 1, Long Papers, Vancouver, Canada, 2017, pp. 1227-1236. July 30 -August 4.

[28] D. Dai, X. Xiao, Y. Lyu, S. Dou, Q. She, H. Wang,Joint extraction of entities and overlapping relations using position-attentive sequence labeling, in:The Thirty-Third AAAI Conference on Artificial Intelligence, 2019, pp. 6300-6308. Honolulu, Hawaii, USA, January 27 - February 1.

[29] X. Ma, E.H. Hovy,End-to-end sequence labeling via bi-directional lstm-cnns-crf, in:Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, vol. 1, Long Papers, Berlin, Germany, 2016, pp. 1064-1074.

[30] K. Cho, B. Van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, Y. Bengio, Learning phrase representations using rnn encoder-decoder for statistical machine translation, in: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, A meeting of SIGDAT, a Special Interest Group of the ACL, Doha, Qatar, 2014, pp. 1724-1734. October 25-29.

[31] X. Xue, Y. Gao, M. Liu, X. Sun, W. Zhang, J. Feng, Gru-based capsule network with an improved loss for personnel performance prediction, Appl. Intell. 51 (7) (2021) 4730-4743.

[32] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, L. Kaiser, I. Polosukhin, Attention is all you need, Adv. Neural Inform.Proc. Syst. 30:Ann. Conf. Neural Inf. Proc. Syst. 2017 (2017) 5998-6008. December 4-9, Long Beach, CA, USA.

[33] L. Hou, J. Zhang, O. Wu, T. Yu, Z. Wang, Z. Li, J. Gao, Y. Ye, R. Yao, Method and dataset entity mining in scientific literature: a CNN þ bilstm model with self-attention, Knowl. Base Syst. 235 (2022) 107621.

[34] L. Yan, J. Song, Granular trees based on different data sets and their modeling applications, Comput. Sci. 41 (3) (2014) 258-262.

[35] X. Han, P. Yu, Z. Liu, M. Sun, P. Li,Hierarchical relation extraction with coarse-to-fine grained attention, in:Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018, pp. 2236-2245. Brussels, Belgium, October 31 - November 4.

[36] A. Komninos, S. Manandhar, Dependency based embeddings for sentence classification tasks, in: K. Knight, A. Nenkova, O. Rambow (Eds.), The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016, pp. 1490-1500. San Diego California, USA, June 12-17, 2016.

[37] A. Akbik, D. Blythe, R. Vollgraf,Contextual string embeddings for sequence labeling, in:Proceedings of the 27th International Conference on Computational Linguistics, 2018, pp. 1638-1649. Santa Fe, New Mexico, USA, August 20-26.

[38] B. Copley, F. Martin, Causation in Grammatical Structures, 2014.

[39] J.A. Bondy, U.S.R. Murty, et al., Graph Theory with Applications, 1976.

[40] J. Li, G. Huang, J. Chen, Y. Wang, Dual CNN for relation extraction with knowledge-based attention and word embeddings, Comput. Intell. Neurosci. 2019 (2019) 6789520.

[41] Q. Li, L. Li, W. Wang, Q. Li, J. Zhong, A comprehensive exploration of semantic relation extraction via pre-trained cnns, Knowl. Base Syst. 194 (2020) 105488.

[42] X. Zeng, D. Zeng, S. He, K. Liu, J. Zhao,Extracting relational facts by an end-to-end neural model with copy mechanism, in:Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, vol. 1, Long Papers, 2018, pp. 506-514. Melbourne, Australia, July 15-20.

[43] J. Devlin, M. Chang, K. Lee, K. Toutanova, BERT: pre-training of deep bidirectional transformers for language understanding,in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1, Long and Short Papers, Minneapolis, MN, USA, 2019, pp. 4171-4186. June 2-7.