Pairwise tagging framework for end-to-end emotion-cause pair extraction

Abstract

Emotion-cause pair extraction (ECPE) aims to extract all the pairs of emotions and corresponding causes in a document. It generally contains three subtasks, emotions extraction, causes extraction, and causal relations detection between emotions and causes. Existing works adopt pipelined approaches or multi-task learning to address the ECPE task. However, the pipelined approaches easily suffer from error propagation in real-world scenarios. Typical multi-task learning cannot optimize all tasks globally and may lead to suboptimal extraction results. To address these issues, we propose a novel framework, Pairwise Tagging Framework (PTF), tackling the complete emotion-cause pair extraction in one unified tagging task. Unlike prior works, PTF innovatively transforms all subtasks of ECPE, i.e., emotions extraction, causes extraction, and causal relations detection between emotions and causes, into one unified clause-pair tagging task. Through this unified tagging task, we can optimize the ECPE task globally and extract more accurate emotion-cause pairs. To validate the feasibility and effectiveness of PTF, we design an end-to-end PTF-based neural network and conduct experiments on the ECPE benchmark dataset. The experimental results show that our method outperforms pipelined approaches significantly and typical multi-task learning approaches.

Graphical abstract

Keywords

emotion-cause pair extraction / pairwise tagging framework / end-to-end / neural network

Zhen Wu is a PhD candidate of the National Key Lab for Novel Software Technology, Department of Computer Science & Technology, Nanjing University, China. He received his BEng degree from Nanjing University of Science and Technology, China in 2016. In the same year, he was admitted to pursue a PhD degree at Nanjing University, China. His research interests include natural language processing and sentiment analysis

Xinyu Dai is a Professor of the School of Artificial Intelligence, Nanjing University, China. He received his PhD degree in the Department of Computer Science & Technology, Nanjing University, China in 2005. He joined Nanjing University, as an assistant professor in 2005, worked as an associate professor from 2008, and became professor in 2017. His research interests majorly include language processing and intelligence, knowledge engineering, and human-machine communication

Rui Xia is a Professor of the School of Computer Science and Engineering, Nanjing University of Science and Technology, China. He received the BSc degree from Southeast University, China in 2004, the MSc degree from East China University of Science and Technology, China in 2007, and the PhD degree from the Institute of Automation, Chinese Academy of Sciences, China in 2011. His research interests include natural language processing, machine learning, and data mining

References

[1] Fox E. Emotion Science: Cognitive and Neuroscientific Approaches to Understanding Human Emotions. New York: Palgrave Macmillan, 2008

[2] Brosch T, Scherer K R, Grandjean D, Sander D . The impact of emotion on perception, attention, memory, and decision-making. Swiss Medical Weekly, 2013, 143: w13786

[3] Quan C, Ren F. Construction of a blog emotion corpus for Chinese emotional expression analysis. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing. 2009, 1446– 1454

[4] Bellegarda J R. Emotion analysis using latent affective folding and embedding. In: Proceedings of the 2010 NAACL HLT Workshop on Computational Approaches to Analysis and Generation of Emotion in Text. 2010, 1– 9

[5] Qadir A, Riloff E. Learning emotion indicators from tweets: hashtags, hashtag patterns, and phrases. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014, 1203– 1209

[6] Lee S Y M, Chen Y, Huang C R. A text-driven rule-based system for emotion cause detection. In: Proceedings of the 2010 NAACL HLT Workshop on Computational Approaches to Analysis and Generation of Emotion in Text. 2010, 45– 53

[7] Russo I, Caselli T, Rubino F, Boldrini E, Martínez-Barco P. EMOCause: an easy-adaptable approach to extract emotion cause contexts. In: Proceedings of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis. 2011, 153– 160

[8] Neviarouskaya A, Aono M. Extracting causes of emotions from text. In: Proceedings of the 6th International Joint Conference on Natural Language Processing. 2013, 932– 936

[9] Gao K, Xu H, Wang J . A rule-based approach to emotion cause detection for Chinese micro-blogs. Expert Systems with Applications, 2015, 42( 9): 4517– 4528

[10] Gui L, Wu D, Xu R, Lu Q, Zhou Y. Event-driven emotion cause extraction with corpus construction. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2016, 1639– 1649

[11] Cheng X, Chen Y, Cheng B, Li S, Zhou G . An emotion cause corpus for Chinese microblogs with multiple-user structures. ACM Transactions on Asian and Low-Resource Language Information Processing, 2018, 17( 1): 6

[12] Fan C, Yan H, Du J, Gui L, Bing L, Yang M, Xu R, Mao R. A knowledge regularized hierarchical approach for emotion cause analysis. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 5614– 5624

[13] Xia R, Ding Z. Emotion-cause pair extraction: a new task to emotion analysis in texts. In: Proceedings of the 57th Conference of the Association for Computational Linguistics. 2019, 1003– 1012

[14] Song H, Zhang C, Li Q, Song D. End-to-end emotion-cause pair extraction via learning to link. 2020, arXiv preprint arXiv: 2002.10710

[15] Tang H, Ji D, Zhou Q . Joint multi-level attentional model for emotion detection and emotion-cause pair extraction. Neurocomputing, 2020, 409: 329– 340

[16] Ding Z, Xia R, Yu J. ECPE-2D: emotion-cause pair extraction based on joint two-dimensional representation, interaction and prediction. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 3161– 3170

[17] Wei P, Zhao J, Mao W. Effective inter-clause modeling for end-to-end emotion-cause pair extraction. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 3171– 3181

[18] Lee S Y M, Chen Y, Huang C R, Li S S . Detecting emotion causes with a linguistic rule-based approach. Computational Intelligence, 2013, 29( 3): 390– 416

[19] Li W, Xu H . Text-based emotion classification using emotion cause extraction. Expert Systems with Applications, 2014, 41( 4): 1742– 1749

[20] Gao K, Xu H, Wang J. Emotion cause detection for Chinese micro-blogs based on ECOCC model. In: Proceedings of the 19th Pacific-Asia Conference on Knowledge Discovery and Data Mining. 2015, 3– 14

[21] Yada S, Ikeda K, Hoashi K, Kageura K. A bootstrap method for automatic rule acquisition on emotion cause extraction. In: Proceedings of the 2017 IEEE International Conference on Data Mining Workshops. 2017, 414– 421

[22] Chen Y, Lee S Y M, Li S, Huang C R. Emotion cause detection with linguistic constructions. In: Proceedings of the 23rd International Conference on Computational Linguistics. 2010, 179– 187

[23] Gui L, Yuan L, Xu R, Liu B, Lu Q, Zhou Y. Emotion cause detection with linguistic construction in Chinese Weibo text. In: Proceedings of the 3rd CCF International Conference on Natural Language Processing and Chinese Computing. 2014, 457– 464

[24] Ghazi D, Inkpen D, Szpakowicz S. Detecting emotion stimuli in emotion-bearing sentences. In: Proceedings of the 16th International Conference on Computational Linguistics and Intelligent Text Processing. 2015, 152– 165

[25] Gui L, Xu R, Lu Q, Wu D, Zhou Y. Emotion cause extraction, a challenging task with corpus construction. In: Proceedings of the 5th National Conference on Social Media Processing. 2016, 98– 109

[26] Xu R, Hu J, Lu Q, Wu D, Gui L . An ensemble approach for emotion cause detection with event extraction and multi-kernel SVMs. Tsinghua Science and Technology, 2017, 22( 6): 646– 659

[27] Gui L, Hu J, He Y, Xu R, Lu Q, Du J. A question answering approach to emotion cause extraction. 2017, arXiv preprint arXiv: 1708.05482

[28] Li X, Song K, Feng S, Wang D, Zhang Y. A co-attention neural network model for emotion cause analysis with emotional context awareness. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018, 4752– 4757

[29] Yu X, Rong W, Zhang Z, Ouyang Y, Xiong Z . Multiple level hierarchical network-based clause selection for emotion cause extraction. IEEE Access, 2019, 7: 9071– 9079

[30] Ding Z, He H, Zhang M, Xia R. From independent prediction to reordered prediction: integrating relative position and global label information to emotion cause identification. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 6343– 6350

[31] Xia R, Zhang M, Ding Z. RTHN: a RNN-transformer hierarchical network for emotion cause extraction. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019, 5285– 5291

[32] Wu S, Chen F, Wu F, Huang Y, Li X. A multi-task learning neural network for emotion-cause pair extraction. In: Proceedings of the 24th European Conference on Artificial Intelligence. 2020, 2212– 2219

[33] Fan C, Yuan C, Gui L, Zhang Y, Xu R . Multi-task sequence tagging for emotion-cause pair extraction via tag distribution refinement. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29: 2339– 2350

[34] Cheng Z, Jiang Z, Yin Y, Yu H, Gu Q. A symmetric local search network for emotion-cause pair extraction. In: Proceedings of the 28th International Conference on Computational Linguistics. 2020, 139– 149

[35] Yu J, Liu W, He Y, Zhang C . A mutually auxiliary multitask model with self-distillation for emotion-cause pair extraction. IEEE Access, 2021, 9: 26811– 26821

[36] Fan C, Yuan C, Du J, Gui L, Yang M, Xu R. Transition-based directed graph construction for emotion-cause pair extraction. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 3707– 3717

[37] Yuan C, Fan C, Bao J, Xu R. Emotion-cause pair extraction as sequence labeling based on a novel tagging scheme. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 2020, 3568– 3573

[38] Cheng Z, Jiang Z, Yin Y, Li N, Gu Q . A unified target-oriented sequence-to-sequence model for emotion-cause pair extraction. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29: 2779– 2791

[39] Yang Z, Yang D, Dyer C, He X, Smola A, Hovy E. Hierarchical attention networks for document classification. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2016, 1480− 1489

[40] Wu Z, Dai X Y, Yin C, Huang S, Chen J. Improving review representations with user attention and product attention for sentiment classification. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018, 5989− 5996

[41] Hochreiter S, Schmidhuber J . Long short-term memory. Neural Computation, 1997, 9( 8): 1735– 1780

[42] Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. In: Proceedings of the 3rd International Conference on Learning Representations. 2015

[43] Devlin J, Chang M W, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019, 4171− 4186

[44] Mikolov T, Sutskever I, Chen K, Corrado G, Dean J. Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013, 3111− 3119

[45] Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R . Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 2014, 15( 1): 1929– 1958

[46] Kingma D P, Ba J. Adam: a method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations. 2015

[47] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser Ł, Polosukhin I. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6000− 6010

Acknowledgements

This work was supported by the National Natural Science Foundation of China (NSFC) (Grant Nos. 61976114 and 61936012) and the National Key R&D Program of China (2018YFB1005102).

RIGHTS & PERMISSIONS

2023 Higher Education Press