Negative focus detection via hypergraph attention LSTM network and LLM-driven data augmentation

Abstract

Negative focuses are the most prominent negated texts for a negative cue or verbal negation in a negative statement. Although previous work adopted sequence labelling framework using LSTM and CRF networks, Negative Focus Detection (NFD) is still faced with several disadvantages involving with data limitation, coarse-grained encoding, and insufficient dependencies of the sequence of words. To solve these problems, we firstly apply data augmentation driven by Large Language Models (LLMs) to produce more samples. Then, we propose a novel HyperGraph attention LSTM network (LSTM-HyG) to capture high-level semantics for sentences, and negated verbs, negative cues. Finally, we predict negative focuses by a fine-grained label scheme that can learn adequate sequential dependency relationship of words. Experimental results on PB-FOC and CNeSp datasets can prove that our proposed model is superior to state-of-the-arts.

Graphical abstract

Keywords

negative focus / LLM-driven data augmentation / HyperGraph attention network

References

[1] Morante R, Sporleder C . Modality and negation: an introduction to the special issue. Computational Linguistics, 2012, 38( 2): 223–260

[2] Morante R, Blanco E. *SEM 2012 shared task: resolving the scope and focus of negation. In: Proceedings of the 1st Joint Conference on Lexical and Computational Semantics. 2012, 265−274

[3] Blanco E, Moldovan D. Semantic representation of negation using focus detection. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011, 581−589

[4] Rosenberg S, Bergler S. UConcordia: CLaC negation focus detection at *Sem 2012. In: Proceedings of the 1st Joint Conference on Lexical and Computational Semantics. 2012, 294−300

[5] Blanco E, Moldovan D. Fine-grained focus for pinpointing positive implicit meaning from negated statements. In: Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2012, 456−465

[6] Zou B, Zhou G, Zhu Q. Negation focus identification with contextual discourse information. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. 2014, 522−530

[7] Zou B, Zhou G, Zhu Q. Unsupervised negation focus identification with word-topic graph model. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015, 1632−1636

[8] Shen L, Zou B, Hong Y, Zhou G, Zhu Q, Aw A. Negative focus detection via contextual attention mechanism. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019, 2251−2261

[9] Hossain M, Hamilton K, Palmer A, Blanco E. Predicting the focus of negation: model and error analysis. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 8389−8401

[10] Szarvas G, Vincze V, Farkas R, Csirik J. The BioScope corpus: annotation for negation, uncertainty and their scope in biomedical texts. In: Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing. 2008, 38−45

[11] Konstantinova N, de Sousa S C M, Cruz N P, Maña M J, Taboada M, Mitkov R. A review corpus annotated for negation, speculation and their scope. In: Proceedings of the 8th International Conference on Language Resources and Evaluation. 2012, 3190−3195

[12] Vincze V, Szarvas G, Farkas R, Móra G, Csirik J . The bioscope corpus: biomedical texts annotated for uncertainty, negation and their scopes. BMC Bioinformatics, 2008, 9( S11): S9

[13] Christen R, Shaitarova A, Stürmer M, Niklaus J. Resolving legalese: a multilingual exploration of negation scope resolution in legal documents. In: Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation. 2024, 13992−14004

[14] Qian Z, Li P, Zhu Q, Zhou G, Luo Z, Luo W. Speculation and negation scope detection via convolutional neural networks. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2016, 815−825

[15] Fei H, Ren Y, Ji D . Negation and speculation scope detection using recursive neural conditional random fields. Neurocomputing, 2020, 374: 22–29

[16] Wu Y, Sun A. Negation scope refinement via boundary shift loss. In: Proceedings of Findings of the Association for Computational Linguistics. 2023, 6090−6099

[17] Yoshida A, Kato Y, Matsubara S. Negation scope conversion: towards a unified negation-annotated dataset. In: Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation. 2024, 12093−12099

[18] Qian Z, Sun T, Zou T, Li P, Zhu Q, Zhou G. Speculation and negation scope resolution via machine reading comprehension formulation with data augmentation. In: Proceedings of the 28th International Conference on Database Systems for Advanced Applications. 2023, 487−496

[19] Qian Z, Zou T, Zhang Z, Li P, Zhu Q, Zhou G . Speculation and negation identification via unified Machine Reading Comprehension frameworks with lexical and syntactic data augmentation. Engineering Applications of Artificial Intelligence, 2024, 131: 107806

[20] Anand P, Martell C. Annotating the focus of negation in terms of questions under discussion. In: Proceedings of the Workshop on Extra-Propositional Aspects of Meaning in Computational Linguistics. 2012, 65−69

[21] Taulé M, Nofre M, González M, Martí M A . Focus of negation: its identification in Spanish. Natural Language Engineering, 2021, 27( 2): 131–152

[22] Matsuyoshi S, Otsuki R, Fukumoto F. Annotating the focus of negation in Japanese text. In: Proceedings of the 9th International Conference on Language Resources and Evaluation. 2014, 1743−1750

[23] Sheng J, Zou B, Shen L, Ye J, Hong Y. Research on Chinese negative focus identification: dataset and baseline. In: Proceedings of the 19th Chinese National Conference on Computational Linguistics. 2020, 550−560

[24] Zou B, Zhu Q, Zhou G. Negation and speculation identification in Chinese language. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015, 656−665

[25] Qian Z, Li P, Zhou G, Zhu Q. Speculation and negation scope detection via bidirectional LSTM neural networks. Journal of Software, 2018, 29(8): 2427−2447

Rights & permissions

Higher Education Press