A syntactic path-based hybrid neural network for negation scope detection

Abstract

The automatic detection of negation is a crucial task in a wide-range of natural language processing (NLP) applications, including medical data mining, relation extraction, question answering, and sentiment analysis. In this paper, we present a syntactic path-based hybrid neural network architecture, a novel approach to identify the scope of negation in a sentence. Our hybrid architecture has the particularity to capture salient information to determine whether a token is in the scope or not, without relying on any human intervention. This approach combines a bidirectional long shortterm memory (Bi-LSTM) network and a convolutional neural network (CNN). The CNN model captures relevant syntactic features between the token and the cue within the shortest syntactic path in both constituency and dependency parse trees. The Bi-LSTM learns the context representation along the sentence in both forward and backward directions. We evaluate our model on the Bioscope corpus, and get 90.82% F-score (78.31% PCS) on the abstract sub-corpus, outperforming features-dependent approaches.

Keywords

natural language processing / negation scope detection / convolutional neural network / recurrent neural network / syntactic path

References

[1] Morante R, Liekens A, Daelemans W. Learning the scope of negation in biomedical texts. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics. 2008, 715–724

[2] Chapman W, Bridewell W, Hanbury P, Cooper G F, Buchanan B G. A simple algorithm for identifying negated findings and diseases in discharge summaries. Journal of Biomedical Informatics, 2001, 34(5): 301–310

[3] Mutalik P G, Deshpande A, Nadkarni P M. Use of general-purpose negation detection to augment concept indexing of medical documents. Journal of the American Medical Informatics Association, 2001, 8(6): 598–609

[4] Huang Y, Lowe H J. A novel hybrid approach to automated negation detection in clinical radiology reports. Journal of the American Medical Informatics Association, 2007, 14(3): 304–311

[5] Vincze V, Szarvas G, Farkas R, Mra G, Csirik J. The BioScope corpus: biomedical texts annotated for uncertainty, negation and their scopes. BMC Bioinformatics, 2008, 9(11): S9

[6] Morante R, Daelemans W. A metalearning approach to processing the scope of negation. In: Proceedings of the 13th Conference on Computational Natural Language Learning, Association for Computational Linguistics. 2009, 21–29

[7] Zou B, Zhou G, Zhu Q. Tree kernel-based negation and speculation scope detection with structured syntactic parse features. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2013, 968–976

[8] Abu-Jbara A, Dragomir R. Umichigan: a conditional random field model for resolving the scope of negation. In: Proceedings of the 1st Joint Conference on Lexical and Computational Semantics–Volume 1: Proceedings of the Main Conference and the Shared Task, and Volume 2: Proceedings of the 6th International Workshop on Semantic Evaluation, Association for Computational Linguistics. 2012, 328–334

[9] Agarwal S, Yu H. Biomedical negation scope detection with conditional random fields. Journal of the American Medical Informatics Association, 2010, 17(6): 696–701

[10] Lazib L, Zhao Y, Qin B, Liu T. Negation scope detection with conditional random field model. High Technology Letters, 2017, 23(2): 191–197

[11] Cho K, Van Merrinboer B, Bahdanau D, Bengio Y. On the properties of neural machine translation: encoder-decoder approaches. In: Proceedings of the 8th Workshop on Syntax, Semantics and Sturucture in Statistical Translation (SSST-8). 2014

[12] Zeng D, Liu K, Chen Y, Zhao J. Distant supervision for relation extraction via piecewise convolutional neural networks. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2015, 1753–1762

[13] Tang D, Qin B, Liu T. Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2015, 1426–1432

[14] Lazib L, Zhao Y, Qin B, Liu T. Negation scope detection with recurrent neural networks models in review texts. In: Proceedings of International Conference of Young Computer Scientists, Engineers and Educators. 2016, 494–508

[15] Lazib L, Zhao Y, Qin B, Liu T. Negation scope detection with recurrent neural networks models in review texts. International Journal of High Performance Computing and Networking, 2019, 13(2): 211–221

[16] Qian Z, Li P, Zhu Q, Zhou G, Luo Z, Luo W. Speculation and negation scope detection via convolutional neural networks. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2016, 815–825

[17] Fancellu F, Lopez A, Webber B L. Neural networks for negation scope detection. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 495–504

[18] Schuster M, Paliwal K K. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 1997, 45(11): 2673–2681

[19] Mikolov T, Karafit M, Burget L, Černocký J, Khudanpur S. Recurrent neural network based language model. In: Proceedings of 11th Annual Conference of the International Speech Communication Association. 2010, 1045–1048

[20] LeCun Y, Bengio Y. Convolutional networks for images, speech, and time series. The Handbook of Brain Theory and Neural Networks, 1995, 3361(10): 1995

[21] Liu Y,Wei F, Li S, Ji H, Zhou M, Wang H. A dependency-based neural network for relation classification. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015, 285–290

[22] Cai R, Zhang X, Wang H. Bidirectional recurrent convolutional neural network for relation classification. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 756–765

[23] Xu Y, Mou L, Li G, Chen Y, Peng H, Jin Z. Classifying relations via long short-term memory networks along shortest dependency paths. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2015, 1785–1794

[24] Øvrelid L, Velldal E, Oepen S. Syntactic scope resolution in uncertainty analysis. In: Proceedings of the 23rd International Conference on Computational Linguistics. 2010, 1379–1387

[25] Lafferty J, McCallum A, Pereira F C. Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of Probabilistic Models for Segmenting and Labeling Sequence Data. 2001, 282–289

[26] White J P. UWashington: negation resolution using machine learning methods. In: Proceedings of the 1st Joint Conference on Lexical and Computational Semantics-Volume 1: Proceedings of the Main Conference and the Shared Task, and Volume 2: Proceedings of the 6th International Workshop on Semantic Evaluation. Association for Computational Linguistics. 2012, 335–339

[27] Huang Z, Xu W, Yu K. Bidirectional LSTM-CRF models for sequence tagging. 2015, arXiv preprint arXiv:1508.01991

[28] Wang P, Qian Y, Soong F K, He L, Zhao H. A unified tagging solution: bidirectional LSTM recurrent neural network with word embedding. 2015, arXiv preprint arXiv: 1511.00215

[29] Taboada M, Anthony C, Voll K. Methods for creating semantic orientation dictionaries. In: Proceedings of the 5th Conference on Language Resources and Evaluation. 2006, 427–432

[30] Zeng D, Liu K, Lai S, Zhou G, Zhao J. Relation classification via convolutional deep neural network. In: Proceedings of the 25th International Conference on Computational Linguistics: Technical Papers. 2014, 2335–2344

[31] Zhang B, Su J, Xiong D, Lu Y, Duan H, Yao J. Shallow convolutional neural network for implicit discourse relation recognition. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2015, 2230–2235

[32] Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation, 1997, 9(8): 1735–1780

[33] Graves A, Schmidhuber J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks, 2005, 18(5): 602–610

[34] Sundermeyer M, Schlter R, Ney H. LSTM neural networks for language modeling. In: Proceedings of the 13th Annual Conference of the International Speech Communication Association. 2013, 194–197

[35] Kadari R, Zhang Y, Zhang W, Liu T. CCG supertagging with bidirectional long short-term memory networks. Natural Language Engineering, 2018, 24(1): 77–90

[36] Kadari R, Zhang Y, Zhang W, Liu T. CCG supertagging via Bidirectional LSTM-CRF neural architecture. Neurocomputing, 2018, 283: 31–37

[37] Graves A, Jaitly N. Towards end-to-end speech recognition with recurrent neural networks. In: Proceedings of the 31st International Conference on Machine Learning. 2014, 1764–1772

[38] Sak H, Senior A W, Beaufays F. Long short-term memory recurrent neural network architectures for large scale acoustic modeling. In: Proceedings of the 15th Annual Conference of the International Speech Communication Association. 2014, 338–342

[39] Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa P. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 2011, 12(Aug): 2493–2537

[40] Collier N, Park H S, Ogata N, Tateishi Y, Nobata C, Ohta T, Sekimizu T, Imai H, Ibushi K, Tsujii J I. The GENIA project: corpus-based knowledge acquisition and information extraction from genome research papers. In: Proceedings of the 9th Conference on European Chapter of the Association for Computational Linguistics. 1999, 271–272

[41] Chollet F. Keras on GitHub, 2015

[42] Klein D, Manning C D. Accurate unlexicalized parsing. In: Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics. 2003

RIGHTS & PERMISSIONS

Higher Education Press and Springer-Verlag GmbH Germany, part of Springer Nature