Nested relation extraction with iterative neural network

Yixuan CAO, Dian CHEN, Zhengqi XU, Hongwei LI, Ping LUO

PDF(1014 KB)
PDF(1014 KB)
Front. Comput. Sci. ›› 2021, Vol. 15 ›› Issue (3) : 153323. DOI: 10.1007/s11704-020-9420-6
RESEARCH ARTICLE

Nested relation extraction with iterative neural network

Author information +
History +

Abstract

Most existing researches on relation extraction focus on binary flat relations like BornIn relation between a Person and a Location. But a large portion of objective facts described in natural language are complex, especially in professional documents in fields such as finance and biomedicine that require precise expressions. For example, “the GDP of the United States in 2018 grew 2.9% compared with 2017” describes a growth rate relation between two other relations about the economic index, which is beyond the expressive power of binary flat relations. Thus, we propose the nested relation extraction problem and formulate it as a directed acyclic graph (DAG) structure extraction problem. Then, we propose a solution using the Iterative Neural Network which extracts relations layer by layer. The proposed solution achieves 78.98 and 97.89 F1 scores on two nested relation extraction tasks, namely semantic cause-and-effect relation extraction and formula extraction. Furthermore, we observe that nested relations are usually expressed in long sentences where entities are mentioned repetitively, which makes the annotation difficult and errorprone. Hence, we extend our model to incorporate a mentioninsensitive mode that only requires annotations of relations on entity concepts (instead of exact mentions) while preserving most of its performance. Our mention-insensitive model performs better than the mention sensitive model when the random level in mention selection is higher than 0.3.

Keywords

nested relation extraction / mention insensitive relation / iterative neural network

Cite this article

Download citation ▾
Yixuan CAO, Dian CHEN, Zhengqi XU, Hongwei LI, Ping LUO. Nested relation extraction with iterative neural network. Front. Comput. Sci., 2021, 15(3): 153323 https://doi.org/10.1007/s11704-020-9420-6

References

[1]
Ernst P, Siu A, Weikum G. Highlife: higher-arity fact harvesting. In: Proceedings of the 2018 World Wide Web Conference. 2018, 1013–1022
CrossRef Google scholar
[2]
Hassan N, Arslan F, Li C, Tremayne M. Toward automated fact-checking: detecting check-worthy factual claims by claimbuster. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2017, 1803–1812
CrossRef Google scholar
[3]
Mintz M, Bills S, Snow R, Dan J. Distant supervision for relation extraction without labeled data. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. 2009, 1003–1011
CrossRef Google scholar
[4]
Aggarwal C C, Zhai C. Mining Text Data. Springer Science & Business Media, 2012
CrossRef Google scholar
[5]
Miwa M, Bansal M. End-to-end relation extraction using LSTMs on sequences and tree structures. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 1105–1116
CrossRef Google scholar
[6]
Xu Y, Mou L, Li G, Chen Y, Peng H, Jin Z. Classifying relations via long short term memory networks along shortest dependency paths. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015, 1785–1794
CrossRef Google scholar
[7]
Zhou P, Shi W, Tian J, Qi Z, Li B, Hao H, Xu B. Attention-based bidirectional long short-term memory networks for relation classification. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 207–212
CrossRef Google scholar
[8]
Zhang Y, Qi P, Manning C D. Graph convolution over pruned dependency trees improves relation extraction. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018, 2205–2215
CrossRef Google scholar
[9]
Katiyar A, Cardie C. Going out on a limb: joint extraction of entity mentions and relations without dependency trees. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017, 917–928
CrossRef Google scholar
[10]
Christopoulou F, Miwa M, Ananiadou S. A walk-based model on entity graphs for relation extraction. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018, 81–88
CrossRef Google scholar
[11]
Zeng W, Lin Y, Liu Z, Sun M. Incorporating relation paths in neural relation extraction. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017, 1768–1777
CrossRef Google scholar
[12]
Suchanek F M, Kasneci G, Weikum G. Yago: a large ontology from wikipedia and wordnet. Journal of Web Semantics, 2008, 6(3): 203–217
CrossRef Google scholar
[13]
Zhou D, Zhong D, He Y. Biomedical relation extraction: from binary to complex. Computational and Mathematical Methods in Medicine, 2014
CrossRef Google scholar
[14]
McDonald R, Pereira F, Kulick S, Winters S, Jin Y, White P. Simple algorithms for complex relation extraction with applications to biomedical IE. In: Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics. 2005, 491–498
CrossRef Google scholar
[15]
Li J, Sun Y, Johnson R J, Sciaky D, Wei C H, Leaman R, Davis A P, Mattingly C J, Wiegers T C, Lu Z. BioCreative V CDR task corpus: a resource for chemical disease relation extraction. Database: the Journal of Biological Databases & Curation, 2016, 2016: baw068
CrossRef Google scholar
[16]
Peng Y, Wei C H, Lu Z. Improving chemical disease relation extraction with rich features and weakly labeled data. Journal of Cheminformatics, 2016, 8(1): 53
CrossRef Google scholar
[17]
Verga P, Strubell E, McCallum A. Simultaneously self-attending to all mentions for full-abstract biological relation extraction. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018, 872–884
CrossRef Google scholar
[18]
Cui L, Wei F, Zhou M. Neural open information extraction. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018, 407–413
CrossRef Google scholar
[19]
Reshadat V, Hoorali M, Faili H. A hybrid method for open information extraction based on shallow and deep linguistic analysis. Interdisciplinary Information Sciences, 2016, 22(1): 87–100
CrossRef Google scholar
[20]
Reshadat V, Faili H. A new open information extraction system using sentence difficulty estimation. Computing and Informatics, 2019, 38(4): 986–1008
CrossRef Google scholar
[21]
Sun M, Li X, Wang X, Fan M, Feng Y, Li P. Logician: a unified end-to-end neural approach for open-domain information extraction. In: Proceedings of the 11th ACM International Conference on Web Search & Data Mining. 2018
CrossRef Google scholar
[22]
Chen Y, Xu L, Liu K, Zeng D, Zhao J. Event extraction via dynamic multi-pooling convolutional neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015, 167–176
CrossRef Google scholar
[23]
Blunsom P, Freitas d N, Grefenstette E, Hermann K M. A deep architecture for semantic parsing. In: Proceedings of the ACL 2014 Workshop on Semantic Parsing. 2014
[24]
Liang C, Berant J, Le Q, Forbus K D, Lao N. Neural symbolic machines: learning semantic parsers on freebase with weak supervision. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017, 23–33
CrossRef Google scholar
[25]
Wang Y, Berant J, Liang P. Building a semantic parser overnight. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015, 1332–1342
CrossRef Google scholar
[26]
Xiao C, Dymetman M, Gardent C. Sequence-based structured prediction for semantic parsing. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 1341–1350
CrossRef Google scholar
[27]
Berant J, Chou A, Frostig R, Liang P. Semantic parsing on freebase from question-answer pairs. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. 2013, 1533–1544
[28]
Hershcovich D, Abend O, Rappoport A. A transition-based directed acyclic graph parser for UCCA. In: Proceedings of the 55th AnnualMeeting of the Association for Computational Linguistics. 2017, 1127–1138
CrossRef Google scholar
[29]
Zhu X, Sobihani P, Guo H. Long short-term memory over recursive structures. In: Proceedings of the 32nd International Conference on Machine Learning. 2015, 1604–1612
[30]
Tai K S, Socher R, Manning C D. Improved semantic representations from tree-structured long short-term memory networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015, 1556–1566
CrossRef Google scholar
[31]
Agerri R, Rigau G. Robust multilingual named entity recognition with shallow semi-supervised features. Artificial Intelligence, 2016, 238: 63–82
CrossRef Google scholar
[32]
Aguilar J, Beller C, McNamee P, Van Durme B, Strassel S, Song Z, Ellis J. A comparison of the events and relations across ACE, ERE, TAC-KBP, and framenet annotation standards. In: Proceedings of the 2nd Workshop on EVENTS: Definition, Detection, Coreference, and Representation. 2014, 45–53
CrossRef Google scholar
[33]
Girju R, Nakov P, Nastase V, Szpakowicz S, Turney P, Yuret D. Semeval- 2007 task 04: classification of semantic relations between nominals. In: Proceedings of the 4th International Workshop on Semantic Evaluations. 2007, 13–18
CrossRef Google scholar
[34]
Hendrickx I, Kim S N, Kozareva Z, Nakov P, Ó Séaghdha D, Padó S, Pennacchiotti M, Romano L, Szpakowicz S. Semeval-2010 task 8: multiway classification of semantic relations between pairs of nominals. In: Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions. 2009, 94–99
CrossRef Google scholar
[35]
Wang Y, Liu X, Shi S. Deep neural solver for math word problems. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017, 845–854
CrossRef Google scholar
[36]
Wang L, Zhang D, Zhang J, Xu X, Gao L, Dai B T, Shen H T. Templatebased math word problem solvers with recursive neural networks. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 7144–7151
CrossRef Google scholar
[37]
Zeng D, Liu K, Chen Y, Zhao J. Distant supervision for relation extraction via piecewise convolutional neural networks. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015, 1753–1762
CrossRef Google scholar
[38]
Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation, 1997, 9(8): 1735–1780
CrossRef Google scholar
[39]
Luong M T, Pham H, Manning C D. Effective approaches to attentionbased neural machine translation. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015, 1412–1421
CrossRef Google scholar
[40]
Cao Y, Li H, Luo P, Yao J. Towards automatic numerical cross-checking: extracting formulas from text. In: Proceedings of the 2018 World Wide Web Conference. 2018, 1795–1804
CrossRef Google scholar
[41]
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser Ł, Polosukhin I. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6000–6010
[42]
Zeiler M D. Adadelta: an adaptive learning rate method. 2012, arXiv preprint arXiv: 1212.5071
[43]
Huang D, Yao J, Lin C, Zhou Q, Yin J. Using intermediate representations to solve math word problems. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018, 419–428
CrossRef Google scholar
[44]
Gu J, Lu Z, Li H, Li V O. Incorporating copying mechanism in sequenceto- sequence learning. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 1631–1640
CrossRef Google scholar
[45]
Klein G, Kim Y, Deng Y, Senellart J, Rush A. OpenNMT: open-source toolkit for neural machine translation. In: Proceedings of Annual Meeting of the Association for Computational Linguistics, System Demonstrations. 2017, 67–72
CrossRef Google scholar
[46]
Zheng S, Wang F, Bao H, Hao Y, Zhou P, Xu B. Joint extraction of entities and relations based on a novel tagging scheme. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017, 1227–1236
CrossRef Google scholar
[47]
Cao Y, Chen D, Li H, Luo P. Nested relation extraction with iterative neural network. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 2019, 1001–1010
CrossRef Google scholar

RIGHTS & PERMISSIONS

2020 Higher Education Press
AI Summary AI Mindmap
PDF(1014 KB)

Accesses

Citations

Detail

Sections
Recommended

/