Sememe knowledge computation: a review of recent advances in application and expansion of sememe knowledge bases

Abstract

A sememe is defined as the minimum semantic unit of languages in linguistics. Sememe knowledge bases are built by manually annotating sememes for words and phrases. HowNet is the most well-known sememe knowledge base. It has been extensively utilized in many natural language processing tasks in the era of statistical natural language processing and proven to be effective and helpful to understanding and using languages. In the era of deep learning, although data are thought to be of vital importance, there are some studies working on incorporating sememe knowledge bases like HowNet into neural network models to enhance system performance. Some successful attempts have been made in the tasks including word representation learning, language modeling, semantic composition, etc. In addition, considering the high cost of manual annotation and update for sememe knowledge bases, some work has tried to use machine learning methods to automatically predict sememes for words and phrases to expand sememe knowledge bases. Besides, some studies try to extend HowNet to other languages by automatically predicting sememes for words and phrases in a new language. In this paper, we summarize recent studies on application and expansion of sememe knowledge bases and point out some future directions of research on sememes.

Keywords

natural language process / semantics / knowledge base / sememe / HowNet

References

[1] Bloomfield L. A set of postulates for the science of language. Language, 1926, 2(3): 153–164

[2] Wierzbicka A. Semantics: Primes and Universals. Oxford: Oxford University Press, 1996

[3] Dong Z, Dong Q. HowNet and the Computation of Meaning. Singapore: World Scientific Publishing, 2006

[4] Gan K W, Wong P W. Annotating information structures in Chinese texts using HowNet. In: Proceedings of the 2nd Chinese Language Processing Workshop. 2000, 85–92

[5] Liu Q, Li S. Word similarity computing based on HowNet. International Journal of Computational Linguistics & Chinese Language Processing, 2002, 7(2): 59–76

[6] Zhang Y, Gong L, Wang Y. Chinese word sense disambiguation using HowNet. In: Proceedings of International Conference on Natural Computation. 2005, 925–932

[7] Duan X, Zhao J, Xu B. Word sense disambiguation through sememe labeling. In: Proceedings of the 20th International Joint Conference on Artificial Intelligence. 2007, 1594–1599

[8] Zhu Y, Min J, Zhou Y, Huang X, Wu L. Semantic orientation computing based on HowNet. Journal of Chinese Information Processing, 2006, 20(1): 14–20

[9] Dang L, Zhang L. Method of discriminant for Chinese sentence sentiment orientation based on HowNet. Application Research of Computers, 2010, 4: 43

[10] Fu X, Liu G, Guo Y, Wang Z. Multi-aspect sentiment analysis for Chinese online social reviews based on topic modeling and HowNet lexicon. Knowledge-Based Systems, 2013, 37: 186–195

[11] Sun J, Cai D, Lv D, Dong Y. HowNet based Chinese question automatic classification. Journal of Chinese Information Processing, 2007, 21(1): 90–95

[12] Moro A, Raganato A, Navigli R. Entity linking meets word sense disambiguation: a unified approach. Transactions of the Association for Computational Linguistics, 2014, 2: 231–244

[13] Faruqui M, Dodge J, Jauhar S K, Dyer C, Hovy E, Smith N A. Retrofitting word vectors to semantic lexicons. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2015, 1606–1615

[14] Chen Q, Zhu X, Ling Z H, Inkpen D, Wei S. Neural natural language inference models enhanced with external knowledge. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018, 2406–2417

[15] Sun M, Chen X. Embedding for words and word senses based on human annotated knowledge base: use HowNet as a case study. Journal of Chinese Information Processing, 2016, 30(6): 1–5

[16] Niu Y, Xie R, Liu Z, Sun M. Improved word representation learning with sememes. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017, 2049–2058

[17] Gu Y, Yan J, Zhu H, Liu Z, Xie R, Sun M, Lin F, Lin L. Language modeling with sparse product of sememe experts. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018, 4642–4651

[18] Zeng X, Yang C, Tu C, Liu Z, Sun M. Chinese LIWC lexicon expansion via hierarchical classification of word embeddings with sememe attention. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018, 5650–5657

[19] Qi F, Huang J, Yang C, Liu Z, Chen X, Liu Q, Sun M. Modeling semantic compositionality with sememe knowledge. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019, 5706–5715

[20] Qin Y, Qi F, Ouyang S, Liu Z, Yang C, Wang Y, Liu Q, Sun M. Enhancing recurrent neural networks with sememes. 2019, arXiv preprint arXiv:1910.08910

[21] Luo L, Ao X, Song Y, Li J, Yang X, He Q, Yu D. Unsupervised neural aspect extraction with sememes. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019, 5123–5129

[22] Zhang L, Qi F, Liu Z, Wang Y, Liu Q, Sun M. Multi-channel reverse dictionary model. In: Proceedings of the AAAI Conference on Artificial Intelligence. 2020, 312–319

[23] Zang Y, Qi F, Yang C, Liu Z, Zhang M, Liu Q, Sun M. Word-level textual adversarial attacking as combinatorial optimization. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 6066–6080

[24] Xie R, Yuan X, Liu Z, Sun M. Lexical sememe prediction via word embeddings and matrix factorization. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017, 4200–4206

[25] Sarwar B, Karypis G, Konstan J, Riedl J. Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th International Conference on World Wide Web. 2001, 285–295

[26] Koren Y, Bell R, Volinsky C. Matrix factorization techniques for recommender systems. Computer, 2009, 42(8): 30–37

[27] Jin H, Zhu H, Liu Z, Xie R, Sun M, Lin F, Lin L. Incorporating Chinese characters of words for lexical sememe prediction. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018, 2439–2449

[28] Du J, Qi F, Sun M, Liu Z. Lexical sememe prediction by dictionary definitions and local semantic correspondence. Journal of Chinese Information Processing, 2020, 34(5): 1–9

[29] Qi F, Lin Y, Sun M, Zhu H, Xie R, Liu Z. Cross-lingual lexical sememe prediction. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018, 358–368

[30] Qi F, Chang L, Sun M, Sicong O, Liu Z. Towards building a multilingual sememe knowledge base: predicting sememes for BabelNet synsets. In: Proceedings of the AAAI Conference on Artificial Intelligence. 2020, 8624–8631

[31] Miller G A. WordNet: a lexical database for English. Communications of the ACM, 1995, 38(11): 39–41

[32] Speer R, Chin J, Havasi C. Conceptnet 5.5: an open multilingual graph of general knowledge. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence. 2017, 4444–4451

[33] Mikolov T, Chen K, Corrado G, Dean J. Efficient estimation of word representations in vector space. In: Proceedings of 2013 International Conference on Learning Representations Workshop. 2013

[34] Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation, 1997, 9(8): 1735–1780

[35] Hinton G. Products of experts. In: Proceedings of the 9th International Conference on Artificial Neural Networks. 1999, 1–6

[36] Pelletier F J. The principle of semantic compositionality. Topoi, 1994, 13(1): 11–24

[37] Pelletier F J. Semantic compositionality. In: Oxford Research Encyclopedia of Linguistics. Oxford University Press, 2016

[38] Mitchell J, Lapata M. Language models based on semantic composition. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing. 2009, 430–439

[39] Socher R, Bauer J, Manning C D, Ng A Y. Parsing with compositional vector grammars. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. 2013, 455–465

[40] Maas A L, Daly R E, Pham P T, Huang D, Ng A Y, Potts C. Learning word vectors for sentiment analysis. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011, 142–150

[41] Socher R, Perelygin A, Wu J Y, Chuang J, Manning C D, Ng A Y, Potts C. Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. 2013, 1631–1642

[42] Mitchell J, Lapata M. Vector-based models of semantic composition. In: Proceedings of ACL-08: HLT. 2008, 236–244

[43] Navigli R, Ponzetto S P. BabelNet: the automatic construction, evaluation and application of a wide-coverage multilingual semantic network. Artificial Intelligence, 2012, 193: 217–250

[44] Chen X, Xu L, Liu Z, Sun M, Luan H. Joint learning of character and word embeddings. In: Proceedings of the 24th International Joint Conference on Artificial Intelligence. 2015, 1236–1242

[45] Camacho-Collados J, Pilehvar M T, Navigli R. Nasari: integrating explicit knowledge and corpus statistics for a multilingual representation of concepts and entities. Artificial Intelligence, 2016, 240: 36–64

[46] Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O. Translating embeddings for modeling multi-relational data. In: Proceedings of the 26th Conference on Neural Information Processing Systems. 2013, 2787–2795

[47] Qi F, Yang C, Liu Z, Dong Q, Sun M, Dong Z. OpenHowNet: an open sememe-based lexical knowledge base. 2019, arXiv preprint arXiv:1901.09957

RIGHTS & PERMISSIONS

2021 Higher Education Press