Aspect-level sentiment analysis based on semantic heterogeneous graph convolutional network

Abstract

The deep learning methods based on syntactic dependency tree have achieved great success on Aspect-based Sentiment Analysis (ABSA). However, the accuracy of the dependency parser cannot be determined, which may keep aspect words away from its related opinion words in a dependency tree. Moreover, few models incorporate external affective knowledge for ABSA. Based on this, we propose a novel architecture to tackle the above two limitations, while fills up the gap in applying heterogeneous graphs convolution network to ABSA. Specially, we employ affective knowledge as an sentiment node to augment the representation of words. Then, linking sentiment node which have different attributes with word node through a specific edge to form a heterogeneous graph based on dependency tree. Finally, we design a multi-level semantic heterogeneous graph convolution network (Semantic-HGCN) to encode the heterogeneous graph for sentiment prediction. Extensive experiments are conducted on the datasets SemEval 2014 Task 4, SemEval 2015 task 12, SemEval 2016 task 5 and ACL 14 Twitter. The experimental results show that our method achieves the state-of-the-art performance.

Graphical abstract

Keywords

heterogeneous graph convolution network / multi-head attention network / aspect-based sentiment analysis / deep learning / affective knowledge

Yufei Zeng is a master student at School of Computer Science and Engineering, Guangxi Normal University, China. His research interests include sentiment analysis and information extraction

Zhixin Li is a professor at School of Computer Science and Engineering, Guangxi Normal University, China. In 2010, He obtained his PhD degree in computer software and theory from Institute of Computing Technology, Chinese Academy of Sciences, China. He obtained his BS degree and MS degree at the Huazhong University of Science and Technology, China in 1992 and 2004 respectively. His research interests include image understanding, machine learning and cross-media computing. He has won the best doctoral dissertation award of Chinese Association of Artificial Intelligence in 2011

Zhenbin Chen is a master student at School of Computer Science and Engineering, Guangxi Normal University, China. His research interests include relation extraction and few-shot learning

Huifang Ma received the BE degree from Northwest Normal University, China in 2003, and the MS degree from Beijing Normal University, China in 2006. She received the PhD degree from Institute of Computing Technology, Chinese Academy of Sciences, China in 2010. She is now a professor at College of Computer Science and Engineering, Northwest Normal University, China. Her research interests include data mining and machine learning

References

[1] Nguyen T H, Shirai K. PhraseRNN: phrase recursive neural network for aspect-based sentiment analysis. In: Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing. 2015, 2509−2514

[2] Tang D, Qin B, Liu T. Aspect level sentiment classification with deep memory network. In: Proceedings of 2016 Conference on Empirical Methods in Natural Language Processing. 2016, 214−224

[3] Wang Y, Huang M, Zhu X, Zhao L. Attention-based LSTM for aspect-level sentiment classification. In: Proceedings of 2016 Conference on Empirical Methods in Natural Language Processing. 2016, 606−615

[4] Tang D, Qin B, Feng X, Liu T. Effective LSTMs for target-dependent sentiment classification. In: Proceedings of the 26th International Conference on Computational Linguistics. 2016, 3298−3307

[5] Chen P, Sun Z, Bing L, Yang W. Recurrent attention network on memory for aspect sentiment analysis. In: Proceedings of 2017 Conference on Empirical Methods in Natural Language Processing. 2017, 452−461

[6] Zhang Y, Qi P, Manning C D. Graph convolution over pruned dependency trees improves relation extraction. In: Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. 2018, 2205−2215

[7] Zhang C, Li Q, Song D. Aspect-based sentiment classification with aspect-specific graph convolutional networks. In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 4568−4578

[8] Li Z, Sun Y, Zhu J, Tang S, Zhang C, Ma H. Improve relation extraction with dual attention-guided graph convolutional networks. Neural Computing and Applications, 2021, 33( 6): 1773–1784

[9] Chen S, Li Z, Huang F, Zhang C, Ma H. Improving object detection with relation mining network. In: Proceedings of 2020 IEEE International Conference on Data Mining. 2020, 52−61

[10] Zhang M, Qian T. Convolution over hierarchical syntactic and lexical graphs for aspect level sentiment analysis. In: Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing. 2020, 3540−3549

[11] Cambria E, Poria S, Hazarika D, Kwok K. SenticNet 5: discovering conceptual primitives for sentiment analysis by means of context embeddings. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence and 30th Innovative Applications of Artificial Intelligence Conference and 8th AAAI Symposium on Educational Advances in Artificial Intelligence. 2018, 219

[12] Ma D, Li S, Zhang X, Wang H. Interactive attention networks for aspect-level sentiment classification. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017, 4068−4074

[13] Fan F, Feng Y, Zhao D. Multi-grained attention network for aspect-level sentiment classification. In: Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. 2018, 3433−3442

[14] Xue W, Li T. Aspect based sentiment analysis with gated convolutional networks. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018, 2514−2523

[15] Tay Y, Tuan L A, Hui S C. Learning to attend via word-aspect associative fusion for aspect-based sentiment analysis. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence and 30th Innovative Applications of Artificial Intelligence Conference and 8th AAAI Symposium on Educational Advances in Artificial Intelligence. 2018, 731

[16] Yao L, Mao C, Luo Y. Graph convolutional networks for text classification. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence and 31st Innovative Applications of Artificial Intelligence Conference and 9th AAAI Symposium on Educational Advances in Artificial Intelligence. 2019, 905

[17] Zhang C, Li Q, Song D. Syntax-aware aspect-level sentiment classification with proximity-weighted convolution network. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2019, 1145−1148

[18] Hu M, Zhao S, Guo H, Cheng R, Su Z. Learning to detect opinion snippet for aspect-based sentiment analysis. In: Proceedings of the 23rd Conference on Computational Natural Language Learning. 2019, 970−979

[19] Xu L, Bing L, Lu W, Huang F. Aspect sentiment classification with aspect-specific opinion spans. In: Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing. 2020, 3561−3567

[20] Wang Y, Chen Q, Shen J, Hou B, Ahmed M, Li Z. Aspect-level sentiment analysis based on gradual machine learning. Knowledge-Based Systems, 2021, 212: 106509

[21] Zhang Z, Hang C W, Singh M P. Octa: omissions and conflicts in target-aspect sentiment analysis. In: Proceedings of the Findings of the Association for Computational Linguistics. 2020, 1651−1662

[22] Cai H, Zheng V W, Chang K C C. A comprehensive survey of graph embedding: problems, techniques, and applications. IEEE Transactions on Knowledge and Data Engineering, 2018, 30( 9): 1616–1637

[23] Sun K, Zhang R, Mensah S, Mao Y, Liu X. Aspect-level sentiment analysis via convolution over dependency tree. In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 5679−5688

[24] Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y. Graph attention networks. In: Proceedings of the ICLR 2018. 2018

[25] Huang B, Carley K. Syntax-aware aspect level sentiment classification with graph attention networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 5469−5477

[26] Wang K, Shen W, Yang Y, Quan X, Wang R. Relational graph attention network for aspect-based sentiment analysis. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 3229−3238

[27] Ratinov L, Roth D. Design challenges and misconceptions in named entity recognition. In: Proceedings of the 30th Conference on Computational Natural Language Learning. 2009, 147−155

[28] Rahman A, Ng V. Coreference resolution with world knowledge. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011, 814−824

[29] Nakashole N, Mitchell T M. A knowledge-intensive model for prepositional phrase attachment. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015, 365−375

[30] Xu Z, Liu B, Wang B, Sun C, Wang X. Incorporating loose-structured knowledge into LSTM with recall gate for conversation modeling. 2016, arXiv preprint arXiv: 1605.05110

[31] Zhang B, Xu X, Yang M, Chen X, Ye Y. Cross-domain sentiment classification by capsule network with semantic rules. IEEE Access, 2018, 6: 58284–58294

[32] Zhang J, Lertvittayakumjorn P, Guo Y. Integrating semantic knowledge to tackle zero-shot text classification. In: Proceedings of NAACL-HLT 2019. 2019, 1031−1040

[33] Hu Z, Ma X, Liu Z, Hovy E, Xing E P. Harnessing deep neural networks with logic rules. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 2410−2420

[34] Dragoni M, Petrucci G. A fuzzy-based strategy for multi-domain sentiment analysis. International Journal of Approximate Reasoning, 2018, 93: 59–73

[35] Zhang B, Li X, Xu X, Leung K C, Chen Z, Ye Y. Knowledge guided capsule attention network for aspect-based sentiment analysis. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2020, 28: 2538–2551

[36] Ma Y, Peng H, Cambria E. Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive LSTM. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence and 30th Innovative Applications of Artificial Intelligence Conference and 8th AAAI Symposium on Educational Advances in Artificial Intelligence. 2018, 721

[37] Cambria E, Poria S, Bajpai R, Schuller B. SenticNet 4: a semantic resource for sentiment analysis based on conceptual primitives. In: Proceedings of the 26th International Conference on Computational Linguistics. 2016, 2666−2677

[38] Zeng B, Yang H, Xu R, Zhou W, Han X. LCF: a local context focus mechanism for aspect-based sentiment classification. Applied Sciences, 2019, 9( 16): 3389

[39] Cambria E, Fu J, Bisio F, Poria S. AffectiveSpace 2: enabling affective intuition for concept-level sentiment analysis. In: Proceedings of the 29th AAAI Conference on Artificial Intelligence. 2015, 508−514

[40] Bingham E, Mannila H. Random projection in dimensionality reduction: applications to image and text data. In: Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2001, 245−250

[41] Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th International Conference on Learning Representations. 2017

[42] Bengio Y, Ducharme R, Vincent P, Janvin C. A neural probabilistic language model. The Journal of Machine Learning Research, 2003, 3: 1137–1155

[43] Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K. Adaptive recursive neural network for target-dependent twitter sentiment classification. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. 2014, 49−54

[44] Kirange D, Deshmukh R R, Kirange M. Aspect based sentiment analysis semeval-2014 task 4. Asian Journal of Computer Science and Information Technology, 2014, 4(8): 72−75

[45] Pontiki M, Galanis D, Papageorgiou H, Manandhar S, Androutsopoulos I. SemEval-2015 task 12: aspect based sentiment analysis. In: Proceedings of the 9th International Workshop on Semantic Evaluation. 2015, 486−495

[46] Pontiki M, Galanis D, Papageorgiou H, Androutsopoulos I, Manandhar S, Al-Smadi M, Al-Ayyoub M, Zhao Y, Qin B, De Clercq O, Hoste V, Apidianaki M, Tannier X, Loukachevitch N, Kotelnikov E, Bel N, Jiménez-Zafra S M, Eryiğit G. SemEval-2016 task 5: aspect based sentiment analysis. In: Proceedings of the 10th International Workshop on Semantic Evaluation. 2016, 19−30

[47] Dozat T, Manning C D. Deep biaffine attention for neural dependency parsing. In: Proceedings of the 5th International Conference on Learning Representations. 2017

[48] Pennington J, Socher R, Manning C. GloVe: global vectors for word representation. In: Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing. 2014, 1532−1543

[49] Kingma D P, Ba J. Adam: a method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations. 2015

[50] He R, Lee W S, Ng H T, Dahlmeier D. Effective attention modeling for aspect-level sentiment classification. In: Proceedings of the 27th International Conference on Computational Linguistics. 2018, 1121−1131

[51] Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation, 1997, 9( 8): 1735–1780

[52] Ali W, Yang Y, Qiu X, Ke Y, Wang Y. Aspect-level sentiment analysis based on bidirectional-GRU in SIoT. IEEE Access, 2021, 9: 69938–69950

[53] Yadav R K, Jiao L, Granmo O C, Goodwin M. Human-level interpretable learning for aspect-based sentiment analysis. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence. 2021, 14203−14212

[54] Li X, Bing L, Lam W, Shi B. Transformation networks for target-oriented sentiment classification. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018, 946−956

[55] Dai J, Yan H, Sun T, Liu P, Qiu X. Does syntax matter? A strong baseline for aspect-based sentiment analysis with RoBERTa. In: Proceedings of 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021, 1816−1829

[56] Chen D, Manning C D. A fast and accurate dependency parser using neural networks. In: Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing. 2014, 740−750

[57] Xu K, Li C, Tian Y, Sonobe T, Kawarabayashi K I, Jegelka S. Representation learning on graphs with jumping knowledge networks. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 5449−5458

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant Nos. 62276073, 61966004), Guangxi Natural Science Foundation (No. 2019GXNSFDA245018), Innovation Project of Guangxi Graduate Education (No. YCSW2022155), Guangxi “Bagui Scholar” Teams for Innovation and Research Project, and Guangxi Collaborative Innovation Center of Multi-source Information Integration and Intelligent Processing.

RIGHTS & PERMISSIONS

2023 Higher Education Press