Enhancing aspect sentiment quad prediction with syntactic information in generation model

Abstract

Utilizing pre-trained generation models for predicting sentiment elements has recently shown significant advancements in aspect sentiment quad prediction benchmarks. However, these models overlook the significance of syntactic information, which have proven to be effective in previous extraction-based approaches. Different from extraction-based models, efficiently encoding the syntactic structure in generation model is challenging because such models are pretrained on natural language, and modeling structured data may lead to catastrophic forgetting of distributional knowledge. In this study, we propose an innovative structure-aware framework that explicitly encodes the syntactic structure into the pre-trained generation model while preserving its original distributional knowledge.

Graphical abstract

Keywords

aspect-based sentiment analysis / aspect sentiment quad prediction / syntactic information

References

[1] Zhang W, Deng Y, Li X, Yuan Y, Bing L, Lam W. Aspect sentiment quad prediction as paraphrase generation. In: Proceedings of 2021 Conference on Empirical Methods in Natural Language Processing. 2021, 9209–9219

[2] Cai H, Xia R, Yu J. Aspect-category-opinion-sentiment quadruple extraction with implicit aspects and opinions. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 2021, 340–350

[3] Wan H, Yang Y, Du J, Liu Y, Qi K, Pan J Z. Target-aspect-sentiment joint detection for aspect-based sentiment analysis. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020, 9122–9129

[4] Hu M, Bai Y, Wu Y, Zhang Z, Zhang L, Gao H, Zhao S, Huang M. Uncertainty-aware unlikelihood learning improves generative aspect sentiment quad prediction. In: Proceedings of the Findings of the Association for Computational Linguistics. 2023, 13481–13494

[5] Tay Y, Tuan L A, Hui S C. Learning to attend via word-aspect associative fusion for aspect-based sentiment analysis. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018

[6] Sun K, Zhang R, Mensah S, Mao Y, Liu X. Aspect-level sentiment analysis via convolution over dependency tree. In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 5678–5687

[7] Huang B, Carley K. Syntax-aware aspect level sentiment classification with graph attention networks. In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 5468–5476

[8] Tang H, Ji D, Li C, Zhou Q. Dependency graph enhanced dual-transformer structure for aspect-based sentiment classification. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 6578–6588

[9] Li S, Wang Z, Xu Y, Zhou G. Structure-aware generation model for cross-domain aspect-based sentiment classification. In: Proceedings of 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation. 2024, 15373–15383

[10] Schlichtkrull M, Kipf T N, Bloem P, van den Berg R, Titov I, Welling M. Modeling relational data with graph convolutional networks. In: Proceedings of the 15th International Conference on the Semantic Web. 2018, 593–607

[11] Wang Q, Wen Z, Zhao Q, Yang M, Xu R. Progressive self-training with discriminator for aspect term extraction. In: Proceedings of 2021 Conference on Empirical Methods in Natural Language Processing. 2021, 257–268

[12] Bu J, Ren L, Zheng S, Yang Y, Wang J, Zhang F, Wu W. ASAP: a Chinese review dataset towards aspect category sentiment analysis and rating prediction. In: Proceedings of 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021, 2069–2079

[13] Hu M, Zhao S, Zhang L, Cai K, Su Z, Cheng R, Shen X. CAN: constrained attention networks for multi-aspect sentiment analysis. In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 4601–4610

[14] Fan Z, Wu Z, Dai X Y, Huang S, Chen J. Target-oriented opinion words extraction with target-fused neural sequence labeling. In: Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019, 2509–2518

[15] Tang D, Qin B, Feng X, Liu T. Effective LSTMs for target-dependent sentiment classification. In: Proceedings of the 26th International Conference on Computational Linguistics. 2016, 3298–3307

[16] Chen C, Teng Z, Wang Z, Zhang Y. Discrete opinion tree induction for aspect-based sentiment analysis. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 2022, 2051–2064

[17] Cao J, Liu R, Peng H, Jiang L, Bai X. Aspect is not you need: no-aspect differential sentiment framework for aspect-based sentiment analysis. In: Proceedings of 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022, 1599–1609

[18] Xu L, Li H, Lu W, Bing L. Position-aware tagging for aspect sentiment triplet extraction. In: Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing. 2020, 2339–2349

[19] Li J, Yu J, Xia R. Generative cross-domain data augmentation for aspect and opinion co-extraction. In: Proceedings of 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022, 4219–4229

[20] Zhao G, Luo Y, Chen Q, Qian X. Aspect-based sentiment analysis via multitask learning for online reviews. Knowledge-Based Systems, 2023, 264: 110326

[21] Zhang W, Li X, Deng Y, Bing L, Lam W. Towards generative aspect-based sentiment analysis. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 2021, 504–510

[22] Zhang W, Zhang X, Cui S, Huang K, Wang X, Liu T. Adaptive data augmentation for aspect sentiment quad prediction. In: Proceedings of 2024 IEEE International Conference on Acoustics, Speech and Signal Processing. 2024, 11176–11180

[23] Zhang H, Cheah Y N, He C, Yi F. An instruction tuning-based contrastive learning framework for aspect sentiment quad prediction with implicit aspects and opinions. In: Proceedings of the Findings of the Association for Computational Linguistics. 2024, 7698–7714

[24] Qiu G, Liu B, Bu J, Chen C. Opinion word expansion and target extraction through double propagation. Computational Linguistics, 2011, 37( 1): 9–27

[25] Wang W, Pan S J, Dahlmeier D, Xiao X. Coupled multi-layer attentions for co-extraction of aspect and opinion terms. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence. 2017, 3316–3322

[26] Peper J, Wang L. Generative aspect-based sentiment analysis with contrastive learning and expressive structure. In: Proceedings of the Findings of the Association for Computational Linguistics. 2022, 6089–6095

[27] Zhou J, Yang H, He Y, Mou H, Yang J. A unified one-step solution for aspect sentiment quad prediction. In: Proceedings of the Findings of the Association for Computational Linguistics. 2023, 12249–12265

[28] Mao Y, Shen Y, Yang J, Zhu X, Cai L. Seq2Path: generating sentiment tuples as paths of a tree. In: Proceedings of the Findings of the Association for Computational Linguistics. 2022, 2215–2225

[29] Yang Y, Zhang Y, Xu R. Enhancing generative aspect-based sentiment analysis with relation-level supervision and prompt. In: Proceedings of 2024 IEEE International Conference on Acoustics, Speech and Signal Processing. 2024, 10526–10530

[30] Lu H Y, Liu T C, Cong R, Yang J, Gan Q, Fang W, Wu X J. QAIE: LLM-based quantity augmentation and information enhancement for few-shot aspect-based sentiment analysis. Information Processing & Management, 2025, 62( 1): 103917

[31] Bao X, Jiang X, Wang Z, Zhang Y, Zhou G. Opinion tree parsing for aspect-based sentiment analysis. In: Proceedings of the Findings of the Association for Computational Linguistics. 2023, 7971–7984

[32] Bao X, Wang Z, Jiang X, Xiao R, Li S. Aspect-based sentiment analysis with opinion tree generation. In: Proceedings of the 31st International Joint Conference on Artificial Intelligence. 2022, 4044–4050

[33] Ma T, Wang Z, Zhou G. Transition-based opinion generation for aspect-based sentiment analysis. In: Proceedings of the Findings of the Association for Computational Linguistics. 2024, 3078–3087

[34] Liu K, Xu L, Liu Y, Zhao J. Opinion target extraction using partially-supervised word alignment model. In: Proceedings of the 23rd International Joint Conference on Artificial Intelligence. 2013, 2134–2140

[35] Lakkaraju H, Socher R, Manning C. Aspect specific sentiment analysis using hierarchical deep learning. In: Proceedings of the 28th Annual Conference on Neural Information Processing Systems. 2014, 1–9

[36] He R, Lee W S, Ng H T, Dahlmeier D. Effective attention modeling for aspect-level sentiment classification. In: Proceedings of the 27th International Conference on Computational Linguistics. 2018

[37] Chen D, Manning C. A fast and accurate dependency parser using neural networks. In: Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing. 2014, 740–750

[38] Dozat T, Manning C D. Deep biaffine attention for neural dependency parsing. In: Proceedings of the 5th International Conference on Learning Representations. 2017

[39] Zheng Y, Zhang R, Mensah S, Mao Y. Replicate, walk, and stop on syntax: an effective neural network model for aspect-level sentiment classification. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020, 9685–9692

[40] Wang K, Shen W, Yang Y, Quan X, Wang R. Relational graph attention network for aspect-based sentiment analysis. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 3229–3238

[41] Zhang C, Li Q, Song D. Aspect-based sentiment classification with aspect-specific graph convolutional networks. In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 4567–4577

[42] Liang B, Su H, Gui L, Cambria E, Xu R. Aspect-based sentiment analysis via affective knowledge enhanced graph convolutional networks. Knowledge-Based Systems, 2022, 235: 107643

[43] Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu P J. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 2020, 21( 1): 140

[44] Hu M, Wu Y, Gao H, Bai Y, Zhao S. Improving aspect sentiment quad prediction via template-order data augmentation. In: Proceedings of 2022 Conference on Empirical Methods in Natural Language Processing. 2022, 7889–7900

[45] Touvron H, Lavril T, Izacard G, Martinet X, Lachaux M A, Lacroix T, Rozière B, Goyal N, Hambro E, Azhar F, Rodriguez A, Joulin A, Grave E, Lample G. LLaMA: open and efficient foundation language models. 2023, arXiv preprint arXiv: 2302.13971

[46] Hu E J, Shen Y, Wallis P, Allen-Zhu Z, Li Y, Wang S, Wang L, Chen W. LoRA: low-rank adaptation of large language models. In: Proceedings of the 10th International Conference on Learning Representations. 2022

[47] Honnibal M, Johnson M. An improved non-monotonic transition system for dependency parsing. In: Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing. 2015, 1373–1378

RIGHTS & PERMISSIONS

Higher Education Press