Multichannel 2D-CNN Attention-Based BiLSTM Method for Low-Resource Ewe Sentiment Analysis

Abstract

The unavailability of an annotated dataset for a low-resource Ewe language makes it difficult to develop an automated system to appropriately evaluate public opinion on events, news, policies, and regulations. In this study, we collected and preprocessed a low-resourced document-level Ewe sentiment dataset based on social media comments. We used three features learned by word embeddings (Global vectors, word-to-vector, and FastText) rather than hand-crafted features. We further proposed a novel method termed MC2D-CNN+BiLSTM-Attn to detect the exact sentiment feature from the Ewe dataset. Extensive experiments indicate that the proposed method efficiently classifies various sentiments and is superior to benchmark deep learning methods. Results show that in detecting the precise sentiments from raw Ewe textual context, the BiLSTM incorporating Glove outperforms Word2Vec and FastText embedding with an accuracy of 0.727. Furthermore, Attn+BiLSTM and multichannel convolutional neural network methods incorporating the Word2Vec embedding layer perform better than Glove and FastText embedding with an accuracy of 0.848 and 0.896. In contrast, our proposed method with the same Word2Vec embedding recorded 0.949.

Keywords

document level-sentiment analysis / EweSentiment / Multichannel CNN / natural language processing

Funding Support

This work was partially supported by the National Key Research and Development Program of China under Grant 2018AAA0100202 and the National Science Foundation of China under Grant 61976043.

Conflicts of Interest

The authors declare that they have no conflicts of interest to this work.

Data Availability Statement

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

Author Contribution Statement

Victor Kwaku Agbesi: Conceptualization, Methodology, Software, Formal analysis, Writing - original draft. Wenyu Chen: Supervision, Project administration. Chiagoziem C. Ukwuoma: Formal analysis, Writing - review & editing, Visualization, Project administration. Noble A. Kuadey: Investigation, Writing - review & editing, Visualization. Collinson Colin M. Agbesi: Investigation, Writing - review & editing. Chukwuebuka J. Ejiyi: Validation, Data curation. Emmanuel S. A. Gyarteng: Validation, Data curation. Gladys W. Muoka: Validation, Visualization. Anthony M. Kuadey: Software, Data curation.

References

[1] Agbesi V. K., Chen W., Gizaw S. M., Ukwuoma C. C., Ameneshewa A. S., & Ejiyi C. J. (2023). Attention based BiGRU-2DCNN with hunger game search technique for low-resource document-level sentiment classification. In Proceedings of the 2023 5th International Conference on Image Processing and Machine Vision,48-54. https://doi.org/10.1145/3582177.3582186

[2] Shirsat A. R., González A. F., & May J. J. (2022). Proposing a model of social media user interaction with fake news. Journal of Information, Communication and Ethics in Society, 20(1), 134-149. https://doi.org/10.1108/JICES-10-2020-0104

[3] Nagamanjula R., & Pethalakshmi A. (2020). A novel framework based on bi-objective optimization and LAN2FIS for Twitter sentiment analysis. Social Network Analysis and Mining, 10(1), 34. https://doi.org/10.1007/s13278-020-00648-5

[4] Naz S., Sharan A., & Malik N. (2018). Sentiment classification on Twitter data using support vector machine. In 2018 IEEE/WIC/ACM International Conference on Web Intelligence, 676-679. https://doi.org/10.1109/WI.2018.00-13

[5] Bibi R., Qamar U., Ansar M., & Shaheen A. (2019). Sentiment analysis for Urdu news tweets using decision tree. In 2019 IEEE 17th International Conference on Software Engineering Research, Management and Application, 66-70. https://doi.org/10.1109/SERA.2019.8886788

[6] Jaha A., Satya Deepthi N., Suryakanth G., & Surya Kala Eswari G. (2020). Text sentiment analysis using Naïve Baye’s classifier. International Journal of Computer Trends and Technology, 64(4), 261-265.

[7] Zhang Z., Zou Y., & Gan C. (2018). Textual sentiment analysis via three different attention convolutional neural networks and cross-modality consistent regression. Neurocomputing, 275, 1407-1415. https://doi.org/10.1016/j.neucom.2017.09.080

[8] Gan C., Wang L., & Zhang Z. (2020). Multi-entity sentiment analysis using self-attention based hierarchical dilated convolutional neural network. Future Generation Computer Systems, 112, 116-125. https://doi.org/10.1016/j.future.2020.05.022

[9] Abdi A., Shamsuddin S. M., Hasan S., & Piran J. (2019). Deep learning-based sentiment classification of evaluative text based on multi-feature fusion. Information Processing & Management, 56(4), 1245-1259. https://doi.org/10.1016/j.ipm.2019.02.018

[10] Zhang Z., Wang L., Zou Y., & Gan C. (2018). The optimally designed dynamic memory networks for targeted sentiment classification. Neurocomputing, 309,36-45. https://doi.org/10.1016/j.neucom.2018.04.068

[11] Basiri M. E., Nemati S., Abdar M., Cambria E., & Acharya U. R. (2021). ABCDM: An attention-based bidirectional CNN-RNN deep model for sentiment analysis. Future Generation Computer Systems, 115, 279-294. https://doi.org/10.1016/j.future.2020.08.005

[12] Vilares D., Peng H., Satapathy R., & Cambria E. (2018). BabelSenticNet: A commonsense reasoning framework for multilingual sentiment analysis. In 2018 IEEE Symposium Series on Computational Intelligence, 1292-1298. https://doi.org/10.1109/SSCI.2018.8628718

[13] Mahmood Z., Safder I., Nawab R. M. A., Bukhari F., Nawaz R., Alfakeeh A. S., :::, & Hassan S. U. (2020). Deep sentiments in Roman Urdu text using recurrent convolutional neural network model. Information Processing & Management, 57(4), 102233. https://doi.org/10.1016/j.ipm.2020.102233

[14] Sharf Z., & Rahman S. U. (2018). Performing natural language processing on Roman Urdu datasets. International Journal of Computer Science and Network Security, 18(1), 141-148.

[15] Mehmood K., Essam D., Shafi K., & Malik M. K. (2020). An unsupervised lexical normalization for Roman Hindi and Urdu sentiment analysis. Information Processing & Management, 57(6), 102368. https://doi.org/10.1016/j.ipm.2020.102368

[16] Sarkar K. (2019). Sentiment polarity detection in Bengali tweets using deep convolutional neural networks. Journal of Intelligent Systems, 28(3), 377-386. https://doi.org/10.1515/jisys-2017-0418

[17] Emon E. A., Rahman S., Banarjee J., Das A. K., & Mittra T.(2019). A deep learning approach to detect abusive Bengali text. In 2019 7th International Conference on Smart Computing and Communications, 1-5. https://doi.org/10.1109/ICSCC.2019.8843606

[18] Irtiza Trinto N., & Eunus Ali M. (2018). Detecting multilabel sentiment and emotions from Bangla Youtube comments. In 2018 International Conference on Bangla Speech and Language Processing, 1-6. https://doi.org/10.1109/ICBSLP.2018.8554875

[19] Mahmud M. S., Islam M. T., Bonny A. J., Shorna R. K., Omi J. H., & Rahman M. S. (2022). Deep learning based sentiment analysis from Bangla text using glove word embedding along with convolutional neural network. In 2022 13th International Conference on Computing Communication and Networking Technologies, 1-6. https://doi.org/10.1109/ICCCNT54827.2022.9984392

[20] Tasnim F., Habiba S. U., Nafisa N., & Ahmed A. (2022). Depressive Bangla text detection from social media post using different data mining techniques. In Computational Intelligence in Machine Learning: Select Proceedings of ICCIML 2021, 237-247. https://doi.org/10.1007/978-981-16-8484-5_21

[21] Sarkar K. (2020). Sentiment analysis of Bengali tweets using deep learning. In Computational Intelligence in Data Science: Third IFIP TC 12 International Conference, 578, 71-84. https://doi.org/10.1007/978-3-030-63467-4_6

[22] Azmin S., & Dhar K. (2019). Emotion detection from Bangla text corpus using Naïve Bayes classifier. In 2019 4th International Conference on Electrical Information and Communication Technology, 1-5. https://doi.org/10.1109/EICT48899.2019.9068797

[23] Sharmin S., & Chakma D. (2021). Attention-based convolutional neural network for Bangla sentiment analysis. AI & Society, 36(1), 381-396. https://doi.org/10.1007/s00146-020-01011-0

[24] Rahman M., Haque S., & Saurav Z. R. (2020). Identifying and categorizing opinions expressed in Bangla sentences using deep learning technique. International Journal of Computer Applications, 176(17), 13-17. https://doi.org/10.5120/ijca2020920119

[25] Islam K. I., Islam M. S., & Amin M. R. (2020). Sentiment analysis in Bengali via transfer learning using multi-lingual BERT. In 23rd International Conference on Computer and Information Technology,1-5. https://doi.org/10.1109/ICCIT51783.2020.9392653

[26] Smetanin S., & Komarov M. (2021). Deep transfer learning baselines for sentiment analysis in Russian. Information Processing & Management, 58(3), 102484. https://doi.org/10.1016/j.ipm.2020.102484

[27] Agbesi V. K., Chen W., Kuadey N. A., & Maale G. T.(2022). Multi-topic categorization in a low-resource ewe language: A modern transformer approach. In 2022 7th International Conference on Computer and Communication Systems, 42-45. https://doi.org/10.1109/ICCCS55155.2022.9846372

[28] Fouad M. M., Mahany A., Aljohani N., Abbasi R. A., & Hassan S. U. (2020). ArWordVec: Efficient word embedding models for Arabic tweets. Soft Computing, 24(11), 8061-8068. https://doi.org/10.1007/s00500-019-04153-6

[29] Rizkallah S., Atiya A. F., Shaheen S., & Mahgoub, H. E. D. (2023). ArSphere: Arabic word vectors embedded in a polar sphere. International Journal of Speech Technology, 26(1), 95-111. https://doi.org/10.1007/s10772-022-09966-9

[30] Alzanin S. M., Azmi A. M., & Aboalsamh H. A. (2022). Short text classification for Arabic social media tweets. Journal of King Saud University - Computer and Information Sciences, 34(9), 6595-6604. https://doi.org/10.1016/j.jksuci.2022.03.020

[31] Mittal N., & Agarwal B. (2013). Sentiment analysis of Hindi review based on negation and discourse relation. In International Joint Conference on Natural Language Processing,45-50.

[32] Al-Amin M., Islam M. S., & Das Uzzal S. (2017). Sentiment analysis of Bengali comments with Word2Vec and sentiment information of words. In 2017 International Conference on Electrical, Computer and Communication Engineering, 186-190. https://doi.org/10.1109/ECACE.2017.7912903

[33] Agbesi V. K., Chen W., Ameneshewa A. S., Odame E., Dumor K., & Browne J. A. (2023). Efficient adaptive convolutional model based on label embedding for text classification using low resource languages. In Proceedings of the 2023 7th International Conference on Intelligent Systems, Meta-heuristics & Swarm Intelligence,144-151. https://doi.org/10.1145/3596947.3596962

[34] Nandwani P., & Verma R. (2021). A review on sentiment analysis and emotion detection from text. Social Network Analysis and Mining, 11(1), 81. https://doi.org/10.1007/s13278-021-00776-6

[35] Hameed Z., & Garcia-Zapirain B. (2020). Sentiment classific-ation using a single-layered BiLSTM model. IEEE Access, 8, 73992-74001. https://doi.org/10.1109/ACCESS.2020.2988550

[36] Samiul Salehin S. M., Miah R., & Saiful Islam M. (2020). A comparative sentiment analysis on Bengali Facebook posts. In Proceedings of the International Conference on Computing Advancements, 21. https://doi.org/10.1145/3377049.3377078

[37] Pennington J., Socher R., & Manning C. D. (2014). GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, 1532-1543. https://doi.org/10.3115/v1/d14-1162

[38] Bojanowski P., Grave E., Joulin A., & Mikolov T. (2017). Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5, 135-146. https://doi.org/10.1162/tacl_a_00051

[39] Mikolov T., Chen K., Corrado G., & Dean J. (2013). Efficient estimation of word representations in vector space. arXiv Preprint: 1301.3781.

[40] Anki P., & Bustamam A. (2021). Measuring the accuracy of LSTM and BiLSTM models in the application of artificial intelligence by applying chatbot programme. Indonesian Journal of Electrical Engineering and Computer Science, 23(1), 197-205. https://doi.org/10.11591/ijeecs.v23.i1.pp197-205

[41] Verma S. (2022). Sentiment analysis of public services for smart society: Literature review and future research directions. Government Information Quarterly, 39(3), 101708. https://doi.org/10.1016/j.giq.2022.101708

[42] Li C., Wang B., Pavlu V., & Aslam J. A. (2016). An empirical study of skip-gram features and regularization for learning on sentiment analysis. In Advances in Information Retrieval: 38th European Conference on IR Research, 72-87. https://doi.org/10.1007/978-3-319-30671-1_6