An ensemble learning framework for text summarization based on an improved multilayer extreme learning machine autoencoder

Abstract

The massive growth of electronic data has created a demand for efficient tools to manage information and support fast decision-making. Automatic text summarization (ATS) addresses this by condensing large texts into concise, relevant summaries rapidly. ATS methods are categorized as extractive, abstractive, or hybrid. Extractive techniques select key sentences from input documents, whereas abstractive techniques generate new sentences to capture meaning. Hybrid methods combine both strategies. However, despite numerous suggested techniques, machine-generated summaries often fail to match the accuracy and coherence of human-written summaries. This study reviewed existing ATS techniques and highlighted their limitations, particularly high computational costs and low training efficiency. To address these problems, this study proposed an improved multilayer extreme learning machine autoencoder (MLELM-AE) and an ensemble learning framework that integrates four machine learning models: Sentence-bidirectional encoder representations from transformers, autoencoder, variational autoencoder, and the improved MLELM-AE. The proposed framework generates summaries through cosine similarity evaluation, followed by voting-based fusion, re-ranking, and optimal sentence selection. Experimental results showed that the proposed improved MLELM-AE model achieved strong performance, with an execution time of 50,015 ms and a recall-oriented understudy for gisting evaluation 1 score of 0.865145. These findings demonstrate that the proposed ensemble framework delivers more accurate and efficient summaries, offering a promising advancement in ATS.

Keywords

Automatic Text Summarization / Bidirectional Encoder Representations from Transformers / Deep Neural Networks / Multilayer Extreme Learning Machine Autoencoder / Word Embedding / Word2vec

References

[1] Adelia, R., Suyanto, S., & Wisesty, U.N. (2019). Indonesian abstractive text summarization using bidirectional gated recurrent unit. Procedia Computer Science, 157, 581-588. https://doi.org/10.1016/j.procs.2019.09.017

[2] Akter, S., Asa, A.S., Uddin, M.P., Hossain, M.D., Roy, S.K., & Afjal, M.I. (2017). An Extractive Text Summarization Technique for Bengali Document(s) using K-Means Clustering Algorithm. In: Proceedings of the 2017 IEEE International Conference on Imaging, Vision and Pattern Recognition (icIVPR). IEEE, p1-6. https://doi.org/10.1109/ICIVPR.2017.7890883

[3] Alami, N., Meknassi, M., & En-nahnahi, N. (2019). Enhancing unsupervised neural networks based text summarization with word embedding and ensemble learning. Expert Systems with Applications, 123, 195-211. https://doi.org/10.1016/j.eswa.2019.01.037

[4] Alotaibi, A., & Nadeem, F. (2025). An unsupervised integrated framework for Arabic aspect-based sentiment analysis and abstractive text summarization of traffic services using transformer models. Smart Cities, 8(2), 62. https://doi.org/10.3390/smartcities8020062

[5] Alzuhair, A., & Al-Dhelaan, M. (2019). An approach for combining multiple weighting schemes and ranking methods in graph-based multi-document summarization. IEEE Access, 7, 120375-120386. https://doi.org/10.1109/access.2019.2936832

[6] Dave, H., & Jaswal, S. (2015). Multiple Text document summarization system using hybrid summarization technique. In: Proceedings of the 2015 1st International Conference on Next Generation Computing Technologies (NGCT), IEEE, p804-808. https://doi.org/10.1109/ngct.2015.7375231

[7] Dilawari, A., Khan, M.U.G., Saleem, S., Zahoor-Ur-Rehman, & Shaikh, F.S. (2023). Neural attention model for abstractive text summarization using linguistic feature space. IEEE Access, 11, 23557-23564. https://doi.org/10.1109/access.2023.3249783

[8] Ding, J., Li, Y., Ni, H., & Yang, Z. (2020). Generative text summary based on enhanced semantic attention and gain-benefit gate. IEEE Access, 8, 92659-92668. https://doi.org/10.1109/access.2020.2994092

[9] Du, Y., & Huo, H. (2020). News text summarization based on multi-feature and fuzzy logic. IEEE Access, 8, 140261-140272. https://doi.org/10.1109/access.2020.3007763

[10] Elbarougy, R., Behery, G., & El Khatib, A. (2020). Extractive Arabic text summarization using modified PageRank algorithm. Egyptian Informatics Journal, 21(2), 73-81. https://doi.org/10.1016/j.eij.2019.11.001

[11] El-Kassas, W.S., Salama, C.R., Rafea, A.A., & Mohamed, H.K. (2020). EdgeSumm: Graph-based framework for automatic text summarization. Information Processing and Management, 57(6), 102264. https://doi.org/10.1016/j.ipm.2020.102264

[12] Gambhir, M., & Gupta, V. (2017). Recent automatic text summarization techniques: A survey. Artificial Intelligence Review, 47(1), 1-66. https://doi.org/10.1007/s10462-016-9475-9

[13] Hassan, A.Q.A., Al-Onazi, B.B., Maashi, M., Darem, A.A., Abunadi, I., & Mahmud, A. (2024). Enhancing extractive text summarization using natural language processing with an optimal deep learning model. AIMS Mathematics, 9(5), 12588-12609. https://doi.org/10.3934/math.2024616

[14] Hernández-Castañeda, Á., García-Hernández, R.A., & Ledeneva, Y. (2023). Toward the automatic generation of an objective function for extractive text summarization. IEEE Access, 11, 51455-51464. https://doi.org/10.1109/access.2023.3279101

[15] Hernández-Castañeda, Á., García-Hernández, R.A., Ledeneva, Y., & Millán-Hernández, C.E. (2020). Extractive automatic text summarization based on lexical-semantic keywords. IEEE Access, 8, 49896-49907. https://doi.org/10.1109/access.2020.2980226

[16] Hernández-Castañeda, Á., García-Hernández, R.A., Ledeneva, Y., & Millán-Hernández, C.E. (2022). Language-independent extractive automatic text summarization based on automatic keyword extraction. Computer Speech and Language, 71, 101256. https://doi.org/10.1016/j.csl.2021.101267

[17] Jadhav, A., Jain, R., Fernandes, S., & Shaikh, S. (2019). Text Summarization using Neural Networks. In 2019 6^th IEEE International Conference on Advances in Computing, Communication and Control (ICAC3). https://doi.org/10.1109/icac347590.2019.9036739

[18] Jain, A., Bhatia, D., & Thakur, M.K. (2017). Extractive text summarization using word vector embedding. In: 2017 International Conference on Machine Learning and Data Science (MLDS), p51-55. https://doi.org/10.1109/mlds.2017.12

[19] Jindal, S.G., & Kaur, A. (2020). Automatic keyword and sentence-based text summarization for software bug reports. IEEE Access, 8, 65352-65370. https://doi.org/10.1109/access.2020.2985222

[20] Joshi, A., Fidalgo, E., Alegre, E., & Fernández-Robles, L.(2019). SummCoder: An unsupervised frameworkfor extractive text summarization based on deep auto-encoders. Expert Systems with Applications, 129, 200-215. https://doi.org/10.1016/j.eswa.2019.03.045

[21] Khan, B., Usman, M., Khan, I., Khan, J., Hussain, D.,& Gu, Y.H. (2025). Next-generation text summarization: A T5-LSTM FusionNet hybrid approach for psychological data. IEEE Access, 13, 1-15.

[22] Liang, Z., Du, J., & Li, C. (2020). Abstractive social media text summarization using selective reinforced Seq2Seq attention model. Neurocomputing, 410,432-440. https://doi.org/10.1016/j.neucom.2020.04.137

[23] Madhuri, J.N., & Kumar, R.G. (2019). Extractive Text Summarization using Sentence Ranking. In: Proceedings of the 2019 International Conference on Data Science and Communication (IconDSC), p1-3. https://doi.org/10.1109/icondsc.2019.8817040

[24] Mitra, M., Buckley, C., & Research, S. (2000). Automatic Text Summarization by Paragraph Extraction. Available from: https://www.researchgate.net/publication/2457789 [Last accessed on 2025 Sep 09].

[25] Moradi, M., Dorffner, G., & Samwald, M. (2020). Deep contextualized embeddings for quantifying the informative content in biomedical text summarization. Computer Methods and Programs in Biomedicine, 184, 105117. https://doi.org/10.1016/j.cmpb.2019.105117

[26] Onan, A., & Alhumyani, H. (2024b). Contextual hypergraph networks for enhanced extractive summarization: Introducing multi-element contextual hypergraph extractive summarizer (MCHES). Applied Sciences, 14(11), 4671. https://doi.org/10.3390/app14114671

[27] Onan, A., & Alhumyani, H.A. (2024a). FuzzyTP-BERT: Enhancing extractive text summarization with fuzzy topic modeling and transformer networks. Journal of King Saud University - Computer and Information Sciences, 36(6), 102080. https://doi.org/10.1016/j.jksuci.2024.102080

[28] Pradip, K.G., & Patil, D.R. (2016). Summarization of Sentences using Fuzzy and Hierarchical Clustering Approach. In: 2016 Symposium on Colossal Data Analysis and Networking (CDAN). IEEE. p1-7. https://doi.org/10.1109/cdan.2016.7570907

[29] Prameswari, P., Zulkarnain, Z, Surjandari, I., & Laoh, E. (2018). Mining Online Reviews in Indonesia’s Priority Tourist Destinations using Sentiment Analysis and Text Summarization Approach. In: 2017 IEEE 8^th International Conference on Awareness Science and Technology (iCAST). IEEE, p36-41. https://doi.org/10.1109/icawst.2017.8256540

[30] Qaroush, A., Abu Farha, I., Ghanem, W., Washaha, M.,& Maali, E. (2021). An efficient single document Arabic text summarization using a combination of statistical and semantic features. Journal of King Saud University - Computer and Information Sciences, 33(6), 677-692. https://doi.org/10.1016/j.jksuci.2019.03.010

[31] Sanchez-Gomez, J.M., Vega-Rodríguez, M.A., & Pérez, C.J. (2019). Parallelizing a multi-objective optimization approach for extractive multi-document text summarization. Journal of Parallel and Distributed Computing, 134, 166-179. https://doi.org/10.1016/j.jpdc.2019.09.001

[32] Toprak, A., & Turan, M. (2025). Enhanced automatic abstractive document summarization using transformers and sentence grouping. The Journal of Supercomputing, 81(4), 1-30. https://doi.org/10.1007/s11227-025-07048-6

[33] Tsai, C.F., Chen, K., Hu, Y.H., & Chen, W.K. (2020). Improving text summarization of online hotel reviews with review helpfulness and sentiment. Tourism Management, 80, 104122. https://doi.org/10.1016/j.tourman.2020.104122

[34] Van Lierde, H., & Chow, T.W.S. (2019). Learning with fuzzy hypergraphs: A topical approach to query-oriented text summarization. Information Sciences, 496, 212-224. https://doi.org/10.1016/j.ins.2019.05.020

[35] Wan, L. (2018). Extraction Algorithm of English Text Summarization for English Teaching. In: 2018 3^rd International Conference on Intelligent Transportation, Big Data and Smart City (ICITBS). https://doi.org/10.1109/icitbs.2018.00085

[36] Yao, K., Zhang, L., Du, D., Luo, T., Tao, L., & Wu, Y. (2018). Dual encoding for abstractive text summarization. IEEE Transactions on Cybernetics. IEEE, New York. https://doi.org/10.1109/tcyb.2018.2876317

[37] Zajic, D.M., Dorr, B.J., & Lin, J. (2008). Single-document and multi-document summarization techniques for email threads using sentence compression. Information Processing and Management, 44(4), 1600-1610. https://doi.org/10.1016/j.ipm.2007.09.007

[38] Zenkert, J., Klahold, A., & Fathi, M. (2018). Towards Extractive Text Summarization using Multidimensional Knowledge Representation. In: 2018 IEEE International Conference on Electro/Information Technology (EIT), p826-831. https://doi.org/10.1109/EIT.2018.8500186