Real-time electricity load forecasting in South Africa using SOM-enriched deep learning ensembles

Abstract

Accurate short-term electrical demand forecasting is critical for maintaining operational efficiency and energy security, especially in power-constrained systems like South Africa’s Eskom. Statistical methods like autoregressive integrated moving average (ARIMA) and exponential smoothing often fail to represent nonlinear and regime-dependent trends in power demand. This study presents a dynamic ensemble that combines deep neural networks (DNN) and long short-term memory (LSTM) architectures, which are both augmented by self-organising maps (SOM)-based clustering. The proposed method divides historical hourly load data from the Drakensberg generation plant into discrete temporal regimes using SOM, then trains the DNN and LSTM architectures within each regime, and dynamically combines their predictions. Shapley additive explanations (SHAP) are used to improve the interpretability of the impact of each cluster and time hierarchies, while mean square error (MSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) measures are used to assess prediction performance. The ensemble architecture delivers a higher accuracy, lowering MAPE to 2.20% while consistently outperforming individual benchmark architectures. The deployment on Amazon Web Services (AWS) proves the model’s scalability and appropriateness for real-time applications. Although performance degrades in irregular demand clusters, adaptive re-clustering may alleviate this constraint. Overall, the combined DNN-LSTM-SOM strategy is a reliable, interpretable, and scalable solution for short-term load forecasting, enabling better operational planning and grid dependability in developing energy systems.

Keywords

deep neural networks / energy security / electricity load forecasting / LSTM networks / nonlinear ensemble learning / self-organising maps

References

[1] Rostum M, Zamel AA, Moustafa H, et al. (2020) Electrical load forecasting: A methodological overview. Int J Eng Technol 9: 842-869. https://doi.org/10.14419/ijet.v9i3.30706

[2] Bunn DW (2000) Forecasting loads and prices in competitive power markets. Proc IEEE 88: 163-169. https://doi.org/10.1109/5.823996

[3] Hobbs BF, Jitprapaikulsarn S, Konda S, et al. (1999) Analysis of the value for unit commitment of improved load forecasts. IEEE Trans Power Syst 14: 1342-1348. https://doi.org/10.1109/59.801894

[4] Dudek G (2016) Pattern-based local linear regression models for short-term load forecasting. Electr Power Syst Res 130: 139-147. https://doi.org/10.1016/j.epsr.2015.09.001

[5] Percuku A, Minkovska D, Hinov N (2025) Enhancing electricity load forecasting with machine learning and deep learning. Technologies 13: 59. https://doi.org/10.3390/technologies13020059

[6] Al-Ani BRK, Erkan TE (2022) A study of load demand forecasting models in electricity using artificial neural networks and fuzzy logic models. Int J Eng 35: 1111-1118. https://doi.org/10.5829/ije.2022.35.06c.02

[7] Eberhart, Shi Y (2001) Particle swarm optimisation: Developments, applications and resources. Proceedings of the 2001 Congress on Evolutionary Computation, IEEE 1: 1-86. https://doi.org/10.1109/CEC.2001.934374

[8] Kim J, Kim H, Kim H, et al. (2025) A comprehensive survey of deep learning for time series forecasting: Architectural diversity and open challenges. Artif Intell Rev 58: 1-95. https://doi.org/10.1007/s10462-025-11223-9

[9] Islam BU, Ahmed SF (2022) Short-term electrical load demand forecasting based on LSTM and RNN deep neural networks. Math Probl Eng 2022: 2316474. https://doi.org/10.1155/2022/2316474

[10] Mohan P, Patil KK (2018) Deep learning-based weighted SOM to forecast weather and crop prediction for agriculture applications. Int J Intell Eng Syst 11: 167-176. https://doi.org/10.22266/ijies2018.0831.17

[11] Guo F, Li S, Zhao G, et al. (2024) A SOM-LSTM combined model for groundwater level prediction in karst critical zone aquifers considering connectivity characteristics. Environ Earth Sci 83: 267. https://doi.org/10.1007/s12665-024-11567-5

[12] Raj RA, Sarathkumar D, Andrews LJB, et al. (2023) Key gases in transformer oil-An analysis using self-organising map (SOM) neural networks. 2023 IEEE 12th International Conference on Communication Systems and Network Technologies (CSNT), 642-647. https://doi.org/10.1109/CSNT57126.2023.10134597

[13] Aguilar Madrid E, Antonio N (2021) Short-term electricity load forecasting with machine learning. Information 12: 50. https://doi.org/10.3390/info12020050

[14] Baur L, Ditschuneit K, Schambach M, et al. (2024) Explainability and interpretability in electric load forecasting using machine learning techniques: A review. Energy AI 16: 100358. https://doi.org/10.1016/j.egyai.2024.100358

[15] Wang A, Yu Q, Wang J, et al. (2023) Electric load forecasting based on deep ensemble learning. Appl Sci 13: 9706. https://doi.org/10.3390/app13179706

[16] Shoko C, Moroke ND, Makatjane K (2024) A deep learning framework for modelling temporal dependencies and hierarchies in hourly electricity demand load. In: Acharjya P, Koley S, Barman S (eds) Machine Learning and Computer Vision for Renewable Energy. IGI Global , 42-65. https://doi.org/10.4018/979-8-3693-2355-7.ch003

[17] Baptista ML, Henriques EM, Goebel K (2021) A self-organising map and a normalising self- organising multi-layer perceptron approach to baselining in prognostics under dynamic regimes. Neurocomputing 456: 268-287. https://doi.org/10.1016/j.neucom.2021.05.031

[18] Sun S, Wang S, Wei Y, et al. (2020) A clustering-based nonlinear ensemble approach for exchange rates forecasting. IEEE Trans Syst Man Cybern Syst 50: 2284-2292. https://doi.org/10.1109/TSMC.2018.2799869

[19] Alotaibi MA (2022) Machine learning approach for short-term load forecasting using deep neural network. Energies 15: 6261. https://doi.org/10.3390/en15176261

[20] Goodfellow I, Bengio Y, Courville A (2016) Deep Learning. MIT Press, Cambridge. Available from: https://virtualmmx.ddns.net/gbooks/DeepLearning.pdf.

[21] Wang H, Li T, Zhuang Z, et al. (2023) Early stopping for deep image prior. Trans Mach Learn Res. Available from: https://openreview.net/forum?id=231ZzrLC8X.

[22] Oruh J, Viriri S, Adegun A (2022) Long short-term memory recurrent neural network for automatic speech recognition. IEEE Access 10: 30069-30079. https://doi.org/10.1109/ACCESS.2022.3159339

[23] Toba AL, Kulkarni S, Khallouli W, et al. (2025) Long-term traffic prediction using deep learning long short-term memory. Smart Cities 8: 126. https://doi.org/10.3390/smartcities8040126

[24] Echrigui R, Hamiche M (2023) Optimising LSTM models for EUR/USD prediction in the context of reducing energy consumption. E3S Web Conf 412: 01069. https://doi.org/10.1051/e3sconf/202341201069

[25] Kim HS, Choi D, Yoo DG, et al. (2022) Hyperparameter sensitivity analysis of deep learning-based pipe burst detection model for multiregional water supply networks. Sustainability 14: 13788. https://doi.org/10.3390/su142113788

[26] He X, Zhao W, Zhang L, et al. (2024) A novel ensemble deep reinforcement learning model for short-term load forecasting based on Q-learning dynamic model selection. J Eng 2024: 1-12. https://doi.org/10.1049/tje2.12409

[27] Murthy R, Padmalatha NA (2025) Iterative error-based ensemble forecasting model: A novel framework for enhanced forecast accuracy. IUP J Appl Econ 24: 85-95. https://doi.org/10.71329/IUPJAE/2025.16.1.85-95

[28] Dantas TM, Oliveira FLC (2018) Improving time series forecasting: An approach combining bootstrap aggregation, clusters and exponential smoothing. Int J Forecast 34: 748-761. https://doi.org/10.1016/j.ijforecast.2018.05.006

[29] Eskom (2025) Power system remains stable with a sustained reduction in unplanned outages. Available from: https://www.eskom.co.za/ power-system-remains-stable-with-a-sustained-reduction-in-unplanned-outages/.

[30] Xaba D, Makatjane K, Senosi A (2024) Prediction accuracy of SARIMA-STAR-CNE. In: Vasant P, Panchenko V, Munapo E, et al (eds) Intelligent Computing and Optimisation. Lect Notes Netw Syst, 1169. Springer, Cham. https://doi.org/10.1007/978-3-031-73324-6 31

[31] Januzaj Y, Beqiri E, Luma A (2023) Determining the optimal number of clusters using silhouette score as a data mining technique. Int J Online Biomed Eng, 19. https://doi.org/10.3991/ijoe.v19i04.37059

[32] Majidi F (2023) A hybrid SOM and K-means model for clustering time series energy consumption clustering. arXiv . https://doi.org/10.48550/arXiv.2312.11475

[33] Toussaint W, Moodley D (2020) Clustering residential electricity consumption data to create archetypes that capture household behaviour in South Africa. S Afr Comput J 32: 1-34. Available from: https://hdl.handle.net/10520/ejc-comp-v32-n2-a3

[34] Sigauke C, Moroke N, Makatjane K, et al. (2025) A deep learning forecasting of downside risk: Application of a combined ESRNN-VAE. Front Appl Math Stat 11: 1662252. https://doi.org/10.3389/fams.2025.1662252

[35] Meswal H, Kumar D, Gupta A, et al. (2024) A weighted ensemble transfer learning approach for melanoma classification from skin lesion images. Multimed Tools Appl 83: 33615-33637. https://doi.org/10.1007/s11042-023-16783-y

[36] Amnuaypongsa W, Wangdee W, Songsiri J (2025) Neural network-based prediction interval estimation with large width penalisation for renewable energy forecasting and system applications. Energy Convers Manag X 27: 101119. https://doi.org/10.1016/j.ecmx.2025.101119

[37] Mazibuko T, Akindeji K (2025) Hybrid forecasting for energy consumption in South Africa: LSTM and XGBoost approach. Energies 18: 4285. https://doi.org/10.3390/en18164285

[38] SANEDI (2023) Short-term electricity demand forecasting for South Africa: Implications for grid planning and policy. South African National Energy Development Institute . Available from: https://sanedi.org.za/.

[39] Eskom (2023) Medium-Term System Adequacy Outlook 2024-2028. Eskom Holdings SOC Ltd, Pretoria. Available from: https://www.eskom.co.za/wp-content/uploads/2023/11/ Medium_Term_System_Adequacy_Outlook_2024-2028.pdf.