Enhancing Power System Flexibility Using AI-Based Forecasting Techniques: A Comparative Study

Abstract

This paper compares seven forecasting models for hourly electricity consumption in a commercial office building using data spanning 2024–2025. Models include XGBoost, LSTM, GRU, 1D-CNN, SARIMA, Prophet, and Seasonal Naive baseline. Features encompass temporal indicators (hour, day of week, month), autoregressive lags (1, 2, 24, 168 h), and rolling statistics. Evaluation uses Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) on a 14-day test set (336 samples) with rigorous hyperparameter tuning via GridSearchCV and TimeSeriesSplit cross-validation. XGBoost achieves superior performance (MAE 6.29 kW, 3.5% MAPE) compared to GRU (10.95 kW), 1D-CNN (11.86 kW), LSTM (14.98 kW), Seasonal Naive (16.15 kW), Prophet (35.72 kW), and SARIMA (48.16 kW). Paired t-tests confirm statistical significance: XGBoost versus GRU (t=18.73,p<.0001) and versus Seasonal Naive (t=13.47,p<.0001). Surprisingly, deep learning models underperformed gradient boosting despite theoretical sequence-modeling advantages, attributed to modest sample size (17,016), rich feature engineering capturing 69.5% of variance through autoregressive features, and single-hour forecasting horizon. Classical statistical models exhibited catastrophic failures, reflecting inadequate modeling of non-stationary, non-linear building consumption with multiple seasonal patterns. Results demonstrate that for structured tabular time series with comprehensive feature engineering, gradient boosting substantially outperforms sequential neural architectures and classical statistical methods. The findings enable high-confidence building energy management decisions (HVAC pre-conditioning, demand response) with ±6.3 kW prediction accuracy. Code and reproducibility documentation are available at https://github.com/SaadHayat91/BEJ-Electricity-Forecasting.

Keywords

AI-based forecasting / anomaly detection / energy consumption / GRU / LSTM / neural tree model / power system flexibility / prophet / XGBoost

References

[1] B. Wang, Y. Song, and X. Liu, “Power System Flexibility: A Review of Concepts and Performance Measures,” Renewable and Sustainable Energy Reviews35 (2021): 233-243.

[2] N. Hatziargyriou, H. Asano, R. Iravani, and C. Marnay, “Microgrids: An Overview of Ongoing Research, Development, and Demonstration Projects,” IEEE Power and Energy Magazine5, no. 4 (2020): 78-94.

[3] S. K. Khaitan and J. D. McCalley, “Design Techniques and Applications of Cyber-Physical Systems: A Survey,” IEEE Systems Journal14, no. 1 (2019): 234-244.

[4] P. Siano, “Demand Response and Smart Grids—A Survey,” Renewable and Sustainable Energy Reviews30 (2014): 461-478.

[5] M. Amin and B. F. Wollenberg, “Toward a Smart Grid: Power Delivery for the 21st Century,” IEEE Power and Energy Magazine3, no. 5 (2019): 34-41.

[6] P. Denholm, M. O'Connell, G. Brinkman, and J. Jorgenson, “Overgeneration From Solar Energy in California: A Field Guide to the Duck Chart,” National Renewable Energy Laboratory (2018).

[7] J. T. Saraiva, “Load Forecasting in Smart Grids: A State-of-the-Art Survey,” Electric Power Systems Research192 (2021): 0.

[8] H. S. Hippert, C. E. Pedreira, and R. C. Souza, “Neural Networks for Short-Term Load Forecasting: A Review and Evaluation,” IEEE Transactions on Power Systems16, no. 1 (2018): 44-55.

[9] S. Rahman and R. Bhatnagar, “An Expert System Based Algorithm for Short Term Load Forecasting,” IEEE Transactions on Power Systems3, no. 2 (2020): 392-399.

[10] A. J. Conejo, M. A. Plazas, R. Espinola, and A. B. Molina, “Day-Ahead Electricity Price Forecasting Using the Wavelet Transform and ARIMA Models,” IEEE Transactions on Power Systems20, no. 2 (2021): 1035-1042.

[11] R. Ahmed, A. Nawaz, Z. Javid, M. Y. A. Khan, A. A. Shah, and O. M. Valarezo, “Optimal Transmission Switching Based on Probabilistic Load Flow in Power System With Large-Scale Renewable Energy Integration,” Electrical Engineering104, no. 2 (2022): 883-898.

[12] J. W. Taylor and R. Buizza, “Using Weather Ensemble Predictions in Electricity Demand Forecasting,” International Journal of Forecasting19, no. 1 (2021): 57-70.

[13] P. C. Refenes, A. N. Burgess, and Y. Bentz, “Neural Networks in Financial Engineering: A Study in Methodology,” IEEE Transactions on Neural Networks8, no. 6 (2019): 1222-1264.

[14] M. Ghiassi, D. K. Zimbra, and H. Saidane, “Medium Term System Load Forecasting With a Dynamic Artificial Neural Network Model,” Electric Power Systems Research76, no. 5 (2018): 302-316.

[15] G. Gross and F. D. Galiana, “Short-Term Load Forecasting,” Proceedings of the IEEE75, no. 12 (2020): 1558-1573.

[16] T. Haida and S. Muto, “Regression Based Peak Load Forecasting Using a Transformation Technique,” IEEE Transactions on Power Systems9, no. 4 (2019): 1788-1794.

[17] Y. Wang, Q. Chen, C. Kang, and M. O. O. Hong, “Sparse and Redundant Representation Based Smart Meter Load Forecasting,” IEEE Transactions on Power Systems29 (2020): 360-372.

[18] A. Nawaz, H. Wang, H. Yang, H. Armghan, and J. Gao, “Risk-Constrained Probabilistic Coordination in Coupled Transmission and Distribution System,” Electric Power Systems Research228 (2024): 110005.

[19] Z. Xiaoli, L. Weijun, and X. Weidou, “Short-Term Load Forecasting Using an ARIMA Model,” IEEE Transactions on Power Systems12, no. 2 (2019): 450-455.

[20] S. Fan and R. J. Hyndman, “Short-Term Load Forecasting Based on a Semi-Parametric Additive Model,” IEEE Transactions on Power Systems27, no. 1 (2020): 358-364.

[21] M. Khashei and M. Bijari, “A Novel Hybridization of Artificial Neural Networks and ARIMA Models for Time Series Forecasting,” Applied Soft Computing11, no. 2 (2019): 2664-2675.

[22] P. J. Brockwell and R. A. Davis, “Introduction to Time Series and Forecasting,” Springer (2018).

[23] G. Dudek, “Pattern-Based Forecasting of Load Time Series,” IEEE Transactions on Power Systems28, no. 3 (2018): 1483-1490.

[24] C. Deb, S. E. Lee, and M. S. Yang, “Predictive Modelling for Adaptive Thermal Comfort in Contemporary Office Buildings,” Energy and Buildings60 (2018): 244-257.

[25] M. Ahmed and A. Khalid, “A Review on the Role of Smart Grid in Mitigating the Impact of Climate Change,” Renewable and Sustainable Energy Reviews63 (2019): 810-821.

[26] D. Nguyen and B. Reiter, “The Role of Big Data and Machine Learning in Smart Grid Systems: A Review,” Journal of Cleaner Production255 (2020).

[27] H. Kim and J. Park, “Short-Term Load Forecasting Using a Deep Neural Network,” IEEE Transactions on Power Systems31, no. 3 (2018): 2024-2031.

[28] N. Amjady and F. Keynia, “Short-Term Load Forecasting of Power Systems by Combination of Wavelet Transform and Neuro-Evolutionary Algorithm,” Energy34, no. 1 (2019): 46-57.

[29] T. Hong, M. Gui, and J. Wilson, “Probabilistic Electric Load Forecasting: A Tutorial Review,” International Journal of Forecasting32, no. 3 (2018): 914-938.

[30] F. M. Alobaidi, M. A. Negnevitsky, and Y. Xue, “Hybrid Wavelet-Neuro-Fuzzy System for Short-Term Load Forecasting,” IEEE Transactions on Power Systems26, no. 1 (2019): 245-256.

[31] Y. Sun, L. M. Su, and X. Lu, “Probabilistic Load Forecasting Based on Quantile Regression Neural Network and Kernel Density Estimation,” Energy113 (2020): 624-636.

[32] X. Wang, Y. Wang, and H. Song, “A Novel Hybrid Model Based on Advanced Feature Selection for Daily Urban Water Demand Forecasting,” Journal of Hydrology560 (2018): 56-67.

[33] S. M. Chen, J. S. Shieh, and B. F. M. Lin, “A Novel Short-Term Load Forecasting System Using an Evolutionary Fuzzy Hybrid Network,” IEEE Transactions on Power Systems27, no. 1 (2018): 141-149.

[34] H. Liu and D. Yang, “A Hybrid Model Based on LSTM and XGboost for Urban Water Demand Forecasting,” Journal of Hydrology561 (2018): 52-60.

[35] A. Z. Zhang, H. N. Wu, and Z. B. Ma, “A Novel Hybrid Short-Term Load Forecasting Model Based on Support Vector Machines and Improved Ant Colony Optimization,” IEEE Transactions on Power Systems29, no. 4 (2019): 1394-1403.

[36] P. S. d. Lima and T. F. O. Ribeiro, “Hybrid Adaptive Model Based on Seasonal Decomposition for Energy Load Forecasting,” IEEE Transactions on Power Systems32, no. 2 (2019): 1552-1563.

[37] J. L. Zhang, L. Wang, and H. Liu, “Short-Term Load Forecasting Using a Hybrid Model Based on Support Vector Machine and Chaotic Differential Evolution Algorithm,” Applied Energy157 (2019): 19-29.

[38] S. Li, X. Li, and X. Bai, “A Novel Hybrid Model for Short-Term Load Forecasting Using ANFIS and Improved Particle Swarm Optimization,” Energy60 (2020): 132-144.

[39] H. Wang, Q. Zhang, and W. Xu, “Probabilistic Short-Term Load Forecasting Based on Multi-Objective Optimization,” IEEE Transactions on Power Systems30, no. 6 (2018): 2561-2569.

[40] R. D. Zimmermann, C. E. Murillo-Sánchez, and R. J. Thomas, “MATPOWER: Steady-State Operations, Planning, and Analysis Tools for Power Systems Research and Education,” IEEE Transactions on Power Systems26, no. 1 (2019): 12-19.

[41] J. Xu, H. Zhang, and S. Ji, “Hybrid Method Combining Ensemble Empirical Mode Decomposition and Long Short-Term Memory Neural Network for Smart Meter Data Forecasting,” IEEE Transactions on Power Systems34, no. 1 (2019): 431-440.

[42] M. R. Jordehi and M. Z. A. A. Kadir, “A Review of Power System Flexibility With High Penetration of Renewable Energy,” Renewable and Sustainable Energy Reviews68 (2018): 306-316.

[43] G. Dudek, “Neural Networks for Short-Term Load Forecasting: A Comprehensive Review,” IEEE Transactions on Power Systems34, no. 3 (2019): 1213-1224.

[44] T. T. Hong, J. G. Price, and J. S. Lee, “Evaluation of Hybrid Methods Combining Artificial Neural Networks and Statistical Models for Short-Term Load Forecasting,” Applied Energy160 (2020): 175-186.

[45] S. Rahman and A. A. Oke, “A Comprehensive Review of Artificial Neural Network Applications in Power System Stability and Load Forecasting,” IEEE Transactions on Power Systems33, no. 4 (2018): 3826-3835.

[46] Y. C. Lee and T. H. Chen, “Applying Data Mining Techniques for Load Forecasting: A Case Study of Taiwan Power Company,” IEEE Transactions on Power Systems25, no. 4 (2019): 1941-1951.

[47] J. E. Espinoza, F. C. Teofilo, and S. L. Souza, “Load Forecasting Using Multi-Scale Deep Learning Models,” IEEE Transactions on Power Systems34, no. 6 (2019): 5586-5597.

[48] R. Weron, “Electricity Price Forecasting: A Review of the State-of-the-Art With a Look Into the Future,” International Journal of Forecasting30, no. 4 (2018): 1030-1081.

[49] C. F. Juarez, S. S. Junior, and M. F. Silva, “Improving Short-Term Load Forecasting Using Wavelet Transform and Neural Networks,” IEEE Transactions on Power Systems34, no. 3 (2019): 2121-2130.

[50] D. W. Bunn and E. D. Farmer, “Comparative Models for Electrical Load Forecasting,” Wiley (2020).

[51] J. E. Cohen and M. L. Puterman, “Renewable Energy Integration and Power System Flexibility: A Review,” Renewable and Sustainable Energy Reviews92 (2020): 187-198.

[52] A. T. Erdogan and M. F. Uygun, “A Hybrid Model for Short-Term Load Forecasting Based on Wavelet Transform and Neural Networks,” Energy119 (2018): 53-62.

[53] A. Z. Zhang and H. N. Wu, “Hybrid Model for Short-Term Load Forecasting Based on Emd and Svm,” Energy68 (2019): 170-178.

[54] F. M. Alobaidi, M. A. Negnevitsky, and Y. Xue, “Short-Term Load Forecasting Using Adaptive Neuro-Fuzzy Inference Systems,” IEEE Transactions on Power Systems27, no. 1 (2020): 123-130.

[55] S. Ong and N. Clark, “Commercial and Residential Hourly Load Profiles for all TMY3 Locations in the United States,” (2014).

[56] L. M. H. Lee, Y. C. Ser, G. Selvachandran, et al., “A Comparative Study of Forecasting Electricity Consumption Using Machine Learning Models,” Mathematics10, no. 8 (2022): 1329.

[57] E. Energy, “Yearly Electricity Data,” (2025), accessed: 2025-01-04. [Online], https://ember-energy.org/data/yearly-electricity-data/.

RIGHTS & PERMISSIONS

2025 The Author(s). Battery Energy published by Xijing University and John Wiley & Sons Australia, Ltd.