Hybrid model with temporal convolutional network and transformer encoder for privacy-preserving wind power forecasting

Abstract

The data dependence of deep learning models for wind power forecasting presents a significant challenge for newly built wind farms, which may lack sufficient data for effective model training. Overcoming this limitation by amalgamating data from other wind turbines introduces an inherent risk of privacy leakage. To address this issue, we introduced a novel hybrid model for wind power forecasting, specifically designed to address data privacy preservation. This model integrates a temporal convolutional network with a transformer encoder, harnessing the strengths of both components. The temporal convolutional network extracts local temporal patterns, whereas the transformer encoder captures the intricate time dependencies between these patterns. To address the data privacy risks caused by traditional centralized forecasting methods, a federated training strategy is implemented to train a collaborative knowledge-sharing model, ensuring that the original data from the source domains remain confidential and undisclosed. The experimental results validate the effectiveness of the proposed model, showing significant improvements in forecasting performance. When federated with other wind turbines, the average mean absolute error and the root mean square error of six wind turbines are 4.624 6 and 6.245 7, respectively. In contrast to traditional centralized training methods, the proposed federated learning method, which was applied to inland wind turbines, achieved almost no increase in losses.

Keywords

Temporal convolutional network / Privacy-preserving / Federated learning / Wind power forecasting / Transformer

References

[1] YanJ, MöhrlenC, GöçmenT, et al. . Uncovering wind power forecasting uncertainty sources and their propagation through the whole modelling chain. Renew Sustain Energy Rev, 2022, 165. 112519

[2] WangY, ZouR, LiuF, et al. . A review of wind speed and wind power forecasting with deep neural networks. Appl Energy, 2021, 304. 117766

[3] KisvariA, LinZ, LiuX. Wind power forecasting–a data-driven method along with gated recurrent neural network. Renew Energy, 2021, 163: 1895-1909.

[4] LiuL, LiuJ, YeY, et al. . Ultra-short-term wind power forecasting based on deep Bayesian model with uncertainty. Renew Energy, 2023, 205: 598-607.

[5] ZhangY, LiY, ZhangG. Short-term wind power forecasting approach based on Seq2Seq model using NWP data. Energy, 2020, 213. 118371

[6] CheJ, YuanF, DengD, et al. . Ultra-short-term probabilistic wind power forecasting with spatial-temporal multi-scale features and K-FSDW based weight. Appl Energy, 2023, 331. 120479

[7] LiL, LiuZ, TsengM, et al. . Using enhanced crow search algorithm optimization-extreme learning machine model to forecast short-term wind power. Expert Syst Appl, 2021, 184. 115579

[8] KarijadiI, ChouSY, DewabharataA. Wind power forecasting based on hybrid CEEMDAN-EWT deep learning method. Renew Energy, 2023, 218. 119357

[9] SunS, LiuY, LiQ, et al. . Short-term multi-step wind power forecasting based on spatio-temporal correlations and transformer neural networks. Energy Convers Manage, 2023, 283. 116916

[10] YildizC, AcikgozH, KorkmazD, et al. . An improved residual-based convolutional neural network for very short-term wind power forecasting. Energy Convers Manage, 2021, 228. 113731

[11] SunZ, ZhaoM, DongY, et al. . Hybrid model with secondary decomposition, randomforest algorithm, clustering analysis and long short memory network principal computing for short-term wind power forecasting on multiple scales. Energy, 2021, 221. 119848

[12] HossainMA, ChakraborttyRK, ElsawahS, et al. . Very short-term forecasting of wind power generation using hybrid deep learning model. J Clean Prod, 2021, 296. 126564

[13] HongYY, RiofloridoCLPP. A hybrid deep learning-based neural network for 24-h ahead wind power forecasting. Appl Energy, 2019, 250: 530-539.

[14] AhmadT, ZhangD. A data-driven deep sequence-to-sequence long-short memory method along with a gated recurrent neural network for wind power forecasting. Energy, 2022, 239. 122109

[15] XiongB, LouL, MengX, et al. . Short-term wind power forecasting based on attention mechanism and deep learning. Electr Power Syst Res, 2022, 206. 107776

[16] MekaR, AlaeddiniA, BhaganagarK. A robust deep learning framework for short-term wind power forecast of a full-scale wind farm using atmospheric variables. Energy, 2021, 221. 119759

[17] TangY, YangK, ZhangS, et al. . Wind power forecasting:a temporal domain generalization approach incorporating hybrid model and adversarial relationship-based training. Appl Energy, 2024, 355. 122266

[18] DongY, ZhangH, WangC, et al. . A novel hybrid model based on Bernstein polynomial with mixture of Gaussians for wind power forecasting. Appl Energy, 2021, 286. 116545

[19] DuanJ, WangP, MaW, et al. . A novel hybrid model based on nonlinear weighted combination for short-term wind power forecasting. Int J Electr Power Energy Syst, 2022, 134. 107452

[20] ZhuJ, SuL, LiY. Wind power forecasting based on new hybrid model with TCN residual modification. Energy AI, 2022, 10. 100199

[21] ZhaoZ, YunS, JiaL, et al. . Hybrid VMD-CNN-GRU-based model for short-term forecasting of wind power considering spatio-temporal features. Eng Appl Artif Intell, 2023, 121. 105982

[22] WangH, SongK, ChengY. A hybrid forecasting model based on CNN and informer for short-term wind power. Front Energy Res, 2022, 9. 788320

[23] XiangL, FuX, YaoQ, et al. . A novel model for ultra-short term wind power prediction based on vision transformer. Energy, 2024, 294. 130854

[24] Al-HajjR, AssiA, NejiB, et al. . Transfer learning for renewable energy systems: a survey. Sustainability, 2023, 15119131.

[25] TangY, YangK, ZhangS, et al. . Photovoltaic power forecasting: a hybrid deep learning model incorporating transfer learning strategy. Renew Sustain Energy Rev, 2022, 162. 112473

[26] LiuY, WangJ. Transfer learning based multi-layer extreme learning machine for probabilistic wind power forecasting. Appl Energy, 2022, 312. 118729

[27] TangY, YangK, ZhangS, et al. . Wind power forecasting: a hybrid forecasting model and multi-task learning-based framework. Energy, 2023, 278. 127864

[28] TangL, XieH, WangX, et al. . Privacy-preserving knowledge sharing for few-shot building energy prediction: a federated learning approach. Appl Energy, 2023, 337. 120860

[29] MoayyedH, MoradzadehA, Mohammadi-IvatlooB, et al. . A cyber-secure generalized supermodel for wind power forecasting based on deep federated learning and image processing. Energy Convers Manage, 2022, 267. 115852

[30] LiY, WangR, LiY, et al. . Wind power forecasting considering data privacy protection: a federated deep reinforcement learning approach. Appl Energy, 2023, 329. 120291

[31] ZhaoY, PanZY, et al. . Ultra-short-term wind power forecasting based on personalized robust federated learning with spatial collaboration. Energy, 2024, 288. 129847

[32] TangY, ZhangS, ZhangZ. A privacy-preserving framework integrating federated learning and transfer learning for wind power forecasting. Energy, 2024, 286. 129639

[33] Bai S, Kolter JZ, Koltun V (2018) An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint. https://doi.org/10.48550/arXiv.1803.01271

[34] Vaswani A, Shazeer N, Parmar N et al (2017) Attention is all you need. In: Proceedings of the 31st international conference on neural information processing systems, Long Beach, California, 4‒9 December, pp 6000‒6010

[35] YangQ, LiuY, ChenT, et al. . Federated machine learning: concept and applications. ACM Trans Intell Syst Technol, 2019, 1021-19.

[36] LiL, FanY, TseM, et al. . A review of applications in federated learning. Comput Ind Eng, 2020, 149. 106854

[37] ZhangC, XieY, BaiH, et al. . A survey on federated learning. Knowl Based Syst, 2021, 216. 106775

[38] ZhangZ, MingY, WangY. A federated transfer learning approach for surface electromyographic hand gesture recognition with emphasis on privacy preservation. Eng Appl Artif Intell, 2024, 136. 108952

[39] MoradzadehA, MoayyedH, Mohammadi-IvatlooB, et al. . A novel cyber-resilient solar power forecasting model based on secure federated deep learning and data visualization. Renew Energy, 2023, 211: 697-705.

[40] WangL, FanW, JiangG, et al. . An efficient federated transfer learning framework for collaborative monitoring of wind turbines in IoE-enabled wind farms. Energy, 2023, 284. 128518

[41] ZhangZ, WangY, RuanX, et al. . Lithium-ion batteries lifetime early prediction using domain adversarial learning. Renew Sustain Energy Rev, 2025, 208. 115035

[42] DingYData science for wind energy, 2024Boca RatonChapman & Hall/CRC Press

[43] ShinYE, DingY, HuangJZ. Covariate matching methods for testing and quantifying wind turbine upgrades. Ann Appl Stat, 2018, 1221271-1292.

[44] EzzatAA, JunM, DingY. Spatio-temporal asymmetry of local wind fields and its impact on short-term wind forecasting. IEEE Trans Sustain Energy, 2018, 931437-1447.

[45] Vaswani A, Bengio S, Brevdo E et al (2018) Tensor2tensor for neural machine translation. arXiv preprint. https://doi.org/10.48550/arXiv.1803.07416

RIGHTS & PERMISSIONS

Shanghai University and Periodicals Agency of Shanghai University and Springer-Verlag GmbH Germany, part of Springer Nature