Enhanced bearing RUL prediction based on dynamic temporal attention and mixed MLP

Abstract

Bearings are critical components in machinery, and accurately predicting their remaining useful life (RUL) is essential for effective predictive maintenance. Traditional RUL prediction methods often rely on manual feature extraction and expert knowledge, which face specific challenges such as handling non-stationary data and avoiding overfitting due to the inclusion of numerous irrelevant features. This paper presents an approach that leverages Continuous Wavelet Transform (CWT) for feature extraction, a Channel-Temporal Mixed MLP (CT-MLP) layer for capturing intricate dependencies, and a dynamic attention mechanism to adjust its focus based on the temporal importance of features within the time series. The dynamic attention mechanism integrates multi-head attention with innovative enhancements, making it particularly effective for datasets exhibiting non-stationary behaviour. An experimental study using the XJTU-SY rolling bearings dataset and the PRONOSTIA bearing dataset revealed that the proposed deep learning algorithm significantly outperforms other state-of-the-art algorithms in terms of RMSE and MAE, demonstrating its robustness and accuracy.

References

[1.] Chen Y., et al.. A novel deep learning method based on attention mechanism for bearing remaining useful life prediction Appl. Soft Comput., 2020, 86. 105919(Available at)

[2.] Ji P., et al.. Deep learning prediction of Amplitude Death Auton. Intell. Syst., 2022, 2(1): 26-36.

[3.] Chen C., Wu D., Liu Y.. Recent advances of AI for Engineering Service and maintenance Auton. Intell. Syst., 2022, 2(1): 19-21.

[4.] Fu H., Liu Y.. A deep learning-based approach for electrical equipment remaining useful life prediction Auton. Intell. Syst., 2022, 2(1): 16-27.

[5.] Zhao M., Tang B., Tan Q.. Bearing remaining useful life estimation based on time–frequency representation and supervised dimensionality reduction Measurement, 2016, 86: 41-55.

[6.] Zhao B., Yuan Q.. A novel deep learning scheme for multi-condition remaining useful life prediction of rolling element bearings J. Manuf. Syst., 2021, 61: 450-460.

[7.] Wang H., et al.. Remaining useful life prediction of bearings based on convolution attention mechanism and temporal convolution network IEEE Access, 2023, 11: 24407-24419.

[8.] Tayade A., et al.. Remaining useful life (RUL) prediction of bearing by using regression model and principal component analysis (PCA) technique Vibroeng. Proc., 2019, 23: 30-36.

[9.] Li T., et al.. WaveletKernelNet: an interpretable deep neural network for industrial intelligent diagnosis IEEE Trans. Syst. Man Cybern. Syst., 2022, 52(4): 2302-2312.

[10.] Shah S.R., et al.. A sequence-to-sequence approach for remaining useful lifetime estimation using attention-augmented bidirectional LSTM Intell. Syst. Appl., 2021, 10–11. 200049

[11.] Liu Y., et al.. Degradation-trend-aware deep neural network with attention mechanism for bearing remaining useful life prediction IEEE Trans. Artif. Intell., 2024, 5(6): 2997-3011.

[12.] Zhao M., Tang B., Tan Q.. Bearing remaining useful life estimation based on time–frequency representation and supervised dimensionality reduction Measurement, 2016, 86: 41-55.

[13.] Ren L., et al.. Bearing remaining useful life prediction based on Deep Autoencoder and Deep Neural Networks J. Manuf. Syst., 2018, 48: 71-77.

[14.] Kamat P.V., Sugandhi R., Kumar S.. Deep learning-based anomaly-onset aware remaining useful life estimation of bearings PeerJ Comput. Sci., 2021, 7. e795

[15.] Liu H., et al.. Remaining useful life prediction using a novel feature-attention-based end-to-end approach IEEE Trans. Ind. Inform., 2021, 17(2): 1197-1207.

[16.] Wu Y., et al.. Remaining useful life estimation of Engineered Systems using Vanilla LSTM neural networks Neurocomputing, 2018, 275: 167-179.

[17.] Li X., Zhang W., Ding Q.. Deep learning-based remaining useful life estimation of bearings using multi-scale feature extraction Reliab. Eng. Syst. Saf., 2019, 182: 208-218. Available at)

[18.] Jin C., Chen X.. An end-to-end framework combining time–frequency expert knowledge and modified transformer networks for vibration signal classification Expert Syst. Appl., 2021, 171. 114570

[19.] Yang F., et al.. Waveform: graph enhanced wavelet learning for long sequence forecasting of multivariate time series Proc. AAAI Conf. Artif. Intell., 2023, 37(9): 10754-10761.

[20.] Hejazi S.Z., Packianather M., Liu Y.. A novel customised load adaptive framework for induction motor fault classification utilising MFPT bearing dataset Machines, 2024, 12(1. 44

[21.] Siddique M.F., et al.. A hybrid deep learning approach: integrating short-time Fourier transform and continuous wavelet transform for improved pipeline leak detection Sensors, 2023, 23 19. 8079

[22.] Shah S.R., et al.. A sequence-to-sequence approach for remaining useful lifetime estimation using attention-augmented bidirectional LSTM Intell. Syst. Appl., 2021, 10–11. 200049

[23.] Chen C., et al.. An integrated deep learning-based approach for automobile maintenance prediction with GIS Data Reliab. Eng. Syst. Saf., 2021, 216. 107919

[24.] Niazi S.G., et al.. Multi-scale time series analysis using TT-CONVLSTM technique for bearing remaining useful life prediction Mech. Syst. Signal Process., 2024, 206. 110888

[25.] Wang D., Liu K., Zhang X.. A generic indirect deep learning approach for multisensor degradation modeling IEEE Trans. Autom. Sci. Eng., 2022, 19 3): 1924-1940.

[26.] Liu J., et al.. Fault prediction of bearings based on LSTM and statistical process analysis Reliab. Eng. Syst. Saf., 2021, 214. 107646

[27.] Zhang J., et al.. Long short-term memory for machine remaining life prediction J. Manuf. Syst., 2018, 48: 78-86.

[28.] Zhu J., Chen N., Peng W.. Estimation of bearing remaining useful life based on multiscale convolutional Neural Network IEEE Trans. Ind. Electron., 2019, 66(4): 3208-3216.

[29.] Z. Li, et al., MTS-mixers: Multivariate Time Series forecasting via factorized temporal and channel mixing (2023). https://arxiv.org/abs/2302.04501 (Accessed: 28 July 2024)

[30.] Liu Z., et al.. A multi-head neural network with unsymmetrical constraints for remaining useful life prediction Adv. Eng. Inform., 2021, 50. 101396

[31.] Li X., et al.. Data alignments in machinery remaining useful life prediction using deep adversarial neural networks Knowl.-Based Syst., 2020, 197. 105843

[32.] Huang C.-G., et al.. A novel deep convolutional neural network-bootstrap integrated method for RUL prediction of rolling bearing J. Manuf. Syst., 2021, 61: 757-772.

[33.] Zou Y., et al.. A method for predicting the remaining useful life of rolling bearings under different working conditions based on multi-domain adversarial networks Measurement, 2022, 188. 110393

[34.] Mo Y., et al.. Remaining useful life estimation via transformer encoder enhanced by a gated convolutional unit J. Intell. Manuf., 2021, 32(7): 1997-2006.

[35.] Zeng F., et al.. A deep attention residual neural network-based remaining useful life prediction of machinery Measurement, 2021, 181. 109642

[36.] Yu X., Yu Z., Ramalingam S.. Learning strict identity mappings in deep residual networks 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018. Preprint])

[37.] Huang X., et al.. Frequency Hoyer attention based convolutional neural network for remaining useful life prediction of machinery Meas. Sci. Technol., 2021, 32(12. 125108

[38.] Jiang M., et al.. MLGN: multi-scale local-global feature learning network for long-term series forecasting Mach. Learn.: Sci. Technol., 2023, 4 4. 045059

[39.] G. Ioannides, A. Chadha, A. Elkins, Gaussian adaptive attention is all you need: Robust contextual representations across multiple modalities (2024). https://arxiv.org/abs/2401.11143 (Accessed: 13 June 2024)

[40.] Zhu L., et al.. Biformer: vision transformer with bi-level routing attention 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. Preprint])

[41.] Shen L., et al.. Improving the robustness of transformer-based large language models with dynamic attention Proceedings 2024 Network and Distributed System Security Symposium, 2024. Preprint])

[42.] Zhang Y., et al.. A free lunch from VIT: adaptive attention multi-scale fusion transformer for fine-grained visual recognition ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022. Preprint])

[43.] Fu S., et al.. MCA-DTCN: a novel dual-task temporal convolutional network with multi-channel attention for first prediction time detection and remaining useful life prediction Reliab. Eng. Syst. Saf., 2024, 241. 109696

[44.] Wang B., Lel Y., Li N., et al.. A hybrid proanostics approach for estimating remaining useful life of rolling element bearinas IEEE Trans. Reliab., 2020, 69 1): 401-412.

[45.] Nectoux P., Gouriveau R., Medjaher K., Ramasso E., Morello B., Zerhouni N., Varnier C.. PRONOSTIA: an experimental platform for bearings accelerated life test IEEE International Conference on Prognostics and Health Management, 2012