A MLP-Mixer and mixture of expert model for remaining useful life prediction of lithium-ion batteries

Abstract

Accurately predicting the Remaining Useful Life (RUL) of lithium-ion batteries is crucial for battery management systems. Deep learning-based methods have been shown to be effective in predicting RUL by leveraging battery capacity time series data. However, the representation learning of features such as long-distance sequence dependencies and mutations in capacity time series still needs to be improved. To address this challenge, this paper proposes a novel deep learning model, the MLP-Mixer and Mixture of Expert (MMMe) model, for RUL prediction. The MMMe model leverages the Gated Recurrent Unit and Multi-Head Attention mechanism to encode the sequential data of battery capacity to capture the temporal features and a re-zero MLP-Mixer model to capture the high-level features. Additionally, we devise an ensemble predictor based on a Mixture-of-Experts (MoE) architecture to generate reliable RUL predictions. The experimental results on public datasets demonstrate that our proposed model significantly outperforms other existing methods, providing more reliable and precise RUL predictions while also accurately tracking the capacity degradation process. Our code and dataset are available at the website of github.

Graphical abstract

Keywords

lithium-ion battery / remaining useful life / deep learning / MLP-Mixer / mixture-of-experts

Lingling Zhao is an associate professor at Faculty of Computing, Harbin Institute of Technology, China. She received the PhD, MS, and BS degrees from Harbin Institute of Technology, China. Her current research interests include machine learning

Shitao Song is currently a postgraduate student in the School of Electrical Engineering, Liaoning University of Technology, China. His research interest includes machine learning and micro-grid scheduling optimization

Pengyan Wang received his BE degree in Computer Science and Technology from Northeast Electric Power University, China in 2021. Currently, he is pursuing a ME degree in Computer Science and Technology at Northeast Electric Power University, China. His research interest includes lithium-ion battery safety

Chunyu Wang is a professor at the Faculty of Computing, Harbin Institute of Technology, China. He received his BS, MS, and PhD degrees in computer science and technology from Harbin Institute of Technology, China. His current research interests include bioinformatics and machine learning

Junjie Wang received the BS degree in Information management and information system from Institute of Disaster Prevention in 2013, the MS degree in software engineering in 2015, and the PhD degree in Computer science and technology in 2020 from the Harbin Institute of Technology, China. He is a Lecturer with the School of Biomedical Engineering and Informatics, Nanjing Medical University, China. His current research interests include bioinformatics and deep learning

Maozu Guo received the PhD degree from the School of Computer Science and Technology, Harbin Institute of Technology, China. He is currently a Professor with the School of Electrical and Information Engineering, Beijing University of Civil Engineering and Architecture, China. His current research interests include machine learning and artificial intelligence

References

[1] Tang X, Liu K, Li K, Widanage W D, Kendrick E, Gao F . Recovering large-scale battery aging dataset with machine learning. Patterns, 2021, 2( 8): 100302

[2] Wang Z, Liu N, Guo Y . Adaptive sliding window LSTM NN based RUL prediction for lithium-ion batteries integrating LTSA feature reconstruction. Neurocomputing, 2021, 466: 178–189

[3] Ge M F, Liu Y, Jiang X, Liu J . A review on state of health estimations and remaining useful life prognostics of lithium-ion batteries. Measurement, 2021, 174: 109057

[4] Rauf H, Khalid M, Arshad N . Machine learning in state of health and remaining useful life estimation: theoretical and technological development in battery degradation modelling. Renewable and Sustainable Energy Reviews, 2022, 156: 111903

[5] Zhai Q, Ye Z S . RUL prediction of deteriorating products using an adaptive wiener process model. IEEE Transactions on Industrial Informatics, 2017, 13( 6): 2911–2921

[6] Wang Y, Tian J, Sun Z, Wang L, Xu R, Li M, Chen Z . A comprehensive review of battery modeling and state estimation approaches for advanced battery management systems. Renewable and Sustainable Energy Reviews, 2020, 131: 110015

[7] Deng Z, Yang L, Deng H, Cai Y, Li D . Polynomial approximation pseudo-two-dimensional battery model for online application in embedded battery management system. Energy, 2018, 142: 838–850

[8] Yang L, Cai Y, Yang Y, Deng Z . Supervisory long-term prediction of state of available power for lithium-ion batteries in electric vehicles. Applied Energy, 2020, 257: 114006

[9] Son J, Zhou S, Sankavaram C, Du X, Zhang Y . Remaining useful life prediction based on noisy condition monitoring signals using constrained Kalman filter. Reliability Engineering & System Safety, 2016, 152: 38–50

[10] Su X, Wang S, Pecht M, Zhao L, Ye Z . Interacting multiple model particle filter for prognostics of lithium-ion batteries. Microelectronics Reliability, 2017, 70: 59–69

[11] Tian J, Xu R, Wang Y, Chen Z . Capacity attenuation mechanism modeling and health assessment of lithium-ion batteries. Energy, 2021, 221: 119682

[12] Li Y, Liu K, Foley A M, Zülke A, Berecibar M, Nanini-Maury E, Van Mierlo J, Hoster H E . Data-driven health estimation and lifetime prediction of lithium-ion batteries: a review. Renewable and Sustainable Energy Reviews, 2019, 113: 109254

[13] Li S, Fang H, Shi B . Remaining useful life estimation of lithium-ion battery based on interacting multiple model particle filter and support vector regression. Reliability Engineering & System Safety, 2021, 210: 107542

[14] Shu X, Li G, Shen J, Lei Z, Chen Z, Liu Y . A uniform estimation framework for state of health of lithium-ion batteries considering feature extraction and parameters optimization. Energy, 2020, 204: 117957

[15] Liu Z, Cheng Y, Wang P, Yu Y, Long Y . A method for remaining useful life prediction of crystal oscillators using the Bayesian approach and extreme learning machine under uncertainty. Neurocomputing, 2018, 305: 27–38

[16] Shen D, Wu L, Kang G, Guan Y, Peng Z . A novel online method for predicting the remaining useful life of lithium-ion batteries considering random variable discharge current. Energy, 2021, 218: 119490

[17] Yang B, Liu R, Zio E . Remaining useful life prediction based on a double-convolutional neural network architecture. IEEE Transactions on Industrial Electronics, 2019, 66( 12): 9521–9530

[18] Ding P, Liu X, Li H, Huang Z, Zhang K, Shao L, Abedinia O . Useful life prediction based on wavelet packet decomposition and two-dimensional convolutional neural network for lithium-ion batteries. Renewable and Sustainable Energy Reviews, 2021, 148: 111287

[19] Zhang Y, Xiong R, He H, Pecht M G . Long short-term memory recurrent neural network for remaining useful life prediction of lithium-ion batteries. IEEE Transactions on Vehicular Technology, 2018, 67( 7): 5695–5705

[20] Zhao S, Zhang C, Wang Y . Lithium-ion battery capacity and remaining useful life prediction using board learning system and long short-term memory neural network. Journal of Energy Storage, 2022, 52: 104901

[21] Chen D, Hong W, Zhou X . Transformer network for remaining useful life prediction of lithium-ion batteries. IEEE Access, 2022, 10: 19621–19628

[22] Zheng L, He Y, Chen X, Pu X . Optimization of dilated convolution networks with application in remaining useful life prediction of induction motors. Measurement, 2022, 200: 111588

[23] Ragab M, Chen Z, Wu M, Kwoh C K, Yan R, Li X . Attention-based sequence to sequence model for machine remaining useful life prediction. Neurocomputing, 2021, 466: 58–68

[24] Wu J Y, Wu M, Chen Z, Li X L, Yan R . Degradation-aware remaining useful life prediction with LSTM autoencoder. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1–10

[25] Jin R, Chen Z, Wu K, Wu M, Li X, Yan R . Bi-LSTM-based two-stream network for machine remaining useful life prediction. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 1–10

[26] Xiao L, Zhang L, Niu F, Su X, Song W . RETRACTED: remaining useful life prediction of wind turbine generator based on 1D-CNN and Bi-LSTM. International Journal of Fatigue, 2022, 163: 107051

[27] Rouhi Ardeshiri R, Ma C . Multivariate gated recurrent unit for battery remaining useful life prediction: a deep learning approach. International Journal of Energy Research, 2021, 45( 11): 16633–16648

[28] Chen Z, Wu M, Zhao R, Guretno F, Yan R, Li X . Machine remaining useful life prediction via an attention-based deep learning approach. IEEE Transactions on Industrial Electronics, 2021, 68( 3): 2521–2531

[29] Tolstikhin I O, Houlsby N, Kolesnikov A, Beyer L, Zhai X, Unterthiner T, Yung J, Steiner A, Keysers D, Uszkoreit J, Lucic M, Dosovitskiy A. MLP-Mixer: an all-MLP architecture for vision. In: Proceedings of the 35th International Conference on Neural Information Processing Systems. 2021

[30] Touvron H, Bojanowski P, Caron M, Cord M, El-Nouby A, Grave E, Izacard G, Joulin A, Synnaeve G, Verbeek J, Jegou H . ResMLP: feedforward networks for image classification with data-efficient training. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45( 4): 5314–5321

[31] Chen S, Xie E, Ge C, Chen R, Liang D, Luo P. CycleMLP: a MLP-like architecture for dense prediction. In: Proceedings of the 10th International Conference on Learning Representations. 2022

[32] Yu T, Li X, Cai Y, Sun M, Li P. S²-MLP: spatial-shift MLP architecture for vision. In: Proceedings of 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 2022, 3615−3624

[33] Jacobs R A, Jordan M I, Nowlan S J, Hinton G E . Adaptive mixtures of local experts. Neural Computation, 1991, 3( 1): 79–87

[34] Liu H, Liu Z, Jia W, Lin X . Remaining useful life prediction using a novel feature-attention-based end-to-end approach. IEEE Transactions on Industrial Informatics, 2021, 17( 2): 1197–1207

[35] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser Ł, Polosukhin I. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6000−6010

[36] Wu Y, Li W, Wang Y, Zhang K . Remaining useful life prediction of lithium-ion batteries using neural network and bat-based particle filter. IEEE Access, 2019, 7: 54843–54854

[37] Liu J, Saxena A, Goebel K, Saha B, Wang W. An adaptive recurrent neural network for remaining useful life prediction of lithium-ion batteries. In: Proceedings of Annual Conference of the Prognostics and Health Management Society. 2010

[38] Williard N, He W, Osterman M, Pecht M. Comparative analysis of features for determining state of health in lithium-ion batteries. International Journal of Prognostics and Health Management, 2013, 4(1): 1−7

[39] Shi Z, Chehade A . A dual-LSTM framework combining change point detection and remaining useful life prediction. Reliability Engineering & System Safety, 2021, 205: 107257

[40] Nagulapati V M, Lee H, Jung D, Brigljevic B, Choi Y, Lim H . Capacity estimation of batteries: influence of training dataset size and diversity on data driven prognostic models. Reliability Engineering & System Safety, 2021, 216: 108048

Acknowledgements

We are very grateful to the anonymous reviewers for their effort in evaluating our paper. This work was supported by the National Natural Science Foundation of China (Grant Nos. 62102191, 61872114, and 61871020).

Competing interests

The authors declare that they have no competing interests or financial conflicts to disclose.

Open Access

This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

RIGHTS & PERMISSIONS

2023 The Author(s) 2023. This article is published with open access at link.springer.com and journal.hep.com.cn