A new automatic convolutional neural network based on deep reinforcement learning for fault diagnosis

Long WEN, You WANG, Xinyu LI

PDF(2174 KB)
PDF(2174 KB)
Front. Mech. Eng. ›› 2022, Vol. 17 ›› Issue (2) : 17. DOI: 10.1007/s11465-022-0673-7
RESEARCH ARTICLE

A new automatic convolutional neural network based on deep reinforcement learning for fault diagnosis

Author information +
History +

Abstract

Convolutional neural network (CNN) has achieved remarkable applications in fault diagnosis. However, the tuning aiming at obtaining the well-trained CNN model is mainly manual search. Tuning requires considerable experiences on the knowledge on CNN training and fault diagnosis, and is always time consuming and labor intensive, making the automatic hyper parameter optimization (HPO) of CNN models essential. To solve this problem, this paper proposes a novel automatic CNN (ACNN) for fault diagnosis, which can automatically tune its three key hyper parameters, namely, learning rate, batch size, and L2-regulation. First, a new deep reinforcement learning (DRL) is developed, and it constructs an agent aiming at controlling these three hyper parameters along with the training of CNN models online. Second, a new structure of DRL is designed by combining deep deterministic policy gradient and long short-term memory, which takes the training loss of CNN models as its input and can output the adjustment on these three hyper parameters. Third, a new training method for ACNN is designed to enhance its stability. Two famous bearing datasets are selected to evaluate the performance of ACNN. It is compared with four commonly used HPO methods, namely, random search, Bayesian optimization, tree Parzen estimator, and sequential model-based algorithm configuration. ACNN is also compared with other published machine learning (ML) and deep learning (DL) methods. The results show that ACNN outperforms these HPO and ML/DL methods, validating its potential in fault diagnosis.

Graphical abstract

Keywords

deep reinforcement learning / hyper parameter optimization / convolutional neural network / fault diagnosis

Cite this article

Download citation ▾
Long WEN, You WANG, Xinyu LI. A new automatic convolutional neural network based on deep reinforcement learning for fault diagnosis. Front. Mech. Eng., 2022, 17(2): 17 https://doi.org/10.1007/s11465-022-0673-7

References

[1]
Zhang X , Huang T , Wu B , Hu Y M , Huang S , Zhou Q , Zhang X . Multi-model ensemble deep learning method for intelligent fault diagnosis with high-dimensional samples. Frontiers of Mechanical Engineering, 2021, 16( 2): 340– 352
CrossRef Google scholar
[2]
Chen X F , Wang S B , Qiao B J , Chen Q . Basic research on machinery fault diagnostics: past, present, and future trends. Frontiers of Mechanical Engineering, 2018, 13( 2): 264– 291
CrossRef Google scholar
[3]
Lei Y G , Yang B , Jiang X W , Jia F , Li N P , Nandi A K . Applications of machine learning to machine fault diagnosis: a review and roadmap. Mechanical Systems and Signal Processing, 2020, 138 : 106587
CrossRef Google scholar
[4]
Nath A G , Udmale S S , Singh S K . Role of artificial intelligence in rotor fault diagnosis: a comprehensive review. Artificial Intelligence Review, 2021, 54 : 2609– 2668
CrossRef Google scholar
[5]
Wang J L , Xu C Q , Dai L , Zhang J , Zhong R Y . An unequal deep learning approach for 3-D point cloud segmentation. IEEE Transactions on Industrial Informatics, 2021, 17( 12): 7913– 7922
CrossRef Google scholar
[6]
Chen Z Y , Mauricio A , Li W H , Gryllias K . A deep learning method for bearing fault diagnosis based on cyclic spectral coherence and convolutional neural networks. Mechanical Systems and Signal Processing, 2020, 140 : 106683
CrossRef Google scholar
[7]
Wolpert D H , Macready W G . No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 1997, 1( 1): 67– 82
CrossRef Google scholar
[8]
Wolpert D H . The supervised learning no-free-lunch theorems. In: Roy R, Köppen M, Ovaska S, Furuhashi T, Hoffmann F, eds. Soft Computing and Industry. London: Springer, 2002, 25– 42
CrossRef Google scholar
[9]
Hutter F Kotthoff L Vanschoren J. Automated Machine Learning: Methods, Systems, Challenges. Cham: Springer, 2019
[10]
Wen L , Li X Y , Gao L . A new reinforcement learning based learning rate scheduler for convolutional neural network in fault classification. IEEE Transactions on Industrial Electronics, 2021, 68( 12): 12890– 12900
CrossRef Google scholar
[11]
Wen L , Ye X C , Gao L . A new automatic machine learning based hyperparameter optimization for workpiece quality prediction. Measurement and Control, 2020, 53( 7−8): 1088– 1098
CrossRef Google scholar
[12]
Feurer M Eggensperger K Falkner S Lindauer M Hutter F. Practical automated machine learning for the AutoML challenge 2018. In: Proceedings of International Workshop on Automatic Machine Learning at ICML. 2018, 1189‒ 1232
[13]
He F X , Liu T L , Tao D C . Control batch size and learning rate to generalize well: theoretical and empirical evidence. In: Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS). Vancouver, 2019, 1143– 1152
[14]
Li Y Z, Wei C, Ma T Y. Towards explaining the regularization effect of initial large learning rate in training neural networks. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. Vancouver, 2019, 11674–11685
[15]
Zhou P , Peng Z K , Chen S Q , Yang Y , Zhang W M . Non-stationary signal analysis based on general parameterized time-frequency transform and its application in the feature extraction of a rotary machine. Frontiers of Mechanical Engineering, 2018, 13( 2): 292– 300
CrossRef Google scholar
[16]
Wang J L , Xu C Q , Yang Z L , Zhang J , Li X O . Deformable convolutional networks for efficient mixed-type wafer defect pattern recognition. IEEE Transactions on Semiconductor Manufacturing, 2020, 33( 4): 587– 596
CrossRef Google scholar
[17]
Xu G W , Liu M , Jiang Z F , Shen W M , Huang C X . Online fault diagnosis method based on transfer convolutional neural networks. IEEE Transactions on Instrumentation and Measurement, 2020, 69( 2): 509– 520
CrossRef Google scholar
[18]
Li Z X, Zheng T S, Wang Y, Cao Z, Guo Z Q, Fu H Y. A novel method for imbalanced fault diagnosis of rotating machinery based on generative adversarial networks. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 3500417
[19]
Chen J B, Huang R Y, Zhao K, Wang W, Liu L C, Li W H. Multiscale convolutional neural network with feature alignment for bearing fault diagnosis. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 3517010
[20]
Jiao J Y , Zhao M , Lin J , Liang K X . A comprehensive review on convolutional neural network in machine fault diagnosis. Neurocomputing, 2020, 417 : 36– 63
CrossRef Google scholar
[21]
Yao Y , Zhang S , Yang S X , Gui G . Learning attention representation with a multi-scale CNN for gear fault diagnosis under different working conditions. Sensors, 2020, 20( 4): 1233
CrossRef Google scholar
[22]
Li S , Wang H Q , Song L Y , Wang P X , Cui L L , Lin T J . An adaptive data fusion strategy for fault diagnosis based on the convolutional neural network. Measurement, 2020, 165 : 108122
CrossRef Google scholar
[23]
Kolar D , Lisjak D , Pająk M , Pavković D . Fault diagnosis of rotary machines using deep convolutional neural network with wide three axis vibration signal input. Sensors, 2020, 20( 14): 4017
CrossRef Google scholar
[24]
Wang R X , Jiang H K , Li X Q , Liu S W . A reinforcement neural architecture search method for rolling bearing fault diagnosis. Measurement, 2020, 154 : 107417
CrossRef Google scholar
[25]
Zhang K Y , Chen J L , He S L , Xu E Y , Li F D , Zhou Z T . Differentiable neural architecture search augmented with pruning and multi-objective optimization for time-efficient intelligent fault diagnosis of machinery. Mechanical Systems and Signal Processing, 2021, 158 : 107773
CrossRef Google scholar
[26]
Cabrera D , Guamán A , Zhang S H , Cerrada M , Sánchez R V , Cevallos J , Long J Y , Li C . Bayesian approach and time series dimensionality reduction to LSTM-based model-building for fault diagnosis of a reciprocating compressor. Neurocomputing, 2020, 380 : 51– 66
CrossRef Google scholar
[27]
Li L , Jamieson K , DeSalvo G , Rostamizadeh A , Talwalkar A . Hyperband: a novel bandit-based approach to hyperparameter optimization. The Journal of Machine Learning Research, 2018, 18( 1): 6765– 6816
CrossRef Google scholar
[28]
Li H , Zhang Q , Qin X R , Sun Y T . Raw vibration signal pattern recognition with automatic hyper-parameter-optimized convolutional neural network for bearing fault diagnosis. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science, 2020, 234( 1): 343– 360
CrossRef Google scholar
[29]
Long J Y , Zhang S H , Li C . Evolving deep echo state networks for intelligent fault diagnosis. IEEE Transactions on Industrial Informatics, 2020, 16( 7): 4928– 4937
CrossRef Google scholar
[30]
Han J H , Choi D J , Park S U , Hong S K . Hyperparameter optimization using a genetic algorithm considering verification time in a convolutional neural network. Journal of Electrical Engineering & Technology, 2020, 15( 2): 721– 726
CrossRef Google scholar
[31]
Wei J A , Huang H S , Yao L G , Hu Y , Fan Q S , Huang D . New imbalanced fault diagnosis framework based on cluster-MWMOTE and MFO-optimized LS-SVM using limited and complex bearing data. Engineering Applications of Artificial Intelligence, 2020, 96 : 103966
CrossRef Google scholar
[32]
Hansen S. Using deep Q-learning to control optimization hyperparameters. 2016, arXiv:1602.04062
[33]
Zhang Z Z , Chen J L , Chen Z B , Li W P . Asynchronous episodic deep deterministic policy gradient: toward continuous control in computationally complex environments. IEEE Transactions on Cybernetics, 2021, 51( 2): 604– 613
CrossRef Google scholar
[34]
Zhu Z Y , Peng G L , Chen Y H , Gao H J . A convolutional neural network based on a capsule network with strong generalization for bearing fault diagnosis. Neurocomputing, 2019, 323 : 62– 75
CrossRef Google scholar
[35]
Wang Y , Ning D J , Feng S L . A novel capsule network based on wide convolution and multi-scale convolution for fault diagnosis. Applied Sciences, 2020, 10( 10): 3659
CrossRef Google scholar
[36]
Wen L , Li X Y , Gao L . A new two-level hierarchical diagnosis network based on convolutional neural network. IEEE Transactions on Instrumentation and Measurement, 2020, 69( 2): 330– 338
CrossRef Google scholar
[37]
Zhang W , Peng G L , Li C H , Chen Y H , Zhang Z J . A new deep learning model for fault diagnosis with good anti-noise and domain adaptation ability on raw vibration signals. Sensors, 2017, 17( 2): 425
CrossRef Google scholar
[38]
Hoang D T , Kang H J . Rolling element bearing fault diagnosis using convolutional neural network and vibration image. Cognitive Systems Research, 2019, 53 : 42– 50
CrossRef Google scholar
[39]
Jiang G Q , He H B , Yan J , Xie P . Multiscale convolutional neural networks for fault diagnosis of wind turbine gearbox. IEEE Transactions on Industrial Electronics, 2019, 66( 4): 3196– 3207
CrossRef Google scholar
[40]
Qiao H H , Wang T Y , Wang P , Zhang L , Xu M D . An adaptive weighted multiscale convolutional neural network for rotating machinery fault diagnosis under variable operating conditions. IEEE Access: Practical Innovations, Open Solutions, 2019, 7 : 118954– 118964
CrossRef Google scholar
[41]
Song Y , Li Y B , Jia L , Qiu M K . Retraining strategy-based domain adaption network for intelligent fault diagnosis. IEEE Transactions on Industrial Informatics, 2020, 16( 9): 6163– 6171
CrossRef Google scholar

Nomenclature

Abbreviations
ACNN Automatic convolutional neural network
ANN Artificial neural network
AVG Average prediction accuracy
AWMS-CNN Adaptive weighted multiscale convolutional neural network
BO Bayesian optimization
CNN Convolutional neural network
DAN Domain adaption network
DDPG Deep deterministic policy gradient
DL Deep learning
DRL Deep reinforcement learning
ELM Extreme learning machine
FC Fully connected
GS Grid search
HCNN Hierarchical convolutional neural network
HPO Hyper parameter optimization
ICN CNN based on a capsule network with an inception block
IF Inner race fault
LSTM Long short-term memory
ML Machine learning
MSCNN Multiscale convolutional neural network
NAS Neural architecture search
OF Outer race fault
RF Roller fault
RL Reinforcement learning
RS Random search
SMAC Sequential model-based algorithm configuration
SVM Support vector machine
TPE Tree Parzen estimator
VI-CNN CNN based on vibration image
WDCNN Deep convolutional neural networks with wide first-layer kernels
WMSCCN Wide convolution and multiscale convolution
Variables
a Action of DDPG algorithm
at Current action
bt Current batch size
bmax, bmin The upper and lower boundaries for batch size
EΠ Expected value under policy Π
f Fourier frequency in short-time Fourier transform
lt Current L2-regulation value
lmax, lmin The upper and lower boundaries for L2-regulation value
lrmax, lrmin The upper and lower boundaries for learning rate, respectivley
lrt Current learning rate
losst Loss value of the CNN model at time step t
L Training loss of critic network
M Number of the sequencing training loss
n Number of samples in the experience storage D
p Transition probability function
QΠ (s, a) Q-value function under policy Π
r Reward of DDPG algorithm
s State of DDPG algorithm
st State at time step t
STFT Short-time Fourier transform formulation
t Time step
w(t) Window function
yt Actual Q-value
α Factor to control the degree of soft updating
γ Discount factor
Π Policy of the agent to choose the action
θμ Online actor network
θμ Target actor network
ωQ Online critic network
ωQ Target critic network

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (Grant Nos. 51805192 and U21B2029), the Major Special Science and Technology Project of Hubei Province, China (Grant No. 2020A EA009), and the State Key Laboratory of Digital Manufacturing Equipment and Technology of Huazhong University of Science and Technology, China (Grant No. DMETKF2020029).

RIGHTS & PERMISSIONS

2022 Higher Education Press 2022
AI Summary AI Mindmap
PDF(2174 KB)

Accesses

Citations

Detail

Sections
Recommended

/