Partial transfer learning network for data imputation and soft sensor under various operation conditions

Abstract

Soft sensor plays a key role in the safe operation of industrial processes and product quality control. In industrial processes, the switching of operation conditions may lead to distribution discrepancy and dimension inconsistency between the training data (source domain) and testing data (target domain), leading to the soft sensor model mismatch problem. In addition, the data may be incomplete because of sensor transmission failure, where the missing values may influence the accuracy of the soft sensor. This article introduces a partial transfer learning network (PTL-Net) for soft sensors under different operation conditions with missing data. First, the imputation and soft sensor modules are constructed for source domain data, where a compactness loss is designed to induce feature aggregation to alleviate the influence of abnormal features mapped from missing data. Then a partial transfer strategy is proposed to reduce the distribution discrepancy between the source and target data. Furthermore, the proposed strategy selects the common components between two domains for partial knowledge transfer rather than inheriting all the parameters directly, which can overcome the model mismatch problem. The effectiveness of PTL-Net is verified under the nuclear dataset and the three-phase flow process.

Keywords

partial transfer / various operation conditions / dimension inconsistency / missing data / soft sensor

References

[1] XieR-m, JanN M, HaoK-r, et al. . Supervised variational autoencoders for soft sensor modeling with missing data [J]. IEEE Transactions on Industrial Informatics, 2020, 16(4): 2820-2828

[2] ZhaoC-hui. Perspectives on nonstationary process monitoring in the era of industrial artificial intelligence [J]. Journal of Process Control, 2022, 116: 255-272

[3] DaiW, LiD-p, ChenQ-x, et al. . Data driven particle size estimation of hematite grinding process using stochastic configuration network with robust technique [J]. Journal of Central South University, 2019, 26: 43-62

[4] ChangS-c, ChenX, ZhaoC-hui. Flexible Clockwork Recurrent Neural Network for multirate industrial soft sensor [J]. Journal of Process Control, 2022, 119: 86-100

[5] AnitescuC, AtroshchenkoE, AlajlanN, et al. . Artificial neural network methods for the solution of second order boundary value problems [J]. Computers, Materials & Continua, 2019, 59(1): 345-359

[6] SamaniegoE, AnitescuC, GoswamiS, et al. . An energy approach to the solution of partial differential equations in computational mechanics via machine learning: Concepts, implementation and applications [J]. Computer Methods in Applied Mechanics and Engineering, 2020, 362: 112790

[7] WangT-s, ZuoH-yan. Fuzzy least squares support vector machine soft measurement model based on adaptive mutative scale chaos immune algorithm [J]. Journal of Central South University, 2014, 21(2): 593-599

[8] ShokriS, SadeghiM T, MarvastM A, et al. . Soft sensor design for hydrodesulfurization process using support vector regression based on WT and PCA [J]. Journal of Central South University, 2015, 22(2): 511-521

[9] WanK-y, DuB-x, WangJ-h, et al. . Deep learning-based intelligent management for sewage treatment plants [J]. Journal of Central South University, 2022, 29(5): 1537-1552

[10] CHEN Xu, ZHENG Jia-le, ZHAO Chun-hui, et al. Full decoupling high-order dynamic mode decomposition for advanced static and dynamic synergetic fault detection and isolation [J]. IEEE Transactions on Automation Science and Engineering, 2022. DOI: https://doi.org/10.1109/tase.2022.3218009.

[11] ShyamaM, PillaiA S. Fault tolerance strategies for wireless sensor networks-A comprehensive survey [C]. 2018 3rd International Conference on Inventive Computation Technologies (ICICT), 2020, Coimbatore, India, IEEE: 707-711

[12] ZhangS-chao. Nearest neighbor selection for iteratively kNN imputation [J]. Journal of Systems and Software, 2012, 85(11): 2541-2552

[13] SONG Peng-yu, ZHAO Chun-hui. Slow down to go better: A survey on slow feature analysis [J]. IEEE Transactions on Neural Networks and Learning Systems, 2022. DOI: https://doi.org/10.1109/TNNLS.2022.3201621.

[14] MccoyJ T, KroonS, AuretL. Variational autoencoders for missing data imputation with application to a simulated milling circuit [J]. IFAC-PapersOnLine, 2018, 51(21): 141-146

[15] YOON J, JORDAN J, SCHAAR M. Gain: Missing data imputation using generative adversarial nets [C]//Proceedings of the 35th International Conference on Machine Learning. Stockholm, Sweden, 2018: 5689–5698. https://proceedings.mlr.press/v80/yoon18a.html.

[16] ChaiZ, ZhaoC-h, HuangB, et al. . A deep probabilistic transfer learning framework for soft sensor modeling with missing data [J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(12): 7598-7609

[17] LiT-f, ZhaoZ-b, SunC, et al. . Domain adversarial graph convolutional network for fault diagnosis under variable working conditions [J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-10

[18] ZhangW, LiX, MaH, et al. . Transfer learning using deep representation regularization in remaining useful life prediction across operating conditions [J]. Reliability Engineering & System Safety, 2021, 211: 107556

[19] ChenX, ZhaoC-hui. Conditional discriminative autoencoder and condition-driven immediate representation of soft transition for monitoring complex nonstationary processes [J]. Control Engineering Practice, 2022, 122105090

[20] ZhangL, WangS-s, HuangG-b, et al. . Manifold criterion guided transfer learning via intermediate domain generation [J]. IEEE Transactions on Neural Networks and Learning Systems, 2019, 30(12): 3759-3773

[21] HsiaoY D, KangJ-l, WongD. Development of robust and physically interpretable soft sensor for industrial distillation column using transfer learning with small datasets [J]. Processes, 2021, 9(4): 667

[22] AraújoR L, de AraújoF H D, SilvaR R V E. Automatic segmentation of melanoma skin cancer using transfer learning and fine-tuning [J]. Multimedia Systems, 2022, 2841239-1250

[23] PanS J, TsangI W, KwokJ T, et al. . Domain adaptation via transfer component analysis [J]. IEEE Transactions on Neural Networks, 2010, 22(2): 199-210

[24] HSIEH Y T, TAO S Y, TSAI Y H H, et al. Recognizing heterogeneous cross-domain data via generalized joint distribution adaptation [C]//2016 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2016. DOI: https://doi.org/10.1109/icme.2016.7552878.

[25] ZhuX-f, ThungK H, AdeliE, et al. . Maximum mean discrepancy based multiple kernel learning for incomplete multimodality neuroimaging data [M]. Medical Image Computing and Computer Assisted Intervention–MICCAI 2017, 2017, Cham, Springer International Publishing: 72-80

[26] WallM E, RechtsteinerA, RochaL M. Singular value decomposition and principal component analysis [M]. A Practical Approach to Microarray Data Analysis, 2005, Boston, USA, Kluwer Academic Publishers: 91109

[27] WenL, GaoL, LiX-yu. A new deep transfer learning based on sparse auto-encoder for fault diagnosis [J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2019, 49(1): 136-144

[28] ChenX, ZhaoC-hui. Linear and nonlinear hierarchical multivariate time delay analytics for dynamic modeling and process monitoring [J]. Journal of Process Control, 2021, 107: 83-93

[29] WangX, LiuHan. Soft sensor based on stacked autoencoder deep neural network for air preheater rotor deformation prediction [J]. Advanced Engineering Informatics, 2018, 36: 112-119

[30] ChaiZ, WangJ-y, ZhaoC-h, et al. . Deep transfer learning methods for typical supervised tasks in industrial monitoring: State-of-the-art, challenges, and perspectives [J]. Sci Sin Inform, 2023, 53: 821-840 in Chinese)

[31] TanC, SunF, KongT, et al. . A survey on deep transfer learning [C]. Artificial Neural Networks and Machine Learning - ICANN 2018: 27th International Conference on Artificial Neural Networks, 2018, Rhodes, Greece, Springer International Publishing: 270279

[32] ShaoS-y, McaleerS, YanR-q, et al. . Highly accurate machine fault diagnosis using deep transfer learning [J]. IEEE Transactions on Industrial Informatics, 2019, 15(4): 2446-2455

[33] ChaiZ, ZhaoC-h, HuangBiao. Variational progressive-transfer network for soft sensing of multirate industrial processes [J]. IEEE Transactions on Cybernetics, 2022, 52(12): 12882-12892

[34] DongY-n, QinS J. A novel dynamic PCA algorithm for dynamic data modeling and process monitoring [J]. Journal of Process Control, 2018, 67: 1-11

[35] HewageP, BeheraA, TrovatiM, et al. . Temporal convolutional neural (TCN) network for an effective weather forecasting using time-series data from the local weather station [J]. Soft Computing, 2020, 24(21): 1-30

[36] HuangS-t, WangD-l, WuX-h, et al. . DSANet: Dual self-attention network for multivariate time series forecasting [C]. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 2019, New York, ACM: 21292132

[37] SatchlerG RIntroduction to nuclear reactions [M], 1990, London, UK, Palgrave Macmillan

[38] WangC-q, SunX-d, SabharwallP. CFD investigation of MHTGR natural circulation and decay heat removal in P-LOFC accident [J]. Frontiers in Energy Research, 2020, 8: 129

[39] StekhovenD, BühlmannP. Miss forest-non-parametric missing value imputation for mixed-type data [J]. Bioinformatics, 2012, 28(1): 112-118

[40] YuM, QuanT-q, PengQ-l, et al. . A model-based collaborate filtering algorithm based on stacked AutoEncoder [J]. Neural Computing and Applications, 2022, 34(4): 2503-2511

[41] Ruiz-CárcelC, CaoY, MbaD, et al. . Statistical process monitoring of a multiphase flow facility [J]. Control Engineering Practice, 2015, 4274-88

[42] CHEN Tian-qi, GUESTRIN C. XGBoost: A scalable tree boosting system [C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2016: https://doi.org/10.1145/2939672.2939785. DOI: https://doi.org/10.1145/2939672.2939785.

[43] LiuY, YangC, LiuK-x, et al. . Domain adaptation transfer learning soft sensor for product quality prediction [J]. Chemometrics and Intelligent Laboratory Systems, 2019, 192: 103813

[44] SongP-y, ZhaoC-h, HuangBiao. MPGE and RootRank: A sufficient root cause characterization and quantification framework for industrial process faults [J]. Neural Networks, 2023, 161397-417