Dynamic soft sensor model based on combination of GRU and TCN-Transformer for chemical process application

Abstract

Soft sensor technology has been widely applied in key areas of industrial process monitoring. To address challenges such as strong nonlinearity, complex temporal dependencies, and dynamic system behavior commonly encountered in industrial soft sensor data modeling, we propose a hybrid dynamic modeling method that integrates gated recurrent unit (GRU) with temporal convolutional network-Transformer (TCN-Transformer) architecture. TCN-Transformer module is employed to extract multi-scale temporal patterns and capture long-range dependencies among auxiliary variables, while GRU network processes the historical information of target variables through its gated memory mechanism. The complementary feature representations from both components are summed before being passed into a fully connected layer for prediction. To validate the effectiveness of GRU-TCN-Transformer framework, comprehensive case studies were conducted on two typical industrial processes: the prediction of butane (C₄) concentration in a debutanizer column and the estimation of hydrogen sulfide (H₂S) and sulfur dioxide (SO₂)concentrations in a sulfur recovery unit (SRU). Experimental results demonstrate that the proposed hybrid dynamic modeling method significantly outperforms traditional dynamic modeling methods—convolutional neural network (CNN), long short-term memory (LSTM), and TCN—across multiple evaluation metrics. Specifically, for C₄ concentration estimation, the proposed method reduced root mean squared error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) by 55.0%, 51.0% and 50.1%, respectively, and improved R² by 2.3% compared to the best-performing TCN-Transformer model. For H₂S estimation, it achieved reductions of 30%, 30.61% and 29.23% in RMSE, MAE, and MAPE, respectively, while increasing R² by 11.09% over the best LSTM-TCN-Transformer model. For SO₂ estimation, the proposed model reduced RMSE, MAE, and MAPE by 7.91%, 9.09% and 9.64%, respectively, with a 0.87% increase in R². These comparative results further confirm the improvements in prediction accuracy, indicating that the proposed model is capable of meeting the stringent requirements of industrial applications.

Keywords

soft sensor modelling / temporal convolutional network (TCN) / Transformer / gated recurrent unit (GRU) / dynamic model / chemical process

Acknowledgement

This work was financially supported by National Natural Science Foundation of China (No. 52467008), Key Project of Natural Science Foundation of Gansu Province (No.25JRRA150), Key Research and Development Planning Project of Gansu Province (No. 23YFWA0007), and Lanzhou Science and Technology Plan Project (No.2023-1-16).

Declaration of conflicting interests

The authors have no conflict of interests related to this publication.

References

[1] JIANG Y C,YIN S,DONG J W,et al. A review on soft sensors for monitoring, control, and optimization of industrial processes. IEEE Sensors Journal, 2021, 21(11): 12868-12881.

[2] MIETTINEN J, TIAINEN T,VIITALA R,et al. Bidirectional LSTM-based soft sensor for rotor displacement trajectory estimation. IEEE Access, 2021, 9: 167556-167569.

[3] SETHI S P,DAS D P,BEHERA S K. Monitoring of arc plasma process parameter using CNN-based deep learning algorithm to accommodate sensor failure. IEEE Transactions on Plasma Science, 2023, 51(6): 1434-1445.

[4] ZHANG Y Q,MA Y M,LIU Y H. Convolution-bidirectional temporal convolutional network for protein secondary structure prediction. IEEE Access, 2022, 10: 117469-117476.

[5] YE W,KUANG H X,DENG K X,et al. LGTCN: a spatial–temporal traffic flow prediction model based on local–global feature fusion temporal convolutional network. Applied Sciences, 2024, 14(19): 8847.

[6] TANG P F,DU P J,XIA J S,et al. Channel attention-based temporal convolutional network for satellite image time series classification. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 8016505.

[7] DONG X C,SUN Y Y,LI Y,et al. Spatio-temporal convolutional network based power forecasting of multiple wind farms. Journal of Modern Power Systems and Clean Energy, 2022, 10(2): 388-398.

[8] YUAN X F,QI S B,WANG Y L,et al. Quality variable prediction for nonlinear dynamic industrial processes based on temporal convolutional networks. IEEE Sensors Journal, 2021, 21(18): 20493-20503.

[9] ZHANG L,REN G F,LI S L,et al. A novel soft sensor approach for industrial quality prediction based TCN with spatial and temporal attention. Chemometrics and Intelligent Laboratory Systems, 2025, 257: 105272.

[10] TUO B B,ZHAO X Q,SUN K W,et al. Soft sensor model for nonlinear dynamic industrial process based on GraphSAGE-IMATCN. Process Safety and Environmental Protection, 2024, 191: 1131-1147.

[11] VASWANI A. Attention is all you need. Advances in Neural Information Processing Systems, 2017.

[12] TUCUDEAN G,BUCOS M,DRAGULESCU B,et al. Natural language processing with transformers: a review. PeerJ Computer Science, 2024, 10: e2222.

[13] CHEN D,LIU J,WEI G W. Multiscale topology-enabled structure-to-sequence transformer for protein-ligand interaction predictions. Nature Machine Intelligence, 2024, 6(7): 799-810.

[14] WEN Q,ZHOU T,ZHANG C,et al. Transformers in time series: A survey. arXiv preprint arXiv:2202.07125, 2022.

[15] LI Y,GE H W. General and robust voxel feature learning with Transformer for 3D object detection. Journal of Measurement Science and Instrumentation, 2022, 13(1): 51-60.

[16] FANG Z Y,GAO S W,DANG X C,et al. Transformer enhanced by local perception self-attention for dynamic soft sensor modeling of industrial processes. Measurement Science and Technology, 2024, 35(5): 055123.

[17] JI S P,MENG Y L,YAN L,et al. GRU-corr neural network optimized by improved PSO algorithm for time series prediction. International Journal on Artificial Intelligence Tools, 2020, 29(7n08): 2040010.

[18] CAO J F,XUE A K,YANG Y,et al. Deep learning based soft sensor for microbial wastewater treatment efficiency prediction. Journal of Water Process Engineering, 2023, 56: 104259.

[19] LIU J P,HE J Z,TANG Z H,et al. Frame-dilated convolutional fusion network and GRU-based self-attention dual-channel network for soft-sensor modeling of industrial process quality indexes. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2022, 52(9): 5989-6002.

[20] XIE R M,HAO K R,HUANG B,et al. Data-driven modeling based on two-stream λ gated recurrent unit network with soft sensor application. IEEE Transactions on Industrial Electronics, 2020, 67(8): 7034-7043.

[21] FORTUNA L,RIZZO A,SINATRA M,et al. Soft analyzers for a sulfur recovery unit. Control Engineering Practice, 2003, 11(12): 1491-1500.

[22] YUAN X F,WANG Y C,WANG C,et al. Variable correlation analysis-based convolutional neural network for far topological feature extraction and industrial predictive modeling. IEEE Transactions on Instrumentation and Measurement, 2024, 73: 3001110.

[23] LUI C F,LIU Y Q,XIE M. A supervised bidirectional long short-term memory network for data-driven dynamic soft sensor modeling. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 2504713.

[24] ZHANG Z X,YANG X,HUANG J,et al. Layer-wise-residual-driven approach for soft sensing in composite dynamic system based on slow and fast time-varying latent variables. Chemometrics and Intelligent Laboratory Systems, 2024, 254: 105245.

[25] MOU M,ZHAO X Q. Gated broad learning system based on deep cascaded for soft sensor modeling of industrial process. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 2508811.

[26] YUAN X F,WANG Y L,YANG C H,et al. Stacked isomorphic autoencoder based soft analyzer and its application to sulfur recovery unit. Information Sciences, 2020, 534: 72-84.

[27] MOU T H,LIU J F,ZOU Y Y,et al. Enhanced industrial process modeling with transfer-incremental-learning: a parallel SAE approach and its application to a sulfur recovery unit. Control Engineering Practice, 2024, 148: 105955.