A hybrid spatial-temporal deep learning prediction model of industrial methanol-to-olefins process

Abstract

Methanol-to-olefins, as a promising non-oil pathway for the synthesis of light olefins, has been successfully industrialized. The accurate prediction of process variables can yield significant benefits for advanced process control and optimization. The challenge of this task is underscored by the failure of traditional methods in capturing the complex characteristics of industrial processes, such as high nonlinearities, dynamics, and data distribution shift caused by diverse operating conditions. In this paper, we propose a novel hybrid spatial-temporal deep learning prediction model to address these issues. Firstly, a unique data normalization technique called reversible instance normalization is employed to solve the problem of different data distributions. Subsequently, convolutional neural network integrated with the self-attention mechanism are utilized to extract the temporal patterns. Meanwhile, a multi-graph convolutional network is leveraged to model the spatial interactions. Afterward, the extracted temporal and spatial features are fused as input into a fully connected neural network to complete the prediction. Finally, the outputs are denormalized to obtain the ultimate results. The monitoring results of the dynamic trends of process variables in an actual industrial methanol-to-olefins process demonstrate that our model not only achieves superior prediction performance but also can reveal complex spatial-temporal relationships using the learned attention matrices and adjacency matrices, making the model more interpretable. Lastly, this model is deployed onto an end-to-end Industrial Internet Platform, which achieves effective practical results.

Graphical abstract

Keywords

methanol-to-olefins / process variables prediction / spatial-temporal / self-attention mechanism / graph convolutional network

References

[1] Zhou J, Gao M, Zhang J, Liu W, Zhang T, Li H, Xu Z, Ye M, Liu Z. Directed transforming of coke to active intermediates in methanol-to-olefins catalyst to boost light olefins selectivity. Nature Communications, 2021, 12(1): 17

[2] Ye M, Tian P, Liu Z M. DMTO: a sustainable methanol-to-olefins technology. Engineering, 2021, 7(1): 17–21

[3] Li C Q, Chen Y Q, Shang Y L. A review of industrial big data for decision making in intelligent manufacturing. Engineering Science and Technology an International Journal, 2022, 29: 101021

[4] Pirdashti M, Curteanu S, Kamangar M H, Hassim M H, Khatami M A. Artificial neural networks: applications in chemical engineering. Reviews in Chemical Engineering, 2013, 29(4): 205–239

[5] Chiang L H, Braun B, Wang Z, Castillo I. Towards artificial intelligence at scale in the chemical industry. AIChE Journal, 2022, 68(6): e17644

[6] Zhu L T, Chen X Z, Ouyang B, Yan W C, Lei H, Chen Z, Luo Z H. Review of machine learning for hydrodynamics, transport, and reactions in multiphase flows and reactors. Industrial & Engineering Chemistry Research, 2022, 61(28): 9901–9949

[7] Wang Z Q, Wang L, Yuan Z H, Chen B Z. Data-driven optimal operation of the industrial methanol to olefin process based on relevance vector machine. Chinese Journal of Chemical Engineering, 2021, 34: 106–115

[8] Zhang H L, Zhu A Q, Xu J, Ge W. Gas-solid reactor optimization based on EMMS-DPM simulation and machine learning. Particuology, 2024, 89: 131–143

[9] Yao L, Ge Z Q. Big data quality prediction in the process industry: a distributed parallel modeling framework. Journal of Process Control, 2018, 68: 1–13

[10] Sun Q Q, Ge Z Q. A Survey on deep learning for data-driven soft sensors. IEEE Transactions on Industrial Informatics, 2021, 17(9): 5853–5866

[11] Yuan X F, Jia Z Z, Li L, Wang K, Ye L J, Wang Y L, Yang C H, Gui W H. A SIA-LSTM based virtual metrology for quality variables in irregular sampled time sequence of industrial processes. Chemical Engineering Science, 2022, 249: 117299

[12] Lee Y S, Chen J H. Developing semi-supervised latent dynamic variational autoencoders to enhance prediction performance of product quality. Chemical Engineering Science, 2023, 265: 118192

[13] Yang F, Sang Y S, Lv J C, Cao J. Prediction of gasoline yield in fluid catalytic cracking based on multiple level LSTM. Chemical Engineering Research & Design, 2022, 185: 119–129

[14] Li J C, Yang B, Li H G, Wang Y J, Qi C, Liu Y. DTDR–ALSTM: extracting dynamic time-delays to reconstruct multivariate data for improving attention-based LSTM industrial time series prediction models. Knowledge-Based Systems, 2021, 211: 106508

[15] Hao X, Huang G, Li Z, Zheng L, Zhao Y. A spatio-temporal data decoupling convolution network model for specific surface area prediction in cement grind process. ISA Transactions, 2023, 135: 380–397

[16] Zhao C H. Perspectives on nonstationary process monitoring in the era of industrial artificial intelligence. Journal of Process Control, 2022, 116: 255–272

[17] Jiang Y C, Yin S, Dong J W, Kaynak O. A review on soft sensors for monitoring, control, and optimization of industrial processes. IEEE Sensors Journal, 2021, 21(11): 12868–12881

[18] De Gooijer J G, Hyndman R J. 25 years of time series forecasting. International Journal of Forecasting, 2006, 22(3): 443–473

[19] Kuo Y H, Kusiak A. From data to big data in production research: the past and future trends. International Journal of Production Research, 2019, 57(15–16): 4828–4853

[20] KumarSHussain LBanarjeeSRezaM. Energy load forecasting using deep learning approach-LSTM and GRU in spark cluster. In: 2018 Fifth International Conference on Emerging Applications of Information Technology. New York: IEEE, 2018, 1–4

[21] Wang Y J, Ren Y M, Li H G. Symbolic multivariable hierarchical clustering based convolutional neural networks with applications in industrial process operating trend predictions. Industrial & Engineering Chemistry Research, 2020, 59(34): 15133–15145

[22] Yan F, Yang C J, Zhang X M. DSTED: a denoising spatial-temporal encoder-decoder framework for multistep prediction of burn-through point in sintering process. IEEE Transactions on Industrial Electronics, 2022, 69(10): 10735–10744

[23] Connor J T, Martin R D, Atlas L E. Recurrent neural networks and robust time series prediction. IEEE Transactions on Neural Networks, 1994, 5(2): 240–254

[24] Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation, 1997, 9(8): 1735–1780

[25] ChoKVan Merrienboer BGulcehreCBahdanauDBougaresF SchwenkHBengio Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv:1406.1078, 2014

[26] O’SheaKNashR. An introduction to convolutional neural networks. arXiv:1511.08458, 2015

[27] Wang Y J, Zhang Y C, Wu Z, Li H G, Christofides P D. Operational trend prediction and classification for chemical processes: a novel convolutional neural network method based on symbolic hierarchical clustering. Chemical Engineering Science, 2020, 225: 115796

[28] Zhou J, Cui G Q, Hu S D, Zhang Z Y, Yang C, Liu Z Y, Wang L F, Li C C, Sun M S. Graph neural networks: a review of methods and applications. AI Open, 2020, 1: 57–81

[29] BahdanauDCho KBengioY. Neural machine translation by jointly learning to align and translate. arXiv:1409.0473, 2014

[30] YinXHanY SunHXuZ YuHDuanX. A multivariate time series prediction schema based on multi-attention in recurrent neural network. In: 2020 IEEE Symposium on Computers and Communications (ISCC). New York: IEEE, 2020, 1–7

[31] Yang Y, Xiong Q, Wu C, Zou Q, Yu Y, Yi H, Gao M. A study on water quality prediction by a hybrid CNN-LSTM model with attention mechanism. Environmental Science and Pollution Research International, 2021, 28(39): 55129–55139

[32] VaswaniAShazeer NParmarNUszkoreitJJonesL GomezA NKaiser LPolosukhinI. Attention is all you need. In: Advances in Neural Information Processing Systems. New York: Curran Associates Inc., 2017

[33] FuX BGao FWuJWeiX YDuanF W. Spatiotemporal attention networks for wind power forecasting. In: 2019 International Conference on Data Mining Workshops. New York: IEEE, 2019, 149–154

[34] HuangS TWang D LWuXTangA. Dsanet: dual self-attention network for multivariate time series forecasting. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management. New York: Association for Computing Machinery, 2019, 2129–2132

[35] WuNGreenB BenXO’Banion S. Deep transformer models for time series forecasting: the influenza prevalence case. arXiv:2001.08317, 2020

[36] Scarselli F, Gori M, Tsoi A C, Hagenbuchner M, Monfardini G. The graph neural network model. IEEE Transactions on Neural Networks, 2009, 20(1): 61–80

[37] YuBYinH T ZhuZ X. Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. arXiv:1709.04875, 2017

[38] WuZ HPan S RLongG DJiangJZhangC Q. Graph wavenet for deep spatial-temporal graph modeling. arXiv:1906.00121, 2019

[39] LuBGanX Y JinH MFu L YZhangH S. Spatiotemporal adaptive gated graph convolution network for urban traffic flow forecasting. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management. New York: Association for Computing Machinery, 2020, 1025–1034

[40] Amornbunchornvej C, Zheleva E, Berger-Wolf T. Variable-lag granger causality and transfer entropy for time series analysis. ACM Transactions on Knowledge Discovery from Data, 2021, 15(4): 1–30

[41] XuH YHuang Y DDuanZ HFengJSongP Y. Multivariate time series forecasting based on causal inference with transfer entropy and graph neural network. arXiv:2005.01185, 2020

[42] He K W, Chen X, Wu Q, Yu S, Zhou Z. Graph attention spatial-temporal network with collaborative global-local learning for citywide mobile traffic prediction. IEEE Transactions on Mobile Computing, 2022, 21(4): 1244–1256

[43] WuZ HPan S RLongG DJiangJChangX J ZhangC Q. Connecting the dots: multivariate time series forecasting with graph neural networks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York: Association for Computing Machinery, 2020, 753–763

[44] KimTKimJ TaeYParkC ChoiJ HChoo J. Reversible instance normalization for accurate time-series forecasting against distribution shift. In: International Conference on Learning Representations, 2022

[45] JinG YXi Z XShaH YFengY HHuangJ C. Deep multi-view spatiotemporal virtual graph neural network for significant citywide ride-hailing demand prediction. arXiv:2007.15189, 2020

[46] Li D F, Lin K X, Li X T, Liao J B, Du R, Chen D Q, Madden A. Improved sales time series predictions using deep neural networks with spatiotemporal dynamic pattern acquisition mechanism. Information Processing & Management, 2022, 59(4): 102987

[47] ChaiDWang LYangQ. Bike flow prediction with multi-graph convolutional networks. In: Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. New York: Association for Computing Machinery, 2018, 397–400

[48] Troyanskaya O, Cantor M, Sherlock G, Brown P, Hastie T, Tibshirani R, Botstein D, Altman R B. Missing value estimation methods for DNA microarrays. Bioinformatics, 2001, 17(6): 520–525

[49] LaiG KChang W CYangY MLiuH X. Modeling long-and short-term temporal patterns with deep neural networks. In: The 41st international ACM SIGIR Conference on Research & Development in Information Retrieval. New York: Association for Computing Machinery, 2018, 95–104

[50] Fan J, Zhang K, Huang Y, Zhu Y, Chen B. Parallel spatio-temporal attention-based TCN for multivariate time series prediction. Neural Computing & Applications, 2023, 35(18): 13109–13118

Competing interests

The authors declare that they have no competing interests.

Acknowledgements

We thank the financial support from the National Natural Science Foundation of China (Grant No. 21991093), the Strategic Priority Research Program of Chinese Academy of Sciences (Grant No. XDA29050200), the Dalian Institute of Chemical Physics (DICP I202135), and the Energy Science and Technology Revolution Project (Grant No. E2010412).

RIGHTS & PERMISSIONS

2024 Higher Education Press