A hybrid spatial-temporal deep learning prediction model of industrial methanol-to-olefins process

Abstract

Methanol-to-olefins, as a promising non-oil pathway for the synthesis of light olefins, has been successfully industrialized. The accurate prediction of process variables can yield significant benefits for advanced process control and optimization. The challenge of this task is underscored by the failure of traditional methods in capturing the complex characteristics of industrial processes, such as high nonlinearities, dynamics, and data distribution shift caused by diverse operating conditions. In this paper, we propose a novel hybrid spatial-temporal deep learning prediction model to address these issues. Firstly, a unique data normalization technique called reversible instance normalization is employed to solve the problem of different data distributions. Subsequently, convolutional neural network integrated with the self-attention mechanism are utilized to extract the temporal patterns. Meanwhile, a multi-graph convolutional network is leveraged to model the spatial interactions. Afterward, the extracted temporal and spatial features are fused as input into a fully connected neural network to complete the prediction. Finally, the outputs are denormalized to obtain the ultimate results. The monitoring results of the dynamic trends of process variables in an actual industrial methanol-to-olefins process demonstrate that our model not only achieves superior prediction performance but also can reveal complex spatial-temporal relationships using the learned attention matrices and adjacency matrices, making the model more interpretable. Lastly, this model is deployed onto an end-to-end Industrial Internet Platform, which achieves effective practical results.

Key words： methanol-to-olefins; process variables prediction; spatial-temporal; self-attention mechanism; graph convolutional network

Competing interests

The authors declare that they have no competing interests.

Acknowledgements

We thank the financial support from the National Natural Science Foundation of China (Grant No. 21991093), the Strategic Priority Research Program of Chinese Academy of Sciences (Grant No. XDA29050200), the Dalian Institute of Chemical Physics (DICP I202135), and the Energy Science and Technology Revolution Project (Grant No. E2010412).

References

1 Zhou J, Gao M, Zhang J, Liu W, Zhang T, Li H, Xu Z, Ye M, Liu Z. Directed transforming of coke to active intermediates in methanol-to-olefins catalyst to boost light olefins selectivity. Nature Communications, 2021, 12(1): 17

2 Ye M, Tian P, Liu Z M. DMTO: a sustainable methanol-to-olefins technology. Engineering, 2021, 7(1): 17–21

3 Li C Q, Chen Y Q, Shang Y L. A review of industrial big data for decision making in intelligent manufacturing. Engineering Science and Technology an International Journal, 2022, 29: 101021

4 Pirdashti M, Curteanu S, Kamangar M H, Hassim M H, Khatami M A. Artificial neural networks: applications in chemical engineering. Reviews in Chemical Engineering, 2013, 29(4): 205–239

5 Chiang L H, Braun B, Wang Z, Castillo I. Towards artificial intelligence at scale in the chemical industry. AIChE Journal, 2022, 68(6): e17644

6 Zhu L T, Chen X Z, Ouyang B, Yan W C, Lei H, Chen Z, Luo Z H. Review of machine learning for hydrodynamics, transport, and reactions in multiphase flows and reactors. Industrial & Engineering Chemistry Research, 2022, 61(28): 9901–9949

7 Wang Z Q, Wang L, Yuan Z H, Chen B Z. Data-driven optimal operation of the industrial methanol to olefin process based on relevance vector machine. Chinese Journal of Chemical Engineering, 2021, 34: 106–115

8 Zhang H L, Zhu A Q, Xu J, Ge W. Gas-solid reactor optimization based on EMMS-DPM simulation and machine learning. Particuology, 2024, 89: 131–143

9 Yao L, Ge Z Q. Big data quality prediction in the process industry: a distributed parallel modeling framework. Journal of Process Control, 2018, 68: 1–13

10 Sun Q Q, Ge Z Q. A Survey on deep learning for data-driven soft sensors. IEEE Transactions on Industrial Informatics, 2021, 17(9): 5853–5866

11 Yuan X F, Jia Z Z, Li L, Wang K, Ye L J, Wang Y L, Yang C H, Gui W H. A SIA-LSTM based virtual metrology for quality variables in irregular sampled time sequence of industrial processes. Chemical Engineering Science, 2022, 249: 117299

12 Lee Y S, Chen J H. Developing semi-supervised latent dynamic variational autoencoders to enhance prediction performance of product quality. Chemical Engineering Science, 2023, 265: 118192

13 Yang F, Sang Y S, Lv J C, Cao J. Prediction of gasoline yield in fluid catalytic cracking based on multiple level LSTM. Chemical Engineering Research & Design, 2022, 185: 119–129

14 Li J C, Yang B, Li H G, Wang Y J, Qi C, Liu Y. DTDR–ALSTM: extracting dynamic time-delays to reconstruct multivariate data for improving attention-based LSTM industrial time series prediction models. Knowledge-Based Systems, 2021, 211: 106508

15 Hao X, Huang G, Li Z, Zheng L, Zhao Y. A spatio-temporal data decoupling convolution network model for specific surface area prediction in cement grind process. ISA Transactions, 2023, 135: 380–397

16 Zhao C H. Perspectives on nonstationary process monitoring in the era of industrial artificial intelligence. Journal of Process Control, 2022, 116: 255–272

17 Jiang Y C, Yin S, Dong J W, Kaynak O. A review on soft sensors for monitoring, control, and optimization of industrial processes. IEEE Sensors Journal, 2021, 21(11): 12868–12881

18 De Gooijer J G, Hyndman R J. 25 years of time series forecasting. International Journal of Forecasting, 2006, 22(3): 443–473

19 Kuo Y H, Kusiak A. From data to big data in production research: the past and future trends. International Journal of Production Research, 2019, 57(15–16): 4828–4853

20 KumarSHussain LBanarjeeSRezaM. Energy load forecasting using deep learning approach-LSTM and GRU in spark cluster. In: 2018 Fifth International Conference on Emerging Applications of Information Technology. New York: IEEE, 2018, 1–4

21 Wang Y J, Ren Y M, Li H G. Symbolic multivariable hierarchical clustering based convolutional neural networks with applications in industrial process operating trend predictions. Industrial & Engineering Chemistry Research, 2020, 59(34): 15133–15145

22 Yan F, Yang C J, Zhang X M. DSTED: a denoising spatial-temporal encoder-decoder framework for multistep prediction of burn-through point in sintering process. IEEE Transactions on Industrial Electronics, 2022, 69(10): 10735–10744

23 Connor J T, Martin R D, Atlas L E. Recurrent neural networks and robust time series prediction. IEEE Transactions on Neural Networks, 1994, 5(2): 240–254

24 Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation, 1997, 9(8): 1735–1780

25 ChoKVan Merrienboer BGulcehreCBahdanauDBougaresF SchwenkHBengio Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv:1406.1078, 2014

26 O’SheaKNashR. An introduction to convolutional neural networks. arXiv:1511.08458, 2015

27 Wang Y J, Zhang Y C, Wu Z, Li H G, Christofides P D. Operational trend prediction and classification for chemical processes: a novel convolutional neural network method based on symbolic hierarchical clustering. Chemical Engineering Science, 2020, 225: 115796

28 Zhou J, Cui G Q, Hu S D, Zhang Z Y, Yang C, Liu Z Y, Wang L F, Li C C, Sun M S. Graph neural networks: a review of methods and applications. AI Open, 2020, 1: 57–81

29 BahdanauDCho KBengioY. Neural machine translation by jointly learning to align and translate. arXiv:1409.0473, 2014

30 YinXHanY SunHXuZ YuHDuanX. A multivariate time series prediction schema based on multi-attention in recurrent neural network. In: 2020 IEEE Symposium on Computers and Communications (ISCC). New York: IEEE, 2020, 1–7

31 Yang Y, Xiong Q, Wu C, Zou Q, Yu Y, Yi H, Gao M. A study on water quality prediction by a hybrid CNN-LSTM model with attention mechanism. Environmental Science and Pollution Research International, 2021, 28(39): 55129–55139

32 VaswaniAShazeer NParmarNUszkoreitJJonesL GomezA NKaiser LPolosukhinI. Attention is all you need. In: Advances in Neural Information Processing Systems. New York: Curran Associates Inc., 2017

33 FuX BGao FWuJWeiX YDuanF W. Spatiotemporal attention networks for wind power forecasting. In: 2019 International Conference on Data Mining Workshops. New York: IEEE, 2019, 149–154

34 HuangS TWang D LWuXTangA. Dsanet: dual self-attention network for multivariate time series forecasting. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management. New York: Association for Computing Machinery, 2019, 2129–2132

35 WuNGreenB BenXO’Banion S. Deep transformer models for time series forecasting: the influenza prevalence case. arXiv:2001.08317, 2020

36 Scarselli F, Gori M, Tsoi A C, Hagenbuchner M, Monfardini G. The graph neural network model. IEEE Transactions on Neural Networks, 2009, 20(1): 61–80

37 YuBYinH T ZhuZ X. Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. arXiv:1709.04875, 2017

38 WuZ HPan S RLongG DJiangJZhangC Q. Graph wavenet for deep spatial-temporal graph modeling. arXiv:1906.00121, 2019

39 LuBGanX Y JinH MFu L YZhangH S. Spatiotemporal adaptive gated graph convolution network for urban traffic flow forecasting. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management. New York: Association for Computing Machinery, 2020, 1025–1034

40 Amornbunchornvej C, Zheleva E, Berger-Wolf T. Variable-lag granger causality and transfer entropy for time series analysis. ACM Transactions on Knowledge Discovery from Data, 2021, 15(4): 1–30

41 XuH YHuang Y DDuanZ HFengJSongP Y. Multivariate time series forecasting based on causal inference with transfer entropy and graph neural network. arXiv:2005.01185, 2020

42 He K W, Chen X, Wu Q, Yu S, Zhou Z. Graph attention spatial-temporal network with collaborative global-local learning for citywide mobile traffic prediction. IEEE Transactions on Mobile Computing, 2022, 21(4): 1244–1256

43 WuZ HPan S RLongG DJiangJChangX J ZhangC Q. Connecting the dots: multivariate time series forecasting with graph neural networks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York: Association for Computing Machinery, 2020, 753–763

44 KimTKimJ TaeYParkC ChoiJ HChoo J. Reversible instance normalization for accurate time-series forecasting against distribution shift. In: International Conference on Learning Representations, 2022

45 JinG YXi Z XShaH YFengY HHuangJ C. Deep multi-view spatiotemporal virtual graph neural network for significant citywide ride-hailing demand prediction. arXiv:2007.15189, 2020

46 Li D F, Lin K X, Li X T, Liao J B, Du R, Chen D Q, Madden A. Improved sales time series predictions using deep neural networks with spatiotemporal dynamic pattern acquisition mechanism. Information Processing & Management, 2022, 59(4): 102987

47 ChaiDWang LYangQ. Bike flow prediction with multi-graph convolutional networks. In: Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. New York: Association for Computing Machinery, 2018, 397–400

48 Troyanskaya O, Cantor M, Sherlock G, Brown P, Hastie T, Tibshirani R, Botstein D, Altman R B. Missing value estimation methods for DNA microarrays. Bioinformatics, 2001, 17(6): 520–525

49 LaiG KChang W CYangY MLiuH X. Modeling long-and short-term temporal patterns with deep neural networks. In: The 41st international ACM SIGIR Conference on Research & Development in Information Retrieval. New York: Association for Computing Machinery, 2018, 95–104

50 Fan J, Zhang K, Huang Y, Zhu Y, Chen B. Parallel spatio-temporal attention-based TCN for multivariate time series prediction. Neural Computing & Applications, 2023, 35(18): 13109–13118