Data Analytics and Machine Learning for Smart Process Manufacturing: Recent Advances and Perspectives in the Big Data Era

Abstract

Safe, efficient, and sustainable operations and control are primary objectives in industrial manufacturing processes. State-of-the-art technologies heavily rely on human intervention, thereby showing apparent limitations in practice. The burgeoning era of big data is influencing the process industries tremendously, providing unprecedented opportunities to achieve smart manufacturing. This kind of manufacturing requires machines to not only be capable of relieving humans from intensive physical work, but also be effective in taking on intellectual labor and even producing innovations on their own. To attain this goal, data analytics and machine learning are indispensable. In this paper, we review recent advances in data analytics and machine learning applied to the monitoring, control, and optimization of industrial processes, paying particular attention to the interpretability and functionality of machine learning models. By analyzing the gap between practical requirements and the current research status, promising future research directions are identified.

Keywords

Big data / Machine learning / Smart manufacturing / Process systems engineering

References

[[1]] Willis MJ, Di Massimo CD, Montague GA, Tham MT, Morris AJ. Artificial neural networks in process engineering. IEE Proc Contr TheorAppl 1991;138 (3):256–66.

[[2]] Willis MJ, Montague GA, Di Massimo C, Tham MT, Morris AJ. Artificial neural networks in process estimation and control. Automatica 1992;28(6):1181–7.

[[3]] MacGregor J, Cinar A. Monitoring, fault diagnosis, fault-tolerant control and optimization: data driven methods. Comput Chem Eng 2012;47:111–20.

[[4]] Pillonetto G, Dinuzzo F, Chen T, De Nicolao G, Ljung L. Kernel methods in system identification, machine learning and function estimation: a survey. Automatica 2014;50(3):657–82.

[[5]] Bengio Y, Courville A, Vincent P. Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell 2013;35(8):1798–828.

[[6]] Nair V, Hinton GE. Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning; 2010 Jun 21–25; Haifa, Israel; 2010. p. 807–14.

[[7]] Bengio Y, Lamblin P, Popovici D, Larochelle H. Greedy layer-wise training of deep networks. In: Schölkopf B, Platt J, Hofmann T, editors. Advances in neural information processing systems 19: Proceedings of the 2006 Conference. Cambridge: The MIT Press; 2007. p. 153–60.

[[8]] Shu Y, Ming L, Cheng F, Zhang Z, Zhao J. Abnormal situation management: challenges and opportunities in the big data era. Comput Chem Eng 2016;91:104–13.

[[9]] Chiang L, Lu B, Castillo I. Big data analytics in chemical engineering. Annu Rev Chem Biomol Eng 2017;8:63–85.

[[10]] Ge Z, Song Z, Ding SX, Huang B. Data mining and analytics in the process industry: the role of machine learning. IEEE Access 2017;5:20590–616.

[[11]] Shang C, Huang B, Yang F, Huang D. Probabilistic slow feature analysis-based representation learning from massive process data for soft sensor modeling. AIChE J 2015;61(12):4126–39.

[[12]] Shang C, Yang F, Gao X, Huang X, Suykens JAK, Huang D. Concurrent monitoring of operating condition deviations and process dynamics anomalies with slow feature analysis. AIChE J 2015;61(11):3666–82.

[[13]] Shang C, Huang B, Yang F, Huang D. Slow feature analysis for monitoring and diagnosis of control performance. J Process Contr 2016;39:21–34.

[[14]] Shang C, Yang F, Huang B, Huang D. Recursive slow feature analysis for adaptive monitoring of industrial processes. IEEE Trans Ind Electron 2018;65 (11):8895–905.

[[15]] Guo F, Shang C, Huang B, Wang K, Yang F, Huang D. Monitoring of operating point and process dynamics via probabilistic slow feature analysis. Chemom Intell Lab Syst 2016;151:115–25.

[[16]] Gao X, Li H, Wang Y, Chen T, Zuo X, Zhong L. Fault detection in managed pressure drilling using slow feature analysis. IEEE Access 2018;6:34262–71.

[[17]] Zhang H, Tian X, Deng X. Batch process monitoring based on multiway global preserving kernel slow feature analysis. IEEE Access 2017;5:2696–710.

[[18]] Zhang S, Zhao C. Slow-feature-analysis-based batch process monitoring with comprehensive interpretation of operation condition deviation and dynamic anomaly. IEEE Trans Ind Electron 2019;66(5):3773–83.

[[19]] Zhang H, Tian X, Deng X, Cao Y. Batch process fault detection and identification based on discriminant global preserving kernel slow feature analysis. ISA Trans 2018;79:108–26.

[[20]] Dong Y, Qin SJ. A novel dynamic PCA algorithm for dynamic data modeling and process monitoring. J Process Contr 2018;67:1–11.

[[21]] Dong Y, Qin SJ. Dynamic latent variable analytics for process operations and control. Comput Chem Eng 2018;114:69–80.

[[22]] Ku W, Storer RH, Georgakis C. Disturbance detection and isolation by dynamic principal component analysis. Chemom Intell Lab Syst 1995;30(1):179–96.

[[23]] Lee JM, Yoo C, Lee IB. Statistical monitoring of dynamic processes based on dynamic independent component analysis. Chem Eng Sci 2004;59 (14):2995–3006.

[[24]] Yu J, Qin SJ. Multimode process monitoring with Bayesian inference-based finite Gaussian mixture models. AIChE J 2008;54(7):1811–29.

[[25]] Wang F, Tan S, Shi H. Hidden Markov model-based approach for multimode process monitoring. Chemom Intell Lab Syst 2015;148:51–9.

[[26]] Bai X, Lu G, Hossain MM, Szuhánszki J, Daood SS, Nimmo W, et al. Multi-mode combustion process monitoring on a pulverised fuel combustion test facility based on flame imaging and random weight network techniques. Fuel 2017;202:656–64.

[[27]] He QP, Wang J. Statistical process monitoring as a big data analytics tool for smart manufacturing. J Process Contr 2018;67:35–43.

[[28]] Brosilow C, Tong M. Inferential control of processes: part II. the structure and dynamics of inferential control systems. AIChE J 1978;24(3):492–500.

[[29]] Shardt YAW, Hao H, Ding SX. A new soft-sensor-based process monitoring scheme incorporating infrequent KPI measurements. IEEE Trans Ind Electron 2015;62(6):3843–51.

[[30]] Kadlec P, Gabrys B, Strandt S. Data-driven soft sensors in the process industry. Comput Chem Eng 2009;33(4):795–814.

[[31]] Ma Y, Huang B. Bayesian learning for dynamic feature extraction with application in soft sensing. IEEE Trans Ind Electron 2017;64(9):7171–80.

[[32]] Ma Y, Huang B. Extracting dynamic features with switching models for process data analytics and application in soft sensing. AIChE J 2018;64 (6):2037–51.

[[33]] Zhong W, Jiang C, Peng X, Li Z, Qian F. Online quality prediction of industrial terephthalic acid hydropurification process using modified regularized slowfeature analysis. Ind Eng Chem Res 2018;57(29):9604–14.

[[34]] Shang C, Yang F, Huang D, Lyu W. Data-driven soft sensor development based on deep learning technique. J Process Contr 2014;24(3):223–33.

[[35]] Gao X, Shang C, Jiang Y, Huang D, Chen T. Refinery scheduling with varying crude: a deep belief network classification and multimodel approach. AIChE J 2014;60(7):2525–32.

[[36]] Li F, Zhang J, Shang C, Huang D, Oko E, Wang M. Modelling of a postcombustion CO2 capture process using deep belief network. Appl Therm Eng 2018;130:997–1003.

[[37]] Zhang Z, Zhao J. A deep belief network based fault diagnosis model for complex chemical processes. Comput Chem Eng 2017;107:395–407.

[[38]] Wu H, Zhao J. Deep convolutional neural network model based chemical process fault diagnosis. Comput Chem Eng 2018;115:185–97.

[[39]] Ge Z. Process data analytics via probabilistic latent variable models: a tutorial review. Ind Eng Chem Res 2018;57(38):12646–112461.

[[40]] Yuan X, Ge Z, Ye L, Song Z. Supervised neighborhood preserving embedding for feature extraction and its application for soft sensor modeling. J Chemometr 2016;30(8):430–41.

[[41]] Chu Y, You F. Model-based integration of control and operations: overview, challenges, advances, and opportunities. Comput Chem Eng 2015;83:2–20.

[[42]] Yan Z, Wang J. Robust model predictive control of nonlinear systems with unmodeled dynamics and bounded uncertainties based on neural networks. IEEE Trans Neural Netw Learn Syst 2014;25(3):457–69.

[[43]] Appino RR, González Ordiano JÁ, Mikut R, Faulwasser T, Hagenmeyer V. On the use of probabilistic forecasts in scheduling of renewable energy sources coupled to storages. Appl Energy 2018;210:1207–18.

[[44]] Saltık MB, Özkan L, Ludlage JHA, Weiland S, Van den Hof PMJ. An outlook on robust model predictive control algorithms: reflections on performance and computational aspects. J Process Contr 2018;61:77–102.

[[45]] Farina M, Giulioni L, Scattolini R. Stochastic linear model predictive control with chance constraints—a review. J Process Contr 2016;44:53–67.

[[46]] Shang C, You F. A data-driven robust optimization approach to scenario-based stochastic model predictive control. J Process Contr 2019;75:24–39.

[[47]] Shang C, Chen WH, Stroock AD, You F. Robust model predictive control of irrigation systems with active uncertainty learning and data analytics. IEEE Trans Contr Syst Technol. Epub 2019 May 31.

[[48]] Rosolia U, Zhang X, Borrelli F. Data-driven predictive control for autonomous systems. Robot Auton Syst 2018;1:259–86.

[[49]] Marti K, Kall P. Stochastic programming. Berlin: Springer; 1994.

[[50]] Ben-Tal A, El Ghaoui L, Nemirovski A. Robust optimization. Princeton: Princeton University Press; 2009.

[[51]] Delage E, Ye Y. Distributionally robust optimization under moment uncertainty with application to data-driven problems. Oper Res 2010;58 (3):595–612.

[[52]] Gebreslassie BH, Yao Y, You F. Design under uncertainty of hydrocarbon biorefinery supply chains: multiobjective stochastic programming models, decomposition algorithm, and a comparison between CVaR and downside risk. AIChE J 2012;58(7):2155–79.

[[53]] Garcia DJ, You F. Supply chain design and optimization: challenges and opportunities. Comput Chem Eng 2015;81:153–70.

[[54]] Bertsimas D, Thiele A. Robust and data-driven optimization: modern decision making under uncertainty. In: Johnson MP, Norman B, Secomandi N, editors. Models, methods, and applications for innovative decision making. Catonsville: The Institute for Operations Research and the Management Sciences; 2006. p. 95–122.

[[55]] Calafiore G, Campi MC. Uncertain convex programs: randomized solutions and confidence levels. Math Program 2005;102(1):25–46.

[[56]] Campi MC, Garatti S. The exact feasibility of randomized solutions of uncertain convex programs. SIAM J Optim 2008;19(3):1211–30.

[[57]] Birge JR, Louveaux F. Introduction to stochastic programming. 2nd ed. New York: Springer Science & Business Media; 2011.

[[58]] You F, Grossmann IE. Multicut Benders decomposition algorithm for process supply chain planning under uncertainty. Ann Oper Res 2013;210 (1):191–211.

[[59]] Carlone L, Srivastava V, Bullo F, Calafiore GC. Distributed random convex programming via constraints consensus. SIAM J Contr Optim 2014;52 (1):629–62.

[[60]] You K, Tempo R, Xie P. Distributed algorithms for robust convex optimization via the scenario approach. IEEE Trans Automat Contr 2019;64 (3):880–95.

[[61]] Shang C, Huang X, You F. Data-driven robust optimization based on kernel learning. Comput Chem Eng 2017;106(2):464–79.

[[62]] Ning C, You F. Data-driven decision making under uncertainty integrating robust optimization with principal component analysis and kernel smoothing methods. Comput Chem Eng 2018;112:190–210.

[[63]] Ning C, You F. Data-driven adaptive nested robust optimization: general modeling framework and efficient computational algorithm for decision making under uncertainty. AIChE J 2017;63(9):3790–817.

[[64]] Ning C, You F. A data-driven multistage adaptive robust optimization framework for planning and scheduling under uncertainty. AIChE J 2017;63 (10):4343–69.

[[65]] Ning C, You F. Data-driven adaptive robust unit commitment under wind power uncertainty: a Bayesian nonparametric approach. IEEE Trans Power Syst 2019;34(3):2409–18.

[[66]] Zhao L, Ning C, You F. Operational optimization of industrial steam systems under uncertainty using data-driven adaptive robust optimization. AIChE J 2019;65(7):e16500.

[[67]] Zhao S, You F. Resilient supply chain design and operations with decisiondependent uncertainty using a data-driven robust optimization approach. AIChE J 2019;65(3):1006–21.

[[68]] Ning C, You F. Data-driven stochastic robust optimization: general computational framework and algorithm leveraging machine learning for optimization under uncertainty in the big data era. Comput Chem Eng 2018;111:115–33.

[[69]] Ning C, You F. Adaptive robust optimization with minimax regret criterion: multiobjective optimization framework and computational algorithm for planning and scheduling under uncertainty. Comput Chem Eng 2018;108:425–47.

[[70]] Bertsimas D, Gupta V, Kallus N. Data-driven robust optimization. Math Program 2018;167(2):235–92.

[[71]] Shang C, You F. Distributionally robust optimization for planning and scheduling under uncertainty. Comput Chem Eng 2018;110:53–68.

[[72]] Gao J, Ning C, You F. Data-driven distributionally robust optimization for shale gas supply chains under uncertainty. AIChE J 2019;65(3):947–63.

[[73]] MacGregor JF, Bruwer MJ, Miletic I, Cardin M, Liu Z. Latent variable models and big data in the process industries. In: Proceedings of 9th International Symposium on Advanced Control of Chemical Processes; 2015 Jun 7–10; Whistler, BC, Canada; 2015. p. 521–5.

[[74]] Shu Y, Zhao J. Data driven causal inference based on a modified transfer entropy. Comput Chem Eng 2013;57:173–80.

[[75]] Qin SJ. Survey on data-driven industrial process monitoring and diagnosis. Annu Rev Contr 2012;36(2):220–34.

[[76]] Pan SJ, Yang Q. A survey on transfer learning. IEEE Trans Knowl Data Eng 2010;22(10):1345–59.

[[77]] Wang R, Edgar TF, Baldea M, Nixon M, Wojsznis W, Dunia R. A geometric method for batch data visualization, process monitoring and fault detection. J Process Contr 2018;67:197–205.

[[78]] Kadlec P, Grbic´ R, Gabrys B. Review of adaptation mechanisms for data-driven soft sensors. Comput Chem Eng 2011;35(1):1–24.

[[79]] Morariu O, Morariu C, Borangiu T, Ra˘ileanu S. Manufacturing systems at scale with big data streaming and online machine learning. In: Borangiu T, Trentesaux D, Thomas A, Cardin O, editors. Service orientation in holonic and multi-agent manufacturing. Cham: Springer; 2018. p. 253–64.

[[80]] Geiger A, Lenz P, Urtasun R. Are we ready for autonomous driving? The kitti vision benchmark suite. In: Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition; 2012 Jun 16–21; Providence, RI, USA. Washington, DC: IEEE; 2012. p. 3354–61.

[[81]] Duchesne C, Liu JJ, MacGregor JF. Multivariate image analysis in the process industries: a review. Chemom Intell Lab Syst 2012;117:116–28.

[[82]] Chen M, Khare S, Huang B. A unified recursive just-in-time approach with industrial near infrared spectroscopy application. Chemom Intell Lab Syst 2014;135:133–40.

[[83]] Duan Y, Chen X, Houthooft R, Schulman J, Abbeel P. Benchmarking deep reinforcement learning for continuous control. In: Proceedings of the 33rd International Conference on Machine Learning; 2016 Jun 19–24; New York, NY, USA; 2016. p. 1329–38.

[[84]] Lewis FL, Vrabie D, Vamvoudakis KG. Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers. IEEE Control Syst 2012;32(6):76–105.

[[85]] Liu D, Yang X, Wang D, Wei Q. Reinforcement-learning-based robust controller design for continuous-time uncertain nonlinear systems subject to input constraints. IEEE Trans Cybern 2015;45(7):1372–85.

[[86]] Wiesemann W, Kuhn D, Sim M. Distributionally robust convex optimization. Oper Res 2014;62(6):1358–76