Using machine learning models to explore the solution space of large nonlinear systems underlying flowsheet simulations with constraints

Abstract

Flowsheet simulations of chemical processes on an industrial scale require the solution of large systems of nonlinear equations, so that solvability becomes a practical issue. Additional constraints from technical, economic, environmental, and safety considerations may further limit the feasible solution space beyond the convergence requirement. A priori, the design variable domains for which a simulation converges and fulfills the imposed constraints are usually unknown and it can become very time-consuming to distinguish feasible from infeasible design variable choices by simply running the simulation for each choice. To support the exploration of the design variable space for such scenarios, an adaptive sampling technique based on machine learning models has recently been proposed. However, that approach only considers the exploration of the convergent domain and ignores additional constraints. In this paper, we present an improvement which particularly takes the fulfillment of constraints into account. We successfully apply the proposed algorithm to a toy example in up to 20 dimensions and to an industrially relevant flowsheet simulation.

Graphical abstract

Keywords

machine learning / flowsheet simulations / constraints / exploration

References

[1] Grossmann I E, Sargent R W H. Optimum design of chemical plants with uncertain parameters. AIChE Journal. American Institute of Chemical Engineers, 1978, 24(6): 1021–1028

[2] Halemane K P, Grossmann I E. Optimal process design under uncertainty. AIChE Journal. American Institute of Chemical Engineers, 1983, 29(3): 425–433

[3] Boukouvala F, Ierapetritou M G. Feasibility analysis of black-box processes using an adaptive sampling Kriging-based method. Computers & Chemical Engineering, 2012, 36: 358–368

[4] Boukouvala F, Ierapetritou M G. Derivative-free optimization for expensive constrained problems using a novel expected improvement objective function. AIChE Journal. American Institute of Chemical Engineers, 2014, 60(7): 2462–2474

[5] Wang Z, Ierapetritou M G. A novel feasibility analysis method for black-box processes using a radial basis function adaptive sampling approach. AIChE Journal. American Institute of Chemical Engineers, 2017, 63(2): 532–550

[6] Rogers A, Ierapetritou M G. Feasibility and flexibility analysis of black-box processes Part 1: surrogate-based feasibility analysis. Chemical Engineering Science, 2015, 137: 986–1004

[7] Shahriari B, Swersky K, Wang Z, Adams R P, de Freitas N. Taking the human out of the loop: a review of Bayesian optimization. Proceedings of the Institute of Electrical and Electronics Engineers, 2016, 104(1): 148–175

[8] Bano G, Wang Z, Facco P, Bezzo F, Barolo M, Ierapetritou M G. A novel and systematic approach to identify the design space of pharmaceutical processes. Computers & Chemical Engineering, 2018, 115: 309–322

[9] Gramacy R B, Lee H K H. Optimization Under Unknown Constraints, Bayesian Statistics 9: Proceedings of the Ninth Valencia International Meeting, 2011, 9, 229–256

[10] Tran A, Sun J, Furlan J M, Pagalthivarthi K V, Visintainer R J, Wang Y. A batch parallel known/unknown constrained Bayesian optimization with feasibility classification and its applications in computational fluid dynamics. Computer Methods in Applied Mechanics and Engineering, 2019, 347: 827–852

[11] Gelbart M A, Snoek J, Adams R P. Bayesian optimization with unknown constraints. arXiv:1403.5607, 2014

[12] Griffiths R, Hernández-Lobato J M. Constrained Bayesian optimization for automatic chemical design using variational autoencoders. Chemical Science (Cambridge), 2020, 11(2): 577–586

[13] Dias L S, Ierapretrou M G. Data-driven feasibility analysis for the integration of planning and scheduling problems. Optimization and Engineering, 2019, 20(4): 1029–1066

[14] Heese R, Walczak M, Seidel T, Asprion N, Bortz M. Optimized data exploration applied to the simulation of a chemical process. Computers & Chemical Engineering, 2019, 124: 326–342

[15] Schonlau M, Welch W J, Jones D R. Global versus local search in constrained optimization of computer models. Institute of Mathematical Statistics Lecture Notes—Monograph Series, 1998, 34: 11–25

[16] Gelbart M A. Constrained Bayesian optimization and applications. Dissertation for the Doctoral Degree. Cambridge (Massachusetts): Harvard University, 2015

[17] Rasmussen C E, Williams C K I. Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning). Cambridge (Massachusetts): The MIT Press, 2005

[18] Gardner J R, Kusner M J, Xu Z, Weinberger K Q, Cunningham J P. Bayesian optimization with inequality constraints. ICML'14: Proceedings of the 31st International Conference on International Conference on Machine Learning, 2014, 32: 937–945

[19] Schölkopf B. The kernel trick for distances. In: Advances in Neural Information Processing Systems. Cambridge (Massachusetts): The MIT Press, 2001, 301–307

[20] Heese R, Walczak M, Bortz M, Schmid J. Calibrated simplex mapping classification. arxiv.org/abs/2103.02926, 2021

[21] Byrd R H, Lu P, Nocedal J, Zhu C. A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing, 1995, 16(5): 1190–1208

[22] Zhu C, Byrd R H, Lu P, Nocedal J. Algorithm 778: L-bfgs-b: Fortran subroutines for large-scale bound-constrained optimization. ACM Transactions on Mathematical Software, 1997, 23(4): 550–560

[23] Virtanen P, Gommers R, Oliphant T E, Haberland M, Reddy T, Cournapeau D, Burovski E, Peterson P, Weckesser W, Bright J, . SciPy 1.0: fundamental algorithms for scientific computing in Python. Nature Methods, 2020, 17(3): 261–272

[24] Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, . Scikit-learn: machine learning in Python. Journal of Machine Learning Research, 2011, 12: 2825–2830

[25] GPy. GPy: a gaussian process framework in python. The website of github, 2012

[26] Biegler L T, Grossmann I E, Westerberg A W. Systematic Methods for Chemical Process Design. New Jersey: Prentice Hall, 1997

[27] Renon H, Prausnitz J M. Local compositions in thermodynamic excess functions for liquid mixtures. AIChE Journal. American Institute of Chemical Engineers, 1968, 14(1): 135–144

[28] Bortz M, Burger J, Asprion N, Blagov S, Böttcher R, Nowak U, Scheithauer A, Welke R, Küfer K H, Hasse H. Multi-criteria optimization in chemical process design and decision support by navigation on pareto sets. Computers & Chemical Engineering, 2014, 60: 354–363

Acknowledgements

This work was developed within the Fraunhofer Cluster of Excellence “Cognitive Internet Technologies”.

Electronic Supplementary Material

Supplementary material is available in the online version of this article at https://dx.doi.org/10.1007/s11705-021-2073-7 and is accessible for authorized users.

RIGHTS & PERMISSIONS

2021 Higher Education Press