Turnpike Properties for Stochastic Linear-Quadratic Optimal Control Problems

Abstract

This paper analyzes the limiting behavior of stochastic linear-quadratic optimal control problems in finite time-horizon [0, T] as T → ∞. The so-called turnpike properties are established for such problems, under stabilizability condition which is weaker than the controllability, normally imposed in the similar problem for ordinary differential systems. In dealing with the turnpike problem, a crucial issue is to determine the corresponding static optimization problem. Intuitively mimicking the deterministic situations, it seems to be natural to include both the drift and the diffusion expressions of the state equation to be zero as constraints in the static optimization problem. However, this would lead us to a wrong direction. It is found that the correct static problem should contain the diffusion as a part of the objective function, which reveals a deep feature of the stochastic turnpike problem.

Keywords

Turnpike property / Stochastic optimal control / Static optimization / Linear-quadratic / Stabilizability / Riccati equation

References

[1] Ait Rami M, Zhou X Y. Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls. IEEE Trans. Automat. Control, 2000, 45: 1131-1143

[2] Breiten T, Pfeiffer L. On the turnpike property and the receding-horizon method for linear-quadratic optimal control problems. SIAM J. Control Optim., 2020, 58: 1077-1102

[3] Carlson D A, Haurie A B, Leizarowitz A. Infinite Horizon Optimal Control-Deterministic and Stochastic Systems, 1991 2nd ed. Berlin: Springer-Verlag

[4] Damm T, Grüne L, Stieler M, Worthmann K. An exponential turnpike theorem for dissipative discrete time optimal control problems. SIAM J. Control Optim., 2014, 52: 1935-1957

[5] Dorfman R, Samuelson P A, Solow R M. Linear Programming and Economics Analysis, 1958, New York: McGraw-Hill

[6] Grüne L, Guglielmi R. Turnpike properties and strict dissipativity for discrete time linear quadratic optimal control problems. SIAM J. Control Optim., 2018, 56: 1282-1302

[7] Grüne L, Guglielmi R. On the relation between turnpike properties and dissipativity for continuous time linear quadratic optimal control problems. Math. Control Relat. Fields, 2021, 11: 169-188

[8] Huang J, Li X, Yong J. A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Math. Control Relat. Fields, 2015, 5: 97-139

[9] Ibañez A. Optimal control of the Lotka-Volterra system: turnpike property and numerical simulations. J. Biol. Dyn., 2017, 11: 25-41

[10] Kalman R E. Contributions to the theory of optimal control. Bol. Soc. Mat. Mexicanna, 1960, 5: 102-119

[11] Li X, Yong J, Zhou Y. Elements in Control Theory, 2010 2nd ed. Beijing: High Education Press (in Chinese)

[12] Liu F, Peng S. On controllability for stochastic control systems when the coefficient is time-variant. J Syst. Soc., 2010, 14: 270-278

[13] Lou H, Wang W. Turnpike properties of optimal relaxed control problems. ESAIM Control Optim. Calc. Var., 2019, 25: 74

[14] McKenzie L W. Turnpike theory. Econometrica, 1976, 44: 841-865

[15] von Neumann J. A model of general economic equilibrium. Rev. Econ. Stud., 1945, 13: 1-9

[16] Peng S. Backward stochastic differential equation and exact controllability of stochastic control systems. Progr. Natural Sci. (English Ed.), 1994, 4: 274-284

[17] Porretta A, Zuazua E. Long time versus steady state optimal control. SIAM J. Control Optim., 2013, 51: 4242-4273

[18] Porretta A, Zuazua E. Remarks on long time versus steady state optimal control, Mathematical Paradigms of Climate Science, 2016, New York: Springer-Verlag 67-89

[19] Rawlings J B, Amrit R. Magni L, Raimondo D M, Allgöwer F. Optimizing process economic performance using model predictive control, Nonlinear Model Predictive Control. Lecture Notes in Control and Information Science, 384, 2009, Berlin: Springer-Verlag 119-138

[20] Sun J, Li X, Yong J. Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems. SIAM J. Control Optim., 2016, 54: 2274-2308

[21] Sun J, Yong J. Stochastic linear quadratic optimal control problems in infinite horizon. Appl. Math. Optim., 2018, 78: 145-183

[22] Sun J, Yong J. Stochastic Linear-Quadratic Optimal Control Theory: Open-Loop and Closed-Loop Solutions, 2020, Cham: Springer-Verlag

[23] Trélat E, Zhang C. Integral and measure-turnpike properties for infinite-dimensional optimal control systems. Math. Control Signals Syst., 2018, 30: 1-34

[24] Trélat E, Zuazua E. The turnpike property in finite-dimensional nonlinear optimal control. J. Differential Equations, 2015, 258: 81-114

[25] Wang Y, Yang D, Yong J, Yu Z. Exact controllability of linear stochastic differential equations and related problems. Math. Control Relat. Fields, 2017, 7: 305-345

[26] Yong J. Optimization Theory: A Concise Introduction, 2018, Singapore: World Scientific

[27] Zaslavski A J. Turnpike Properties in the Calculus of Variations and Optimal Control, 2006, New York: Springer-Verlag

[28] Zaslavski A J. Turnpike properties of approximate solutions for discrete-time control systems. Commun. Math. Anal., 2011, 11: 36-45

[29] Zaslavski A J. Turnpike Conditions in Infinite Dimensional Optimal Control, 2019, Cham: Springer-Verlag

[30] Zuazua E. Large time control and turnpike properties for wave equations. Annu. Rev. Control, 2017, 44: 199-210