Maximum principle for a discrete-time robust stochastic optimal control problem

Abstract

This paper presents the necessary and sufficient conditions for a kind of discrete-time robust stochastic optimal control problem with convex control domains. The classical variational method is invalid in this context because it is an “inf sup problem”. We obtain the variational inequality with a common reference probability by systematically using weak convergence approach and the minimax theorem. Moreover, a discrete-time robust investment problem is also studied where the explicit optimal control is given.

Keywords

Stochastic maximum principle / Discrete-time system / Robust control / Investment problem

References

[1] Aoki, M. Optimization of stochastic systems: topics in discrete-time systems. Elsevier, 2016.

[2] Bäuerle, N., and Glauner, A., Distributionally robust Markov decision processes and their connection to risk measures, Mathematics of Operations Research, 2022, 47(3): 1757-1780

[3] Beghi, A., and D’alessandro, D., Discrete-time optimal control with control-dependent noise and generalized Riccati difference equations, Automatica, 1998, 34(8): 1031-1034

[4] Beissner, P., Lin, Q., and Riedel, F., Dynamically consistent alpha-maxmin expected utility, Mathematical Finance, 2020, 30(3): 1073-1102

[5] Bensoussan, A., Maximum principle and dynamic programming approaches of the optimal control of partially observed diffusions, Stochastics: An International Journal of Probability and Stochastic Processes, 1983, 9(3): 169-222

[6] Bertsekas, D., and Shreve, S. E. Stochastic optimal control: the discrete-time case ( Vol. 5). Athena Scientific, 1996.

[7] Bielecki, T. R., Chen, T., Cialenco, I., Cousin, A., & Jeanblanc, M., Adaptive robust control under model uncertainty, SIAM Journal on Control and Optimization, 2019, 57(2): 925-946

[8] Bismut, J. M., Conjugate convex functions in optimal stochastic control, Journal of Mathematical Analysis and Applications, 1973, 44(2): 384-404

[9] Cadenillas, A., and Karatzas, I., The stochastic maximum principle for linear, convex optimal control with random coefficients, SIAM journal on control and optimization, 1995, 33(2): 590-624

[10] Dong, B., Nie, T., and Wu, Z., Maximum principle for discrete-time stochastic control problem of mean-field type, Automatica, 2022, 144: 110497

[11] Elliott, R., Li, X., and Ni, Y. H., Discrete time mean-field stochastic linear-quadratic optimal control problems, Automatica, 2013, 49(11): 3222-3233

[12] Gilboa, I., and Schmeidler, D., Maxmin expected utility with non-unique prior, Journal of Mathematical Economics, 1989, 18(2): 141-153

[13] González-Trejo, J. I., Hernández-Lerma, O., and Hoyos-Reyes, L. F., Minimax control of discrete-time stochastic systems, SIAM Journal on Control and Optimization, 2002, 41(5): 1626-1659

[14] Halkin, H., A maximum principle of the Pontryagin type for systems described by nonlinear difference equations, SIAM Journal on control, 1966, 4(1): 90-111

[15] Hansen, L. P., Sargent, T. J., Turmuhambetova, G., and Williams, N., Robust control and model misspecification, Journal of Economic Theory, 2006, 128(1): 45-90

[16] He, W., Luo, P., and Wang, F. Maximum principle for mean-field SDEs under model uncertainty. Applied Mathematics and Optimization, 2023, 87(3), 59.

[17] Hu, M., and Ji, S., Stochastic maximum principle for stochastic recursive optimal control problem under volatility ambiguity, SIAM Journal on Control and Optimization, 2016, 54(2): 918-945

[18] Hu, M., Ji, S., andLi, X. Maximum principle for discrete-time stochastic optimal control problem under distribution uncertainty. arXiv preprint arXiv: 2206.12846, 2022.

[19] Hu, M., and Wang, F., Maximum principle for stochastic recursive optimal control problem under model uncertainty, SIAM Journal on Control and Optimization, 2020, 58(3): 1341-1370

[20] Ji, S., and Liu, H., Maximum principle for stochastic optimal control problem of forward-backward stochastic difference systems, International Journal of control, 2022, 95(7): 1979-1992

[21] Kushner, H. J., On the stochastic maximum principle: Fixed time of control, Journal of Mathematical Analysis and Applications, 1965, 11: 78-92

[22] Kushner, H. J., Necessary conditions for continuous parameter stochastic optimization problems, SIAM Journal on Control, 1972, 10(3): 550-565

[23] Lin, X., and Zhang, W., A maximum principle for optimal control of discrete-time stochastic systems with multiplicative noise, IEEE Transactions on Automatic Control, 2014, 60(4): 1121-1126

[24] Maccheroni, F., Marinacci, M., and Rustichini, A., Ambiguity aversion, robustness, and the variational representation of preferences, Econometrica, 2006, 74(6): 1447-1498

[25] Ni, Y. H., Elliott, R., and Li, X., Discrete-time mean-field stochastic linear-quadratic optimal control problems, II: Infinite horizon case, Automatica, 2015, 57: 65-77

[26] Paruchuri, P., and Chatterjee, D., Discrete time Pontryagin maximum principle under state-action-frequency constraints, IEEE Transactions on Automatic Control, 2019, 64(10): 4202-4208

[27] Peng, S., A general stochastic maximum principle for optimal control problems, SIAM Journal on control and optimization, 1990, 28(4): 966-979

[28] Peng, S. G-expectation, G-Brownian Motion and related stochastic calculus of Itô type. In Stochastic Analysis and Applications (pp.541-567). Springer, Berlin, Heidelberg, 2007.

[29] Peng, S., Multi-dimensional G-Brownian motion and related stochastic calculus under G-expectation, Stochastic Processes and their Applications, 2008, 118(12): 2223-2253

[30] Peng, S. Nonlinear expectations and stochastic calculus under uncertainty. Springer-Verlag Berlin Heidelberg, 2019.

[31] Pham, H. Continuous-time stochastic control and optimization with financial applications ( Vol. 61). Springer Science & Business Media, 2009.

[32] Pham, H., & Wei, X., Discrete time McKean-Vlasov control problem: a dynamic programming approach, Applied Mathematics & Optimization, 2016, 74: 487-506

[33] Rami, M. A., Chen, X., and Zhou, X. Y., Discrete-time indefinite LQ control with state and control dependent noises, Journal of Global Optimization, 2002, 23: 245-265

[34] Rudin, W. Real and complex analysis. Tata McGraw-hill education, 2006.

[35] Song, T., and Liu, B., A maximum principle for fully coupled controlled forward-backward stochastic difference systems of mean-field type, Advances in Difference Equations, 2020, 2020: 1-24

[36] Wang, G., and Xing, Z., Two-stage linear quadratic stochastic optimal control problem under model uncertainty, Science China Information Sciences, 2025, 68(9): 1-14

[37] Wang, G., and Xing, Z., Robust optimal control of biobjective linear-quadratic system with noisy observation, IEEE Transactions on Automatic Control, 2023, 69(1): 303-308

[38] Wu, Z., and Zhang, F., Maximum principle for discrete-time stochastic optimal control problem and stochastic game, Mathematical Control & Related Fields, 2022, 12(2): 475

[39] Yong, J., and Zhou, X. Y. Stochastic controls: Hamiltonian systems and HJB equations ( Vol. 43). Springer Science and Business Media, 1999.

[40] Zhang, H., Qi, Q., and Fu, M., Optimal stabilization control for discrete-time mean-field stochastic systems, IEEE Transactions on Automatic Control, 2018, 64(3): 1125-1136