Optimization of Markov jump linear system with controlled modes jump probabilities

Abstract

The optimal control of a Markov jump linear quadratic model with controlled jump probabilities of modes is investigated. Two kinds of mode control policies, i.e., open-loop control policy and closed-loop control policy, are considered. Using the concepts of policy iteration and performance potential, the sufficient condition needed for the optimal closed-loop control policy to perform better than the optimal open-loop control policy is proposed. The condition is helpful for the design of an optimal controller. Furthermore, an efficient algorithm to construct a closed-loop control policy, which is better than the optimal open-loop control policy, is given with policy iteration.

Keywords

Markov jump system / optimal control / policy iteration

References

[1] ChengD Z, GuoY Q. Advances on switched systems. Control Theory and Applications, 2005, 22(6): 954–960(in Chinese)

[2] Abou-kandilH, De SmetO, FreilingG, . Flow control in a failure-prone multi-machine manufacturing system. In: Proceedings of INRIA/IEEE Symposium on Emerging Technologies and Factory Automation. 1995, 2: 575–583

[3] BoukasE K, ShiP, AndijaniA. Robust inventory-production control problem with stochastic demand. Optimal Control Applications and Methods, 1999, 20(1): 1–20

[4] JiY, ChizeckH J. Controllability, stabilizability, and continuous-time Markovian jump linear quadratic control. IEEE Transactions on Automatic Control, 1990, 35(7): 777–788

[5] CostaO, FragosoM D, MarquesR P. Discrete-Time Markov Jump Linear Systems. London: Springer-Verlag, 2005

[6] XueF, GuoL. Necessary and sufficient conditions for adaptive stabilizability of jump linear systems. Communications in Information and Systems, 2001, 1(2): 205–224

[7] ZhangL J, LiC W, ChengD Z. Robust adaptive control of Markov jump systems with parameter uncertainties. Control and Decision, 2005, 20(9): 1030–1033(in Chinese)

[8] LiuF. Robust L₂-L_∞ filtering for uncertain jump systems. Control and Decision, 2005, 20(1): 32–35(in Chinese)

[9] LiuF, ZhangX H. Robust control for jump systems with L₂ gain constraints. Control Theory and Applications, 2006, 23(3): 373–377(in Chinese)

[10] LiuF, SuH Y, ChuJ. Robust positive real control of Markov jump systems with parametric uncertainties. Acta Automatica Sinica, 2003, 29(5): 761–766(in Chinese)

[11] JiY, ChizeckH J. Optimal quadratic control of jump linear systems with separately controlled transition probabilities. International Journal of Control, 1989, 49(2): 481–491

[12] BoukasE K, LiuZ K. Jump linear quadratic regulator with controlled jump rates. IEEE Transactions on Automatic Control, 2001, 46(2): 301–305

[13] XuY K, ChenX. Discrete-time JLQG with dependently controlled jump probabilities. In: Proceedings of IEEE 22nd International Symposium on Intelligent Control, Singapore. 2007: 441–445

[14] CaoX R. Stochastic Learning and Optimization: A Sensitivity-Based Approach. New York: Springer, 2007

[15] ZhangK J, XuY K, ChenX, . Policy iteration based feedback control. Automatica, 2008, 44(4): 1055–1061

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant Nos. 60574064, 60736027).

RIGHTS & PERMISSIONS

2014 Higher Education Press and Springer-Verlag Berlin Heidelberg