Optimization of Markov jump linear system with controlled modes jump probabilities

Abstract

The optimal control of a Markov jump linear quadratic model with controlled jump probabilities of modes is investigated. Two kinds of mode control policies, i.e., open-loop control policy and closed-loop control policy, are considered. Using the concepts of policy iteration and performance potential, the sufficient condition needed for the optimal closed-loop control policy to perform better than the optimal open-loop control policy is proposed. The condition is helpful for the design of an optimal controller. Furthermore, an efficient algorithm to construct a closed-loop control policy, which is better than the optimal open-loop control policy, is given with policy iteration.

Key words： Markov jump system; optimal control; policy iteration

Introduction

In recent years, switching systems have received great attention because of their potential applications in engineering systems [1]. The Markov jump linear system (MJLS) is a class of switching systems that has been fully studied. MJLS is widely applied in systems with abrupt changes in operating points or disturbances [2,3], including flexible manufacturing systems, power systems, economic systems, fault-tolerant systems and inventory systems[ 4–10].

This paper considers the discrete-time jump linear quadratic Gaussian (JLQG) model. The mode jump of a standard Markov jump system is governed by a Markov chain, whose transition probability matrix is given a priori. In practice, the jump of modes is random, but the jump probabilities can often be controlled (or selected from a finite set). For example, the probability of the jump from normal mode to fault mode of a machine depends on daily maintenance frequency; and the switch between received mode and lost mode of data packages in networked control systems depends on the strength of the communication signal. However, problems with controlled jump probabilities of modes are rarely studied [11–13]. This paper discusses two classes of control policies for modes jump probabilities: open-loop mode control and closed-loop mode control. The relation between the two mode control policies and the standard JLQG model is analyzed. Generally, optimization of the open-loop mode control is an easier problem in a smaller policy space compared to optimization of the closed-loop mode control. By using the performance potential concept and policy iteration approach, this paper presents the sufficient condition under which the optimal closed-loop mode control is better than the optimal open-loop mode control. Based on this condition, we can easily construct a closed-loop mode control policy, which has better performance than the optimal open-loop mode control.

Formulation

The Markov jump system is a class of hybrid systems with two kinds of dynamics, i.e., the mode described by a discrete Markov chain, and the state described by the state-space equation under a mode. Consider the following discrete-time linear system

(1)x_{k+1}=A\left({\theta }_k\right)x_k+B\left({\theta }_k\right)u_k+\sigma \left({\theta }_k\right)w_k,

where k is the discrete time epoch, BoldItalic_k ∈ Rⁿ is the state, {\theta }_k\in =\{1,2,\dots ,S\} is the mode, BoldItalic_k ∈ R^m is the control variable, and w_k is an i.i.d. random variable with mean zero and covariance matrix BoldItalic (identity matrix). BoldItalic(θ_k), BoldItalic(θ_k), BoldItalic(θ_k) are suitably-dimensional matrices depending on the mode. The mode θ_k satisfies the Markov chain whose jump probability matrix is p={\left\{p_{ij}\right\}}_{i,j\in S}. Assume that the Markov chain is ergodic, and it has a steady-state probability row vector BoldItalic = [π₁,π₂,...,π_S]. The criterion to be optimized is

(2)J\left(x_0,{\theta }_0\right)=\underset{K\to \infty }{\lim }\frac{1}{K}E\left\{{\displaystyle \sum _{k=0}^{K-1}\left[f^u\left({\theta }_k,x_k\right)\right]}|x_0,{\theta }_0\right\},

where the cost function is

f^u\left({\theta }_k,x_k\right)=x_{k}M\left({\theta }_k\right)x_k+u_{k}N\left({\theta }_k\right)u_k.

Suppose that Eq. (1) is stochastically stabilizable. For a stable system, performance criterion Eq. (2) exists and is independent of the initial values [5]. For simplicity, when θ_k = i, the matrices BoldItalic(θ_k), BoldItalic(θ_k), BoldItalic(θ_k), BoldItalic(θ_k) and BoldItalic(θ_k) are written as BoldItalic_i, BoldItalic_i, BoldItalic_i, BoldItalic_i and BoldItalic_i, respectively. Let a_i={\sigma }_i{\sigma }_i^{\text{T}}. Assume that BoldItalic_i and BoldItalic_i are positive semi-definite matrices.

If the jump probabilities \left\{p_{ij},\forall j\in S\right\} from mode i to other modes are not given a priori, but can be chosen from a finite set Y_i, it can be concluded that the jump probabilities p={\left\{p_{ij}\right\}}_{i,j\in S} of the modes are controllable. This paper studies the optimization problem of JLQG with controlled mode jump probabilities. At this point, the admissible control policy ℒ is a combination of the feedback control law u and the mode control policy, denoted as \text{ℒ}=\{u,p\}. Consider two classes of mode control polices: For the first class, the jump probability is independent of the current system state BoldItalic, i.e., the jump probability takes the same value for any state BoldItalic, thus we call it an open-loop mode control; For the second class, the jump probability depends on the current system state BoldItalic, i.e., the jump probability takes a different value for a different state BoldItalic, thus we call it a closed-loop mode control, and the corresponding jump probability is denoted as p_ij(BoldItalic). Consider the following optimization problems:

Problem 1 Find a state feedback control law u(i,BoldItalic) and a closed-loop mode control p_ij(BoldItalic) to minimize performance Eq. (2). Denote the admissible control policy of Problem 1 as L_1=\left\{u(i,x),p_{ij}(x)\right\}, where the optimal policy is denoted as {\text{ℒ}}_1^{*}.

Problem 2 Find a state feedback control law u(i,BoldItalic) and an open-loop mode control p_ij to minimize performance Eq. (2). Denote the admissible control policy of Problem 2 as {\text{ℒ}}_2=\left\{u(i,x),p_{ij}\right\}, where the optimal policy is denoted as {\text{ℒ}}_2^{*}.

Problem 3 Given a mode jump probability matrix BoldItalic, find a state feedback control law u(i,BoldItalic) to minimize performance Eq. (2). Denote the admissible control policy of Problem 3 as {\text{ℒ}}_3(p)=\{u(i,x)\}, where the optimal policy is denoted as {\text{ℒ}}_3^{*}(p). Problem 3 is a typical JLQG problem. The following lemma gives its optimal solution.

Lemma 1 [5] Given a mode's jump probability p={\left\{p_{ij}\right\}}_{i,j\in S}, the optimal feedback control law is

u^{*}(i,x)=-L_{i}x,

{ℒ}_i={\left[N_i+B_i^{\text{T}}{F}_i{B}_i\right]}^{-1}{B}_i^{\text{T}}{F}_i{A}_i,

where F_i={\displaystyle \sum _{j\in }{p}_{ij}{K}_j}, BoldItalic_i is the unique solution to the coupled Riccati equation:

K_i=A_i^{\text{T}}{F}_i{A}_i+M_i-A_i^{\text{T}}{F}_i{B}_i{L}_i.

Then the optimal performance is

J^{{�}_3^{*}(p)}={\displaystyle \sum _{i\in �}{\pi }_i\text{tr}\left(a_i{F}_i\right)}.

For Problem 3, by applying the results in Ref. [5] and the approach in Ref. [13], it is easy to verify that the value function (also called performance potential [14]) of the optimal feedback control law u^*(i,BoldItalic) is

(3)g^{L_3^{*}\left(p\right)}(i,x)=x^{\text{T}}{K}_{i}x+q_i.

where q_i is the solution to the following equation:

(4)\left(I-p\right)q+J^{{\text{ℒ}}_3^{*}\left(p\right)}e-\tilde{f}=0,

where BoldItalic is the identity matrix, BoldItalic = [q₁,q₂,...,q_S]^T, BoldItalic = [1,1,...,1]^T, \tilde{f}={\left[{\tilde{f}}_1,{\tilde{f}}_2,\dots ,{\tilde{f}}_S\right]}^{\text{T}} and {\tilde{f}}_i=\text{tr}\left(a_i{F}_i\right). The solution of Eq. (4) is not unique, i.e., ifq_i,\forall i\in S is a solution, then for any constant c, q_i,\forall i\in S is also a solution to Eq. (4). Reference [14] gives a special solution in the following form:

(5)q={\left(I-p+e\pi \right)}^{-1}\tilde{f}.

If p_ij is controlled, \left\{{\text{ℒ}}_3^{*}(p),p_{ij}\right\} is an admissible control policy of Problem 2. Let

p^{*}=\underset{p}{\arg \min }\left\{J^{\left\{{\text{ℒ}}_3^{*}(p),p_{ij}\right\}}\right\},

then {\text{ℒ}}_2^{*}=\left\{{\text{ℒ}}_3^{*}(p^{*}),p_{ij}^{*}\right\} is the optimal admissible control policy of Problem 2, and the performance of {\text{ℒ}}_3^{*}\left(p^{*}\right) is the same as that of {\text{ℒ}}_2^{*}, i.e., J^{{\text{ℒ}}_2^{*}}=J^{{\text{ℒ}}_3^{*}\left(p^{*}\right)}. By solving Problem 3, we can find the optimal open-loop mode control in the set q_i,\forall i\in S, and thus obtain the optimal policy {\text{ℒ}}_2^{*} of Problem 2. Another way to address Problem 2 is to apply the gradient-based method introduced in Ref. [13] to find the optimal policy.

Solving Problem 1 is rather difficult. An admissible control policy of Problem 2 can be viewed as a special admissible control policy of Problem 1, thus the policy space of Problem 2 is a subset of the policy space of Problem 1. Therefore, the optimal performance of Problem 1 is no worse than the optimal performance of Problem 2.

Dynamic systems can be formulated with Markov systems. We define transition function P_i(B|BoldItalic) as the probability of the transition from mode i, state BoldItalic to Borel set B, and P(j,B|i,BoldItalic) as the probability of the transition from mode i, state BoldItalic to mode j, Borel set B. Then we have P(j,B|i,BoldItalic) = p_ijP_i(B|BoldItalic). For any cost function f(i,BoldItalic), define a transition operation

(6)Pf(i,x)={\displaystyle \sum _{j\in S}{p}_{ij}{\displaystyle \underset{y\in R^n}{\int }{P}_i(\text{d}y|x)f(j,y)}}.

In the above definition, f(i,x) represents the expected cost function at the next time epoch after one step transition when the current mode is i and the state is BoldItalic. Let \text{ℒ} be a control policy which can be any admissible control policy of Problems 1, 2 and 3. Under this policy, the transition function, cost function and performance are denoted as P^{\text{ℒ}},f^{\text{ℒ}} and J^{\text{ℒ}} respectively. Reference [15 ] gives the policy iteration formula to optimize performance.

Lemma 2 [15] Policy \text{ℒ}\text{'} is better than policy \text{ℒ}, if and only if

For any i\in S,x\in R^n, we have

{}^{\text{ℒ}\text{'}}{g}^{\text{ℒ}}(i,x)+f^{\text{ℒ}\text{'}}(i,x)\le {}^{\text{ℒ}}{g}^{\text{ℒ}}(i,x)+f^{\text{ℒ}}(i,x);

There exists l\in S and a set X ⊂ Rⁿ with non-zero measure, such that for all BoldItalic ∈ X, we have

g^{\text{ℒ}} is the performance potential under policy \text{ℒ}, which satisfies the Poisson equation:

J^{\text{ℒ}}+g^{\text{ℒ}}=f^{\text{ℒ}}+{}^{\text{ℒ}}{g}^{\text{ℒ}}.

Analysis of two classes of mode control policies

We shall discuss the condition under which the optimal admissible control policy of Problem 1 is better than that of Problem 2.

Lemma 3 If \exists l\in S,X ⊂ Rⁿ and {\overline{p}}_{lj}\in Y_l, such that for l and BoldItalic ∈ X, we have

(7)

then {\text{ℒ}}_1^{*} is better than {\text{ℒ}}_2^{*}.

Proof{\text{ℒ}}_2^{*} can be considered as a special admissible control policy of Problem 1:

{\widetilde{ℒ}}_1=\left\{{\text{ℒ}}_3^{*}\left(p^{*}\right),{\tilde{p}}_{ij}(x)=p_{ij}^{*},\forall x\right\}.

Policies {\text{ℒ}}_2^{*} and {\widetilde{ℒ}}_1 have the same performance. We can construct an admissible control policy for Problem 1 as follows:

{{ℒ}^{\prime }}_1=\left\{{\text{ℒ}}_3^{*}\left(p^{*}\right),{p^{\prime }}_{ij}(x)\right\},

where

(8){p^{\prime }}_{ij}(x)=\{\begin{array}{l}{\overline{p}}_{ij},\text{if}i=l\text{and}x\in X,\\ p_{ij}^{*},\text{otherwise}.\end{array}

Note that the cost function f^u(i,BoldItalic) only depends on the feedback control law, and is independent of mode control policy, thus

\begin{array}{l}{\wp }^{{{ℒ}^{\prime }}_1}{g}^{{\widetilde{ℒ}}_1}(i,x)+f^{{{ℒ}^{\prime }}_1}(i,x)\\ ={\displaystyle \sum _{j\in S}{p^{\prime }}_{ij}(x){\displaystyle \underset{y\in R^n}{\int }{P}_i^{{\text{ℒ}}_3^{*}\left(p^{*}\right)}(\text{d}y|x)g^{{\text{ℒ}}_2^{*}}(j,y)}}+f^{{\text{ℒ}}_3^{*}\left(p^{*}\right)}(i,x),\end{array}

and

\begin{array}{l}{\wp }^{{\widetilde{ℒ}}_1}{g}^{{\widetilde{ℒ}}_1}(i,x)+f^{{\widetilde{ℒ}}_1}(i,x)\\ ={\displaystyle \sum _{j\in S}{p}_{ij}^{*}{\displaystyle \underset{y\in R^n}{\int }{P}_i^{{\text{ℒ}}_3^{*}\left(p^{*}\right)}(\text{d}y|x)g^{{\text{ℒ}}_2^{*}}(j,y)}}+f^{{\text{ℒ}}_3^{*}\left(p^{*}\right)}(i,x).\end{array}

According to Eqs. (7) and (8), for l and BoldItalic ∈ X, we have

For any other mode i and state BoldItalic, we have

{\wp }^{{{ℒ}^{\prime }}_1}{g}^{{\widetilde{ℒ}}_1}(i,x)+f^{{{ℒ}^{\prime }}_1}(i,x)={\wp }^{{\widetilde{ℒ}}_1}{g}^{{\widetilde{ℒ}}_1}(i,x)+f^{{\widetilde{ℒ}}_1}(i,x).

According to Lemma 2, {{ℒ}^{\prime }}_1 is better than {\widetilde{ℒ}}_1. And {\text{ℒ}}_1^{*} is no worse than {{ℒ}^{\prime }}_1, thus {\text{ℒ}}_1^{*} is better than {\widetilde{ℒ}}_1. Since {\text{ℒ}}_2^{*}={\widetilde{ℒ}}_1, {\text{ℒ}}_1^{*} is better than {\text{ℒ}}_2^{*}.

Lemma 4 If \exists l\in S,X ⊂ Rⁿ and {\overline{p}}_{lj}\in Y_l, such that for l and BoldItalic ∈ X, we have

(9)

Thus {\text{ℒ}}_1^{*} is better than {\text{ℒ}}_2^{*}. K_i,q_i,L_i,\forall i\in S are solutions to Problem 3 with a given BoldItalic*.

Proof Put Eq. (3) into Eq. (7), and the proof can be obtained with Lemmas 3 and 4.

Let

\Pi \triangleq {\left(A_l-B_l{L}_l\right)}^{\text{T}}{\displaystyle \sum _{j\in }{K}_j\left({\overline{p}}_{lj}-p_{lj}^{*}\right)\left(A_l-B_l{L}_l\right)},

and

\text{Γ}\triangleq {\displaystyle \sum _{j\in }\left(\text{tr}(a_l{K}_j)+q_j\right)\left(p_{lj}^{*}-{\overline{p}}_{lj}\right)},

then Eq. (9) is equivalent to

(10)

With the above lemmas, we obtain the sufficient condition under which the optimal closed-loop mode control is better than the optimal open-loop mode control.

Theorem 1 If \exists l\in S,X ⊂ Rⁿ and {\overline{p}}_{lj}\in Y_l, such that one of the following three conditions is satisfied, then {\text{ℒ}}_1^{*} is better than {\text{ℒ}}_2^{*}.

Π is an indefinite matrix;

Π is a non-zero positive semi-definite matrix, and Γ > 0;

Π is a non-zero negative semi-definite matrix, and Γ < 0.

K_i,q_i,L_i,\forall i\in S are solutions to Problem 3 when BoldItalic* is given.

Proof First, Eq. (10) does not constantly hold. Otherwise, from Lemma 2 we obtain that {\overline{p}}_{lj} is better than p_{lj}^{*}. This conclusion conflicts with the definition that BoldItalic* is the jump probability matrix corresponding to the optimal admissible control policy of Problem 2. Thus the solution to Eq. (10) is not Rⁿ space. Therefore, the Lemma 4 condition is equivalent to that for the optimal admissible control policy of Problem 2, {\text{ℒ}}_2^{*},\exists l\in S and {\overline{p}}_{lj}\in Y_l, such that Eq. (10) has a solution.

The matrix Π may be found in four cases: non-zero positive semi-definite, non-zero negative semi-definite, zero and indefinite. We will discuss these four cases respectively.

When condition 1) holds, i.e., Π is indefinite, then Eq. (10) always has a solution no matter what Γ is.

When condition 2) holds, i.e., Π is non-zero positive semi-definite, then we have 0 ≤ BoldItalic^TΠBoldItalic < +∞. Thus, Eq. (10) has a solution and does not hold constantly, which means Γ > 0.

When condition 3) holds, i.e., Π is non-zero negative semi-definite, then we have -∞ < BoldItalic^TΠBoldItalic ≤ 0. If Γ ≥ 0, then from Lemma 2 we obtain that {\overline{p}}_{lj} is better than p_{lj}^{*}, which conflicts with the definition of BoldItalic*. Therefore, Γ < 0.

If Π is zero, when Γ > 0, the solution of Eq. (10) is Rⁿ. And when Γ ≤ 0, Eq. (10) has no solution. Therefore, Π cannot be zero. Theorem 1 is proved.

Theorem 1 gives the sufficient condition under which the optimal closed-loop mode control is better than the optimal open-loop mode control. If this condition is satisfied, the closed-loop mode control achieves better performance than the open-loop mode control. This result is helpful for designing controllers. For the optimization problem of JLQG with controlled jump probabilities, seeking the optimal open-loop mode control is relatively simpler with a smaller policy space. However, seeking the optimal closed-loop mode control is rather difficult, since if the jump of modes depends on system state, the state feedback control law then cannot be solved by the standard JLQG model any more. The only existing methods are dynamic programming or simulation-based policy iteration approaches [15] which are time consuming. If the optimal open-loop mode control policy is obtained, we can apply the sufficient condition provided by Theorem 1 to check if there is any better closed-loop mode control policy. Thereafter, we can improve the jump probability of modes.

For a special case with two modes and a one-dimensional state, Theorem 1 can be simplified.

Corollary 1m = n = 1 and the mode set is S=\{1,2\}. If \exists l\in S, {\overline{p}}_{lj}\in Y_l, {\overline{p}}_{lj}\ne p_{lj}^{*}, such that

(11)\frac{\left({\sigma }_l^2{K}_2+q_2\right)-\left({\sigma }_l^2{K}_1+q_1\right)}{{\left(A_l-B_l{L}_l\right)}^2\left(K_1-K_2\right)}>0,

then {\text{ℒ}}_1^{*} is better than {\text{ℒ}}_2^{*}. K_i,q_i,L_i,\forall i\in S are the solutions to Problem 3 when BoldItalic* is given.

Proof In this case,

\Pi ={\left(A_l-B_l{L}_l\right)}^2\left({\overline{p}}_{l1}-p_{l1}^{*}\right)\left(K_1-K_2\right),

\text{Γ}=\left(p_{l1}^{*}-{\overline{p}}_{l1}\right)\left[\left({\sigma }_l^2{K}_1+q_1\right)-\left({\sigma }_l^2{K}_2+q_2\right)\right].

According to Theorem 1, it is easy to obtain the proof for Corollary 1.

Furthermore, we can solve the interval X. Solve the equation Πx² = Γ, we have

x_{+}=\sqrt{\frac{\left({\sigma }_l^2{K}_2+q_2\right)-\left({\sigma }_l^2{K}_1+q_1\right)}{{\left(A_l-B_l{L}_l\right)}^2\left(K_1-K_2\right)}},

(12)x_{-}=-\sqrt{\frac{\left({\sigma }_l^2{K}_2+q_2\right)-\left({\sigma }_l^2{K}_1+q_1\right)}{{\left(A_l-B_l{L}_l\right)}^2\left(K_1-K_2\right)}}.

If Eq. (11) holds, x₊ and x_- exist. If \left({\overline{p}}_{l1}-p_{l1}^{*}\right)\left(K_1-K_2\right)>0, we have X = [x_-,x₊]. If , the interval becomes X = (-∞,x_-] ∪ [x₊,+∞).

If the condition in Theorem 1 is satisfied, we should seek an admissible control policy of Problem 1 to improve system performance. Seeking {\text{ℒ}}_1^{*} is rather difficult. As a compromise, we tried to find a good admissible control policy for Problem 1, which improves system performance significantly compared with {\text{ℒ}}_2^{*}. All the proofs of the above lemmas and theorems are all constructive, thus we can construct a closed-loop mode control following the same idea. Combining the original feedback control law and the closed-loop mode control, we obtain a new policy, which is better than {\text{ℒ}}_2^{*} and BoldItalic*.

Algorithm 1 Construct an admissible control policy for Problem 1.

Solve the optimal admissible control policy of Problem 2, {\text{ℒ}}_2^{*}.

Check the condition contained in Theorem 1. When it is satisfied, there may exist more than one mode l\in S, and even in one Y_l there may exist several possible {\overline{p}}_{lj}\text{'}\text{s}. Then choose any mode l\in S, which satisfies the condition and corresponding X ⊂ Rⁿ and {\overline{p}}_{lj}\in Y_l.

Implement a one-step policy iteration for {\text{ℒ}}_2^{*}. Construct {{ℒ}^{\prime }}_1=\left\{{\text{ℒ}}_3^{*}\left(p^{*}\right),{p^{\prime }}_{ij}(x)\right\} by using Eq. (8).

Numerical example

Example 1 Consider a system with a one-dimensional state and two modes, S=\{1,2\}.

\begin{array}{l}{A}_1=0.5;B_1=2;M_1=1;N_1=10;{\sigma }_1=2;\\ A_2=1;B_2=1;M_2=2;N_2=10;{\sigma }_2=1;\\ Y_1=\{[0.80.2],[0.20.8]\},Y_2=\{[0.50.5]\}.\end{array}

First, find the optimal open-loop mode control. Therefore, we have

p^{*}=\left[\begin{array}{cc}0.8& 0.2\\ 0.5& 0.5\end{array}\right].

Then K₁ = 1.236, K₂ = 2.614, q₁ = 6.549, q₂ = 0.664, L₁ = 0.094, L₂ = 0.161. When l = 1, we have

\begin{array}{l}\left[\left({\sigma }_1^2{K}_2+q_2\right)-\left({\sigma }_1^2{K}_1+q_1\right)\right]/\left[{\left(A_1-B_1{L}_1\right)}^2\left(K_1-K_2\right)\right]\\ =2.772>0,\end{array}

thus the optimal admissible control policy of Problem 1, {\text{ℒ}}_1^{*}, is better than {\text{ℒ}}_2^{*}. According to Eq. (12), we have

X=[-1.665,1.665].

Using Algorithm 1, we can construct a better closed-loop mode control: When x ∈ X,

p^{\prime }(x)=\left[\begin{array}{cc}0.2& 0.8\\ 0.5& 0.5\end{array}\right];

otherwise, BoldItalic′(x) = BoldItalic*. Through simulation we have that the system performance under policy {\text{ℒ}}_2^{*} is 4.87, while the performance under constructed policy {{ℒ}^{\prime }}_1 is 4.06, which achieves a 17% improvement than {\text{ℒ}}_2^{*}.

Conclusions

The paper considers two classes of mode control policies, and gives the sufficient condition under which the optimal closed-loop mode control is better than the optimal open-loop mode control. When optimizing the JLQG model with controlled jump probabilities of modes, we can first find the optimal feedback control law and then the optimal open-loop mode control, after which we can check the condition that is sufficient. If it is satisfied, we can further seek better closed-loop mode control policies; otherwise, the open-loop mode control is good enough, and we can stop searching to avoid huge computations. If the optimal closed-loop mode control is difficult to obtain, we can apply Algorithm 1 to construct a closed-loop mode control, which significantly improves system performance. The numerical example verifies the efficiency of the proposed algorithm.

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant Nos. 60574064, 60736027).

References

1 ChengD Z, GuoY Q. Advances on switched systems. Control Theory and Applications, 2005, 22(6): 954–960(in Chinese)

2 Abou-kandilH, De SmetO, FreilingG, . Flow control in a failure-prone multi-machine manufacturing system. In: Proceedings of INRIA/IEEE Symposium on Emerging Technologies and Factory Automation. 1995, 2: 575–583

3 BoukasE K, ShiP, AndijaniA. Robust inventory-production control problem with stochastic demand. Optimal Control Applications and Methods, 1999, 20(1): 1–20

4 JiY, ChizeckH J. Controllability, stabilizability, and continuous-time Markovian jump linear quadratic control. IEEE Transactions on Automatic Control, 1990, 35(7): 777–788

5 CostaO, FragosoM D, MarquesR P. Discrete-Time Markov Jump Linear Systems. London: Springer-Verlag, 2005

6 XueF, GuoL. Necessary and sufficient conditions for adaptive stabilizability of jump linear systems. Communications in Information and Systems, 2001, 1(2): 205–224

7 ZhangL J, LiC W, ChengD Z. Robust adaptive control of Markov jump systems with parameter uncertainties. Control and Decision, 2005, 20(9): 1030–1033(in Chinese)

8 LiuF. Robust L₂-L_∞ filtering for uncertain jump systems. Control and Decision, 2005, 20(1): 32–35(in Chinese)

9 LiuF, ZhangX H. Robust control for jump systems with L₂ gain constraints. Control Theory and Applications, 2006, 23(3): 373–377(in Chinese)

10 LiuF, SuH Y, ChuJ. Robust positive real control of Markov jump systems with parametric uncertainties. Acta Automatica Sinica, 2003, 29(5): 761–766(in Chinese)

11 JiY, ChizeckH J. Optimal quadratic control of jump linear systems with separately controlled transition probabilities. International Journal of Control, 1989, 49(2): 481–491

12 BoukasE K, LiuZ K. Jump linear quadratic regulator with controlled jump rates. IEEE Transactions on Automatic Control, 2001, 46(2): 301–305

13 XuY K, ChenX. Discrete-time JLQG with dependently controlled jump probabilities. In: Proceedings of IEEE 22nd International Symposium on Intelligent Control, Singapore. 2007: 441–445

14 CaoX R. Stochastic Learning and Optimization: A Sensitivity-Based Approach. New York: Springer, 2007

15 ZhangK J, XuY K, ChenX, . Policy iteration based feedback control. Automatica, 2008, 44(4): 1055–1061