Part III Applications in Systems and Control
11.2 Stochastic Stability for Markovian Antilinear Systems
In previous section, the stability of ordinary antilinear systems has been investigated.
In this section, the stability is considered for the following discrete-time Markovian jump (DTMJ) antilinear system
x(t+1)=Ar(t)x(t),x(0)=x0, (11.13) wherex(t)∈Cnis the state of the system with the initial conditionx(0)=x0, and r(t)represents a homogeneous discrete-time Markovian chain that takes values in a
discrete finite setS = {1, 2,. . .,N}with transition probability matrixP= pi j
N×N
given by
pi j =Pr{r(t+1)= j|r(t)=i}. (11.14) The setScontainsNmodes of the system (11.13), and forr(t)=i ∈S, the system matrix of thei-th mode is assumed to be given and denoted byAi. Similar to the case of linear systems with Markovian jump parameters [27], the concept of stochastic stability is given below for the DTMJ antilinear system (11.13).
Definition 11.4 The DTMJ antilinear system (11.13) with transition probability (11.14) is called stochastically stable, if for all finite x0 ∈ Cn andi ∈ S, there exists a positive definite matrixPsatisfying
E ∞
t=0
xH(t,x0,i)x(t,x0,i)|x(0)=x0, r(0)=i
≤x0HP x 0, (11.15)
wherex(t,x0,i)represents the corresponding solution of the system (11.13) at time t when the initial conditions ofx(t)andr(t)are respectively taken asx(0)= x0
andr(0)=i.E represents the mathematical expectation.
Before proceeding, some well-known properties are given for Hermitian matrices.
Lemma 11.1 Let A∈Cn×n be a positive definite Hermitian matrix. Then (1) xHAx is real for all x ∈Cn. Further, xHAx=xTAx.
(2) xHAx >0for all non-zero x∈Cn.
The main aim of this section is to derive some necessary and sufficient conditions for the DTMJ antilinear system (11.13)–(11.14) to be stochastically stable. By using a stochastic version of Lyapunov’s second method, criteria are derived for the stochastic stability of a Markovian jump antilinear system.
Theorem 11.2 The DTMJ antilinear system (11.13) with transition probability (11.14) is stochastically stable if and only if, for any given positive definite Hermitian matrices Qi, i ∈I[1,N], there exist unique positive definite Hermitian matrices Ki, i ∈I[1,N], such that the following N coupled equations hold
AiH
⎛
⎝N
j=1
pi jKj
⎞
⎠Ai−Ki = −Qi, i∈I[1,N]. (11.16)
Proof For the DTMJ antilinear system (11.13), the following stochastic Lyapunov function is considered
V(x(t),r(t)=i)=V(x(t),i)=xH(t)Kix(t), (11.17)
where Ki is a positive definite Hermitian matrix. By using Lemma 11.1, one can see that V(x(t),r(t) = i)is real and V(x(t),r(t) = i) > 0 for x(t) = 0. Let V(x(t),i)be defined as
V(x(t),i)=E [V(x(t+1),r(t+1))|x(t),r(t)=i]−V(x(t),i). (11.18) By the properties of conditional expectation, from (11.17) and (11.18) one can obtain that
V(x(t),i)=
⎛
⎝N
j=1
pi j Aix(t)H
KjAix(t)
⎞
⎠−xH(t)Kix(t).
By using the first item of Lemma11.1, one yields
V(x(t),i)= N
j=1
pi j Aix(t)H
KjAix(t)−xH(t)Kix(t)
= N
j=1
pi j
Aix(t)H
KjAix(t)−xH(t)Kix(t)
=xH(t)ATi
⎛
⎝N
j=1
pi jKj
⎞
⎠Aix(t)−xH(t)Kix(t)
=xH(t)
⎡
⎣ATi
⎛
⎝N
j=1
pi jKj
⎞
⎠Ai−Ki
⎤
⎦x(t)
= −xH(t)Qix(t).
With this relation, it can be obtained that forx(t)=0, there holds V(x(t),i)
V(x(t),i) = −xH(t)Qix(t)
xH(t)Kix(t) ≤ −γ = −min
i∈S
λmin[Qi]
λmax[Ki], (11.19) where λmin[Qi]denotes the minimal eigenvalue of Qi, andλmax[Ki] denotes the maximal eigenvalue ofKi.
With the relation (11.19), one has
V(x(t),i)≤ −γV(x(t),i). (11.20) According to the definition (11.18) ofV(x(t),i), from (11.20), one can get
E [V(x(t+1),r(t+1))|x(t),r(t)=i]≤(1−γ )[V(x(t),i)].
With this relation, by using the properties of conditional expectation one has E [V(x(t+1),r(t+1))]
=E{E[V(x(t+1),r(t+1))|x(t),r(t)=i]} (11.21)
≤(1−γ )EV(x(t),i).
Here, note thatγ should satisfy 0≤1−γ <1.Suppose thatγ >1, it implies that E [V(x(t+1),r(t+1))]<0,
which contradicts the fact thatKi,i ∈S, are positive definite Hermitian matrices.
Since the relation (11.21) holds for anyi ∈S, one has
E [V(x(t+1),r(t+1))]≤(1−γ )E[V(x(t),r(t))]. (11.22) It follows from (11.22) that
E [V(x(t+1),r(t+1))]≤(1−γ )t+1E[V(x0,r(0))].
From this, one has
E [V(x(t+1),r(t+1)) |x0,r(0)]≤(1−γ )t+1V(x0,r(0)).
Sinceγ <1, summing both sides of the preceding expression from 0 to∞, gives E
∞
t=0
xH(t)Kr(t)x(t)|x0,r(0)=i
≤∞
t=0
(1−γ )tx0HKix0 = 1
γx0HKix0. (11.23) SinceKiis a positive definite Hermitian matrix for eachi∈ S, then
E ∞
t=0
xH(t)Kr(t)x(t)|x0,r(0)=i
≥E ∞
t=0
λmin
Kr(t)
xH(t)x(t)|x0,r(0)=i
≥min
j∈S λmin
Kj
E ∞
t=0
xH(t)x(t)|x0,r(0)=i
. (11.24)
Combining (11.23) with (11.24), yields minj∈S λmin
Kj
E ∞
t=0
xH(t)x(t)|x0,r(0)=i
≤ 1
γx0HKix0.
Thus, it can be concluded that for alli ∈S E
∞
t=0
xH(t)x(t)|x0,r(0)=i
≤x0HP x 0, (11.25)
where Pis a positive definite Hermitian matrix satisfying P=max
i,j∈S
Ki
γ λmin
Kj
.
The sufficiency is thus proven.
Now, let us show the necessity. It is assumed that the DTMJ antilinear system (11.13) is stochastically stable, that is, for anyx0∈Cnand anyi∈ S, there exists a positive definite Hermitian matrix Psatisfying
E ∞
t=0
xH(t)x(t)|x(0)=x0,r(0)=i
≤x0HP x 0. (11.26)
Let{Ri :i ∈S}be a set of positive definite matrices. Define the following function
xH(t)K(L−t,r(t))x(t)=E L
k=t
xH(k)Rr(k)x(k)|x(t),r(t)
, whereL is a positive integer.
SinceRr(k),k∈I[t,L], are positive definite Hermitian matrices, thenxH(t)K(L− t,r(t))x(t)is a non-decreasing function ofL.It can be obtained from (11.26) that xH(t)K(L−t,r(t))x(t)is bounded from above. Hence, one can find a matrixKr(t)
such that
xH(t)Kr(t)x(t)= lim
L→∞xH(t)K(L−t,r(t))x(t) (11.27)
= lim
L→∞E
L
k=t
xH(k)Rr(k)x(k)|x(t),r(t)
. Since (11.27) holds for anyr(t)=i,one has
Ki= lim
L→∞K(L−t,r(t)=i). (11.28) Note thatRiis a positive definite Hermitian matrix, it can be concluded from (11.27) thatKi is also positive definite. Now let us consider
E
xH(t)K(L−t,r(t))x(t)−
xH(t+1)K(L−t−1,r(t+1))x(t+1)|x(t),r(t)
=E
E L
k=t
xH(k)Rr(k)x(k)|x(t),r(t)
− E
L
k=t+1
xH(k)Rr(k)x(k)|x(t+1),r(t+1)
|x(t),r(t)
. For the right-hand side of the preceding equation, one has
E
E L
k=t
xH(k)Rr(k)x(k)|x(t),r(t)
− E
L
k=t+1
xH(k)Rr(k)x(k)|x(t+1),r(t+1)
|x(t),r(t)
=xH(t)Rr(t)x(t).
In addition, by using the definition of conditional expectation, one can obtain that E
xH(t)K(L−t,r(t))x(t)−xH(t+1)K(L−t−1,r(t+1))x(t+1)|x(t),r(t)=i
=xH(t)K(L−t,r(t)=i)x(t)− Ar(t)=ix(t)H
⎛
⎝N
j=1
pi jK(L−t−1,j)
⎞
⎠ Ar(t)=ix(t) .
It follows from the preceding three relations that for anyx(t)there holds
xH(t)K(L−t,r(t)=i)x(t)− Ar(t)=ix(t)H
⎛
⎝N
j=1
pi jK(L−t−1,j)
⎞
⎠ Ar(t)=ix(t)
(11.29)
=xH(t)Rr(t)=ix(t).
LetL−t → ∞, then from (11.29) one has
Aix(t)H⎛
⎝N
j=1
pi jKj
⎞
⎠ Aix(t)
−xH(t)Kix(t)= −xH(t)Rix(t). (11.30)
From the first item of Lemma11.1, it can be obtained that
Aix(t)H
⎛
⎝N
j=1
pi jKj
⎞
⎠ Aix(t)
=xH(t)AiT
⎛
⎝N
j=1
pi jKj
⎞
⎠Aix(t).
Substituting it into (11.30), gives
xH(t)
⎡
⎣ATi
⎛
⎝N
j=1
pi jKj
⎞
⎠Ai−Ki
⎤
⎦x(t)= −xH(t)Rix(t),
for anyi∈ S. Since this relation holds for anyx(t), one can conclude that
ATi
⎛
⎝N
j=1
pi jKj
⎞
⎠Ai−Ki = −Ri. (11.31)
LetQi =Ri >0,i ∈I[1,N]. Then, (11.31) can be equivalently rewritten as (11.16).
Hence,{Ki : i ∈ S}is the positive definite Hermitian solution of (11.16). By (11.27), (11.28) and the relation Qi = Ri, it can be easily obtained that Ki is uniquely defined by Qi. Since (11.16) is derived from (11.27), Ki is the unique
solution of (11.16). Thus, the proof is completed.
Obviously, the coupled matrix equation (11.16) is not linear. In order to distinguish from the coupled Lyapunov equations appearing in DTMJ linear systems, the coupled equation (11.16) is referred to as the coupled anti-Lyapunov matrix equation.
Remark 11.1 The coupled anti-Lyapunov matrix equation in (11.16) can be equiv- alently written as
√ piiAi
H
Ki√
piiAi−Ki = −AHi
⎛
⎝ N
j=i,j=1
pi jKj
⎞
⎠Ai−Qi,i ∈I[1,N].
By the result in Sect.11.1, it can be obtained from this relation that a necessary condition of the DTMJ antilinear system (11.13) to be stochastically stable is that the following discrete-time antilinear systems are stable:
x(t+1)=√
piiAix(t), i∈I[1,N]. (11.32) Thus, in view of 0≤ pii ≤1 the stability of all subsystems is not necessary for the system (11.13) to be stochastically stable.
Example 11.1 Consider a one-dimensional Markovian jump antilinear system x(t+1)=Ar(t)x(t),x(0)=x0,
where the dynamic process is a 2-state Markov chain with the following transition probability matrix
P=
0.3 0.7 0.5 0.5
,
and the subsystem matrices are
A1= −1+0.5i,A2=0.5−0.5i.
ChooseQi =1,i =1, 2. Then, it is easily obtained that the solution of anti-Lyapunov matrix equation (11.16) is(K1,K2)=(6.5,3.5), which are positive definite. Hence, according to Theorem 11.2, this Markovian jump antilinear system is stochasti- cally stable. On the other hand, one can take advantage of the stability condition in Sect.11.1for antilinear systems to check the stability of these two subsystems.
As a result, the subsystem with the system matrix A1 is not stable while the other subsystem is stable. Thus, it can be seen that the stability of all subsystems is not necessary for the system (11.13) to be stochastically stable.
The following result provides another necessary and sufficient condition for the stochastic stability of Markovian jump antilinear systems. In comparison with The- orem11.2, a different Lyapunov function is chosen in this result.
Theorem 11.3 The DTMJ antilinear system (11.13) with transition probability (11.14) is stochastically stable if and only if, for any given positive definite Hermitian matrices Qi, i ∈I[1,N], there exist unique positive definite Hermitian matrices Ki, i ∈I[1,N], satisfying the following coupled anti-Lyapunov matrix equation:
N j=1
pi jAHjKjAj−Ki = −Qi, i∈I[1,N]. (11.33)
Proof For the Markovian jump antilinear system (11.13), the following stochastic Lyapunov function is considered
V(x(t),r(t−1)=i)=V(x(t),i)=xH(t)Kix(t), (11.34) withKi being Hermitian and positive definite. The proof is very similar to that of Theorem11.2. LetV(x(t),i)be defined as
V(x(t),i)=E [V(x(t+1),r(t))|x(t),r(t−1)=i]−V(x(t),i). (11.35) By the definition of conditional expectation, it follows from (11.34) and (11.35) that
V(x(t),i)= N
j=1
pi j Ajx(t)H
Kj Ajx(t)
−xH(t)Kix(t)
= N
j=1
pi jx(t)HAHjKjAjx(t)−xH(t)Kix(t).
By using the first item of Lemma11.1, one has
V(x(t),i)= N
j=1
pi jx(t)HAHjKjAjx(t)−xH(t)Kix(t)
=xH(t)
⎛
⎝N
j=1
pi jATjKjAj
⎞
⎠x(t)−xH(t)Kix(t)
=xH(t)
⎡
⎣N
j=1
pi jATjKjAj−Ki
⎤
⎦x(t)
= −xH(t)Qix(t),
from which, it can be obtained that forx(t)=0, there holds V(x(t),i)
V(x(t),i) = −xH(t)Qix(t)
xH(t)Kix(t) ≤ −γ = −min
i∈S
λmin[Qi]
λmax[Ki]. (11.36) From (11.36), it can be derived that
V(x(t),i)≤ −γV(x(t),i). (11.37) With the definition (11.35) ofV(x(t),i), it follows from (11.37) that
E[V(x(t+1),r(t))|x(t),r(t−1)=i]≤(1−γ )V(x(t),i). With this relation, by using the properties of conditional expectation one has
E [V(x(t+1),r(t))]
=E{E [V(x(t+1),r(t))|x(t),r(t−1)=i]} (11.38)
=(1−γ )E [V(x(t),i)] .
Similarly to the proof of the preceding theorem, it can be obtained that 0≤1−γ <1.
Since the relation (11.38) holds for anyi ∈S, one has
E [V(x(t+1),r(t))|x(t),r(t−1)]≤(1−γ )E[V(x(t),r(t−1))]. (11.39) Repeating this iteration, it can be obtained that
E [V(x(t+1),r(t))]≤(1−γ )tE[V(x(1),r(0))].
From this, one has
E [V(x(t+1),r(t))|x(1),r(0)]≤(1−γ )t[V(x(1),r(0))].
Summing both sides of the preceding expression from 1 to∞and using the fact that γ <1, yields
E ∞
t=1
xH(t)Kr(t−1)x(t)|x(1),r(0)
≤∞
t=1
(1−γ )t−1xH(1)Kr(0)x(1) (11.40)
= 1
γxH(1)Kr(0)x(1).
By the first item of Lemma11.1, it follows from (11.40) and (11.13) that E
∞
t=1
xH(t)Kr(t−1)x(t)|x(0)=x0,r(0)=i
≤ 1
γ (Aix0)HKi(Aix0)
= 1
γ(Aix0)HKi(Aix0)
= 1
γx0HATiKiAix0.
SinceKiis a positive definite Hermitian matrix for eachi∈ S, then E
∞
t=1
xH(t)Kr(t−1)x(t)|x(0)=x0,r(0)=i
≥E ∞
t=1
λmin
Kr(t−1)
xH(t)x(t)|x(0)=x0,r(0)=i
≥min
j∈S λmin
Kj
E ∞
t=1
xH(t)x(t)|x(0)=x0,r(0)=i
. From the preceding two relations, one has
minj∈S λmin
Kj
E ∞
t=1
xH(t)x(t)|x(0)=x0,r(0)=i
≤ 1
γx0HATiKiAix0.
Thus, one can obtain that for alli ∈ S E
∞
t=0
xH(t)x(t)|x(0)=x0,r(0)=i
≤x0HP x0,
where Pis a positive definite matrix defined as below P=I+max
i,j∈S
ATiKiAi
γ λmin
Kj
.
This shows the sufficiency.
Next, the necessity will be shown. It is assumed that the Markovian jump antilinear system (11.13) is stochastically stable, that is, for anyx0 ∈Cnand anyi ∈S, there exists a positive definite Hermitian matrix Psatisfying
E ∞
t=0
xH(t)x(t)|x(0)=x0,r(0)=i
≤x0HP x 0. (11.41)
Let{Ri :i ∈S}be a set of positive definite Hermitian matrices. Define the following function
xH(t)K(L−t,r(t−1))x(t)=E L
k=t
xH(k)Rr(k−1)x(k)|x(t),r(t−1)
,
whereLis a positive integer. Obviously,xH(t)K(L−t,r(t−1))x(t)is a nondecreas- ing function ofL.It can be obtained from (11.41) thatxH(t)K(L−t,r(t−1))x(t) is bounded from above. Hence, one can find a matrixKr(t−1)such that
xH(t)Kr(t−1)x(t)
= lim
L→∞xH(t)K(L−t,r(t−1))x(t) (11.42)
= lim
L→∞E
L
k=t
xH(k)Rr(k−1)x(k)|x(t),r(t−1)
. Since (11.42) holds for anyr(t−1)=i, it can be derived that
Ki= lim
L→∞K(L−t,r(t−1)=i). (11.43) Since {Ri : i ∈ S}is a set of positive definite matrices, it can be obtained from (11.42) thatKiis also Hermitian and positive definite. Now the following relation is considered:
E
xH(t)K(L−t,r(t−1))x(t)−
xH(t+1)K(L−t−1,r(t))x(t+1)|x(t),r(t−1)
=E
E L
k=t
xH(k)Rr(k−1)x(k)|x(t),r(t−1)
− E
L
k=t+1
xH(k)Rr(k−1)x(k)|x(t+1),r(t)
|x(t),r(t−1)
.
For the expression on the right-hand side of the preceding relation, it can be readily obtained that
E
E L
k=t
xH(k)Rr(k−1)x(k)|x(t),r(t−1)
− E
L
k=t+1
xH(k)Rr(k−1)x(k)|x(t+1),r(t)
|x(t),r(t−1)
=xH(t)Rr(t−1)x(t).
By using the properties of conditional expectation, it follows from (11.13) that
E
xH(t)K(L−t,r(t−1))x(t)−xH(t+1)K(L−t−1,r(t))x(t+1)|x(t),r(t−1)=i
=xH(t)K(L−t,r(t−1)=i)x(t)− N j=1
pi j Ar(t)=jx(t)H
K(L−t−1,r(t)=j) Ar(t)=jx(t) .
Combining the preceding three relations, gives xH(t)K(L−t,r(t−1)=i)x(t)−
N j=1
pi j Ajx(t)H
K(L−t−1,r(t)= j) Ajx(t)
=xH(t)Rr(t−1)=ix(t) (11.44)
for anyx(t). For (11.44), letL→ ∞, then one has N
j=1
pi j Ajx(t)H
Kj Ajx(t)
−xH(t)Kix(t)= −xH(t)Rix(t). (11.45)
By the first item of Lemma11.1, it can be obtained that N
j=1
pi j Ajx(t)H
Kj Ajx(t)
=xH(t)
⎛
⎝N
j=1
pi jATjKjAj
⎞
⎠x(t).
Thus, it follows from (11.45) that N
j=1
pi jATjKjAj−Ki= −Ri, (11.46)
for anyi ∈ S. Let Qi = Ri > 0,i ∈ I[1, N]. Then, (11.46) can be equivalently rewritten as (11.33).
Hence,{Ki :i ∈ S}is a set of positive definite Hermitian solutions of (11.33).
By (11.42), (11.43), and the relationQi = Ri, it can be easily obtained thatKi is uniquely determined byQi. Since (11.16) is derived from (11.27),Kiis the unique
solution of (11.33). Thus, the proof is completed.
The results of the previous two theorems are based on the choice of a Lyapunov function by using a stochastic version of Lyapunov’s second method. A natural candidate for a Lyapunov function is an appropriately chosen quadratic form. In the proof of Theorem 11.2, the Lyapunov function is chosen asV(x(t),r(t)) = xH(t)Kr(t)x(t), where x(t)is available with respect to r(t) and the matrix Kr(t)
depends only on r(t). On the other hand, Theorem 11.3 is proved by using the Lyapunov functionV(x(t),r(t))=xH(t)Kr(t−1)x(t). In this case, the matrixKr(t−1)
depends only onr(t−1).
Remark 11.2 The necessary and sufficient conditions given in (11.16) and (11.33) are equivalent.
Remark 11.3 By analyzing the coupled anti-Lyapunov equation in (11.33), a similar conclusion as in Remark11.1can also be obtained.
Remark 11.4 For the case of DTMJ linear systems, the involved coupled Lyapunov matrix equations are linear, and thus the stochastic stability can be checked by the spectral radius of an augmented matrix being less than one [124]. Since the coupled anti-Lyapunov matrix equations are not linear, one can not resort such a method to check the stochastic stability for a DTMJ antilinear system.
Theorem11.3provides an alternative criterion based on coupled anti-Lyapunov equation for the stochastic stability of the DTMJ antilinear system (11.13). Since the conditions in Theorems11.2and11.3are necessary and sufficient for the system (11.13) to be stochastically stable, there may exist a link between solutions of the coupled anti-Lyapunov equations (11.16) and (11.33). But it is a pity that currently, such a relation can not be established directly. In addition, it is not obvious which of these two necessary and sufficient conditions is better for practical applications.
In general, one has to solve N coupled anti-Lyapunov equations when Theorem 11.2or Theorem11.3is applied. However, for some special cases, compared with Theorems11.2and11.3actually provides an easier test for checking the stochastic stability of the DTMJ antilinear system. This is shown in the following corollary by using Theorem11.3.
Corollary 11.1 Suppose that{r(t)}is a finite-state independent and identically dis- tributed (i.i.d.) random sequence with the probability distribution{p1, p2,. . ., pN}.
Then the DTMJ antilinear system (11.13) is stochastically stable if and only if, for any given positive definite Hermitian matrix Q, there exists a unique positive definite Hermitian matrix K such that the following matrix equation holds:
N j=1
pjAHjK Aj−K = −Q.
Remark 11.5 For i.i.d. case, if the necessary and sufficient condition in Theorem 11.2is adopted, one needs to solve coupled anti-Lyapunov equations. This is more complicated than the method in Corollary11.1.