×

Quadratic costs and second moments of jump linear systems with general Markov chain. (English) Zbl 1248.93173

Summary: This paper presents an analytic, systematic approach to handle quadratic functionals associated with Markov jump linear systems with general jumping state. The Markov chain is finite state, but otherwise general, possibly reducible and periodic. We study how the second moment dynamics are affected by the additive noise and the asymptotic behaviour, either oscillatory or invariant, of the Markov chain. The paper comprises a series of evaluations that lead to a tight two-sided bound for quadratic cost functionals. A tight two-sided bound for the norm of the second moment of the system is also obtained. These bounds allow us to show that the long-run average cost is well defined for system that are stable in the mean square sense, in spite of the periodic behaviour of the chain and taking into consideration that it may not be unique, as it may depend on the initial distribution. We also address the important question of approximation of the long-run average cost via adherence of finite horizon costs.

MSC:

93E20 Optimal stochastic control
49N10 Linear-quadratic optimal control problems
93C05 Linear systems in control theory
60J05 Discrete-time Markov processes on general state spaces
74H55 Stability of dynamical problems in solid mechanics
74H40 Long-time behavior of solutions for dynamical problems in solid mechanics
74G45 Bounds for solutions of equilibrium problems in solid mechanics
60J75 Jump processes (MSC2010)
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Costa OLV, Fragoso MD (1993) Stability results for discrete-time linear systems with Markovian jumping parameters. J Math Anal Appl 179: 154–178 · Zbl 0790.93108 · doi:10.1006/jmaa.1993.1341
[2] Ji Y, Chizeck HJ (1990) Jump linear quadratic Gaussian control: steady-state solution and testable conditions. Control Theory Adv Technol 6(3): 289–319
[3] Costa OLV, Fragoso MD, Marques RP (2005) Discrete-time Markovian jump linear systems. Springer, New York
[4] Zampolli F (2006) Optimal monetary policy in a regime-switching economy: the response to abrupt shifts in exchange rate dynamics. J Econ Dynam Control 30: 1527–1567 · Zbl 1162.91508 · doi:10.1016/j.jedc.2005.10.013
[5] Costa OLV, de Paulo WL (2007) Indefinite quadratic with linear costs optimal control of Markov jump with multiplicative noise systems. Automatica 43: 587–597 · Zbl 1115.49021 · doi:10.1016/j.automatica.2006.10.022
[6] Khanbaghi M, Malhame RP, Perrier M (2002) Optimal white water and broke recirculation policies in paper mills via jump linear quadratic control. IEEE Trans Automat Control 10(4): 578–588
[7] Hernández-Lerma O, Lasserre JB (1996) Discrete-time Markov control processes: basic optimality criteria. Springer, New York
[8] Arapostathis A, Borkar VS, Fernández-Gaucherand E, Ghosh MK, Marcus SI (1993) Discrete-time controlled Markov processes with average cost criterion: a survey. SIAM J Control Optim 31(2): 282–344 · Zbl 0770.93064 · doi:10.1137/0331018
[9] Grimm G, Messina MJ, Tuna SE, Teel AR (2005) Model predictive control: for want of a local control Lyapunov function, all is not lost. IEEE Trans Automat Control 50: 546–558 · Zbl 1365.93263 · doi:10.1109/TAC.2005.847055
[10] Costa EF, do Val JBR (2009) Uniform approximation of infinite horizon control problems for nonlinear systems and stability of the approximating controls. IEEE Trans Automat Control 54(4): 881–886 · Zbl 1367.93195 · doi:10.1109/TAC.2008.2010970
[11] Costa EF, do Val JBR (2006) Obtaining stabilizing stationary controls via finite horizon cost. In: Proc. American Control Conference, Minneapolis, Minnesota, pp 4297–4302
[12] Jadbabaie A, Hauser J (2005) On the stability of receding horizon control with a general terminal cost. IEEE Trans Automat Control 50: 674–678 · Zbl 1365.93176 · doi:10.1109/TAC.2005.846597
[13] Costa EF, do Val JBR, Fragoso MD (2005) A new approach to detectability of discrete-time Markov jump linear systems. SIAM J Control Optim 43(6): 2132–2156 · Zbl 1116.93050 · doi:10.1137/S036301290342992X
[14] Çinlar E (1975) Introduction to stochastic processes. Prentice Hall, New York · Zbl 0341.60019
[15] Costa OLV, Fragoso MD (1995) Discrete-time LQ-optimal control problems for finite Markov jump parameters systems. IEEE Trans Automat Control 40: 2076–2088 · Zbl 0843.93091 · doi:10.1109/9.478328
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.