zbMATH — the first resource for mathematics

Topics in the constructive theory of countable Markov chains. (English) Zbl 0823.60053
Cambridge: Cambridge University Press. 169 p. (1995).
The monograph contains seven chapters. In the first chapter the authors introduce basic notions and results from the theory of discrete time homogeneous countable Markov chains. At the end of this chapter classical examples are presented. In the second chapter several criteria for general countable Markov chains are given and proved. The authors study also criteria involving semi-martingales. In the third chapter they offer techniques for an explicit geometrical construction of Lyapunov functions. In Section 3.1 the simplest case is introduced. In Section 3.2 the principal definition of a space homogeneous random walk in \(\mathbb Z^ N_ +\) is given. The classification for \(N = 2\) is presented in Sections 3.3 and 3.4. A special type of random walk in \(\mathbb Z^ N_ +\) is obtained by Jackson networks for which necessary and sufficient conditions of ergodicity are proved, by constructing explicit Lyapunov functions. The zero drift case in \(\mathbb Z^ 2_ +\) and almost zero drift one-dimensional examples constitute new directions of development introduced by J. Lamperti (1960). In the fourth chapter the authors get sufficient conditions for a random walk to have a Lyapunov function satisfying the Foster criteria. “Random walks in two-dimensional complexes” is the title of the fifth chapter. Transience is proved by using a method to construct global Lyapunov functions. The sixth chapter, entitled “Stability”, treats the continuity of stationary distributions for families of homogeneous irreducible and aperiodic Markov chains. In chapter seven probabilistic criteria using Lyapunov functions and Foster’s theorems are presented for a family of Markov chains to be an analytic family.

60Jxx Markov processes
60-02 Research exposition (monographs, survey articles) pertaining to probability theory
Full Text: DOI