Quasi-stationary distributions. Markov chains, diffusions and dynamical systems.

*(English)*Zbl 1261.60002
Probability and Its Applications. Berlin: Springer (ISBN 978-3-642-33130-5/hbk; 978-3-642-33131-2/ebook). xv, 280 p. (2013).

Let \(Y=(Y_t,\operatorname{P}_x)\) be a continuous time Markov process with state space \(\mathcal X\). Denote by \(\partial \mathcal X\) the set of the so-called forbidden states, let \(T=T_{\partial \mathcal X}:=\inf\{t>0:\, Y_t\in \partial \mathcal X\}\) be the hitting time of \(\partial \mathcal X\), and assume that \(\operatorname{P}_x(T<\infty)=1\) for all \(x\in \mathcal X\setminus \partial \mathcal X\). If it is assumed that \(Y\) is killed at time \(T\), then, being non-conservative, the killed process clearly does not have a stationary distribution. As a replacement, one can consider a probability measure \(\nu\) on \(\mathcal X\setminus \partial \mathcal X\) having the property that, for all measurable \(B\subset \mathcal X\setminus \partial \mathcal X\) and all time \(t\geq 0\), it holds that \(\operatorname{P}_{\nu}(Y_t\in B \big| T>t)=\nu(B)\). Such probability measure \(\nu\) is called a quasi-stationary distribution (QSD). It explains the long term behavior of the almost surely killed process when it is conditioned to survive. A necessary condition for \(\nu\) to be a a QSD is the existence of (positive) exponential moments of time \(T\) under \(\operatorname{P}_{\nu}\) implying that the survival of the process (starting from a QSD) is exponentially distributed. A related concept is that of quasi-limiting distribution (or Yaglom limit): this is a probability measure \(\pi\) on \(\mathcal X\setminus \partial \mathcal X\) for which there exists a probability measure \(\nu\) satisfying \(\lim_{t\to \infty}\operatorname{P}_{\nu}(X_t\in \cdot \, \big|\, T>t)=\pi(\cdot)\). The measure \(\nu\) is then said to be in the domain of attraction of \(\pi\). Clearly, any QSD is also a quasi-limiting distribution.

The monograph under review is a thorough study of QSDs and related concepts for Markov chains, diffusions and dynamical systems. The authors focus on four essential problems, i.e., the existence of a QSD, the uniqueness of a QSD, the problem of domains of attraction, and the existence and classification of the process conditioned to survive forever.

The book starts with an instructive introduction, where the main concepts are explained for discrete-time finite state Markov chains by using Perron-Frobenius theory. The second chapter lays the ground for the rest of the book. Here, QSD is introduced in the general framework and necessary conditions for its existence are given.

The next three chapters are devoted to Markov chains in continuous time. Chapter 3 deals with the main problems of QSDs for irreducible Markov chains on finite state space. It is shown that there exists a unique QSD which is the normalized left Perron-Frobenius eigenvector of the jump-rates matrix. The process conditioned to survive forever is a Markov chain which is an \(h\)-process with \(h\) given by the right eigenvector. Markov chains on an infinite countable space \(\mathcal X\) are studied in Chapter 4. The main result shows that a necessary and sufficient condition for the existence of a QSD if the process does not come from infinity is the existence of exponential moments. The chapter also gives the infinitesimal characterization of QSDs (in terms of the jump-rates matrix), and sufficient conditions for an exponential rate of survival to coincide with exponential rates of transition probabilities (Kingman’s parameter). Chapter 5 is about birth-and-death chains on \(\mathbb Z_+\), where the forbidden state is \(0\) (i.e., \(T=\inf\{t>0: Y_t=0\}\)). The main result is a description of the structure of the set of QSDs: exponential killing implies the existence of a QSD. In case \(\infty\) is a natural boundary, there is a continuum of QSDs, while, if \(\infty\) is an entrance boundary, the QSD is unique. It is further shown that the extremal QSD is a quasi-limiting distribution, the survival process is studied and its classification is given, and the spectral methods are introduced. The chapter ends with several examples.

The focus in Chapters 6 and 7 is on diffusions on \(\mathcal X=[0,\infty)\) with 0 being the forbidden state. The diffusions studied are given by the SDE \(dY_t=dB_t-\alpha(Y_t)\, dt\), were \(B\) is the standard one-dimensional Brownian motion and \(\alpha\in C^1([0,\infty))\). In Chapter 6, the theory is developed under several assumptions, the main one being that \(\infty\) is a natural boundary for \(Y\). After looking at the finite interval case, the authors move to proving several fundamental results for the infinite interval case. These include the existence of the Yaglom limit, the existence of the process conditioned to survive and its classification. The spectral methods play a prominent role in this study. Again, the chapter ends with some examples. Chapter 7 is similar in spirit to the previous one, the main difference being that now \(\infty\) is assumed to be the entrance boundary (i.e., the process comes down from infinity) and the drift \(\alpha\) is allowed to have a singularity at zero. Such situation is different from the usually studied ones and invites spectral theory of semigroups as a tool. The main results are the existence of Yaglom limit and uniqueness of QSD. The whole chapter is based on the paper [P. Cattiaux et al., Ann. Probab. 37, No. 5, 1926–1969 (2009; Zbl 1176.92041)].

Finally, in the last chapter, the authors apply similar ideas to the case of dynamical systems with a trap in the state space. This case has many analogies with the stochastic one. A QSD in this context deals with trajectories that do not fall into the trap. The authors discuss Gibbs QSDs for symbolic systems and Pianigiani-Yorke QSDs.

The book provides a systematic approach to various topics related to quasi-stationary distributions. Although “it is not intended to be a complete exposition of the theory of QSDs”, it does cover many important parts of the field. The exposition of the chosen material is well-organized and proofs are mostly given in complete detail. Historical references are scattered throughout the text. The book will be very useful to researchers and graduate students who want to learn more about the subject, as well as to the experts in the field.

The monograph under review is a thorough study of QSDs and related concepts for Markov chains, diffusions and dynamical systems. The authors focus on four essential problems, i.e., the existence of a QSD, the uniqueness of a QSD, the problem of domains of attraction, and the existence and classification of the process conditioned to survive forever.

The book starts with an instructive introduction, where the main concepts are explained for discrete-time finite state Markov chains by using Perron-Frobenius theory. The second chapter lays the ground for the rest of the book. Here, QSD is introduced in the general framework and necessary conditions for its existence are given.

The next three chapters are devoted to Markov chains in continuous time. Chapter 3 deals with the main problems of QSDs for irreducible Markov chains on finite state space. It is shown that there exists a unique QSD which is the normalized left Perron-Frobenius eigenvector of the jump-rates matrix. The process conditioned to survive forever is a Markov chain which is an \(h\)-process with \(h\) given by the right eigenvector. Markov chains on an infinite countable space \(\mathcal X\) are studied in Chapter 4. The main result shows that a necessary and sufficient condition for the existence of a QSD if the process does not come from infinity is the existence of exponential moments. The chapter also gives the infinitesimal characterization of QSDs (in terms of the jump-rates matrix), and sufficient conditions for an exponential rate of survival to coincide with exponential rates of transition probabilities (Kingman’s parameter). Chapter 5 is about birth-and-death chains on \(\mathbb Z_+\), where the forbidden state is \(0\) (i.e., \(T=\inf\{t>0: Y_t=0\}\)). The main result is a description of the structure of the set of QSDs: exponential killing implies the existence of a QSD. In case \(\infty\) is a natural boundary, there is a continuum of QSDs, while, if \(\infty\) is an entrance boundary, the QSD is unique. It is further shown that the extremal QSD is a quasi-limiting distribution, the survival process is studied and its classification is given, and the spectral methods are introduced. The chapter ends with several examples.

The focus in Chapters 6 and 7 is on diffusions on \(\mathcal X=[0,\infty)\) with 0 being the forbidden state. The diffusions studied are given by the SDE \(dY_t=dB_t-\alpha(Y_t)\, dt\), were \(B\) is the standard one-dimensional Brownian motion and \(\alpha\in C^1([0,\infty))\). In Chapter 6, the theory is developed under several assumptions, the main one being that \(\infty\) is a natural boundary for \(Y\). After looking at the finite interval case, the authors move to proving several fundamental results for the infinite interval case. These include the existence of the Yaglom limit, the existence of the process conditioned to survive and its classification. The spectral methods play a prominent role in this study. Again, the chapter ends with some examples. Chapter 7 is similar in spirit to the previous one, the main difference being that now \(\infty\) is assumed to be the entrance boundary (i.e., the process comes down from infinity) and the drift \(\alpha\) is allowed to have a singularity at zero. Such situation is different from the usually studied ones and invites spectral theory of semigroups as a tool. The main results are the existence of Yaglom limit and uniqueness of QSD. The whole chapter is based on the paper [P. Cattiaux et al., Ann. Probab. 37, No. 5, 1926–1969 (2009; Zbl 1176.92041)].

Finally, in the last chapter, the authors apply similar ideas to the case of dynamical systems with a trap in the state space. This case has many analogies with the stochastic one. A QSD in this context deals with trajectories that do not fall into the trap. The authors discuss Gibbs QSDs for symbolic systems and Pianigiani-Yorke QSDs.

The book provides a systematic approach to various topics related to quasi-stationary distributions. Although “it is not intended to be a complete exposition of the theory of QSDs”, it does cover many important parts of the field. The exposition of the chosen material is well-organized and proofs are mostly given in complete detail. Historical references are scattered throughout the text. The book will be very useful to researchers and graduate students who want to learn more about the subject, as well as to the experts in the field.

Reviewer: Zoran Vondraček (Zagreb)

##### MSC:

60-02 | Research exposition (monographs, survey articles) pertaining to probability theory |

60J10 | Markov chains (discrete-time Markov processes on discrete state spaces) |

60J27 | Continuous-time Markov processes on discrete state spaces |

60J60 | Diffusion processes |

60J65 | Brownian motion |

60J80 | Branching processes (Galton-Watson, birth-and-death, etc.) |

93E03 | Stochastic systems in control theory (general) |

92Dxx | Genetics and population dynamics |

37Dxx | Dynamical systems with hyperbolic behavior |

35Pxx | Spectral theory and eigenvalue problems for partial differential equations |