×

Countable state Markov processes: non-explosiveness and moment function. (English) Zbl 1379.60088

Summary: The existence of a moment function satisfying a drift function condition is well known to guarantee non-explosiveness of the associated minimal Markov process (cf. [W. J. Anderson, Continuous-time Markov chains. An applications-oriented approach. New York etc.: Springer-Verlag (1991; Zbl 0731.60067); S. P. Meyn and R. L. Tweedie, Adv. Appl. Probab. 25, No. 3, 518–548 (1993; Zbl 0781.60053)]), under standard technical conditions. Surprisingly, the reverse is true as well for a countable space Markov process. We prove this result by showing that recurrence of an associated jump process, that we call the \(\alpha\)-jump process, is equivalent to non-explosiveness. Non-explosiveness corresponds in a natural way to the validity of the Kolmogorov integral relation for the function identically equal to 1. In particular, we show that positive recurrence of the \(\alpha\)-jump chain implies that all bounded functions satisfy the Kolmogorov integral relation. We present a drift function criterion characterizing positive recurrence of this \(\alpha\)-jump chain. Suppose that to a drift function \(V\) there corresponds another drift function \(W\), which is a moment with respect to \(V\). Via a transformation argument, the above relations hold for the transformed process with respect to \(V\). Transferring the results back to the original process, allows to characterize the \(V\)-bounded functions that satisfy the Kolmogorov forward equation.

MSC:

60J27 Continuous-time Markov processes on discrete state spaces
PDF BibTeX XML Cite
Full Text: DOI

References:

[1] Anderson, W. J., Continuous-time Markov chains, (1991), New York: Springer-Verlag, New York · Zbl 0721.60081
[2] Avrachenkov, K., Piunovskiy, A., and Zhang, Y. (2013). Markov processes with restart. J. Appl. Prob. 50: 960-968.3161367 · Zbl 1295.60086
[3] Brémaud, P. (1999). Markov chains. Gibbs fields, Monte Carlo Simulation, and Queues. Number 31 in Texts in Applied Mathematics. New York: Springer-Verlag.1689633 · Zbl 0949.60009
[4] Chen, M. F., Coupling for jump processes, Acta Mathematica Sinica, 2, 2, 123-136, (1986) · Zbl 0615.60078
[5] Chen, M. F., From Markov Chains to Non-equilibrium Particle Systems, (1992), Singapore: World Scientific Publishing Co, Singapore
[6] Guo, X.P. and Hernández-Lerma, O. (2009). Continuous-time Markov decision processes. In Rozovskii, B. & Grimmett, G. (eds), Stochastic Modelling and Applied Probability, Vol. 62. Heidelberg: Springer-Verlag.
[7] Guo, X.P., Hernández-Lerma, O., and Prieto-Rumeau, T. (2006). A survey of recent results on continuous-time Markov decision processes. TOP14: 177-261. doi:10.1007/BF028375622286961 · Zbl 1278.90427
[8] Mertens, J.-F., Samuel-Cahn, E., and Zamir, S. (1978). Necessary and sufficient conditions for recurrence and transience of Markov chains in terms of inequalities. Journal of Applied Probability15: 848-851. doi:10.2307/3213440 · Zbl 0398.60068
[9] Meyn, S.P. and Tweedie, R.L. (1995). Stability of Markovian processes III: Foster-Lyapunov criteria for continuous time processes. Advances in Applied Probability25: 518-548. · Zbl 0781.60053
[10] Norris, J.R. (2004). Markov Chains. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge: Cambridge University Press.
[11] Piunovskiy, A. and Zhang, Y. (2012). The transformation method for continuous-time Markov decision processes. Journal of Optimization Theory and Applications154(2): 691-712. doi:10.1007/s10957-012-0015-8 · Zbl 1256.90048
[12] Prieto-Rumeau, T. and Hernández-Lerma, O. (2012). Selected Topics on Continuous-Time Controlled Markov Chains and Markov Games, ICP Advanced Texts in Mathematics. Vol. 5. London: Imperial College Press. · Zbl 1269.60004
[13] Prieto-Rumeau, T. and Hernández-Lerma, O. (2012). Uniform ergodicity of continuous-time controlled Markov chains: a survey and new results. Annals of Operations Research doi: doi:10.1007/s10479-012-1184-4. · Zbl 1386.60263
[14] Reuter, G. E.H., Denumerable Markov processes and the associated contraction semigroups on l, Acta Mathematica, 97, 1-46, (1957) · Zbl 0079.34703
[15] Spieksma, F.M. (2013). Kolmogorov forward equation and explosiveness in countable state Markov processes. Annals of Operations Research doi: doi:10.1007/s10479-012-1262-7. 24049244 · Zbl 1348.90621
[16] Tweedie, R. L., Some ergodic properties of the Feller minimal process, Journal of Mathematics, 25, 2, 485-493, (1974) · Zbl 0309.60046
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.