×

Semi-additive functionals of semi-Markov processes and measure-valued Poisson equation. (English) Zbl 1407.60118

Summary: In this paper, we consider a continuous-time semi-Markov process (SMP) in Polish spaces. We first introduce the semi-additive functional in semi-Markov cases, a natural generalization of the additive functional of Markov process (MP). The main contribution of this paper is the properties of semi-additive functional aforementioned. First, semi-additive functionals of SMPs are characterized in terms of a càdlàg function with zero initial value and a measurable function. Second, the necessary and sufficient conditions are investigated under which a semi-additive functional of SMP is a semimartingale, a local martingale, or a special semimartingale respectively. By the way, the Itô type formula is given. Finally, we study the expected cumulative discounted value of the semi-additive functional of an SMP. We prove that it solves a measure-valued Poisson equation and give the uniqueness conditions.

MSC:

60K15 Markov renewal processes, semi-Markov processes
60J55 Local time and additive functionals
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Bhattacharya, R. N.; Majumdar, M., Controlled semi-markov models - the discounted case, J. Statist. Plann. Inference, 21, 3, 365-381 (1989) · Zbl 0673.93089
[2] Boucherie, R. J.; van Dijk, N. M., Markov Decision Processes in Practice (2017), Springer · Zbl 1369.90001
[3] Cao, X. R., Semi-Markov decision problems and performance sensitivity analysis, IEEE Trans. Automat. Control, 48, 5, 758-769 (2003) · Zbl 1364.90344
[4] Cherenkov, A. P., Additive functionals of semi-Markov processes with absorption, Math. Notes Acad. Sci. USSR, 17, 2, 189-194 (1975) · Zbl 0361.60078
[5] Cherenkov, A. P., Asymptote of additive functionals of semi-markov processes with arbitrary sets of states, Math. Notes Acad. Sci. USSR, 21, 2, 119-124 (1977) · Zbl 0376.60089
[6] Dai, G.; Yin, B.; Li, Y.; Xi, H., Performance optimization algorithms based on potentials for semi-Markov control processes, Internat. J. Control, 78, 11, 801-812 (2005) · Zbl 1121.90414
[7] Glynn, P. W.; Meyn, S. P., A liapounov bound for solutions of the poisson equation, Ann. Probab., 24, 2, 916-931 (1996) · Zbl 0863.60063
[8] Grabski, F., Semi-Markov Processes: Applications in System Reliability and Maintenance (2015), Elsevier · Zbl 1326.60006
[9] Guo, X. P.; Hernández-Lerma, O., Constrained continuous-time markov control processes with discounted criteria, Stoch. Anal. Appl., 21, 2, 379-399 (2003) · Zbl 1099.90071
[10] Guo, X. P.; Hernández-Lerma, O., Continuous-Time Markov Decision Processes: Theory and Applications (2009), Springer New York · Zbl 1209.90002
[11] Harlamov, B. P., Stochastic integral with respect to a semi-Markov process of diffusion type, J. Math. Sci., 139, 3, 6643-6656 (2006)
[12] Harlamov, B., Continuous Semi-Markov Processes (2008), John Wiley and Sons, New York · Zbl 1156.60004
[13] Huang, Y. H.; Guo, X. P., First passage models for denumerable semi-markov decision processes with nonnegative discounted costs, Acta Math. Appl. Sin. Engl. Ser., 27, 2, 177-190 (2011) · Zbl 1235.90177
[14] Jacod, J.; Skorokhod, A. V., Jumping markov processes, Ann. Inst. H. Poincaré Probab. Statist., 32, 1, 11-67 (1996) · Zbl 0841.60066
[15] Janssen, J.; Limnios, N., Semi-Markov Models and Applications (1999), Springer US · Zbl 0937.00015
[16] Koroliuk, V. S.; Limnios, N., Stochastic Systems in Merging Phase Space (2005), World Scientific: World Scientific Singapore · Zbl 1101.60003
[17] Lévy, P., Processus semi-Markoviens, (Proceedings of the International Congress of Mathematicians (1954), North-Holland), 416-426
[18] Limnios, N.; Oprişan, G., Semi-Markov Processes and Reliability (2001), Birkhäuser Boston · Zbl 0990.60004
[19] Neveu, J., Potentiel Markovien récurrent des chaînes de Harris., Ann. Inst. Fourier, 22, 2, 85-130 (1972) · Zbl 0226.60084
[20] Nummelin, E., On the Poisson equation in the potential theory of a single kernel, Math. Scand., 68, 1, 59-82 (1991) · Zbl 0748.31012
[21] Pyke, R., Markov renewal processes: definitions and preliminary properties, Ann. Math. Stat., 32, 4, 1231-1242 (1961) · Zbl 0267.60089
[22] Pyke, R., Markov renewal processes with finitely many states, Ann. Math. Stat., 32, 4, 1243-1259 (1961) · Zbl 0201.49901
[23] Smith, W. L., Regenerative stochastic processes, Proc. R. Soc. Lond., 232, 1188, 6-31 (1955) · Zbl 0067.36301
[24] Veretennikov, A. Y.; Kulik, A. M., Extended poisson equation for weakly ergodic markov processes, Theory Probab. Math. Statist., 85, 23-39 (2013) · Zbl 1278.60098
[25] Yin, B.; Li, Y.; Zhou, Y.; Xi, H., Performance optimization of semi-markov decision processes with discounted-cost criteria, Eur. J. Control, 14, 3, 213-222 (2008) · Zbl 1360.93791
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.