# zbMATH — the first resource for mathematics

Measure-valued differentiation for stationary Markov chains. (English) Zbl 1278.90428
Summary: We study general state-space Markov chains that depend on a parameter, say,$${\theta}$$. Sufficient conditions are established for the stationary performance of such a Markov chain to be differentiable with respect to $${\theta}$$. Specifically, we study the case of unbounded performance functions and thereby extend the result on weak differentiability of stationary distributions of Markov chains to unbounded mappings. First, a closed-form formula for the derivative of the stationary performance of a general state-space Markov chain is given using an operator-theoretic approach. In a second step, we translate the derivative formula into unbiased gradient estimators. Specifically, we establish phantom-type estimators and score function estimators. We illustrate our results with examples from queueing theory.

##### MSC:
 90C40 Markov and semi-Markov decision processes
Full Text: