Perturbation realization, potentials, and sensitivity analysis of Markov processes.

*(English)*Zbl 0889.93039Suppose \(\{X_t^{\{i\}}\}\) and \(\{\widetilde X_t^{\{j\}}\}, (t\geq 0)\) are two independent normalized Markov processes with the same state space \(Z^+\) and the same infinitesimal generator, but with different initial states \(i\) and \(j\) respectively. Two fundamental concepts, perturbation realization factors \((d_{ij})\) and performance potentials \((g_i)\), are proposed here; they are defined as: \(d_{ij} \triangleq E\{\int_0^{T\{i,j\}} [f(\widetilde X_t^{\{j\}}) -f(X_t^{\{i\}})] dt\}\) and \(d_{ij} =g_j-g_i\) (the reasonableness of it is well proved), where \(T\{i,j\} =\inf \{t:(X_t^{\{i\}}, \widetilde X_t^{\{j\}}) =(k,k)\), \(\forall k\in Z^+\}\) and \(f\) is a performance function. An easy approach is provided here for estimating the \(d_{ij}\)’s based on a single sample path. It is also shown under some minor conditions that the realization matrix \(D= [d_{ij}]\) and the performance potential \(g=[g_i]\) satisfy the Lyapunov equation and the Poisson equation respectively, and they can uniquely determine \(D\) and \(g\). These results provide a sound theoretical foundation for some algorithms for estimating performance sensitivities based on a single sample path of a Markov process.

Reviewer: Wu Chengxun (Shanghai)

##### MSC:

93C73 | Perturbations in control/observation systems |

93B35 | Sensitivity (robustness) |

93C30 | Control/observation systems governed by functional relations other than differential equations (such as hybrid and switching systems) |

93E99 | Stochastic systems and control |

60G17 | Sample path properties |