zbMATH — the first resource for mathematics

Perturbation realization, potentials, and sensitivity analysis of Markov processes. (English) Zbl 0889.93039
Suppose \(\{X_t^{\{i\}}\}\) and \(\{\widetilde X_t^{\{j\}}\}, (t\geq 0)\) are two independent normalized Markov processes with the same state space \(Z^+\) and the same infinitesimal generator, but with different initial states \(i\) and \(j\) respectively. Two fundamental concepts, perturbation realization factors \((d_{ij})\) and performance potentials \((g_i)\), are proposed here; they are defined as: \(d_{ij} \triangleq E\{\int_0^{T\{i,j\}} [f(\widetilde X_t^{\{j\}}) -f(X_t^{\{i\}})] dt\}\) and \(d_{ij} =g_j-g_i\) (the reasonableness of it is well proved), where \(T\{i,j\} =\inf \{t:(X_t^{\{i\}}, \widetilde X_t^{\{j\}}) =(k,k)\), \(\forall k\in Z^+\}\) and \(f\) is a performance function. An easy approach is provided here for estimating the \(d_{ij}\)’s based on a single sample path. It is also shown under some minor conditions that the realization matrix \(D= [d_{ij}]\) and the performance potential \(g=[g_i]\) satisfy the Lyapunov equation and the Poisson equation respectively, and they can uniquely determine \(D\) and \(g\). These results provide a sound theoretical foundation for some algorithms for estimating performance sensitivities based on a single sample path of a Markov process.

93C73 Perturbations in control/observation systems
93B35 Sensitivity (robustness)
93C30 Control/observation systems governed by functional relations other than differential equations (such as hybrid and switching systems)
93E99 Stochastic systems and control
60G17 Sample path properties
Full Text: DOI