×

zbMATH — the first resource for mathematics

Perturbation realization, potentials, and sensitivity analysis of Markov processes. (English) Zbl 0889.93039
Suppose \(\{X_t^{\{i\}}\}\) and \(\{\widetilde X_t^{\{j\}}\}, (t\geq 0)\) are two independent normalized Markov processes with the same state space \(Z^+\) and the same infinitesimal generator, but with different initial states \(i\) and \(j\) respectively. Two fundamental concepts, perturbation realization factors \((d_{ij})\) and performance potentials \((g_i)\), are proposed here; they are defined as: \(d_{ij} \triangleq E\{\int_0^{T\{i,j\}} [f(\widetilde X_t^{\{j\}}) -f(X_t^{\{i\}})] dt\}\) and \(d_{ij} =g_j-g_i\) (the reasonableness of it is well proved), where \(T\{i,j\} =\inf \{t:(X_t^{\{i\}}, \widetilde X_t^{\{j\}}) =(k,k)\), \(\forall k\in Z^+\}\) and \(f\) is a performance function. An easy approach is provided here for estimating the \(d_{ij}\)’s based on a single sample path. It is also shown under some minor conditions that the realization matrix \(D= [d_{ij}]\) and the performance potential \(g=[g_i]\) satisfy the Lyapunov equation and the Poisson equation respectively, and they can uniquely determine \(D\) and \(g\). These results provide a sound theoretical foundation for some algorithms for estimating performance sensitivities based on a single sample path of a Markov process.

MSC:
93C73 Perturbations in control/observation systems
93B35 Sensitivity (robustness)
93C30 Control/observation systems governed by functional relations other than differential equations (such as hybrid and switching systems)
93E99 Stochastic systems and control
60G17 Sample path properties
PDF BibTeX XML Cite
Full Text: DOI