# zbMATH — the first resource for mathematics

Perturbation realization, potentials, and sensitivity analysis of Markov processes. (English) Zbl 0889.93039
Suppose $$\{X_t^{\{i\}}\}$$ and $$\{\widetilde X_t^{\{j\}}\}, (t\geq 0)$$ are two independent normalized Markov processes with the same state space $$Z^+$$ and the same infinitesimal generator, but with different initial states $$i$$ and $$j$$ respectively. Two fundamental concepts, perturbation realization factors $$(d_{ij})$$ and performance potentials $$(g_i)$$, are proposed here; they are defined as: $$d_{ij} \triangleq E\{\int_0^{T\{i,j\}} [f(\widetilde X_t^{\{j\}}) -f(X_t^{\{i\}})] dt\}$$ and $$d_{ij} =g_j-g_i$$ (the reasonableness of it is well proved), where $$T\{i,j\} =\inf \{t:(X_t^{\{i\}}, \widetilde X_t^{\{j\}}) =(k,k)$$, $$\forall k\in Z^+\}$$ and $$f$$ is a performance function. An easy approach is provided here for estimating the $$d_{ij}$$’s based on a single sample path. It is also shown under some minor conditions that the realization matrix $$D= [d_{ij}]$$ and the performance potential $$g=[g_i]$$ satisfy the Lyapunov equation and the Poisson equation respectively, and they can uniquely determine $$D$$ and $$g$$. These results provide a sound theoretical foundation for some algorithms for estimating performance sensitivities based on a single sample path of a Markov process.

##### MSC:
 93C73 Perturbations in control/observation systems 93B35 Sensitivity (robustness) 93C30 Control/observation systems governed by functional relations other than differential equations (such as hybrid and switching systems) 93E99 Stochastic systems and control 60G17 Sample path properties
Full Text: