×

Théorèmes de convergence presque sure pour une classe d’algorithmes stochastiques à pas decroissant. (French) Zbl 0588.62153

The paper studies the pathwise asymptotic behaviour of stochastic algorithms of the following general form \[ \theta_{n+1}=\theta_ n+\gamma_{n+1}f(\theta_ n,Y_{n+1}), \] the hypothesis allowing discontinuities on the adaptation term f. The process \((Y_ n)_{n\geq 0}\) is a Markov chain ”controlled by \((\theta_ n)''\). For each \(\theta\) fixed the Markov chain \((Y_ n^{\theta})_{n\geq 0}\) is essentially of positive recurrent type.
A first theorem generalizes to this situation a theorem of L. Ljung [IEEE Trans. Autom. Control AC-22, 551-575 (1977; Zbl 0362.93031)]. An almost sure convergence theorem is proved under the existence of a global Lyapunov function for the associated deterministic differential equation d\({\bar \theta}\)(t)/dt\(=h({\bar \theta}(t))\) where \(h(\theta)=\int f(\theta,y)\Gamma_{\theta}(dy)\) and \(\Gamma_{\theta}\) is the invariant probability of the Markov chain \((Y_ n^{\theta})_{n\geq 0}\).

MSC:

62L20 Stochastic approximation
60F15 Strong limit theorems
60J20 Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.)
62L12 Sequential estimation

Citations:

Zbl 0362.93031
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Benveniste, A.; Goursat, G.; Ruget, G., Analysis of stochastic approximation schemes with discontinuous dependent forcing terms, IEEE Trans. Autom. Control, 25, 6 (1980) · Zbl 0485.93079
[2] Eweda, E.; Macchi, O., Convergence analysis of self adaptive equalizers, IEEE Trans. Information Theory, 30, 161-176 (1984) · Zbl 0554.94009 · doi:10.1109/TIT.1984.1056896
[3] Krasovski, N. N., Stability of motion (1963), Stanford: University Press, Stanford · Zbl 0109.06001
[4] Kushner, H., Stochastic approximation with discontinuous dynamics and state dependent noise: W.p 1 and weak convergence, J. Math. Anal. Appl., 82, 527-542 (1981) · Zbl 0473.62074 · doi:10.1016/0022-247X(81)90214-6
[5] Kushner, H.; Shwartz, A., An invariant measure approach to the convergence of stochastic approximations with state dependent noise, SIAM J. Control Optimization, 22, 13-27 (1984) · Zbl 0535.62069 · doi:10.1137/0322002
[6] Kushner, H.; Clark, D. S., Stochastic approximation for constrained and unconstrained systems (1978), Berlin Heidelberg New York: Springer, Berlin Heidelberg New York · Zbl 0381.60004
[7] Ljung, L., Analysis of recursive stochastic algorithms, IEEE Trans. Autom. Control, 22, 551-575 (1977) · Zbl 0362.93031 · doi:10.1109/TAC.1977.1101561
[8] Ljung, L., Analysis of stochastic gradient algorithms for linear regression problems, IEEE Trans. Inf. Theory., 30, 151-160 (1984) · Zbl 0589.65097 · doi:10.1109/TIT.1984.1056895
[9] Metivier, M.; Priouret, P., Applications of a Kushner and Clark lemma to general classes of stochastic algorithms, IEEE Trans. Inf. Theory., 30, 140-150 (1984) · Zbl 0546.62056 · doi:10.1109/TIT.1984.1056894
[10] Metivier, M.; Priouret, P., Convergence avec probabilité (1-ε) d’algorithmes stochastiques. Application au cas de l’égaliseur aveugle, Ann. Télécommunt, 56, 41, 269-279 (1986) · Zbl 0607.62097
[11] Sunyach, C., Une classe de chaînes récurrentes sur un espace métrique complet, Ann. Inst. H. Poincaré, I, 325-343 (1975) · Zbl 0324.60059
[12] Freidlin, M. I.; Wentzel, A. D., Random perturbations of dynamical systems (1984), Berlin Heidelberg New York: Springer, Berlin Heidelberg New York · Zbl 0522.60055
[13] Heyde, C. C.; Hall, D., Martingale limit theory and its applications (1980), New York: Academic Press, New York · Zbl 0462.60045
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.