an:06504590 Zbl 1394.60084 Etessami, Kousha; Stewart, Alistair; Yannakakis, Mihalis Greatest fixed points of probabilistic min/max polynomial equations, and reachability for branching Markov decision processes EN Halld??rsson, Magn??s M. (ed.) et al., Automata, languages, and programming. 42nd international colloquium, ICALP 2015, Kyoto, Japan, July 6--10, 2015. Proceedings. Part II. Berlin: Springer (ISBN 978-3-662-47665-9/pbk; 978-3-662-47666-6/ebook). Lecture Notes in Computer Science 9135, 184-196 (2015). 2015
a
60J80 branching Markov decision processes; probabilistic polynomial system; Bellman optimality Summary: We give polynomial time algorithms for quantitative (and qualitative) reachability analysis for branching Markov decision processes (BMDPs). Specifically, given a BMDP, and given an initial population, where the objective of the controller is to maximize (or minimize) the probability of eventually reaching a population that contains an object of a desired (or undesired) type, we give algorithms for approximating the supremum (infimum) reachability probability, within desired precision $$\epsilon > 0$$, in time polynomial in the encoding size of the BMDP and in $$\log (1/\epsilon )$$. We furthermore give P-time algorithms for computing $$\epsilon$$-optimal strategies for both maximization and minimization of reachability probabilities. We also give P-time algorithms for all associated qualitative analysis problems, namely: deciding whether the optimal (supremum or infimum) reachability probabilities are 0 or 1. Prior to this paper, approximation of optimal reachability probabilities for BMDPs was not even known to be decidable.{ }Our algorithms exploit the following basic fact: we show that for any BMDP, its maximum (minimum) non-reachability probabilities are given by the greatest fixed point (GFP) solution $$g^* \in [0,1]^n$$ of a corresponding monotone max (min) probabilistic polynomial system of equations (max/min-PPS), $$x=P(x)$$, which are the Bellman optimality equations for a BMDP with non-reachability objectives. We show how to compute the GFP of max/min PPSs to desired precision in P-time. For the entire collection see [Zbl 1316.68013].