×

Dynamic programming for optimal control of stochastic McKean-Vlasov dynamics. (English) Zbl 1361.93069

Summary: We study optimal control of the general stochastic McKean-Vlasov equation. Such a problem is motivated originally from the asymptotic formulation of cooperative equilibrium for a large population of particles (players) in mean-field interaction under common noise. Our first main result is to state a dynamic programming principle for the value function in the Wasserstein space of probability measures, which is proved from a flow property of the conditional law of the controlled state process. Next, by relying on the notion of differentiability with respect to probability measures due to [P. L. Lions, Cours au Collège de France: Théorie des jeux à champ moyens, (2012), pp. 2006–2012] and Itô’s formula along a flow of conditional measures, we derive the dynamic programming Hamilton-Jacobi–Bellman equation and prove the viscosity property together with a uniqueness result for the value function. Finally, we solve explicitly the linear-quadratic stochastic McKean-Vlasov control problem and give an application to an interbank systemic risk model with common noise.

MSC:

93E20 Optimal stochastic control
60H30 Applications of stochastic analysis (to PDEs, etc.)
60K35 Interacting random processes; statistical mechanics type models; percolation theory
90C39 Dynamic programming
49L20 Dynamic programming in optimal control and differential games
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] S. Ahuja, {\it Wellposedness of mean field games with common noise under a weak monotonicity condition}, SIAM J. Control Optim., 54 (2016), pp. 30-48. · Zbl 1327.93403
[2] L. Ambrosio, N. Gigli, and G. Savaré, {\it Gradient Flows: In Metric Spaces and in the Space of Probability Measures,} Lectures Math. ETH zürich, Birkhäuser Verlag, Basel, 2005. · Zbl 1090.35002
[3] D. Andersson and B. Djehiche, {\it A maximum principle for SDEs of mean-field type}, Appl. Math. Optim., 63 (2010), pp. 341-356. · Zbl 1215.49034
[4] A. Bain and D. Crisan, {\it Fundamentals of Stochastic Filtering}, Stoch. Model. Appl. Probab., 60, Springer, New York, 2009. · Zbl 1176.62091
[5] E. Bayraktar, A. Cosso, and H. Pham, {\it Randomized dynamic programming principle and Feynman-Kac representation for optimal control of McKean-Vlasov dynamics,} Trans. Amer. Math. Soc. to appear, , 2016. · Zbl 1381.93102
[6] A. Bensoussan, J. Frehse, and P. Yam, {\it The Master equation in mean-field theory}, J. Math. Pures Appl., 103 (2015), pp. 1441-1474. · Zbl 1325.35232
[7] A. Bensoussan, J. Frehse, and P. Yam, {\it On the interpretation of the Master equation}, Stoch. Process Appl., 127 (2017). · Zbl 1379.60063
[8] R. Buckdahn, B. Djehiche, and J. Li, {\it A general maximum principle for SDEs of mean-field type}, Appl. Math. Optim., 64 (2011), pp. 197-216. · Zbl 1245.49036
[9] R. Buckdahn, J. Li, and J. Ma, {\it A mean-field stochastic control problem with partial observations}, Ann. Appl Probab., to appear. · Zbl 1380.93282
[10] R. Buckdahn, J. Li, S. Peng, and C. Rainer, {\it Mean-field stochastic differential equations and associated PDEs}, Ann. Probab., to appear, , 2014. · Zbl 1402.60070
[11] P. Cardaliaguet, {\it Notes on Mean Field Games: Notes from P. L. Lions Lectures at Collège de France}, (2012).
[12] P. Cardaliaguet, F. Delarue, J. M. Lasry, and P. L. Lions, {\it The Master Equation and the Convergence Problem in Mean Field Games}, , 2015. · Zbl 1430.91002
[13] R. Carmona and F. Delarue, {\it Forward-backward stochastic differential equations and controlled McKean Vlasov dynamics}, Ann. Probab., 43 (2015), pp. 2647-2700. · Zbl 1322.93103
[14] R. Carmona and F. Delarue, {\it The Master equation for large population equilibriums}, in Stochastic Analysis and Applications 2014, D. Crisan et al., eds., Springer Proc. Math. Statist. 100, Springer, New York, 2014. · Zbl 1391.92036
[15] R. Carmona, F. Delarue, and A. Lachapelle, {\it Control of McKean-Vlasov dynamics versus mean field games}, Math. Financ. Econ., 7 (2013), pp. 131-166. · Zbl 1269.91012
[16] R. Carmona, F. Delarue, and D. Lacker {\it Mean field games with common noise}, Ann. Probab., 44 (2016), pp. 3740-3803. · Zbl 1422.91083
[17] R. Carmona, J. P. Fouque, and L. Sun, {\it Mean field games and systemic risk}, Commun. Math. Sci., 13 (2015), pp. 911-933. · Zbl 1337.91031
[18] R. Carmona and X. Zhu, {\it A Probabilistic approach to mean field games with major and minor players}, Ann. Appl. Probab., 26 (2016), pp. 1535-1580. · Zbl 1342.93121
[19] J. F. Chassagneux, D. Crisan, and F. Delarue, {\it A probabilistic approach to classical solutions of the master equation for large population equilibria}, , 2014. · Zbl 1520.91003
[20] J. Claisse, D. Talay, and X. Tan, {\it A pseudo-Markov property for controlled diffusion processes,} SIAM J. Control Optim., 54 (2016), pp. 1017-1029. · Zbl 1341.60097
[21] D. Dawson and J. Vaillancourt, {\it Stochastic McKean-Vlasov equations}, Nonlinear Differential Equations Appl., 2 (1995), pp. 199-229. · Zbl 0830.60091
[22] J. L. Doob, {\it Measure Theory}, Springer, New York, 1994. · Zbl 0791.28001
[23] G. Fabbri, F. Gozzi, and A. Swiech, {\it Stochastic Optimal Control in Infinite Dimension: Dynamic Programming and HJB Equations}, , (2015). · Zbl 1379.93001
[24] J. Feng and M. Katsoulakis, {\it A comparison principle for Hamilton-Jacobi equations related to controlled gradient flows in infinite dimensions}, Arch. Ration. Mech. Anal., 192 (2009), pp. 275-310. · Zbl 1159.49036
[25] M. Fuhrman and H. Pham, {\it Randomized and backward SDE representation for optimal control of non-Markovian SDEs}, Ann. Appl. Probab, 25 (2015), pp. 2134-2167. · Zbl 1322.60087
[26] W. Gangbo, T. Nguyen, and A. Tudorascu, {\it Hamilton-Jacobi equations in the Wasserstein space}, Methods Appl Anal., 15 (2008), pp. 155-184. · Zbl 1171.49308
[27] W. Gangbo and A. Swiech, {\it Existence of a solution to an equation arising from mean field games}, J. Differential Equations, to appear. · Zbl 1277.35129
[28] M. Huang, P. Caines, and R. Malhamé, {\it Large population stochastic dynamic games: Closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle}, Commun. Inf. Syst., 6 (2006), pp. 221-252. · Zbl 1136.91349
[29] N. Krylov, {\it Controlled Diffusion Processes}, Appl. Math., 14, Springer, New York, 1980. · Zbl 0459.93002
[30] T. Kurtz and J. Xiong, {\it Particle representations for a class of nonlinear SPDEs}, Stochastic Process Appl., 83 (1999), pp. 103-126. · Zbl 0996.60071
[31] J. M. Lasry and P. L. Lions, {\it Mean-field games}, Jpn. J. Math., 2 (2007), pp. 229-260. · Zbl 1156.91321
[32] M. Laurière and O. Pironneau, {\it Dynamic programming for mean-field type control}, C. R. Math., 352 (2014), pp. 707-713. · Zbl 1310.90117
[33] P. L. Lions, {\it Viscosity solutions of fully nonlinear second-order equations and optimal control in infinite dimension. Part} I: {\it The case of bounded stochastic evolutions}, Acta Math., 161 (1988), pp. 243-278. · Zbl 0757.93082
[34] P. L. Lions, {\it Viscosity solutions of fully nonlinear second-order equations and optimal control in infinite dimension. Part} III: {\it Uniqueness of viscosity solutions for general second-order equations}, J. Funct. Anal., 86 (1989), pp. 1-18. · Zbl 0757.93084
[35] P. L. Lions, {\it Cours au Collège de France: Théorie des jeux à champ moyens}, audio conference, 2012.
[36] H. Pham and X. Wei, {\it Bellman equation and viscosity solutions for mean-field stochastic control problem}, , 2015. · Zbl 1396.93134
[37] P. Protter, {\it Stochastic Integration and Differential Equations}, 2nd ed., Springer-Verlag, Heidelberg, 2005.
[38] D. Revuz and M. Yor, {\it Continuous Martingales and Brownian Motion}, 3rd ed., Springer, New York, 1999. · Zbl 0917.60006
[39] M. Soner and N. Touzi, {\it Dynamic programming for stochastic target problems and geometric flows}, J. Eur. Math. Soc., 4 (2002), pp. 201-236. · Zbl 1003.49003
[40] C. Villani, {\it Topics in Optimal Transportation}, Grad. Stud. Math., AMS, Providence, RI, 2003. · Zbl 1106.90001
[41] D. Wagner, {\it Survey of measurable selection theorems: an update}, Lecture Notes in Math. 794, Springer-Verlag, Berlin, 1980. · Zbl 0427.28009
[42] W. Wonham, {\it On a matrix Riccati equation of stochastic control}, SIAM J. Control Optim., 6 (1968), pp. 681-697. · Zbl 0182.20803
[43] J. Yong, {\it A linear-quadratic optimal control problem for mean-field stochastic differential equations}, SIAM J. Control Optim.., 51 (2013), pp. 2809-2838. · Zbl 1275.49060
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.