Gong, Bo; Liu, Wenbin; Tang, Tao; Zhao, Weidong; Zhou, Tao An efficient gradient projection method for stochastic optimal control problems. (English) Zbl 1386.60239 SIAM J. Numer. Anal. 55, No. 6, 2982-3005 (2017). Summary: In this work, we propose a simple yet effective gradient projection algorithm for a class of stochastic optimal control problems. We first reduce the optimal control problem to an optimization problem for a convex functional by means of a projection operator. Then we propose a convergent iterative scheme for the optimization problem. The key issue in our iterative scheme is to compute the gradient of the objective functional by solving the adjoint equations that are given by backward stochastic differential equations (BSDEs). The Euler method is used to solve the resulting BSDEs. Rigorous convergence analysis is presented, and it is shown that the entire numerical algorithm admits a first order rate of convergence. Several numerical examples are carried out to support the theoretical finding. Cited in 11 Documents MSC: 60H35 Computational methods for stochastic equations (aspects of stochastic analysis) 65C20 Probabilistic models, generic numerical methods in probability and statistics 35Q99 Partial differential equations of mathematical physics and other areas of application 35R35 Free boundary problems for PDEs Keywords:stochastic optimal control; gradient projection methods; backward stochastic differential equations; conditional expectations PDFBibTeX XMLCite \textit{B. Gong} et al., SIAM J. Numer. Anal. 55, No. 6, 2982--3005 (2017; Zbl 1386.60239) Full Text: DOI References: [1] O. Bahn, A. Haurie, and R. Malhame, {\it A stochastic control model for optimal timing of climate policies}, Automatica J. IFAC, 44 (2008), pp. 1545-1558. · Zbl 1283.93305 [2] M. Bardi and I. Capuzzo-Dolcetta, {\it Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations}, Birkhäuser Boston, Boston, 1997. · Zbl 0890.49011 [3] K. Barty, J.-S. Roy, and C. Strugarek, {\it A stochastic gradient type algorithm for closed-loop problems}, Math. Program., 119 (2009), pp. 51-78. · Zbl 1159.62053 [4] A. Bensoussan, {\it Lecture on stochastic control}, in Nonlinear Filtering and Stochastic Control, Lecture Notes in Math. 972, Springer-Verlag, Berlin, New York, 1982, pp. 1-62. · Zbl 0505.93079 [5] A. Bensoussan, {\it Stochastic Control by Functional Analysis Methods}, North-Holland, New York, 1982. · Zbl 0474.93002 [6] D.P. Bertsekas, {\it Dynamic Programming and Optimal Control}, Vol. II., Athena Scientific, Belmont, MA, 2007. · Zbl 1209.90343 [7] F. Campillo, J. Nekkachi, and E. Pardoux, {\it Numerical methods in ergodic stochastic control: Application to semi-active vehicle suspensions}, in Proceedings of the 29th IEEE Conference on Decision and Control (Honolulu, HI), IEEE, Washington, DC, 1990. [8] P. Carpentier, G. Cohen, and A. Dallagi, {\it Particle methods for stochastic optimal control problems}, Comput. Optim. Appl., 56 (2013), pp. 635-674. · Zbl 1284.93259 [9] J. Chen, {\it Some Problems of Optimal Control and the Application in Financial Mathematics}, Master’s thesis, Tongji University, Shanghai, China, 2007. [10] M. Crandall and P. L. Lions, {\it Viscosity solutions of Hamilton-Jacobi equations}, Trans. Amer. Math. Soc., 277 (1983), pp. 1-42. · Zbl 0599.35024 [11] A. Dallagi, {\it Méthodes particulaires en commande optimale stochastique}, Thèse de doctorat, Université Paris 1, Panthéon-Sorbonne, Paris, France, 2007. [12] N. Du, J. T. Shi, and W. B. Liu, {\it An effective gradient projection method for stochastic optimal control}, Int. J. Numer. Anal. Model., 4 (2013), pp. 757-774. · Zbl 1280.65061 [13] R. J. Elliott, {\it The optimal control of diffusions}, Appl. Math. Optim., 22 (1990), pp. 229-240. · Zbl 0718.49013 [14] W. H. Fleming and R. W. Rishel, {\it Deterministic and Stochastic Optimal Control}, Springer, New York, 2012. · Zbl 0323.49001 [15] P. Girardeau, {\it A comparison of sample-based stochastic optimal control methods}, Stochastic Program. E-Print Ser., 2010 (2010), . [16] E. Gobet, J. P. Lemor, and X. Warin, {\it A regression-based Monte Carlo method to solve backward stochastic differential equations}, Ann. Appl. Probab., 15 (2005), pp. 2172-2202. · Zbl 1083.60047 [17] M. Gunzburger, {\it Perspectives in Flow Control and Optimization}, SIAM, Philadelphia, 2003, . · Zbl 1088.93001 [18] M. Gunzburger, H. Lee, and J. Lee, {\it Error estimates of stochastic optimal Neumann boundary control problems}, SIAM J. Numer. Anal., 49 (2011), pp. 1532-1552, . · Zbl 1243.65008 [19] F. B. Hanson, {\it Applied Stochastic Processes and Control for Jump-Diffusions: Modeling, Analysis, and Computation}, SIAM, Philadelphia, 2007. · Zbl 1145.60003 [20] U. G. Haussmann, {\it Some examples of optimal stochastic controls or: The stochastic maximum principle at work}, SIAM Rev., 23 (1981), pp. 292-307, . · Zbl 0466.93073 [21] H. Heitsch and W. Römisch, {\it Scenario Reduction Algorithms in Stochastic Programming}, Comput. Optim. Appl., 24 (2003), pp. 187-206. · Zbl 1094.90024 [22] H. Heitsch, W. Römisch, and C. Strugarek, {\it Stability of multistage stochastic programs}, SIAM J. Optim., 17 (2006), pp. 511-525, . · Zbl 1165.90582 [23] C. S. Huang, S. Wang, and K. L. Teo, {\it Solving Hamilton-Jacobi-Bellman equations by a modified method of characteristics}, Nonlinear Anal., 40 (2000), pp. 279-293. · Zbl 0959.49021 [24] R. Korn, {\it Optimal Portfolios: Stochastic Models for Optimal Investment and Risk Management in Continuous Time}, World Scientific, Singapore, 1997. · Zbl 0931.91017 [25] R. Korn and H. Kraft, {\it A stochastic control approach to portfolio problems with stochastic interest rates}, SIAM J. Control Optim., 40 (2001), pp. 1250-1269, . · Zbl 1020.93029 [26] D. Kuhn, P. Parpas, and B. Rustem, {\it Stochastic optimization of investment planning problems in the electric power industry}, in Energy Systems Engineering, Process Systems Engrg. 5, Wiley-VCH Verlag, Weinheim, Germany, 2008, pp. 215-230. [27] H. J. Kushner, {\it Necessary conditions for continuous parameter stochastic optimization problems}, SIAM J. Control, 10 (1972), pp. 550-565, . · Zbl 0242.93063 [28] H. J. Kushner and P. Dupuis, {\it Numerical Methods for Stochastic Control Problems in Continuous Time}, 2nd ed., Springer-Verlag, New York, 2001. · Zbl 0968.93005 [29] A. Li, E. M. Feng, and X. L. Sun, {\it Stochastic optimal control and algorithm of the trajectory of horizontal wells}, J. Comput. Appl. Math., 212 (2008), pp. 419-430. · Zbl 1136.93048 [30] J. Ma and J. Yong, {\it Forward-Backward Stochastic Differential Equations and Their Applications}, Springer, New York, 2007. · Zbl 0927.60004 [31] G. Mastroianni and G. Monegato, {\it Error estimates for Gauss-Laguerre and Gauss-Hermite quadrature formulas}, in Approximation and Computation: A Festschrift in Honor of Walter Gautschi, Birkhäuser Boston, Boston, 1994, pp. 421-434. · Zbl 0819.41024 [32] R. C. Merton, {\it Continuous-Time Finance}, Blackwell, Cambridge, MA, 1992. · Zbl 1019.91502 [33] G. N. Milstein and M. V. Tretyakov, {\it Numerical algorithms for forward-backward stochastic differential equations}, SIAM J. Sci. Comput., 28 (2006), pp. 561-582, . · Zbl 1114.60054 [34] E. Pardoux and S. Peng, {\it Adapted solution of a backward stochastic differential equation}, Systems Control Lett., 14 (1990), pp. 55-61. · Zbl 0692.93064 [35] S. G. Peng, {\it A general stochastic maximum principle for optimal control problems}, SIAM J. Control Optim., 28 (1990), pp. 966-979, . · Zbl 0712.93067 [36] S. G. Peng, {\it Probabilistic interpretation for systems of quasilinear parabolic partial differential equations}, Stochastics Stochastics Rep., 37 (1991), pp. 61-74. · Zbl 0739.60060 [37] S. G. Peng, {\it Backward stochastic differential equations and applications to optimal control}, Appl. Math. Optim., 27 (1993), pp. 125-144. · Zbl 0769.60054 [38] R. Raffard, J. Hu, and C. Tomlin, {\it Adjoint-based optimal control of the expected exit time for stochastic hybrid systems}, in Hybrid Systems: Computation and Control, M. Morari and L. Thiele, eds., Lecture Notes in Comput. Sci. 3414, Springer-Verlag, Berlin, 2005, pp. 557-572. · Zbl 1078.93064 [39] L. C. G. Rogers, {\it Pathwise stochastic optimal control}, SIAM J. Control Optim., 46 (2007), pp. 1116-1132, . · Zbl 1155.90019 [40] S. P. Sethi and Q. Zhang, {\it Hierarchical Decision Making in Stochastic Manufacturing Systems}, Birkhäuser Verlag, Basel, 1994. · Zbl 0923.90002 [41] A. Shapiro and A. Ruszczynski, eds., {\it Stochastic Programming}, Elsevier, Amsterdam, 2003. · Zbl 1115.90001 [42] J. T. Shi and Z. Wu, {\it A stochastic maximum principle for optimal control of jump diffusions and applications to finance}, Chinese J. Appl. Probab. Statist., 27 (2011), pp. 127-137. · Zbl 1240.93364 [43] J. C. Spall, {\it Stochastic optimization} II.6, in Handbook of Computational Statistics, J. Gentle, W. Härdle, and Y. Mori, eds., Springer-Verlag, New York, 2004, pp. 169-197. [44] T. Tang, W. Zhao, and T. Zhou, {\it Deferred correction methods for forward backward stochastic differential equations}, Numer. Math., 10 (2017), pp. 222-242. · Zbl 1389.60083 [45] W. S. Xu, {\it Maximum principle for a stochastic optimal control problem and application to portfolio/consumption choice}, J. Optim. Theory Appl., 98 (1998), pp. 719-731. · Zbl 0915.90033 [46] J. Yong and X. Y. Zhou, {\it Stochastic Controls: Hamiltonian Systems and HJB Equations}, Springer, New York, 1999. · Zbl 0943.93002 [47] W. Zhao, L. Chen, and S. Peng, {\it A new kind of accurate numerical method for backward stochastic differential equations}, SIAM J. Sci. Comput., 28 (2006), pp. 1563-1581, . · Zbl 1121.60072 [48] W. Zhao, Y. Fu, and T. Zhou, {\it New kinds of high-order multistep schemes for coupled forward backward stochastic differential equations}, SIAM J. Sci. Comput., 36 (2014), pp. A1731-A1751, . · Zbl 1316.65014 [49] W. Zhao, T. Zhou, and T. Kong, {\it High order numerical schemes for second order FBSDEs with applications to stochastic optimal control}, Commun. Comput. Phys., 21 (2017), pp. 808-834. · Zbl 1499.65027 This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.