×

Second-order necessary conditions for stochastic optimal control problems. (English) Zbl 1380.93294

Summary: The main purpose of this paper is to present some of our recent results about the second-order necessary conditions for stochastic optimal controls with the control variable entering into both the drift and the diffusion terms. In particular, when the control region is convex, a pointwise second-order necessary condition for stochastic singular optimal controls in the classical sense is established, whereas when the control region is allowed to be nonconvex, we obtain a pointwise second-order necessary condition for stochastic singular optimal controls in the sense of the Pontryagin-type maximum principle. Unlike deterministic optimal control problems or stochastic optimal control problems with control-independent diffusions, there exist some essential difficulties in deriving the pointwise second-order necessary optimality conditions from the integral conditions when the controls act in the diffusion terms of the stochastic control systems. Some techniques from Malliavin calculus are employed to overcome these difficulties. Moreover, it is found that, in contrast to the first-order necessary conditions, the correction part of the solution to the second-order adjoint equation appears in the pointwise second-order necessary conditions whenever the diffusion term depends on the control variable, even if the control region is convex.

MSC:

93E20 Optimal stochastic control
60H07 Stochastic calculus of variations and the Malliavin calculus
60H10 Stochastic ordinary differential equations (aspects of stochastic analysis)
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Ch. A. Agayeva, {\it Second order necessary conditions of optimality for stochastic systems with variable delay}, Teor. Ĭmovīr. Mat. Stat., 83 (2010), pp. 1-12; translation in Theoret. Probab. Math. Statist., 83 (2011), pp. 1-12. · Zbl 1238.93122
[2] D. J. Bell and D. H. Jacobson, {\it Singular Optimal Control Problems}, Math. Sci. Eng. 117, Academic Press, New York, 1975. · Zbl 0338.49006
[3] A. Bensoussan, {\it Lectures on stochastic control}, in Nonlinear Filtering and Stochastic Control, Lecture Notes in Math. 972, Springer-Verlag, Berlin, 1981, pp. 1-62.
[4] J.-M. Bismut, {\it An introductory approach to duality in optimal stochastic control}, SIAM Rev., 20 (1978), pp. 62-78, . · Zbl 0378.93049
[5] J. F. Bonnans and A. Hermant, {\it Second-order analysis for optimal control problems with pure state constraints and mixed control-state constraints}, Ann. Inst. H. Poincaré Anal. Non Linéaire, 26 (2009), pp. 561-598. · Zbl 1158.49023
[6] J. F. Bonnans and F. J. Silva, {\it First and second order necessary conditions for stochastic optimal control problems}, Appl. Math. Optim., 65 (2012), pp. 403-439. · Zbl 1244.49045
[7] P. Cannarsa, H. Frankowska, and E. M. Marchini, {\it Existence and Lipschitz regularity of solutions to Bolza problems in optimal control}, Trans. Amer. Math. Soc., 361 (2009), pp. 4491-4517. · Zbl 1169.49001
[8] F. H. Clarke and R. B. Vinter, {\it Regularity properties of optimal controls}, SIAM J. Control Optim., 28 (1990), pp. 980-997, . · Zbl 0715.49039
[9] D. J. Clements and B. D. O. Anderson, {\it Singular Optimal Control: The Linear-Quadratic Problem}, Lecture Notes in Control and Inform. Sci. 5, Springer-Verlag, New York, 1978. · Zbl 0371.49002
[10] N. El Karoui, S. Peng, and M. C. Quenez, {\it Backward stochastic differential equations in finance}, Math. Finance, 7 (1997), pp. 1-71. · Zbl 0884.90035
[11] H. Frankowska and D. Tonon, {\it Pointwise second-order necessary optimality conditions for the Mayer problem with control constraints}, SIAM J. Control Optim., 51 (2013), pp. 3814-3843, . · Zbl 1285.49013
[12] H. Frankowska, H. Zhang, and X. Zhang, {\it First and second order necessary conditions for stochastic optimal controls}, J. Differential Equations, 262 (2017), pp. 3689-3736. · Zbl 1352.93100
[13] H. Frankowska, H. Zhang, and X. Zhang, {\it Stochastic optimal control problems with control and initial-final states constraints}, submitted for publication. · Zbl 1391.93294
[14] H. Frankowska, H. Zhang, and X. Zhang, {\it Necessary Optimality Conditions for Local Minimizers of Stochastic Optimal Control Problems with State Constraints}, submitted for publication. · Zbl 1417.93335
[15] R. Gabasov and F. M. Kirillova, {\it High order necessary conditions for optimality}, SIAM J. Control, 10 (1972), pp. 127-168, . · Zbl 0236.49005
[16] R. F. Gabasov and F. M. Kirillova, {\it Singular Optimal Controls}, Izdat. “Nauka”, Moscow, 1973.
[17] B. S. Goh, {\it Necessary conditions for singular extremals involving multiple control variables}, SIAM J. Control, 4 (1966), pp. 716-731, . · Zbl 0161.29004
[18] U. G. Haussmann, {\it General necessary conditions for optimal control of stochastic systems}, Math. Program. Study, 6 (1976), pp. 30-48.
[19] D. Hoehener, {\it Variational approach to second-order optimality conditions for control problems with pure state constraints}, SIAM J. Control Optim., 50 (2012), pp. 1139-1173, . · Zbl 1246.49016
[20] H.-W. Knobloch, {\it Higher Order Necessary Conditions in Optimal Control Theory}, Lecture Notes in Comput. Sci. 34, Springer-Verlag, New York, 1981. · Zbl 0474.49001
[21] A. J. Krener, {\it The high order maximal principle and its application to singular extremals}, SIAM J. Control Optim., 15 (1977), pp. 256-293, . · Zbl 0354.49008
[22] H. J. Kushner, {\it Necessary conditions for continuous parameter stochastic optimization problems}, SIAM J. Control Optim., 10 (1972), pp. 550-565, . · Zbl 0242.93063
[23] H. Lou, {\it Second-order necessary/sufficient conditions for optimal control problems in the absence of linear structure}, Discrete Contin. Dyn. Syst. Ser. B, 14 (2010), pp. 1445-1464. · Zbl 1219.49016
[24] Q. Lü, {\it Second order necessary conditions for optimal control problems of stochastic evolution equations}, in Proceedings of the 35th Chinese Control Conference, Chengdu, China, 2016, pp. 2620-2625.
[25] Q. Lü, H. Zhang, and X. Zhang, {\it Second Order Optimality Conditions for Optimal Control Problems of Stochastic Evolution Equations}, preprint.
[26] Q. Lü and X. Zhang, {\it General Pontryagin-Type Stochastic Maximum Principle and Backward Stochastic Evolution Equations in Infinite Dimensions}, Springer Briefs in Mathematics, Springer, New York, 2014. · Zbl 1316.49004
[27] N. I. Mahmudov and A. E. Bashirov, {\it First order and second order necessary conditions of optimality for stochastic systems}, in Statistics and Control of Stochastic Process (Moscow, 1995/1996), World Scientific, River Edge, NJ, 1997, pp. 283-295. · Zbl 0928.93068
[28] L. Mou and J. Yong, {\it A variational formula for stochastic controls and some applications}, Pure Appl. Math. Q., 3 (2007), pp. 539-567. · Zbl 1142.49013
[29] D. Nualart, {\it The Malliavin Calculus and Related Topics}, 2nd ed., Springer-Verlag, Berlin, 2006. · Zbl 1099.60003
[30] N. P. Osmolovskii and H. Maurer, {\it Applications to Regular and Bang-Bang Control. Second-Order Necessary and Sufficient Optimality Conditions in Calculus of Variations and Optimal Control}, SIAM, Philadelphia, 2012, . · Zbl 1263.49002
[31] Z. Páles and V. Zeidan, {\it Optimal control problems with set-valued control and state constraints}, SIAM J. Optim., 14 (2003), pp. 334-358, . · Zbl 1041.49025
[32] S. Peng, {\it A general stochastic maximum principle for optimal control problems}, SIAM J. Control Optim., 28 (1990), pp. 966-979, . · Zbl 0712.93067
[33] L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, and E. F. Mishchenko, {\it The Mathematical Theory of Optimal Processes}, John Wiley, New York, 1962. · Zbl 0102.32001
[34] S. Tang, {\it A second-order maximum principle for singular optimal stochastic controls}, Discrete Contin. Dyn. Syst. Ser. B, 14 (2010), pp. 1581-1599. · Zbl 1219.93147
[35] J. Warga, {\it A second-order condition that strengthens Pontryagin’s maximum principle}, J. Differential Equations, 28 (1978), pp. 284-307. · Zbl 0346.49012
[36] J. Yong and X.Y. Zhou, {\it Stochastic Controls: Hamiltonian Systems and HJB Equations}, Springer-Verlag, New York, 2000. · Zbl 0943.93002
[37] H. Zhang and X. Zhang, {\it Some results on pointwise second-order necessary conditions for stochastic optimal controls}, Sci. China Math., \rm 59 (2016), pp. 227-238. · Zbl 1338.93409
[38] H. Zhang and X. Zhang, {\it Pointwise second-order necessary conditions for stochastic optimal controls, Part I: The case of convex control constraint}, SIAM J. Control Optim., 53 (2015), pp. 2267-2296, . · Zbl 1337.49045
[39] H. Zhang and X. Zhang, {\it Pointwise second-order necessary conditions for stochastic optimal controls, Part II: The general case}, SIAM J. Control Optim., 55 (2017), pp. 2841-2875, . · Zbl 1371.93222
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.