zbMATH — the first resource for mathematics

Using dynamic programming with adaptive grid scheme for optimal control problems in economics. (English) Zbl 1202.49026
Summary: The study of the solutions of dynamic models with optimizing agents has often been limited by a lack of available analytical techniques to explicitly find the global solution paths. On the other hand, the application of numerical techniques such as dynamic programming to find the solution in interesting regions of the state was restricted by the use of fixed grid size techniques. Following L. Grüne [Numer. Math. 75, No. 3, 319–337 (1997; Zbl 0880.65045); Numer. Math. 99, No. 1, 85–112 (2004; Zbl 1074.65009)], in this paper an adaptive grid scheme is used for finding the global solutions of discrete time Hamilton-Jacobi-Bellman equations. Local error estimates are established and an adapting iteration for the discretization of the state space is developed. The advantage of the use of adaptive grid scheme is demonstrated by computing the solutions of one- and two-dimensional economic models which exhibit steep curvature, complicated dynamics due to multiple equilibria, thresholds (Skiba sets) separating domains of attraction and periodic solutions. We consider deterministic and stochastic model variants. The studied examples are from economic growth, investment theory, environmental and resource economics.

49L20 Dynamic programming in optimal control and differential games
49N90 Applications of optimal control and differential games
91B55 Economic dynamics
49M25 Discrete approximations in optimal control
90C39 Dynamic programming
Full Text: DOI
[1] Azariadis, C.; Drazen, A., Thresholds externalities in economic development, Quarterly journal of economics, 105, 2, 501-526, (1990)
[2] Bardi, M.; Capuzzo Dolcetta, I., Optimal control and viscosity solutions of hamilton – jacobi – bellman equations, (1997), Birkhäuser Boston, MA · Zbl 0890.49011
[3] Benhabib, J., Farmer, R., 1999. Indeterminacy in macroeconomics. In: Taylor, J., Woodford, M. (Eds.), Handbook of Macroeconomics. Elsevier, Amsterdam.
[4] Benhabib, J.; Perli, R., Uniqueness and indeterminacyon the dynamics of endogenous growth, Journal of economic theory, 63, 1, 113-142, (1994) · Zbl 0803.90023
[5] Benhabib, J.; Perli, R.; Xie, D., Monopolistic competition, indeterminacy and growth, Ricerche economiche, 48, 279-298, (1994) · Zbl 0826.90020
[6] Blanchard, O.J., Debt and current account deficit in Brazil, (), 187-197
[7] Brock, W.; Mirman, L., Optimal economic growth and uncertainty: the discounted case, Journal of economic theory, 4, 479-513, (1972)
[8] Brock, W.A., Starrett, D., 1999. Nonconvexities in ecological management problems. SSRI Working Paper 2026, Department of Economics, University of Wisconsin, Madison, WI.
[9] Burnside, C., Discrete state-space methods for the study of dynamic economies, (), 95-113
[10] Camilli, F.; Falcone, M., An approximation scheme for the optimal control of diffusion processes, RAIRO. modélisation math. anal. numér, 29, 97-122, (1995) · Zbl 0822.65044
[11] Candler, G.V., Finite-difference methods for continuous-time dynamic programming, (), 172-194
[12] Capuzzo Dolcetta, I., On a discrete approximation of the hamilton – jacobi equation of dynamic programming, Appl. math. optim, 10, 367-377, (1983) · Zbl 0582.49019
[13] Capuzzo Dolcetta, I.; Falcone, M., Discrete dynamic programming and viscosity solutions of the Bellman equation, Ann. inst. Henri Poincaré, anal. non linéaire, 6, supplement, 161-184, (1989) · Zbl 0674.49028
[14] Carlini, E., Falcone, M., Ferretti, R., 2002. An efficient algorithm for Hamilton-Jacobi equations in high dimensions. Report 2002/7, Università di Roma ‘La Sapienza’, Dipartimento di Matematica ‘Guido Castelnuovo’, submitted for publication. · Zbl 1070.65072
[15] Chow, C.-S.; Tsitsiklis, J.N., An optimal one-way multigrid algorithm for discrete – time stochastic control, IEEE transactions on automatic control, 36, 898-914, (1991) · Zbl 0752.93078
[16] Daniel, J.W., Splines and efficiency in dynamic programming, Journal of mathematical analysis and applications, 54, 402-407, (1976) · Zbl 0345.90041
[17] Deissenberg, D., Feichtinger, G., Semmler, W., Wirl, F., 2001. History dependence and global dynamics in models with multiple equilibria. Working paper No. 12, Center for Empirical Macroeconomics, University of Bielefeld, www.wiwi.uni-bielefeld.de.
[18] Falcone, M., 1987. A numerical approach to the infinite horizon problem of deterministic control theory, Applied Mathematical and Optimization 15, 1-13. Corrigenda, ibid, 1991. 23, 213-214. · Zbl 0715.49023
[19] Falcone, M.; Ferretti, R., Convergence analysis for a class of high-order semi-Lagrangian advection schemes, SIAM journal of numerical analysis, 35, 909-940, (1998) · Zbl 0914.65097
[20] Falcone, M.; Giorgi, T., An approximation scheme for evolutive hamilton – jacobi equations, (), 288-303 · Zbl 0931.65067
[21] Feichtinger, G., Haunschmied, J.L., Kort, P.M., Hartl, R.F., 2000. A DNS-curve in a two state capital accumulation model: a numerical analysis. Forschungsbericht 242 des Instituts für Ökonometrie, OR und Systemtheorie. University of Technology, Vienna, submitted for publication. · Zbl 1029.91029
[22] Feichtinger, G.; Kort, P.; Hartl, R.F.; Wirl, F., The dynamics of a simple relative adjustment-cost framework, German economic review, 2, 3, 255-268, (2001)
[23] González, R.L.V.; Sagastizábal, C.A., Un algorithme pour la résolution rapide d’équations discrètes de hamilton – jacobi – bellman, C. R. acad. sci., Paris, Sér. I, 311, 45-50, (1990) · Zbl 0719.65053
[24] Grüne, L., An adaptive grid scheme for the discrete hamilton – jacobi – bellman equation, Numerical mathematics, 75, 3, 319-337, (1997) · Zbl 0880.65045
[25] Grüne, L., 2003. Error estimation and adaptive discretization for the discrete stochastic Hamilton-Jacobi-Bellman equation. Preprint, Universität Bayreuth, submitted.
[26] Grüne, L.; Metscher, M.; Ohlberger, M., On numerical algorithm and interactive visualization for optimal control problems, Computing and visualization in science, 1, 4, 221-229, (1999) · Zbl 0970.65073
[27] Grüne, L., Semmler, W., Sieveking, M., 2001. Thresholds in a credit market model with multiple equilibria. Discussion Paper No. 482, Department of Economics, University of Bielefeld, submitted for publication.
[28] Grüne, L., Kato, M., Semmler, W., 2002. Numerical study of an ecological management problem. CEM Bielefeld, Working paper.
[29] Hartl, R.F., Kort, P., Feichtinger, G., Wirl, F., 2000. Multiple equilibria and thresholds due to relative investment costs: non-concave – concave, focus – node, continuous – discontinuous. Working paper, University of Technology, Vienna.
[30] Jermann, U.J., Asset pricing in production economies, Journal of monetary economics, 41, 257-275, (1998)
[31] Johnson, S.A.; Stedinger, J.R.; Shoemaker, C.A.; Li, Y.; Tejada-Guibert, J.A., Numerical solution of continuous-state dynamic programs using linear and spline interpolation, Operations research, 41, 484-500, (1993) · Zbl 0777.90074
[32] Judd, K.L., 1996, Approximation, perturbation, and projection methods in economic analysis. In: Amman, H.M., Kendrick, D.A., Rust, J. (Eds.), Handbook of Computational Economics. Elsevier, Amsterdam, pp. 511-585 (Chapter 12). · Zbl 1126.91300
[33] Judd, K.L., Numerical methods in economics, (1998), MIT Press Cambridge, MA · Zbl 0941.00048
[34] Judd, K.L.; Guu, S.-M., Asymptotic methods for aggregate growth models, Journal of economic dynamics & control, 21, 1025-1042, (1997) · Zbl 0901.90039
[35] Keane, M.P.; Wolpin, K.I., The solution and estimation of discrete choice dynamic programming models by simulation and interpolationmonte Carlo evidence, The review of economics & statistics, 76, 648-672, (1994)
[36] Ljungqvist, L.; Sargent, T.J., Recursive macroeconomic theory, (2001), MIT Press Cambridge, MA
[37] Marcet, A., Simulation analysis of stochastic dynamic models: applications to theory and estimation, (), 81-118
[38] Munos, R.; Moore, A., Variable resolution discretization in optimal control, Machine learning, 49, 291-323, (2002) · Zbl 1005.68086
[39] Reiter, M., Solving higher-dimensional continuous-time stochastic control problems by value function regression, Journal of economic dynamics and control, 23, 1329-1353, (1999) · Zbl 0949.93080
[40] Rust, J., Numerical dynamic programming in economics, (), 620-729 · Zbl 1126.65316
[41] Rust, J., Using randomization to break the curse of dimensionality, Econometrica, 65, 478-516, (1997) · Zbl 0872.90107
[42] Santos, M.S., Vigo-Aguiar, J., 1995. Accuracy estimates for a numerical approach to stochastic growth models. Discussion Paper 107, Institute for Empirical Macroeconomics, Federal Reserve Bank of Minneapolis.
[43] Santos, M.S.; Vigo-Aguiar, J., Analysis of a numerical dynamic programming algorithm applied to economic models, Econometrica, 66, 2, 409-426, (1998) · Zbl 1010.90091
[44] Seeck, A., Iterative Lösungen der hamilton – jacobi – bellman-gleichung bei unendlichem zeithorizont, (1997), Diplomarbeit Universität Kiel
[45] Semmler, W.; Sieveking, M., On optimal exploitation of interacting resources, Journal of economics, 59, 1, 23-49, (1994) · Zbl 0798.90018
[46] Sieveking, M.; Semmler, W., The present value of resources with large discount rates, Applied mathematics and optimization, 35, 283-309, (1997) · Zbl 0881.49002
[47] Skiba, A.K., Optimal growth with a convex – concave a production function, Econometrica, 46, 3, 527-539, (1978) · Zbl 0383.90020
[48] Tauchen, G.; Hussey, R., Quadrature-based methods for obtaining approximate solutions to nonlinear asset-price models, Econometrica, 59, 371-396, (1991) · Zbl 0735.90012
[49] Taylor, J.B.; Uhlig, H., Solving non-linear stochastic growth modelsa comparison of alternative solution methods, Journal of business and economic studies, 8, 1-18, (1990)
[50] Trick, M.A., Zin, S.E., 1993. A linear programming approach to solving stochastic dynamic programs, Working paper, Carnegie-Mellon University.
[51] Trick, M.A.; Zin, S.E., Spline approximations to value functionsa linear programming approach, Macroeconomic dynamics, 1, 255-277, (1997) · Zbl 0914.90209
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.