×

Goal-oriented optimal design of experiments for large-scale Bayesian linear inverse problems. (English) Zbl 1475.65037

Summary: We develop a framework for goal-oriented optimal design of experiments (GOODE) for large-scale Bayesian linear inverse problems governed by PDEs. This framework differs from classical Bayesian optimal design of experiments (ODE) in the following sense: we seek experimental designs that minimize the posterior uncertainty in the experiment end-goal, e.g. a quantity of interest (QoI), rather than the estimated parameter itself. This is suitable for scenarios in which the solution of an inverse problem is an intermediate step and the estimated parameter is then used to compute a QoI. In such problems, a GOODE approach has two benefits: the designs can avoid wastage of experimental resources by a targeted collection of data, and the resulting design criteria are computationally easier to evaluate due to the often low-dimensionality of the QoIs. We present two modified design criteria, A-GOODE and D-GOODE, which are natural analogues of classical Bayesian A- and D-optimal criteria. We analyze the connections to other ODE criteria, and provide interpretations for the GOODE criteria by using tools from information theory. Then, we develop an efficient gradient-based optimization framework for solving the GOODE optimization problems. Additionally, we present comprehensive numerical experiments testing the various aspects of the presented approach. The driving application is the optimal placement of sensors to identify the source of contaminants in a diffusion and transport problem. We enforce sparsity of the sensor placements using an \(\ell_1\)-norm penalty approach, and propose a practical strategy for specifying the associated penalty parameter.

MSC:

65K05 Numerical mathematical programming methods
62F15 Bayesian inference
65F22 Ill-posedness and regularization problems in numerical linear algebra
90C30 Nonlinear programming
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Atkinson, A. C.; Donev, A. N., Optimum Experimental Designs, (1992), Oxford: Clarendon Press, Oxford · Zbl 0829.62070
[2] Chaloner, K.; Verdinelli, I., Bayesian experimental design: a review, Stat. Sci., 10, 273-304, (1995) · Zbl 0955.62617 · doi:10.1214/ss/1177009939
[3] Pukelsheim, F., Optimal Design of Experiments, (1993), New York: Wiley, New York · Zbl 0834.62068
[4] Uciński, D., Optimal Measurement Methods for Distributed Parameter System Identification, (2005), Boca Raton, FL: CRC Press, Boca Raton, FL · Zbl 1155.93003
[5] Pázman, A., Foundations of Optimum Experimental Design, (1986), Heidelberg: Springer, Heidelberg · Zbl 0588.62117
[6] Bauer, I.; Bock, H. G.; Körkel, S.; Schlöder, J. P., Numerical methods for optimum experimental design in DAE systems, J. Comput. Appl. Math., 120, 1-25, (2000) · Zbl 0998.65083 · doi:10.1016/S0377-0427(00)00300-9
[7] Körkel, S.; Kostina, E.; Bock, H. G.; Schlöder, J. P., Numerical methods for optimal control problems in design of robust optimal experiments for nonlinear dynamic processes, Optim. Methods Softw., 19, 327-338, (2004) · Zbl 1060.49025 · doi:10.1080/10556780410001683078
[8] Haber, E.; Horesh, L.; Tenorio, L., Numerical methods for the design of large-scale nonlinear discrete ill-posed inverse problems, Inverse Problems, 26, (2010) · Zbl 1189.65073 · doi:10.1088/0266-5611/26/2/025002
[9] Horesh, L.; Haber, E.; Tenorio, L., Optimal Experimental Design for the Large-Scale Nonlinear Ill-Posed Problem of Impedance Imaging, 273-290, (2010), New York: Wiley, New York
[10] Chung, M.; Haber, E., Experimental design for biological systems, SIAM J. Control Optim., 50, 471-489, (2012) · Zbl 1243.93130 · doi:10.1137/100791063
[11] Huan, X.; Marzouk, Y. M., Simulation-based optimal Bayesian experimental design for nonlinear systems, J. Comput. Phys., 232, 288-317, (2013) · doi:10.1016/j.jcp.2012.08.013
[12] Long, Q.; Scavino, M.; Tempone, R.; Wang, S., Fast estimation of expected information gains for Bayesian experimental designs based on Laplace approximations, Comput. Methods Appl. Mech. Eng., 259, 24-39, (2013) · Zbl 1286.62068 · doi:10.1016/j.cma.2013.02.017
[13] Sandu, A.; Cioaca, A.; Rao, V., Dynamic sensor network configuration in infosymbiotic systems using model singular vectors, 18, 1909-1918, (2013)
[14] Huan, X.; Marzouk, Y. M., Gradient-based stochastic optimization methods in Bayesian experimental design, Int. J. Uncertain. Quantification, 4, 479-510, (2014) · doi:10.1615/Int.J.UncertaintyQuantification.2014006730
[15] Alexanderian, A.; Petra, N.; Stadler, G.; Ghattas, O., A-optimal design of experiments for infinite-dimensional Bayesian linear inverse problems with regularized \( \newcommand{\e}{{\rm e}} ℓ_0\)-sparsification, SIAM J. Sci. Comput., 36, A2122-A2148, (2014) · Zbl 1314.62163 · doi:10.1137/130933381
[16] Long, Q.; Motamed, M.; Tempone, R., Fast Bayesian optimal experimental design for seismic source inversion, Comput. Methods Appl. Mech. Eng., 291, 123-145, (2015) · doi:10.1016/j.cma.2015.03.021
[17] Uciński, D., An algorithm for construction of constrained D-optimum designs, Stochastic Models, Statistics and their Applications, 461-468, (2015), New York: Springer, New York · Zbl 1349.62349
[18] Alexanderian, A.; Petra, N.; Stadler, G.; Ghattas, O., A fast and scalable method for A-optimal design of experiments for infinite-dimensional Bayesian nonlinear inverse problems, SIAM J. Sci. Comput., 38, A243-A272, (2016) · Zbl 06536072 · doi:10.1137/140992564
[19] Alexanderian, A.; Gloor, P. J.; Ghattas, O., On Bayesian A-and D-optimal experimental designs in infinite dimensions, Bayesian Anal., 11, 671-695, (2016) · Zbl 1359.62315 · doi:10.1214/15-BA969
[20] Bisetti, F.; Kim, D.; Knio, O.; Long, Q.; Tempone, R., Optimal Bayesian experimental design for priors of compact support with application to shock-tube experiments for combustion kinetics, Int. J. Numer. Methods Eng., 108, 136-155, (2016) · doi:10.1002/nme.5211
[21] Crestel, B.; Alexanderian, A.; Stadler, G.; Ghattas, O., A-optimal encoding weights for nonlinear inverse problems, with application to the Helmholtz inverse problem, Inverse problems, 33, (2017) · Zbl 1370.65064 · doi:10.1088/1361-6420/aa6d8e
[22] Yu, J.; Zavala, V. M.; Anitescu, M., A scalable design of experiments framework for optimal sensor placement, J. Process Control, 67, 44-55, (2018) · doi:10.1016/j.jprocont.2017.03.011
[23] Walsh, S. N.; Wildey, T. M.; Jakeman, J. D., Optimal experimental design using A consistent Bayesian approach, ASCE-ASME J. Risk Uncertain. Eng. Syst. B, 4, (2018)
[24] Ruthotto, L.; Chung, J.; Chung, M., Optimal experimental design for constrained inverse problems submitted, (2017)
[25] Haber, E.; Horesh, L.; Tenorio, L., Numerical methods for experimental design of large-scale linear ill-posed inverse problems, Inverse Problems, 24, 125-137, (2008) · Zbl 1153.65062 · doi:10.1088/0266-5611/24/5/055012
[26] Haber, E.; Magnant, Z.; Lucero, C.; Tenorio, L., Numerical methods for A-optimal designs with a sparsity constraint for ill-posed inverse problems, Comput. Optim. Appl., 52, 293-314, (2012) · Zbl 1259.90135
[27] Tenorio, L.; Lucero, C.; Ball, V.; Horesh, L., Experimental design in the context of Tikhonov regularized inverse problems, Stat. Modelling, 13, 481-507, (2013) · doi:10.1177/1471082X13494613
[28] Beck, J.; Dia, B. M.; Espath, L. F R.; Long, Q.; Tempone, R., Fast Bayesian experimental design: Laplace-based importance sampling for the expected information gain, Comput. Methods Appl. Mech. Eng., 334, 523-553, (2018) · doi:10.1016/j.cma.2018.01.053
[29] Khodja, M.; Prange, M.; Djikpesse, H., Guided Bayesian optimal experimental design, Inverse Problems, 26, (2010) · Zbl 1187.86006 · doi:10.1088/0266-5611/26/5/055008
[30] Djikpesse, H. A.; Khodja, M. R.; Prange, M. D.; Duchenne, S.; Menkiti, H., Bayesian survey design to optimize resolution in waveform inversion, Geophysics, 77, R81-R93, (2012) · doi:10.1190/geo2011-0143.1
[31] Alexanderian, A.; Saibaba, A. K., Efficient D-optimal design of experiments for infinite-dimensional Bayesian linear inverse problems, SIAM J. Sci. Comput. SISC, (2018)
[32] Lieberman, C.; Willcox, K., Goal-oriented inference: approach, linear theory, and application to advection diffusion, SIAM Rev., 55, 493-519, (2013) · Zbl 1273.35303 · doi:10.1137/130913110
[33] Lieberman, C.; Willcox, K., Nonlinear goal-oriented Bayesian inference: application to carbon capture and storage, SIAM J. Sci. Comput., 36, B427-B449, (2014) · Zbl 1429.62711 · doi:10.1137/130928315
[34] Spantini, A.; Cui, T.; Willcox, K.; Tenorio, L.; Marzouk, Y., Goal-oriented optimal approximations of Bayesian linear inverse problems, SIAM J. Sci. Comput., 39, S167-S196, (2017) · Zbl 1373.15027 · doi:10.1137/16M1082123
[35] Avron, H.; Toledo, S., Randomized algorithms for estimating the trace of an implicit symmetric positive semi-definite matrix, J. ACM, 58, 17, (2011) · Zbl 1327.68331 · doi:10.1145/1944345.1944349
[36] Saibaba, A. K.; Alexanderian, A.; Ipsen, I. C., Randomized matrix-free trace and log-determinant estimators, Numer. Math., 137, 353-395, (2017) · Zbl 1378.65094 · doi:10.1007/s00211-017-0880-z
[37] Bui-Thanh, T.; Ghattas, O.; Martin, J.; Stadler, G., A computational framework for infinite-dimensional Bayesian inverse problems part I: the linearized case, with application to global seismic inversion, SIAM J. Sci. Comput., 35, A2494-A2523, (2013) · Zbl 1287.35087 · doi:10.1137/12089586X
[38] Tarantola, A., Inverse Problem Theory and Methods for Model Parameter Estimation, (2005), Philadelphia, PA: SIAM, Philadelphia, PA · Zbl 1074.65013
[39] Bisetti, F.; Kim, D.; Knio, O.; Long, Q.; Tempone, R., Optimal Bayesian experimental design for priors of compact support with application to shock-tube experiments for combustion kinetics, Int. J. Numer. Methods Eng., 108, 136-155, (2016)
[40] Long, Q.; Scavino, M.; Tempone, R.; Wang, S., A Laplace method for under-determined Bayesian optimal experimental designs, Comput. Methods Appl. Mech. Eng., 285, 849-876, (2015) · doi:10.1016/j.cma.2014.12.008
[41] Bernardo, J. M., Expected information as expected utility, Ann. Stat., 7, 686-690, (1979) · Zbl 0407.62002 · doi:10.1214/aos/1176344689
[42] Choice of response surface design and alphabetic optimality, Technical Report, (1982) · Zbl 0524.62073
[43] Dette, H., A note on Bayesian c-and D-optimal designs in nonlinear regression models, Ann. Stat., 24, 1225-1234, (1996) · Zbl 0866.62046 · doi:10.1214/aos/1032526965
[44] Elfving, G., Optimum allocation in linear regression theory, Ann. Math. Stat., 23, 255-262, (1952) · Zbl 0047.13403 · doi:10.1214/aoms/1177729442
[45] Kiefer, J.; Wolfowitz, J., Optimum designs in regression problems, Ann. Math. Stat., 30, 271-294, (1959) · Zbl 0090.11404 · doi:10.1214/aoms/1177706252
[46] Lindley, D. V., On a measure of the information provided by an experiment, Ann. Math. Stat., 27, 986-1005, (1956) · Zbl 0073.14103 · doi:10.1214/aoms/1177728069
[47] Kullback, S.; Leibler, R. A., On information and sufficiency, Ann. Math. Stat., 22, 79-86, (1951) · Zbl 0042.38403 · doi:10.1214/aoms/1177729694
[48] Papalambros, P. Y.; Wilde, D. J., Principles of Optimal Design and Computation, (2000), Cambridge: Cambridge University Press, Cambridge · Zbl 0962.90002
[49] Krause, A.; Singh, A.; Guestrin, C., Near-optimal sensor placements in Gaussian processes: theory, efficient algorithms and empirical studies, J. Mach. Learn. Res., 9, 235-284, (2008) · Zbl 1225.68192
[50] Attia, A.; Rao, V.; Sandu, A., A sampling approach for four dimensional data assimilation, Dynamic Data-Driven Environmental Systems Science, 215-226, (2015), New York: Springer, New York
[51] Attia, A.; Rao, V.; Sandu, A., A hybrid Monte-Carlo sampling smoother for four dimensional data assimilation, Int. J. Numer. Methods Fluids, 83, 90-112, (2017)
[52] Navon, I. M., Data assimilation for numerical weather prediction: a review, Data Assimilation for Atmospheric, Oceanic and Hydrologic Applications, 21-65, (2009), New York: Springer, New York
[53] Isaac, T.; Petra, N.; Stadler, G.; Ghattas, O., Scalable and efficient algorithms for the propagation of uncertainty from data through inference to prediction for large-scale problems, with application to flow of the Antarctic ice sheet, J. Comput. Phys., 296, 348-368, (2015) · Zbl 1352.86017 · doi:10.1016/j.jcp.2015.04.047
[54] Akçelik, V.; Biros, G.; Drăgănescu, A.; Ghattas, O.; Hill, J.; van Bloeman Waanders, B., Dynamic data-driven inversion for terascale simulations: real-time identification of airborne contaminants, (2005)
[55] Flath, P. H.; Wilcox, L. C.; Akçelik, V.; Hill, J.; van Bloemen Waanders, B.; Ghattas, O., Fast algorithms for Bayesian uncertainty quantification in large-scale linear inverse problems based on low-rank partial Hessian approximations, SIAM J. Sci. Comput., 33, 407-432, (2011) · Zbl 1229.65174 · doi:10.1137/090780717
[56] Petra, N.; Stadler, G., Model variational inverse problems governed by partial differential equations, Technical Report 11-05, (2011)
[57] Villa, U.; Petra, N.; Ghattas, O., hIPPYlib: an extensible software framework for large-scale deterministic and linearized Bayesian inversion, (2016)
[58] Logg, A.; Mardal, K-A; Wells, G., Automated Solution of Differential Equations by the Finite Element Method: The FEniCS Book, vol 48, (2012), New York: Springer, New York · Zbl 1247.65105
[59] Zhu, C.; Byrd, R. H.; Lu, P.; Nocedal, J., Algorithm 778: L-BFGS-B: fortran subroutines for large-scale bound-constrained optimization, ACM Trans. Math. Softw., 23, 550-560, (1997) · Zbl 0912.65057 · doi:10.1145/279232.279236
[60] Byrd, R. H.; Lu, P.; Nocedal, J.; Zhu, C., A limited memory algorithm for bound constrained optimization, SIAM J. Sci. Comput., 16, 1190-1208, (1995) · Zbl 0836.65080 · doi:10.1137/0916069
[61] Morales, J. L.; Nocedal, J., Remark on ‘algorithm 778: L-BFGS-B: fortran subroutines for large-scale bound constrained optimization, ACM Trans. Math. Softw., 38, 7, (2011) · Zbl 1365.65164 · doi:10.1145/2049662.2049669
[62] Hansen, P. C.; Johnston, P., The L-curve and its use in the numerical treatment of inverse probems, Computational Inverse Problems in Electrocardiology, 119-142, (2001), Southampton: WIT Press, Southampton
[63] Phillips, D. L., A technique for the numerical solution of certain integral equations of the first kind, J. ACM, 9, 84-97, (1962) · Zbl 0108.29902 · doi:10.1145/321105.321114
[64] Tikhonov, A., Solution of incorrectly formulated problems and the regularization method, Sov. Meth. Dokl., 4, 1035-1038, (1963) · Zbl 0141.11001
[65] Attia, A.; Stefanescu, R.; Sandu, A., The reduced-order hybrid Monte Carlo sampling smoother, Int. J. Numer. Methods Fluids, 83, 28-51, (2017)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.