×

Bayesian optimal sequential design for nonparametric regression via inhomogeneous evolutionary MCMC. (English) Zbl 1486.62103

Summary: We develop a novel computational methodology for Bayesian optimal sequential design for nonparametric regression. This computational methodology, that we call inhomogeneous evolutionary Markov chain Monte Carlo, combines ideas of simulated annealing, genetic or evolutionary algorithms, and Markov chain Monte Carlo. Our framework allows optimality criteria with general utility functions and general classes of priors for the underlying regression function. We illustrate the usefulness of our novel methodology with applications to experimental design for nonparametric function estimation using Gaussian process priors and free-knot cubic splines priors.

MSC:

62G08 Nonparametric regression and quantile regression
62F15 Bayesian inference
65C40 Numerical analysis or methods applied to Markov chains
65C05 Monte Carlo methods
62K05 Optimal statistical designs
62L05 Sequential statistical design

Software:

fda (R); spBayes
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] (Abramowitz, M.; Stegun, I., Handbook of Mathematical Functions (1965), Dover: Dover New York) · Zbl 0171.38503
[2] Amzal, B.; Bois, F. Y.; Parent, E.; Robert, C. P., Bayesian-optimal design via interacting particle systems, J. Amer. Statist. Assoc., 101, 773-785 (2006) · Zbl 1119.62308
[3] Banerjee, S.; Carlin, B. P.; Gelfand, A. E., Hierarchical Modeling and Analysis for Spatial Data (2012), Chapman & Hall/CRC: Chapman & Hall/CRC Boca Raton
[4] Berger, J., Statistical Decision Theory and Bayesian Analysis (1985), Springer-Verlag: Springer-Verlag New York · Zbl 0572.62008
[5] Chaloner, K.; Verdinelli, I., Bayesian experimental design: a review, Statist. Sci., 10, 273-304 (1995) · Zbl 0955.62617
[6] Denison, D. G.T.; Mallick, B. K.; Smith, A. F.M., Automatic Bayesian curve fitting, J. R. Stat. Soc. Ser. B, 60, 333-350 (1998) · Zbl 0907.62031
[7] Dimatteo, I.; Genovese, C. R.; Kass, R. E., Bayesian curve-fitting with free-knot splines, Biometrika, 88, 4, 1055-1071 (2001) · Zbl 0986.62026
[8] Donoho, D. L.; Johnstone, I. M., Ideal spatial adaptation via wavelet shrinkage, Biometrika, 81, 425-455 (1994) · Zbl 0815.62019
[9] Doucet, A.; de Freitas, N.; Gordon, N., Sequential Monte Carlo Methods in Practice (2001), Springer: Springer New York · Zbl 0967.00022
[10] Drugan, M.; Thierens, D., Evolutionary Markov chain Monte Carlo, (Liardet, P.; Collet, P.; Fonlupt, C.; Luton, E.; Shoenauer, M., Artificial Evolution: 6th International Conference. Artificial Evolution: 6th International Conference, Lecture Notes in Computer Science (2004), Springer: Springer Berlin), 63-76 · Zbl 1098.68650
[11] Ferreira, M. A.R.; Lee, H. K.H., (Multiscale Modeling: A Bayesian Perspective. Multiscale Modeling: A Bayesian Perspective, Springer Series in Statistics (2007), Springer: Springer New York) · Zbl 1183.62046
[12] Finley, A.; Banerjee, S.; Carlin, B. P., spBayes: a program for multivariate point-referenced spatial modeling, J. Stat. Softw., 19, 4 (2007)
[13] Folland, G. B., Real Analysis: Modern Techniques and their Applications (1999), Wiley: Wiley New York · Zbl 0924.28001
[14] Gamerman, D.; Lopes, H. F., Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference (2006), Chapman and Hall/CRC: Chapman and Hall/CRC Boca Raton, FL · Zbl 1137.62011
[15] Goswami, G. R.; Liu, J. S., On learning strategies for evolutionary Monte Carlo, Stat. Comput., 17, 23-38 (2007)
[16] Gramacy, R. B.; Lee, H. K.H., Adaptive design of supercomputer experiments, Technometrics, 51, 130-145 (2009)
[17] Gramacy, R. B.; Lee, H. K.H., Bayesian treed Gaussian process models with an application to computer modeling, J. Amer. Statist. Assoc., 103, 1119-1130 (2008) · Zbl 1205.62218
[18] Gramacy, R. B.; Polson, N. G., Particle learning of Gaussian process models for sequential design and optimization, J. Comput. Graph. Statist., 20, 102-118 (2011)
[19] Green, P. J., Reversible jump Markov chain Monte Carlo computation and Bayesian model determination, Biometrika, 82, 711-732 (1995) · Zbl 0861.62023
[20] Green, P. J.; Silverman, B. W., Nonparametric Regression and Generalized Linear Models: A Roughness Penalty Approach (1994), Chapman and Hall · Zbl 0832.62032
[21] Higdon, D. M., (Anderson, C. W.; Barnett, V.; Chatwin, P. C.; El-Shaarawi, A. H., Space and Space-Time Modeling Using Process Convolutions. Space and Space-Time Modeling Using Process Convolutions, Quantitative Methods for Current Environmental Issues (2002), Springer-Verlag: Springer-Verlag New York), 37-54 · Zbl 1255.86016
[22] Jasra, A.; Stephens, D. A.; Holmes, C. C., Population-based reversible jump Markov chain Monte Carlo, Biometrika, 94, 787-807 (2007) · Zbl 1158.62019
[23] Johansen, A. M.; Doucet, A.; Davy, M., Particle methods for maximum likelihood estimation in latent variable models, Stat. Comput., 18, 47-57 (2008)
[24] Kass, R. E.; Wasserman, L., A reference Bayesian test for nested hypotheses and its relationship to the schwarz criterion, J. Amer. Statist. Assoc., 90, 431, 928-934 (1995) · Zbl 0851.62020
[25] Kim, H.; Mallick, B. K.; Holmes, C. C., Analyzing spatial data using piecewise Gaussian processes, J. Amer. Statist. Assoc., 100, 653-668 (2005) · Zbl 1117.62368
[26] Laskey, K. B.; Myers, J. W., Population Markov chain Monte Carlo, Mach. Learn., 50, 175-196 (2003) · Zbl 1028.68167
[27] Liang, F.; Wong, W. H., Evolutionary Monte Carlo: applications to \(C_p\) model sampling and change point problem, Statist. Sinica, 10, 317-342 (2000) · Zbl 1054.65500
[28] Liang, F.; Wong, W. H., Real-parameter evolutionary Monte Carlo with applications to Bayesian mixture models, J. Amer. Statist. Assoc., 96, 653-666 (2001) · Zbl 1017.62022
[29] Liu, J. S., Monte Carlo Strategies in Scientific Computing (2004), Springer: Springer New York
[30] Mukherjee, B., A note on sampling designs for random processes with no quadratic mean derivative, Aust. N.Z. J. Stat., 48, 305-319 (2006) · Zbl 1108.62082
[31] Müller-Gronbach, T., Optimal designs for approximating the path of a stochastic process, J. Statist. Plann. Inference, 49, 371-385 (1996) · Zbl 0881.62080
[32] Müller-Gronbach, T.; Ritter, K., Spatial adaption for predicting random functions, Ann. Statist., 26, 2264-2288 (1998) · Zbl 0927.62098
[33] Müller, P.; Sansó, B.; Iorio, M. D., Optimal Bayesian design by inhomogeneous Markov chain simulation, J. Amer. Statist. Assoc., 99, 788-798 (2004) · Zbl 1117.62404
[34] O’Hagan, A., Curve fitting and optimal design for prediction, J. R. Stat. Soc. Ser. B, 40, 1-42 (1978) · Zbl 0374.62070
[35] Ramsay, J. O.; Silverman, B. W., Functional Data Analysis (2006), Springer: Springer New York · Zbl 1079.62006
[36] Robert, C. P.; Casella, G., Monte Carlo Statistical Methods (2004), Springer-Verlag · Zbl 1096.62003
[37] Ruiz-Cárdenas, R.; Ferreira, M. A.R.; Schmidt, A. M., Stochastic search algorithms for optimal monitoring network designs, Environmetrics, 21, 102-112 (2010)
[38] Ruiz-Cárdenas, R.; Ferreira, M. A.R.; Schmidt, A. M., Evolutionary Markov chain Monte Carlo algorithms for optimal monitoring network designs, Stat. Methodol., 9, 185-194 (2012) · Zbl 1248.60088
[39] Taddy, M. A.; Gramacy, R. B.; Polson, N. G., Dynamic trees for learning and design, J. Amer. Statist. Assoc., 106, 109-123 (2011) · Zbl 1396.62158
[40] Vidakovic, B., Statistical Modeling by Wavelets (1999), Wiley: Wiley New York · Zbl 0924.62032
[41] Wallstrom, G.; Liebner, J.; Kass, R. E., An implementation of Bayesian adaptive regression splines in C with S and R wrappers, J. Stat. Softw., 26, 1, 1-21 (2008)
[42] Zhu, Z.; Stein, M. L., Spatial sampling design for parameter estimation of the covariance function, J. Statist. Plann. Inference, 134, 583-603 (2005) · Zbl 1066.62092
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.