×

Maximum likelihood estimation for Gaussian processes under inequality constraints. (English) Zbl 1428.62420

Summary: We consider covariance parameter estimation for a Gaussian process under inequality constraints (boundedness, monotonicity or convexity) in fixed-domain asymptotics. We address the estimation of the variance parameter and the estimation of the microergodic parameter of the Matérn and Wendland covariance functions. First, we show that the (unconstrained) maximum likelihood estimator has the same asymptotic distribution, unconditionally and conditionally to the fact that the Gaussian process satisfies the inequality constraints. Then, we study the recently suggested constrained maximum likelihood estimator. We show that it has the same asymptotic distribution as the (unconstrained) maximum likelihood estimator. In addition, we show in simulations that the constrained maximum likelihood estimator is generally more accurate on finite samples. Finally, we provide extensions to prediction and to noisy observations.

MSC:

62M30 Inference from spatial processes
62F12 Asymptotic properties of parametric estimators
60G15 Gaussian processes
PDFBibTeX XMLCite
Full Text: DOI arXiv Euclid

References:

[1] P. Abrahamsen. A review of Gaussian random fields and correlation functions. Technical report, Norwegian computing center, 1997.
[2] M. Abramowitz and I. A. Stegun., Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. Dover, New York, ninth Dover printing, tenth GPO printing edition, 1964. · Zbl 0171.38503
[3] R. J. Adler., An introduction to continuity, extrema, and related topics for general Gaussian processes. IMS, 1990. · Zbl 0747.60039
[4] R. J. Adler and J. E. Taylor., Random fields and geometry. Springer Monographs in Mathematics. Springer, New York, 2007. · Zbl 1149.60003
[5] E. Anderes. On the consistent separation of scale and variance for Gaussian random fields., The Annals of Statistics, 38:870-893, 2010. · Zbl 1204.60041 · doi:10.1214/09-AOS725
[6] J.-M. Azaïs, F. Bachoc, T. Klein, A. Lagnoux, and T. M. N. Nguyen. Semi-parametric estimation of the variogram of a Gaussian process with stationary increments., arXiv :1806.03135, 2018.
[7] J.-M. Azaïs and M. Wschebor., Level sets and extrema of random processes and fields. John Wiley & Sons, Inc., Hoboken, NJ, 2009. · Zbl 1168.60002
[8] F. Bachoc. Cross validation and maximum likelihood estimations of hyper-parameters of Gaussian processes with model mispecification., Computational Statistics and Data Analysis, 66:55-69, 2013. · Zbl 1471.62021 · doi:10.1016/j.csda.2013.03.016
[9] F. Bachoc. Asymptotic analysis of the role of spatial sampling for covariance parameter estimation of Gaussian processes., Journal of Multivariate Analysis, 125:1-35, 2014. · Zbl 1280.62100 · doi:10.1016/j.jmva.2013.11.015
[10] F. Bachoc. Asymptotic analysis of the role of spatial sampling for covariance parameter estimation of Gaussian processes., Journal of Multivariate Analysis, 125:1-35, 2014. · Zbl 1280.62100 · doi:10.1016/j.jmva.2013.11.015
[11] F. Bachoc, K. Ammar, and J. Martinez. Improvement of code behavior in a design of experiments by metamodeling., Nuclear science and engineering, 183(3):387-406, 2016.
[12] F. Bachoc, G. Bois, J. Garnier, and J. Martinez. Calibration and improved prediction of computer models by universal Kriging., Nuclear Science and Engineering, 176(1):81-97, 2014.
[13] F. Bachoc et al. Asymptotic analysis of covariance parameter estimation for Gaussian processes in the misspecified case., Bernoulli, 24(2) :1531-1575, 2018. · Zbl 1429.60035 · doi:10.3150/16-BEJ906
[14] J. Bect, F. Bachoc, and D. Ginsbourger. A supermartingale approach to Gaussian process based sequential design of experiments., Bernoulli, forthcoming, 2018. · Zbl 1428.62369 · doi:10.3150/18-BEJ1074
[15] M. Bevilacqua, T. Faouzi, R. Furrer, and E. Porcu. Estimation and prediction using generalized Wendland covariance functions under fixed domain asymptotics., The Annals of Statistics, 47(2):828-856, 2019. · Zbl 1418.62365 · doi:10.1214/17-AOS1652
[16] C.-H. Chang, H.-C. Huang, C.-K. Ing, et al. Mixed domain asymptotics for a stochastic process model with time trend and measurement error., Bernoulli, 23(1):159-190, 2017. · Zbl 1359.62353 · doi:10.3150/15-BEJ740
[17] H.-S. Chen, D. Simpson, and Z. Ying. Infill asymptotics for a stochastic process model with measurement error., Statistica Sinica, 10:141-156, 2000. · Zbl 0970.62061
[18] A. Cousin, H. Maatouk, and D. Rullière. Kriging of financial term-structures., European Journal of Operational Research, 255(2):631-648, 2016. · Zbl 1346.91264
[19] S. Da Veiga and A. Marrel. Gaussian process modeling with inequality constraints., Annales de la faculté des sciences de Toulouse Mathématiques, 21(3):529-555, 4 2012. · Zbl 1279.60047 · doi:10.5802/afst.1344
[20] J. Du, H. Zhang, and V. Mandrekar. Fixed-domain asymptotic properties of tapered maximum likelihood estimators., The Annals of Statistics, 37 :3330-3361, 2009. · Zbl 1369.62248 · doi:10.1214/08-AOS676
[21] T. Gneiting. Compactly supported correlation functions., Journal of Multivariate Analysis, 83(2):493-508, 2002. · Zbl 1011.60015 · doi:10.1006/jmva.2001.2056
[22] S. Golchi, D. R. Bingham, H. Chipman, and D. A. Campbell. Monotone emulation of computer experiments., SIAM/ASA Journal on Uncertainty Quantification, 3(1):370-392, 2015. · Zbl 1327.62146 · doi:10.1137/140976741
[23] I. Ibragimov and Y. Rozanov., Gaussian Random Processes. Springer-Verlag, New York, 1978. · Zbl 0392.60037
[24] J. Istas and G. Lang. Quadratic variations and estimation of the local Hölder index of a Gaussian process., Annales de l’Institut Henri Poincaré, 33:407-436, 1997. · Zbl 0882.60032
[25] D. Jones, M. Schonlau, and W. Welch. Efficient global optimization of expensive black box functions., Journal of Global Optimization, 13:455-492, 1998. · Zbl 0917.90270 · doi:10.1023/A:1008306431147
[26] C. Kaufman and B. Shaby. The role of the range parameter for estimation and prediction in geostatistics., Biometrika, 100:473-484, 2013. · Zbl 1284.62590 · doi:10.1093/biomet/ass079
[27] M. A. Lifshits., Gaussian random functions, volume 322 of Mathematics and its Applications. Kluwer Academic Publishers, Dordrecht, 1995. · Zbl 0832.60002
[28] A. F. López-Lopera., lineqGPR: Gaussian Process Regression Models with Linear Inequality Constraints, 2018. R package version 0.0.3.
[29] A. F. López-Lopera, F. Bachoc, N. Durrande, and O. Roustand. Finite-dimensional Gaussian approximation with linear inequality constraints., SIAM/ASA Journal on Uncertainty Quantification, 6(3) :1224-1255, 2018. · Zbl 1405.60047 · doi:10.1137/17M1153157
[30] A. F. López-lopera, S. John, and N. Durrande. Gaussian process modulated Cox processes under linear inequality constraints. In, AISTATS, volume 89 of Proceedings of Machine Learning Research, pages 1997-2006, 16-18 Apr 2019.
[31] H. Maatouk and X. Bay. Gaussian process emulators for computer experiments with inequality constraints., Mathematical Geosciences, 49(5):557-582, 2017. · Zbl 1371.65006 · doi:10.1007/s11004-017-9673-2
[32] G. Matheron., La Théorie des Variables Régionalisées et ses Applications. Fasicule 5 in Les Cahiers du Centre de Morphologie Mathématique de Fontainebleau. Ecole Nationale Supérieure des Mines de Paris, 1970.
[33] H. Nickisch and C. E. Rasmussen. Approximations for binary Gaussian process classification., Journal of Machine Learning Research, 9(Oct) :2035-2078, 2008. · Zbl 1225.62087
[34] A. Pakman and L. Paninski. Exact Hamiltonian Monte Carlo for truncated multivariate Gaussians., Journal of Computational and Graphical Statistics, 23(2):518-542, 2014.
[35] R. Paulo, G. Garcia-Donato, and J. Palomo. Calibration of computer models with multivariate output., Computational Statistics and Data Analysis, 56 :3959-3974, 2012. · Zbl 1255.62187 · doi:10.1016/j.csda.2012.05.023
[36] H. Putter and G. A. Young. On the effect of covariance function estimation on the accuracy of Kriging predictors., Bernoulli, 7(3):421-438, 2001. · Zbl 0987.62061 · doi:10.2307/3318494
[37] C. Rasmussen and C. Williams., Gaussian Processes for Machine Learning. The MIT Press, Cambridge, 2006. · Zbl 1177.68165
[38] J. Riihimäki and A. Vehtari. Gaussian processes with monotonicity information. In, Journal of Machine Learning Research: Workshop and Conference Proceedings, volume 9, pages 645-652, 2010.
[39] O. Roustant, D. Ginsbourger, Y. Deville, et al. DiceKriging, DiceOptim: Two R packages for the analysis of computer experiments by Kriging-based metamodeling and optimization., Journal of Statistical Software, 51(i01), 2012.
[40] J. Sacks, W. Welch, T. Mitchell, and H. Wynn. Design and analysis of computer experiments., Statistical Science, 4:409-423, 1989. · Zbl 0955.62619 · doi:10.1214/ss/1177012413
[41] T. Santner, B. Williams, and W. Notz., The Design and Analysis of Computer Experiments. Springer, New York, 2003. · Zbl 1041.62068
[42] M. Stein. Asymptotically efficient prediction of a random field with a misspecified covariance function., The Annals of Statistics, 16:55-63, 1988. · Zbl 0637.62088
[43] M. Stein. Bounds on the efficiency of linear predictions using an incorrect covariance function., The Annals of Statistics, 18 :1116-1138, 1990. · Zbl 0749.62061 · doi:10.1214/aos/1176347742
[44] M. Stein. A comparison of generalized cross validation and modified maximum likelihood for estimating the parameters of a stochastic process., The Annals of Statistics, 18 :1139-1157, 1990. · Zbl 0734.62091 · doi:10.1214/aos/1176347743
[45] M. Stein. Uniform asymptotic optimality of linear predictions of a random field using an incorrect second-order structure., The Annals of Statistics, 18:850-872, 1990. · Zbl 0716.62099 · doi:10.1214/aos/1176347629
[46] M. Stein., Interpolation of Spatial Data: Some Theory for Kriging. Springer, New York, 1999. · Zbl 0924.62100
[47] A. W. van der Vaart., Asymptotic statistics, volume 3 of Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge, 1998. · Zbl 0910.62001
[48] D. Wang., Fixed Domain Asymptotics and Consistent Estimation for Gaussian Random Field Models in Spatial Statistics and Computer Experiments. PhD thesis, Nat. Univ. Singapore, Singapore, 2010.
[49] D. Wang and W.-L. Loh. On fixed-domain asymptotics and covariance tapering in Gaussian random field models., Electronic Journal of Statistics, 5:238-269, 2011. · Zbl 1274.62643 · doi:10.1214/11-EJS607
[50] H. Wendland., Scattered data approximation, volume 17. Cambridge university press, 2004. · Zbl 1185.65022
[51] Z. Ying. Asymptotic properties of a maximum likelihood estimator with data from a Gaussian process., Journal of Multivariate Analysis, 36:280-296, 1991. · Zbl 0733.62091 · doi:10.1016/0047-259X(91)90062-7
[52] Z. Ying. Maximum likelihood estimation of parameters under a spatial sampling scheme., The Annals of Statistics, 21 :1567-1590, 1993. · Zbl 0797.62019 · doi:10.1214/aos/1176349272
[53] H. Zhang. Inconsistent estimation and asymptotically equivalent interpolations in model-based geostatistics., Journal of the American Statistical Association, 99:250-261, 2004. · Zbl 1089.62538 · doi:10.1198/016214504000000241
[54] H. Zhang and Y. Wang. Kriging and cross validation for massive spatial data., Environmetrics, 21:290-304, 2010.
[55] D.-X. Zhou. Derivative reproducing properties for kernel methods in learning theory., Journal of computational and Applied Mathematics, 220(1):456-463, 2008. · Zbl 1152.68049 · doi:10.1016/j.cam.2007.08.023
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.