×

zbMATH — the first resource for mathematics

Adaptive estimation over anisotropic functional classes via oracle approach. (English) Zbl 1328.62213
Summary: We address the problem of adaptive minimax estimation in white Gaussian noise models under \(\mathbb{L}_{p}\)-loss, \(1\leq p\leq\infty\), on the anisotropic Nikol’skii classes. We present the estimation procedure based on a new data-driven selection scheme from the family of kernel estimators with varying bandwidths. For the proposed estimator we establish so-called \(\mathbb{L}_{p}\)-norm oracle inequality and use it for deriving minimax adaptive results. We prove the existence of rate-adaptive estimators and fully characterize behavior of the minimax risk for different relationships between regularity parameters and norm indexes in definitions of the functional class and of the risk. In particular some new asymptotics of the minimax risk are discovered, including necessary and sufficient conditions for the existence of a uniformly consistent estimator. We provide also a detailed overview of existing methods and results and formulate open problems in adaptive minimax estimation.

MSC:
62G05 Nonparametric estimation
62G20 Asymptotic properties of nonparametric inference
Software:
EBayesThresh
PDF BibTeX XML Cite
Full Text: DOI Euclid arXiv
References:
[1] Akakpo, N. (2012). Adaptation to anisotropy and inhomogeneity via dyadic piecewise polynomial selection. Math. Methods Statist. 21 1-28. · Zbl 1308.62070
[2] Baraud, Y. and Birgé, L. (2014). Estimating composite functions by model selection. Ann. Inst. Henri Poincaré Probab. Stat. 50 285-314. · Zbl 1281.62093
[3] Baraud, Y., Birgé, L. and Sart, M. (2014). A new method for estimation and model selection: \(\rho\)-estimation. Available at . arXiv:1403.6057v1 · Zbl 1373.62141
[4] Baraud, Y., Giraud, C. and Huet, S. (2014). Estimator selection in the Gaussian setting. Ann. Inst. Henri Poincaré Probab. Stat. 50 1092-1119. · Zbl 1298.62113
[5] Barron, A., Birgé, L. and Massart, P. (1999). Risk bounds for model selection via penalization. Probab. Theory Related Fields 113 301-413. · Zbl 0946.62036
[6] Bertin, K. (2005). Sharp adaptive estimation in sup-norm for \(d\)-dimensional Hölder classes. Math. Methods Statist. 14 267-298.
[7] Birgé, L. (2008). Model selection for density estimation with \({\mathbb{L}}_{2}\)-loss. Available at . arXiv:0808.1416v2
[8] Birgé, L. and Massart, P. (2001). Gaussian model selection. J. Eur. Math. Soc. ( JEMS ) 3 203-268. · Zbl 1037.62001
[9] Bunea, F., Tsybakov, A. B. and Wegkamp, M. H. (2007). Aggregation for Gaussian regression. Ann. Statist. 35 1674-1697. · Zbl 1209.62065
[10] Cai, T. T. (1999). Adaptive wavelet estimation: A block thresholding and oracle inequality approach. Ann. Statist. 27 898-924. · Zbl 0954.62047
[11] Cai, T. T. and Low, M. G. (2005). On adaptive estimation of linear functionals. Ann. Statist. 33 2311-2343. · Zbl 1086.62031
[12] Cai, T. T. and Low, M. G. (2006). Optimal adaptive estimation of a quadratic functional. Ann. Statist. 34 2298-2325. · Zbl 1110.62048
[13] Cavalier, L. and Golubev, Y. (2006). Risk hull method and regularization by projections of ill-posed inverse problems. Ann. Statist. 34 1653-1677. · Zbl 1246.62082
[14] Cavalier, L. and Tsybakov, A. B. (2001). Penalized blockwise Stein’s method, monotone oracles and sharp adaptive estimation. Math. Methods Statist. 10 247-282. · Zbl 1005.62027
[15] Chichignoud, M. (2012). Minimax and minimax adaptive estimation in multiplicative regression: Locally Bayesian approach. Probab. Theory Related Fields 153 543-586. · Zbl 1318.62128
[16] Chichignoud, M. and Lederer, J. (2014). A robust, adaptive M-estimator for pointwise estimation in heteroscedastic regression. Bernoulli 20 1560-1599. · Zbl 1303.62035
[17] Comte, F. and Lacour, C. (2013). Anisotropic adaptive kernel deconvolution. Ann. Inst. Henri Poincaré Probab. Stat. 49 569-609. · Zbl 1348.62121
[18] Dalalyan, A. and Tsybakov, A. B. (2008). Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity. Mach. Learn. 72 39-61.
[19] de Guzmán, M. (1975). Differentiation of Integrals in \(R^{n}\). Lecture Notes in Mathematics 481 . Springer, Berlin.
[20] Delyon, B. and Juditsky, A. (1996). On minimax wavelet estimators. Appl. Comput. Harmon. Anal. 3 215-228. · Zbl 0865.62023
[21] Devroye, L. and Lugosi, G. (1996). A universally acceptable smoothing factor for kernel density estimates. Ann. Statist. 24 2499-2512. · Zbl 0867.62024
[22] Devroye, L. and Lugosi, G. (1997). Nonasymptotic universal smoothing factors, kernel complexity and Yatracos classes. Ann. Statist. 25 2626-2637. · Zbl 0897.62035
[23] Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1996). Density estimation by wavelet thresholding. Ann. Statist. 24 508-539. · Zbl 0860.62032
[24] Efroĭmovich, S. Y. and Pinsker, M. S. (1984). A self-training algorithm for nonparametric filtering. Automat. Remote Control 45 58-65.
[25] Efroĭmovich, S. Y. (1986). Nonparametric estimation of the density with unknown smoothness. Theory Probab. Appl. 30 557-568.
[26] Efromovich, S. (2008). Adaptive estimation of and oracle inequalities for probability densities and characteristic functions. Ann. Statist. 36 1127-1155. · Zbl 1360.62118
[27] Efromovich, S. and Low, M. G. (1994). Adaptive estimates of linear functionals. Probab. Theory Related Fields 98 261-275. · Zbl 0796.62037
[28] Folland, G. B. (1999). Real Analysis : Modern Techniques and Their Applications , 2nd ed. Wiley, New York. · Zbl 0924.28001
[29] Gach, F., Nickl, R. and Spokoiny, V. (2013). Spatially adaptive density estimation by localised Haar projections. Ann. Inst. Henri Poincaré Probab. Stat. 49 900-914. · Zbl 1355.62009
[30] Giné, E. and Nickl, R. (2009). An exponential inequality for the distribution function of the kernel density estimator, with applications to adaptive estimation. Probab. Theory Related Fields 143 569-596. · Zbl 1160.62032
[31] Goldenshluger, A. (2009). A universal procedure for aggregating estimators. Ann. Statist. 37 542-568. · Zbl 1155.62018
[32] Goldenshluger, A. and Lepski, O. (2008). Universal pointwise selection rule in multivariate function estimation. Bernoulli 14 1150-1190. · Zbl 1168.62323
[33] Goldenshluger, A. and Lepski, O. (2009). Structural adaptation via \(\mathbb{L}_{p}\)-norm oracle inequalities. Probab. Theory Related Fields 143 41-71. · Zbl 1149.62020
[34] Goldenshluger, A. and Lepski, O. (2011). Bandwidth selection in kernel density estimation: Oracle inequalities and adaptive minimax optimality. Ann. Statist. 39 1608-1632. · Zbl 1234.62035
[35] Goldenshluger, A. V. and Lepski, O. V. (2013). General selection rule from a family of linear estimators. Theory Probab. Appl. 57 209-226. · Zbl 1417.62056
[36] Goldenshluger, A. and Lepski, O. (2014). On adaptive minimax density estimation on \(R^{d}\). Probab. Theory Related Fields 159 479-543. · Zbl 1342.62053
[37] Goldenshluger, A. and Nemirovski, A. (1997). Spatial adaptive estimation of smooth nonparametric regression functions. Math. Methods Statist. 6 135-170. · Zbl 0892.62018
[38] Golubev, G. K. (1992). Nonparametric estimation of smooth densities of a distribution in \(L_{2}\). Probl. Inform. Transm. 1 52-62.
[39] Golubev, G. K. and Nussbaum, M. (1992). Adaptive spline estimates in a nonparametric regression model. Theory Probab. Appl. 37 553-560. · Zbl 0787.62044
[40] Hasminskii, R. and Ibragimov, I. (1990). On density estimation in the view of Kolmogorov’s ideas in approximation theory. Ann. Statist. 18 999-1010. · Zbl 0705.62039
[41] Horowitz, J. L. and Mammen, E. (2007). Rate-optimal estimation for a general class of nonparametric regression models with unknown link functions. Ann. Statist. 35 2589-2619. · Zbl 1129.62034
[42] Hristache, M., Juditsky, A., Polzehl, J. and Spokoiny, V. (2001). Structure adaptive approach for dimension reduction. Ann. Statist. 29 1537-1566. · Zbl 1043.62052
[43] Ibragimov, I. A. and Has’minskiĭ, R. Z. (1981). Statistical Estimation : Asymptotic Theory. Applications of Mathematics 16 . Springer, New York.
[44] Johnstone, I. M. and Silverman, B. W. (2005). Empirical Bayes selection of wavelet thresholds. Ann. Statist. 33 1700-1752. · Zbl 1078.62005
[45] Juditsky, A. (1997). Wavelet estimators: Adapting to unknown smoothness. Math. Methods Statist. 6 1-25. · Zbl 0871.62039
[46] Juditsky, A. and Lambert-Lacroix, S. (2004). On minimax density estimation on \(\mathbb{R}\). Bernoulli 10 187-220. · Zbl 1076.62037
[47] Juditsky, A. B., Lepski, O. V. and Tsybakov, A. B. (2009). Nonparametric estimation of composite functions. Ann. Statist. 37 1360-1404. · Zbl 1160.62030
[48] Juditsky, A. and Nemirovski, A. (2000). Functional aggregation for nonparametric regression. Ann. Statist. 28 681-712. · Zbl 1105.62338
[49] Kerkyacharian, G., Lepski, O. and Picard, D. (2001). Nonlinear estimation in anisotropic multi-index denoising. Probab. Theory Related Fields 121 137-170. · Zbl 1010.62029
[50] Kerkyacharian, G., Lepski, O. and Picard, D. (2008). Nonlinear estimation in anisotropic multiindex denoising. Sparse case. Theory Probab. Appl. 52 58-77. · Zbl 1315.62031
[51] Kerkyacharian, G., Nickl, R. and Picard, D. (2012). Concentration inequalities and confidence bands for needlet density estimators on compact homogeneous manifolds. Probab. Theory Related Fields 153 363-404. · Zbl 06062621
[52] Kerkyacharian, G., Thanh, M. and Picard, D. (2011). Localized spherical deconvilution. Ann. Statist. 39 1042-1068. · Zbl 1216.62059
[53] Lepski, O. (2013). Multivariate density estimation under sup-norm loss: Oracle approach, adaptation and independence structure. Ann. Statist. 41 1005-1034. · Zbl 1360.62158
[54] Lepski, O. V. (2015). Upper functions for \({\mathbb{L}}_{p}\)-norm of Gaussian random fields. Bernoulli . To appear. Available at . arXiv:1311.4996v1
[55] Lepski, O. V. and Levit, B. Y. (1998). Adaptive minimax estimation of infinitely differentiable functions. Math. Methods Statist. 7 123-156. · Zbl 1103.62332
[56] Lepski, O. V., Mammen, E. and Spokoiny, V. G. (1997). Optimal spatial adaptation to inhomogeneous smoothness: An approach based on kernel estimates with variable bandwidth selectors. Ann. Statist. 25 929-947. · Zbl 0885.62044
[57] Lepski, O. and Serdyukova, N. (2014). Adaptive estimation under single-index constraint in a regression model. Ann. Statist. 42 1-28. · Zbl 1302.62077
[58] Lepskiĭ, O. V. (1990). A problem of adaptive estimation in Gaussian white noise. Theory Probab. Appl. 35 459-470. · Zbl 0725.62075
[59] Lepskiĭ, O. V. (1991). Asymptotically minimax adaptive estimation. I. Upper bounds. Optimally adaptive estimates. Theory Probab. Appl. 36 682-697. · Zbl 0776.62039
[60] Lepskiĭ, O. V. (1992a). Asymptotically minimax adaptive estimation. II. Schemes without optimal adaptation. Adaptive estimates. Theory Probab. Appl. 37 468-481.
[61] Lepskiĭ, O. V. (1992b). On problems of adaptive estimation in white Gaussian noise. In Topics in Nonparametric Estimation. Adv. Soviet Math. 12 87-106. Amer. Math. Soc., Providence, RI.
[62] Leung, G. and Barron, A. R. (2006). Information theory and mixing least-squares regressions. IEEE Trans. Inform. Theory 52 3396-3410. · Zbl 1309.94051
[63] Müller, H.-G. and Stadtmüller, U. (1987). Variable bandwidth kernel estimators of regression curves. Ann. Statist. 15 182-201. · Zbl 0634.62032
[64] Nemirovski, A. (2000). Topics in nonparametric statistics. In Lectures on Probability Theory and Statistics ( Saint-Flour , 1998). Lecture Notes in Math. 1738 85-277. Springer, Berlin. · Zbl 0998.62033
[65] Nemirovskiy, A. S. (1985). Nonparametric estimation of smooth regression functions. Soviet J. Comput. Systems Sci. 23 1-11. · Zbl 0604.62033
[66] Neumann, M. H. (2000). Multivariate wavelet thresholding in anisotropic function spaces. Statist. Sinica 10 399-431. · Zbl 0982.62039
[67] Nikol’skiĭ, S. M. (1977). Priblizhenie Funktsii Mnogikh Peremennykh i Teoremy Vlozheniya , 2nd ed. Nauka, Moscow.
[68] Reynaud-Bouret, P., Rivoirard, V. and Tuleau-Malot, C. (2011). Adaptive density estimation: A curse of support? J. Statist. Plann. Inference 141 115-139. · Zbl 1197.62033
[69] Rigollet, P. (2006). Adaptive density estimation using the blockwise Stein method. Bernoulli 12 351-370. · Zbl 1098.62040
[70] Rigollet, P. and Tsybakov, A. B. (2007). Linear and convex aggregation of density estimators. Math. Methods Statist. 16 260-280. · Zbl 1231.62057
[71] Rigollet, P. and Tsybakov, A. (2011). Exponential screening and optimal rates of sparse estimation. Ann. Statist. 39 731-771. · Zbl 1215.62043
[72] Samarov, A. and Tsybakov, A. (2007). Aggregation of density estimators and dimension reduction. In Advances in Statistical Modeling and Inference. Ser. Biostat. 3 233-251. World Scientific, Hackensack, NJ.
[73] Tsybakov, A. B. (1998). Pointwise and sup-norm sharp adaptive estimation of functions on the Sobolev classes. Ann. Statist. 26 2420-2469. · Zbl 0933.62028
[74] Tsybakov, A. (2003). Optimal rate of aggregation. In Proc. COLT. Lecture Notes in Artificial Intelligence 2777 303-313. Springer, New York. · Zbl 1208.62073
[75] Tsybakov, A. B. (2009). Introduction to Nonparametric Estimation . Springer, New York. · Zbl 1176.62032
[76] Wegkamp, M. (2003). Model selection in nonparametric regression. Ann. Statist. 31 252-273. · Zbl 1019.62037
[77] Zhang, C.-H. (2005). General empirical Bayes wavelet methods and exactly adaptive minimax estimation. Ann. Statist. 33 54-100. · Zbl 1064.62009
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.