zbMATH — the first resource for mathematics

Confidence sets based on penalized maximum likelihood estimators in Gaussian regression. (English) Zbl 1329.62156
Summary: Confidence intervals based on penalized maximum likelihood estimators such as the LASSO, adaptive LASSO, and hard-thresholding are analyzed. In the known-variance case, the finite-sample coverage properties of such intervals are determined and it is shown that symmetric intervals are the shortest. The length of the shortest intervals based on the hard-thresholding estimator is larger than the length of the shortest interval based on the adaptive LASSO, which is larger than the length of the shortest interval based on the LASSO, which in turn is larger than the standard interval based on the maximum likelihood estimator. In the case where the penalized estimators are tuned to possess the ‘sparsity property’, the intervals based on these estimators are larger than the standard interval by an order of magnitude. Furthermore, a simple asymptotic confidence interval construction in the ‘sparse’ case, that also applies to the smoothly clipped absolute deviation estimator, is discussed. The results for the known-variance case are shown to carry over to the unknown-variance case in an appropriate asymptotic sense.

62F25 Parametric tolerance and confidence regions
62J07 Ridge regression; shrinkage estimators (Lasso)
Full Text: DOI Euclid arXiv
[1] Fan, J. and Li, R. (2001). Variable Selection Via Nonconcave Penalized Likelihood and Its Oracle Properties., Journal of the American Statistical Association 96 1348-1360. · Zbl 1073.62547 · doi:10.1198/016214501753382273
[2] Frank, I. E. and Friedman, J. H. (1993). A Statistical View of Some Chemometrics Regression Tools (with discussion)., Technometrics 35 109-148. · Zbl 0775.62288 · doi:10.2307/1269656
[3] Joshi, V. M. (1969). Admissibility of the usual confidence sets for the mean of a univariate or bivariate normal population., Annals of Mathematical Statistics 40 1042-1067. · Zbl 0205.46202 · doi:10.1214/aoms/1177697608
[4] Kagan, A. and Nagaev, A. V. (2008). A lemma on stochastic majorization and properties of the Student distribution., Theory of Probability and its Applications 52 160-164. · Zbl 1147.60307 · doi:10.1137/S0040585X97982931
[5] Knight, K. and Fu, W. (2000). Asymptotics of Lasso-Type Estimators., Annals of Statistics 28 1356-1378. · Zbl 1105.62357 · doi:10.1214/aos/1015957397
[6] Leeb, H. and Pötscher, B. M. (2008). Sparse Estimators and the Oracle Property, or the Return of Hodges’ Estimator., Journal of Econometrics 142 201-211. · Zbl 1418.62272 · doi:10.1016/j.jeconom.2007.05.017
[7] Pötscher, B. M. (2009). Confidence Sets Based on Sparse Estimators Are Necessarily Large., Sankya 71-A 1-18. · Zbl 1192.62096 · sankhya.isical.ac.in
[8] Pötscher, B. M. and Leeb, H. (2009). On the Distribution of Penalized Maximum Likelihood Estimators: The LASSO, SCAD, and Thresholding., Journal of Multivariate Analysis 100 2065-2082. · Zbl 1170.62046 · doi:10.1016/j.jmva.2009.06.010
[9] Pötscher, B. M. and Schneider, U. (2009). On the Distribution of the Adaptive LASSO Estimator., Journal of Statistical Planning and Inference 139 2775-2790. · Zbl 1162.62063 · doi:10.1016/j.jspi.2009.01.003
[10] Tibshirani, R. (1996). Regression Shrinkage and Selection Via the Lasso., Journal of the Royal Statistical Society Series B 58 267-288. · Zbl 0850.62538
[11] Zou, H. (2006). The Adaptive Lasso and Its Oracle Properties., Journal of the American Statistical Association 101 1418-1429. · Zbl 1171.62326 · doi:10.1198/016214506000000735
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.