×

zbMATH — the first resource for mathematics

Asymptotic properties of bridge estimators in sparse high-dimensional regression models. (English) Zbl 1133.62048
Summary: We study the asymptotic properties of bridge estimators in sparse, high-dimensional, linear regression models when the number of covariates may increase to infinity with the sample size. We are particularly interested in the use of bridge estimators to distinguish between covariates whose coefficients are zero and covariates whose coefficients are nonzero. We show that under appropriate conditions, bridge estimators correctly select covariates with nonzero coefficients with probability converging to one and that the estimators of nonzero coefficients have the same asymptotic distribution that they would have if the zero coefficients were known in advance. Thus, bridge estimators have an oracle property in the sense of J. Fan and R. Li [J. Am. Stat. Assoc. 96, No. 456, 1348–1360 (2001; Zbl 1073.62547)] and J. Fan and H. Peng [Ann. Stat. 32, No. 3, 928–961 (2004; Zbl 1092.62031)]. In general, the oracle property holds only if the number of covariates is smaller than the sample size. However, under a partial orthogonality condition in which the covariates of the zero coefficients are uncorrelated or weakly correlated with the covariates of nonzero coefficients, we show that marginal bridge estimators can correctly distinguish between covariates with nonzero and zero coefficients with probability converging to one even when the number of covariates is greater than the sample size.

MSC:
62J05 Linear regression; mixed models
62F12 Asymptotic properties of parametric estimators
62E20 Asymptotic distribution theory in statistics
62J07 Ridge regression; shrinkage estimators (Lasso)
62H12 Estimation in multivariate analysis
PDF BibTeX XML Cite
Full Text: DOI arXiv
References:
[1] Bair, E., Hastie, T., Paul, D. and Tibshirani, R. (2006). Prediction by supervised principal components. J. Amer. Statist. Assoc. 101 119-137. · Zbl 1118.62326 · doi:10.1198/016214505000000628 · miranda.asa.catchword.org
[2] Bülhman, P. (2006). Boosting for high-dimensional linear models. Ann. Statist. 34 559-583. · Zbl 1095.62077 · doi:10.1214/009053606000000092
[3] Fan, J. (1997). Comment on “Wavelets in statistics: A review,” by A. Antoniadis. J. Italian Statist. Soc. 6 131-138.
[4] Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348-1360. JSTOR: · Zbl 1073.62547 · doi:10.1198/016214501753382273 · links.jstor.org
[5] Fan, J. and Li, R. (2006). Statistical challenges with high dimensionality: Feature extraction in knowledge discovery. In International Congress of Mathematicians III 595-622. Eur. Math. Soc., Zürich. · Zbl 1117.62137
[6] Fan, J. and Lv, J. (2006). Sure independence screening for ultra-high dimensional feature space. Preprint, Dept. Operational Research and Financial Engineering, Princeton Univ.
[7] Fan, J. and Peng, H. (2004). Nonconcave penalized likelihood with a diverging number of parameters. Ann. Statist. 32 928-961. · Zbl 1092.62031 · doi:10.1214/009053604000000256
[8] Fan, J., Peng, H. and Huang, T. (2005). Semilinear high-dimensional model for normalization of microarray data: A theoretical analysis and partial consistency (with discussion). J. Amer. Statist. Assoc. 100 781-813. · Zbl 1117.62330 · doi:10.1198/016214504000001781 · miranda.asa.catchword.org
[9] Frank, I. E. and Friedman, J. H. (1993). A statistical view of some chemometrics regression tools (with discussion). Technometrics 35 109-148. · Zbl 0775.62288 · doi:10.2307/1269656
[10] Fu, W. J. (1998). Penalized regressions: The bridge versus the Lasso. J. Comput. Graph. Statist. 7 397-416. JSTOR: · doi:10.2307/1390712 · links.jstor.org
[11] Hoerl, A. E. and Kennard, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12 55-67. · Zbl 0202.17205 · doi:10.2307/1267351
[12] Huang, J., Wang, D. L. and Zhang, C.-H. (2005). A two-way semilinear model for normalization and analysis of cDNA microarray data. J. Amer. Statist. Assoc. 100 814-829. · Zbl 1117.62358 · doi:10.1198/016214504000002032 · miranda.asa.catchword.org
[13] Huang, J. and Zhang, C.-H. (2005). Asymptotic analysis of a two-way semiparametric regression model for microarray data. Statist. Sinica 15 597-618. · Zbl 1086.62122
[14] Huber, P. J. (1981). Robust Statistics . Wiley, New York. · Zbl 0536.62025
[15] Hunter, D. R. and Li, R. (2005). Variable selection using MM algorithms. Ann. Statist. 33 1617-1642. · Zbl 1078.62028 · doi:10.1214/009053605000000200
[16] Knight, K. and Fu, W. J. (2000). Asymptotics for lasso-type estimators. Ann. Statist. 28 1356-1378. · Zbl 1105.62357 · doi:10.1214/aos/1015957397
[17] Kosorok, M. R. and Ma, S. (2007). Marginal asymptotics for the “large p , small n ” paradigm: With applications to microarray data. Ann. Statist. 35 1456-1486. · Zbl 1123.62005 · doi:10.1214/009053606000001433
[18] Portnoy, S. (1984). Asymptotic behavior of M estimators of p regression parameters when p 2 / n is large. I. Consistency. Ann. Statist. 12 1298-1309. · Zbl 0584.62050 · doi:10.1214/aos/1176346793
[19] Portnoy, S. (1985). Asymptotic behavior of M estimators of p regression parameters when p 2 / n is large. II. Normal approximation. Ann. Statist. 13 1403-1417. · Zbl 0601.62026 · doi:10.1214/aos/1176349744
[20] Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. J. Roy. Statist. Soc. Ser. B 58 267-288. JSTOR: · Zbl 0850.62538 · links.jstor.org
[21] van der Laan, M. J. and Bryan, J. (2001). Gene expression analysis with the parametric bootstrap. Biostatistics 2 445-461. · Zbl 1097.62571 · doi:10.1093/biostatistics/2.4.445
[22] Van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes : With Applications to Statistics . Springer, New York. · Zbl 0862.60002
[23] Zhao, P. and YU, B. (2006). On model selection consistency of Lasso. J. Mach. Learn. Res. 7 2541-2563. · Zbl 1222.62008 · www.jmlr.org
[24] Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net. J. Roy. Statist. Soc. Ser. B 67 301-320. · Zbl 1069.62054 · doi:10.1111/j.1467-9868.2005.00503.x
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.