×

Resampling-based information criteria for best-subset regression. (English) Zbl 1253.62050

Summary: When a linear model is chosen by searching for the best subset among a set of candidate predictors, a fixed penalty such as that imposed by the Akaike information criterion may penalize model complexity inadequately, leading to biased model selection. We study resampling-based information criteria that aim to overcome this problem through improved estimation of the effective model dimension. The first proposed approach builds upon previous work on bootstrap-based model selection. We then propose a more novel approach based on cross-validation. Simulations and analyses of a functional neuroimaging data set illustrate the strong performance of our resampling-based methods, which are implemented in a new R package.

MSC:

62J05 Linear regression; mixed models
62B10 Statistical aspects of information-theoretic topics
62H12 Estimation in multivariate analysis
92C55 Biomedical imaging and signal processing
65C60 Computational problems in statistics (MSC2010)

Software:

subselect; R; leaps; gamair
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Akaike H. (1973) Information theory and an extension of the maximum likelihood principle. In: Petrov B.N., Csàki F. (eds) Second International Symposium on Information Theory. Akademiai Kiàdo, Budapest, pp 267–281 · Zbl 0283.62006
[2] Akaike H. (1974) A new look at the statistical model identification. IEEE Transactions on Automatic Control 19: 716–723 · Zbl 0314.62039 · doi:10.1109/TAC.1974.1100705
[3] Biswal B., Yetkin F.Z., Haughton V.M., Hyde J.S. (1995) Functional connectivity in the motor cortex of resting human brain using echo-planar MRI. Magnetic Resonance in Medicine 34: 537–541 · doi:10.1002/mrm.1910340409
[4] Cerdeira, J. O., Duarte Silva, P., Cadima, J., Minhoto, M. (2009). subselect: Selecting variable subsets. R package version 0.10-1. http://CRAN.R-project.org/package=subselect
[5] Davison A.C. (2003) Statistical Models. Cambridge University Press, Cambridge · Zbl 1044.62001
[6] Efron B. (1983) Estimating the error rate of a prediction rule: Improvement on cross-validation. Journal of the American Statistical Association 78: 316–331 · Zbl 0543.62079 · doi:10.1080/01621459.1983.10477973
[7] Efron B. (2004) The estimation of prediction error: Covariance penalties and cross-validation (with discussion). Journal of the American Statistical Association 99: 619–642 · Zbl 1117.62324 · doi:10.1198/016214504000000692
[8] Efron B., Tibshirani R. (1997) Improvements on cross-validation: The .632+ bootstrap method. Journal of the American Statistical Association 92: 548–560 · Zbl 0887.62044
[9] Foster D.P., George E.I. (1994) The risk inflation criterion for multiple regression. Annals of Statistics 22: 1947–1975 · Zbl 0829.62066 · doi:10.1214/aos/1176325766
[10] George E.I., Foster D.P. (2000) Calibration and empirical Bayes variable selection. Biometrika 87: 731–747 · Zbl 1029.62008 · doi:10.1093/biomet/87.4.731
[11] Harville D.A. (2008) Matrix Algebra from a Statistician’s Perspective. Springer, New York · Zbl 1142.15001
[12] Helland I.S. (1988) On the structure of partial least squares regression. Communications in Statistics: Theory and Methods 17: 588–607 · Zbl 0695.62167
[13] Hoerl A.E., Kennard R.W. (1970) Ridge regression: applications to nonorthogonal problems. Technometrics 12: 69–82 · Zbl 0202.17206 · doi:10.1080/00401706.1970.10488635
[14] Hurvich C.M., Tsai C.-L. (1989) Regression and time series model selection in small samples. Biometrika 76: 297–307 · Zbl 0669.62085 · doi:10.1093/biomet/76.2.297
[15] Ishiguro M., Sakamoto Y., Kitagawa G. (1997) Bootstrapping log likelihood and EIC, an extension of AIC. Annals of the Institute of Statistical Mathematics 49: 411–434 · Zbl 0935.62033 · doi:10.1023/A:1003158526504
[16] Konishi S., Kitagawa G. (1996) Generalised information criteria in model selection. Biometrika 83: 875–890 · Zbl 0883.62004 · doi:10.1093/biomet/83.4.875
[17] Konishi S., Kitagawa G. (2008) Information Criteria and Statistical Modeling. Springer, New York · Zbl 1172.62003
[18] Lawless J.F., Singhal K. (1978) Efficient screening of nonnormal regression models. Biometrics 34: 318–327 · doi:10.2307/2530022
[19] Lumley, T. (2009). using Fortran code by A. Miller. leaps: regression subset selection. R package version 2.9. http://CRAN.R-project.org/package=leaps
[20] Luo X., Stefanski L.A., Boos D.D. (2006) Tuning variable selection procedures by adding noise. Technometrics 48: 165–175 · doi:10.1198/004017005000000319
[21] Magnus J.R. (1986) The exact moments of a ratio of quadratic forms in normal variables. Annales d’Économie et de Statistique 4: 95–109
[22] Miller A. (2002) Subset Selection in Regression, 2nd ed. Chapman & Hall/CRC, Boca Raton · Zbl 1051.62060
[23] Pan W., Le C.T. (2001) Bootstrap model selection in generalized linear models. Journal of Agricultural, Biological, and Environmental Statistics 6: 49–61 · doi:10.1198/108571101300325139
[24] R Development Core Team (2010). R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria, ISBN 3-900051-07-0. http://www.R-project.org .
[25] Reiss, P. T., Huang, L., Mennes, M. (2010). Fast function-on-scalar regression with penalized basis expansions. International Journal of Biostatistics, 6, article 28.
[26] Rosenberg M. (1965) Society and the Adolescent Self-Image. Princeton University Press, Princeton, NJ
[27] Shao J. (1996) Bootstrap model selection. Journal of the American Statistical Association 91: 655–665 · Zbl 0869.62030 · doi:10.1080/01621459.1996.10476934
[28] Shen X., Ye J. (2002) Adaptive model selection. Journal of the American Statistical Association 97: 210–221 · Zbl 1073.62509 · doi:10.1198/016214502753479356
[29] Stark D.E., Margulies D.S., Shehzad Z., Reiss P.T., Kelly A.M.C., Uddin L.Q., Gee D., Roy A.K., Banich M.T., Castellanos F.X., Milham M.P. (2008) Regional variation in interhemispheric coordination of intrinsic hemodynamic fluctuations. Journal of Neuroscience 28: 13754–13764 · doi:10.1523/JNEUROSCI.4544-08.2008
[30] Stein J.L., Wiedholz L.M., Bassett D.S., Weinberger D.R., Zink C.F., Mattay V.S., Meyer-Lindenberg A. (2007) A validated network of effective amygdala connectivity. NeuroImage 36: 736–745 · doi:10.1016/j.neuroimage.2007.03.022
[31] Sugiura N. (1978) Further analysis of the data by Akaike’s information criterion and the finite corrections. Communicatons in Statistics: Theory and Methods 7: 13–26 · Zbl 0382.62060 · doi:10.1080/03610927808827599
[32] Tibshirani R. (1996) Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B 58: 267–288 · Zbl 0850.62538
[33] Tibshirani R., Knight K. (1999) The covariance inflation criterion for adaptive model selection. Journal of the Royal Statistical Society, Series B 61: 529–546 · Zbl 0924.62031 · doi:10.1111/1467-9868.00191
[34] Watson, D., Weber, K., Assenheimer, J. S., Clark, L. A., Strauss, M. E., McCormick, R. A. (1995). Testing a tripartite model: I. Evaluating the convergent and discriminant validity of anxiety and depression symptom scales. Journal of Abnormal Psychology, 104, 3–14.
[35] Wood S.N. (1994) Monotonic smoothing splines fitted by cross validation. SIAM Journal on Scientific Computing, 15: 1126–1133 · Zbl 0821.65002 · doi:10.1137/0915069
[36] Wood S.N. (2006) Generalized Additive Models: An Introduction with R. Chapman and Hall/CRC, Boca Raton
[37] Ye J. (1998) On measuring and correcting the effects of data mining and model selection. Journal of the American Statistical Association 93: 120–131 · Zbl 0920.62056 · doi:10.1080/01621459.1998.10474094
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.