×

zbMATH — the first resource for mathematics

Component selection in the additive regression model. (English) Zbl 1364.62091
Summary: Similar to variable selection in the linear model, selecting significant components in the additive model is of great interest. However, such components are unknown, unobservable functions of independent variables. Some approximation is needed. We suggest a combination of penalized regression spline approximation and group variable selection, called the group-bridge-type spline method (GBSM), to handle this component selection problem with a diverging number of correlated variables in each group. The proposed method can select significant components and estimate non-parametric additive function components simultaneously. To make the GBSM stable in computation and adaptive to the level of smoothness of the component functions, weighted power spline bases and projected weighted power spline bases are proposed. Their performance is examined by simulation studies. The proposed method is extended to a partial linear regression model analysis with real data, and gives reliable results.

MSC:
62G08 Nonparametric regression and quantile regression
62J07 Ridge regression; shrinkage estimators (Lasso)
Software:
Excel; hgam
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Albright, Data analysis and decision making with Microsoft Excel (1999)
[2] Amato, Adaptive wavelet series estimation in separable non-parametric regression models, Statist. Comput. 11 pp 371– (2001) · doi:10.1023/A:1011929305660
[3] Amato, Fourier series approximation of separable models, J. Comput. Appl. Math. 146 pp 459– (2002) · Zbl 1058.62034 · doi:10.1016/S0377-0427(02)00398-9
[4] Buja, Linear smoothers and additive models, Ann. Statist. 17 pp 435– (1989) · Zbl 0689.62029
[5] Chen, Nonparametric estimation of additive separable regression models, Statistical Theory and Computational Aspects of Smoothing (1996) · doi:10.1007/978-3-642-48425-4_18
[6] De Boor, A practical guide to splines (1978) · doi:10.1007/978-1-4612-6333-3
[7] Donoho, Ideal spatial adaptation by wavelet shrinkage, Biometrika 81 pp 425– (1994) · Zbl 0815.62019 · doi:10.1093/biomet/81.3.425
[8] Eilers, Generalized linear additive smooth structures, J. Comput. Graph. Statist. 11 pp 758– (2002) · doi:10.1198/106186002844
[9] Fan, Nonparametric independence screening in sparse ultra-high dimensional additive models, J. Amer. Statist. Assoc. 116 pp 544– (2009)
[10] Fan, Direct estimation of additive and linear components for high-dimensional data, Ann. Statist. 26 pp 943– (1998) · Zbl 1073.62527 · doi:10.1214/aos/1024691083
[11] Fan, Variable selection via nonconcave penalized likelihood and its oracle properties, J. Amer. Statist. Assoc. 96 pp 1348– (2001) · Zbl 1073.62547 · doi:10.1198/016214501753382273
[12] Fan, Nonconcave penalized likelihood with a diverging number of parameters, Ann Statist. 32 pp 928– (2004) · Zbl 1092.62031 · doi:10.1214/009053604000000256
[13] Friedman, Flexible parsimonious smoothing and additive modelling, Technometrics 31 pp 3– (1989) · Zbl 0672.65119 · doi:10.1080/00401706.1989.10488470
[14] Friedman, Projection pursuit regression, J. Amer. Statist. Assoc. 76 pp 817– (1981) · doi:10.1080/01621459.1981.10477729
[15] Hastie, Generalized additive models (1990) · Zbl 0747.62061
[16] Huang, Variable selection in non-parametric additive models, Ann. Statist. 38 pp 2282– (2010) · Zbl 1202.62051 · doi:10.1214/09-AOS781
[17] Huang, A group bridge approach for variable selection, Biometrika 96 pp 339– (2009) · Zbl 1163.62050 · doi:10.1093/biomet/asp020
[18] Lam, Profile-kernel likelihood inference with diverging number of parameters, Ann. Statist. 36 pp 2232– (2008) · Zbl 1274.62289 · doi:10.1214/07-AOS544
[19] Lin, Component selection and smoothing in multivariate non-parametric regression, Ann. Statist. 34 pp 2272– (2006) · Zbl 1106.62041 · doi:10.1214/009053606000000722
[20] Linton, A kernel method of estimating structured nonparametric regression based on marginal integration, Biometrika 82 pp 93– (1995) · Zbl 0823.62036 · doi:10.1093/biomet/82.1.93
[21] Luo, Hybrid adaptive splines, J. Amer. Statist. Assoc. 92 pp 107– (1997) · doi:10.1080/01621459.1997.10473607
[22] Mammen, The existence and asymptotic properties of a backfitting projection algorithm under weak conditions, Ann. Statist. 27 pp 1443– (1999) · Zbl 0986.62028 · doi:10.1214/aos/1017939138
[23] Mammen, Bandwidth selection for smooth backfitting in additive models, Ann. Statist. 33 pp 1260– (2005) · Zbl 1072.62025 · doi:10.1214/009053605000000101
[24] Mammen, A simple smooth backfitting method for additive models, Ann. Statist. 34 pp 2252– (2006) · Zbl 1106.62042 · doi:10.1214/009053606000000696
[25] Mammen, Locally adaptive regression splines, Ann. Statist. 25 pp 387– (1997) · Zbl 0871.62040 · doi:10.1214/aos/1034276635
[26] Meier, High-dimensional additive modelling, Ann. Statist. 37 pp 3779– (2009) · Zbl 1360.62186 · doi:10.1214/09-AOS692
[27] Nielsen, Smooth backfitting in practice., J. Roy. Statist. Soc. Ser. B Statist. Methodol. 67 pp 43– (2005) · Zbl 1060.62048 · doi:10.1111/j.1467-9868.2005.00487.x
[28] Opsomer, A fully automated bandwith selection method for fitting additive models, J. Amer. Statist. Assoc. 93 pp 605– (1998) · Zbl 0953.62034 · doi:10.1080/01621459.1998.10473714
[29] Ravikumar, Sparse additive models, J. Roy. Statist. Soc. Ser. B Statist. Methodol. 71 pp 1009– (2009) · doi:10.1111/j.1467-9868.2009.00718.x
[30] Sardy, AMlet, RAMlet, & GAMlet: automatic nonlinear fitting of additive models, robust and generalised with wavelets, J. Comput. Graph. Statist. 13 pp 283– (2004) · doi:10.1198/1061860043434
[31] Shively, Variable selection and function estimation in additive non-parametric regression using a data-based prior, J. Amer. Statist. Assoc. 94 pp 777– (1999) · Zbl 0994.62033 · doi:10.1080/01621459.1999.10474180
[32] Smith, Non-parametric regression using bayesian variable selection, J. Econometrics 75 pp 317– (1996) · Zbl 0864.62025 · doi:10.1016/0304-4076(95)01763-1
[33] Stone, Additive regression and other non-parametric models, Ann. Statist. 13 pp 689– (1985) · Zbl 0605.62065 · doi:10.1214/aos/1176349548
[34] Stone, The dimensionality reduction principal for generalized additive models, Ann. Statist. 14 pp 590– (1986) · Zbl 0603.62050 · doi:10.1214/aos/1176349940
[35] Tjøstheim, J. Amer. Statist. Assoc. 89 pp 1398– (1994)
[36] van de Geer, Empirical processes in M-estimation (2000)
[37] Wang, Tuning parameter selectors for the smoothly clipped absolute deviation method, Biometrika 94 pp 553– (2007) · Zbl 1135.62058 · doi:10.1093/biomet/asm053
[38] Yu, Smooth backfitting in generalized additive models, Ann. Statist. 36 pp 228– (2008) · Zbl 1132.62028 · doi:10.1214/009053607000000596
[39] Zhang, Prediction error estimation under bregman divergence for non-parametric regression and classification, Scand. J. Statist. 35 pp 496– (2008) · Zbl 1195.62043 · doi:10.1111/j.1467-9469.2008.00593.x
[40] Zhou, Croup variable selection via a hierarchical lasso and its oracle property, Statist. Its Interface 3 pp 557– (2010) · Zbl 1245.62183 · doi:10.4310/SII.2010.v3.n4.a13
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.