×

Penalized estimation equation for an extended single-index model. (English) Zbl 1398.62182

Summary: The single-index model is a useful extension of the linear regression model. In [Ann. Stat. 39, No. 3, 1658–1688 (2011; Zbl 1221.62062)], X. Cui et al. proposed an estimating function method for the estimation of index vector in an extended single-index model (ESIM). Nevertheless, how to conduct variable selection for ESIM has not been studied. To solve this problem, we penalize the estimating equation with some types of penalty, such as smoothly clipped absolute deviation penalty and adaptive Lasso penalty. Under some regularity conditions, the oracle property is established, i.e., the resulting estimator can be as efficient as the oracle estimator, thus we improve the explanatory ability and accuracy of estimator for the ESIM. A novel algorithm is proposed to solve the penalized estimating equation by combining quasi-Fisher scoring type algorithm and MM algorithm. Simulation study and real data application demonstrate the excellent performance of the proposed estimators.

MSC:

62J05 Linear regression; mixed models
62J07 Ridge regression; shrinkage estimators (Lasso)
62H12 Estimation in multivariate analysis
62G08 Nonparametric regression and quantile regression

Citations:

Zbl 1221.62062
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Carroll, R. J., Fan, J., Gijbels, I., Wand, M. P. (1997). Generalized partially linear single-index models. Journal of the American Statistical Association, 92, 477-489. · Zbl 0890.62053
[2] Cui, X., Härdle, W., Zhu, L. X. (2011). The EFM approach for single-index models. Annals of Statistics, 39, 1658-1688. · Zbl 1221.62062
[3] Donoho, D. L., Johnstone, I. M. (1994). Ideal spatial adaptation by wavelet shrinkage. Biometrika, 81, 425-455. · Zbl 0815.62019
[4] Fan, J., Gijbels, I. (1996). Local polynomial modeling and its applications. London: Chapman Hall. · Zbl 0873.62037
[5] Fan, J., Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96, 1348-1360. · Zbl 1073.62547
[6] Fan, J., Heckman, N. E., Wand, M. P. (1995). Local polynomial kernel regression for generalized linear models and quasi-likehood functions. Journal of the American Statistical Association, 90, 141-150. · Zbl 0818.62036
[7] Härdle, W., Stoker, T. M. (1989). Investigating smooth multiple regression by the method of average derivatives. Journal of the American Statistical Association, 84, 985-995. · Zbl 0703.62052
[8] Härdle, W., Hall, P., Ichimura, H. (1993). Optimal smoothing in single-index models. Annals of Statistics, 21, 157-178. · Zbl 0770.62049
[9] Hristache, M., Juditsky, A., Spokoiny, V. (2001). Direct estimation of the index coeffcient in a single-index model. Annals of Statistics, 29, 595-623. · Zbl 1012.62043
[10] Hunter, D. R., Li, R. Z. (2005). Variable selection using MM algorithms. Annals of Statistics, 33, 1617-1642. · Zbl 1078.62028
[11] Jeng, X. J., Daye, Z. J. (2011). Sparse convariance thresholding for high-dimensional variable selection. Statistica Sinica, 21, 625-657. · Zbl 1214.62059
[12] Johnson, B. A., Lin, D. Y., Zeng, D. L. (2008). Penalized estimating functions and variable selection in semiparametric regression models. Journal of the American Statistical Association, 103, 672-680. · Zbl 1471.62330
[13] Kane, M., Holt, J., Alen, B. (2004). Results concerning the generalized partically linear single-index model. Journal of Statistical Computation and Simulation, 72, 897-912. · Zbl 1054.62044
[14] Kong, E., Xia, Y. C. (2007). Variable selection for the single-index model. Biometrika, 94, 217-229. · Zbl 1142.62353
[15] Li, K. C. (1991). Sliced inverse regression for dimension reduction (with discussion). Journal of the American Statistical Association, 86, 316-342. · Zbl 0742.62044 · doi:10.1080/01621459.1991.10475035
[16] Powell, J. L., Stock, J. H., Stoker, T. M. (1989). Semiparametric estimation of index coeffcients. Econometrika, 57, 1403-1430. · Zbl 0683.62070
[17] Tibshirani, R. J. (1996). Regression shrinkage and selection via the LASSO. Journal of the Royal Statistical Society : Series B, 58, 267-288. · Zbl 0850.62538
[18] Wang, H., Leng, C. (2007). Unified lasso estimation via least squares approximation. Journal of the American Statistical Association, 102, 1039-1048. · Zbl 1306.62167
[19] Wang, H. S., Xia, Y. C. (2009). Shrinkage estimation of the varying coefficient model. Journal of the American Statistical Association, 104, 747-757. · Zbl 1388.62213
[20] Xia, Y. C., Tong, H., Li, W. K., Zhu, L. X. (2002). An adaptive estimation of optimal regression subspace. Journal of the Royal Statistical Society: Series B, 64, 363-410. · Zbl 1091.62028
[21] Xu, P. R., Zhu, L. X. (2012). Estimation for a marginal generalized single-index longitudinal model. Journal of Multivariate Analysis, 105, 285-299. · Zbl 1233.62073
[22] Zeng, P., He, T. H., Zhu, Y. (2012). A Lasso-type approach for estimation and variable selection in single index models. Journal of Computational and Graphical Statistics, 21, 92-109.
[23] Zhang, Q. Z., Wang, Q. H. (2013). Local least absolute relative error estimating approach for partially linear multiplicative model. Statistica Sinica, 23, 1091-1116. · Zbl 06202699
[24] Zhu, L. P., Zhu, L. X. (2009). Nonconcave penalized inverse regression in single-index models with high dimensional predictors. Journal of Multivariate Analysis, 100, 862-875. · Zbl 1157.62037
[25] Zhu, L. P., Qian, L. Y., Lin, J. G. (2011). Variable selection in a class of single-index models. Annals of the Institute of Statistical Mathematics, 63, 1277-1293. · Zbl 1230.62062
[26] Zou, H. (2006). The adaptive lasso and its oracle properties. Journal of the American Statistical Association, 101, 1418-1429. · Zbl 1171.62326 · doi:10.1198/016214506000000735
[27] Zou, H., Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B, 67, 310-320. · Zbl 1069.62054
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.