×

Simultaneous estimation and factor selection in quantile regression via adaptive sup-norm regularization. (English) Zbl 1243.62049

Summary: Some regularization methods, including the group lasso and the adaptive group lasso, have been developed for the automatic selection of grouped variables (factors) in conditional mean regression. In many practical situations, such a problem arises naturally when a set of dummy variables is used to represent a categorical factor and/or when a set of basis functions of a continuous variable is included in the predictor set. Complementary to these earlier works, the simultaneous and automatic factor selection is examined in quantile regression. To incorporate the factor information into regularized model fitting, the adaptive sup-norm regularized quantile regression is proposed, which penalizes the empirical check loss function by the sum of factor-wise adaptive sup-norm penalties. It is shown that the proposed method possesses the oracle property. A simulation study demonstrates that the proposed method is a more appropriate tool for factor selection than the adaptive lasso regularized quantile regression.

MSC:

62G08 Nonparametric regression and quantile regression
65C60 Computational problems in statistics (MSC2010)
90C05 Linear programming
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Fan, J.; Li, R., Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of the American Statistical Association, 96, 1348-1360 (2001) · Zbl 1073.62547
[2] Hendricks, W.; Koenker, R., Hierarchical spline models for conditional quantiles and the demand for electricity, Journal of the American Statistical Association, 87, 58-68 (1992)
[3] Hosmer, D. W.; Lemeshow, S., Applied Logistic Regression (2000), Wiley: Wiley New York · Zbl 0967.62045
[4] Kim, Y.; Kim, J.; Kim, Y., Blockwise sparse regression, Statisitca Sinica, 16, 375-390 (2006) · Zbl 1096.62076
[5] Knight, K., Limiting distributions for L1 regression estimators under general conditions, The Annals of Statistics, 26, 755-770 (1998) · Zbl 0929.62021
[6] Knight, K., Asymptotics for L1-estimators of regression parameters under heteroscedasticity, The Canadian Journal of Statistics, 27, 497-507 (1999) · Zbl 0946.62024
[7] Koenker, R., Quantile regression for longitudinal data, Journal of Multivariate Analysis, 91, 74-89 (2004) · Zbl 1051.62059
[8] Koenker, R., Quantile Regression (2005), Cambridge University Press: Cambridge University Press New York · Zbl 1111.62037
[9] Koenker, R.; Bassett, G., Regression quantiles, Econometrica, 4, 33-50 (1978) · Zbl 0373.62038
[10] Koenker, R.; Geling, R., Reappraising medfly longevity: a quantile regression survival analysis, Journal of the American Statistical Association, 96, 458-468 (2001) · Zbl 1019.62100
[11] Koenker, R.; Hallock, K., Quantile regression, Journal of Economic Perspectives, 15, 143-156 (2001)
[12] Koenker, R.; Ng, P.; Portnoy, S., Quantile smoothing splines, Biometrika, 81, 673-680 (1994) · Zbl 0810.62040
[13] Kotz, S.; Kozubowski, T.; Podgorski, K., The Laplace Distribution and Generalizations: A Revisit with Applications to Communications, Economics, Engineering, and Finance (2001), Birkhauser: Birkhauser Boston · Zbl 0977.62003
[14] Kwon, S.; Choi, H.; Kim, Y., Quadratic approximation on SCAD penalized estimation, Computational Statistics and Data Analysis, 55, 421-428 (2011) · Zbl 1247.62107
[15] Kyung, M.; Gill, J.; Ghosh, M.; Casella, G., Penalized regression, standard errors, and Bayesian lassos, Bayesian Analysis, 5, 1-44 (2010) · Zbl 1330.62289
[16] Li, Y.; Zhu, J., The l1 norm quantile regression, Journal of Computational and Graphical Statistics, 17, 163-185 (2008)
[17] Meier, L.; van de Geer, S.; Buhlmann, P., The group lasso for logistic regression, Journal of the Royal Statistical Society, Series B, 70, 53-71 (2008) · Zbl 1400.62276
[18] Pollard, D., Asymptotics for least absolute deviation regression estimators, Econometric Theory, 7, 186-199 (1991)
[19] Schwarz, G., Estimating the dimension of a model, The Annals of Statistics, 6, 461-464 (1978) · Zbl 0379.62005
[20] Tibshirani, R., Regression shrinkage and selection via the lasso, Journal of the Royal Statistical Society, Series B, 58, 267-288 (1996) · Zbl 0850.62538
[21] Tibshirani, R.; Saunders, M.; Rosset, S.; Zhu, J.; Knight, K., Sparsity and smoothness via the fused lasso, Journal of the Royal Statistical Society, Series B, 67, 91-108 (2005) · Zbl 1060.62049
[22] Turlach, B., 2006. Simultaneous variable selection. In: Joint Statistical Meeting, Seattle, USA, Invited Talk (http://www.maths.uwa.edu.au/ berwin/; Turlach, B., 2006. Simultaneous variable selection. In: Joint Statistical Meeting, Seattle, USA, Invited Talk (http://www.maths.uwa.edu.au/ berwin/
[23] Turlach, B.; Venables, W.; Wright, S., Simultaneous variable selection, Technometrics, 47, 349-363 (2005)
[24] Wang, H.; He, X., Detecting differential expressions in genechip microarray studies: a quantile approach, Journal of the American Statistical Association, 102, 104-112 (2007) · Zbl 1284.62439
[25] Wang, H.; Leng, C., A note on adaptive group lasso, Computational Statistics and Data Analysis, 52, 5277-5286 (2008) · Zbl 1452.62524
[26] Wang, H.; Li, G.; Jiang, G., Robust regression shrinkage and consistent variable selection via the LAD-LASSO, Journal of Business and Economics Statistics, 25, 347-355 (2007)
[27] Wang, H.; Li, R.; Tsai, C., Tuning parameter selectors for the smoothly clipped absolute deviation method, Biometrika, 94, 553-568 (2007) · Zbl 1135.62058
[28] Wang, X.; Park, T.; Carriere, K., Regularized variable selection via combined penalization for high-dimensional data analysis, Computational Statistics and Data Analysis, 54, 2230-2243 (2010) · Zbl 1284.62065
[29] Wu, Y.; Liu, Y., Variable selection in quantile regression, Statistica Sinica, 19, 801-817 (2009) · Zbl 1166.62012
[30] Xu, J.; Ying, Z., Simultaneous estimation and variable selection in median regression using lasso-type penalty, Annals of the Institute of Statistical Mathematics, 62, 487-514 (2010) · Zbl 1440.62280
[31] Yang, S., Censored median regression using weighted empirical survival and hazard functions, Journal of the American Statistical Association, 94, 137-145 (1999) · Zbl 0997.62080
[32] Ye, G.; Xie, X., Split Bregman method for large scale fused lasso, Computational Statistics and Data Analysis, 55, 1552-1569 (2011) · Zbl 1328.65048
[33] Yuan, M., GACV for quantile smooting splines, Computational Statistics and Data Analysis, 50, 813-829 (2006) · Zbl 1432.62090
[34] Yuan, M.; Lin, Y., Model selection and estimation in regression with grouped variables, Journal of the Royal Statistical Society, Series B, 68, 49-67 (2006) · Zbl 1141.62030
[35] Zhang, H.; Liu, Y.; Wu, Y.; Zhu, J., Variable selection for multicategory svm via sup-norm regularization, Electronic Journal of Statistics, 2, 149-167 (2008) · Zbl 1135.62056
[36] Zhao, P.; Rocha, G.; Yu, B., The composite absolute penalties family for grouped and hierarchical variable selection, The Annals of Statistics, 37, 3468-3497 (2009) · Zbl 1369.62164
[37] Zou, H., The adaptive lasso and its oracle properties, Journal of the American Statistical Association, 101, 1418-1429 (2006) · Zbl 1171.62326
[38] Zou, H.; Li, R., One-step sparse estimates in nonconcave penalized likelihood models, The Annals of Statistics, 36, 1509-1533 (2008) · Zbl 1142.62027
[39] Zou, H.; Yuan, M., Regularized simultaneous model selection in multiple quantiles regression, Computational Statistics and Data Analysis, 52, 5296-5304 (2008) · Zbl 1452.62301
[40] Zou, H.; Yuan, M., The F1-norm support vector machine, Statisitca Sinica, 18, 379-398 (2008) · Zbl 1416.62370
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.