×

Quantile regression with group Lasso for classification. (English) Zbl 1414.62318

Summary: Applications of regression models for binary response are very common and models specific to these problems are widely used. Quantile regression for binary response data has recently attracted attention and regularized quantile regression methods have been proposed for high dimensional problems. When the predictors have a natural group structure, such as in the case of categorical predictors converted into dummy variables, then a group lasso penalty is used in regularized methods. In this paper, we present a Bayesian Gibbs sampling procedure to estimate the parameters of a quantile regression model under a group lasso penalty for classification problems with a binary response. Simulated and real data show a good performance of the proposed method in comparison to mean-based approaches and to quantile-based approaches which do not exploit the group structure of the predictors.

MSC:

62J07 Ridge regression; shrinkage estimators (Lasso)
62H30 Classification and discrimination; cluster analysis (statistical aspects)
62F15 Bayesian inference
PDF BibTeX XML Cite
Full Text: DOI Link

References:

[1] Alhamzawi, R.; Yu, K., Conjugate priors and variable selection for Bayesian quantile regression, Comput Stat Data Anal, 64, 209-219, (2013) · Zbl 1468.62015
[2] Alhamzawi, R.; Yu, K.; Benoit, D., Bayesian adaptive lasso quantile regression, Stat Model, 12, 279-297, (2012) · Zbl 1306.65029
[3] Andrews, DF; Mallows, CL, Scale mixtures of normal distributions, J R Stat Soc Ser B, 36, 99-102, (1974) · Zbl 0282.62017
[4] Bach, F., Consistency of the group lasso and multiple kernel learning, J Mach Learn Res, 9, 1179-1225, (2008) · Zbl 1225.68147
[5] Bae, K.; Mallick, B., Gene selection using a two-level hierarchical Bayesian model, Bioinformatics, 20, 3423-3430, (2004)
[6] Belloni, A.; Chernozhukov, V., Post l\(_1\)-penalized quantile regression in high-dimensional sparse models, Ann Stat, 39, 82-130, (2011) · Zbl 1209.62064
[7] Benoit, D.; Poel, D., Binary quantile regression: a Bayesian approach based on the asymmetric laplace density, J Appl Econ, 27, 1174-1188, (2012)
[8] Breheny, P.; Huang, J., Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors, Stat Comput, 25, 173-187, (2015) · Zbl 1331.62359
[9] Davino C, Furno M, Vistocco D (2013) Quantile regression: theory and applications. Wiley, Chichester · Zbl 1286.62004
[10] Friedman, J.; Hastie, T.; Tibshirani, R., Regularization paths for generalized linear models via coordinate descent, J Stat Softw, 31, 1-22, (2010)
[11] Genkin, A.; Lewis, DD; Madigan, D., Large-scale Bayesian logistic regression for text categorization, Technometrics, 49, 291-304, (2007)
[12] Geraci, M.; Bottai, M., Quantile regression for longitudinal data using the asymmetric Laplace distribution, Biostatistics, 8, 140-154, (2007) · Zbl 1170.62380
[13] Gramacy, R.; Polson, N., Simulation-based regularized logistic regression, Bayesian Anal, 7, 503-770, (2012) · Zbl 1330.62301
[14] Hand, D.; Vinciotti, V., Local versus global models for classification problems: fitting models where it matters, Am Stat, 57, 124-131, (2003)
[15] Huang, J.; Zhang, T., The benefit of group sparsity, Ann Stat, 38, 1978-2004, (2010) · Zbl 1202.62052
[16] Ji, Y.; Lin, N.; Zhang, B., Model selection in binary and tobit quantile regression using the Gibbs sampler, Comput Stat Data Anal, 56, 827-839, (2012) · Zbl 1243.62033
[17] Koenker R (2005) Quantile regression. CRC Press, Boca Raton · Zbl 1111.62037
[18] Koenker, R.; Bassett, GW, Regression quantiles, Econometrica, 46, 33-50, (1978) · Zbl 0373.62038
[19] Kordas G (2002) Credit scoring using binary quantile regression. In: Statistical data analysis based on the L1-norm and related methods. Statistics for industry and technology. Birkhäuser, Basel, pp 125-137 · Zbl 1145.62386
[20] Kordas, G., Smoothed binary regression quantiles, J Appl Econ, 21, 387-407, (2006)
[21] Kozumi, H.; Kobayashi, G., Gibbs sampling methods for Bayesian quantile regression, J Stat Comput Simul, 81, 1565-1578, (2011) · Zbl 1431.62018
[22] Krishnapuram, B.; Carin, L.; Figueiredo, MA; Hartemink, AJ, Sparse multinomial logistic regression: fast algorithms and generalization bounds, IEEE Trans Pattern Anal Mach Intell, 27, 957-968, (2005)
[23] Li, Y.; Zhu, J., L1-norm quantile regressions, J Comput Graph Stat, 17, 163-185, (2008)
[24] Li, Q.; Xi, R.; Lin, N., Bayesian regularized quantile regression, Bayesian Anal, 5, 1-24, (2010) · Zbl 1330.62143
[25] Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml
[26] Liu, X.; Wang, Z.; Wu, Y., Group variable selection and estimation in the tobit censored response model, Comput Stat Data Anal, 60, 80-89, (2013) · Zbl 1365.62281
[27] Lounici, K.; Pontil, M.; Tsybakov, A.; Geer, S., Oracle inequalities and optimal inference under group sparsity, Ann Stat, 39, 2164-2204, (2011) · Zbl 1306.62156
[28] Lum, K.; Gelfand, A., Spatial quantile multiple regression using the asymmetric laplace process, Bayesian Anal, 7, 235-258, (2012) · Zbl 1330.62197
[29] Manski, C., Maximum score estimation of the stochastic utility model of choice, J Econ, 3, 205-228, (1975) · Zbl 0307.62068
[30] Manski, C., Semiparametric analysis of discrete response: asymptotic properties of the maximum score estimator, J Econ, 27, 313-333, (1985) · Zbl 0567.62096
[31] Meier, L.; Geer, S.; Bühlmann, P., The group lasso for logistic regression, J R Stat Soc Ser B, 70, 53-71, (2008) · Zbl 1400.62276
[32] Miguéis, LV; Benoit, DF; Poel, D., Enhanced decision support in credit scoring using Bayesian binary quantile regression, J Oper Res Soc, 64, 1374-1383, (2013)
[33] Powell, J., Least absolute deviations estimation for the censored regression model, J Econ, 25, 303-325, (1984) · Zbl 0571.62100
[34] Sharma, D.; Bondell, H.; Zhang, H., Consistent group identification and variable selection in regression with correlated predictors, J Comput Graph Stat, 22, 319-340, (2013)
[35] Simon, N.; Friedman, J.; Hastie, T.; Tibshirani, R., A sparse-group lasso, J Comput Graph Stat, 22, 231-245, (2013)
[36] Tibshirani, R., Regression shrinkage and selection via the lasso, J R Stat Soc Ser B, 58, 267-288, (1996) · Zbl 0850.62538
[37] Tibshirani, R., The lasso method for variable selection in the cox model, Stat Med, 16, 385-395, (1997)
[38] Wei, F.; Huang, J., Consistent group selection in high-dimensional linear regression, Stat Med, 16, 1369-1384, (2010) · Zbl 1207.62146
[39] Yang Y, Zou H (2015) A fast unified algorithm for solving group-lasso penalized learning problems. Stat Comput (to appear) · Zbl 1331.62343
[40] Yu, K.; Moyeed, R., Bayesian quantile regression, Stat Probab Lett, 54, 437-447, (2001) · Zbl 0983.62017
[41] Yu, K.; Cathy, C.; Reed, C.; Dunson, D., Bayesian variable selection in quantile regression, Stat Interface, 6, 261-274, (2013) · Zbl 1327.62135
[42] Yuan, M.; Lin, Y., Model selection and estimation in regression with grouped variables, J R Stat Soc Ser B, 68, 49-67, (2006) · Zbl 1141.62030
[43] Zou, H., The adaptive lasso and its oracle properties, J Am Stat Assoc, 101, 1418-1429, (2006) · Zbl 1171.62326
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.