×

Adaptive sparse group LASSO in quantile regression. (English) Zbl 07433030

Summary: This paper studies the introduction of sparse group LASSO (SGL) to the quantile regression framework. Additionally, a more flexible version, an adaptive SGL is proposed based on the adaptive idea, this is, the usage of adaptive weights in the penalization. Adaptive estimators are usually focused on the study of the oracle property under asymptotic and double asymptotic frameworks. A key step on the demonstration of this property is to consider adaptive weights based on a initial \(\sqrt{n}\)-consistent estimator. In practice this implies the usage of a non penalized estimator that limits the adaptive solutions to low dimensional scenarios. In this work, several solutions, based on dimension reduction techniques PCA and PLS, are studied for the calculation of these weights in high dimensional frameworks. The benefits of this proposal are studied both in synthetic and real datasets.

MSC:

62J07 Ridge regression; shrinkage estimators (Lasso)

Software:

CVXPY; ECOS
PDFBibTeX XMLCite
Full Text: DOI Link

References:

[1] Chatterjee S, Banerjee, Arindam S, Ganguly AR (2011) Sparse Group Lasso for regression on land climate variables. In: IEEE 11th international conference on data mining workshops. IEEE, pp 1-8
[2] Chiang, AP; Beck, JS; Yen, H-J; Tayeh, MK; Scheetz, TE; Swiderski, RE; Nishimura, DY; Braun, TA; Kim, K-YA; Huang, J.; Elbedour, K.; Carmi, R.; Slusarski, DC; Casavant, TL; Stone, EM; Sheffield, VC, Homozygosity mapping with SNP arrays identifies TRIM32, an E3 ubiquitin ligase, as a Bardet-Biedl syndrome gene (BBS11), Proc Natl Acad Sci, 103, 16, 6287-6292 (2006) · doi:10.1073/pnas.0600158103
[3] Chun, H.; Keleş, S., Sparse partial least squares regression for simultaneous dimension reduction and variable selection, J R Stat Soc Ser B Stat Methodol, 72, 1, 3-25 (2010) · Zbl 1411.62184 · doi:10.1111/j.1467-9868.2009.00723.x
[4] Ciuperca, G., Adaptive fused LASSO in grouped quantile regression, J Stat Theory Pract, 11, 1, 107-125 (2017) · Zbl 1420.62130 · doi:10.1080/15598608.2016.1258601
[5] Ciuperca, G., Adaptive group LASSO selection in quantile models, Stat Pap, 60, 1, 173-197 (2019) · Zbl 1411.62185 · doi:10.1007/s00362-016-0832-1
[6] Diamond S, Boyd S (2016) CVXPY: a Python-embedded modeling language for convex optimization. arXiv:1603.00943 · Zbl 1360.90008
[7] Domahidi A, Chu E, Boyd S (2013) ECOS: an SOCP solver for embedded systems. In: European control conference (ECC)
[8] Fan, J.; Li, R., Variable selection via nonconcave penalized likelihood and its oracle properties, J Am Stat Assoc, 96, 456, 1348-1360 (2001) · Zbl 1073.62547 · doi:10.1198/016214501753382273
[9] Fan, J.; Peng, H., Nonconcave penalized likelihood with a diverging number of parameters, Ann Stat, 32, 3, 928-961 (2004) · Zbl 1092.62031 · doi:10.1214/009053604000000256
[10] Friedman J, Hastie T, Tibshirani R (2010) A note on the group lasso and a sparse group lasso, pp 1-8. ArXiv:1001.0736
[11] Ghosh, S., On the grouped selection and model complexity of the adaptive elastic net, Stat Comput, 21, 451-462 (2011) · Zbl 1255.62215 · doi:10.1007/s11222-010-9181-4
[12] Huang, J.; Horowitz, JL; Ma, S., Asymptotic properties of bridge estimators in sparse high-dimensional regression models, Ann Stat, 36, 2, 587-613 (2008) · Zbl 1133.62048 · doi:10.1214/009053607000000875
[13] Huang, J.; Ma, S.; Zhang, C-H, Adaptive Lasso for sparse high-dimensional regression, Stat Sin, 1, 374, 1-28 (2008) · Zbl 1255.62198
[14] Huber, PJ; Ronchetti, EM, Robust statistics. Wiley series in probability and statistics (2009), Hoboken: Wiley, Hoboken · Zbl 1276.62022
[15] Kim, Y.; Choi, H.; Oh, HS, Smoothly clipped absolute deviation on high dimensions, J Am Stat Assoc, 103, 484, 1665-1673 (2008) · Zbl 1286.62062 · doi:10.1198/016214508000001066
[16] Koenker, R., Quantile regression (2005), Cambridge: Cambridge University Press, Cambridge · Zbl 1111.62037 · doi:10.1017/CBO9780511754098
[17] Koenker, R.; Bassett, G., Regression quantiles, Econometrica, 46, 1, 33-50 (1978) · Zbl 0373.62038 · doi:10.2307/1913643
[18] Laria, JC; Aguilera-Morillo, MC; Lillo, RE, An iterative sparse-group Lasso, J Comput Graph Stat, 28, 722-731 (2019) · Zbl 07499089 · doi:10.1080/10618600.2019.1573687
[19] Li, Y.; Zhu, J., \(L_1\)-Norm quantile regression, J Comput Graph Stat, 17, 1, 1-23 (2008) · doi:10.1198/106186008X289155
[20] Loh, PL, Statistical consistency and asymptotic normality for high-dimensional robust m-estimators, Ann Stat, 45, 2, 866-896 (2017) · Zbl 1371.62023 · doi:10.1214/16-AOS1471
[21] Nardi, Y.; Rinaldo, A., On the asymptotic properties of the group lasso estimator for linear models, Electron J Stat, 2, 605-633 (2008) · Zbl 1320.62167 · doi:10.1214/08-EJS200
[22] Poignard, B., Asymptotic theory of the adaptive Sparse Group Lasso, Ann Inst Stat Math, 72, 297-328 (2018) · Zbl 1436.62342 · doi:10.1007/s10463-018-0692-7
[23] Scheetz, TE; Kim, K-YA; Swiderski, RE; Philp, AR; Braun, TA; Knudtson, KL; Dorrance, AM; DiBona, GF; Huang, J.; Casavant, TL; Sheffield, VC; Stone, EM, Regulation of gene expression in the mammalian eye and its relevance to eye disease, Proc Natl Acad Sci, 103, 39, 14429-14434 (2006) · doi:10.1073/pnas.0602562103
[24] Simon, N.; Friedman, J.; Hastie, T.; Tibshirani, R., A sparse-group lasso, J Comput Graph Stat, 22, 2, 231-245 (2013) · doi:10.1080/10618600.2012.681250
[25] Subramanian, A.; Tamayo, P.; Mootha, VK; Mukherjee, S.; Ebert, BL; Gillette, MA; Paulovich, A.; Pomeroy, SL; Golub, TR; Lander, ES; Mesirov, JP, Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles, Proc Natl Acad Sci, 102, 43, 15545-15550 (2005) · doi:10.1073/pnas.0506580102
[26] Tibshirani, R., Regression shrinkage and selection via the lasso, J R Stat Soc Ser B (Methodol), 58, 1, 267-288 (1996) · Zbl 0850.62538
[27] Wang, L.; Wu, Y.; Li, R., Quantile regression for analyzing heterogeneity in ultra-high dimension, J Am Stat Assoc, 107, 497, 214-222 (2012) · Zbl 1328.62468 · doi:10.1080/01621459.2012.656014
[28] Wright, J.; Ma, Y.; Mairal, J.; Sapiro, G.; Huang, TS; Yan, S., Sparse representation for computer vision and pattern recognition, Proc IEEE, 98, 6, 1031-1044 (2010) · doi:10.1109/JPROC.2010.2044470
[29] Wu, Y.; Liu, Y., Variable selection in quantile regression, Stat Sin, 19, 2, 801-817 (2009) · Zbl 1166.62012
[30] Yahya Algamal, Z.; Hisyam Lee, M., A two-stage sparse logistic regression for optimal gene selection in high-dimensional microarray data classification, Adv Data Anal Classif, 13, 753-771 (2019) · Zbl 1474.62273 · doi:10.1007/s11634-018-0334-1
[31] Yuan, M.; Lin, Y., Model selection and estimation in regression with grouped variables, J R Stat Soc Ser B (Methodol), 68, 1, 49-67 (2006) · Zbl 1141.62030 · doi:10.1111/j.1467-9868.2005.00532.x
[32] Zhao, W.; Zhang, R.; Liu, J., Sparse group variable selection based on quantile hierarchical Lasso, J Appl Stat, 41, 8, 1658-1677 (2014) · Zbl 1514.62979 · doi:10.1080/02664763.2014.888541
[33] Zhou, N.; Zhu, J., Group variable selection via a hierarchical lasso and its oracle property, Stat Interface, 3, 557-574 (2010) · Zbl 1245.62183 · doi:10.4310/SII.2010.v3.n4.a13
[34] Zou, H., The adaptive lasso and its oracle properties, J Am Stat Assoc, 101, 476, 1418-1429 (2006) · Zbl 1171.62326 · doi:10.1198/016214506000000735
[35] Zou, H.; Hastie, T.; Tibshirani, R., Sparse principal component analysis, J Comput Graph Stat, 15, 2, 265-286 (2006) · doi:10.1198/106186006X113430
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.