zbMATH — the first resource for mathematics

The accessible Lasso models. (English) Zbl 1440.62286
Summary: A new line of research [J. D. Lee et al., Ann. Stat. 44, No. 3, 907–927 (2016; Zbl 1341.62061)] on the Lasso [R. Tibshirani, J. R. Stat. Soc., Ser. B 58, No. 1, 267–288 (1996; Zbl 0850.62538)] exploits the beautiful geometric fact that the Lasso fit is the residual from projecting the response vector \(y\) onto a certain convex polytope [R. J. Tibshirani and J. Taylor, Ann. Stat. 40, No. 2, 1198–1232 (2012; Zbl 1274.62469)]. This geometric picture also allows an exact geometric description of the set of accessible Lasso models for a given design matrix, that is, which configurations of the signs of the coefficients it is possible to realize with some choice of \(y\). In particular, the accessible Lasso models are those that correspond to a face of the convex hull of all the feature vectors together with their negations. This convex hull representation then permits the enumeration and bounding of the number of accessible Lasso models, which in turn provides a direct proof of model selection inconsistency when the size of the true model is greater than half the number of observations.

62J07 Ridge regression; shrinkage estimators (Lasso)
52B12 Special polytopes (linear programming, centrally symmetric, etc.)
Full Text: DOI arXiv