×

On model selection consistency of Lasso. (English) Zbl 1222.62008

Summary: Sparsity or parsimony of statistical models is crucial for their proper interpretations, as in sciences and social sciences. Model selection is a commonly used method to find such models, but usually involves a computationally heavy combinatorial search. Lasso is now being used as a computationally feasible alternative to model selection. Therefore it is important to study Lasso for model selection purposes. We prove that a single condition, which we call the irrepresentable condition, is almost necessary and sufficient for Lasso to select the true model both in the classical fixed \(p\) setting and in the large \(p\) setting as the sample size \(n\) gets large. Based on these results, sufficient conditions that are verifiable in practice are given to relate to previous works and help applications of Lasso for feature selection and sparse representation. This irrepresentable condition, which depends mainly on the covariance of the predictor variables, states that Lasso selects the true model consistently if and (almost) only if the predictors that are not in the true model are “irrepresentable” (in a sense to be clarified) by predictors that are in the true model. Furthermore, simulations are carried out to provide insights and understanding of this result.

MSC:

62A01 Foundations and philosophical topics in statistics
65C60 Computational problems in statistics (MSC2010)
62H99 Multivariate analysis
PDFBibTeX XMLCite
Full Text: Link