×

An \(\ell_{1}\)-oracle inequality for the Lasso in finite mixture Gaussian regression models. (English) Zbl 1395.62166

Summary: We consider a finite mixture of Gaussian regression models for high-dimensional heterogeneous data where the number of covariates may be much larger than the sample size. We propose to estimate the unknown conditional mixture density by an \(\ell_1\)-penalized maximum likelihood estimator. We shall provide an \(\ell_1\)-oracle inequality satisfied by this Lasso estimator with the Kullback-Leibler loss. In particular, we give a condition on the regularization parameter of the Lasso to obtain such an oracle inequality. Our aim is twofold: to extend the \(\ell_1\)-oracle inequality established by P. Massart and C. Meynet [Electron. J. Stat. 5, 669–687 (2011; Zbl 1274.62468)] in the homogeneous Gaussian linear regression case, and to present a complementary result to N. Städler et al. [Test 19, No. 2, 209–256 (2010; Zbl 1203.62128)], by studying the Lasso for its \(\ell_1\)-regularization properties rather than considering it as a variable selection procedure. Our oracle inequality shall be deduced from a finite mixture Gaussian regression model selection theorem for \(\ell_1\)-penalized maximum likelihood conditional density estimation, which is inspired from V. Vapnik’s method of structural risk minimization [Estimation of dependences based on empirical data. New York, NY: Springer (1982; Zbl 0499.62005)] and from the theory on model selection for maximum likelihood estimators developed by P. Massart [Concentration inequalities and model selection. Ecole d’Eté de Probabilités de Saint-Flour XXXIII – 2003. Berlin: Springer (2007; Zbl 1170.60006)].

MSC:

62H30 Classification and discrimination; cluster analysis (statistical aspects)
62G08 Nonparametric regression and quantile regression
62J07 Ridge regression; shrinkage estimators (Lasso)
PDFBibTeX XMLCite
Full Text: DOI