Model-based regression clustering for high-dimensional data: application to functional data. (English) Zbl 1414.62238

Summary: Finite mixture regression models are useful for modeling the relationship between response and predictors arising from different subpopulations. In this article, we study high-dimensional predictors and high-dimensional response and propose two procedures to cluster observations according to the link between predictors and the response. To reduce the dimension, we propose to use the Lasso estimator, which takes into account the sparsity and a maximum likelihood estimator penalized by the rank, to take into account the matrix structure. To choose the number of components and the sparsity level, we construct a collection of models, varying those two parameters and we select a model among this collection with a non-asymptotic criterion. We extend these procedures to functional data, where predictors and responses are functions. For this purpose, we use a wavelet-based approach. For each situation, we provide algorithms and apply and evaluate our methods both on simulated and real datasets, to understand how they work in practice.


62H30 Classification and discrimination; cluster analysis (statistical aspects)
Full Text: DOI arXiv


[1] Anderson, TW, Estimating linear restrictions on regression coefficients for multivariate normal distributions, Ann Math Stat, 22, 327-351, (1951) · Zbl 0043.13902
[2] Baudry, J-P; Maugis, C.; Michel, B., Slope heuristics: overview and implementation, Stat Comput, 22, 455-470, (2012) · Zbl 1322.62007
[3] Birgé L, Massart P (2007) Minimal penalties for Gaussian model selection. Probab Theory Relat Fields 138(1-2):33-73 · Zbl 1112.62082
[4] Bühlmann P, van de Geer S (2011) Statistics for high-dimensional data: methods, theory and applications, Springer Series in Statistics. Springer, Berlin
[5] Bunea, F.; She, Y.; Wegkamp, M., Joint variable and rank selection for parsimonious estimation of high-dimensional matrices, Ann Stat, 40, 2359-2388, (2012) · Zbl 1373.62246
[6] Celeux, G.; Govaert, G., Gaussian parsimonious clustering models, Pattern Recognit, 28, 781-793, (1995)
[7] Ciarleglio, A.; Ogden, T., Wavelet-based scalar-on-function finite mixture regression models, Comput Stat Data Anal, 93, 86-96, (2014) · Zbl 1468.62042
[8] Dempster, AP; Laird, NM; Rubin, DB, Maximum likelihood from incomplete data via the EM algorithm. Discussion, J R Stat Soc Ser B, 39, 1-38, (1977) · Zbl 0364.62022
[9] Devijver, E., Finite mixture regression: a sparse variable selection by model selection for clustering, Electron J Stat, 9, 2642-2674, (2015) · Zbl 1329.62279
[10] Devijver E (2015) Joint rank and variable selection for parsimonious estimation in high-dimension finite mixture regression model. arXiv:1501.00442 · Zbl 1362.62127
[11] Ferraty F, Vieu P (2006) Nonparametric functional data analysis: theory and practice, Springer series in statistics. Springer, New York · Zbl 1119.62046
[12] Gareth, J.; Sugar, C., Clustering for sparsely sampled functional data, J Am Stat Assoc, 98, 397-408, (2003) · Zbl 1041.62052
[13] Giraud, C., Low rank multivariate regression, Electron J Stat, 5, 775-799, (2011) · Zbl 1274.62434
[14] Hubert, L.; Arabie, P., Comparing partitions, J Classif, 2, 193-218, (1985) · Zbl 0587.62128
[15] Izenman, A., Reduced-rank regression for the multivariate linear model, J Multivar Anal, 5, 248-264, (1975) · Zbl 0313.62042
[16] Jones, PN; McLachlan, GJ, Fitting finite mixture models in a regression context, Aust J Stat, 34, 233-240, (1992)
[17] Mallat, S., A theory for multiresolution signal decomposition: the wavelet representation, IEEE Trans Pattern Anal Mach Intell, 11, 674-693, (1989) · Zbl 0709.94650
[18] Mallat S (1999) A wavelet tour of signal processing. Academic Press, Dublin · Zbl 0998.94510
[19] McLachlan G, Peel D (2004) Finite Mixture Models, Wiley series in probability and statistics. Wiley, New York
[20] Meinshausen, N.; Bühlmann, P., Stability selection, J R Stat Soc Ser B Stat Methodol, 72, 417-473, (2010)
[21] Meynet C, Maugis-Rabusseau C (2012) A sparse variable selection procedure in model-based clustering. Research report, September
[22] Misiti M, Misiti Y, Oppenheim G, Poggi J-M (2004) Matlab Wavelet Toolbox User’s Guide. Version 3. The Mathworks Inc, Natick, MA · Zbl 1126.94301
[23] Misiti M, Misiti Y, Oppenheim G, Poggi J-M (2007) Clustering signals using wavelets. In: Sandoval Francisco, Prieto Alberto, Cabestany Joan, Graña Manuel (eds) Computational and Ambient Intelligence, vol 4507, Lecture Notes in Computer Science. Springer, Berlin Heidelberg, pp 514-521 · Zbl 1167.42300
[24] Park, T.; Casella, G., The Bayesian lasso, J Am Stat Assoc, 103, 681-686, (2008) · Zbl 1330.62292
[25] Ramsay JO, Silverman BW (2005) Functional data analysis, Springer series in statistics. Springer, New York
[26] Simon N, Friedman J, Hastie T, Tibshirani R (2013) A sparse-group lasso. J Comput Graph Stat 22:231-245
[27] Städler N, Bühlmann P, Van de Geer S (2010) \(ℓ _{1}\)-penalization for mixture regression models. Test 19(2):209-256
[28] Tibshirani, R., Regression shrinkage and selection via the lasso, J R Stat Soc Ser B, 58, 267-288, (1996) · Zbl 0850.62538
[29] Tseng, P., Convergence of a block coordinate descent method for nondifferentiable minimization, J Optim Theory Appl, 109, 475-494, (2001) · Zbl 1006.65062
[30] Wu, J., On the convergence properties of the EM algorithm, Ann Stat, 1, 95-103, (1983) · Zbl 0517.62035
[31] Yao, F.; Fu, Y.; Lee, T., Functional mixture regression, Biostatistics, 2, 341-353, (2011)
[32] Zhao, Y.; Ogden, T.; Reiss, P., Wavelet-based LASSO in functional linear regression, J Comput Graph Stat, 21, 600-617, (2012)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.