Constrained monotone EM algorithms for finite mixture of multivariate Gaussians. (English) Zbl 1445.62116

Summary: The likelihood function for normal multivariate mixtures may present both local spurious maxima and also singularities and the latter may cause the failure of the optimization algorithms. Theoretical results assure that imposing some constraints on the eigenvalues of the covariance matrices of the multivariate normal components leads to a constrained parameter space with no singularities and at least a smaller number of local maxima of the likelihood function. Conditions assuring that an EM algorithm implementing such constraints maintains the monotonicity property of the usual EM algorithm are provided. Different approaches are presented and their performances are evaluated and compared using numerical experiments.


62H12 Estimation in multivariate analysis
62H30 Classification and discrimination; cluster analysis (statistical aspects)
62-08 Computational methods for problems pertaining to statistics
Full Text: DOI


[1] Axelsson, O., Iterative solution methods, (1996), Cambridge University Press Cambridge
[2] Banfield, J.D.; Raftery, A.E., Model-based gaussian and non-Gaussian clustering, Biometrics, 49, 803-821, (1993) · Zbl 0794.62034
[3] Biernacki, C., 2004a. Degeneracy in the maximum likelihood estimation of univariate Gaussian mixtures for grouped data and behaviour of the EM algorithm. Technical Report, Université de Franche-Comté. · Zbl 1150.62010
[4] Biernacki, C., 2004b. An asymptotic upper bound of the likelihood to prevent Gaussian mixtures from degenerating. Technical Report, Université de Franche-Comté.
[5] Biernacki, C.; Chrétien, S., Degeneracy in the maximum likelihood estimation of univariate Gaussian mixtures with EM, Statist. probab. lett., 61, 373-382, (2003) · Zbl 1038.62023
[6] Burden, R.L.; Faires, J.D., Numerical analysis, (1985), Prindle, Weber & Schmidt Boston
[7] Celeux, G.; Govaert, G., Gaussian parsimonious clustering models, Pattern recognition, 28, 781-793, (1995)
[8] Ciuperca, G.; Ridolfi, A.; Idier, J., Penalized maximum likelihood estimator for normal mixtures, Scand. J. statist., 30, 45-59, (2003) · Zbl 1034.62018
[9] Frayley, C.; Raftery, A.E., Model-based clustering, discriminant analysis and density estimation, J. amer. statist. assoc., 97, 611-631, (2002) · Zbl 1073.62545
[10] Hathaway, R.J., A constrained formulation of maximum-likelihood estimation for normal mixture distributions, Ann. statist., 13, 795-800, (1985) · Zbl 0576.62039
[11] Horn, R.A.; Johnson, C.R., Matrix analysis, (1999), Cambridge University Press New York
[12] Ingrassia, S., A likelihood-based constrained algorithm for multivariate normal mixture models, Statist. methods appl., 13, 151-166, (2004) · Zbl 1205.62066
[13] Ingrassia, S.; Rocci, R., Monotone constrained EM algorithms for multinormal mixture models, (), 111-118
[14] McLachlan, G.J.; Krishnan, T., The EM algorithm and extensions, (1997), Wiley New York · Zbl 0882.62012
[15] McLachlan, G.J.; Peel, D., Finite mixture models, (2000), Wiley New York · Zbl 0963.62061
[16] Meng, X.L.; Rubin, D.B., Maximum likelihood estimation via the ECM algorithm: a general framework, Biometrika, 80, 267-278, (1993) · Zbl 0778.62022
[17] Snoussi, H.; Mohammad-Djafari, A., Penalized maximum likelihood for multivariate Gaussian mixture, (), 36-46
[18] Theobald, C.M., An inequality with applications to multivariate analysis, Biometrika, 62, 461-466, (1975) · Zbl 0316.62020
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.