×

A comparison of the \(L_2\) minimum distance estimator and the EM-algorithm when fitting \(k\)-component univariate normal mixtures. (English) Zbl 1416.62148

Summary: The method of maximum likelihood using the EM-algorithm for fitting finite mixtures of normal distributions is the accepted method of estimation ever since it has been shown to be superior to the method of moments. Recent books testify to this. There has however been criticism of the method of maximum likelihood for this problem, the main criticism being when the variances of component distributions are unequal the likelihood is in fact unbounded and there can be multiple local maxima. Another major criticism is that the maximum likelihood estimator is not robust. Several alternative minimum distance estimators have since been proposed as a way of dealing with the first problem. This paper deals with one of these estimators which is not only superior due to its robustness, but in fact can have an advantage in numerical studies even at the model distribution. Importantly, robust alternatives of the EM-algorithm, ostensibly fitting \(t\) distributions when in fact the data are mixtures of normals, are also not competitive at the normal mixture model when compared to the chosen minimum distance estimator. It is argued for instance that natural processes should lead to mixtures whose component distributions are normal as a result of the Central Limit Theorem. On the other hand data can be contaminated because of extraneous sources as are typically assumed in robustness studies. This calls for a robust estimator.

MSC:

62F10 Point estimation
62H30 Classification and discrimination; cluster analysis (statistical aspects)
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Amemiya T (1985) Advanced econometrics. Harvard University Press, Cambridge
[2] Benaglia T, Chauveau D, Hunter DR, Young D (2009) Mixtools: an R package for analysing finite mixture models. J Stat Soft 32(6):1-29 · doi:10.18637/jss.v032.i06
[3] Basford KE, Greenway DR, McLachlan GJ, Peel D (1997) Standard errors of fitted means under normal mixture. Comput Stat 12:1-17 · Zbl 0924.62055
[4] Biernacki C, Chretien S (2003) Degeneracy in the maximum likelihood estimation of univariate gaussian mixtures with em. Stat Prob Lett 61:373-382 · Zbl 1038.62023 · doi:10.1016/S0167-7152(02)00396-6
[5] Biernacki C, Celeux G, Govaert G (2003) Strategies for getting the highest likelihoods in mixture models. Guest Editors: Böhning and Seidel. Comput Stat Data Anal 41:561-575 · Zbl 1429.62235 · doi:10.1016/S0167-9473(02)00163-9
[6] Choi K, Bulgren WG (1968) An estimation procedure for mixtures of distributions. J R Stat Soc B 30:444-460 · Zbl 0187.15804
[7] Clarke BR (1989) An unbiased minimum distance estimator of the proportion parameter in a mixture of two normal distributions. Stat Prob Lett 7(4):275-281 · Zbl 0667.62015 · doi:10.1016/0167-7152(89)90105-3
[8] Clarke BR (2000) A review of differentiability in relation to robustness with an application to seismic data analysis. PINSA 66A:467-482
[9] Clarke BR, Heathcote CR (1978) Comment on “Estimating mixtures of normal distributions and switching regressions” by Quandt, R.E. and Ramsey, J.B. J Am Stat Assoc 73:749-750 · doi:10.1080/01621459.1978.10480092
[10] Clarke BR, Heathcote CR (1994) Robust estimation of \[k\] k-component univariate normal mixtures. Ann Inst Stat Math 46:83-93 · Zbl 0802.62039 · doi:10.1007/BF00773595
[11] Clarke BR, Futshik A (2007) On the convergence of Newton’s method when estimating higher dimensional parameters. J Multivar Anal 98:916-931 · Zbl 1118.65004 · doi:10.1016/j.jmva.2006.12.002
[12] Cutler A, Cordiero-Braña OI (1996) Minimum Hellinger distance estimation for finite mixture models. J Am Stat Assoc 91:1716-1723 · Zbl 0881.62035 · doi:10.1080/01621459.1996.10476743
[13] Dempster AP, Laird NM, Rubin DP (1977) Maximum likelihood from incomplete data via the EM algorithm (with discussion). J R Stat Soc B 39:1-38 · Zbl 0364.62022
[14] Depraetere N, Vandebroek M (2014) Order selection in finite mixtures of linear regressions. Stat Pap 55:871-911 · Zbl 1334.62138 · doi:10.1007/s00362-013-0534-x
[15] Fisher RA (1947) The analysis of covariance method for the relation between a part and the whole. Biometrics 3:65-68 · doi:10.2307/3001641
[16] Fryer JG, Robertson CA (1972) A comparison of some methods for estimating mixed normal distributions. Biometrika 59:639-648 · Zbl 0255.62033 · doi:10.1093/biomet/59.3.639
[17] Hasselman B (2013) nleqslv: solve systems of non linear equations. R package version 2.0
[18] Huber PJ, Ronchetti EM (2009) Robust statistics, 2nd edn. Wiley, Hoboken · Zbl 1276.62022 · doi:10.1002/9780470434697
[19] Klar B, Meintanis SG (2005) Tests for normal mixtures based on the empirical characteristic function. Comput Stat Data Anal 49:227-242 · Zbl 1429.62156 · doi:10.1016/j.csda.2004.05.011
[20] Lee SX, McLachlan GJ (2013) On mixtures of skew normal and skew t-distributions. Adv Data Anal Classif 7(3):241-266 · Zbl 1273.62115 · doi:10.1007/s11634-013-0132-8
[21] Lee SX, McLachlan GJ (2014) EMMIXuskew: an R package for fitting mixtures of multivariate skew t-distributions via the EM algorithm. J Stat Soft 55(12):1-22
[22] Macdonald PDM (1971) Comment on a paper by Choi, K. and Bulgren, W.G. J R Stat Soc B 33:326-329
[23] McLachlan GJ, Krishnan T (2008) The EM algorithm and extensions, 2nd edn. Wiley, New York · Zbl 1165.62019 · doi:10.1002/9780470191613
[24] McLachlan GJ, Peel D (2000) Finite mixture models. Wiley, New York · Zbl 0963.62061 · doi:10.1002/0471721182
[25] McLachlan GJ, Peel D, Basford KE, Adams P (1999) The EMMIX software for the fitting of mixtures of normal and t-components. J Stat Softw 4(2):1-14 · doi:10.18637/jss.v004.i02
[26] Pearson K (1894) Contributions to the mathematical theory of evolution. Philos Trans R Soc Lond A 185(1887-1895):71-110 · JFM 25.0347.02 · doi:10.1098/rsta.1894.0003
[27] Peel D, McLachlan GJ (2000) Robust mixture modelling using the t distribution. Stat Comput 10:339-348 · doi:10.1023/A:1008981510081
[28] Quandt RE, Ramsey JB (1978) Estimating mixtures of normal distributions and switching regressions. J Am Stat Assoc 73:730-738 · Zbl 0401.62024 · doi:10.1080/01621459.1978.10480085
[29] R Development Core Team (2014) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria, ISBN 3-900051-07-0. http://www.r-project.org/ · Zbl 1429.62235
[30] Redner RA, Walker HF (1984) Mixture densities, maximum likelihood and the EM algorithm. SIAM Rev 26:195-239 · Zbl 0536.62021 · doi:10.1137/1026034
[31] Seidel W, Mosler K, Alker M (2000) A cautionary note on likelihood ratio tests in mixture models. Ann Inst Stat Math 52:481-487 · Zbl 0960.62025 · doi:10.1023/A:1004117419204
[32] Seidel W, Ševčíková H (2004) Types of likelihood maxima in mixture models and their implication on the performance of tests. Ann Inst Stat Math 56:631-654 · Zbl 1078.62014 · doi:10.1007/BF02506480
[33] Tan WY, Chang WC (1972) Some comparisons of the method of moments and the method of maximum likelihood in estimating parameters of a mixture of two normal densities. J Am Stat Assoc 67:702-708 · Zbl 0245.62039 · doi:10.1080/01621459.1972.10481282
[34] Titterington DM, Smith AFM, Makov UE (1985) Statistical analysis of finite mixture distributions. Wiley, New York · Zbl 0646.62013
[35] Wang K, McLachlan GJ, Ng A, Peel D (2009) EMMIX-skew EM algorithm for mixture of multivariate skew normal/t distributions. EMMIX was originally written in Fortran by David Peel, R package version 1.0.20. http://www.maths.uq.edu.au/gjm/mix_soft/EMMIX-skew
[36] Woodward WA, Parr WC, Schucany WR, Lindsey H (1984) A comparison of minimum distance and maximum likelihood estimation of a mixture proportion. J Am Stat Assoc 79:590-598 · Zbl 0547.62017 · doi:10.1080/01621459.1984.10478085
[37] Wu CFJ (1983) On the convergence properties of the EM algorithm. Ann Stat 11:95-103 · Zbl 0517.62035 · doi:10.1214/aos/1176346060
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.