×

Model-based classification via mixtures of multivariate \(t\)-distributions. (English) Zbl 1247.62151

Summary: A novel model-based classification technique is introduced based on mixtures of multivariate t-distributions. A family of four mixture models is defined by constraining, or not, the covariance matrices and the degrees of freedom to be equal across mixture components. Parameters for each of the resulting four models are estimated using a multicycle expectation-conditional maximization algorithm, where convergence is determined using a criterion based on the Aitken acceleration. A straightforward, but very effective, technique for the initialization of the unknown component memberships is introduced and compared with a popular, more sophisticated, initialization procedure. This novel four-member family is applied to real and simulated data, where it gives good classification performance, even when compared with more established techniques.

MSC:

62H30 Classification and discrimination; cluster analysis (statistical aspects)
62H10 Multivariate distribution of statistics
65C60 Computational problems in statistics (MSC2010)

Software:

R; GGobi; PGMM; mclust; BRENT
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Aitken, A. C., On Bernoulli’s numerical solution of algebraic equations, Proceedings of the Royal Society of Edinburgh, 46, 289-305 (1926) · JFM 52.0098.05
[2] Andrews, J.L., McNicholas, P.D., 2010. Extending mixtures of multivariate \(t\) doi:10.1007/s11222-010-9175-2; Andrews, J.L., McNicholas, P.D., 2010. Extending mixtures of multivariate \(t\) doi:10.1007/s11222-010-9175-2 · Zbl 1255.62171
[3] Biernacki, C.; Celeux, G.; Govaert, G., Assessing a mixture model for clustering with the integrated completed likelihood, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 7, 719-725 (2000)
[4] Binder, D. A., Bayesian cluster analysis, Biometrika, 65, 31-38 (1978) · Zbl 0376.62007
[5] Böhning, D.; Dietz, E.; Schaub, R.; Schlattmann, P.; Lindsay, B., The distribution of the likelihood ratio for mixtures of densities from the one-parameter exponential family, Annals of the Institute of Statistical Mathematics, 46, 373-388 (1994) · Zbl 0802.62017
[6] Bouveyron, C.; Girard, S.; Schmid, C., High-dimensional data clustering, Computational Statistics and Data Analysis, 52, 1, 502-519 (2007) · Zbl 1452.62433
[7] Brent, R., Algorithms for Minimization without Derivatives (1973), Prentice-Hall: Prentice-Hall New Jersey · Zbl 0245.65032
[8] Celeux, G.; Govaert, G., Gaussian parsimonious clustering models, Pattern Recognition, 28, 781-793 (1995)
[9] Cook, D.; Swayne, D. F., Interactive and Dynamic Graphics for Data Analysis: With R and GGobi (Use R) (2007), Springer · Zbl 1154.62006
[10] Dasgupta, A.; Raftery, A. E., Detecting features in spatial point processes with clutter via model-based clustering, Journal of the American Statistical Association, 93, 294-302 (1998) · Zbl 0906.62105
[11] Dean, N.; Murphy, T. B.; Downey, G., Using unlabelled data to update classification rules with applications in food authenticity studies, Journal of the Royal Statistical Society. Series C, 55, 1, 1-14 (2006) · Zbl 1490.62155
[12] Dempster, A. P.; Laird, N. M.; Rubin, D. B., Maximum likelihood from incomplete data via the EM algorithm, Journal of the Royal Statistical Society. Series B, 39, 1, 1-38 (1977) · Zbl 0364.62022
[13] Edwards, A. W.F.; Cavalli-Sforza, L. L., A method for cluster analysis, Biometrics, 21, 362-375 (1965)
[14] Forina, M.; Armanino, C.; Castino, M.; Ubigli, M., Multivariate data analysis as a discriminating method of the origin of wines, Vitis, 25, 189-201 (1986)
[15] Forina, M.; Armanino, C.; Lanteri, S.; Tiscornia, E., Classification of olive oils from their fatty acid composition, (Martens, H.; Russwurm, H., Food Research and Data Analysis (1983), Applied Science Publishers: Applied Science Publishers London), 189-214
[16] Forina, M.; Tiscornia, E., Pattern recognition methods in the prediction of Italian olive oil origin by their fatty acid content, Annali di Chimica, 72, 143-155 (1982)
[17] Fraley, C.; Raftery, A. E., MCLUST: software for model-based cluster analysis, Journal of Classification, 16, 297-306 (1999) · Zbl 0951.91500
[18] Fraley, C.; Raftery, A. E., Model-based clustering, discriminant analysis, and density estimation, Journal of the American Statistical Association, 97, 458, 611-631 (2002) · Zbl 1073.62545
[19] Fraley, C.; Raftery, A. E., Enhanced software for model-based clustering, density estimation, and discriminant analysis: MCLUST, Journal of Classification, 20, 263-286 (2003) · Zbl 1055.62071
[20] Fraley, C., Raftery, A.E., 2006. MCLUST version 3 for R: normal mixture modeling and model-based clustering. Technical Report 504. Department of Statistics, University of Washington. Minor revisions January 2007 and November 2007.; Fraley, C., Raftery, A.E., 2006. MCLUST version 3 for R: normal mixture modeling and model-based clustering. Technical Report 504. Department of Statistics, University of Washington. Minor revisions January 2007 and November 2007.
[21] Frühwirth-Schnatter, S., Finite Mixture and Markov Switching Models (2006), Springer-Verlag: Springer-Verlag New York, New York, LLC · Zbl 1108.62002
[22] Hubert, L.; Arabie, P., Comparing partitions, Journal of Classification, 2, 193-218 (1985)
[23] Hurley, C., Clustering visualizations of multivariate data, Journal of Computational and Graphical Statistics, 13, 4, 788-806 (2004)
[24] Keribin, C., Consistent estimation of the order of mixture models, Sankhyā. The Indian Journal of Statistics. Series A, 62, 1, 49-66 (2000) · Zbl 1081.62516
[25] Leroux, B. G., Consistent estimation of a mixing distribution, The Annals of Statistics, 20, 1350-1360 (1992) · Zbl 0763.62015
[26] McLachlan, G. J., (The Classification and Mixture Maximum Likelihood Approaches to Cluster Analysis. The Classification and Mixture Maximum Likelihood Approaches to Cluster Analysis, Handbook of Statistics, vol. 2 (1982), North-Holland: North-Holland Amsterdam), 199-208 · Zbl 0513.62064
[27] McLachlan, G. J.; Basford, K. E., Mixture Models: Inference and Applications to Clustering (1988), Marcel Dekker Inc.: Marcel Dekker Inc. New York · Zbl 0697.62050
[28] McLachlan, G. J.; Bean, R. W.; Jones, L. B.-T., Extension of the mixture of factor analyzers model to incorporate the multivariate \(t\)-distribution, Computational Statistics and Data Analysis, 51, 11, 5327-5338 (2007) · Zbl 1445.62053
[29] McLachlan, G. J.; Krishnan, T., The EM Algorithm and Extensions (1997), Wiley: Wiley New York · Zbl 0882.62012
[30] McLachlan, G. J.; Peel, D., Robust cluster analysis via mixtures of multivariate \(t\)-distributions, (Lecture Notes in Computer Science, vol. 1451 (1998), Springer-Verlag: Springer-Verlag Berlin), 658-666
[31] McLachlan, G. J.; Peel, D., Finite Mixture Models (2000), John Wiley & Sons: John Wiley & Sons New York · Zbl 0963.62061
[32] McLachlan, G. J.; Peel, D.; Bean, R. W., Modelling high-dimensional data by mixtures of factor analyzers, Computational Statistics and Data Analysis, 41, 3-4, 379-388 (2003) · Zbl 1256.62036
[33] McNicholas, P. D., Model-based classification using latent Gaussian mixture models, Journal of Statistical Planning and Inference, 140, 5, 1175-1181 (2010) · Zbl 1181.62095
[34] McNicholas, P. D.; Murphy, T. B., Parsimonious Gaussian mixture models, Statistics and Computing, 18, 285-296 (2008)
[35] McNicholas, P. D.; Murphy, T. B., Model-based clustering of longitudinal data, The Canadian Journal of Statistics, 38, 1, 153-168 (2010) · Zbl 1190.62120
[36] McNicholas, P. D.; Murphy, T. B.; McDaid, A. F.; Frost, D., Serial and parallel implementations of model-based clustering via parsimonious Gaussian mixture models, Computational Statistics and Data Analysis, 54, 3, 711-723 (2010) · Zbl 1464.62131
[37] Meng, X. L.; Rubin, D. B., Maximum likelihood estimation via the ECM algorithm: a general framework, Biometrika, 80, 267-278 (1993) · Zbl 0778.62022
[38] R Development Core Team, 2009. R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria.; R Development Core Team, 2009. R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria.
[39] Raftery, A. E.; Dean, N., Variable selection for model-based clustering, Journal of the American Statistical Association, 101, 473, 168-178 (2006) · Zbl 1118.62339
[40] Rand, W. M., Objective criteria for the evaluation of clustering methods, Journal of the American Statistical Association, 66, 846-850 (1971)
[41] Schwarz, G., Estimating the dimension of a model, Annals of Statistics, 6, 31-38 (1978)
[42] Scrucca, L., 2009. Dimension reduction for model-based clustering. Statistics and Computing (in press).; Scrucca, L., 2009. Dimension reduction for model-based clustering. Statistics and Computing (in press).
[43] Swayne, D., Cook, D., Buja, A., Lang, D., Wickham, H., Lawrence, M., 2006. GGobi manual. Sourced from: www.ggobi.org/docs/manual.pdf; Swayne, D., Cook, D., Buja, A., Lang, D., Wickham, H., Lawrence, M., 2006. GGobi manual. Sourced from: www.ggobi.org/docs/manual.pdf
[44] Wolfe, J. H., Pattern clustering by multivariate mixture analysis, Multivariate Behavioral Research, 5, 329-350 (1970)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.