Mixture model modal clustering. (English) Zbl 1474.62218

Summary: The two most extended density-based approaches to clustering are surely mixture model clustering and modal clustering. In the mixture model approach, the density is represented as a mixture and clusters are associated to the different mixture components. In modal clustering, clusters are understood as regions of high density separated from each other by zones of lower density, so that they are closely related to certain regions around the density modes. If the true density is indeed in the assumed class of mixture densities, then mixture model clustering allows to scrutinize more subtle situations than modal clustering. However, when mixture modeling is used in a nonparametric way, taking advantage of the denseness of the sieve of mixture densities to approximate any density, then the correspondence between clusters and mixture components may become questionable. In this paper we introduce two methods to adopt a modal clustering point of view after a mixture model fit. Examples are provided to illustrate that mixture modeling can also be used for clustering in a nonparametric sense, as long as clusters are understood as the domains of attraction of the density modes. Finally, a simulation study reveals that the new methods are extremely efficient from a computational point of view, while at the same time they retain a high level of accuracy.


62H30 Classification and discrimination; cluster analysis (statistical aspects)
68T10 Pattern recognition, speech recognition
91C20 Clustering in the social and behavioral sciences
Full Text: DOI arXiv


[1] Aghaeepour, N., Finak, G., The FlowCAP Consortium, The DREAM Consortium, Hoos, H., Mosmann, T. R., Brinkman, R., Gottardo, R. and Scheuermann, R. H., Critical assessment of automated flow cytometry analysis techniques, Nat Methods, 10, 228-238, (2013)
[2] Aliyari Ghassabeh, Y., A sufficient condition for the convergence of the mean shift algorithm with Gaussian kernel, J Multivar Anal, 135, 1-10, (2015) · Zbl 1308.62118
[3] Arias-Castro, E.; Mason, D.; Pelletier, B., On the estimation of the gradient lines of a density and the consistency of the mean-shift algorithm, J Mach Learn Res, 17, 1-28, (2016) · Zbl 1360.62150
[4] Azzalini, A.; Bowman, AW, A look at some data on the Old Faithful geyser, Appl Stat, 39, 357-365, (1990) · Zbl 0707.62186
[5] Azzalini, A.; Torelli, N., Clustering via nonparametric density estimation, Stat Comput, 17, 71-80, (2007)
[6] Baudry J-P (2010) Sélection de Modèle pour la Classifcation Non Supervisée. Choix du Nombre de Classes. Ph.D. Thesis, Université Paris-Sud 11
[7] Baudry, J-P; Raftery, AE; Celeux, G.; Lo, K.; Gottardo, R., Combining mixture components for clustering, J Comput Graph Stat, 19, 332-353, (2010)
[8] Bock H-H (1974) Automatische Klassifikation (Clusteranalyse). Vandenhoeck & Ruprecht, Göttingen
[9] Brinkman, RR; Gasparetto, M.; Lee, S-JJ; Ribickas, AJ; Perkins, J.; Janssen, W.; Smiley, R.; Smith, C., High-content flow cytometry and temporal data analysis for defining a cellular signature of Graft-versus-Host Disease, Biol Blood Marrow Transpl, 13, 691-700, (2007)
[10] Carlsson, G.; Mémoli, F., Classifying clustering schemes, Found Comput Math, 13, 221-252, (2013) · Zbl 1358.62057
[11] Carreira-Perpiñán, MÁ, Mode-finding for mixtures of Gaussian distributions, IEEE Trans Pattern Anal Mach Intell, 22, 1318-1323, (2000)
[12] Carreira-Perpiñán MÁ (2006) Acceleration strategies for Gaussian mean-shift image segmentation. In: IEEE conference on computer vision and pattern recognition (CVPR 2006), pp 1160-1167
[13] Carreira-Perpiñán, MÁ, Gaussian mean shift is an EM algorithm, IEEE Trans Pattern Anal Mach Intell, 29, 767-776, (2007)
[14] Carreira-Perpiñán MÁ, Williams CKI (2003a) On the number of modes of a Gaussian mixture. In: Scale-space methods in computer vision. Lecture notes in computer science, vol 2695, pp 625-640. Springer, Berlin · Zbl 1067.68724
[15] Carreira-Perpiñán MÁ, Williams CKI (2003b) An isotropic Gaussian mixture can have more modes than components. Technical report EDI-INF-RR-0185, School of Informatics, University of Edinburgh, UK
[16] Chacón JE (2012) Identifying nonstandard group shapes in mixture model clustering through the mean shift algorithm. In: Programme and abstracts of the 5th international conference of the ERCIM working group on computing and statistics, p 122
[17] Chacón, JE, A population background for nonparametric density-based clustering, Stat Sci, 30, 518-532, (2015) · Zbl 1426.62181
[18] Chacón, JE; Duong, T., Bandwidth selection for multivariate density derivative estimation, with applications to clustering and bump hunting, Electron J Stat, 7, 499-532, (2013) · Zbl 1337.62067
[19] Chacón JE, Monfort P (2014) A comparison of bandwidth selectors for mean shift clustering. In: Skiadas CH (ed) Theoretical and applied issues in statistics and demography, pp 47-59. International Society for the Advancement of Science and Technology (ISAST), Athens
[20] Comaniciu, D., An algorithm for data-driven bandwidth selection, IEEE Trans Pattern Anal Mach Intell, 25, 281-288, (2003)
[21] Comaniciu, D.; Meer, P., Mean shift: a robust approach toward feature space analysis, IEEE Trans Pattern Anal Mach Intell, 24, 603-619, (2002)
[22] Cuevas, A.; Febrero, M.; Fraiman, R., Cluster analysis: a further approach based on density estimation, Comput Stat Data Anal, 36, 441-459, (2001) · Zbl 1053.62537
[23] Dennis JE, Schnabel RB (1996) Numerical methods for unconstrained optimization and nonlinear equations. SIAM, Philadelphia · Zbl 0847.65038
[24] Denœud, L., Transfer distance between partitions, Adv Data Anal Classif, 2, 279-294, (2008) · Zbl 1284.05319
[25] Duong, T.; Cowling, A.; Koch, I.; Wand, MP, Feature significance for multivariate kernel density estimation, Comput Stat Data Anal, 52, 4225-4242, (2008) · Zbl 1452.62265
[26] Edelsbrunner, H.; Fasy, BT; Rote, G., Add isotropic Gaussian kernels at own risk: more and more resilient modes in higher dimensions, Discrete Comput Geom, 49, 797-822, (2013) · Zbl 1282.26019
[27] Edelsbrunner, H.; Harer, J., Persistent homology—a survey, Contemp Math, 453, 257-282, (2008) · Zbl 1145.55007
[28] Ester, M.; Aggarwal, CC (ed.); Reddy, CK (ed.), Density-based clustering, 111-126, (2014), Boca Raton
[29] Ester M, Kriegel H, Sander J, Xu X (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proceedings of the 2nd international conference on knoledge discovery and data mining, pp 226-231. AAAI Press, Portland
[30] Forina, M.; Armanino, C.; Castino, M.; Ubigli, M., Multivariate data analysis as a discriminating method of the origin of wines, Vitis, 25, 189-201, (1986)
[31] Fraley, C.; Raftery, AE, Model-based clustering, discriminant analysis, and density estimation, J Am Stat Assoc, 97, 611-631, (2002) · Zbl 1073.62545
[32] Fraley, C.; Raftery, AE; Scrucca, L., mclust: Gaussian mixture modelling for model-based clustering, classification, and density estimation, R package version, 5, 2, (2016)
[33] Fukunaga, K.; Hostetler, LD, The estimation of the gradient of a density function, with applications in pattern recognition, IEEE Trans Inf Theory, 21, 32-40, (1975) · Zbl 0297.62025
[34] Hartigan JA (1975) Clustering algorithms. Wiley, New York · Zbl 0372.62040
[35] Hennig, C., Methods for merging Gaussian mixture components, Adv Data Anal Classif, 4, 3-34, (2010) · Zbl 1306.62141
[36] Hennig C (2015) fpc: flexible procedures for clustering. R package version 2.1-10. https://CRAN.R-project.org/package=fpc
[37] Lee, SX; McLachlan, GJ, EMMIXuskew: an R package for fitting mixtures of multivariate skew \(t\) distributions via the EM algorithm, J Stat Softw, 55, 1-22, (2013)
[38] Li JQ, Barron AR (2000) Mixture density estimation. In Solla SA, Leen TK, Mueller K-R (eds) Adv Neural Inf Process Syst 12:279-285
[39] Li, X.; Hu, Z.; Wu, F., A note on the convergence of the mean shift, Pattern Recognit, 40, 1756-1762, (2007) · Zbl 1111.68111
[40] Lin, T-I, Maximum likelihood estimation for multivariate skew normal mixture models, J Multivar Anal, 100, 257-265, (2009) · Zbl 1152.62034
[41] Lin, T-I; Ho, HJ; Lee, C-R, Flexible mixture modelling using the multivariate skew-\(t\)-normal distribution, Stat Comput, 24, 531-546, (2014) · Zbl 1325.62113
[42] Lin, T-I; McLachlan, GJ; Lee, S-X, Extending mixtures of factor models using the restricted multivariate skew-normal distribution, J Multivar Anal, 143, 398-413, (2016) · Zbl 1328.62378
[43] Lo, K.; Brinkman, RR; Gottardo, R., Automated gating of flow cytometry data via robust model-based clustering, Cytom A, 73, 321-332, (2008)
[44] McLachlan GJ, Basford KE (1988) Mixture models: inference and applications to clustering. Marcel Dekker Inc, New York · Zbl 0697.62050
[45] Papadimitriou C, Steiglitz K (1982) Combinatorial optimization: algorithms and complexity. Prentice Hall, Englewood Cliffs · Zbl 0503.90060
[46] Priebe, CE, Adaptive mixtures, J Am Stat Assoc, 89, 796-806, (1994) · Zbl 0825.62445
[47] Ray, S.; Lindsay, BG, The topography of multivariate normal mixtures, Ann Stat, 33, 2042-2065, (2005) · Zbl 1086.62066
[48] Ray, S.; Ren, D., On the upper bound of the number of modes of a multivariate normal mixture, J Multivar Anal, 108, 41-52, (2012) · Zbl 1238.62064
[49] Rinaldo, A.; Singh, A.; Nugent, R.; Wasserman, L., Stability of density-based clustering, J Mach Learn Res, 13, 905-948, (2012) · Zbl 1283.62130
[50] Schnell, P., Eine methode zur auffindung von gruppen, Biometrische Zeitschrift, 6, 47-48, (1964)
[51] Scrucca, L., Identifying connected components in Gaussian finite mixture models for clustering, Comput Stat Data Anal, 93, 5-17, (2016) · Zbl 1468.62174
[52] Scrucca, L.; For, M.; Murphy, TB; Raftery, AE, mclust 5: clustering, classification and density estimation using gaussian finite mixture models, R J, 8, 289-317, (2016)
[53] Stuetzle, W., Estimating the cluster tree of a density by analyzing the minimal spanning tree of a sample, J Classif, 20, 25-47, (2003) · Zbl 1055.62075
[54] Walther, G.; Misra, JC (ed.), Bikernel mixture analysis, 586-604, (2003), New Delhi
[55] Wand, MP; Jones, MC, Comparison of smoothing parameterizations in bivariate kernel density estimation, J Am Stat Assoc, 88, 520-528, (1993) · Zbl 0775.62105
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.