Latent class model with conditional dependency per modes to cluster categorical data. (English) Zbl 1414.62253

Summary: We propose a parsimonious extension of the classical latent class model to cluster categorical data by relaxing the conditional independence assumption. Under this new mixture model, named conditional modes model (CMM), variables are grouped into conditionally independent blocks. Each block follows a parsimonious multinomial distribution where the few free parameters model the probabilities of the most likely levels, while the remaining probability mass is uniformly spread over the other levels of the block. Thus, when the conditional independence assumption holds, this model defines parsimonious versions of the standard latent class model. Moreover, when this assumption is violated, the proposed model brings out the main intra-class dependencies between variables, summarizing thus each class with relatively few characteristic levels. The model selection is carried out by an hybrid MCMC algorithm that does not require preliminary parameter estimation. Then, the maximum likelihood estimation is performed via an EM algorithm only for the best model. The model properties are illustrated on simulated data and on three real data sets by using the associated R package CoModes. The results show that this model allows to reduce biases involved by the conditional independence assumption while providing meaningful parameters.


62H30 Classification and discrimination; cluster analysis (statistical aspects)
62F15 Bayesian inference


Rmixmod; MULTIMIX; CoModes; R
Full Text: DOI HAL


[1] Agresti A (2002) Categorical data analysis, vol 359. Wiley, New York · Zbl 1018.62002
[2] Allman, E.; Matias, C.; Rhodes, J., Identifiability of parameters in latent structure models with many observed variables, Ann Stat, 37, 3099-3132, (2009) · Zbl 1191.62003
[3] Bartholomew D, Knott M, Moustaki I (2011) Latent variable models and factor analysis: a unified approach, vol 899. Wiley, New York · Zbl 1266.62040
[4] Biernacki, C.; Celeux, G.; Govaert, G., Exact and Monte Carlo calculations of integrated likelihoods for the latent class model, J Stat Plan Inference, 140, 2991-3002, (2010) · Zbl 1203.62027
[5] Bretagnolle V (2007) Personal communication. source: Museum
[6] Celeux, G.; Govaert, G., Clustering criteria for discrete data and latent class models, J Classif, 8, 157-176, (1991) · Zbl 0775.62150
[7] Chavent, M.; Kuentz, V.; Saracco, J., A partitioning method for the clustering of categorical variables, 91-99, (2010), Berlin Heidelberg
[8] Choirat, C.; Seri, R., Estimation in discrete parameter models, Stat Sci, 27, 278-293, (2012) · Zbl 1330.62306
[9] Czerniak J, Zarzycki H (2003) Application of rough sets in the presumptive diagnosis of urinary system diseases. Artifical intelligence and security in computing systems, ACS’2002 9th International Conference Proceedings, pp 41-51
[10] Dempster, A.; Laird, N.; Rubin, D., Maximum likelihood from incomplete data via the EM algorithm, J R Stat Soc. Ser B (Method), 39, 1-38, (1977) · Zbl 0364.62022
[11] Espeland, M.; Handelman, S., Using Latent class models to characterize and assess relative error in discrete measurements, Biometrics, 45, 587-599, (1989) · Zbl 0718.62220
[12] Gollini, I.; Murphy, T., Mixture of latent trait analyzers for model-based clustering of categorical data, Stat Comput, 24, 569-588, (2014) · Zbl 1325.62122
[13] Goodman, L., Exploratory latent structure analysis using both identifiable and unidentifiable models, Biometrika, 61, 215-231, (1974) · Zbl 0281.62057
[14] Govaert G (2010) Data analysis, vol 136. Wiley, New york · Zbl 1304.62026
[15] Hagenaars, J., Latent structure models with direct effects between indicators local dependence models, Sociol Methods & Res, 16, 379-405, (1988)
[16] Hand, D.; Yu, K., Idiot’s bayes not so stupid after all?, Int Stat Rev, 69, 385-398, (2001) · Zbl 1213.62010
[17] Huang, J.; Ng, M.; Rong, H.; Li, Z., Automated variable weighting in k-means type clustering, Pattern Anal Mach Intell, IEEE Trans On, 27, 657-668, (2005)
[18] Hubert, L.; Arabie, P., Comparing partitions, J Classif, 2, 193-218, (1985) · Zbl 0587.62128
[19] Jajuga K, Sokołowski A, Bock H (2002) Classification, clustering and data analysis: recent advances and applications. Springer, Berlin · Zbl 1026.00018
[20] Jorgensen, M.; Hunt, L., Mixture model clustering of data sets with categorical and continuous variables, In Proc Conf ISIS, 96, 375-384, (1996)
[21] Kruskal, J., More factors than subjects, tests and treatments: An indeterminacy theorem for canonical decomposition and individual differences scaling, Psychometrika, 41, 281-293, (1976) · Zbl 0339.92015
[22] Kruskal, J., Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics, Linear Algebra Appl, 18, 95-138, (1977) · Zbl 0364.15021
[23] Lebret R, Iovleff S, Langrognet F, Biernacki C, Celeux G, Govaert G (2014) Rmixmod: the R package of the model-based unsupervised, supervised and semi-supervised classification mixmod library. J Stat Softw (in press)
[24] McLachlan G, Krishnan T (1997) The EM algorithm. Applied probability and statistics., Probability and statisticsWiley, New York · Zbl 0882.62012
[25] McLachlan G, Peel D (2000) Finite mixutre models. Applied probability and statistics., Probability and statisticsWiley, New York · Zbl 0963.62061
[26] Moran, M.; Walsh, C.; Lynch, A.; Coen, R.; Coakley, D.; Lawlor, B., Syndromes of behavioural and psychological symptoms in mild alzheimer’s disease, Int J Geriatr Psychiatry, 19, 359-364, (2004)
[27] Qu, Y.; Tan, M.; Kutner, M., Random effects models in latent class analysis for evaluating accuracy of diagnostic tests, Biometrics, 52, 797-810, (1996) · Zbl 0875.62551
[28] Schwarz, G., Estimating the dimension of a model, Ann Stat, 6, 461-464, (1978) · Zbl 0379.62005
[29] Hattum, P.; Hoijtink, H., Market segmentation using brand strategy research: bayesian inference with respect to mixtures of log-linear models, J Classif, 26, 297-328, (2009) · Zbl 1337.62144
[30] Vermunt, J., Multilevel latent class models, Sociol Method, 33, 213-239, (2003)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.