Factorial and reduced \(K\)-means reconsidered. (English) Zbl 1284.62396

Summary: Factorial \(K\)-means analysis (FKM) and Reduced \(K\)-means analysis (RKM) are clustering methods that aim at simultaneously achieving a clustering of the objects and a dimension reduction of the variables. Because a comprehensive comparison between FKM and RKM is lacking in the literature so far, a theoretical and simulation-based comparison between FKM and RKM is provided. It is shown theoretically how FKM’s versus RKM’s performances are affected by the presence of residuals within the clustering subspace and/or within its orthocomplement in the observed data. The simulation study confirmed that for both FKM and RKM, the cluster membership recovery generally deteriorates with increasing amount of overlap between clusters. Furthermore, the conjectures were confirmed that for FKM the subspace recovery deteriorates with increasing relative sizes of subspace residuals compared to the complement residuals, and that the reverse holds for RKM. As such, FKM and RKM complement each other. When the majority of the variables reflect the clustering structure, and/or standardized variables are being analyzed, RKM can be expected to perform reasonably well. However, because both RKM and FKM may suffer from subspace and membership recovery problems, it is essential to critically evaluate their solutions on the basis of the content of the clustering problem at hand.


62H30 Classification and discrimination; cluster analysis (statistical aspects)


Full Text: DOI


[1] American Psychiatric Association, Diagnostic and statistical manual of mental disorders, (1968), American Psychiatric Association Washington, DC
[2] Arabie, P.; Hubert, L., Cluster analysis in marketing research, ()
[3] Bock, H.H., On the interface between cluster analysis, principal component analysis, and multidimensional scaling, (), 17-34
[4] Calinski, R.B.; Harabasz, J., A dendrite method for cluster analysis, Communications in statistics, 3, 1-27, (1974) · Zbl 0273.62010
[5] Cattell, R.B., The scree test for the number of factors, Multivariate behavioral research, 1, 245-276, (1966)
[6] Ceulemans, E.; Kiers, H.A.L., Selecting among three-mode principal component models of different types and complexities: A numerical convex hull based method, British journal of mathematical and statistical psychology, 59, 133-150, (2006)
[7] Cliff, N., Orthogonal rotation to congruence, Psychometrika, 31, 33-42, (1966)
[8] De Soete, G.; Carroll, J.D., K-means clustering in a low-dimensional Euclidean space, (), 212-219
[9] Hubert, L.; Arabie, P., Comparing partitions, Journal of classification, 2, 193-218, (1985)
[10] Kaiser, H.F., The varimax criterion for analytic rotation in factor analysis, Psychometrika, 23, 187-200, (1958) · Zbl 0095.33603
[11] Kaufman, L.; Rousseeuw, P.J., Finding groups in data: an introduction to cluster analysis, (1990), Wiley New York · Zbl 1345.62009
[12] MacQueen, J., Some methods for classification and analysis of multivariate observations, (), 281-296 · Zbl 0214.46201
[13] Mezzich, J.E.; Solomon, H., Taxonomy and behavioral science, (1980), Academic Press London
[14] Milligan, G.W., Clustering validation: results and implications for applied analysis, (), 341-375 · Zbl 0895.62069
[15] Milligan, G.W.; Cooper, M.C., A study of standardization of variables in cluster analysis, Journal of classification, 5, 181-204, (1988)
[16] Schepers, J.; Ceulemans, E.; Van Mechelen, I., Selecting among multi-mode partitioning models of different complexities: A comparison of four model selection criteria, Journal of classification, 25, 67-85, (2008) · Zbl 1260.62048
[17] Steinley, D.; Brusco, M.J., Selection of variables in cluster analysis: an empirical comparison of eight procedures, Psychometrika, 73, 125-144, (2008) · Zbl 1143.62327
[18] Steinley, D.; Henson, R., OCLUS: an analytic method for generating clusters with known overlap, Journal of classification, 22, 221-250, (2005) · Zbl 1336.62191
[19] Tucker, L.R., 1951. A method for synthesis of factor analysis studies, Personnel Research Section Report No. 984. Department of the Army, Washington, DC.
[20] Vichi, M.; Kiers, H.A.L., Factorial \(k\)-means analysis for two-way data, Computational statistics and data analysis, 37, 49-64, (2001) · Zbl 1051.62056
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.