A mixture of generalized hyperbolic factor analyzers. (English) Zbl 1414.62278

Summary: The mixture of factor analyzers model, which has been used successfully for the model-based clustering of high-dimensional data, is extended to generalized hyperbolic mixtures. The development of a mixture of generalized hyperbolic factor analyzers is outlined, drawing upon the relationship with the generalized inverse Gaussian distribution. An alternating expectation-conditional maximization algorithm is used for parameter estimation, and the Bayesian information criterion is used to select the number of factors as well as the number of components. The performance of our generalized hyperbolic factor analyzers model is illustrated on real and simulated data, where it performs favourably compared to its Gaussian analogue and other approaches.


62H30 Classification and discrimination; cluster analysis (statistical aspects)
62H25 Factor analysis and principal components; correspondence analysis
Full Text: DOI arXiv


[1] Aitken, A., On Bernoulli’s numerical solution of algebraic equations, Proc R Soc Edim, 46, 289-305, (1926) · JFM 52.0098.05
[2] Andrews, JL; McNicholas, PD, Extending mixtures of multivariate t-factor analyzers, Stat Comput, 21, 361-373, (2011) · Zbl 1255.62171
[3] Andrews, JL; McNicholas, PD, Mixtures of modified t-factor analyzers for model-based clustering, classification, and discriminant analysis, J Stat Plan Inference, 141, 1479-1486, (2011) · Zbl 1204.62098
[4] Andrews, JL; McNicholas, P., Model-based clustering, classification, and discriminant analysis via mixtures of multivariate \(t\)-distributions, Stat Comput, 22, 1021-1029, (2012) · Zbl 1252.62062
[5] Baek, J.; McLachlan, GJM; Flack, L., Mixtures of factor analyzers with common factor loadings: Applications to the clustering and visualization of high-dimensional data, IEEE Trans Pattern Anal Mach Intell, 32, 1298-1309, (2010)
[6] Barndorff-Nielsen, O.; Halgreen, C., Infinite divisibility of the hyperbolic and generalized inverse Gaussian distributions, Z. Wahrscheinlichkeitstheor Verw. Geb, 38, 309-311, (1977) · Zbl 0403.60026
[7] Bergé, L.; Bouveyron, C.; Girard, S., Hdclassif: high dimensional supervised classification and clustering, R Package Version, 1, 2, (2013)
[8] Bhattacharya, S.; McNicholas, PD, A LASSO-penalized BIC for mixture model selection, Adv Data Anal Classif, 8, 45-61, (2014)
[9] Blæsild P (1978) The shape of the generalized inverse Gaussian and hyperbolic distributions. In: Research Report 37, Department of Theoretical Statistics. Aarhus University, Denmark
[10] Böhning, D.; Diez, E.; Scheub, R.; Schlattmann, P.; Lindsay, B., The distribution of the likelihood ratio for mixtures of densities from the one-parameter exponential family, Ann Inst Stat Math, 46, 373-388, (1994) · Zbl 0802.62017
[11] Bouveyron, C.; Girard, S.; Schmid, C., High-dimensional data clustering, Comput Stat Data Anal, 52, 502-519, (2007) · Zbl 1452.62433
[12] Bouveyron, C.; Brunet-Saumard, C., Model-based clustering of high-dimensional data: a review, Comput Stat Data Anal, 71, 52-78, (2014) · Zbl 1306.65033
[13] Browne RP, McNicholas PD (2015) A mixture of generalized hyperbolic distributions. Can J Stat. doi:10.1002/cjs.11246
[14] Browne, RP; McNicholas, PD; Sparling, MD, Model-based learning using a mixture of mixtures of Gaussian and uniform distributions, IEEE Trans Pattern Anal Mach Intell, 34, 814-817, (2012)
[15] Browne, RP; McNicholas, PD, Estimating common principal components in high dimensions, Adv Data Anal Classif, 8, 217-226, (2014)
[16] Campbell, JG; Fraley, F.; Murtagh, F.; Raftery, AE, Linear flaw detection in woven textiles using model-based clustering, Pattern Recogn Lett, 18, 1539-1548, (1997)
[17] Chen, X.; Cheung, ST; So, S.; Fan, ST; Barry, C.; Higgins, J.; Lai, K-M; Ji, J.; Dudoit, S.; Ng, IO; Rijn, M.; Botstein, D.; Brown, PO, Gene expression patterns in human liver cancers, Mol Biol Cell, 13, 1929-1939, (2002)
[18] Dasgupta, A.; Raftery, AE, Detecting features in spatial point processed with clutter via model-based clustering, J Am Stat Assoc, 93, 294-302, (1998) · Zbl 0906.62105
[19] Dempster, AP; Laird, NM; Rubin, DB, Maximum likelihood from incomplete data via the EM algorithm, J R Stat Soc Ser B, 39, 1-38, (1977) · Zbl 0364.62022
[20] Forina, M.; Armanino, C., Eigenvector projection and simplified non linear mapping of fatty acid content of Italian olive oils, Ann Chim, 72, 127-141, (1982)
[21] Forina, M.; Tiscornia, E., Pattern recognition methods in the prediction of Italian olive oil origin by their fatty acid content, Ann Chim, 72, 143-155, (1982)
[22] Forina, M.; Armanino, C.; Castino, M.; Ubigli, M., Multivariate data analysis as a discriminating method of the origin of wines, Vitis, 25, 189-201, (1986)
[23] Franczak BC, McNicholas PD, Browne RP, Murray PM (2013) Parsimonious shifted asymmetric Laplace mixtures. ArXiv preprint arXiv:1311.0317
[24] Franczak, BC; Browne, RP; McNicholas, PD, Mixtures of shifted asymmetric Laplace distributions, IEEE Trans Pattern Anal Mach Intell, 36, 1149-1157, (2014)
[25] Ghahramani Z, Hinton GE (1997) The EM algorithm for factor analyzers. In: Technical Report CRG-TR-96-1. University of Toronto, Toronto
[26] Good, IJ, The population frequencies of species and the estimation of population parameters, Biometrika, 40, 237-260, (1953) · Zbl 0051.37103
[27] Gorman, RP; Sejnowski, TJ, Analysis of hidden units in a layered network trained to classify sonar targets, Neural Netw, 1, 75-89, (1988)
[28] Halgreen, C., Self-decomposibility of the generalized inverse Gaussian and hyperbolic distributions, Z. Wahrscheinlichkeitstheor Verw. Geb, 47, 13-18, (1979) · Zbl 0377.60020
[29] Hennig, C., Methods for merging Gaussian mixture components, Adv Data Anal Classif, 4, 3-34, (2010) · Zbl 1306.62141
[30] Hubert, L.; Arabie, P., Comparing partitions, J Classif, 2, 193-218, (1985) · Zbl 0587.62128
[31] Jørgensen B (1982) Statistical properties of the generalized inverse Gaussian distribution. Springer, New York · Zbl 0486.62022
[32] Karlis, D.; Santourian, A., Model-based clustering with non-elliptically contoured distributions, Stat Comput, 19, 73-83, (2009)
[33] Lee, SX; McLachlan, GJ, On mixtures of skew normal and skew t-distributions, Adv Data Anal Classif, 7, 241-266, (2013) · Zbl 1273.62115
[34] Lee S, McLachlan G (2013a). EMMIXuskew: fitting unrestricted multivariate skew t mixture models. R package version 0.11-5
[35] Lin T-I, McLachlan GJ, Lee SX (2013) Extending mixtures of factor models using the restricted multivariate skew-normal distribution. ArXiv preprint arXiv:1307.1748
[36] Lin, T-I, Maximum likelihood estimation for multivariate skew normal mixture models, J Multivar Anal, 100, 257-265, (2009) · Zbl 1152.62034
[37] Lin, T-I, Robust mixture modeling using multivariate skew t distributions, Stat Comput, 20, 343-356, (2010)
[38] Lin, T-I; McNicholas, PD; Hsiu, JH, Capturing patterns via parsimonious t mixture models, Stat Probab Lett, 88, 80-87, (2014) · Zbl 1369.62131
[39] Lindsay B (1995). Mixture models: theory, geometry and applications. In: NSF-CBMS regional conference series in probability and statistics, vol 5. Institute of Mathematical Statistics, Hayward, California
[40] Lopes, HF; West, M., Bayesian model assessment in factor analysis, Stat Sin, 14, 41-67, (2004) · Zbl 1035.62060
[41] Markos A, Iodice D’Enza A, Van de Velden M (2013) clustrd: methods for joint dimension reduction and clustering. R package version 0.1.2
[42] Maugis, C.; Celeux, G.; Martin-Magniette, M., Variable selection in model-based clustering: a general variable role modeling, Comput Stat Data Anal, 53, 3872-3882, (2009) · Zbl 1453.62154
[43] McLachlan GJ, Peel D (2000) Mixtures of factor analyzers. In: Proceedings of the seventh international conference on machine learning. San Francisco, Morgan Kaufmann, pp 599-606
[44] McLachlan, GJ; Peel, D.; Bean, RW, Modelling high-dimensional data by mixtures of factor analyzers, Comput Stat Data Anal, 41, 379-388, (2003) · Zbl 1256.62036
[45] McLachlan, GJ; Bean, RW; Jones, LB-T, Extension of the mixture of factor analyzers model to incorporate the multivariate t-distribution, Comput Stat Data Anal, 51, 5327-5338, (2007) · Zbl 1445.62053
[46] McNicholas SM, McNicholas PD, Browne RP (2013) Mixtures of variance-gamma distributions. Arxiv preprint arXiv:1309.2695
[47] McNicholas, PD; Murphy, TB, Parsimonious Gaussian mixture models, Stat Comput, 18, 285-296, (2008)
[48] McNicholas, PD, Model-based classification using latent Gaussian mixture models, J Stat Plan Inference, 140, 1175-1181, (2010) · Zbl 1181.62095
[49] McNicholas, PD; Murphy, TB, Model-based clustering of microarray expression data via latent Gaussian mixture models, Bioinformatics, 26, 2705-2712, (2010)
[50] McNicholas, PD; Jampani, KR; McDaid, AF; Murphy, TB; Banks, L., Pgmm: parsimonious Gaussian mixture models, R Package Version, 1, 1, (2014)
[51] Meng, X.; Dyk, D., The EM algorithm-an old folk song sung to a fast new tune, J R Stat Soc Ser B (Stat Methodol), 59, 511-567, (1997) · Zbl 1090.62518
[52] Montanari, A.; Viroli, C., Maximum likelihood estimation of mixtures of factor analyzers, Comput Stat Data Anal, 55, 2712-2723, (2011) · Zbl 1464.62134
[53] Morris, K.; McNicholas, PD; Scrucca, L., Dimension reduction for model-based clustering via mixtures of multivariate t-distributions, Adv Data Anal Classif, 7, 321-338, (2013) · Zbl 1273.62141
[54] Morris, K.; McNicholas, PD, Dimension reduction for model-based clustering via mixtures of shifted asymmetric Laplace distributions, Stat Probab Lett, 83, 2088-2093, (2013) · Zbl 1282.62153
[55] Murray PM, Browne RB, McNicholas PD (2013) Mixtures of ‘unrestricted’ skew-t factor analyzers. Arxiv preprint arXiv:1310.6224
[56] Murray, PM; Browne, RB; McNicholas, PD, Mixtures of skew-t factor analyzers, Comput Stat Data Anal, 77, 326-335, (2014) · Zbl 06984029
[57] Murray, PM; McNicholas, PD; Browne, RB, A mixture of common skew-\(t\) factor analyzers, Stat, 3, 68-82, (2014)
[58] O’Hagan A, Murphy TB, Gormley IC, McNicholas PD, Karlis D (2014) Clustering with the multivariate normal inverse Gaussian distribution. Comput Stat Data Anal. doi:10.1016/j.csda.2014.09.006
[59] R Core Team (2014) R: a language and environment for statistical computing. In: R foundation for statistical computing. Vienna, Austria
[60] Rand, WM, Objective criteria for the evaluation of clustering methods, J Am Stat Assoc, 66, 846-850, (1971)
[61] Ritter G (2014) Robust cluster analysis and variable selection. Chapman & Hall, Boca Raton · Zbl 1341.62037
[62] Rocci, R.; Gattone, SA; Vichi, M., A new dimension reduction method: factor discriminant k-means, J Classif, 28, 210-226, (2011) · Zbl 1226.62062
[63] Schwarz, G., Estimating the dimension of a model, Ann Stat, 6, 461-464, (1978) · Zbl 0379.62005
[64] Steane, MA; McNicholas, PD; Yada, R., Model-based classification via mixtures of multivariate t-factor analyzers, Commun Stat-Simul Comput, 41, 510-523, (2012) · Zbl 1294.62142
[65] Subedi, S.; McNicholas, PD, Variational Bayes approximations for clustering via mixtures of normal inverse Gaussian distributions, Adv Data Anal Classif, 8, 167-193, (2014)
[66] Tan PJ, Dowe DL (2005) MML inference of oblique decision trees. In: AI 2004: advances in artificial intelligence. Springer, Berlin, Heidelberg, pp 1082-1088
[67] Timmerman ME, Ceulemans E, De Roover K, Van Leeuwen K (2013) Subspace K-means clustering. Behav Res Methods 45(4):1011-1023
[68] Tortora, C.; Browne, RP; Franczak, BC; McNicholas, PD, MixGHD: model based clustering and classification using the mixture of generalized hyperbolic distributions, R Package Version, 1, 4, (2015)
[69] Vichi, M.; Kiers, H., Factorial k-means analysis for two way data, Comput Stat Data Anal, 37, 29-64, (2001) · Zbl 1051.62056
[70] Vrbik, I.; McNicholas, PD, Analytic calculations for the EM algorithm for multivariate skew-mixture models, Stat Probab Lett, 82, 1169-1174, (2012) · Zbl 1244.65012
[71] Vrbik, I.; McNicholas, PD, Parsimonious skew mixture models for model-based clustering and classification, Comput Stat Data Anal, 71, 196-210, (2014) · Zbl 1471.62202
[72] Wang K, Ng A, McLachlan G (2013) EMMIXskew: the EM algorithm and skew mixture distribution. R Package Version 1:1
[73] Wei Y, McNicholas PD (2014) Mixture model averaging for clustering. Adv Data Anal Classif. doi:10.1007/s11634-014-0182-6
[74] Woodbury M (1950) Inverting modified matrices. In: Technical Report 42. Princeton University, Princeton
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.