Local statistical modeling via a cluster-weighted approach with elliptical distributions. (English) Zbl 1360.62335

Summary: Cluster-weighted modeling (CWM) is a mixture approach to modeling the joint probability of data coming from a heterogeneous population. Under Gaussian assumptions, we investigate statistical properties of CWM from both theoretical and numerical point of view; in particular, we show that Gaussian CWM includes mixtures of distributions and mixtures of regressions as special cases. Further, we introduce CWM based on Student-\(t\) distributions, which provides a more robust fit for groups of observations with longer than normal tails or noise data. Theoretical results are illustrated using some empirical studies, considering both simulated and real data. Some generalizations of such models are also outlined.


62H30 Classification and discrimination; cluster analysis (statistical aspects)
62F10 Point estimation
62J05 Linear regression; mixed models


CapeML; flexmix
Full Text: DOI


[1] ANDERSON, JA, Separate sample logistic discrimination, Biometrika, 59, 19-35, (1972) · Zbl 0231.62080
[2] ANDREWS, RL; ANSARI, A; CURRIM, IS, Hierarchical Bayes versus finite mixture conjoint analysis models: A comparison of fit, prediction, and partworth recovery, Journal of Marketing, 39, 87-98, (2002)
[3] ANDREWS, JL; McNICHOLAS, PD, Extending mixtures of multivariate T-factor analyzers, Statistics and Computing, 21, 361-373, (2011) · Zbl 1255.62171
[4] Baek, J; McLACHLAN, GJ, Mixtures of common T-factor analyzers for clustering high-dimensional microarray data, Bioinformatics, 27, 1269-1276, (2011)
[5] BERNARDO, JM; GIRÓN, FJ; Bernardo, JM (ed.); Berger, JO (ed.); Dawid, AP (ed.); Smith, AFM (ed.), Robust sequential prediction from non- random samples: the election night forecasting case, 61-77, (1992), Oxford
[6] CAMPBELL, NA; MAHON, RJ, A multivariate study of variation in two species of rock crab of genus letpograspus, Australian Journal of Zoology, 22, 417-455, (1974)
[7] Cerioli, A, Multivariate outlier detection with high-breakdown estimators, Journal of the American Statistical Society, 105, 147-156, (2010) · Zbl 1397.62167
[8] CUESTA-ALBERTOS, JA; MATRÁN, C; MAYO-ISCAR, A, Trimming and likelihood: robust location and dispersion estimation in the elliptical model, The Annals of Statistics, 36, 2284-2318, (2008) · Zbl 1148.62038
[9] DAYTON, CM; MACREADY, GB, Concomitant-variable latent-class models, Journal of the American Statistical Association, 83, 173-178, (1988)
[10] DESARBO, WS; CRON, WL, A maximum likelihood methodology for cluster wise linear regression, Journal of Classification, 5, 249-282, (1988) · Zbl 0692.62052
[11] DICKEY, JT, Matricvariate generalizations of the multivariate t distribution and the inverted multivariate t distribution, The Annals of Mathematical Statistics, 38, 511-518, (1967) · Zbl 0158.18403
[12] EVERITT, B.S., and HAND, D.J. (1981), Finite Mixture Distributions, London: Chapman & Hall. · Zbl 0466.62018
[13] Faria, S; Soromenho, G, Fitting mixtures of linear regressions, Journal of Statistical Computation and Simulation, 80, 201-225, (2010) · Zbl 1184.62118
[14] FONSECA, J.R.S. (2008), “Mixture Modeling and Information Criteria for Discovering Patterns in Continuous Data”, Eighth International Conference on Hybrid Intelligent Systems, IEEE Computer Society. · Zbl 1248.62091
[15] FRÜWIRTH-SCHNATTER, S. (2005), Finite Mixture and Markov Switching Models, Heidelberg: Springer.
[16] GALLEGOS, MT; RITTER, G, Trimming- algorithms for clustering contaminated grouped data and their robustness, Advances in Data Analysis and Classification, 3, 135-167, (2009) · Zbl 1284.62372
[17] GALLEGOS, MT; RITTER, G, Trimmed ML estimation of contaminated mixtures, Sankhya, 71-A, Part, 2, 164-220, (2009) · Zbl 1193.62021
[18] Gershenfeld, N, Non linear inference and cluster-weighted modeling, Annals of the New York Academy of Sciences, 808, 18-24, (1997)
[19] GERSHENFELD, N. (1999), The Nature of Mathematical Modelling, Cambridge: Cambridge University Press, pp. 101-130. · Zbl 0905.00015
[20] Gershenfeld, N; Schöner, B; Metois, E, Cluster-weighted modelling for time-series analysis, Nature, 397, 329-332, (1999)
[21] Greselin, F; Ingrassia, S, Constrained monotone EM algorithms of multivariate \(t\) distributions, Statistics & Computing, 20, 9-22, (2010)
[22] Ingrassia, S, A likelihood-based constrained algorithm for multivariate normal mixture models, Statistical Methods & Applications, 13, 151-166, (2004) · Zbl 1205.62066
[23] Ingrassia, S; Rocci, R, Constrained monotone EM algorithms for finite mixture of multivariate gaussians, Computational Statistics & Data Analysis, 51, 5339-5351, (2007) · Zbl 1445.62116
[24] JANSEN, RC, Maximum likelihood in a generalized linear finite mixture model by using the EM algorithm, Biometrics, 49, 227-231, (1993)
[25] JORDAN, M.I. (1995), “Why the Logistic Function? A Tutorial Discussion on Probabilities and Neural Networks”, MIT Computational Cognitive Science Report 9503.
[26] JORDAN, MI; JACOBS, RA, Hierarchical mixtures of experts and the EM algorithm, Neural Computation, 6, 181-224, (1994)
[27] KAN, R., and ZHOU, G. (2006), “Modelling Non-Normality Using Multivariate t: Implications for Asset Pricing”, Working paper, Washington University, St. Louis. · Zbl 1445.62116
[28] LANGE, KL; LITTLE, RJA; TAYLOR, JMG, Robust statisticalmodeling using the \(t\) distribution, Journal of the American Statistical Society, 84, 881-896, (1989)
[29] Leisch, F, Flexmix: A general framework for finite mixture models and latent class regression in R, Journal of Statistical Software, 11, 1-18, (2004)
[30] Liu, C; RUBIN, DM, ML estimation of the \(t\) distribution using EM and its extensions, ECM and ECME, Statistica Sinica, 5, 19-39, (1995) · Zbl 0824.62047
[31] MARDIA, K.V., KENT, J.T., and BIBBY, J.M. (1979), Multivariate Analysis, London: Academic Press. · Zbl 0432.62029
[32] McLACHLAN, G.J., and BASFORD, K.E. (1988), Mixture Models: Inference and Applications to Clustering, New York: Marcel Dekker. · Zbl 0697.62050
[33] McLACHLAN, GJ; PEEL, D; Amin, A (ed.); Dori, D (ed.); Pudil, P (ed.); Freeman, H (ed.), Robust cluster analysis via mixtures of multivariate t-distributions, No. 1451, 658-666, (1998), Berlin
[34] McLACHLAN, G.J., and PEEL, D. (2000), Finite Mixture Models, New York: Wiley. · Zbl 0963.62061
[35] Nadarajah, S; Kotz, S, Mathematical properties of the multivariate \(t\) distributions, Acta Applicandae Mathematicae, 89, 53-84, (2005) · Zbl 1092.62060
[36] Newcomb, S, A generalized theory of the combination of observations so as to obtain the best result, American Journal of Mathematics, 8, 343-366, (1886) · JFM 18.0183.01
[37] NG, SK; McLACHLAN, GJ, Extension of mixture-of-experts networks for binary classification of hierarchical data, Artificial Intelligence in Medicine, 41, 57-67, (2007)
[38] NG, SK; McLACHLAN, GJ; Peters, H (ed.); Vogel, M (ed.), Expert networks with mixed continuous and categorical feature variables: A location modeling approach, 355-368, (2008), New York
[39] NIERENBERG, DW; STUKEL, TA; BARON, J; DAIN, BJ; GREENBERG, R, Determinants of plasma levels of beta-carotene and retinol, American Journal of Epidemiology, 130, 511-521, (1989)
[40] Pearson, K, Contributions to the mathematical theory of evolution, Philosophical Transactions of the Royal Society of London A, 185, 71-110, (1894) · JFM 25.0347.02
[41] Peel, D; McLACHLAN, GJ, Robust mixture modelling using the \(t\) distribution, Statistics & Computing, 10, 339-348, (2000)
[42] Peng, F; JACOBS, RA; TANNER, MA, Bayesian inference in mixtures of- experts and hierarchical mixtures-of-experts models with an application to speech recognition, Journal of the American Statistical Association, 91, 953-960, (1996) · Zbl 0882.62022
[43] PINHEIRO, JC; LIU, C; WU, YN, Efficient algorithms for robust estimation in linear mixed-effects models using the multivariate \(t\) distribution, Journal of Computational and Graphical Statistics, 10, 249-276, (2001)
[44] QUANDT, RE, A new approach to estimating switching regressions, Journal of the American Statistical Society, 67, 306-310, (1972) · Zbl 0237.62047
[45] RIANI, M; CERIOLI, A; ATKINSON, AC; PERROTTA, D; TORTI, F; Fogelman-Soulié, F (ed.); Perrotta, D (ed.); Piskorki, J (ed.); Steinberg, R (ed.), Fitting mixtures of regression lines with the forward search, 271-286, (2008), Amsterdam
[46] Riani, M; ATKINSON, AC; CERIOLI, A, Finding an unknown number of multivariate outliers, Journal of the Royal Statistical Society B, 71, 447-466, (2009) · Zbl 1248.62091
[47] SCHLATTMANN, P. (2009), Medical Applications of Finite Mixture Models, Berlin-Heidelberg: Springer-Verlag. · Zbl 1158.62082
[48] SCHÖNER, B. (2000), Probabilistic Characterization and Synthesis of Complex Data Driven Systems, Ph.D. Thesis, MIT. · Zbl 0348.62026
[49] SCHÖNER, B; GERSHENFELD, N; Mees, AI (ed.), Cluster weighted modeling: probabilistic time series prediction, characterization, and synthesis, 365-385, (2001), Boston
[50] TITTERINGTON, D.M., SMITH, A.F.M., and MAKOV, U.E. (1985), Statistical Analysis of Finite Mixture Distributions, New York: Wiley. · Zbl 0646.62013
[51] Wang, P; PUTERMAN, ML; COCKBURN, I; LE, N, Mixed Poisson regression models with covariate dependent rates, Biometrics, 52, 381-400, (1996) · Zbl 0875.62407
[52] Wedel, M, Concomitant variables in finite mixture models, Statistica Nederlandica, 56, 362-375, (2002) · Zbl 1076.62531
[53] Wedel, M; Desarbo, W, A mixture likelihood approach for generalized linear models, Journal of Classification, 12, 21-55, (1995) · Zbl 0825.62611
[54] Wedel, M; Desarbo, W, Market segment derivation and profiling via a finite mixture model framework, Marketing Letters, 13, 17-25, (2002)
[55] WEDEL, M., and KAMAMURA, W.A. (2000), Market Segmentation. Conceptual and Methodological Foundations, Boston: Kluwer Academic Publishers.
[56] Zellner, A, Bayesian and non-Bayesian analysis of the regressionmodel with multivariate student-\(t\) error terms, Journal of the American Statistical Society, 71, 400-405, (1976) · Zbl 0348.62026
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.