A maximum likelihood methodology for clusterwise linear regression. (English) Zbl 0692.62052

Summary: This paper presents a conditional mixture, maximum likelihood methodology for performing clusterwise linear regression. This new methodology simultaneously estimates separate regression functions and membership in K clusters or groups. A review of related procedures is discussed with an associated critique.
The conditional mixture, maximum likelihood methodology is introduced together with the E-M algorithm utilized for parameter estimation. A Monte Carlo analysis is performed via a fractional factorial design to examine the performance of the procedure. Next, a marketing application is presented concerning the evaluations of trade show performance by senior marketing executives. Finally, other potential applications and directions for future research are identified.


62H30 Classification and discrimination; cluster analysis (statistical aspects)
62P20 Applications of statistics to economics
62P99 Applications of statistics
62J05 Linear regression; mixed models


Algorithm 39
Full Text: DOI


[1] ADDELMAN, S. (1962), ”Orthogonal Main Effects Plans for Asymmetrical Factorial Experiments,”Technometrics, 4, 21–46. · Zbl 0116.36704
[2] AKAIKE, H. (1974), ”A New Look at Statistical Model Identification,”IEEE Transactions on Automatic Control, AC-19, 716–723. · Zbl 0314.62039
[3] BASFORD, K.E., and MCLACHLAN, G.J. (1985), ”The Mixture Method of Clustering Applied to Three-Way Data,”Journal of Classification, 2, 109–125.
[4] BINDER, D.A. (1978), ”Bayesian Cluster Analysis,”Biometrika, 65, 31–38. · Zbl 0376.62007
[5] BONOMA, T.V. (1983), ”Get More Out of Your Trade Shows,”Harvard Business Review, 61, 75–83.
[6] BOZDOGAN, H. (1983), ”Determining the Number of Component Clusters in Standard Multivariate Normal Mixture Models Using Model-Selection Criterion,”Technical Report VIC/DOM/A83-1, Army Research Office.
[7] CAVANAUGH, S. (1976), ”Setting Objectives and Evaluating the Effectiveness of Trade Show Exhibits,”Journal of Marketing, 40, 100–103.
[8] CHARLIER, C.V.L., and WICKSELL, S.D. (1924), ”On the Dissection of Frequency Functions,”Arkiv för Matematik, Astronomi Och Fysik., Bd.18, No.6, 85–98. · JFM 50.0657.01
[9] CLEAVER, J. (1982), ”You Don’t Have to be a Star in this Show,”Advertising Age, 53, 9.
[10] COHEN, A.C. (1967), ”Estimation in Mixtures of Two Normal Distributions,”Technometrics, 9, 15–28. · Zbl 0147.18104
[11] COOPER, P. W. (1964), ”Non Supervised Adaptive Signal Detection and Pattern Recognition,”Information and Control, 7, 416–444. · Zbl 0199.22402
[12] DAY, N.E. (1969), ”Estimating the Components of a Mixture of Normal Distributions,”Biometrika, 56, 463–474. · Zbl 0183.48106
[13] DEMPSTER, A.P., LAIRD, N.M., and RUBIN, D.B. (1977), ”Maximum likelihood from Incomplete Data Via the E-M Algorithm,”Journal of the Royal Statistical Society, B39, 1–38. · Zbl 0364.62022
[14] DESARBO, W.S. (1982), ”GENNCLUS: New Models for General Nonhierarchical Clustering Analysis,”Psychometrika, 47, 449–476. · Zbl 0566.62057
[15] DESARBO, W.S., and CARROLL, J.D. (1985), ”Three Way Metric Unfolding Via Alternating Weighted Least Squares,”Psychometrika, 50, 275–300. · Zbl 0597.62113
[16] DESARBO, W.S., CARROLL, J.D., CLARK, L.A., and GREEN, P.E. (1984), ”Synthesized Clustering: A Method for Amalgamating Alternative Clustering Bases with Differential Weighting of Variables,”Psychometrika, 49, 57–78. · Zbl 0594.62067
[17] DESARBO, W.S., and MAHAJAN, V. (1984), ”Constrained Classification: The Use of a Priori Information in Cluster Analysis,”Psychometrika, 49, 187–215. · Zbl 0554.62050
[18] DESARBO, W.S., OLIVER, R., and RANGASWAMY, A. (1988), ”A Simulated Annealing Methodology for Clusterwise Linear Regression,”Working Paper, University of Michigan, Ann Arbor, MI.
[19] De SOETE, G., DESARBO, W.S., FURNAS, G.W., and CARROLL, J.D. (1984), ”The Representation of Nonsymmetric Rectangular Proximity Data by Ultrametric and Path Length Tree Structure,”Psychometrika, 49, 289–310.
[20] De SOETE, G., DESARBO, W.S., and CARROLL, J.D. (1985), ”Optimal Variable Weighting for Hierarchical Clustering: An Alternating Least Squares Algorithm,”Journal of Classification, 2, 173–192. · Zbl 0585.62111
[21] DUNN, S.W., and BARBAN, A.M. (1986),Advertising, 6th ed., Hinsdale, IL: Dryden.
[22] DYNKIN, E.B. (1961), ”Necessary and Sufficient Statistics for a Family of Probability Distributions,”Selected Translations in Mathematical Statistics and Probability, Providence, RI: American Mathematical Society, 17–40. · Zbl 0112.11106
[23] EVERITT, B.S., and HAND, D.J. (1981), Finite Mixture Distribution, New York: Chapman and Hall. · Zbl 0466.62018
[24] GANESALINGAM, S., and MCLACHLAN, G.J. (1981), ”Some Efficiency Results for the Estimation of the Mixing Proportion in a Mixture of Two Normal Distributions,”Biometrics, 37, 23–33. · Zbl 0498.62049
[25] GREEN, P.E., and RAO, V.R. (1971), ”Conjoint Measurement for Quantifying Judgmental Data,”Journal of Marketing Research, 8, 355–363.
[26] GREEN, P.E., and TULL, D.S. (1978),Research for Marketing Decisions, 4th ed., Englewood Cliffs, NJ: Prentice-Hall.
[27] GUILFORD, J.A. (1954),Psychometric Methods, 2nd ed., New York: McGraw-Hill.
[28] HAAS, R.W. (1982),Industrial Marketing Management, 2nd ed., Boston: Kent.
[29] HARTIGAN, J.A. (1975),Clustering Algorithms, New York: Wiley. · Zbl 0372.62040
[30] HARTIGAN, J.A. (1977), ”Distribution Problems in Clustering,” inClassification and Clustering, ed. J. Van Ryzin, New York: Academic Press, 45–71.
[31] HASSELBLAD, V. (1966), ”Estimation of Parameters for a Mixture of Normal Distributions,”Technometrics, 8, 431–444.
[32] HATCH, M. (1981),How to Improve Sales Success at Trade Shows, New Canaan, CT: Trade Show Bureau.
[33] HOSMER, D.W. (1974), ”Maximum Likelihood Estimates of the Parameters of a Mixture of Two Regression Lines,”Communications in Statistics, 3, 995–1006. · Zbl 0294.62085
[34] HUTT, M.D., and SPEH, T.W. (1985),Industrial Marketing Management, 2nd ed., Hinsdale, IL: Dryden.
[35] JOHNSTON, J. (1984),Econometric Methods, 3rd ed., New York: McGraw-Hill.
[36] JUDGE, G.C., GRIFFITHS, W.E., HILL, R.C., LUTKEPOHL, H., and LEE, T.C. (1985),Theory and Practice of Econometrics, New York: Wiley.
[37] KERIN, R.A., and CRON, W.L. (1987), ”Assessing Trade Show Functions and Performance: An Exploratory Study,”Journal of Marketing, 51, 87–94.
[38] KONIKOW, R.B. (1983),How to Participate Profitably in Trade Shows, rev. ed., Chicago: Dartnell.
[39] KRAYENBUEHL, T.E. (1985),Country Risk: Assessment and Monitoring, Lexington, MA: Lexington.
[40] LACHENBRUCH, P.A. (1967), ”An Almost Unbiased Method of Obtaining Confidence Intervals for the Probability of Misclassification in Discriminant Analysis,”Biometrics, 23, 639–645.
[41] LACHENBRUCH, P.A. (1975),Discriminant Analysis, New York: Hafner Press. · Zbl 0354.62050
[42] LILIEN, G.L. (1983), ”A Descriptive Model of the Trade Show Budgeting Decision Process,”Industrial Marketing Management, 12, 25–29.
[43] LOUIS, T.A. (1982), ”Finding the Observed Information Matrix When Using the E-M Algorithm,”Journal of the Royal Statistical Society, B44, 226–233. · Zbl 0488.62018
[44] MACQUEEN, J. (1967), ”Some Methods for Classification and Analysis of Multivariate Observations,” in the5th Berkeley Symposium of Mathematics, Statistics and Probability, Vol. 1, eds. L.M. LeCam and J. Neyman, Los Angeles, CA: University of California Press, 281–298. · Zbl 0214.46201
[45] MADDALA, G.S. (1976),Econometrics, New York: McGraw-Hill.
[46] MARRIOTT, F.H.C. (1975), ”Separating Mixtures of Normal Distributions,”Biometrics, 31, 767–769. · Zbl 0308.62050
[47] MCLACHLAN, G.J. (1982), ”The Classification and Mixture Maximum Likelihood Approaches to Cluster Analysis,” inHandbook of Statistics, Vol. 2, eds. P.R. Krishnaiah and L.N. Kanal, Amsterdam: North-Holland, 199–208. · Zbl 0513.62064
[48] MCLACHLAN, G.J. (1987), ”On Bootstrapping the Likelihood Ratio Test Statistic for the Number of Components in a Normal Mixture,”Applied Statistics, 36, 318–324.
[49] MCLACHLAN, G.J. and BASFORD, K.E. (1988),Mixture Models: Inference and Applications to Clustering, New York: Marcel Dekker. · Zbl 0697.62050
[50] MEE, W.W. (1982),Trade Show Exhibit Cost Analysis, East Orleans, MA: Trade Show Bureau.
[51] MEE, W.W. (1983a),Trade Show Industry Growth 1972–1981, Projected Growth 1981–1991, East Orleans, MA: Trade Show Bureau.
[52] MEE, W.W. (1983b),The Exhibitors: Their Trade Show Practices, East Orleans, MA: Trade Show Bureau.
[53] MEE, W.W. (1984),Audience Characteristics – Regional and National Trade Shows, East Orleans, MA: Trade Show Bureau.
[54] MORRISON, D. (1969), ”On the Interpretation of Discriminant Analysis,”Journal of Marketing Research, 6, 156–163.
[55] PEARSON, K. (1894), ”Contribution to the Mathematical Theory of Evolution,”Philosophical Transactions of the Royal Society of London, A, 185, 71–110. · JFM 25.0347.02
[56] PETERS, B.C., and WALKER, H.F. (1978), ”An Iterative Procedure for Obtaining Maximum Likelihood Estimates of the Parameters for a Mixture of Normal Distributions,”SIAM Journal on Applied Mathematics, 35, 362–378. · Zbl 0443.65112
[57] PORTER, M.E. (1980),Competitive Strategy, New York: Free Press.
[58] QUANDT, R.E. (1972), ”A New Approach to Estimating Switching Regressions,”Journal of the American Statistical Association, 67, 306–310. · Zbl 0237.62047
[59] QUANDT, R.E., and RAMSEY, J.B. (1978), ”Estimating Mixtures of Normal Distributions and Switching Regressions,”Journal of the American Statistical Association, 73, 730–738. · Zbl 0401.62024
[60] REDNER, R.A., and WALKER, H.F. (1984), ”Mixture Densities, Maximum Likelihood, and the E-M Algorithm,”SIAM Review, 2, 195–239. · Zbl 0536.62021
[61] RICH, M. (1985), ”Regional Shows Give Small Marketers an Even Break,”Marketing News, May 27, 19, p. 15.
[62] SCLOVE, S.C. (1977), ”Population Mixture Models and Clustering Algorithms,”Communication in Statistics, A6, 417–434. · Zbl 0368.62036
[63] SCLOVE, S.L. (1983), ”Application of the Conditional Population Mixture Model to Image Segmentation,”IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-5, 428–433. · Zbl 0514.62065
[64] SCLOVE, S.L. (1987), ”Application of Model-Selection Criteria to Some Problems in Multivariate Analysis,”Psychometrika, 52, 333–343.
[65] SCOTT, A.J., and SYMONS, M.J. (1971), ”Clustering Methods Based on Likelihood Ratio Criteria,”Biometrics, 27, 238–397.
[66] SPÄTH, H. (1979), ”Algorithm 39: Clusterwise Linear Regression,”Computing, 22, 367–373. · Zbl 0387.65028
[67] SPÄTH, H. (1981), ”Correction to Algorithm 39: Clusterwise Linear Regression,”Computing, 26, 275. · Zbl 0444.65020
[68] SPÄTH, H. (1982), ”Algorithm 48: A Fast Algorithm for Clusterwise Linear Regression,”Computing, 29, 175–181. · Zbl 0485.65030
[69] SPÄTH, H. (1985),Cluster Dissection and Analysis, New York: Wiley. · Zbl 0584.62094
[70] SYMONS, M.J. (1981), ”Clustering Criteria and Multivariate Normal Mixtures,”Biometrics, 37, 35–43. · Zbl 0473.62048
[71] TEICHER, H. (1961), ”Identifiability of Mixtures,”Annals of Mathematical Statistics, 32, 244–248. · Zbl 0146.39302
[72] TEICHER, H. (1963), ”Identifiability of Finite Mixtures,”Annals of Mathematical Statistics, 34, 1265–1269. · Zbl 0137.12704
[73] THEIL, H. (1971),Principles of Econometrics, New York: Wiley. · Zbl 0221.62002
[74] TITTERINGTON, D.M., SMITH, A.F.M., and MAKOV, U.E. (1985),Statistical Analysis of Finite Mixture Distributions, New York: Wiley. · Zbl 0646.62013
[75] VEAUX, R.D. (1986), ”Parameter Estimation for a Mixture of Linear Regressions,”Technical Report No.247, Department of Statistics, Stanford University, Stanford, CA.
[76] WILSON, D.L., and SARGENT, R.G. (1979), ”Some Results of Monte Carlo Experiments in Estimating the Parameters of the Finite Mixed Exponential Distribution,” in theProceedings of the Twelfth Annual Symposium Interface, ed. J.F. Gentleman, University of Waterloo, Ontario, Canada, 461–465.
[77] WOLFE, J.H. (1965), ”A Computer Programs for the Maximum Likelihood Analysis of Types.” Technical Bulletin, 65-15, U.S. Naval Personnel Research Activity, San Diego, CA.
[78] WOLFE, J.H. (1967), ”NORMIX: Computational Methods for Estimating the Parameters of Multivariate Normal Mixtures of Distributions,”Research Memorandum SRM 68-2, U.S. Naval Personnel Research Activity, San Diego, CA.
[79] WOLFE, J.H. (1970), ”Pattern Clustering by Multivariate Mixture Analysis,”Multivariate Behavioral Research, 5, 329–350.
[80] WOLFE, J.H. (1971), ”A Monte Carlo Study of the Sampling Distribution of the Likelihood Ratio for Mixture of Multinormal Distributions,”Technical Bulletin: Naval Personnel and Training Research Laboratory, STB 72-2, San Diego, CA.
[81] YAKOWITZ, S.J. (1970), ”Unsupervised Learning and the Identification of Finite Mixtures,”IEEE Transactions on Information Theory and Control, IT-16, 330–338. · Zbl 0197.45502
[82] YAKOWITZ, S.J., and SPRAGINS, J.D. (1968), ”On the Identifiability of Finite Mixtures,”Annals of Mathematical Statistics, 39, 209–214. · Zbl 0155.25703
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.