Regularized fuzzy clusterwise ridge regression. (English) Zbl 1306.62166

Summary: Fuzzy clusterwise regression has been a useful method for investigating cluster-level heterogeneity of observations based on linear regression. This method integrates fuzzy clustering and ordinary least-squares regression, thereby enabling to estimate regression coefficients for each cluster and fuzzy cluster memberships of observations simultaneously. In practice, however, fuzzy clusterwise regression may suffer from multicollinearity as it builds on ordinary least-squares regression. To deal with this problem in fuzzy clusterwise regression, a new method, called regularized fuzzy clusterwise ridge regression, is proposed that combines ridge regression with regularized fuzzy clustering in a unified framework. In the proposed method, ridge regression is adopted to estimate clusterwise regression coefficients while handling potential multicollinearity among predictor variables. In addition, regularized fuzzy clustering based on maximizing entropy is utilized to systematically determine an optimal degree of fuzziness in memberships. A simulation study is conducted to evaluate parameter recovery of the proposed method as compared to the extant non-regularized counterpart. The usefulness of the proposed method is illustrated by an application concerning the relationship among the characteristics of used cars.


62J07 Ridge regression; shrinkage estimators (Lasso)
62H86 Multivariate analysis and fuzziness
62H30 Classification and discrimination; cluster analysis (statistical aspects)


Algorithm 39
Full Text: DOI


[1] Aurifeille J-M, Quester PG (2003) Predicting business ethical tolerance in international markets: a concomitant clusterwise regression analysis. Int Bus Rev 12: 253–272
[2] Belsley DA, Kuh E, Welsch RE (1980) Regression diagnostics: identifying influential data and sources of collinearity. Wiley, New York · Zbl 0479.62056
[3] Bezdek JC (1981) Pattern recognition with fuzzy objective function algorithms. Plenum Press, New York · Zbl 0503.68069
[4] Brusco MJ, Cradit JD, Steinley D, Fox GL (2008) Cautionary remarks on the use of clusterwise regression. Multivar Behav Res 43: 29–49
[5] DeSarbo WS, Cron WL (1988) A maximum likelihood methodology for clusterwise linear regression. J Classif 5: 249–282 · Zbl 0692.62052
[6] DeSarbo WS, Oliver RL, Rangaswamy A (1989) A simulated annealing methodology for clusterwise linear regression. Psychometrika 54: 707–736 · Zbl 04567867
[7] DeVeaux RD (1989) Mixtures of linear regressions. Comput Stat Data Anal 8: 227–245 · Zbl 0726.62109
[8] D’Urso P (2003) Linear regression analysis for fuzzy/crisp input and fuzzy/crisp output data. Comput Stat Data Anal 42: 47–72 · Zbl 1429.62337
[9] D’Urso P, Santoro A (2006) Fuzzy clusterwise linear regression analysis with symmetrical fuzzy output variable. Comput Stat Data Anal 51: 287–313 · Zbl 1157.62461
[10] Efron B (1982) The jackknife, the bootstrap, and other resampling plans. Society of Industrial and Applied Mathematics CBMS-NSF Monographs 38 · Zbl 0496.62036
[11] Falk TH, Shatkay H, Chan W-Y (2006) Breast cancer prognosis via Gaussian mixture regression. Canadian conference on electrical and computer engineering, Queen’s University, Canada, pp 987–990
[12] Gordon AD (1999) Classification. Chapman and Hall/CRC, Boca Raton
[13] Hathaway RJ, Bezdek JC (1993) Switching regression models and fuzzy clustering. IEEE Trans Fuzzy Syst 1: 195–204
[14] Heiser WJ, Groenen PJF (1997) Cluster differences scaling with a within-cluster loss component and a fuzzy successive approximation strategy to avoid local minima. Psychometrika 62: 63–83 · Zbl 0889.92037
[15] Hoerl AE, Kennard RW (1970) Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12: 55–67 · Zbl 0202.17205
[16] Hoerl AE, Kennard RW (1970) Ridge regression: applications to nonorthogonal problems. Technometrics 12: 69–82 · Zbl 0202.17206
[17] Hosmer DW (1974) Maximum likelihood estimates of the parameters of a mixture of two regression lines. Commun Stat 3: 995–1006 · Zbl 0294.62085
[18] Hruschka H (1986) Market definition and segmentation using fuzzy clustering methods. Int J Res Mark 3: 117–134
[19] Kuiper S (2008) Introduction to multiple regression: how much is your car worth? J Stat Educ 16. Available via http://www.amstat.org/publications/jse/v16n3/datasets.kuiper.html . Accessed 25 May 2009
[20] Kutner MH, Nachtsheim C, Neter J (2004) Applied linear regression models. McGraw-Hill/Irwin, Boston
[21] Laviolette M, Seaman JW (1992) Evaluating fuzzy representations of uncertainty. Math Sci 17: 26–41 · Zbl 0800.94349
[22] Laviolette M, Seaman JW (1994) Unity and diversity of fuzziness–from a probability viewpoint. IEEE Trans Fuzzy Syst 2: 38–42
[23] Laviolette M, Seaman JW, Barrett JD, Woodall WH (1995) A probabilistic and statistical view of fuzzy methods. Technometrics 37: 249–261 · Zbl 0837.62081
[24] Li R-P, Mukaidono M (1995) A maximum entropy approach to fuzzy clustering. Proceedings of the 4th IEEE international conference on fuzzy systems, Yokohama, Japan, pp 2227–2232
[25] McBratney AB, Moore AW (1985) Application of fuzzy sets to climatic classification. Agric For Meteorol 35: 165–185
[26] Miyamoto S (1998) An overview and new methods in fuzzy clustering. In: Jain LC, Jain RK (eds) Second international conference on knowledge-based intelligent electronic systems, Adelaide, Australia, pp 21–23
[27] Miyamoto S, Mukaidono M (1997) Fuzzy c-means as a regularization and maximum entropy approach. 7th international fuzzy systems association world congress, Prague, Czech Republic, pp 86–92
[28] Miyamoto S, Ichihashi H, Honda K (2008) Algorithms for fuzzy clustering: methods in c-means clustering with applications. Springer, Berlin · Zbl 1147.68073
[29] Quandt RE (1958) The estimation of the parameters of a linear regression system obeying two separate regimes. J Am Stat Assoc 53: 873–880 · Zbl 0116.37304
[30] Quandt RE (1972) A new approach to estimating switching regressions. J Am Stat Assoc 67: 306–310 · Zbl 0237.62047
[31] Quandt RE, Ramsey JB (1978) Estimating mixtures of normal distributions and switching regressions. J Am Stat Assoc 73: 730–738 · Zbl 0401.62024
[32] Späth H (1979) Algorithm 39: clusterwise linear regression. Computing 22: 367–373 · Zbl 0387.65028
[33] Späth H (1981) Correction to algorithm 39: clusterwise linear regression. Computing 26: 275 · Zbl 0444.65020
[34] Späth H (1982) Algorithm 48: a fast algorithm for clusterwise linear regression. Computing 29: 175–181 · Zbl 0485.65030
[35] Späth H (1985) Cluster dissection and analysis. Wiley, New York · Zbl 0584.62094
[36] Takane Y, Hwang H (2007) Regularized linear and kernel redundancy analysis. Comput Stat Data Anal 52: 394–405 · Zbl 1452.62421
[37] Tran D, Wagner M (2000) Fuzzy entropy clustering. The 9th IEEE international conference on fuzzy systems, San Antonio, pp 152–157
[38] Wedel M, DeSarbo WS (1995) A mixture likelihood approach for generalized linear models. J Classif 12: 21–55 · Zbl 0825.62611
[39] Wedel M, Kistemaker C (1989) Consumer benefit segmentation using clusterwise linear regression. Int J Res Mark 6: 45–49
[40] Wedel M, Steenkamp J-BEM (1989) A fuzzy clusterwise regression approach to benefit segmentation. Int J Res Mark 6: 241–258
[41] Wedel M, Steenkamp J-BEM (1991) A clusterwise regression method for simultaneous fuzzy market structuring and benefit segmentation. J Mark Res 28: 385–396
[42] Yang M-S, Ko C-H (1997) On cluster-wise fuzzy regression analysis. IEEE Trans Syst Man Cybern B 27: 1–13
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.