On the choice of the kernel function in kernel discriminant analysis using information complexity. (English) Zbl 1435.62227

Zani, Sergio (ed.) et al., Data analysis, classification and the forward search. Proceedings of the meeting of the Classification and Data Analysis Group (CLADAG) of the Italian Statistical Society, Parma, Italy, June 6–8, 2005. Berlin: Springer. Stud. Classification Data Anal. Knowl. Organ., 11-21 (2006).
Summary: In this short paper we shall consider the Kernel Fisher Discriminant Analysis (KFDA) and extend the idea of Linear Discriminant, Analysis (LDA) to nonlinear feature space. We shall present a new method of choosing the optimal kernel function and its effect on the KDA classifier using information-theoretic complexity measure.
For the entire collection see [Zbl 1097.62504].


62H30 Classification and discrimination; cluster analysis (statistical aspects)
62B10 Statistical aspects of information-theoretic topics
94A17 Measures of information, entropy


Full Text: DOI


[1] ARONSZAJN, N. (1950): Theory of reproducing kernels. Transactions of the American Mathematical Society, 68, 337-404. · Zbl 0037.20701
[2] BOZDOGAN, H. (1988): ICOMP: A New Model-Selection Criterion. In Hans H. Bock (Ed.): Classification and Related Methods of Data, Analysis, Elsevier Science Publishers B. V. (North Holland), Amsterdam, 599-608. · Zbl 0729.62551
[3] BOZDOGAN, H. (1990): On the information-based measure of covariance complexity and its application to the evaluation of multivariate linear models. Communications in Statistics Theory and Methods, 19(1), 221-278. · Zbl 0900.62041
[4] BOZDOGAN, H. (1994): Mixture-model cluster analysis using model selection criteria and a new informational measure of complexity. In H. Bozdogan (Ed.). Multivariate Statistical Modeling, Vol. 2. Kluwer Academic Publishers, Dordrecht, the Netherlands. 69-113.
[5] BOZDOGAN, H. (2000): Akaike’s information criterion and recent developments in informational complexity. Journal of Mathematical Psychology, 44, 62-91. · Zbl 1047.62501
[6] BOZDOGAN, H. (2004). Intelligent statistical data mining with information complexity and genetic algorithms. In H. Bozdogan (Ed.) Statistical Data Mining and, Knowledge Discovery. Chapman and Hall/CRC, Boca Raton, Florida. 15-56.
[7] BOZDOGAN, H. (2006). Information Complexity and, Multivariate Learning Theory with Data, Mining Applications. Forthcoming book.
[8] CAWLEY, G. and TALBOT, N.L.C. (2003): Efficient leave-one-out cross-validation of kernel Fisher discriminant classifiers. Paper submitted to Elsevier Science. · Zbl 1059.68101
[9] CHEN, M. C-F. (1976): Estimation of covariance matrices under a quadratic loss function. Research Report S-46, Department of Mathematics, SUNY at Albany, Albany, N.Y., 1-33.
[10] CRISTIANINI N. and SHAWE-TAYLOR, J. (2000): An introduction to Support Vector Machines. Cambridge University Press. · Zbl 0994.68074
[11] FRANC, V. and HLAVÁČ, V. (2004): Statistical Pattern Recognition Toolbox for Matlab. Research Report of CMP, Czech Technical University in Prague, No.8.
[12] MERCER, J. (1909): Functions of positive and negative type and their connection with the theory of integral equations. Philosophical Transactions Royal Society London, A209, 415-446. · JFM 40.0408.02
[13] MIKA, S. (2002): Kernel Fisher discriminants. Ph.D. Thesis, Technical University of Berlin, Berlin, Germany. · Zbl 1149.68413
[14] MORGERA. S. D. (1985). Information theoretic covariance complexity and its relation to pattern recognition. IEEE Transactions on Systems, Man, and Cybernetics, SMC-15, 608-619. · Zbl 0582.62003
[15] PRESS, S.J. (1975): Estimation of a normal covariance matrix. University of British Columbia.
[16] RIPLEY, B.D. (1994): Neural networks and related methods for classification (with discussion). J. Royal Statistical Soc. Series B, 56: 409-456. · Zbl 0815.62037
[17] SHURYGIN, A. M. (1983): The linear combination of the simplest discriminator and Fisher’s one. In Nauka (Ed.). Applied Statistics. Moscow, 144-158 (in Russian).
[18] VAN EMDEN, M.H. (1971): An Analysis of Complexity. Mathematical Centre Tracts, Amsterdam, 35. · Zbl 0225.68015
[19] VAPNIK, V.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.