zbMATH — the first resource for mathematics

Linear dimensionality reduction using relevance weighted LDA. (English) Zbl 1079.68613
Summary: The Linear Discriminant Analysis (LDA) is one of the most traditional linear dimensionality reduction methods. This paper incorporates the inter-class relationships as relevance weights into the estimation of the overall within-class scatter matrix in order to improve the performance of the basic LDA method and some of its improved variants. We demonstrate that in some specific situations the standard multi-class LDA almost totally fails to find a discriminative subspace if the proposed relevance weights are not incorporated. In order to estimate the relevance weights of individual within-class scatter matrices, we propose several methods of which one employs the evolution strategies.

68T10 Pattern recognition, speech recognition
Full Text: DOI
[1] Fukunaga, K., Introduction to statistical pattern recognition, (1990), Academic Press New York · Zbl 0711.62052
[2] Jin, Z.; Yang, J.Y.; Hu, Z.S.; Lou, Z., Face recognition based on the uncorrelated discriminant transformation, Pattern recognition, 34, 1405-1416, (2001) · Zbl 0978.68118
[3] El-Feghi, I.; Sid-Ahmed, M.A.; Ahmadi, M., Automatic localization of craniofacial landmarks for assisted cephalometry, Pattern recognition, 37, 609-621, (2004)
[4] Petridis, S.; Perantonis, S.J., On the relation between discriminant analysis and mutual information for supervised linear feature extraction, Pattern recognition, 37, 857-874, (2004) · Zbl 1072.68553
[5] Kumar, N.; Andreou, A.G., Heteroscedastic discriminant analysis and reduced rank HMMS for improved speech recognition, Speech comm., 26, 283-297, (1998)
[6] Loog, M.; Duin, R.P.W., Linear dimensionality reduction via a heteroscedastic extension of ldathe Chernoff criterion, IEEE trans. pattern anal. Mach. intell., 26, 732-739, (2004)
[7] Hastie, T.; Tibshirani, R., Discriminant analysis by Gaussian mixtures, J. roy. statist. soc. B, 58, 155-176, (1996) · Zbl 0850.62476
[8] Hoffbeck, J.P.; Landgrebe, D.A., Covariance matrix estimation and classification with limited training data, IEEE trans. pattern anal. Mach. intell., 18, 763-767, (1996)
[9] Friedman, J.H., Regularized discriminant analysis, J. am. statist. assoc., 84, 165-175, (1989)
[10] Yu, H.; Yang, J., A direct LDA algorithm for high-dimensional data—with application to face recognition, Pattern recognition, 34, 2067-2070, (2001) · Zbl 0993.68091
[11] Liu, C.; Wechsler, H., Robust coding schemes for indexing and retrieval from large face databases, IEEE trans. on image processing, 9, 132-137, (2000)
[12] Loog, M.; Duin, R.P.W.; Haeb-Umbach, R., Multiclass linear dimension reduction by weighted pairwise Fisher criteria, IEEE trans. pattern anal. Mach. intell., 23, 762-766, (2001)
[13] Lotlikar, R.; Kothari, R., Fractional-step dimensionality reduction, IEEE trans. pattern anal. Mach. intell., 22, 623-627, (2000)
[14] Back, T.; Schwefel, H.P., An overview of evolutionary algorithms for parameter optimization, Evol. comput., 1, 1-23, (1993)
[15] Kim, D.; Bang, S.-Y., A handwritten numeral character classification using tolerant rough set, IEEE trans. pattern anal. Mach. intell., 22, 923-937, (2000)
[16] Liu, C.; Wechsler, H., Evolutionary pursuit and its application to face recognition, IEEE trans. pattern anal. Mach. intell., 22, 570-582, (2000)
[17] Raymer, M.L.; Punch, W.F.; Goodman, E.D.; Kuhn, L.A.; Jain, A.K., Dimensionality reduction using genetic algorithms, IEEE trans. on evol. comput., 4, 164-171, (2000)
[18] Rizki, M.M.; Zmuda, M.A.; Tamburino, L.A., Evolving pattern recognition systems, IEEE trans. evol. comput., 6, 594-609, (2002)
[19] M. Kotani, M. Nakai, K. Akazawa, Feature extraction using evolutionary computation, Proceedings of the 1999 Congress on Evolutionary Computation, 1999.
[20] Liu, Y.; Yao, X.; Higuchi, T., Evolutionary ensembles with negative correlation learning, IEEE trans. evol. comput., 4, 380-387, (2000)
[21] Khoo, K.G.; Suganthan, P.N., Structural pattern recognition using genetic algorithms with specialized operators, IEEE trans. systems man cybernet. part B, 33, 1, 156-165, (2003)
[22] C.L. Blake, C.J. Merz, UCI Repository of Machine Learning Databases, University of California, Irvine, Department of Information and Computer Sciences, 1996, http://www.ics.uci.edu/\(\sim\)mlearn/MLRepository.html.
[23] Fisher, R.A., The statistical utilization of multiple measurements, Ann. eugenics, 8, 376-386, (1938) · Zbl 0019.35703
[24] Rao, C.R., The utilization of multiple measurements in problems of biological classification, J. roy. statist. soc. B, 10, 159-203, (1948) · Zbl 0034.07902
[25] Yao, X.; Liu, Y., Fast evolution strategies, Control cybernet., 26, 467-496, (1997) · Zbl 0900.93354
[26] Islam, M.M.; Yao, X.; Murase, K., A constructive algorithm for training cooperative neural network ensembles, IEEE trans. neural networks, 14, 820-834, (2003)
[27] Beyer, H.-G.; Deb, K., On self-adaptive features in real-parameter evolutionary algorithms, IEEE trans. evol. comput., 5, 250-270, (2001)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.