Wu, Yiming; Liu, Xiuwen; Mio, Washington Learning representations for object classification using multi-stage optimal component analysis. (English) Zbl 1254.68231 Neural Netw. 21, No. 2-3, 214-221 (2008). Summary: Learning data representations is a fundamental challenge in modeling neural processes and plays an important role in applications such as object recognition. Optimal component analysis (OCA) formulates the problem in the framework of optimization on a Grassmann manifold and a stochastic gradient method is used to estimate the optimal basis. OCA has been successfully applied to image classification problems arising in a variety of contexts. However, as the search space is typically very high dimensional, OCA optimization often requires expensive computational cost. In multi-stage OCA, we first hierarchically project the data onto several low-dimensional subspaces using standard techniques, then OCA learning is performed hierarchically from the lowest to the highest levels to learn about a subspace that is optimal for data discrimination based on the K-nearest neighbor classifier. One of the main advantages of multi-stage OCA lies in the fact that it greatly improves the computational efficiency of the OCA learning algorithm without sacrificing the recognition performance, thus enhancing its applicability to practical problems. In addition to the nearest neighbor classifier, we illustrate the effectiveness of the learned representations on object classification used in conjunction with classifiers such as neural networks and support vector machines. Cited in 1 Document MSC: 68T05 Learning and adaptive systems in artificial intelligence 62H30 Classification and discrimination; cluster analysis (statistical aspects) 68T10 Pattern recognition, speech recognition Keywords:dimension reduction; linear data representations; feature selection; object recognition; object classification; optimal component analysis Software:AR face; COIL-100; CMU PIE PDFBibTeX XMLCite \textit{Y. Wu} et al., Neural Netw. 21, No. 2--3, 214--221 (2008; Zbl 1254.68231) Full Text: DOI References: [1] Anderson, T. W., An introduction to multivariate statistical analysis (2003), John Wiley & Sons · Zbl 0083.14601 [2] Bartlett, M. S.; Movellen, J. R.; Sejnowski, T. J., Face recognition by independent component analysis, IEEE Transactions on Neural Networks, 13, 1450-1464 (2002) [3] Belhumeour, P. N.; Hespanha, J. P.; Kriegman, D. J., Eigenfaces vs. fisherfaces: Recognition using class specific linear projection, IEEE Transactions on Pattern Analysis and Machine Intelligence, 19, 711-720 (1997) [4] Bishop, C. M., Neural networks for pattern recognition (1995), Oxford University Press [5] Comon, P., Independent component analysis, a new concept?, Signal Processing, 36, 287-314 (1994) · Zbl 0791.62004 [6] Duda, R. O.; Hart, P. E.; Stock, D., Pattern classification (2000), John Wiley & Sons [7] Geman, S.; Bienenstock, E., Neural network and the bias/variance dilemma, Neural Computations, 4, 1-58 (1992) [8] Eleyan, A.; Demirel, H., Face recognition system based on PCA and feedforward neural networks, Computational Intelligence and Bioinspired Systems, 3512, 935-942 (2005) [9] Etemad, K.; Chellappa, R., Discriminant analysis for recognition of human face images, Journal of the Optical Society of America A: Optics, Image Science, and Vision, 14, 1724-1733 (1997) [10] Forsyth, D. A.; Ponce, J., Computer vision, a modern approach (2003), Prentice Hall, pp. 315-317 [11] Hyvarinen, A.; Karhunen, J.; Oja, E., Independent component analysis (2001), John Wiley & Sons [12] Jolliffe, I. T., Principal component analysis (1986), Springer-Verlag · Zbl 1011.62064 [13] Kwak, K. C.; Pedrycz, W., Face recognition using an enhanced independent component analysis approach, IEEE Transactions on Neural Networks, 18, 530-541 (2007) [14] Liu, X., & Mio, W. (2005). Kernel methods for nonlinear discriminative analysis. In Proceedings of the international workshop on energy minimization methods in computer vision and pattern recognition; Liu, X., & Mio, W. (2005). Kernel methods for nonlinear discriminative analysis. In Proceedings of the international workshop on energy minimization methods in computer vision and pattern recognition [15] Liu, X.; Srivastava, A.; Gallivan, K. A., Optimal linear representation of images for object recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, 26, 662-666 (2004) [16] Lu, X., & Jain, A.K. (2003). Resampling for face recognition. In AVBPA03; Lu, X., & Jain, A.K. (2003). Resampling for face recognition. In AVBPA03 · Zbl 1050.68816 [17] MacQueen, J. B., Some methods for classification and analysis of multivariate observations, (Proceedings of 5-th berkeley symposium on mathematical statistics and probability (1967), University of California Press: University of California Press Berkeley), 281-297 [18] Martinez, A.M., & Benavente, R. (1998). The AR face database. CRC Technical Report #24. 1998; Martinez, A.M., & Benavente, R. (1998). The AR face database. CRC Technical Report #24. 1998 [19] Martinez, A. M.; Kak, A. C., PCA versus LDA, IEEE Transactions on Pattern Analysis and Machine Intelligence, 23, 228-233 (2001) [20] Nayar, S.K., Nene, S.A., & Murase, H. (1996). Columbia object image library (COIL-100). Tech. Rep. CUCS-006-96; Nayar, S.K., Nene, S.A., & Murase, H. (1996). Columbia object image library (COIL-100). Tech. Rep. CUCS-006-96 [21] Reiter, H., Donner, R., Langs, G., & Bischof, H. (2006). Estimation of face depth maps from color textures using canonical correlation analysis. In Computer Vision Winter Workshop (pp. 6-8); Reiter, H., Donner, R., Langs, G., & Bischof, H. (2006). Estimation of face depth maps from color textures using canonical correlation analysis. In Computer Vision Winter Workshop (pp. 6-8) [22] Roth, D.; Yang, M.; Ahuja, N., Learning to recognition objects, Neural Computation, 14, 1071-1104 (2002) [23] Rumelhart, D. E.; Hinton, G. E.; Williams, R. J., Learning internal representations by backpropagation errors, Nature, 323, 533-536 (1986) · Zbl 1369.68284 [24] Samaria, F.; Harter, A., Parameterization of a stochastic model for human face identification, (Proceedings of 2nd IEEE workshop on applications of computer vision (1994), Sarasota: Sarasota FL) [25] Schölkopf, B.; Burges, C. J.C.; Smola, A. J., Advances in kernel methods: Support vector machine (1998), The MIT Press · Zbl 0935.68084 [26] Sim, T., Simon, B., & Bsat, M. (2001). The CMU pose, illumination, and expression (PIE) database of human faces. Tech. Report, CMU-RI-TR-01-02. Pittsburgh, PA. Robotics Institute, Carnegie Mellon University; Sim, T., Simon, B., & Bsat, M. (2001). The CMU pose, illumination, and expression (PIE) database of human faces. Tech. Report, CMU-RI-TR-01-02. Pittsburgh, PA. Robotics Institute, Carnegie Mellon University [27] Srivastava, A.; Liu, X., Tools for application-driven dimensional reduction, Neuro Computation, 67, 136-160 (2005) [28] Srivastava, A.; Liu, X.; Grenander, U., Universal analytical forms for modeling image probabilities, IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 1200-1214 (2002) [29] Turk, M.; Pentland, A., Eigenfaces for recognition, Journal of Cognitive Neuro-science, 3, 71-86 (1991) [30] Vapnik, V., The nature of statistical learning theory (1995), Springer-Verlag · Zbl 0833.62008 [31] Wu, Y., Liu, X., Mio, W., & Gallivan, K.A. (2007). Two-stage optimal component analysis via dimensional reduction. Computer Vision and Image Understanding (in press); Wu, Y., Liu, X., Mio, W., & Gallivan, K.A. (2007). Two-stage optimal component analysis via dimensional reduction. Computer Vision and Image Understanding (in press) [32] Wu, Y., Liu, X., & Mio, W. (2007). Multi-stage optimal component analysis. In Proceedings of the international joint conference on neural network; Wu, Y., Liu, X., & Mio, W. (2007). Multi-stage optimal component analysis. In Proceedings of the international joint conference on neural network · Zbl 1254.68231 [33] Zhang, Q., & Liu, X. (2003). Hierarchical learning of optimal linear representations. In Proceedings of the international joint conference on neural networks; Zhang, Q., & Liu, X. (2003). Hierarchical learning of optimal linear representations. In Proceedings of the international joint conference on neural networks [34] Zhao, W., Chellappa, R., & Phillips, P. (1994). Subspace linear discriminant analysis for face recognition. Center for Automation Research. University of Maryland. College Park. Technical Report CAR-TR-914; Zhao, W., Chellappa, R., & Phillips, P. (1994). Subspace linear discriminant analysis for face recognition. Center for Automation Research. University of Maryland. College Park. Technical Report CAR-TR-914 [35] Zhu, J., & Yu, Y.L. (1994). Face recognition with eigenfaces. In Proceeding of the IEEE international conference on industrial technology; Zhu, J., & Yu, Y.L. (1994). Face recognition with eigenfaces. In Proceeding of the IEEE international conference on industrial technology This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.