×

Manifold adaptive kernelized low-rank representation for semisupervised image classification. (English) Zbl 1390.68738

Summary: Constructing a powerful graph that can effectively depict the intrinsic connection of data points is the critical step to make the graph-based semisupervised learning algorithms achieve promising performance. Among popular graph construction algorithms, Low-Rank Representation (LRR) is a very competitive one that can simultaneously explore the global structure of data and recover the data from noisy environments. Therefore, the learned low-rank coefficient matrix in LRR can be used to construct the data affinity matrix. Consider the existing problems such as the following: (1) the essentially linear property of LRR makes it not appropriate to process the possible nonlinear structure of data and (2) learning performance can be greatly enhanced by exploring the structure information of data; we propose a new Manifold Kernelized Low-Rank Representation (MKLRR) model that can perform LRR in the data manifold adaptive kernel space. Specifically, the manifold structure can be incorporated into the kernel space by using graph Laplacian and thus the underlying geometry of data is reflected by the wrapped kernel space. Experimental results of semisupervised image classification tasks show the effectiveness of MKLRR. For example, MKLRR can, respectively, obtain 96.13%, 98.09%, and 96.08% accuracies on ORL, Extended Yale B, and PIE data sets when given 5, 20, and 20 labeled face images per subject.

MSC:

68U10 Computing methodologies for image processing
62H30 Classification and discrimination; cluster analysis (statistical aspects)
94A08 Image processing (compression, reconstruction, etc.) in information and communication theory

Software:

l1_ls
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Zhu, X.; Ghahramani, Z.; Lafferty, J., Semi-supervised learning using gaussian fields and harmonic functions, Proceedings of the 20th International Conference on Machine Learning (ICML ’03)
[2] Zhu, X., Semi-supervised learning literature survey, Computer Sciences Technical Report, Madison, WI, USA: University of Wisconsin, Madison, WI, USA
[3] Cai, D.; He, X.; Han, J., Semi-supervised discriminant analysis, Proceedings of the 11th IEEE International Conference on Computer Vision (ICCV ’07) · doi:10.1109/ICCV.2007.4408856
[4] Zhou, Z.-H.; Li, M., Semi-supervised learning by disagreement, Knowledge and Information Systems, 24, 3, 415-439, (2010) · doi:10.1007/s10115-009-0209-z
[5] Wang, Y.; Chen, S., Safety-aware semi-supervised classification, IEEE Transactions on Neural Networks and Learning Systems, 24, 11, 1763-1772, (2013) · doi:10.1109/TNNLS.2013.2263512
[6] Jiang, Z.; Zhang, S.; Zeng, J., A hybrid generative/discriminative method for semi-supervised classification, Knowledge-Based Systems, 37, 137-145, (2013) · doi:10.1016/j.knosys.2012.07.020
[7] Forestier, G.; Wemmert, C., Semi-supervised learning using multiple clusterings with limited labeled data, Information Sciences, 361-362, 48-65, (2016) · doi:10.1016/j.ins.2016.04.040
[8] Yan, S.; Wang, H., Semi-supervised learning by sparse representation, Proceedings of the SIAM International Conference on Data Mining
[9] Zhuang, L.; Gao, H.; Lin, Z.; Ma, Y.; Zhang, X.; Yu, N., Non-negative low rank and sparse graph for semi-supervised learning, Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR ’12) · doi:10.1109/cvpr.2012.6247944
[10] Zheng, Y.; Zhang, X.; Yang, S.; Jiao, L., Low-rank representation with local constraint for graph construction, Neurocomputing, 122, 398-405, (2013) · doi:10.1016/j.neucom.2013.06.013
[11] Yang, S.; Feng, Z.; Ren, Y.; Liu, H.; Jiao, L., Semi-supervised classification via kernel low-rank representation graph, Knowledge-Based Systems, 69, 1, 150-158, (2014) · doi:10.1016/j.knosys.2014.06.007
[12] Wright, J.; Ma, Y.; Mairal, J.; Sapiro, G.; Huang, T. S.; Yan, S., Sparse representation for computer vision and pattern recognition, Proceedings of the IEEE, 98, 6, 1031-1044, (2010) · doi:10.1109/JPROC.2010.2044470
[13] Peng, Y.; Lu, B.-L., Robust structured sparse representation via half-quadratic optimization for face recognition, Multimedia Tools and Applications, 76, 6, 8859-8880, (2017) · doi:10.1007/s11042-016-3510-3
[14] Peng, Y.; Lu, B.-L., Discriminative extreme learning machine with supervised sparsity preserving for image classification, Neurocomputing, 261, 242-252, (2017) · doi:10.1016/j.neucom.2016.05.113
[15] Liu, G.; Lin, Z.; Yu, Y., Robust subspace segmentation by low-rank representation, Proceedings of the 27th International Conference on Machine Learning (ICML ’10)
[16] Jiang, W.; Liu, J.; Qi, H.; Dai, Q., Robust subspace segmentation via nonconvex low rank representation, Information Sciences, 340/341, 144-158, (2016) · Zbl 1395.68233 · doi:10.1016/j.ins.2015.12.038
[17] Lu, X.; Wang, Y.; Yuan, Y., Graph-regularized low-rank representation for destriping of hyperspectral images, IEEE Transactions on Geoscience and Remote Sensing, 51, 7, 4009-4018, (2013) · Zbl 1395.90100 · doi:10.1109/tgrs.2012.2226730
[18] Peng, Y.; Lu, B.-L.; Wang, S., Enhanced low-rank representation via sparse manifold adaption for semi-supervised learning, Neural Networks, 65, 1-17, (2015) · Zbl 1394.68307 · doi:10.1016/j.neunet.2015.01.001
[19] Peng, Y.; Long, X.; Lu, B.-L., Graph Based Semi-Supervised Learning via Structure Preserving Low-Rank Representation, Neural Processing Letters, 41, 3, 389-406, (2015) · doi:10.1007/s11063-014-9396-z
[20] Li, S.; Fu, Y., Low-rank coding with b-matching constraint for semi-supervised classification, Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI ’13)
[21] Nguyen, H.; Yang, W.; Shen, F.; Sun, C., Kernel Low-Rank Representation for face recognition, Neurocomputing, 155, 32-42, (2015) · doi:10.1016/j.neucom.2014.12.051
[22] Zheng, M.; Bu, J.; Chen, C.; Wang, C.; Zhang, L.; Qiu, G.; Cai, D., Graph regularized sparse coding for image representation, IEEE Transactions on Image Processing, 20, 5, 1327-1336, (2011) · Zbl 1372.94314 · doi:10.1109/TIP.2010.2090535
[23] Cai, D.; He, X.; Han, J.; Huang, T. S., Graph regularized nonnegative matrix factorization for data representation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 33, 8, 1548-1560, (2011) · doi:10.1109/TPAMI.2010.231
[24] Cai, D.; He, X.; Han, J., Locally consistent concept factorization for document clustering, IEEE Transactions on Knowledge and Data Engineering, 23, 6, 902-913, (2011) · doi:10.1109/TKDE.2010.165
[25] He, X. F.; Cai, D.; Shao, Y. L.; Bao, H.; Han, J., Laplacian regularized Gaussian mixture model for data clustering, IEEE Transactions on Knowledge and Data Engineering, 23, 9, 1406-1418, (2011) · doi:10.1109/tkde.2010.259
[26] Peng, Y.; Wang, S.; Long, X.; Lu, B.-L., Discriminative graph regularized extreme learning machine and its application to face recognition, Neurocomputing, 340-353, (2015) · doi:10.1016/j.neucom.2013.12.065
[27] Hadsell, R.; Chopra, S.; LeCun, Y., Dimensionality reduction by learning an invariant mapping, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR ’06) · doi:10.1109/cvpr.2006.100
[28] Sindhwani, V.; Niyogi, P.; Belkin, M., Beyond the point cloud: from transductive to semi-supervised learning, Proceedings of the 22nd International Conference on Machine Learning (ICML ’05)
[29] Candès, E. J.; Li, X.; Ma, Y.; Wright, J., Robust principal component analysis?, Journal of the ACM, 58, 3, (2011) · Zbl 1327.62369 · doi:10.1145/1970392.1970395
[30] Cai, J.-F.; Candès, E. J.; Shen, Z., A singular value thresholding algorithm for matrix completion, SIAM Journal on Optimization, 20, 4, 1956-1982, (2010) · Zbl 1201.90155 · doi:10.1137/080738970
[31] Lin, Z.; Chen, M.; Ma, Y., The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices
[32] Peng, Y.; Wang, S.; Lu, B.-L., Structure preserving low-rank representation for semi-supervised face recognition, Proceedings of the International Conference on Neural Information Processing
[33] Schölkopf, B.; Smola, A. J., Learning with kernels, (2002), The MIT Press
[34] Belkin, M.; Sindhwani, V.; Niyogi, P., Manifold regularization: a geometric framework for learning from labeled and unlabeled examples, Journal of Machine Learning Research, 7, 2399-2434, (2006) · Zbl 1222.68144
[35] Niyogi, P.; Smale, S.; Weinberger, S., Finding the homology of submanifolds with high confidence from random samples, Discrete & Computational Geometry, 39, 1-3, 419-441, (2008) · Zbl 1148.68048 · doi:10.1007/s00454-008-9053-2
[36] Chung, F. R., Spectral graph theory, 92, (1997), AMS Bookstore · Zbl 0867.05046
[37] Liu, G.; Lin, Z.; Yan, S.; Sun, J.; Yu, Y.; Ma, Y., Robust recovery of subspace structures by low-rank representation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 35, 1, 171-184, (2013) · doi:10.1109/TPAMI.2012.88
[38] Cheng, B.; Yang, J.; Yan, S.; Fu, Y.; Huang, T. S., Learning with-graph for image analysis, IEEE Transactions on Image Processing, 19, 4, 858-866, (2010) · Zbl 1371.68229 · doi:10.1109/TIP.2009.2038764
[39] Koh, K.; Kim, S.; Boyd, S., l1_ls: A Matlab Solver for Large-Scale l1-Regularized Least Squares Problems, Stanford University
[40] Wang, F.; Zhang, C., Label propagation through linear neighborhoods, IEEE Transactions on Knowledge and Data Engineering, 20, 1, 55-67, (2008) · doi:10.1109/tkde.2007.190672
[41] He, R.; Zheng, W.-S.; Hu, B.-G.; Kong, X.-W., Nonnegative sparse coding for discriminative semi-supervised learning, Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2011 · doi:10.1109/CVPR.2011.5995487
[42] Ji, S.; Ye, J., An accelerated gradient method for trace norm minimization, Proceedings of the 26th International Conference On Machine Learning (ICML ’09), ACM
[43] Cai, D.; He, X., Manifold adaptive experimental design for text categorization, IEEE Transactions on Knowledge and Data Engineering, 24, 4, 707-719, (2012) · doi:10.1109/TKDE.2011.104
[44] Hull, J. J., A database for handwritten text recognition research, IEEE Transactions on Pattern Analysis and Machine Intelligence, 16, 5, 550-554, (1994) · doi:10.1109/34.291440
[45] Li, P.; Bu, J.; Zhang, L.; Chen, C., Graph-based local concept coordinate factorization, Knowledge and Information Systems, 43, 1, 103-126, (2015) · doi:10.1007/s10115-013-0715-x
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.