×

A robust classification framework with mixture correntropy. (English) Zbl 1454.62202

Summary: In this paper, we define a mixture correntropy criterion where two different kernel functions are combined. We induce a more general nonconvex robust loss function by this heterogenous mixture correntropy. The proposed mixture correntropy is also a local similarity measure that not only improves the limitations of correntropy under a single kernel, but also handles heterogeneous data more flexibly and stably. The induced loss amalgamates the superiors of the state-of-the-art robust loss functions and is more effective. What’s more, we verify the Fisher consistency of the induced loss and analyze the robustness from the view point of robust estimation. With this induced loss, we propose a robust support vector machine (SVM) framework and adopt half quadratic optimization algorithm to handle the nonconvexity and further improve convergent rate. Furthermore, we generate heterogenous structured artificial datasets and impose different levels of label noise on benchmark datasets. Implements on these two types of datasets show the superior flexibility and effectiveness of the proposed framework.

MSC:

62H30 Classification and discrimination; cluster analysis (statistical aspects)
62B10 Statistical aspects of information-theoretic topics

Software:

SHOGUN; UCI-ml
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Blake, C.; Merz, C., Uci repository of machine learning databases (1998)
[2] Chen, B.; Principe, J. C., Maximum correntropy estimation is a smoothed map estimation, IEEE Signal Process. Lett., 19, 8, 491-494 (2012)
[3] Chen, B.; Wang, X.; Lu, N.; Wang, S.; Cao, J.; Qin, J., Mixture correntropy for robust learning, Pattern Recogn., 79, 318-327 (2018)
[4] Chen, B.; Xing, L.; Liang, J.; Zheng, N.; Principe, J. C., Steady-state mean-square error analysis for adaptive filtering under the maximum correntropy criterion, IEEE Signal Process. Lett., 21, 7, 880-884 (2014)
[5] Chen, B.; Xing, L.; Zhao, H.; Zheng, N.; Principe, J. C., Generalized correntropy for robust adaptive filtering, IEEE Trans. Signal Process., 64, 13, 3376-3387 (2016) · Zbl 1414.94113
[6] Chen, L.; Qu, H.; Zhao, J., Generalized correntropy based deep learning in presence of non-gaussian noises, Neurocomputing (2017)
[7] Cortes, C.; Vapnik, V., Support-vector networks, Mach. Learn., 20, 3, 273-297 (1995) · Zbl 0831.68098
[8] Cortes, C.; Vapnik, V., Soft Margin Classifier (1997)
[9] Du, B.; Tang, X.; Wang, Z.; Zhang, L.; Tao, D., Robust graph-based semisupervised learning for noisy labeled data via maximum correntropy criterion, IEEE Trans. Cybern., 99, 1-14 (2018)
[10] Du, B.; Wang, Z.; Zhang, L.; Zhang, L.; Tao, D., Robust and discriminative labeling for multi-label active learning based on maximum correntropy criterion, IEEE Trans. Image Process., 99 (2017) · Zbl 1409.94133
[11] Fawcett, T., An introduction to roc analysis, Pattern Recogn. Lett., 27, 8, 861-874 (2006)
[12] He, R.; Hu, B.; Zheng, W.; Kong, X., Robust principal component analysis based on maximum correntropy criterion, IEEE Trans. Image Process., 20, 6, 1485 (2011) · Zbl 1372.94369
[13] He, R.; Zheng, W.; Tan, T.; Sun, Z., Half-quadratic-based iterative minimization for robust sparse representation, IEEE Trans. Pattern Anal. Mach. Intell., 36, 2, 261-275 (2013)
[14] He, R.; Zheng, W. S.; Hu, B. G., Maximum correntropy criterion for robust face recognition, IEEE Trans. Pattern Anal. Mach. Intell., 33, 8, 1561-1576 (2010)
[15] Hearst, M. A.; Dumais, S. T.; Osman, E.; Platt, J.; Scholkopf, B., Support Vector Machines (1998), IEEE Educational Activities Department
[16] Kanamori, T.; Fujiwara, S.; Takeda, A., Robustness of learning algorithms using hinge loss with outlier indicators, Neural Netw., 94, 173-191 (2017) · Zbl 1429.68220
[17] Lin, Y., A note on margin-based loss functions in classification, Stat. Probab. Lett., 68, 1, 73-82 (2004) · Zbl 1058.62052
[18] Liu, W.; Pokharel, P. P.; Principe, J. C., Correntropy: a localized similarity measure, International Joint Conference on Neural Networks, 4919-4924 (2006)
[19] Liu, W.; Pokharel, P. P.; Principe, J. C., Error entropy, correntropy and m-estimation, Machine Learning for Signal Processing, 2006. Proceedings of the 2006 IEEE Signal Processing Society Workshop on, 179-184 (2006)
[20] Liu, W.; Pokharel, P. P.; Principe, J. C., Correntropy: Properties and applications in non-gaussian signal processing, IEEE Trans. Signal Process., 55, 11, 5286-5298 (2007) · Zbl 1390.94277
[21] Mandanas, F. D.; Kotropoulos, C. L., Robust multidimensional scaling using a maximum correntropy criterion, IEEE Trans. Signal Process., 65, 4, 919-932 (2016) · Zbl 1414.94400
[22] Santamaria, I.; Pokharel, P. P.; Principe, J. C., Generalized correlation function: definition, properties, and application to blind equalization, IEEE Trans. Signal Process., 54, 6, 2187-2197 (2006) · Zbl 1374.94594
[23] Singh, A.; Pokharel, R.; Principe, J., The c-loss function for pattern classification, Pattern Recogn., 47, 1, 441-453 (2014) · Zbl 1326.68253
[24] Singh, A.; Principe, J. C., Using correntropy as a cost function in linear adaptive filters, International Joint Conference on Neural Networks, pages 2950-2955 (2009)
[25] Singh, A.; Principe, J. C., A loss function for classification based on a robust similarity metric, International Joint Conference on Neural Networks, pages 1-6 (2010)
[26] Sonnenburg, S.; Rätsch, G.; Schäfer, C.; Schölkopf, B., Large scale multiple kernel learning, J. Mach. Learn. Res., 7, 1531-1565 (2006) · Zbl 1222.90072
[27] Suykens, J. A.K.; Vandewalle, J., Least squares support vector machine classifiers, Neural Process. Lett., 9, 3, 293-300 (1999)
[28] Xu, G.; Cao, Z.; Hu, B.; Principe, J. C., Robust support vector machines based on the rescaled hinge loss function, Pattern Recogn., 63, 139-148 (2016) · Zbl 1429.68242
[29] Yang, L.; Ren, Z.; Wang, Y.; Dong, H., A robust regression framework with laplace kernel-induced loss, Neural Comput., 1-26 (2017)
[30] Zhang, J.; Qiu, T., A robust correntropy based subspace tracking algorithm in impulsive noise environments, Digital Signal Process., 62, 168-175 (2017)
[31] Zhang, L.; Zhang, L.; Tao, D.; Huang, X., On combining multiple features for hyperspectral remote sensing image classification, IEEE Trans. Geosci.Remote Sens., 50, 3, 879-893 (2012)
[32] Zhang, L.; Zhang, Q.; Zhang, L.; Tao, D.; Huang, X.; Du, B., Ensemble manifold regularized sparse low-rank approximation for multiview feature embedding, Pattern Recogn., 48, 10, 3102-3112 (2015)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.