×

Joint distance and similarity measure learning based on triplet-based constraints. (English) Zbl 1429.68229

Summary: Distance and similarity measures usually are complementary to pattern classification. With pairwise constraints, several approaches have been proposed to combine distance and similarity measures. However, it remains less investigated to use triplets of samples for joint learning of distance and similarity measures. Moreover, the kernel extension of triplet-based model is also nontrivial and computationally expensive. In this paper, we propose a novel method to learn a combined distance and similarity measure (CDSM). By incorporating with the max-margin model, we suggest a triplet-based CDSM learning model with a unified regularizer of the Frobenius norm. A support vector machine (SVM)-based algorithm is then adopted to solve the optimization problem. Furthermore, we extend CDSM for learning nonlinear measures via the kernel trick. Two effective strategies are adopted to speed up training and testing of kernelized CDSM. Experiments on the UCI, handwritten digits and person re-identification datasets demonstrate that CDSM and kernelized CDSM outperform several state-of-the-art metric learning methods.

MSC:

68T05 Learning and adaptive systems in artificial intelligence

Software:

LMNN
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] An, L.; Chen, X.; Yang, S.; Bhanu, B., Sparse representation matching for person re-identification, Inf. Sci. (Ny), 355-356, 74-89 (2016)
[2] Bellet, A.; Habrard, A.; Sebban, M., A survey on metric learning for feature vectors and structured data, arXiv preprint arXiv:1306.6709 (2013)
[3] Biehl, M.; Hammer, B.; Schneider, P.; Villmann, T., Metric Learning for Prototype-based Classification, Innovations in Neural Information Paradigms and Applications, 183-199 (2009), Springer
[4] Brunner, C.; Fischer, A.; Luig, K.; Thies, T., Pairwise support vector machines and their application to large scale problems, J. Mach. Learn. Res., 13, 1, 2279-2292 (2012) · Zbl 1433.68329
[5] Cao, Q.; Ying, Y.; Li, P., Similarity metric learning for face recognition, Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2408-2415 (2013), IEEE
[6] Chechik, G.; Shalit, U.; Sharma, V.; Bengio, S., An online algorithm for large scale image similarity learning, Advances in Neural Information Processing Systems (NIPS), 306-314 (2009)
[7] Chen, D.; Cao, X.; Wang, L.; Wen, F.; Sun, J., Bayesian Face Revisited: A Joint Formulation, European Conference on Computer Vision (ECCV), 566-579 (2012), Springer
[8] Davis, J. V.; Kulis, B.; Jain, P.; Sra, S.; Dhillon, I. S., Information-theoretic metric learning, Proceedings of the 24th International Conference on Machine Learning (ICML), 209-216 (2007), ACM
[9] France, S. L.; Carroll, J. D.; Xiong, H., Distance metrics for high dimensional nearest neighborhood recovery: compression and normalization, Inf. Sci. (Ny), 184, 1, 92-110 (2012)
[10] Gani, A.; Siddiqa, A.; Shamshirband, S.; Hanum, F., A survey on indexing techniques for big data: taxonomy and performance evaluation, Knowl. Inf. Syst., 46, 2, 241-284 (2016)
[11] Goldberger, J.; Hinton, G. E.; Roweis, S. T.; Salakhutdinov, R., Neighbourhood components analysis, Advances in Neural Information Processing Systems (NIPS), 513-520 (2004)
[12] Gray, D.; Brennan, S.; Tao, H., Evaluating appearance models for recognition, reacquisition, and tracking, Proc. IEEE International Workshop on Performance Evaluation for Tracking and Surveillance (PETS), 3 (2007), Citeseer
[13] Guillaumin, M.; Verbeek, J.; Schmid, C., Is that you? metric learning approaches for face identification, Proceedings of the IEEE International Conference on Computer Vision (ICCV), 498-505 (2009), IEEE
[14] Hoi, S. C.; Liu, W.; Chang, S.-F., Semi-supervised distance metric learning for collaborative image retrieval, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1-7 (2008), IEEE
[15] Javanmardi, S.; Shojafar, M.; Shariatmadari, S.; Ahrabi, S. S., Fr trust: a fuzzy reputation-based model for trust management in semantic p2p grids, Int. J. Grid Util. Comput., 6, 1, 57-66 (2014)
[16] Kato, T.; Nagano, N., Metric learning for enzyme active-site search, Bioinformatics, 26, 21, 2698-2704 (2010)
[17] Koestinger, M.; Hirzer, M.; Wohlhart, P.; Roth, P. M.; Bischof, H., Large scale metric learning from equivalence constraints, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2288-2295 (2012), IEEE
[18] Kostinger, M.; Wohlhart, P.; Roth, P. M.; Bischof, H., Joint learning of discriminative prototypes and large margin nearest neighbor classifiers, Proceedings of the IEEE International Conference on Computer Vision (ICCV), 3112-3119 (2013), IEEE
[19] Li, Z.; Chang, S.; Liang, F.; Huang, T.; Cao, L.; Smith, J., Learning locally-adaptive decision functions for person verification, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3610-3617 (2013), IEEE
[20] Liao, S.; Hu, Y.; Zhu, X.; Li, S. Z., Person re-identification by local maximal occurrence representation and metric learning, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2197-2206 (2015), IEEE
[21] Maji, P.; Shah, E.; Paul, S., Relsim: an integrated method to identify disease genes using gene expression profiles and {PPIN} based similarity measure, Inf. Sci. (Ny), 384, 110-125 (2017)
[22] Paredes, R.; Vidal, E., Learning prototypes and distances: a prototype reduction technique based on nearest neighbor error minimization, Pattern Recognit., 39, 2, 180-188 (2006) · Zbl 1080.68647
[23] Peng, X.; Xu, D., Twin mahalanobis distance-based support vector machines for pattern recognition, Inf. Sci. (Ny), 200, 22-37 (2012) · Zbl 1248.68416
[24] Qamar, A. M.; Gaussier, E., Online and batch learning of generalized cosine similarities, Ninth IEEE International Conference on Data Mining (ICDM), 926-931 (2009), IEEE
[25] Qamar, A. M.; Gaussier, E.; Chevallet, J.-P.; Lim, J. H., Similarity learning for nearest neighbor classification, Proceedings of the Eighth IEEE International Conference on Data Mining (ICDM), 983-988 (2008), IEEE
[26] Reitmaier, T.; Sick, B., The responsibility weighted mahalanobis kernel for semi-supervised training of support vector machines for classification, Inf. Sci. (Ny), 323, 179-198 (2015)
[27] Shamshirband, S.; Anuar, N. B.; Kiah, M. L.M.; Rohani, V. A.; Petković, D.; Misra, S.; Khan, A. N., Co-fais: cooperative fuzzy artificial immune system for detecting intrusion in wireless sensor networks, Journal of Network and Computer Applications, 42, 102-117 (2014)
[28] Shamshirband, S.; Mohammadi, K.; Tong, C. W.; Zamani, M.; Motamedi, S.; Ch, S., A hybrid svm-ffa method for prediction of monthly mean global solar radiation, Theor. Appl. Climatol., 1-13 (2015)
[29] Shaw, B.; Huang, B.; Jebara, T., Learning a distance metric from a network, Advances in Neural Information Processing Systems (NIPS), 1899-1907 (2011)
[30] Shen, C.; Kim, J.; Wang, L., A scalable dual approach to semidefinite metric learning, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2601-2608 (2011), IEEE
[31] Shen, C.; Kim, J.; Wang, L.; Van Den Hengel, A., Positive semidefinite metric learning using boosting-like algorithms, J. Mach. Learn. Res., 13, 1, 1007-1036 (2012) · Zbl 1283.68303
[32] Tian, Q.; Chen, S.; Qiao, L., Ordinal margin metric learning and its extension for cross-distribution image data, Inf. Sci. (Ny), 349, 50-64 (2016) · Zbl 1398.68461
[33] Tran, D.; Sorokin, A., Human Activity Recognition with Metric Learning, European Conference on Computer Vision (ECCV), 548-561 (2008), Springer
[34] Verma, N.; Mahajan, D.; Sellamanickam, S.; Nair, V., Learning hierarchical similarity metrics, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2280-2287 (2012), IEEE
[35] Vert, J.-P.; Qiu, J.; Noble, W. S., A new pairwise kernel for biological network inference with support vector machines, BMC Bioinformatics, 8, Suppl 10, S8 (2007)
[36] Wang, F.; Zuo, W.; Zhang, L.; Meng, D.; Zhang, D., A kernel classification framework for metric learning, IEEE Trans. Neural Netw. Learn Syst., 26, 9, 1950-1962 (2015)
[37] Wang, J.; Gao, X.; Wang, Q.; Li, Y., Prodis-contshc: learning protein dissimilarity measures and hierarchical context coherently for protein-protein comparison in protein database retrieval, BMC Bioinf., 13, 7, 1 (2012)
[38] Wang, J.; Kalousis, A.; Woznica, A., Parametric local metric learning for nearest neighbor classification, Advances in Neural Information Processing Systems (NIPS), 1601-1609 (2012)
[39] Weinberger, K. Q.; Blitzer, J.; Saul, L. K., Distance metric learning for large margin nearest neighbor classification, Advances in Neural Information Processing Systems (NIPS), 1473-1480 (2005)
[40] Weinberger, K. Q.; Saul, L. K., Fast solvers and efficient implementations for distance metric learning, Proceedings of the 25th international conference on Machine learning (ICML), 1160-1167 (2008), ACM
[41] Weinberger, K. Q.; Saul, L. K., Distance metric learning for large margin nearest neighbor classification, The Journal of Machine Learning Research, 10, 207-244 (2009) · Zbl 1235.68204
[42] Xu, J.; Yang, J.; Lai, Z., K-Local hyperplane distance nearest neighbor classifier oriented local discriminant analysis, Inf. Sci. (Ny), 232, 11-26 (2013) · Zbl 1293.68248
[43] Ying, Y.; Li, P., Distance metric learning with eigenvalue optimization, J. Mach. Learn. Res., 13, 1, 1-26 (2012) · Zbl 1283.68309
[44] Yoon, S.-H.; Kim, S.-W.; Park, S., C-Rank: a link-based similarity measure for scientific literature databases, Inf. Sci. (Ny), 326, 25-40 (2016)
[45] Yu, D.; Yu, X.; Hu, Q.; Liu, J.; Wu, A., Dynamic time warping constraint learning for large margin nearest neighbor classification, Inf. Sci. (Ny), 181, 13, 2787-2796 (2011)
[46] Yu, J.; Tao, D.; Li, J.; Cheng, J., Semantic preserving distance metric learning and applications, Inf. Sci. (Ny), 281, 674-686 (2014)
[47] Zhu, P.; Hu, Q.; Zuo, W.; Yang, M., Multi-granularity distance metric learning via neighborhood granule margin maximization, Inf. Sci. (Ny), 282, 321-331 (2014)
[48] Zhu, P.; Zhang, L.; Zuo, W.; Zhang, D., From point to set: Extend the learning of distance metrics, Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2664-2671 (2013), IEEE
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.