×

Latent common manifold learning with alternating diffusion: analysis and applications. (English) Zbl 1441.62241

The article proposes a mathematical model based on alternating diffusion that is extended to more than two sensors. The model captures the nonlinear manifold structure by preserving both the geometrical and topological structures, it does not reply on prior knowledge and it can integrate subtle patterns of the data and even multiple data sets that contain complementary information. The efficiency of the proposed method is demonstrated and validated on several applications, from toy problems to complex tasks.

MSC:

62M10 Time series, auto-correlation, regression, etc. in statistics (GARCH)
62R30 Statistics on manifolds
60J60 Diffusion processes
28-08 Computational methods for problems pertaining to measure and integration
62P10 Applications of statistics to biology and medical sciences; meta analysis
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Bach, F. R.; Jordan, M. I., Kernel independent component analysis, J. Mach. Learn. Res., 3, 1-48 (2003) · Zbl 1088.68689
[2] Bach, F. R.; Lanckriet, G. R.G.; Jordan, M. I., Multiple kernel learning, conic duality, and the SMO algorithm, (Proceedings of the Twenty-First International Conference on Machine Learning (2004), ACM), 6
[3] Bai, X.; Wang, B.; Yao, C.; Liu, W.; Tu, Z., Co-transduction for shape retrieval, IEEE Trans. Image Process., 21, 5, 2747-2757 (2012) · Zbl 1373.94034
[4] Belkin, M.; Niyogi, P., Laplacian eigenmaps for dimensionality reduction and data representation, Neural Comput., 15, 6, 1373-1396 (June 2003) · Zbl 1085.68119
[5] Boots, B.; Gordon, G. J., Two-manifold problems with applications to nonlinear system identification, (Proc. 29th Intl. Conf. on Machine Learning (ICML) (2012))
[6] Coifman, R. R.; Lafon, S., Diffusion maps, Appl. Comput. Harmon. Anal., 21, 1, 5-30 (2006) · Zbl 1095.68094
[7] Coifman, R. R.; Lafon, S., Geometric harmonics: a novel tool for multiscale out-of-sample extension of empirical functions, Appl. Comput. Harmon. Anal., 21, 1, 31-52 (2006) · Zbl 1095.68095
[8] Davenport, M.; Hegde, C.; Duarte, M. F.; Baraniuk, R. G., Joint manifolds for data fusion, IEEE Trans. Image Process., 19, 10, 2580-2594 (2010) · Zbl 1371.68304
[9] De Livera, A. M.; Hyndman, R. J.; Snyder, R. D., Forecasting time series with complex seasonal patterns using exponential smoothing, J. Amer. Statist. Assoc., 106, 496, 1513-1527 (2011) · Zbl 1234.62123
[10] de Sa, V. R., Spectral clustering with two views, (ICML Workshop on Learning with Multiple Views (2005))
[11] de Sa, V. R.; Gallagher, P. W.; Lewis, J. M.; Malave, V. L., Multi-view kernel construction, Mach. Learn., 79, 1-2, 47-71 (May 2010) · Zbl 1475.62185
[12] do Carmo, M. P.; Flaherty, F., Riemannian Geometry (1992), Birkhäuser: Birkhäuser Boston · Zbl 0752.53001
[13] D.L. Donoho, M. Gavish, I.M. Johnstone, Optimal shrinkage of eigenvalues in the spiked covariance model, arXiv e-prints, November 2013.; D.L. Donoho, M. Gavish, I.M. Johnstone, Optimal shrinkage of eigenvalues in the spiked covariance model, arXiv e-prints, November 2013. · Zbl 1403.62099
[14] Donoho, D. L.; Grimes, C., Hessian eigenmaps: new locally linear embedding techniques for high-dimensional data, Proc. Nat. Acad. Sci., 100, 5591-5596 (2003) · Zbl 1130.62337
[15] El Karoui, N., On information plus noise kernel random matrices, Ann. Statist., 38, 5, 3191-3216 (2010) · Zbl 1200.62056
[16] El Karoui, N.; Wu, H.-T., Connection graph Laplacian methods can be made robust to noise, Ann. Statist., 44, 1, 346-372 (2016) · Zbl 1350.60036
[17] Eynard, D.; Glashoff, K.; Bronstein, M. M.; Bronstein, A. M., Multimodal diffusion geometry by joint diagonalization of Laplacians (2012), preprint
[18] Fowlkes, C.; Belongie, S.; Chung, F.; Malik, J., Spectral grouping using the Nystrom method, IEEE Trans. Pattern Anal. Mach. Intell., 26, 2, 214-225 (2004)
[19] Gould, P. G.; Koehler, A. B.; Ord, J. K.; Snyder, R. D.; Hyndman, R. J.; Vahid-Araghi, F., Forecasting time series with multiple seasonal patterns, European J. Oper. Res., 191, 1, 207-222 (2008) · Zbl 1142.62070
[20] Harvey, A.; Koopman, S. J., Forecasting hourly electricity demand using time-varying splines, J. Amer. Statist. Assoc., 88, 424, 1228-1236 (1993)
[21] Harvey, A.; Koopman, S. J.; Riani, M., The modeling and seasonal adjustment of weekly observations, J. Bus. Econom. Statist., 15, 3, 354-368 (1997)
[22] Hotelling, H., Relations between two sets of variates, Biometrika, 28, 3/4, 321 (December 1936) · Zbl 0015.40705
[23] Huang, H.-C.; Chuang, Y.-Y.; Chen, C.-S., Affinity aggregation for spectral clustering, (2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012), IEEE), 773-780
[24] Keller, Y.; Coifman, R.; Lafon, S.; Zucker, S. W., Audio-visual group recognition using diffusion maps, IEEE Trans. Signal Process., 58, 1, 403-413 (January 2010) · Zbl 1392.94627
[25] Kumar, A.; Daumé, H., A co-training approach for multi-view spectral clustering, (Proceedings of the 28th International Conference on Machine Learning (ICML-11) (2011)), 393-400
[26] Lafon, S.; Keller, Y.; Coifman, R. R., Data fusion and multicue data matching by diffusion maps, IEEE Trans. Pattern Anal. Mach. Intell., 28, 11, 1784-1797 (2006)
[27] Lahat, D.; Adali, T.; Jutten, C., Multimodal data fusion: an overview of methods, challenges, and prospects, Proc. IEEE, 103, 9, 1449-1477 (2015)
[28] Lai, P. L.; Fyfe, C., Kernel and nonlinear canonical correlation analysis, Int. J. Neural Syst., 10, 05, 365-377 (2000)
[29] Lanckriet, G. R.G.; Cristianini, N.; Bartlett, P.; El Ghaoui, L.; Jordan, M. I., Learning the kernel matrix with semidefinite programming, J. Mach. Learn. Res., 5, 27-72 (2004) · Zbl 1222.68241
[30] Lederman, R. R.; Talmon, R., Learning the geometry of common latent variables using alternating-diffusion, Appl. Comput. Harmon. Anal., 44, 3, 509-536 (2018) · Zbl 1394.94297
[31] Lederman, R. R.; Talmon, R.; Wu, H.-T.; Lo, Y.-L.; Coifman, R. R., Alternating diffusion for common manifold learning with application to sleep stage assessment, (IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2015), IEEE), 5758-5762
[32] Lee-Chiong, T., Sleep Medicine: Essentials and Review (2008), Oxford University Press
[33] Lindenbaum, O.; Yeredor, A.; Salhov, M., Learning coupled embedding using multiview diffusion maps, (Proc. Latent Variable Analysis and Signal Separation (2015), Springer), 127-134
[34] Lindenbaum, O.; Yeredor, A.; Salhov, M.; Averbuch, A., Multiview diffusion maps (2015), preprint
[35] Luo, L.; Jia, W.; Zhang, C., Mixed propagation for image retrieval, (IEEE Fourth International Conference on Multimedia Information Networking and Security (MINES) (2012)), 237-240
[36] Luo, L.; Shen, C.; Zhang, C.; van den Hengel, A. J., Shape similarity analysis by self-tuning locally constrained mixed-diffusion, IEEE Trans. Multimedia, 15, 5, 1174-1183 (2013)
[37] Marshall, N. F.; Hirn, M. J., Time coupled diffusion maps (2016)
[38] Michaeli, T.; Wang, W.; Livescu, K., Nonparametric canonical correlation analysis (2015), preprint
[39] Nadler, B.; Lafon, S.; Coifman, R. R.; Kevrekidis, I. G., Diffusion maps, spectral clustering and reaction coordinates of dynamical systems, Appl. Comput. Harmon. Anal., 21, 1, 113-127 (2006) · Zbl 1103.60069
[40] Niyogi, P.; Smale, S.; Weinberger, S., Finding the homology of submanifolds with high confidence from random samples, Conference in Honor of John Franks and Clark Robinson. Conference in Honor of John Franks and Clark Robinson, Northwestern University, April 2003. Conference in Honor of John Franks and Clark Robinson. Conference in Honor of John Franks and Clark Robinson, Northwestern University, April 2003, Discrete Comput. Geom., 23, 1-23 (2006)
[41] Nyström, Evert Johannes, Über die praktische Auflösung von linearen Integralgleichungen mit Anwendungen auf Randwertaufgaben der Potentialtheorie (1929), Akademische Buchhandlung · JFM 55.0819.02
[42] Roweis, S. T.; Saul, L. K., Nonlinear dimensionality reduction by locally linear embedding, Science, 260, 2323-2326 (2000)
[43] Singer, A.; Wu, H.-T., Vector diffusion maps and the connection Laplacian, Comm. Pure Appl. Math., 65, 8, 1067-1144 (2012) · Zbl 1320.68146
[44] Singer, A.; Wu, H.-T., Spectral convergence of the connection Laplacian from random samples, Inform. Infer., J. IMA, 6, 1, 58-123 (2017) · Zbl 1386.94055
[45] Takens, F., Detecting strange attractors in turbulence, (Rand, David; Young, Lai-Sang, Dynamical Systems and Turbulence. Dynamical Systems and Turbulence, Lecture Notes in Math., vol. 898 (1981), Springer Berlin Heidelberg), 366-381 · Zbl 0513.58032
[46] Taylor, J. W., Short-term electricity demand forecasting using double seasonal exponential smoothing, J. Oper. Res. Soc., 54, 8, 799-805 (2003) · Zbl 1097.91541
[47] Tenenbaum, J. B.; de Silva, V.; Langford, J. C., A global geometric framework for nonlinear dimensionality reduction, Science, 260, 2319-2323 (2000)
[48] Wang, B.; Jiang, J.; Wang, W.; Zhou, Z.-H.; Tu, Z., Unsupervised metric fusion by cross diffusion, (Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)), 2997-3004
[49] Wu, H.-T.; Talmon, R.; Lo, Y.-L., Assess sleep stage by modern signal processing techniques, IEEE Trans. Biomed. Eng., 62, 4, 1159-1168 (2015)
[50] Wu, Z.; Wang, Y.; Shou, R.; Chen, B.; Liu, X., Unsupervised co-segmentation of 3d shapes via affinity aggregation spectral clustering, Computers & Graphics, 37, 6, 628-637 (October 2013)
[51] Zhou, Y.; Bai, X.; Liu, W.; Latecki, L. J., Fusion with diffusion for robust visual tracking, (Advances in Neural Information Processing Systems (2012)), 2978-2986
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.