×

Rate-invariant analysis of covariance trajectories. (English) Zbl 1433.68472

Summary: Statistical analysis of dynamic systems, such as videos and dynamic functional connectivity, is often translated into a problem of analyzing trajectories of relevant features, particularly covariance matrices. As an example, in video-based action recognition, a natural mathematical representation of activity videos is as parameterized trajectories on the set of symmetric, positive-definite matrices (SPDMs). The execution rates of actions, implying arbitrary parameterizations of trajectories, complicate their analysis. To handle this challenge, we represent covariance trajectories using transported square-root vector fields, constructed by parallel translating scaled-velocity vectors of trajectories to their starting points. The space of such representations forms a vector bundle on the SPDM manifold. Using a natural Riemannian metric on this vector bundle, we approximate geodesic paths and geodesic distances between trajectories in the space of this vector bundle. This metric is invariant to the action of the re-parameterization group, and leads to a rate-invariant analysis of trajectories. In the process, we remove the parameterization variability and temporally register trajectories. We demonstrate this framework in multiple contexts, using both generative statistical models and discriminative data analysis. The latter is illustrated using several applications involving video-based action recognition and dynamic functional connectivity analysis.

MSC:

68T45 Machine vision and scene understanding
37D40 Dynamical systems of geometric origin and hyperbolicity (geodesic and horocycle flows, etc.)
53C22 Geodesics in global differential geometry
62M40 Random fields; image analysis

Software:

fda (R)
PDFBibTeX XMLCite
Full Text: DOI Link

References:

[1] Afsari, B; Tron, R; Vidal, R, On the convergence of gradient descent for finding the Riemannian center of mass, SIAM J. Control Optim., 51, 2230-2260, (2013) · Zbl 1285.90031 · doi:10.1137/12086282X
[2] Aron, AR; Robbins, TW; Poldrack, RA, Inhibition and the right inferior frontal cortex, Trends Cogn. Sci. (Regul. Ed.), 8, 170-177, (2004) · doi:10.1016/j.tics.2004.02.010
[3] Arsigny, V; Fillard, P; Pennec, X; Ayache, N, Log-Euclidean metrics for fast and simple calculus on diffusion tensors, Magn. Reson. Med., 56, 411-421, (2006) · doi:10.1002/mrm.20965
[4] Basser, PJ; Pierpaoli, C, Microstructural and physiological features of tissues elucidated by quantitative-diffusion-tensor MRI, J. Magn. Reson. B, 111, 209-219, (1996) · doi:10.1006/jmrb.1996.0086
[5] Brigant, A.L.: Computing distances and geodesics between manifold-valued curves in the SRV framework. arXiv:1601.02358 (2016) · Zbl 1366.58005
[6] Brigant, A.L., Arnaudon, M., Barbaresco, F.: Reparameterization invariant metric on the space of curves. arXiv:1507.06503 (2015) · Zbl 1396.58011
[7] Celledoni, E; Eslitzbichler, M; Schmeding, A, Shape analysis on Lie groups with applications in computer animation, J. Geom. Mech., 8, 273-304, (2016) · Zbl 1366.65018 · doi:10.3934/jgm.2016008
[8] Dai, M., Zhang, Z., Srivastava, A.: Testing stationarity of brain functional connectivity using change-point detection in fMRI data. In: CVPR Workshops Diff-CVML, pp. 981-989 (2016)
[9] Dai, M., Zhang, Z., Srivastava, A.: Discovering change-point patterns in dynamic functional brain connectivity of a population. In: IPMI (2017)
[10] Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: International Conference on CVPR, vol. 2, pp. 886-893 (2005)
[11] Destrieux, C; Fischl, B; Dale, A; Halgren, E, Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclature, Neuroimage, 53, 1-15, (2010) · doi:10.1016/j.neuroimage.2010.06.010
[12] Dryden, IL; Koloydenko, AA; Zhou, D, Non-Euclidean statistics for covariance matrices with applications to diffusion tensor imaging, Ann. Appl. Stat., 3, 1102?1123, (2009) · Zbl 1196.62063 · doi:10.1214/09-AOAS249
[13] Faraki, M; Palhang, M; Sanderson, C, Log-Euclidean bag of words for human action recognition, IET Comput. Vision, 9, 331-339, (2014) · doi:10.1049/iet-cvi.2014.0018
[14] Glasser, MF; Sotiropoulos, SN; Wilson, JA; Coalson, TS; Fischl, B; Andersson, JL; Xu, J; Jbabdi, S; Webster, M; Polimeni, JR; Essen, DC; Jenkinson, M, The minimal preprocessing pipelines for the human connectome project, Neuroimage, 80, 105-124, (2013) · doi:10.1016/j.neuroimage.2013.04.127
[15] Guo, K., Ishwar, P., Konrad, J.: Action recognition in video by sparse representation on covariance manifolds of silhouette tunnels. In: Proceedings of the 20th International Conference on Recognizing Patterns in Signals, Speech, Images, and Videos, pp. 294-305 (2010)
[16] Harandi, M; Hartley, R; Shen, C; Lovell, B; Sanderson, C, Extrinsic methods for coding and dictionary learning on Grassmann manifolds, Int. J. Comput. Vision, 114, 113-136, (2015) · doi:10.1007/s11263-015-0833-x
[17] Harandi, M.T., Sanderson, C., Wiliem, A., Lovell, B.C.: Kernel analysis over Riemannian manifolds for visual recognition of actions, pedestrians and textures. In: Proceedings of the 2012 IEEE Workshop on the Applications of Computer Vision, pp. 433-439 (2012)
[18] Hutchison, RM; etal., Dynamic functional connectivity: promise, issues, and interpretations, Neuroimage, 80, 360-378, (2013) · doi:10.1016/j.neuroimage.2013.05.079
[19] Jost, J.: Riemannian Geometry and Geometric Analysis. Springer, New York (2005) · Zbl 1083.53001
[20] Jupp, PE; Kent, JT, Fitting smooth paths to spherical data, J. R. Stat. Soc.: Ser. C (Appl. Stat.), 36, 34-46, (1987) · Zbl 0613.62086
[21] Kendall, WS, Probability, convexity, and harmonic maps with small image I: uniqueness and fine existence, Proceed. Lond. Math. Soc., 3, 371-406, (1990) · Zbl 0675.58042 · doi:10.1112/plms/s3-61.2.371
[22] Kendall, W.S.: Barycenters and hurricane trajectories. arXIV:1406.7173 (2014)
[23] Kim, TK; Cipolla, R, Canonical correlation analysis of video volume tensors for action categorization and detection, IEEE Trans. Pattern Anal. Mach. Intell., 31, 1415-1428, (2009) · doi:10.1109/TPAMI.2008.167
[24] Kim, T.K., Wong, K.Y.K., Cipolla, R.: Tensor canonical correlation analysis for action classification. In: IEEE Conference on CVPR, pp. 1-8 (2007)
[25] Kume, A; Dryden, IL; Le, H, Shape-space smoothing splines for planar landmark data, Biometrika, 94, 513-528, (2007) · Zbl 1134.62044 · doi:10.1093/biomet/asm047
[26] Kurtek, S; Srivastava, A; Klassen, E; Ding, Z, Statistical modeling of curves using shapes and related features, J. Am. Stat. Assoc., 107, 1152-1165, (2012) · Zbl 1443.62389 · doi:10.1080/01621459.2012.699770
[27] Lahiri, S; Robinson, D; Klassen, E, Precise matching of PL curves in \(R^N\) in square-root velocity framework, Geom. Imaging Comput., 2, 133-186, (2015) · Zbl 1403.94020 · doi:10.4310/GIC.2015.v2.n3.a1
[28] Lui, YM, Human gesture recognition on product manifolds, J. Mach. Learn. Res., 13, 3297-3321, (2012) · Zbl 1433.68402
[29] Lui, Y.M., Beveridge, J., Kirby, M.: Action classification on product manifolds. In: IEEE Conference on CVPR, pp. 833-839 (2010)
[30] Morris, R.J., Kent, J., Mardia, K.V., Fidrich, M., Aykroyd, R.G., Linney, A.: Analysing growth in faces. In: International conference on Imaging Science, Systems and Technology (1999)
[31] Pennec, X; Fillard, P; Ayache, N, A Riemannian framework for tensor computing, Int. J. Comput. Vision, 66, 41-66, (2006) · Zbl 1287.53031 · doi:10.1007/s11263-005-3222-z
[32] Ramsay, J.O., Silverman, B.W.: Functional Data Analysis. Springer Series in Statistics. Springer, New York (2005)
[33] Samir, C; Absil, PA; Srivastava, A; Klassen, E, A gradient-descent method for curve Fitting on Riemannian manifolds, Found. Comput. Math., 12, 49-73, (2012) · Zbl 1245.65017 · doi:10.1007/s10208-011-9091-7
[34] Schwartzman, A; Mascarenhas, W; Taylor, J, Inference for eigenvalues and eigenvectors of Gaussian symmetric matrices, Ann. Stat., 36, 2886-2919, (2008) · Zbl 1196.62067 · doi:10.1214/08-AOS628
[35] Srivastava, A., Klassen, E.: Functional and Shape Data Analysis. Springer, New York (2016) · Zbl 1376.62003 · doi:10.1007/978-1-4939-4020-2
[36] Srivastava, A; Klassen, E; Joshi, S; Jermyn, I, Shape analysis of elastic curves in Euclidean spaces, IEEE Trans. Pattern Anal. Mach. Intell., 33, 1415-1428, (2011) · doi:10.1109/TPAMI.2010.184
[37] Su, J; Dryden, IL; Klassen, E; Le, H; Srivastava, A, Fitting optimal curves to time-indexed, noisy observations on nonlinear manifolds, J. Image Vis. Comput., 30, 428-442, (2012) · doi:10.1016/j.imavis.2011.09.006
[38] Su, J; Kurtek, S; Klassen, E; Srivastava, A, Statistical analysis of trajectories on Riemannian manifolds: bird migration, hurricane tracking and video surveillance, Ann. Appl. Stat., 8, 530-552, (2014) · Zbl 1454.62554 · doi:10.1214/13-AOAS701
[39] Su, J., Srivastava, A., de Souza, F., Sarkar, S.: Rate-invariant analysis of trajectories on Riemannian manifolds with application in visual speech recognition. In: 2014 IEEE Conference on CVPR, pp. 620-627 (2014)
[40] Tuzel, O., Porikli, F., Meer, P.: Region covariance: a fast descriptor for detection and classification. In: 9th European Conference on Computer Vision, pp. 589-600 (2006)
[41] Zhang, Z., Su, J., Klassen, E., Le, H., Srivastava, A.: Video-based action recognition using rate-invariant analysis of covariance trajectories. arXiv:1503.06699 (2015)
[42] Zhao, G; Barnard, M; Pietikäinen, M, Lipreading with local spatiotemporal descriptors, IEEE Trans. Multimed., 11, 1254-1265, (2009) · doi:10.1109/TMM.2009.2030637
[43] Zhao, G., Pietikäinen, M., Hadid, A.: Local spatiotemporal descriptors for visual recognition of spoken phrases. In: Proceedings of the International Workshop on Human-centered Multimedia, HCM ’07, pp. 57-66 (2007)
[44] Dryden, IL; Koloydenko, A; Zhou, D, Non-Euclidean statistics for covariance matrices, with applications to diffusion tensor imaging, Ann. Appl. Stat., 3, 1102-1123, (2009) · Zbl 1196.62063 · doi:10.1214/09-AOAS249
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.