×

Linear multiscale analysis of similarities between images on Riemannian manifolds: practical formula and affine covariant metrics. (English) Zbl 1343.68269

Summary: In this paper we study the problem of comparing two patches of images defined on Riemannian manifolds which in turn can be defined by each image domain with a suitable metric depending on the image. For that we single out one particular instance of a set of models defining image similarities that was earlier studied in [C. Ballester et al., Multiscale Model. Simul. 12, No. 2, 616–649 (2014; Zbl 1328.68266)], using an axiomatic approach that extended the classical Álvarez-Guichard-Lions-Morel work to the nonlocal case. Namely, we study a linear model to compare patches defined on two images in \(\mathbb{R}^N\) endowed with some metric. Besides its genericity, this linear model is selected by its computational feasibility since it can be approximated leading to an algorithm that has the complexity of the usual patch comparison using a weighted Euclidean distance. Moreover, we propose and study some intrinsic metrics which we define in terms of affine covariant structure tensors and we discuss their properties. These tensors are defined for any point in the image and are intrinsically endowed with affine covariant neighborhoods. We also discuss the effect of discretization over the affine covariance properties of the tensors. We illustrate our theoretical results with numerical experiments.

MSC:

68U10 Computing methodologies for image processing
35K65 Degenerate parabolic equations
58J60 Relations of PDEs with special manifold structures (Riemannian, Finsler, etc.)
94A08 Image processing (compression, reconstruction, etc.) in information and communication theory

Citations:

Zbl 1328.68266

Software:

ASIFT; Vlfeat; SIFT
PDFBibTeX XMLCite
Full Text: DOI Link

References:

[1] S. Agarwal and D. Roth, {\it Learning a sparse representation for object detection}, in Proceedings of the Seventh European Conference on Computer Vision (ECCV 2002), Lecture Notes in Comput. Sci. 2353, Springer, Berlin, 2002, pp. 113-127. · Zbl 1039.68590
[2] L. Alvarez, F. Guichard, P.-L. Lions, and J. M. Morel, {\it Axioms and fundamental equations of image processing}, Arch. Rational Mech. Anal., 123 (1993), pp. 199-257. · Zbl 0788.68153
[3] C. Ballester, F. Calderero, V. Caselles, and G. Facciolo, {\it Multiscale analysis of similarities between images on Riemannian manifolds}, Multiscale Model. Simul., 12 (2014), pp. 616-649. · Zbl 1328.68266
[4] A. Baumberg, {\it Reliable feature matching across widely separated views}, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2000), Vol. 1, 2000, pp. 1774-1781.
[5] T. Brox, R. Boomgaard, F. Lauze, J. Weijer, J. Weickert, P. Mrázek, and P. Kornprobst, {\it Adaptive structure tensors and their applications}, in Visualization and Processing of Tensor Fields, Springer, Berlin, 2006, pp. 17-47, 417-420.
[6] T. Brox, J. Weickert, B. Burgeth, and P. Mrázek, {\it Nonlinear structure tensors}, Image Vision Comput., 24 (2006), pp. 41-55.
[7] R. Brunelli and T. Poggio, {\it Face recognition: Features versus templates}, IEEE Trans. Pattern Anal. Mach. Intell., 15 (1993), pp. 1042-1052.
[8] A. Buades, B. Coll, and J.-M. Morel, {\it A non-local algorithm for image denoising}, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2005), Vol. 2, 2005, pp. 60-65. · Zbl 1108.94004
[9] M. C. Burl, M. Weber, and P. Perona, {\it A probabilistic approach to object recognition using local photometry and global geometry}, in Proceedings of the 10th European Conference on Computer Vision (ECCV’ 1998), Lecture Notes in Comput. Sci. 1407, Springer, Berlin, 1998, pp. 628-641.
[10] F. Calderero and V. Caselles, {\it Multiscale analysis for images on Riemannian manifolds}, SIAM J. Imaging Sci., 7 (2014), pp. 1108-1170. · Zbl 1304.68189
[11] V. Caselles, L. Igual, and O. Sander, {\it An axiomatic approach to scalar data interpolation on surfaces}, Numer. Math., 102 (2006), pp. 383-411. · Zbl 1092.65007
[12] V. Caselles, J. M. Morel, and C. Sbert, {\it An axiomatic approach to image interpolation}, IEEE Trans. Image Process., 7 (1998), pp. 376-386. · Zbl 0993.94504
[13] P. Chatterjee and P. Milanfar, {\it Patch-based near-optimal image denoising}, IEEE Trans. Image Process., 21 (2012), pp. 1635-1649. · Zbl 1373.94069
[14] J. K. Cohen, F. G. Hagin, and J. B. Keller, {\it Short time asymptotic expansions of solutions of parabolic equations}, J. Math. Anal. Appl., 38 (1972), pp. 82-91. · Zbl 0201.43202
[15] C. A. Deledalle, V. Duval, and J. Salmon, {\it Non-local methods with shape-adaptive patches (NLM-SAP)}, J. Math. Imaging Vision, 43 (2012), pp. 103-120. · Zbl 1255.68227
[16] R. Fergus, P. Perona, and A. Zisserman, {\it Object class recognition by unsupervised scale-invariant learning}, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2003), Vol. 2, 2003, pp. 264-271.
[17] A. Foi, V. Katkovnik, and K. Egiazarian, {\it Pointwise shape-adaptive DCT for high-quality denoising and deblocking of grayscale and color images}, IEEE Trans. Image Process., 16 (2007), pp. 1395-1411.
[18] J. Garding and T. Lindeberg, {\it Direct estimation of local surface shape in a fixating binocular vision system}, in Proceedings of the Third European Conference on Computer Vision (ECCV 1994), J.-O. Eklundh, ed., Lecture Notes in Comput. Sci. 800, Springer, Berlin, Heidelberg, 1994, pp. 365-376.
[19] F. Guichard and J.-M. Morel, {\it Image Analysis and PDEs}, IPAM GBM Tutorials, 2001.
[20] N. Kawai, T. Sato, and N. Yokoya, {\it Image inpainting considering brightness change and spatial locality of textures and its evaluation}, in Advances in Image and Video Technology, Lecture Notes in Comput. Sci. 5414, Springer, Berlin, 2009, pp. 271-282.
[21] R. Kimmel, R. Malladi, and N. Sochen, {\it Images as embedded maps and minimal surfaces: Movies, color, texture, and volumetric medical images}, Int. J. Comput. Vis., 39 (2000), pp. 111-129. · Zbl 1060.68656
[22] D. G. Lowe, {\it Distinctive image features from scale-invariant keypoints}, Int. J. Comput. Vis., 60 (2004), pp. 91-110.
[23] J. Matas, O. Chum, M. Urban, and T. Pajdla, {\it Robust wide-baseline stereo from maximally stable extremal regions}, Image Vision Comput., 22 (2004), pp. 761-767.
[24] K. Mikolajczyk, {\it Affine Covariant Features: Dataset,} http://www.robots.ox.ac.uk/ vgg/research/affine/ (2007).
[25] K. Mikolajczyk and C. Schmid, {\it Scale & affine invariant interest point detectors}, Int. J. Comput. Vis., 60 (2004), pp. 63-86.
[26] J.-M. Morel and G. Yu, {\it ASIFT: A new framework for fully affine invariant image comparison}, SIAM J. Imaging Sci., 2 (2009), pp. 438-469. · Zbl 1181.68252
[27] J.-M. Morel and G. Yu, {\it Is sift scale invariant?}, Inverse Probl. Imaging, 5 (2011), pp. 115-136. · Zbl 1217.68195
[28] G. Peyré, {\it Manifold models for signals and images}, Comput. Vis. Image Underst., 113 (2009), pp. 249-260.
[29] N. Sabater, J.-M. Morel, and A. Almansa, {\it How accurate can block matches be in stereo vision?}, SIAM J. Imaging Sci., 4 (2011), pp. 472-500. · Zbl 1215.68261
[30] N. Sochen, R. Kimmel, and A. M. Bruckstein, {\it Diffusions and confusions in signal and image processing}, J. Math. Imaging Vision, 14 (2001), pp. 195-209. · Zbl 0994.94011
[31] E. Stiefel, {\it Richtungsfelder und fernparallelismus in n-dimensionalen mannigfaltigkeiten}, Comment. Math. Helv., 8 (1935), pp. 305-353. · Zbl 0014.41601
[32] J. Sullivan, A. Blake, M. Isard, and J. MacCormick, {\it Bayesian object localisation in images}, Int. J. Comput. Vis., 44 (2001), pp. 111-135. · Zbl 0981.68527
[33] E. Thomas, {\it Vector fields on manifolds}, Bull. Amer. Math. Soc., 75 (1969), pp. 643-683. · Zbl 0183.51703
[34] C. Tomasi and R. Manduchi, {\it Bilateral filtering for gray and color images}, in Proceedings of the Sixth International IEEE Conference on Computer Vision (ICCV 1998), 1998, pp. 839-846.
[35] T. Tuytelaars and K. Mikolajczyk, {\it Local invariant feature detectors: A survey}, Found. Trends Comp. Graph. Vis., 3 (2008), pp. 177-280.
[36] T. Tuytelaars and L. Van Gool, {\it Matching widely separated views based on affine invariant regions}, Int. J. Comput. Vis., 59 (2004), pp. 61-85.
[37] S. R. S. Varadhan, {\it On the behavior of the fundamental solution of the heat equation with variable coefficients}, Comm. Pure Appl. Math., 20 (1967), pp. 431-455. · Zbl 0155.16503
[38] A. Vedaldi and B. Fulkerson, {\it VLFeat: An Open and Portable Library of Computer Vision Algorithms}, http://www.vlfeat.org/ (2008).
[39] J. Weickert, {\it Anisotropic Diffusion in Image Processing}, Vol. 1, Teubner, Stuttgart, 1998. · Zbl 0886.68131
[40] J. Weickert, {\it Coherence-enhancing diffusion filtering}, Int. J. Comput. Vis., 31 (1999), pp. 111-127. · Zbl 1505.94010
[41] K. J. Yoon and I. S. Kweon, {\it Adaptive support-weight approach for correspondence search}, IEEE Trans. Pattern Anal. Mach. Intell., 28 (2006), pp. 650-656.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.