×

zbMATH — the first resource for mathematics

Blind super-resolution restoration with frame-by-frame nonparametric blur estimation. (English) Zbl 1380.94032
Summary: In this paper, we investigate super-resolution image restoration from multiple images, which are possibly degraded with large nonparametric motion blur. The blur kernel for each input image is separately estimated. This is unlike many existing super-resolution algorithms, which assume identical blur kernel for all input images. We also do not make any restrictions on the motion field among images; that is, we estimate dense motion field without simplifications such as parametric motion. We present a two-step algorithm: In the first step, each input image is deblurred using its estimated blur kernel. In the second step, multi-frame super-resolution restoration is applied to the deblurred images. Because the estimated blur kernels may not be accurate, we propose a weighted cost function for the super-resolution restoration step, where a weight associated with an input image reflects the reliabilities of the corresponding kernel estimate and deblurred image. We provide experimental results with both simulated and real data, and show the effectiveness and robustness of the proposed method compared to some alternative approaches and state-of-the-art methods.
MSC:
94A08 Image processing (compression, reconstruction, etc.) in information and communication theory
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Audiovisual communications lcav. [Online]. http://lcav.epfl.ch/software/superresolution/. · Zbl 1373.94115
[2] Bascle, B., Blake, A., & Zisserman, A. (1996). Motion deblurring and super-resolution from an image sequence, In European conference on computer vision (vol. 2, pp. 573-582), Springer, Berlin.
[3] Cai, J., Ji, H., Liu, C., & Shen, Z. (2009). Blind motion deblurring from a single image using sparse approximation, In IEEE conference on computer vision and pattern recognition (pp. 104-111).
[4] Capel, D. (Ed.). (2006). Image mosaicing and super-resolution. Berlin: Springer. · Zbl 1060.68132
[5] Cho, S., & Lee, S. (2009). Fast motion deblurring, ACM transactions on graphics. (vol. 28, p. Article no. 145), Dec 2009.
[6] Farsiu, S; Robinson, MD; Elad, M; Milanfar, P, Fast and robust multipleframe super resolution, IEEE Transactions on Image Processing, 13, 1327-1343, (2004)
[7] Fergus, R; Singh, B; Hertzmann, A; Freeman, WT, Removing camera shake from a single photograph, ACM Transactions on Graphics, 25, 787-794, (2006) · Zbl 1371.94125
[8] Freeman, WT; Jones, TR; Pasztor, EC, Example-based super-resolution, IEEE Computer Graphics and Applications, 22, 56-65, (2002)
[9] Gevrekci, M., & Gunturk, B. K. (2007a). Superresolution under photometric diversity of images. EURASIP Journal on Advances in Signal Processing, 2007(1), 205. · Zbl 1168.94329
[10] Gevrekci, M., & Gunturk, B. K. (2007b). On geometric and photometric registration of images. In IEEE international conference on acoustics, speech, signal processing (ICASSP), Vol. 1, pp. 1261-1264. · Zbl 1168.94329
[11] Gunturk, BK; Gevrekci, M, High-resolution image reconstruction from multiple differently exposed images, IEEE Signal Processing Letters, 13, 197-200, (2006)
[12] Gunturk, BK; Lukac, R (ed.), Super-resolution imaging, (2010), Boca Raton
[13] Gunturk, B. K., & Li, X. (Eds.). (2012). Image restoration: Fundamentals and advances. Boca Raton: CRC Press. · Zbl 1252.68033
[14] Hossain, I; Gunturk, BK, High dynamic range imaging of non-static scenes, SPIE electronic imaging, 7876, 78 760p1-78 760p9, (2011)
[15] Joshi, N., Szeliski, R., & Kriegman, D. (2008). Psf estimation using sharp edge prediction. In IEEE conference on computer vision and pattern recognition (pp. 104-111).
[16] Katsaggelos, A., Molina, R., & Mateos, J. (Eds.). (2006). Super-resolution of images. San Rafael: Morgan and Claypool Publishers. · Zbl 1140.68523
[17] Kim, T. H., & Lee, K. M. (2014). Segmentation-free dynamic scene deblurring. In IEEE conference on computer vision and pattern recognition (pp. 2766-2773).
[18] Kim, T. H., Ahn, B., & Lee, K. M. (2013). Dynamic scene deblurring. In IEEE conference on computer vision (pp. 3160-3167).
[19] Kim, KI; Kwon, Y, Single-image super-resolution using sparse regression and natural image prior, IEEE Transactions on Pattern Analysis and Machine Intelligence, 32, 1127-1133, (2010)
[20] Liu, C. (2009). Beyond pixels: Exploring new representations and applications for motion analysis, In Doctoral Thesis. Massachusetts Institute of Technology.
[21] Liu, C., & Sun, D.Q. (2011). A bayesian approach to adaptive video super resolution. In IEEE conference on computer vision and pattern recognition (vol 2, pp. 209-216).
[22] Michaeli, T., & Irani, M. (2013). Nonparametric blind super-resolution, In IEEE conference on computer vision (pp. 945-952).
[23] Middleburry optical flow database and evaluation. [Online]. vision.middlebury.edu/flow/.
[24] Milanfar, P. (Ed.). (2010). Super-resolution imaging. Boca Raton: CRC Press.
[25] Mitzel, D., Pock, T., Schoenemann, T., & Cremers, D. (2009). Video super resolution using duality based tv-l1 optical flow. In DAGM symposium on pattern recognition (pp 432-441), Springer, Berlin.
[26] Park, SC; Park, MK; Kang, MG, Super-resolution image reconstruction: A technical overview, IEEE Signal Processing Magazine, 20, 21-36, (2003)
[27] Qian, Q., & Gunturk, B. K. (2014). Space-varying blur kernel estimation and image deblurring. In SPIE conference on electronic imaging (pp. 90 230E1-90 230E11).
[28] Qian, Q., & Gunturk, B. K. (2014). Super-resolution restoration of motion blurred images. In SPIE conference on electronic imaging (pp. 90 230F1-90 230F9).
[29] Rav-Acha, A., & Peleg, S. (2000). Restoration of multiple images with motion blur in different directions. In IEEE workshop on computer vision (pp. 1-7).
[30] Shan, Q; Jia, J; Agarwala, A, High-quality motion deblurring from a single image, ACM Transactions on Graphics, 27, 1-10, (2008)
[31] Sroubek, F; Cristobal, G; Flusser, J, Simultaneous super-resolution and blind deconvolution, Journal of Physics: Conference Series, 124, 012 048:1-012 048:8, (2008)
[32] Su, H; Tang, L; Wu, Y, Unified blind method for multi-image super-resolution and single/multi-image blur deconvolution, IEEE Transactions on Image Processing, 22, 2101-2114, (2013) · Zbl 1373.94115
[33] Wang, JF; Zhang, Y; Yin, WT, An efficient tvl1 algorithm for deblurring multichannel images corrupted by impulsive noise, SIAM Journal of Scientific Computing, 31, 2842-2865, (2009) · Zbl 1195.68110
[34] Xu, L., & Jia, J.Y. (2010). Two-phase kernel estimation for robust motion deblurring. In European conference on computer vision (pp. 157-170), Springer, Berlin. · Zbl 1371.94125
[35] Xu, L., Zheng, S., & Jia, J. (2013). Unnatural l0 sparse representation for natural image deblurring. In Computer vision and pattern recognition (pp. 1107-1114).
[36] Yamaguchi, T., Fukuda, H., Furukawa, R., Kawasaki, H., & Sturm, P. (2010). Video deblurring and super-resolution technique for multiple moving objects. In Asian conference on computer vision (vol. 4, pp. 127-140).
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.