×

Non-uniform deblurring for shaken images. (English) Zbl 1254.68287

Summary: Photographs taken in low-light conditions are often blurry as a result of camera shake, i.e. a motion of the camera while its shutter is open. Most existing deblurring methods model the observed blurry image as the convolution of a sharp image with a uniform blur kernel. However, we show that blur from camera shake is in general mostly due to the 3D rotation of the camera, resulting in a blur that can be significantly non-uniform across the image. We propose a new parametrized geometric model of the blurring process in terms of the rotational motion of the camera during exposure. This model is able to capture non-uniform blur in an image due to camera shake using a single global descriptor, and can be substituted into existing deblurring algorithms with only small modifications. To demonstrate its effectiveness, we apply this model to two deblurring problems; first, the case where a single blurry image is available, for which we examine both an approximate marginalization approach and a maximum a posteriori approach, and second, the case where a sharp but noisy image of the scene is available in addition to the blurry image. We show that our approach makes it possible to model and remove a wider class of blurs than previous approaches, including uniform blur as a special case, and demonstrate its effectiveness with experiments on synthetic and real images.

MSC:

68T45 Machine vision and scene understanding
68U10 Computing methodologies for image processing

Software:

PRMLT
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Banham, M. R., & Katsaggelos, A. K. (1997). Digital image restoration. IEEE Signal Processing Magazine, 14(2), 24–41. · doi:10.1109/79.581363
[2] Bishop, C. M. (2006). Pattern recognition and machine learning (information science and statistics). Berlin: Springer. ISBN 0387310738. · Zbl 1107.68072
[3] Cai, J.-F., Ji, H., Liu, C., & Shen, Z. (2009). Blind motion deblurring from a single image using sparse approximation. In Proc. CVPR.
[4] Chakrabarti, A., Zickler, T., & Freeman, W. T. (2010). Analyzing spatially-varying blur. In Proc. CVPR.
[5] Chan, T. F., & Wong, C.-K. (1998). Total variation blind deconvolution. IEEE Transactions on Image Processing, 7(3).
[6] Chen, J., Yuan, L., Tang, C.-K., & Quan, L. (2008). Robust dual motion deblurring. In Proc. CVPR.
[7] Cho, S., & Lee, S. (2009). Fast motion deblurring. ACM Transactions on Graphics, 28(5), 145:1–145:8 (Proc. SIGGRAPH Asia 2009). · doi:10.1145/1618452.1618491
[8] Cho, S., Matsushita, Y., & Lee, S. (2007). Removing non-uniform motion blur from images. In Proc. ICCV.
[9] Couzinie-Devy, F., Mairal, J., Bach, F., & Ponce, J. (2011). Dictionary learning for deblurring and digital zoom (submitted). Preprint HAL: inria-00627402.
[10] Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2008). Image restoration by sparse 3D transform-domain collaborative filtering. In SPIE electronic imaging.
[11] Efron, B., Hastie, T., Johnstone, L., & Tibshirani, R. (2004). Least angle regression. Annals of Statistics, 32(2), 407–499. · Zbl 1091.62054 · doi:10.1214/009053604000000067
[12] Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T., & Freeman, W. T. (2006). Removing camera shake from a single photograph. ACM Transactions on Graphics, 25(3), 787–794 (Proc. SIGGRAPH 2006). · Zbl 1371.94125 · doi:10.1145/1141911.1141956
[13] Gupta, A., Joshi, N., Zitnick, C. L., Cohen, M., & Curless, B. (2010). Single image deblurring using motion density functions. In Proc. ECCV.
[14] Hartley, R. I., & Zisserman, A. (2004). Multiple view geometry in computer vision (2nd edn.). Cambridge: CUP. ISBN 0521540518. · Zbl 1072.68104
[15] Hirsch, M., Sra, S., Schölkopf, B., & Harmeling, S. (2010). Efficient filter flow for space-variant multiframe blind deconvolution. In Proc. CVPR.
[16] Joshi, N., Kang, S. B., Zitnick, C. L., & Szeliski, R. (2010). Image deblurring using inertial measurement sensors. ACM Transactions on Graphics, 29(4), 30:1–30:9 (Proc. SIGGRAPH 2010).
[17] Kim, S.-J., Koh, K., Lustig, M., Boyd, S., & Gorinevsky, D. (2007). An interior-point method for large-scale 1-regularized least squares. IEEE Journal of Selected Topics in Signal Processing, 1(4), 606–617. · doi:10.1109/JSTSP.2007.910971
[18] Klein, G., & Drummond, T. (2005). A single-frame visual gyroscope. In Proc. BMVC.
[19] Krishnan, D., & Fergus, R. (2009). Fast image deconvolution using hyper-Laplacian priors. In NIPS.
[20] Lee, D. D., & Seung, H. S. (2001). Algorithms for non-negative matrix factorization. In NIPS.
[21] Levin, A. (2006). Blind motion deblurring using image statistics. In NIPS.
[22] Levin, A., Weiss, Y., Durand, F., & Freeman, W. T. (2009). Understanding and evaluating blind deconvolution algorithms. In Proc. CVPR.
[23] Lim, S. H., & Silverstein, A. (2008). Estimation and removal of motion blur by capturing two images with different exposures. Technical Report HPL-2008-170, HP Laboratories.
[24] Lucy, L. B. (1974). An iterative technique for the rectification of observed distributions. Astronomical Journal, 79(6), 745–754. · doi:10.1086/111605
[25] Mairal, J., Bach, F., Ponce, J., & Sapiro, G. (2010). Online learning for matrix factorization and sparse coding. Journal of Machine Learning Research, 11, 19–60. · Zbl 1242.62087
[26] Miskin, J. W., & MacKay, D. J. C. (2000). Ensemble learning for blind image separation and deconvolution. In M. Girolani (Ed.), Advances in independent component analysis. Berlin: Springer.
[27] Nagy, J. G., & O’Leary, D. P. (1998). Restoring images degraded by spatially variant blur. SIAM Journal on Scientific Computing, 19(4), 1063–1082. · Zbl 0919.65091 · doi:10.1137/S106482759528507X
[28] Osher, S., & Rudin, L. I. (1990). Feature oriented image enhancement using shock filters. SIAM Journal on Numerical Analysis, 27(4), 919–940. · Zbl 0714.65096 · doi:10.1137/0727053
[29] Pérez, P., Gangnet, M., & Blake, A. (2003). Poisson image editing. ACM Transactions on Graphics, 22(3), 313–318 (Proc. SIGGRAPH 2003). · Zbl 05457528 · doi:10.1145/882262.882269
[30] Puetter, R. C., Gosnell, T. R., & Yahil, A. (2005). Digital image reconstruction: deblurring and denoising. Annual Review of Astronomy and Astrophysics, 43, 139–194. · doi:10.1146/annurev.astro.43.112904.104850
[31] Rav-Acha, A., & Peleg, S. (2005). Two motion-blurred images are better than one. Pattern Recognition Letters, 26(3).
[32] Richardson, W. H. (1972). Bayesian-based iterative method of image restoration. Journal of the Optical Society of America, 62(1), 55–59. · doi:10.1364/JOSA.62.000055
[33] Sawchuk, A. A. (1974). Space-variant image restoration by coordinate transformations. Journal of the Optical Society of America, 64(2), 138–144. · doi:10.1364/JOSA.64.000138
[34] Shan, Q., Xiong, W., & Jia, J. (2007). Rotational motion deblurring of a rigid object from a single image. In Proc. ICCV.
[35] Shan, Q., Jia, J., & Agarwala, A. (2008). High-quality motion deblurring from a single image. ACM Transactions on Graphics, 27(3) (Proc. SIGGRAPH 2008).
[36] Tai, Y.-W., Tan, P., Gao, L., & Brown, M. S. (2009). Richardson-Lucy deblurring for scenes under projective motion path. Technical report, KAIST.
[37] Tai, Y.-W., Du, H., Brown, M. S., & Lin, S. (2010a). Correction of spatially varying image and video motion blur using a hybrid camera. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(6), 1012–1028. · doi:10.1109/TPAMI.2009.97
[38] Tai, Y.-W., Kong, N., Lin, S., & Shin, S. Y. (2010b). Coded exposure imaging for projective motion deblurring. In Proc. CVPR.
[39] Tai, Y.-W., Tan, P., & Brown, M. S. (2011). Richardson-Lucy deblurring for scenes under a projective motion path. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(8), 1603–1618. · doi:10.1109/TPAMI.2010.222
[40] Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Joutnal of the Royal Statistical Society, Series B, 58(1), 267–288. · Zbl 0850.62538
[41] Tomasi, C., & Manduchi, R. (1998). Bilateral filtering for gray and color images. In Proc. ICCV.
[42] Vio, R., Nagy, J., Tenorio, L., & Wamsteker, W. (2005). Multiple image deblurring with spatially variant PSFs. Astronomy & Astrophysics, 434, 795–800. · doi:10.1051/0004-6361:20035754
[43] Xu, L., & Jia, J. (2010). Two-phase kernel estimation for robust motion deblurring. In Proc. ECCV.
[44] Yuan, L., Sun, J., Quan, L., & Shum, H.-Y. (2007a). Image deblurring with blurred/noisy image pairs. ACM Transactions on Graphics, 26(3) (Proc. SIGGRAPH 2007).
[45] Yuan, L., Sun, J., Quan, L., & Shum, H.-Y. (2007b). Blurred/non-blurred image alignment using sparseness prior. In Proc. ICCV.
[46] Yuan, L., Sun, J., Quan, L., & Shum, H.-Y. (2008). Progressive inter-scale and intra-scale non-blind image deconvolution. ACM Transactions on Graphics, 27(3) (Proc. SIGGRAPH 2008)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.