×

Efficient and effective total variation image super-resolution: a preconditioned operator splitting approach. (English) Zbl 1213.94013

Summary: Super-resolution is a fusion process for reconstructing a high-resolution image from a set of low-resolution images. This paper proposes a novel approach to image super-resolution based on total variation (TV) regularization. We applied the Douglas-Rachford splitting technique to the constrained TV-based variational SR model which is separated into three subproblems that are easy to solve. Then, we derive an efficient and effective iterative scheme, which includes a fast iterative shrinkage/thresholding algorithm for denoising problem, a very simple noniterative algorithm for fusion part, and linear equation systems for deblurring process. Moreover, to speed up convergence, we provide an accelerated scheme based on precondition design of initial guess and forward-backward splitting technique which yields linear systems of equations with a nice structure. The proposed algorithm shares a remarkable simplicity together with a proven global rate of convergence which is significantly better than currently known lagged diffusivity fixed point iteration algorithm and fast decoupling algorithm by exploiting the alternating minimizing approach. Experimental results are presented to illustrate the effectiveness of the proposed algorithm.

MSC:

94A08 Image processing (compression, reconstruction, etc.) in information and communication theory

Software:

RecPF
PDFBibTeX XMLCite
Full Text: DOI EuDML

References:

[1] R. Tsai and T. Huang, “Multi-frame image restoration and registration,” Advances in Computer Vision and Image Processing, vol. 1, no. 2, pp. 317-339, 1984.
[2] S. P. Kim, N. K. Bose, and H. M. Valenzuela, “Recursive reconstruction of high resolution image from noisy undersampled multiframes,” IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 38, no. 6, pp. 1013-1027, 1990. · doi:10.1109/29.56062
[3] S. P. Kim and W. Y. Su, “Recursive high-resolution reconstruction of blurred multiframe images,” IEEE Transactions on Image Processing, vol. 2, no. 4, pp. 534-539, 1993. · doi:10.1109/83.242363
[4] S. Rhee and M. G. Kang, “Discrete cosine transform based regularized high-resolution image reconstruction algorithm,” Optical Engineering, vol. 38, no. 8, pp. 1348-1356, 1999. · doi:10.1117/1.602177
[5] R. H. Chan, T. F. Chan, L. Shen, and Z. Shen, “Wavelet algorithms for high-resolution image reconstruction,” SIAM Journal on Scientific Computing, vol. 24, no. 4, pp. 1408-1432, 2003. · Zbl 1031.68127 · doi:10.1137/S1064827500383123
[6] M. K. Ng, C. K. Sze, and S. P. Yung, “Wavelet algorithms for deblurring models,” International Journal of Imaging Systems and Technology, vol. 14, no. 3, pp. 113-121, 2004. · doi:10.1002/ima.20014
[7] N. Nguyen and P. Milanfar, “A wavelet-based interpolation-restoration method for superresolution (wavelet superresolution),” Circuits, Systems, and Signal Processing, vol. 19, no. 4, pp. 321-338, 2000. · Zbl 1052.94500 · doi:10.1007/BF01200891
[8] S. Borman and R. L. Stevenson, “Super-resolution from image sequences-a review,” in Proceedings of the Midwest Symposium on Circuits and Systems, pp. 9-12, Notre Dame, Ind, USA, August 1998.
[9] A. J. Patti, M. I. Sezan, and A. M. Tekalp, “Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time,” IEEE Transactions on Image Processing, vol. 6, no. 8, pp. 1064-1076, 1997. · doi:10.1109/83.605404
[10] H. Stark and P. Oskoui, “High-resolution image recovery from image-plane arrays, using convex projections,” Journal of the Optical Society of America A, vol. 6, no. 11, pp. 1715-1726, 1989. · doi:10.1364/JOSAA.6.001715
[11] S. Peleg, D. Keren, and L. Schweitzer, “Improving image resolution using subpixel motion,” CVGIP: Graphical Models and Image Processing, vol. 54, pp. 181-186, 1992. · doi:10.1016/1049-9652(92)90065-6
[12] M. Irani and S. Peleg, “Improving resolution by image registration,” CVGIP: Graphical Models and Image Processing, vol. 53, no. 3, pp. 231-239, 1991. · doi:10.1016/1049-9652(91)90045-L
[13] M. Elad and A. Feuer, “Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images,” IEEE Transactions on Image Processing, vol. 6, no. 12, pp. 1646-1658, 1997. · doi:10.1109/83.650118
[14] M. Protter, M. Elad, H. Takeda, and P. Milanfar, “Generalizing the nonlocal-means to super-resolution reconstruction,” IEEE Transactions on Image Processing, vol. 18, no. 1, pp. 36-51, 2009. · Zbl 1371.94300 · doi:10.1109/TIP.2008.2008067
[15] H. Takeda, P. Milanfar, M. Protter, and M. Elad, “Super-resolution without explicit subpixel motion estimation,” IEEE Transactions on Image Processing, vol. 18, no. 9, pp. 1958-1975, 2009. · Zbl 1371.94363 · doi:10.1109/TIP.2009.2023703
[16] J. Chung, E. Haber, and J. Nagy, “Numerical methods for coupled super-resolution,” Inverse Problems, vol. 22, no. 4, pp. 1261-1272, 2006. · Zbl 1147.93302 · doi:10.1088/0266-5611/22/4/009
[17] R. C. Hardie, K. J. Barnard, and E. E. Armstrong, “Joint MAP registration and high-resolution image estimation using a sequence of undersampled images,” IEEE Transactions on Image Processing, vol. 6, no. 12, pp. 1621-1633, 1997. · doi:10.1109/83.650116
[18] N. A. Woods, N. P. Galatsanos, and A. K. Katsaggelos, “Stochastic methods for joint registration, restoration, and interpolation of multiple undersampled images,” IEEE Transactions on Image Processing, vol. 15, no. 1, pp. 201-213, 2006. · Zbl 05453423 · doi:10.1109/TIP.2005.860355
[19] H. Shen, L. Zhang, B. Huang, and P. Li, “A MAP approach to joint motion estimation, segmentation, and super resolution,” IEEE Transactions on Image Processing, vol. 16, no. 2, pp. 479-490, 2007. · Zbl 05453860 · doi:10.1109/TIP.2006.888334
[20] R. Sasahara, H. Hasegawa, I. Yamada, and K. Sakaniwa, “A color super-resolution with multiple nonsmooth constraints by hybrid steepest descent method,” in Proceedings of the IEEE International Conference on Image Processing (ICIP ’05), vol. 1, pp. 857-860, Genova, Italy, September 2005. · doi:10.1109/ICIP.2005.1529886
[21] S. Farsiu, M. Elad, and P. Milanfar, “Multiframe demosaicing and super-resolution of color images,” IEEE Transactions on Image Processing, vol. 15, no. 1, pp. 141-159, 2006. · Zbl 05452873 · doi:10.1109/TIP.2005.860336
[22] T. Akgun, Y. Altunbasak, and R. M. Mersereau, “Super-resolution reconstruction of hyperspectral images,” IEEE Transactions on Image Processing, vol. 14, no. 11, pp. 1860-1875, 2005. · Zbl 05452731 · doi:10.1109/TIP.2005.854479
[23] C. A. Segall, A. K. Katsaggelos, R. Molina, and J. Mateos, “Bayesian resolution enhancement of compressed video,” IEEE Transactions on Image Processing, vol. 13, no. 7, pp. 898-910, 2004. · Zbl 05453310 · doi:10.1109/TIP.2004.827230
[24] C. A. Segall, R. Molina, and A. K. Katsaggelos, “High-resolution images from low-resolution compressed video,” IEEE Signal Processing Magazine, vol. 20, no. 3, pp. 37-48, 2003. · doi:10.1109/MSP.2003.1203208
[25] N. Nguyen, P. Milanfar, and G. Golub, “A computationally efficient super-resolution image reconstruction algorithm,” IEEE Transactions on Image Processing, vol. 10, no. 4, pp. 573-583, 2001. · Zbl 1040.68567 · doi:10.1109/83.913592
[26] A. C. Yau, N. K. Bose, and M. K. Ng, “An efficient algorithm for superresolution in medium field imaging,” Multidimensional Systems and Signal Processing, vol. 18, no. 2-3, pp. 173-188, 2007. · Zbl 1147.68822 · doi:10.1007/s11045-007-0020-5
[27] N. K. Bose, M. K. Ng, and A. C. Yau, “A fast algorithm for image super-resolution from blurred observations,” EURASIP Journal on Applied Signal Processing, vol. 2006, 14 pages, 2006. · doi:10.1155/ASP/2006/35726
[28] M. K. Ng, H. Shen, E. Y. Lam, and L. Zhang, “A total variation regularization based super-resolution reconstruction algorithm for digital video,” EURASIP Journal on Advances in Signal Processing, vol. 2007, Article ID 74585, 16 pages, 2007. · Zbl 1168.94345 · doi:10.1155/2007/74585
[29] T. F. Chan, M. K. Ng, A. C. Yau, and A. M. Yip, “Superresolution image reconstruction using fast inpainting algorithms,” Applied and Computational Harmonic Analysis, vol. 23, no. 1, pp. 3-24, 2007. · Zbl 1116.94002 · doi:10.1016/j.acha.2006.09.005
[30] C. B. Xiao, J. Yu, and Y. Xue, “A high-efficiency super-resolution reconstruction algorithm from image/video sequences,” in Proceedings of the 3rd IEEE International Conference on Signal Image Technologies and Internet Based Systems (SITIS ’07), pp. 573-580, Shanghai, China, December 2007. · doi:10.1109/SITIS.2007.48
[31] M. Elad and Y. Hel-Or, “A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur,” IEEE Transactions on Image Processing, vol. 10, no. 8, pp. 1187-1193, 2001. · Zbl 1062.68590 · doi:10.1109/83.935034
[32] S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Transactions on Image Processing, vol. 13, no. 10, pp. 1327-1344, 2004. · Zbl 05452876 · doi:10.1109/TIP.2004.834669
[33] L. L. Huang, L. Xiao, Z. H. Wei, and J. Zhang, “A fast decoupling algorithm for image super-resolution reconstruction of space-invariant system,” Acta Automatica Sinica, vol. 36, no. 2, pp. 229-236, 2010 (Chinese). · doi:10.3724/SP.J.1004.2010.00229
[34] S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, “Advances and challenges in super-resolution,” International Journal of Imaging Systems and Technology, vol. 14, no. 2, pp. 47-57, 2004. · doi:10.1002/ima.20007
[35] L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Physica D, vol. 60, no. 1-4, pp. 259-268, 1992. · Zbl 0780.49028 · doi:10.1016/0167-2789(92)90242-F
[36] L. Zhang, H. Zhang, H. Shen, and P. Li, “A super-resolution reconstruction algorithm for surveillance images,” Signal Processing, vol. 90, no. 3, pp. 848-859, 2010. · Zbl 1177.94037 · doi:10.1016/j.sigpro.2009.09.002
[37] Y. Wang, W. Yin, and Y. Zhang, “A fast algorithm for image deblurring with total variation regularization,” Tech. Rep. 07-10, Department of Computational and Applied Mathematics, Rice University, Houston, Tex, USA, 2007.
[38] Y. Wang, J. Yang, W. Yin, and Y. Zhang, “A new alternating minimization algorithm for total variation image reconstruction,” SIAM Journal on Imaging Sciences, vol. 1, no. 3, pp. 248-272, 2008. · Zbl 1187.68665 · doi:10.1137/080724265
[39] Y. Huang, M. K. Ng, and Y.-W. Wen, “A fast total variation minimization method for image restoration,” Multiscale Modeling & Simulation, vol. 7, no. 2, pp. 774-795, 2008. · Zbl 1172.94316 · doi:10.1137/070703533
[40] T. Goldstein and S. Osher, “The split Bregman method for L1-regularized problems,” SIAM Journal on Imaging Sciences, vol. 2, no. 2, pp. 323-343, 2009. · Zbl 1177.65088 · doi:10.1137/080725891
[41] L. Xiao, L. L. Huang, and Z. H. Wei, “A weberized total variation regularization-based image multiplicative noise removal algorithm,” EURASIP Journal on Advances in Signal Processing, vol. 2010, Article ID 490384, 2010. · Zbl 05848129 · doi:10.1155/2010/490384
[42] H. G. Chen, Forward backward splitting techniques: theory and applications, Ph.D. thesis, University of Washington, 1994.
[43] S. Setzer, Splitting methods in image processing, Ph.D. thesis, University of Mannheim, 2009.
[44] J. Yang, Y. Zhang, and W. Yin, “A fast alternating direction method for TVL1-L2 signal reconstruction from partial Fourier data,” IEEE Journal on Selected Topics in Signal Processing, vol. 4, no. 2, pp. 288-297, 2010. · doi:10.1109/JSTSP.2010.2042333
[45] P. L. Combettes and J. C. Pesquet, “A Douglas-Rachford splitting approach to nonsmooth convex variational signal recovery,” IEEE Journal on Selected Topics in Signal Processing, vol. 1, no. 4, pp. 564-574, 2007. · doi:10.1109/JSTSP.2007.910264
[46] G. Steidl and T. Teuber, “Removing multiplicative noise by douglas-rachford splitting methods,” Journal of Mathematical Imaging and Vision, vol. 36, no. 2, pp. 168-184, 2010. · Zbl 1287.94016 · doi:10.1007/s10851-009-0179-5
[47] J. Eckstein and D. P. Bertsekas, “On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators,” Mathematical Programming, vol. 55, no. 3, pp. 293-318, 1992. · Zbl 0765.90073 · doi:10.1007/BF01581204
[48] E. Esser, “Applications of Lagrangian-based alternating direction methods and connections to split Bregman,” Tech. Rep. 09-31, UCLA CAM, 2009.
[49] S. Setzer, “Split Bregman algorithm, Douglas-Rachford splitting, and frame shrinkage,” in Proceedings of the 2nd International Conference on Scale Space Methods and Variational Methods in Computer Vision, vol. 5567 of Lecture Notes in Computer Science, pp. 464-476, Springer, 2009.
[50] S. Setzer, G. Steidl, and T. Teuber, “Deblurring Poissonian images by split Bregman techniques,” Journal of Visual Communication and Image Representation, vol. 21, no. 3, pp. 193-199, 2010. · Zbl 05742901 · doi:10.1016/j.jvcir.2009.10.006
[51] M. V. Afonso, J. M. Bioucas-Dias, and M. A. T. Figueiredo, “Fast image recovery using variable splitting and constrained optimization,” IEEE Transactions on Image Processing, vol. 19, no. 9, pp. 2345-2356, 2010. · Zbl 1371.94018 · doi:10.1109/TIP.2010.2047910
[52] D. Bertsekas, Nonlinear Programming, Athena Scientific, Belmont, Mass, USA, 1995. · Zbl 0935.90037
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.