Wavelet frame based blind image inpainting. (English) Zbl 1261.94006
Summary: Image inpainting has been widely used in practice to repair damaged/missing pixels of given images. Most of the existing inpainting techniques require knowing beforehand where those damaged pixels are, either given as a priori or detected by some pre-processing. However, in certain applications, such information is neither available nor can be reliably pre-detected, e.g., removing random-valued impulse noise from images or removing certain scratches from archived photographs. This paper introduces a blind inpainting model to solve this type of problems, i.e., a model of simultaneously identifying and recovering damaged pixels of the given image. A tight frame-based regularization approach is developed in this paper for such blind inpainting problems, and the resulting minimization problem is solved by the split Bregman algorithm first proposed by Goldstein and Osher (2009). The proposed blind inpainting method is applied to various challenging image restoration tasks, including recovering images that are blurry and damaged by scratches and removing image noise mixed with both Gaussian and random-valued impulse noise. The experiments show that our method compares favorably with many available two-staged methods in these applications.
MSC:
| 94A08 | Image processing (compression, reconstruction, etc.) in information and communication theory |
| 65T60 | Numerical methods for wavelets |
Keywords:
image inpainting; sparse approximation; split Bregman algorithm; wavelet frame; image restorationReferences:
| [1] | Goldstein, T.; Osher, S., The split Bregman algorithm for L1 regularized problems, SIAM J. Imaging Sci., 2, 2, 323-343 (2009) · Zbl 1177.65088 |
| [2] | M. Bertalmio, G. Sapiro, V. Caselles, C. Ballester, Image inpainting, in: SIGGRAPH, 2000, pp. 417-424.; M. Bertalmio, G. Sapiro, V. Caselles, C. Ballester, Image inpainting, in: SIGGRAPH, 2000, pp. 417-424. |
| [3] | Bertalmio, M.; Vese, L.; Sapiro, G.; Osher, S., Simultaneous structure and texture image inpainting, IEEE Trans. Image Process., 12, 8, 882-889 (2003) |
| [4] | Chan, T.; Shen, J., Variational image inpainting, Comm. Pure Appl. Math., 58, 579-619 (2005) · Zbl 1067.68168 |
| [5] | Chan, T.; Shen, J.; Zhou, H., Total variation wavelet inpainting, J. Math. Imaging Vision, 25, 1, 107-125 (2006) · Zbl 1478.94019 |
| [6] | Chan, T.; Kang, S.; Shen, J., Eulerʼs elastica and curvature-based inpainting, SIAM J. Appl. Math., 63, 2, 564-592 (2002) · Zbl 1028.68185 |
| [7] | Cai, J.; Chan, R.; Shen, L.; Shen, Z., Convergence analysis of tight framelet approach for missing data recovery, Adv. Comput. Math., 1-27 (2008) |
| [8] | Cai, J.; Chan, R.; Shen, Z., A framelet-based image inpainting algorithm, Appl. Comput. Harmon. Anal., 24, 2, 131-149 (2008) · Zbl 1135.68056 |
| [9] | Cai, J.; Chan, R.; Shen, Z., Simultaneous cartoon and texture inpainting, Inverse Probl. Imaging, 4, 3, 379-395 (2010) · Zbl 1198.94021 |
| [10] | Cai, J.; Chan, R.; Shen, Z., A framelet-based image inpainting algorithm, Appl. Comput. Harmon. Anal., 24, 2, 131-149 (2008) · Zbl 1135.68056 |
| [11] | Z. Shen, Wavelet frames and image restorations, in: Proceedings of the International Congress of Mathematicians, Hyderabad, India.; Z. Shen, Wavelet frames and image restorations, in: Proceedings of the International Congress of Mathematicians, Hyderabad, India. · Zbl 1228.42036 |
| [12] | B. Dong, Z. Shen, MRA based wavelet frames and applications, in: Summer Program on “The Mathematics of Image Processing”, in: IAS/Park City Math. Ser., Amer. Math. Soc.; B. Dong, Z. Shen, MRA based wavelet frames and applications, in: Summer Program on “The Mathematics of Image Processing”, in: IAS/Park City Math. Ser., Amer. Math. Soc. |
| [13] | Chan, T. F.; Shen, J., Image Processing and Analysis: Variational, PDE, Wavelet, and Stochastic Methods (2005), Society for Industrial Mathematics · Zbl 1095.68127 |
| [14] | Gonzalez, R. C.; Woods, R. E., Digital Image Processing (2002), Prentice Hall |
| [15] | H. Ji, Z. Shen, Y.-H. Xu, Robust video restoration by simultaneous sparse and low-rank matrices decomposition, Tech. Rep., NUS, 2010.; H. Ji, Z. Shen, Y.-H. Xu, Robust video restoration by simultaneous sparse and low-rank matrices decomposition, Tech. Rep., NUS, 2010. |
| [16] | Gilboa, G.; Osher, S., Nonlocal operators with applications to image processing, Multiscale Model. Simul., 7, 3, 1005-1028 (2008) · Zbl 1181.35006 |
| [17] | Ji, H.; Shen, Z.; Xu, Y.-H., Wavelet frame based image restoration with missing/damaged pixels, East Asia J. Appl. Math., 1, 2, 108-131 (2011) · Zbl 1286.94017 |
| [18] | A. Ayvaci, M. Raptis, S. Soatto, Optical flow and occlusion detection with convex optimization, in: Proc. of Neuro Information Processing Systems (NIPS), 2010.; A. Ayvaci, M. Raptis, S. Soatto, Optical flow and occlusion detection with convex optimization, in: Proc. of Neuro Information Processing Systems (NIPS), 2010. · Zbl 1235.68243 |
| [19] | Coifman, R.; Donoho, D., Translation-Invariant De-Noising, (Lecture Notes in Statist. (1995), Springer-Verlag: Springer-Verlag New York), 125 · Zbl 0866.94008 |
| [20] | Candes, E.; Donoho, D., New tight frames of curvelets and optimal representations of objects with C2 singularities, Comm. Pure Appl. Math., 56, 219-266 (2004) · Zbl 1038.94502 |
| [21] | Cai, J.; Osher, S.; Shen, Z., Split Bregman methods and frame based image restoration, Multiscale Model. Simul., 8, 2, 337-369 (2009) · Zbl 1189.94014 |
| [22] | Chaux, C.; Combettes, P.; Pesquet, J.; Wajs, V., A variational formulation for frame-based inverse problems, Inverse Problems, 23, 1495-1518 (2007) · Zbl 1141.65366 |
| [23] | Daubechies, I.; Teschke, G.; Vese, L., Iteratively solving linear inverse problems under general convex constraints, Inverse Probl. Imaging, 1, 1, 29 (2007) · Zbl 1123.65044 |
| [24] | Elad, M.; Starck, J.; Querre, P.; Donoho, D., Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA), Appl. Comput. Harmon. Anal., 19, 3, 340-358 (2005) · Zbl 1081.68732 |
| [25] | Fadili, M.; Starck, J.; Murtagh, F., Inpainting and zooming using sparse representations, Comput. J., 52, 1, 64 (2009) |
| [26] | Chai, A.; Shen, Z., Deconvolution: A wavelet frame approach, Numer. Math., 106, 4, 529-587 (2007) · Zbl 1120.65134 |
| [27] | Goldstein, T.; Bresson, X.; Osher, S., Geometric applications of the split Bregman method: Segmentation and surface reconstruction, J. Sci. Comput., 45, 1, 272-293 (2009) · Zbl 1203.65044 |
| [28] | Cai, J.; Osher, S.; Shen, Z., Linearized Bregman iterations for compressed sensing, Math. Comp., 78, 1515-1536 (2009) · Zbl 1198.65102 |
| [29] | Daubechies, I., Ten Lectures on Wavelets (1992), Society for Industrial Mathematics · Zbl 0776.42018 |
| [30] | Ron, A.; Shen, Z., Affine systems in \(L_2(R^d)\): The analysis of the analysis operator, J. Funct. Anal., 148, 2, 408-447 (1997) · Zbl 0891.42018 |
| [31] | Daubechies, I.; Han, B.; Ron, A.; Shen, Z., Framelets: MRA-based constructions of wavelet frames, Appl. Comput. Harmon. Anal., 14, 1, 1-46 (2003) · Zbl 1035.42031 |
| [32] | Cai, J.; Nikolova, M.; Chan, R., Fast two-phase image deblurring under impulse noise, J. Math. Imaging Vision, 36, 46-53 (2010) |
| [33] | Bar, L.; Kiryati, N.; Sochen, N., Image deblurring in the presence of impulse noise, Int. J. Comput. Vis., 70, 3, 279-298 (2006) · Zbl 1477.68502 |
| [34] | Bar, L.; Sochen, N.; Kiryati, N., Image deblurring in the presence of salt-and-pepper noise, (Scale Space and PDE Methods in Computer Vision. Scale Space and PDE Methods in Computer Vision, Lecture Notes in Comput. Sci., vol. 3459 (2005)), 107-118 · Zbl 1119.68520 |
| [35] | Healey, G.; Kondepudy, R., Radiometric ccd camera calibration and noise estimation, IEEE Trans. PAMI, 16, 3, 267-276 (1994) |
| [36] | Alliney, S., Digital filters as absolute norm regularizers, IEEE Trans. Signal Process., 40, 6, 1548-1562 (1992) · Zbl 0859.93037 |
| [37] | Chan, T.; Esedoglu, S., Aspects of total variation regularized \(L_1\) function approximation, SIAM J. Appl. Math., 65, 5, 1817 (2005) · Zbl 1096.94004 |
| [38] | Nikolova, M., A variational approach to remove outliers and impulse noise, J. Math. Imaging Vision, 20, 1, 99-120 (2004) · Zbl 1366.94065 |
| [39] | Zhang, X.; Burger, M.; Bresson, X.; Osher, S., Bregmanized nonlocal regularization for deconvolution and sparse reconstruction, SIAM J. Imaging Sci., 3, 1, 253-276 (2010) · Zbl 1191.94030 |
| [40] | E. Esser, Applications of Lagrangian-based alternating direction methods and connections to split Bregman, Tech. Rep. TR 09-31, UCLA, March 2009.; E. Esser, Applications of Lagrangian-based alternating direction methods and connections to split Bregman, Tech. Rep. TR 09-31, UCLA, March 2009. |
| [41] | Tai, X.; Wu, C., Augmented Lagrangian method, dual methods and split Bregman iteration for ROF model, Scale Space Var. Methods in Comput. Vision, 502-513 (2009) · Zbl 1371.94362 |
| [42] | Glowinski, R.; Le Tallec, P., Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics (1989), Society for Industrial Mathematics · Zbl 0698.73001 |
| [43] | Setzer, S., Split Bregman algorithm, Douglas-Rachford splitting and frame shrinkage, Scale Space Var. Methods in Comput. Vision, 464-476 (2009) |
| [44] | Donoho, D., De-noising by soft-thresholding, IEEE Trans. Inform. Theory, 41, 3, 613-627 (1995) · Zbl 0820.62002 |
| [45] | Combettes, P.; Wajs, V., Signal recovery by proximal forward-backward splitting, Multiscale Model. Simul., 4, 4, 1168-1200 (2006) · Zbl 1179.94031 |
| [46] | Dong, B.; Savitsky, E.; Osher, S., A Novel method for enhanced needle localization using ultrasound-guidance, Adv. Vis. Comput., 914-923 (2009) |
| [47] | Chen, T.; Wu, H. R., Adaptive impulse detection using center-weighted median filters, IEEE Signal Process. Lett., 8, 1, 1-3 (2001) |
| [48] | Hwang, H.; Haddad, R., Adaptive median filters: new algorithms and results, IEEE Trans. Image Process., 4, 4, 499-502 (1995) |
| [49] | Dong, Y.; Chan, R. F.; Xu, S., A detection statistic for random-valued impulse noise, IEEE Trans. Image Process., 16, 4, 1112-1120 (2007) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
