Large region inpainting by re-weighted regularized methods. (English) Zbl 1480.65054
Summary: In the development of imaging science and image processing request in our daily life, inpainting large regions always plays an important role. However, the existing local regularized models and some patch manifold based non-local models are often not effective in restoring the features and patterns in the large missing regions. In this paper, we will apply a strategy of inpainting from outside to inside and propose a re-weighted matching algorithm by closest patch (RWCP), contributing to further enhancing the features in the missing large regions. Additionally, we propose another re-weighted matching algorithm by distance-based weighted average (RWWA), leading to a result with higher PSNR value in some cases. Numerical simulations will demonstrate that for large region inpainting, the proposed method is more applicable than most canonical methods. Moreover, combined with image denoising methods, the proposed model is also applicable for noisy image restoration with large missing regions.
MSC:
| 65D18 | Numerical aspects of computer graphics, image analysis, and computational geometry |
| 68U10 | Computing methodologies for image processing |
References:
| [1] | M. Aharon, M. Elad and A. Bruckstein, An algorithm for designing overcomplete dictionaries for sparse representation, IEEE Transactions on Signal Processing, 54 (2006). · Zbl 1375.94040 |
| [2] | P. Arias, V. Caselles and G. Sapiro, A variational framework for non-local image inpainting, 08 (2009), 345-358. |
| [3] | M. Bertalmio, G. Sapiro, V. Caselles and C. Ballester, Image inpainting, Siggraph’00, (2000), 417-424. |
| [4] | M. Bertalmio; L. Vese; G. Sapiro; S. Osher, Simultaneous structure and texture image inpainting, IEEE Transactions on Image Processing, 12, 882-889 (2003) · doi:10.1109/TIP.2003.815261 |
| [5] | F. Bornemann; T. äMrz, Fast image inpainting based on coherence transport, Journal of Mathematical Imaging & Vision, 28, 259-278 (2007) · Zbl 1451.94007 · doi:10.1007/s10851-007-0017-6 |
| [6] | A. Buades; B. Coll; J.-M. Morel, A non-local algorithm for image denoising, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), 2, 60-65 (2005) · Zbl 1108.94004 |
| [7] | A. Buades, B. Coll and J.-M. Morel, Image enhancement by non-local reverse heat equation, Preprint CMLA, 22 (2006), 2006. |
| [8] | J.-F. Cai; R. H. Chan; L. Shen; Z. Shen, Convergence analysis of tight framelet approach for missing data recovery, Advances in Computational Mathematics, 31, 87-113 (2009) · Zbl 1172.94309 · doi:10.1007/s10444-008-9084-5 |
| [9] | J.-F. Cai; R. H. Chan; Z. Shen, A framelet-based image inpainting algorithm, Applied and Computational Harmonic Analysis, 24, 131-149 (2008) · Zbl 1135.68056 · doi:10.1016/j.acha.2007.10.002 |
| [10] | J.-F. Cai; R. H. Chan; Z. Shen, Simultaneous cartoon and texture inpainting, Inverse Probl. Imaging, 4, 379-395 (2010) · Zbl 1198.94021 · doi:10.3934/ipi.2010.4.379 |
| [11] | J.-F. Cai; H. Ji; Z. Shen; G.-B. Ye, Data-driven tight frame construction and image denoising, Applied and Computational Harmonic Analysis, 37, 89-105 (2014) · Zbl 1336.94008 · doi:10.1016/j.acha.2013.10.001 |
| [12] | R. Chan, L. Shen and Z. Shen, A framelet-based approach for image inpainting, Research Report, 4 (2005), 325. |
| [13] | T. F. Chan; S. H. Kang; J. Shen, Euler’s elastica and curvature-based inpainting, SIAM Journal on Applied Mathematics, 63, 564-592 (2002) · Zbl 1028.68185 · doi:10.1137/S0036139901390088 |
| [14] | T. F. Chan and J. Shen, Image Processing and Analysis: Variational, PDE, Wavelet, and Stochastic Methods, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2005. · Zbl 1095.68127 |
| [15] | T. F. Chan; J. Shen, Variational image inpainting, Commun. Pure Appl. Math, 58, 579-619 (2005) · Zbl 1067.68168 · doi:10.1002/cpa.20075 |
| [16] | T. F. Chan; J. Shen; H.-M. Zhou, Total variation wavelet inpainting, Journal of Mathematical Imaging and Vision, 25, 107-125 (2006) · Zbl 1478.94019 · doi:10.1007/s10851-006-5257-3 |
| [17] | A. Criminisi; P. Pérez; K. Toyama, Region filling and object removal by exemplar-based image inpainting, IEEE Transactions on Image Processing, 13, 1200-1212 (2004) |
| [18] | K. Dabov, A. Foi, V. Katkovnik and K. Egiazarian, Image denoising with block-matching and 3D filtering, in Electronic Imaging 2006, International Society for Optics and Photonics, (2006), 606414-606414. |
| [19] | B. Dong; H. Ji; J. Li; Z. Shen; Y. Xu, Wavelet frame based blind image inpainting, Applied and Computational Harmonic Analysis, 32, 268-279 (2012) · Zbl 1261.94006 · doi:10.1016/j.acha.2011.06.001 |
| [20] | B. Dong and Z. Shen, MRA based wavelet frames and applications, IAS Lecture Notes Series, Summer Program on “Mathematics of Image Processing”, IAS/Park City Math. Ser., 19, Amer. Math. Soc., Providence, RI, 2013, 9-158. · Zbl 1344.94009 |
| [21] | W. Dong; G. Shi; X. Li; Y. Ma; F. Huang, Compressive sensing via nonlocal low-rank regularization, IEEE Transactions on Image Processing, 23, 3618-3632 (2014) · Zbl 1374.94085 · doi:10.1109/TIP.2014.2329449 |
| [22] | W. Dong; G. Shi; X. li, Nonlocal image restoration with bilateral variance estimation: A low-rank approach, IEEE Transactions on Image Processing, 22, 700-711 (2013) · Zbl 1373.94102 · doi:10.1109/TIP.2012.2221729 |
| [23] | M. Elad; J.-L. Starck; P. Querre; D. L. Donoho, Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA), Applied and Computational Harmonic Analysis, 19, 340-358 (2005) · Zbl 1081.68732 · doi:10.1016/j.acha.2005.03.005 |
| [24] | M. J. Fadili; J. L. Starck; F. Murtagh, Inpainting and zooming using sparse representations, Comput. J., 52, 64-79 (2009) |
| [25] | G. Gilboa; S. Osher, Nonlocal operators with applications to image processing, Multiscale Model. Simul., 7, 1005-1028 (2008) · Zbl 1181.35006 · doi:10.1137/070698592 |
| [26] | G. Gilboa, N. Sochen and Y. Zeevi, Image enhancement and denoising by complex diffusion processes, Pattern Analysis and Machine Intelligence, IEEE Transactions on, 26 (2004), 1020-1036. |
| [27] | G. Gilboa; N. Sochen; Y. Y. Zeevi, Forward-and-backward diffusion processes for adaptive image enhancement and denoising, IEEE Transactions on Image Processing, 11, 689-703 (2002) · doi:10.1109/TIP.2002.800883 |
| [28] | R. Glowinski and P. Le Tallec, Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics, SIAM, 1989. · Zbl 0698.73001 |
| [29] | H. Ji; Z. Shen; Y. Xu, Wavelet frame based image restoration with missing/damaged pixels, East Asia Journal on Applied Mathematics, 1, 108-131 (2011) · Zbl 1286.94017 |
| [30] | R. Lai; J. Li, Manifold based low-rank regularization for image restoration and semi-supervised learning, Journal of Scientific Computing, 74, 1241-1263 (2018) · Zbl 1489.94014 · doi:10.1007/s10915-017-0492-x |
| [31] | F. Li; T. Zeng, A universal variational framework for sparsity-based image inpainting, IEEE Transactions on Image Processing, 23, 4242-4254 (2014) · Zbl 1374.94201 · doi:10.1109/TIP.2014.2346030 |
| [32] | T. März, Image inpainting based on coherence transport with adapted distance functions, SIAM Journal on Imaging Sciences, 4, 981-1000 (2011) · Zbl 1232.65038 · doi:10.1137/100807296 |
| [33] | A. Newson; A. Almansa; Y. Gousseau; P. Pérez, Non-local patch-based image inpainting, IPOL J. Image Processing On Line, 7, 373-385 (2017) · doi:10.5201/ipol.2017.189 |
| [34] | S. Osher; Z. Shi; W. Zhu, Low dimensional manifold model for image processing, SIAM Journal on Imaging Sciences, 10, 1669-1690 (2017) · Zbl 1401.49042 · doi:10.1137/16M1058686 |
| [35] | A. Ron; Z. Shen, Affine Systems in \(\begin{document} L_2(\mathbb{R}^d)\end{document} \): The Analysis of the Analysis Operator, Journal of Functional Analysis, 148, 408-447 (1997) · Zbl 0891.42018 · doi:10.1006/jfan.1996.3079 |
| [36] | Z. Shen, Wavelet frames and image restorations, Proceedings of the International Congress of Mathematicians, Volume IV, Hindustan Book Agency, New Delhi, 2010, 2834-2863. · Zbl 1228.42036 |
| [37] | Z. Shi, S. Osher and W. Zhu, Weighted graph laplacian and image inpainting, Journal of Scientific Computing, 577 (2017). |
| [38] | J. Yu, Z. Lin, J. Yang, X. Shen, X. Lu and T. Huang, Generative image inpainting with contextual attention, 06 (2018), 5505-5514. |
| [39] | K. Zhang; W. Zuo; Y. Chen; D. Meng; L. Zhang, Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising, IEEE Transactions on Image Processing, 26, 3142-3155 (2017) · Zbl 1409.94754 · doi:10.1109/TIP.2017.2662206 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
