Sparsity-promoting and edge-preserving maximum a posteriori estimators in non-parametric Bayesian inverse problems. (English) Zbl 06866427
Summary: We consider the inverse problem of recovering an unknown functional parameter \(u\) in a separable Banach space, from a noisy observation vector \(y\) of its image through a known possibly non-linear map \({{\mathcal G}}\) . We adopt a Bayesian approach to the problem and consider Besov space priors (see [M. Lassas et al., Inverse Probl. Imaging 3, No. 1, 87–122 (2009; Zbl 1191.62046)]), which are well-known for their edge-preserving and sparsity-promoting properties and have recently attracted wide attention especially in the medical imaging community.
Our key result is to show that in this non-parametric setup the maximum a posteriori (MAP) estimates are characterized by the minimizers of a generalized Onsager-Machlup functional of the posterior. This is done independently for the so-called weak and strong MAP estimates, which as we show coincide in our context. In addition, we prove a form of weak consistency for the MAP estimators in the infinitely informative data limit. Our results are remarkable for two reasons: first, the prior distribution is non-Gaussian and does not meet the smoothness conditions required in previous research on non-parametric MAP estimates. Second, the result analytically justifies existing uses of the MAP estimate in finite but high dimensional discretizations of Bayesian inverse problems with the considered Besov priors.
Our key result is to show that in this non-parametric setup the maximum a posteriori (MAP) estimates are characterized by the minimizers of a generalized Onsager-Machlup functional of the posterior. This is done independently for the so-called weak and strong MAP estimates, which as we show coincide in our context. In addition, we prove a form of weak consistency for the MAP estimators in the infinitely informative data limit. Our results are remarkable for two reasons: first, the prior distribution is non-Gaussian and does not meet the smoothness conditions required in previous research on non-parametric MAP estimates. Second, the result analytically justifies existing uses of the MAP estimate in finite but high dimensional discretizations of Bayesian inverse problems with the considered Besov priors.
MSC:
| 65J22 | Numerical solution to inverse problems in abstract spaces |
| 62G07 | Density estimation |
| 62F15 | Bayesian inference |
Citations:
Zbl 1191.62046Software:
L1-MAGICReferences:
| [1] | Agapiou S, Larsson S and Stuart A M 2013 Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems {\it Stoch. Process. Appl.}123 3828-60 · Zbl 1284.62289 |
| [2] | Agapiou S, Stuart A M and Zhang Y X 2014 Bayesian posterior contraction rates for linear severely ill-posed inverse problems {\it J. Inverse Ill-Posed Problems}22 297-321 · Zbl 1288.62036 |
| [3] | Anderson T W 1955 The integral of a symmetric unimodal function over a symmetric convex set and some probability inequalities {\it Proc. Am. Math. Soc.}6 170-6 · Zbl 0066.37402 |
| [4] | Bardsley J M, Calvetti D and Somersalo E 2010 Hierarchical regularization for edge-preserving reconstruction of pet images {\it Inverse Problems}26 035010 · Zbl 1188.41023 |
| [5] | Bogachev V I 1998 {\it Gaussian Measures}{\it (Mathematical Surveys and Monographs vol 62)} (Providence, RI: American Mathematical Society) · Zbl 0913.60035 |
| [6] | Bogachev V I 2010 {\it Differentiable Measures and the Malliavin Calculus}{\it (Mathematical Surveys and Monographs vol 164)} (Providence, RI: American Mathematical Society) · Zbl 1247.28001 |
| [7] | Bogachev V I and Smolyanov O G 1990 Analytic properties of infinite-dimensional distributions {\it Usp. Mat. Nauk}45 3-83, 221 · Zbl 0708.60049 |
| [8] | Borell C 1974 Convex measures on locally convex spaces {\it Ark. Mat.}12 239-52 · Zbl 0297.60004 |
| [9] | Bui-Thanh T and Ghattas O 2015 A scalable algorithm for MAP estimators in Bayesian inverse problems with Besov priors {\it Inverse Problems Imaging}9 27-53 · Zbl 1347.49045 |
| [10] | Calvetti D and Somersalo E 2007 A gaussian hypermodel to recover blocky objects {\it Inverse Problems}23 733 · Zbl 1112.62018 |
| [11] | Calvetti D and Somersalo E 2008 Hypermodels in the bayesian imaging framework {\it Inverse Problems}24 034013 · Zbl 1137.62062 |
| [12] | Calvetti D, Somersalo E and Spies R 2015 Variable order smoothness priors for ill-posed inverse problems {\it Math. Comput.}84 1753-73 · Zbl 1309.65045 |
| [13] | Candes E and Romberg J 2005 l1-magic: recovery of sparse signals via convex programming 4 14 www.acm.caltech.edu/l1magic/downloads/l1magic.pdf |
| [14] | Candes E J and Donoho D L 2000 Curvelets: a surprisingly effective nonadaptive representation for objects with edges {\it Technical Report} |
| [15] | Da Prato G 2006 {\it An Introduction to Infinite-Dimensional Analysis} (Berlin: Springer) · Zbl 1109.46001 |
| [16] | Dashti M and Stuart A M 2015 The Bayesian approach to inverse problems {\it Handbook of Uncertainty Quantification} ed R Ghanem {\it et al} (Berlin: Springer) |
| [17] | Dashti M, Harris S and Stuart A 2012 Besov priors for Bayesian inverse problems {\it Inverse Problem Imaging}6 183-200 · Zbl 1243.62032 |
| [18] | Dashti M, Law K J H, Stuart A M and Voss J 2013 MAP estimators and their consistency in Bayesian nonparametric inverse problems {\it Inverse Problems}29 095017 · Zbl 1281.62089 |
| [19] | Daubechies I 1992 {\it Ten Lectures on Wavelets}{\it (CBMS-NSF Regional Conference Series in Applied Mathematics vol 61)} (Philadelphia, PA: Society for Industrial and Applied Mathematics) · Zbl 0776.42018 |
| [20] | Daubechies I, Defrise M and De Mol C 2004 An iterative thresholding algorithm for linear inverse problems with a sparsity constraint {\it Commun. Pure Appl. Math.}57 1413-57 · Zbl 1077.65055 |
| [21] | Dharmadhikari S and Joag-Dev K 1988 {\it Unimodality, Convexity, and Applications} (New York: Elsevier) · Zbl 0646.62008 |
| [22] | Dobson D C and Santosa F 1994 An image-enhancement technique for electrical impedance tomography {\it Inverse Problems}10 317 · Zbl 0805.35149 |
| [23] | Dobson D C and Santosa F 1996 Recovery of blocky images from noisy and blurred data {\it SIAM J. Appl. Math.}56 1181-98 · Zbl 0858.68121 |
| [24] | Donoho D L and Elad M 2003 Optimally sparse representation in general (nonorthogonal) dictionaries via 1 minimization {\it Proc. Natl Acad. Sci.}100 2197-202 · Zbl 1064.94011 |
| [25] | Donoho D L and Huo X 2001 Uncertainty principles and ideal atomic decomposition {\it IEEE Trans. Inf. Theory}47 2845-62 · Zbl 1019.94503 |
| [26] | Dunlop M and Stuart A 2016 MAP estimators for piecewise continuous inversion {\it Inverse Problems}32 105003 · Zbl 1354.65112 |
| [27] | Dürr D and Bach A 1978 The Onsager-Machlup function as Lagrangian for the most probable path of a diffusion process {\it Commun. Math. Phys.}60 153-70 · Zbl 0377.60044 |
| [28] | Engl H W, Hanke M and Neubauer A 1996 {\it Regularization of Inverse Problems} vol 375 (Berlin: Springer) · Zbl 0859.65054 |
| [29] | Franklin J N 1970 Well-posed stochastic extensions of ill-posed linear problems {\it J. Math. Anal. Appl.}31 682-716 · Zbl 0198.20601 |
| [30] | Gerth D and Ramlau R 2014 A stochastic convergence analysis for Tikhonov regularization with sparsity constraints {\it Inverse Problems}30 055009 · Zbl 1317.47014 |
| [31] | Helin T 2009 On infinite-dimensional hierarchical probability models in statistical inverse problems {\it Inverse Problems Imaging}3 567-97 · Zbl 1232.62047 |
| [32] | Helin T and Burger M 2015 Maximum a posteriori probability estimates in infinite-dimensional Bayesian inverse problems {\it Inverse Problems}31 085009 · Zbl 1325.62058 |
| [33] | Helin T and Lassas M 2011 Hierarchical models in statistical inverse problems and the Mumford-Shah functional {\it Inverse Problems}27 015008 · Zbl 1215.65022 |
| [34] | Ikeda N and Watanabe S 2014 {\it Stochastic Differential Equations and Diffusion Processes} vol 24 (New York: Elsevier) · Zbl 0495.60005 |
| [35] | Jia J, Peng J and Gao J 2016 Bayesian approach to inverse problems for functions with a variable-index Besov prior {\it Inverse Problems}32 085006 · Zbl 1366.62048 |
| [36] | Johnstone I M 1994 Minimax bayes, asymptotic minimax and sparse wavelet priors {\it Statistical Decision Theory and Related Topics} (Berlin: Springer) pp 303-26 · Zbl 0815.62017 |
| [37] | Kaipio J and Somersalo E 2006 {\it Statistical and Computational Inverse Problems} vol 160 (Berlin: Springer) · Zbl 1068.65022 |
| [38] | Kallenberg O 2002 {\it Probability and its Applications} 2nd edn (New York: Springer) · Zbl 0996.60001 |
| [39] | Kekkonen H, Lassas M and Siltanen S 2016 Posterior consistency and convergence rates for bayesian inversion with hypoelliptic operators {\it Inverse Problems}32 085005 · Zbl 1390.35422 |
| [40] | Knapik B T and Salomond J B 2014 A general approach to posterior contraction in nonparametric inverse problems {\it Bernoulli}24 2091-21 · Zbl 1419.62089 |
| [41] | Knapik B T, Szabó B T, Vaart A W and Zanten J H 2016 Bayes procedures for adaptive inference in inverse problems for the white noise model {\it Probab. Theory Relat. Fields}164 771-813 · Zbl 1334.62039 |
| [42] | Knapik B T, van Der Vaart A W and van Zanten J H 2011 Bayesian inverse problems with Gaussian priors {\it Ann. Stat.}39 2626-57 · Zbl 1232.62079 |
| [43] | Knapik B T, van der Vaart A W and van Zanten J H 2013 Bayesian recovery of the initial condition for the heat equation {\it Commun. Stat.-Theory Methods}42 1294-313 · Zbl 1347.62057 |
| [44] | Kolehmainen V, Lassas M, Niinimäki K and Siltanen S 2012 Sparsity-promoting Bayesian inversion {\it Inverse Problems}28 025005 · Zbl 1233.62046 |
| [45] | Kolehmainen V, Siltanen S, Järvenpää S, Kaipio J, Koistinen P, Lassas M, Pirttilä J and Somersalo E 2003 Statistical inversion for medical x-ray tomography with few radiographs: II. Application to dental radiology {\it Phys. Med. Biol.}48 1465 |
| [46] | Lasanen S 2012 Non-Gaussian statistical inverse problems. Part i: posterior distributions {\it Inverse Problems Imaging}6 215-66 · Zbl 1263.62041 |
| [47] | Lassas M and Siltanen S 2004 Can one use total variation prior for edge-preserving Bayesian inversion? {\it Inverse Problems}20 1537-63 · Zbl 1062.62260 |
| [48] | Lassas M, Saksman E and Siltanen S 2009 Discretization-invariant Bayesian inversion and Besov space priors {\it Inverse Problems Imaging}3 87-122 · Zbl 1191.62046 |
| [49] | Lehtinen M S, Paivarinta L and Somersalo E 1989 Linear inverse problems for generalised random variables {\it Inverse Problems}5 599 · Zbl 0681.60015 |
| [50] | Leporini D and Pesquet J C 2001 Bayesian wavelet denoising: Besov priors and non-Gaussian noises {\it Signal Process.}81 55-67 · Zbl 1098.94560 |
| [51] | Lie H C and Sullivan T J 2017 Equivalence of weak and strong modes of measures on topological vector spaces (arXiv:1708.02516) · Zbl 1465.65047 |
| [52] | Loris I, Nolet G, Daubechies I and Dahlen F 2007 Tomographic inversion using 1-norm regularization of wavelet coefficients {\it Geophys. J. Int.}170 359-70 |
| [53] | Mandelbaum A 1984 Linear estimators and measurable linear transformations on a Hilbert space {\it Z. Wahrsch. Verwandte Geb.}65 385-97 · Zbl 0506.60004 |
| [54] | Papaspiliopoulos O, Pokern Y, Roberts G O and Stuart A M 2012 Nonparametric estimation of diffusions: a differential equations approach {\it Biometrika}99 511-31 · Zbl 1437.62575 |
| [55] | Perona P and Malik J 1990 Scale-space and edge detection using anisotropic diffusion {\it IEEE Trans. Pattern Anal. Mach. Intell.}12 629-39 |
| [56] | Pokern Y, Stuart A M and van Zanten J H 2013 Posterior consistency via precision operators for Bayesian nonparametric drift estimation in SDEs {\it Stoch. Process. Appl.}123 603-28 · Zbl 1256.62047 |
| [57] | Pulkkinen A, Cox B T, Arridge S R, Kaipio J P and Tarvainen T 2016 Bayesian parameter estimation in spectral quantitative photoacoustic tomography {\it Proc. SPIE}9708 97081G · Zbl 1301.92044 |
| [58] | Rantala M, Vanska S, Jarvenpaa S, Kalke M, Lassas M, Moberg J and Siltanen S 2006 Wavelet-based reconstruction for limited-angle x-ray tomography {\it IEEE Trans. Med. Imaging}25 210-7 |
| [59] | Ray K 2013 Bayesian inverse problems with non-conjugate priors {\it Electron. J. Stat.}7 2516-49 · Zbl 1294.62107 |
| [60] | Rudin L I, Osher S and Fatemi E 1992 Nonlinear total variation based noise removal algorithms {\it Physica} D 60 259-68 · Zbl 0780.49028 |
| [61] | Rudin W 1987 {\it Real and Complex Analysis} 3rd edn (New York: McGraw-Hill) · Zbl 0925.00005 |
| [62] | Shepp L 1965 Distinguishing a sequence of random variables from a translate of itself {\it Ann. Math. Stat.}36 1107-12 · Zbl 0136.40108 |
| [63] | Siltanen S, Kolehmainen V, Järvenpää S, Kaipio J, Koistinen P, Lassas M, Pirttilä J and Somersalo E 2003 Statistical inversion for medical x-ray tomography with few radiographs: I. General theory {\it Phys. Med. Biol.}48 1437 |
| [64] | Stern A S, Donoho D L and Hoch J C 2007 NMR data processing using iterative thresholding and minimum l 1-norm reconstruction {\it J. Magn. Reson.}188 295-300 |
| [65] | Strong D and Chan T 2003 Edge-preserving and scale-dependent properties of total variation regularization {\it Inverse Problems}19 S165 · Zbl 1043.94512 |
| [66] | Stuart A M 2010 Inverse problems: a Bayesian perspective {\it Acta Numer.}19 451-559 · Zbl 1242.65142 |
| [67] | Sullivan T J 2017 Well-posed Bayesian inverse problems and heavy-tailed stable quasi-Banach space priors {\it Inverse Problems Imaging}11 857-74 · Zbl 1368.65085 |
| [68] | Szabó B T, van der Vaart A W and van Zanten J H 2013 Empirical Bayes scaling of Gaussian priors in the white noise model {\it Electron. J. Stat.}7 991-1018 · Zbl 1336.62039 |
| [69] | Tarantola A 2005 {\it Inverse Problem Theory and Methods for Model Parameter Estimation} (Philadelphia: SIAM) · Zbl 1074.65013 |
| [70] | Tick J, Pulkkinen A and Tarvainen T 2016 Image reconstruction with uncertainty quantification in photoacoustic tomography {\it J. Acoust. Soc. Am.}139 1951-61 |
| [71] | van Waaij J and van Zanten H 2016 Gaussian process methods for one-dimensional diffusions: optimal rates and adaptation {\it Electron. J. Stat.}10 628-45 · Zbl 1403.62152 |
| [72] | Vollmer S J 2013 Posterior consistency for Bayesian inverse problems through stability and regression results {\it Inverse Problems}29 125011 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
