Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems. (English) Zbl 1161.65308
Summary: We consider a Bayesian approach to nonlinear inverse problems in which the unknown quantity is a spatial or temporal field, endowed with a hierarchical Gaussian process prior. Computational challenges in this construction arise from the need for repeated evaluations of the forward model (e.g., in the context of Markov chain Monte Carlo) and are compounded by high dimensionality of the posterior. We address these challenges by introducing truncated Karhunen-Loève expansions, based on the prior distribution, to efficiently parameterize the unknown field and to specify a stochastic forward problem whose solution captures that of the deterministic forward model over the support of the prior. We seek a solution of this problem using Galerkin projection on a polynomial chaos basis, and use the solution to construct a reduced-dimensionality surrogate posterior density that is inexpensive to evaluate. We demonstrate the formulation on a transient diffusion equation with prescribed source terms, inferring the spatially-varying diffusivity of the medium from limited and noisy data.
MSC:
| 65C30 | Numerical solutions to stochastic differential and integral equations |
| 60H15 | Stochastic partial differential equations (aspects of stochastic analysis) |
| 60H35 | Computational methods for stochastic equations (aspects of stochastic analysis) |
| 35R30 | Inverse problems for PDEs |
| 35R60 | PDEs with randomness, stochastic partial differential equations |
| 46E22 | Hilbert spaces with reproducing kernels (= (proper) functional Hilbert spaces, including de Branges-Rovnyak and other structured spaces) |
Keywords:
inverse problems; Bayesian inference; dimensionality reduction; polynomial chaos; Markov chain Monte Carlo; Galerkin projection; Gaussian processes; Karhunen-Loève expansion; reproducing kernel Hilbert space; numerical examplesReferences:
| [1] | Vogel, C. R., Computational Methods for Inverse Problems (2002), SIAM · Zbl 1008.65103 |
| [2] | Evans, S. N.; Stark, P. B., Inverse problems as statistics, Inverse Problems, 18, R55-R97 (2002) · Zbl 1039.62007 |
| [3] | Tenorio, L., Statistical regularization of inverse problems, SIAM Rev., 43, 2, 347-366 (2001) · Zbl 0976.65114 |
| [4] | Kaipio, J.; Somersalo, E., Statistical and Computational Inverse Problems (2005), Springer · Zbl 1068.65022 |
| [5] | Tarantola, A., Inverse Problem Theory and Methods for Model Parameter Estimation (2005), SIAM · Zbl 1074.65013 |
| [6] | Marzouk, Y. M.; Najm, H. N.; Rahn, L. A., Stochastic spectral methods for efficient Bayesian solution of inverse problems, J. Comput. Phys., 224, 2, 560-586 (2007) · Zbl 1120.65306 |
| [7] | Wiener, N., The homogeneous chaos, Am. J. Math., 60, 897-936 (1938) · JFM 64.0887.02 |
| [8] | Cameron, R.; Martin, W., The orthogonal development of nonlinear functionals in series of Fourier-Hermite functionals, Ann. Math., 48, 385-392 (1947) · Zbl 0029.14302 |
| [9] | Ghanem, R.; Spanos, P., Stochastic Finite Elements: A Spectral Approach (1991), Springer-Verlag: Springer-Verlag New York · Zbl 0722.73080 |
| [10] | Xiu, D.; Karniadakis, G., The Wiener-Askey polynomial chaos for stochastic differential equations, SIAM J. Sci. Comput., 24, 2, 619-644 (2002) · Zbl 1014.65004 |
| [11] | Debusschere, B.; Najm, H.; Pébay, P.; Knio, O.; Ghanem, R.; Le Maître, O., Numerical challenges in the use of polynomial chaos representations for stochastic processes, SIAM J. Sci. Comput., 26, 2, 698-719 (2004) · Zbl 1072.60042 |
| [12] | Wan, X.; Karniadakis, G. E., An adaptive multi-element generalized polynomial chaos method for stochastic differential equations, J. Comput. Phys., 209, 617-642 (2005) · Zbl 1078.65008 |
| [13] | Ghanem, R., Probabilistic characterization of transport in heterogeneous media, Comput. Methods Appl. Mech. Eng., 158, 199-220 (1998) · Zbl 0954.76079 |
| [14] | R. Ghanem, J. Red-Horse, A. Sarkar, Modal properties of a space-frame with localized system uncertainties, in: A. Kareem, A. Haldar, B.S., Jr., E. Johnson (Eds.), 8th ASCE Specialty Conference of Probabilistic Mechanics and Structural Reliability, PMC200-269, ASCE, 2000.; R. Ghanem, J. Red-Horse, A. Sarkar, Modal properties of a space-frame with localized system uncertainties, in: A. Kareem, A. Haldar, B.S., Jr., E. Johnson (Eds.), 8th ASCE Specialty Conference of Probabilistic Mechanics and Structural Reliability, PMC200-269, ASCE, 2000. |
| [15] | Le Maître, O.; Knio, O.; Najm, H.; Ghanem, R., A stochastic projection method for fluid flow I. Basic formulation, J. Comput. Phys., 173, 481-511 (2001) · Zbl 1051.76056 |
| [16] | Le Maître, O.; Reagan, M.; Najm, H.; Ghanem, R.; Knio, O., A stochastic projection method for fluid flow II. Random process, J. Comput. Phys., 181, 9-44 (2002) · Zbl 1052.76057 |
| [17] | Xiu, D.; Lucor, D.; Su, C.-H.; Karniadakis, G., Stochastic modeling of flow-structure interactions using generalized polynomial chaos, ASME J. Fluids Eng., 124, 51-59 (2002) |
| [18] | Debusschere, B.; Najm, H.; Matta, A.; Knio, O.; Ghanem, R.; Le Maître, O., Protein labeling reactions in electrochemical microchannel flow: numerical simulation and uncertainty propagation, Phys. Fluids, 15, 8, 2238-2250 (2003) · Zbl 1186.76133 |
| [19] | Reagan, M.; Najm, H.; Debusschere, B.; Le Maître, O.; Knio, O.; Ghanem, R., Spectral stochastic uncertainty quantification in chemical systems, Comb. Theory Model., 8, 607-632 (2004) |
| [20] | Wang, J.; Zabaras, N., Using Bayesian statistics in the estimation of heat source in radiation, Int. J. Heat Mass Transfer, 48, 15-29 (2005) · Zbl 1122.80307 |
| [21] | Berkooz, G.; Holmes, P.; Lumley, J. L., The proper orthogonal decomposition in the analysis of turbulent flows, Annu. Rev. Fluid Mech., 25, 539-575 (1993) |
| [22] | Balakrishnan, S.; Roy, A.; Ierapetritou, M. G.; Flach, G. P.; Georgopoulos, P. G., Uncertainty reduction and characterization for complex environmental fate and transport models: an empirical Bayesian framework incorporating the stochastic response surface method, Water Resour. Res., 39, 12, 1350 (2003) |
| [23] | Kennedy, M. C.; O’Hagan, A., Bayesian calibration of computer models, J. Royal Stat. Soc.: Series B, 63, 3, 425-464 (2001) · Zbl 1007.62021 |
| [24] | Christen, J. A.; Fox, C., MCMC using an approximation, J. Comput. Graph. Stat., 14, 4, 795-810 (2005) |
| [25] | Higdon, D.; Lee, H.; Holloman, C., Markov chain Monte Carlo-based approaches for inference in computationally intensive inverse problems, Bayesian Stat., 7, 181-197 (2003) |
| [26] | Geyer, C. J., Markov chain Monte Carlo maximum likelihood, (Keramidas, E. M., Computing Science and Statistics: Proceedings of the 23rd Symposium on the Interface, vol. 23 (1991), Interface Foundation of North America), 156-163 |
| [27] | Efendiev, Y.; Hou, T. Y.; Luo, W., Preconditioning Markov chain Monte Carlo simulations using coarse-scale models, SIAM J. Sci. Comput., 28, 776-803 (2006) · Zbl 1111.65003 |
| [28] | Lee, H.; Higdon, D.; Bi, Z.; Ferreira, M.; West, M., Markov random field models for high-dimensional parameters in simulations of fluid flow in porous media, Technometrics, 44, 3, 230-241 (2002) |
| [29] | Wang, J.; Zabaras, N., A Markov random field model of contamination source identification in porous media flow, Int. J. Heat Mass Transfer, 49, 939-950 (2006) · Zbl 1189.76615 |
| [30] | Kaipio, J.; Somersalo, E., Statistical inverse problems: discretization, model reduction, and inverse crimes, J. Comput. Appl. Math., 198, 2, 493-504 (2007) · Zbl 1101.65008 |
| [31] | Li, W.; Cirpka, O. A., Efficient geostatistical inverse methods for structured and unstructured grids, Water Resour. Res., 42, W06402 (2006) |
| [32] | Xiu, D.; Karniadakis, G., Modeling uncertainty in steady state diffusion problems via generalized polynomial chaos, Comput. Methods Appl. Mech. Eng., 191, 4927-4948 (2002) · Zbl 1016.65001 |
| [33] | Le Maître, O.; Reagan, M.; Debusschere, B.; Najm, H.; Ghanem, R.; Knio, O., Natural convection in a closed cavity under stochastic, non-Boussinesq conditions, SIAM J. Sci. Comput., 26, 2, 375-394 (2004) · Zbl 1075.60077 |
| [34] | Frauenfelder, P.; Schwab, C.; Todor, R., Finite elements for elliptic problems with stochastic coefficients, Comput. Methods Appl. Mech. Eng., 194, 205-228 (2005) · Zbl 1143.65392 |
| [35] | Matthies, H. G.; Keese, A., Galerkin methods for linear and nonlinear elliptic stochastic partial differential equations, Comput. Methods Appl. Mech. Eng., 194, 1295-1331 (2005) · Zbl 1088.65002 |
| [36] | Grigoriu, M., Stochastic Calculus (2002), Birkhäuser · Zbl 1015.60001 |
| [37] | Hoel, P. G.; Port, S. C.; Stone, C. J., Introduction to Stochastic Processes (1987), Waveland Press |
| [38] | Seeger, M., Gaussian processes for machine learning, Int. J. Neural Syst., 14, 2, 1-38 (2004) |
| [39] | Williams, C. K.I.; Rasmussen, C. E., Gaussian processes for regression, (Teuretzky, D. S.; Mozer, M. C.; Hasselmo, M. E., Advances in Neural Information Processing Systems, vol. 8 (1996), MIT Press) |
| [40] | Cressie, N. A.C., Statistics for Spatial Data (1993), Wiley · Zbl 0468.62095 |
| [41] | Rasmussen, C. E.; Williams, C. K.I., Gaussian Processes for Machine Learning (2006), MIT Press · Zbl 1177.68165 |
| [42] | Genton, M. G., Classes of kernels for machine learning: a statistics perspective, J. Mach. Learning Res., 2, 299-312 (2001) · Zbl 1037.68113 |
| [43] | Loève, M., Probability Theory II (1978), Springer · Zbl 0385.60001 |
| [44] | Adler, R. J.; Taylor, J. E., Random Fields and Geometry (2007), Springer · Zbl 1149.60003 |
| [45] | P. Abrahamsen, A review of Gaussian random fields and correlation functions, Technical Report 917, Norwegian Computing Center, Oslo, Norway, 1997.; P. Abrahamsen, A review of Gaussian random fields and correlation functions, Technical Report 917, Norwegian Computing Center, Oslo, Norway, 1997. |
| [46] | Courant, R.; Hilbert, D., Methods of Mathematical Physics, vol. 1 (1953), Wiley-Interscience · Zbl 0729.00007 |
| [47] | Sirovich, L., Turbulence and the dynamics of coherent structures. Part 1: coherent structures, Quart. Appl. Math., 45, 3, 561-571 (1987) · Zbl 0676.76047 |
| [48] | A. Mohammad-Djafari, Bayesian inference for inverse problems, in: Bayesian inference and Maximum Entropy Methods in Science and Engineering, vol. 21, 2002, pp. 477-496.; A. Mohammad-Djafari, Bayesian inference for inverse problems, in: Bayesian inference and Maximum Entropy Methods in Science and Engineering, vol. 21, 2002, pp. 477-496. |
| [49] | Aster, R.; Borchers, B.; Thurber, C., Parameter Estimation and Inverse Problems (2004), Academic Press |
| [50] | Gouveia, W. P.; Scales, J. A., Resolution of seismic waveform inversion: Bayes versus Occam, Inverse Problems, 13, 323-349 (1997) · Zbl 1026.86005 |
| [51] | Malinverno, A., Parsimonious Bayesian Markov chain Monte Carlo inversion in a nonlinear geophysical problem, Geophys. J. Int., 151, 675-688 (2002) |
| [52] | Jackson, C.; Sen, M. K.; Stoffa, P. L., An efficient stochastic Bayesian approach to optimal parameter and uncertainty estimation for climate model predictions, J. Clim., 17, 2828-2841 (2004) |
| [53] | Wang, J.; Zabaras, N., Hierarchical Bayesian models for inverse problems in heat conduction, Inverse Prob., 21, 183-206 (2005) · Zbl 1060.62036 |
| [54] | MacKay, D. J.C., Comparison of approximate methods for handling hyperparameters, Neural Comput., 11, 5, 1035-1068 (1999) |
| [55] | Gilks, W. R.; Richardson, S.; Spiegelhalter, D. J., Markov Chain Monte Carlo in Practice (1996), Chapman & Hall: Chapman & Hall London · Zbl 0832.00018 |
| [56] | Kakutani, S., Spectral analysis of stationary Gaussian processes, (Neyman, J., Proceedings of the 4th Berkeley Symposium on Mathematical Statistics and Probability, vol. 2 (1961), University of California Press), 239-247 · Zbl 0129.30103 |
| [57] | Schoutens, W., Stochastic Processes and Orthogonal Polynomials (2000), Springer · Zbl 0960.60076 |
| [58] | Reagan, M.; Najm, H.; Ghanem, R.; Knio, O., Uncertainty quantification in reacting flow simulations through non-intrusive spectral projection, Combustion and Flame, 132, 545-555 (2003) |
| [59] | Wahba, G., Spline Models for Observational Data (1990), SIAM · Zbl 0813.62001 |
| [60] | Kimeldorf, G. S.; Wahba, G., A correspondence between Bayesian estimation on stochastic processes and smoothing by splines, Ann. Math. Stat., 41, 2, 495-502 (1970) · Zbl 0193.45201 |
| [61] | Oliver, D. S.; Cunha, L.; Reynolds, A. C., Markov chain Monte Carlo methods for conditioning a permeability field to pressure data, Math. Geol., 29, 1, 61-91 (1997) |
| [62] | Sommeijer, B.; Shampine, L.; Verwer, J., RKC: an explicit solver for parabolic PDEs, J. Comput. Appl. Math., 88, 315-326 (1997) · Zbl 0910.65067 |
| [63] | J.G. Verwer, B.P. Sommeijer, W. Hundsdorfer, RKC time-stepping for advection-diffusion-reaction problems, CWI Report MAS-E0405, CWI, Amsterdam, The Netherlands, 2004.; J.G. Verwer, B.P. Sommeijer, W. Hundsdorfer, RKC time-stepping for advection-diffusion-reaction problems, CWI Report MAS-E0405, CWI, Amsterdam, The Netherlands, 2004. · Zbl 1059.65085 |
| [64] | Gelman, A.; Carlin, J. B.; Stern, H. S.; Rubin, D. B., Bayesian Data Analysis (2003), Chapman & Hall: Chapman & Hall CRC |
| [65] | Gelman, A., Prior distributions for variance parameters in hierarchical models, Bayesian Anal., 1, 3, 515-534 (2006) · Zbl 1331.62139 |
| [66] | H.K.H. Lee, C.H. Holloman, C.A. Calder, D.M. Higdon, Flexible Gaussian processes via convolution, Discussion paper 2002-09, Duke University, Department of Statistical Science, 2002.; H.K.H. Lee, C.H. Holloman, C.A. Calder, D.M. Higdon, Flexible Gaussian processes via convolution, Discussion paper 2002-09, Duke University, Department of Statistical Science, 2002. |
| [67] | Kaipio, J. P.; Kolehmainen, V.; Vauhkonen, M.; Somersalo, E., Inverse problems with structural prior information, Inverse Prob., 15, 713-729 (1999) · Zbl 0949.65144 |
| [68] | Calvetti, D.; Somersalo, E., A Gaussian hypermodel to recover blocky objects, Inverse Prob., 23, 733-754 (2007) · Zbl 1112.62018 |
| [69] | Press, W.; Teukolsky, S. A.; Vetterling, W.; Flannery, B., Numerical Recipes in C: The Art of Scientific Computing (1992), Cambridge University Press: Cambridge University Press Cambridge, UK · Zbl 0845.65001 |
| [70] | Robert, C. P.; Casella, G., Monte Carlo Statistical Methods (2004), Springer · Zbl 1096.62003 |
| [71] | Ghanem, R. G., The nonlinear Gaussian spectrum of log-normal stochastic processes and variables, ASME J. Appl. Mech., 66, 4, 964-973 (1999) |
| [72] | W. Luo, Wiener chaos expansion and numerical solutions of stochastic partial differential equations, Ph.D. thesis, California Institute of Technology, May 2006.; W. Luo, Wiener chaos expansion and numerical solutions of stochastic partial differential equations, Ph.D. thesis, California Institute of Technology, May 2006. |
| [73] | Dunkley, J.; Bucher, M.; Ferreira, P. G.; Moodley, K., Fast and reliable Markov chain Monte Carlo technique for cosmological parameter estimation, Mon. Notices Roy. Astronom. Soc., 356, 3, 925-936 (2005) |
| [74] | Lorenz, E. N., Deterministic nonperiodic flow, J. Atmos. Sci., 20, 2, 130-141 (1963) · Zbl 1417.37129 |
| [75] | Hou, T. Y.; Luo, W.; Rozovskii, B.; Zhou, H.-M., Wiener Chaos expansions and numerical solutions of randomly-forced equations of fluid mechanics, J. Comput. Phys., 216, 687-706 (2006) · Zbl 1095.76047 |
| [76] | Doostan, A.; Ghanem, R. G.; Red-Horse, J., Stochastic model reduction for chaos representations, Comput. Methods Appl. Mech. Eng., 196, 3951-3966 (2007) · Zbl 1173.74411 |
| [77] | Babuška, I.; Nobile, F.; Tempone, R., A stochastic collocation method for elliptic partial differential equations with random input data, SIAM J. Numer. Anal., 45, 3, 1005-1034 (2007) · Zbl 1151.65008 |
| [78] | Xiu, D.; Hesthaven, J. S., High-order collocation methods for differential equations with random inputs, SIAM J. Sci. Comput., 27, 3, 1118-1139 (2005) · Zbl 1091.65006 |
| [79] | Kraut, S.; Anderson, R. H.; Krolik, J. L., A generalized Karhunen-Loève basis for efficient estimation of tropospheric refractivity using radar clutter, IEEE Trans. Signal Process., 52, 1, 48-60 (2004) · Zbl 1369.94198 |
| [80] | Aronszajn, N., Theory of reproducing kernels, Trans. Am. Math. Soc., 68, 3, 337-404 (1950) · Zbl 0037.20701 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
