Asymptotic frequentist coverage properties of Bayesian credible sets for sieve priors. (English) Zbl 1471.62350
Uncertainty quantification is crucial in statistical sciences. In statistics, uncertainty about an estimator is described with the help of confidence sets. But the construction of confidence sets can be challenging, especially in complex, nonparametric problems. A popular aspect of the Bayesian approach is the way of quantifying uncertainty with the help of sets with prescribed high posterior probability (called credible sets). In parametric models credible sets are asymptotically confidence sets as well. However, in nonparametric and high-dimensional models the question is still unanswered about how much one can trust Bayesian credible sets as a measure of confidence in the statistical procedure from a frequentist perspective (and a general approach for understanding the coverage of credible sets is missing, too). The paper under review aims to partially fill the gap by contributing to the fundamental understanding of this field. The authors derive results for general choices of models and sieve type of priors, in the spirit of [S. Ghosal et al., ibid. 28, No. 2, 500–531 (2000; Zbl 1105.62315); S. Ghosal and A. van der Vaart, ibid. 35, No. 1, 192–223 (2007; Zbl 1114.62060); the authors, ibid. 45, No. 2, 833–865 (2017; Zbl 1371.62048)]. Namely, the frequentist properties of Bayesian credible sets resulting from the hierarchical and the empirical Bayes procedures are investigated. The authors introduce conditions under which credible sets have honest frequentist coverage and rate adaptive size in the considered general setting. The derived results are then applied to various choices of sampling and prior models.
Reviewer: Joseph Melamed (Los Angeles)
MSC:
62G20 | Asymptotic properties of nonparametric inference |
62G05 | Nonparametric estimation |
62G08 | Nonparametric regression and quantile regression |
62G07 | Density estimation |
62G15 | Nonparametric tolerance and confidence regions |
Keywords:
uncertainty quantification; coverage; posterior contraction rates; adaptation; empirical Bayes; hierarchical Bayes; nonparametric regression; density estimation; classification; sieve priorReferences:
[1] | Arbel, J., Gayraud, G. and Rousseau, J. (2013). Bayesian optimal adaptive estimation using a sieve prior. Scand. J. Stat. 40 549-570. Zentralblatt MATH: 1364.62102 Digital Object Identifier: doi:10.1002/sjos.12002 · Zbl 1364.62102 · doi:10.1002/sjos.12002 |
[2] | Belitser, E. (2017). On coverage and local radial rates of credible sets. Ann. Statist. 45 1124-1151. Zentralblatt MATH: 1371.62044 Digital Object Identifier: doi:10.1214/16-AOS1477 Project Euclid: euclid.aos/1497319690 · Zbl 1371.62044 · doi:10.1214/16-AOS1477 |
[3] | Belitser, E. and Nurushev, N. (2015). Needles and straw in a haystack: Empirical Bayes confidence for possibly sparse sequences. Preprint. Available at arXiv:1511.01803. arXiv: 1511.01803 Zentralblatt MATH: 07140497 Digital Object Identifier: doi:10.3150/19-BEJ1122 Project Euclid: euclid.bj/1574758826 · Zbl 1441.62110 · doi:10.3150/19-BEJ1122 |
[4] | Bull, A. D. (2012). Honest adaptive confidence bands and self-similar functions. Electron. J. Stat. 6 1490-1516. Zentralblatt MATH: 1295.62049 Digital Object Identifier: doi:10.1214/12-EJS720 · Zbl 1295.62049 · doi:10.1214/12-EJS720 |
[5] | Bull, A. D. and Nickl, R. (2013). Adaptive confidence sets in \(L^2\). Probab. Theory Related Fields 156 889-919. Zentralblatt MATH: 1273.62105 Digital Object Identifier: doi:10.1007/s00440-012-0446-z · Zbl 1273.62105 · doi:10.1007/s00440-012-0446-z |
[6] | Cai, T. T. and Low, M. G. (2004). An adaptation theory for nonparametric confidence intervals. Ann. Statist. 32 1805-1840. Zentralblatt MATH: 1056.62060 Digital Object Identifier: doi:10.1214/009053604000000049 Project Euclid: euclid.aos/1098883773 · Zbl 1056.62060 · doi:10.1214/009053604000000049 |
[7] | Carpentier, A. (2013). Honest and adaptive confidence sets in \(L_p\). Electron. J. Stat. 7 2875-2923. Zentralblatt MATH: 1280.62055 Digital Object Identifier: doi:10.1214/13-EJS867 · Zbl 1280.62055 · doi:10.1214/13-EJS867 |
[8] | Carpentier, A. and Nickl, R. (2015). On signal detection and confidence sets for low rank inference problems. Electron. J. Stat. 9 2675-2688. Zentralblatt MATH: 1329.62216 Digital Object Identifier: doi:10.1214/15-EJS1087 · Zbl 1329.62216 · doi:10.1214/15-EJS1087 |
[9] | Castillo, I. (2008). Lower bounds for posterior rates with Gaussian process priors. Electron. J. Stat. 2 1281-1299. Zentralblatt MATH: 1320.62067 Digital Object Identifier: doi:10.1214/08-EJS273 · Zbl 1320.62067 · doi:10.1214/08-EJS273 |
[10] | Castillo, I. and Mismer, R. (2018). Empirical Bayes analysis of spike and slab posterior distributions. Electron. J. Stat. 12 3953-4001. Zentralblatt MATH: 1409.62026 Digital Object Identifier: doi:10.1214/18-EJS1494 · Zbl 1409.62026 · doi:10.1214/18-EJS1494 |
[11] | Castillo, I. and Nickl, R. (2013). Nonparametric Bernstein-von Mises theorems in Gaussian white noise. Ann. Statist. 41 1999-2028. Zentralblatt MATH: 1285.62052 Digital Object Identifier: doi:10.1214/13-AOS1133 Project Euclid: euclid.aos/1382547511 · Zbl 1285.62052 · doi:10.1214/13-AOS1133 |
[12] | Castillo, I. and Nickl, R. (2014). On the Bernstein-von Mises phenomenon for nonparametric Bayes procedures. Ann. Statist. 42 1941-1969. Zentralblatt MATH: 1305.62190 Digital Object Identifier: doi:10.1214/14-AOS1246 Project Euclid: euclid.aos/1410440630 · Zbl 1305.62190 · doi:10.1214/14-AOS1246 |
[13] | Castillo, I. and Rousseau, J. (2015). A Bernstein-von Mises theorem for smooth functionals in semiparametric models. Ann. Statist. 43 2353-2383. Zentralblatt MATH: 1327.62302 Digital Object Identifier: doi:10.1214/15-AOS1336 Project Euclid: euclid.aos/1444222078 · Zbl 1327.62302 · doi:10.1214/15-AOS1336 |
[14] | Castillo, I. and Szabo, B. (2020). Spike and slab empirical Bayes sparse credible sets. Bernoulli 26 127-158. Zentralblatt MATH: 07140495 Digital Object Identifier: doi:10.3150/19-BEJ1119 Project Euclid: euclid.bj/1574758824 · Zbl 1441.62077 · doi:10.3150/19-BEJ1119 |
[15] | Chernozhukov, V., Chetverikov, D. and Kato, K. (2014). Anti-concentration and honest, adaptive confidence bands. Ann. Statist. 42 1787-1818. Zentralblatt MATH: 1305.62161 Digital Object Identifier: doi:10.1214/14-AOS1235 Project Euclid: euclid.aos/1410440625 · Zbl 1305.62161 · doi:10.1214/14-AOS1235 |
[16] | Cox, D. D. (1993). An analysis of Bayesian inference for nonparametric regression. Ann. Statist. 21 903-923. Zentralblatt MATH: 0778.62003 Digital Object Identifier: doi:10.1214/aos/1176349157 Project Euclid: euclid.aos/1176349157 · Zbl 0778.62003 · doi:10.1214/aos/1176349157 |
[17] | Freedman, D. (1999). On the Bernstein-von Mises theorem with infinite-dimensional parameters. Ann. Statist. 27 1119-1140. Zentralblatt MATH: 0957.62002 Project Euclid: euclid.aos/1017938917 · Zbl 0957.62002 |
[18] | Gao, C. and Zhou, H. H. (2016). Rate exact Bayesian adaptation with modified block priors. Ann. Statist. 44 318-345. Zentralblatt MATH: 1331.62215 Digital Object Identifier: doi:10.1214/15-AOS1368 Project Euclid: euclid.aos/1449755965 · Zbl 1331.62215 · doi:10.1214/15-AOS1368 |
[19] | Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500-531. Zentralblatt MATH: 1105.62315 Digital Object Identifier: doi:10.1214/aos/1016218228 Project Euclid: euclid.aos/1016218228 · Zbl 1105.62315 · doi:10.1214/aos/1016218228 |
[20] | Ghosal, S. and van der Vaart, A. (2007). Convergence rates of posterior distributions for non-i.i.d. observations. Ann. Statist. 35 192-223. Zentralblatt MATH: 1114.62060 Digital Object Identifier: doi:10.1214/009053606000001172 Project Euclid: euclid.aos/1181100186 · Zbl 1114.62060 · doi:10.1214/009053606000001172 |
[21] | Giné, E. and Nickl, R. (2010). Confidence bands in density estimation. Ann. Statist. 38 1122-1170. Zentralblatt MATH: 1183.62062 Digital Object Identifier: doi:10.1214/09-AOS738 Project Euclid: euclid.aos/1266586625 · Zbl 1183.62062 · doi:10.1214/09-AOS738 |
[22] | Giné, E. and Nickl, R. (2016). Mathematical Foundations of Infinite-Dimensional Statistical Models. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge Univ. Press, New York. · Zbl 1358.62014 |
[23] | Hadji, A. and Szabo, B. (2019). Can we trust bayesian uncertainty quantification from Gaussian process priors with squared exponential covariance kernel? Preprint. Available at arXiv:1904.01383. arXiv: 1904.01383 |
[24] | Hoffmann, M. and Nickl, R. (2011). On adaptive inference and confidence bands. Ann. Statist. 39 2383-2409. Zentralblatt MATH: 1232.62072 Digital Object Identifier: doi:10.1214/11-AOS903 Project Euclid: euclid.aos/1322663462 · Zbl 1232.62072 · doi:10.1214/11-AOS903 |
[25] | Hoffmann, M., Rousseau, J. and Schmidt-Hieber, J. (2015). On adaptive posterior concentration rates. Ann. Statist. 43 2259-2295. Zentralblatt MATH: 1327.62306 Digital Object Identifier: doi:10.1214/15-AOS1341 Project Euclid: euclid.aos/1442364152 · Zbl 1327.62306 · doi:10.1214/15-AOS1341 |
[26] | Kirichenko, A. and van Zanten, H. (2017). Estimating a smooth function on a large graph by Bayesian Laplacian regularisation. Electron. J. Stat. 11 891-915. Zentralblatt MATH: 1362.62112 Digital Object Identifier: doi:10.1214/17-EJS1253 · Zbl 1362.62112 · doi:10.1214/17-EJS1253 |
[27] | Knapik, B. T., van der Vaart, A. W. and van Zanten, J. H. (2011). Bayesian inverse problems with Gaussian priors. Ann. Statist. 39 2626-2657. Zentralblatt MATH: 1232.62079 Digital Object Identifier: doi:10.1214/11-AOS920 Project Euclid: euclid.aos/1324563350 · Zbl 1232.62079 · doi:10.1214/11-AOS920 |
[28] | Leahu, H. (2011). On the Bernstein-von Mises phenomenon in the Gaussian white noise model. Electron. J. Stat. 5 373-404. Zentralblatt MATH: 1274.62290 Digital Object Identifier: doi:10.1214/11-EJS611 · Zbl 1274.62290 · doi:10.1214/11-EJS611 |
[29] | Low, M. G. (1997). On nonparametric confidence intervals. Ann. Statist. 25 2547-2554. Zentralblatt MATH: 0894.62055 Digital Object Identifier: doi:10.1214/aos/1030741084 Project Euclid: euclid.aos/1030741084 · Zbl 0894.62055 · doi:10.1214/aos/1030741084 |
[30] | McVinish, R., Rousseau, J. and Mengersen, K. (2009). Bayesian goodness of fit testing with mixtures of triangular distributions. Scand. J. Stat. 36 337-354. Zentralblatt MATH: 1195.62053 Digital Object Identifier: doi:10.1111/j.1467-9469.2008.00620.x · Zbl 1195.62053 · doi:10.1111/j.1467-9469.2008.00620.x |
[31] | Mukherjee, R. and Sen, S. (2018). Optimal adaptive inference in random design binary regression. Bernoulli 24 699-739. Zentralblatt MATH: 1387.62053 Digital Object Identifier: doi:10.3150/16-BEJ893 Project Euclid: euclid.bj/1501142460 · Zbl 1387.62053 · doi:10.3150/16-BEJ893 |
[32] | Nickl, R. and Szabó, B. (2016). A sharp adaptive confidence ball for self-similar functions. Stochastic Process. Appl. 126 3913-3934. Zentralblatt MATH: 1348.62165 Digital Object Identifier: doi:10.1016/j.spa.2016.04.017 · Zbl 1348.62165 · doi:10.1016/j.spa.2016.04.017 |
[33] | Nickl, R. and van de Geer, S. (2013). Confidence sets in sparse regression. Ann. Statist. 41 2852-2876. Zentralblatt MATH: 1288.62108 Digital Object Identifier: doi:10.1214/13-AOS1170 Project Euclid: euclid.aos/1387313392 · Zbl 1288.62108 · doi:10.1214/13-AOS1170 |
[34] | Petrone, S., Rousseau, J. and Scricciolo, C. (2014). Bayes and empirical Bayes: Do they merge? Biometrika 101 285-302. Zentralblatt MATH: 06329732 Digital Object Identifier: doi:10.1093/biomet/ast067 · Zbl 1452.62148 · doi:10.1093/biomet/ast067 |
[35] | Picard, D. and Tribouley, K. (2000). Adaptive confidence interval for pointwise curve estimation. Ann. Statist. 28 298-335. Zentralblatt MATH: 1106.62331 Digital Object Identifier: doi:10.1214/aos/1016120374 Project Euclid: euclid.aos/1016120374 · Zbl 1106.62331 · doi:10.1214/aos/1016120374 |
[36] | Ray, K. (2017). Adaptive Bernstein-von Mises theorems in Gaussian white noise. Ann. Statist. 45 2511-2536. Zentralblatt MATH: 1384.62158 Digital Object Identifier: doi:10.1214/16-AOS1533 Project Euclid: euclid.aos/1513328581 · Zbl 1384.62158 · doi:10.1214/16-AOS1533 |
[37] | Rivoirard, V. and Rousseau, J. (2012). Bernstein-von Mises theorem for linear functionals of the density. Ann. Statist. 40 1489-1523. Zentralblatt MATH: 1257.62036 Digital Object Identifier: doi:10.1214/12-AOS1004 Project Euclid: euclid.aos/1346850063 · Zbl 1257.62036 · doi:10.1214/12-AOS1004 |
[38] | Rivoirard, V. and Rousseau, J. (2012). Posterior concentration rates for infinite dimensional exponential families. Bayesian Anal. 7 311-333. Zentralblatt MATH: 1330.62179 Digital Object Identifier: doi:10.1214/12-BA710 · Zbl 1330.62179 · doi:10.1214/12-BA710 |
[39] | Robins, J. and van der Vaart, A. (2006). Adaptive nonparametric confidence sets. Ann. Statist. 34 229-253. Zentralblatt MATH: 1091.62039 Digital Object Identifier: doi:10.1214/009053605000000877 Project Euclid: euclid.aos/1146576262 · Zbl 1091.62039 · doi:10.1214/009053605000000877 |
[40] | Rousseau, J. (2007). Approximating interval hypothesis: \(p\)-values and Bayes factors. In Bayesian Statistics 8 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.). <i/> Oxford Sci. Publ. 417-452. Oxford Univ. Press, Oxford. Zentralblatt MATH: 1252.62028 · Zbl 1252.62028 |
[41] | Rousseau, J. and Szabo, B. (2017). Asymptotic behaviour of the empirical Bayes posteriors associated to maximum marginal likelihood estimator. Ann. Statist. 45 833-865. Zentralblatt MATH: 1371.62048 Digital Object Identifier: doi:10.1214/16-AOS1469 Project Euclid: euclid.aos/1494921959 · Zbl 1371.62048 · doi:10.1214/16-AOS1469 |
[42] | Rousseau, J. and Szabo, B. (2020). Supplement to “Asymptotic frequentist coverage properties of Bayesian credible sets for sieve priors.” https://doi.org/10.1214/19-AOS1881SUPP. |
[43] | Rudelson, M. (1999). Random vectors in the isotropic position. J. Funct. Anal. 164 60-72. Zentralblatt MATH: 0929.46021 Digital Object Identifier: doi:10.1006/jfan.1998.3384 · Zbl 0929.46021 · doi:10.1006/jfan.1998.3384 |
[44] | Scricciolo, C. (2007). On rates of convergence for Bayesian density estimation. Scand. J. Stat. 34 626-642. Zentralblatt MATH: 1150.62018 Digital Object Identifier: doi:10.1111/j.1467-9469.2006.00540.x · Zbl 1150.62018 · doi:10.1111/j.1467-9469.2006.00540.x |
[45] | Serra, P. and Krivobokova, T. (2017). Adaptive empirical Bayesian smoothing splines. Bayesian Anal. 12 219-238. Zentralblatt MATH: 1384.62118 Digital Object Identifier: doi:10.1214/16-BA997 · Zbl 1384.62118 · doi:10.1214/16-BA997 |
[46] | Sniekers, S. and van der Vaart, A. (2015). Adaptive Bayesian credible sets in regression with a Gaussian process prior. Electron. J. Stat. 9 2475-2527. Zentralblatt MATH: 1327.62300 Digital Object Identifier: doi:10.1214/15-EJS1078 · Zbl 1327.62300 · doi:10.1214/15-EJS1078 |
[47] | Söhl, J. and Trabs, M. (2016). Adaptive confidence bands for Markov chains and diffusions: Estimating the invariant measure and the drift. ESAIM Probab. Stat. 20 432-462. Zentralblatt MATH: 1357.62198 Digital Object Identifier: doi:10.1051/ps/2016017 · Zbl 1357.62198 · doi:10.1051/ps/2016017 |
[48] | Szabó, B. (2015). On Bayesian based adaptive confidence sets for linear functionals. In Bayesian Statistics from Methods to Models and Applications. Springer Proc. Math. Stat. 126 91-105. Springer, Cham. |
[49] | Szabó, B., van der Vaart, A. W. and van Zanten, J. H. (2015). Frequentist coverage of adaptive nonparametric Bayesian credible sets. Ann. Statist. 43 1391-1428. Zentralblatt MATH: 1317.62040 Digital Object Identifier: doi:10.1214/14-AOS1270 Project Euclid: euclid.aos/1434546205 · Zbl 1317.62040 · doi:10.1214/14-AOS1270 |
[50] | Tsybakov, A. B. (2009). Introduction to Nonparametric Estimation. Springer Series in Statistics. Springer, New York. Zentralblatt MATH: 1176.62032 · Zbl 1176.62032 |
[51] | van der Pas, S., Szabó, B. and van der Vaart, A. (2017). Uncertainty quantification for the horseshoe (with discussion). Bayesian Anal. 12 1221-1274. Zentralblatt MATH: 1384.62155 Digital Object Identifier: doi:10.1214/17-BA1065 · Zbl 1384.62155 · doi:10.1214/17-BA1065 |
[52] | van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics 3. Cambridge Univ. Press, Cambridge. · Zbl 0910.62001 |
[53] | van der Vaart, A. W. and van Zanten, J. H. (2008). Rates of contraction of posterior distributions based on Gaussian process priors. Ann. Statist. 36 1435-1463. Zentralblatt MATH: 1141.60018 Digital Object Identifier: doi:10.1214/009053607000000613 Project Euclid: euclid.aos/1211819570 · Zbl 1141.60018 · doi:10.1214/009053607000000613 |
[54] | Verdinelli, I. and Wasserman, L. (1998). Bayesian goodness-of-fit testing using infinite-dimensional exponential families. Ann. Statist. 26 1215-1241. Zentralblatt MATH: 0930.62027 Digital Object Identifier: doi:10.1214/aos/1024691240 Project Euclid: euclid.aos/1024691240 · Zbl 0930.62027 · doi:10.1214/aos/1024691240 |
[55] | Weimin Yoo, W. and van der Vaart, A. W. (2017). The Bayes Lepski’s method and credible bands through volume of tubular neighborhoods. ArXiv e-prints. |
[56] | Yang, Y. (1999). Minimax nonparametric classification. I. Rates of convergence. IEEE Trans. Inform. Theory 45 2271-2284. Zentralblatt MATH: 0962.62026 Digital Object Identifier: doi:10.1109/18.796368 · Zbl 0962.62026 · doi:10.1109/18.796368 |
[57] | Yang, Y. (1999). Minimax nonparametric classification. II. Model selection for adaptation. IEEE Trans. Inform. Theory 45 2285-2292. Zentralblatt MATH: 0962.62027 Digital Object Identifier: doi:10.1109/18.796369 · Zbl 0962.62027 · doi:10.1109/18.796369 |
[58] | Yoo, W. · Zbl 1338.62121 · doi:10.1214/15-AOS1398 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.