×

Asymptotic frequentist coverage properties of Bayesian credible sets for sieve priors. (English) Zbl 1471.62350

Uncertainty quantification is crucial in statistical sciences. In statistics, uncertainty about an estimator is described with the help of confidence sets. But the construction of confidence sets can be challenging, especially in complex, nonparametric problems. A popular aspect of the Bayesian approach is the way of quantifying uncertainty with the help of sets with prescribed high posterior probability (called credible sets). In parametric models credible sets are asymptotically confidence sets as well. However, in nonparametric and high-dimensional models the question is still unanswered about how much one can trust Bayesian credible sets as a measure of confidence in the statistical procedure from a frequentist perspective (and a general approach for understanding the coverage of credible sets is missing, too). The paper under review aims to partially fill the gap by contributing to the fundamental understanding of this field. The authors derive results for general choices of models and sieve type of priors, in the spirit of [S. Ghosal et al., ibid. 28, No. 2, 500–531 (2000; Zbl 1105.62315); S. Ghosal and A. van der Vaart, ibid. 35, No. 1, 192–223 (2007; Zbl 1114.62060); the authors, ibid. 45, No. 2, 833–865 (2017; Zbl 1371.62048)]. Namely, the frequentist properties of Bayesian credible sets resulting from the hierarchical and the empirical Bayes procedures are investigated. The authors introduce conditions under which credible sets have honest frequentist coverage and rate adaptive size in the considered general setting. The derived results are then applied to various choices of sampling and prior models.

MSC:

62G20 Asymptotic properties of nonparametric inference
62G05 Nonparametric estimation
62G08 Nonparametric regression and quantile regression
62G07 Density estimation
62G15 Nonparametric tolerance and confidence regions

References:

[1] Arbel, J., Gayraud, G. and Rousseau, J. (2013). Bayesian optimal adaptive estimation using a sieve prior. Scand. J. Stat. 40 549-570. Zentralblatt MATH: 1364.62102
Digital Object Identifier: doi:10.1002/sjos.12002
· Zbl 1364.62102 · doi:10.1002/sjos.12002
[2] Belitser, E. (2017). On coverage and local radial rates of credible sets. Ann. Statist. 45 1124-1151. Zentralblatt MATH: 1371.62044
Digital Object Identifier: doi:10.1214/16-AOS1477
Project Euclid: euclid.aos/1497319690
· Zbl 1371.62044 · doi:10.1214/16-AOS1477
[3] Belitser, E. and Nurushev, N. (2015). Needles and straw in a haystack: Empirical Bayes confidence for possibly sparse sequences. Preprint. Available at arXiv:1511.01803. arXiv: 1511.01803
Zentralblatt MATH: 07140497
Digital Object Identifier: doi:10.3150/19-BEJ1122
Project Euclid: euclid.bj/1574758826
· Zbl 1441.62110 · doi:10.3150/19-BEJ1122
[4] Bull, A. D. (2012). Honest adaptive confidence bands and self-similar functions. Electron. J. Stat. 6 1490-1516. Zentralblatt MATH: 1295.62049
Digital Object Identifier: doi:10.1214/12-EJS720
· Zbl 1295.62049 · doi:10.1214/12-EJS720
[5] Bull, A. D. and Nickl, R. (2013). Adaptive confidence sets in \(L^2\). Probab. Theory Related Fields 156 889-919. Zentralblatt MATH: 1273.62105
Digital Object Identifier: doi:10.1007/s00440-012-0446-z
· Zbl 1273.62105 · doi:10.1007/s00440-012-0446-z
[6] Cai, T. T. and Low, M. G. (2004). An adaptation theory for nonparametric confidence intervals. Ann. Statist. 32 1805-1840. Zentralblatt MATH: 1056.62060
Digital Object Identifier: doi:10.1214/009053604000000049
Project Euclid: euclid.aos/1098883773
· Zbl 1056.62060 · doi:10.1214/009053604000000049
[7] Carpentier, A. (2013). Honest and adaptive confidence sets in \(L_p\). Electron. J. Stat. 7 2875-2923. Zentralblatt MATH: 1280.62055
Digital Object Identifier: doi:10.1214/13-EJS867
· Zbl 1280.62055 · doi:10.1214/13-EJS867
[8] Carpentier, A. and Nickl, R. (2015). On signal detection and confidence sets for low rank inference problems. Electron. J. Stat. 9 2675-2688. Zentralblatt MATH: 1329.62216
Digital Object Identifier: doi:10.1214/15-EJS1087
· Zbl 1329.62216 · doi:10.1214/15-EJS1087
[9] Castillo, I. (2008). Lower bounds for posterior rates with Gaussian process priors. Electron. J. Stat. 2 1281-1299. Zentralblatt MATH: 1320.62067
Digital Object Identifier: doi:10.1214/08-EJS273
· Zbl 1320.62067 · doi:10.1214/08-EJS273
[10] Castillo, I. and Mismer, R. (2018). Empirical Bayes analysis of spike and slab posterior distributions. Electron. J. Stat. 12 3953-4001. Zentralblatt MATH: 1409.62026
Digital Object Identifier: doi:10.1214/18-EJS1494
· Zbl 1409.62026 · doi:10.1214/18-EJS1494
[11] Castillo, I. and Nickl, R. (2013). Nonparametric Bernstein-von Mises theorems in Gaussian white noise. Ann. Statist. 41 1999-2028. Zentralblatt MATH: 1285.62052
Digital Object Identifier: doi:10.1214/13-AOS1133
Project Euclid: euclid.aos/1382547511
· Zbl 1285.62052 · doi:10.1214/13-AOS1133
[12] Castillo, I. and Nickl, R. (2014). On the Bernstein-von Mises phenomenon for nonparametric Bayes procedures. Ann. Statist. 42 1941-1969. Zentralblatt MATH: 1305.62190
Digital Object Identifier: doi:10.1214/14-AOS1246
Project Euclid: euclid.aos/1410440630
· Zbl 1305.62190 · doi:10.1214/14-AOS1246
[13] Castillo, I. and Rousseau, J. (2015). A Bernstein-von Mises theorem for smooth functionals in semiparametric models. Ann. Statist. 43 2353-2383. Zentralblatt MATH: 1327.62302
Digital Object Identifier: doi:10.1214/15-AOS1336
Project Euclid: euclid.aos/1444222078
· Zbl 1327.62302 · doi:10.1214/15-AOS1336
[14] Castillo, I. and Szabo, B. (2020). Spike and slab empirical Bayes sparse credible sets. Bernoulli 26 127-158. Zentralblatt MATH: 07140495
Digital Object Identifier: doi:10.3150/19-BEJ1119
Project Euclid: euclid.bj/1574758824
· Zbl 1441.62077 · doi:10.3150/19-BEJ1119
[15] Chernozhukov, V., Chetverikov, D. and Kato, K. (2014). Anti-concentration and honest, adaptive confidence bands. Ann. Statist. 42 1787-1818. Zentralblatt MATH: 1305.62161
Digital Object Identifier: doi:10.1214/14-AOS1235
Project Euclid: euclid.aos/1410440625
· Zbl 1305.62161 · doi:10.1214/14-AOS1235
[16] Cox, D. D. (1993). An analysis of Bayesian inference for nonparametric regression. Ann. Statist. 21 903-923. Zentralblatt MATH: 0778.62003
Digital Object Identifier: doi:10.1214/aos/1176349157
Project Euclid: euclid.aos/1176349157
· Zbl 0778.62003 · doi:10.1214/aos/1176349157
[17] Freedman, D. (1999). On the Bernstein-von Mises theorem with infinite-dimensional parameters. Ann. Statist. 27 1119-1140. Zentralblatt MATH: 0957.62002
Project Euclid: euclid.aos/1017938917
· Zbl 0957.62002
[18] Gao, C. and Zhou, H. H. (2016). Rate exact Bayesian adaptation with modified block priors. Ann. Statist. 44 318-345. Zentralblatt MATH: 1331.62215
Digital Object Identifier: doi:10.1214/15-AOS1368
Project Euclid: euclid.aos/1449755965
· Zbl 1331.62215 · doi:10.1214/15-AOS1368
[19] Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500-531. Zentralblatt MATH: 1105.62315
Digital Object Identifier: doi:10.1214/aos/1016218228
Project Euclid: euclid.aos/1016218228
· Zbl 1105.62315 · doi:10.1214/aos/1016218228
[20] Ghosal, S. and van der Vaart, A. (2007). Convergence rates of posterior distributions for non-i.i.d. observations. Ann. Statist. 35 192-223. Zentralblatt MATH: 1114.62060
Digital Object Identifier: doi:10.1214/009053606000001172
Project Euclid: euclid.aos/1181100186
· Zbl 1114.62060 · doi:10.1214/009053606000001172
[21] Giné, E. and Nickl, R. (2010). Confidence bands in density estimation. Ann. Statist. 38 1122-1170. Zentralblatt MATH: 1183.62062
Digital Object Identifier: doi:10.1214/09-AOS738
Project Euclid: euclid.aos/1266586625
· Zbl 1183.62062 · doi:10.1214/09-AOS738
[22] Giné, E. and Nickl, R. (2016). Mathematical Foundations of Infinite-Dimensional Statistical Models. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge Univ. Press, New York. · Zbl 1358.62014
[23] Hadji, A. and Szabo, B. (2019). Can we trust bayesian uncertainty quantification from Gaussian process priors with squared exponential covariance kernel? Preprint. Available at arXiv:1904.01383. arXiv: 1904.01383
[24] Hoffmann, M. and Nickl, R. (2011). On adaptive inference and confidence bands. Ann. Statist. 39 2383-2409. Zentralblatt MATH: 1232.62072
Digital Object Identifier: doi:10.1214/11-AOS903
Project Euclid: euclid.aos/1322663462
· Zbl 1232.62072 · doi:10.1214/11-AOS903
[25] Hoffmann, M., Rousseau, J. and Schmidt-Hieber, J. (2015). On adaptive posterior concentration rates. Ann. Statist. 43 2259-2295. Zentralblatt MATH: 1327.62306
Digital Object Identifier: doi:10.1214/15-AOS1341
Project Euclid: euclid.aos/1442364152
· Zbl 1327.62306 · doi:10.1214/15-AOS1341
[26] Kirichenko, A. and van Zanten, H. (2017). Estimating a smooth function on a large graph by Bayesian Laplacian regularisation. Electron. J. Stat. 11 891-915. Zentralblatt MATH: 1362.62112
Digital Object Identifier: doi:10.1214/17-EJS1253
· Zbl 1362.62112 · doi:10.1214/17-EJS1253
[27] Knapik, B. T., van der Vaart, A. W. and van Zanten, J. H. (2011). Bayesian inverse problems with Gaussian priors. Ann. Statist. 39 2626-2657. Zentralblatt MATH: 1232.62079
Digital Object Identifier: doi:10.1214/11-AOS920
Project Euclid: euclid.aos/1324563350
· Zbl 1232.62079 · doi:10.1214/11-AOS920
[28] Leahu, H. (2011). On the Bernstein-von Mises phenomenon in the Gaussian white noise model. Electron. J. Stat. 5 373-404. Zentralblatt MATH: 1274.62290
Digital Object Identifier: doi:10.1214/11-EJS611
· Zbl 1274.62290 · doi:10.1214/11-EJS611
[29] Low, M. G. (1997). On nonparametric confidence intervals. Ann. Statist. 25 2547-2554. Zentralblatt MATH: 0894.62055
Digital Object Identifier: doi:10.1214/aos/1030741084
Project Euclid: euclid.aos/1030741084
· Zbl 0894.62055 · doi:10.1214/aos/1030741084
[30] McVinish, R., Rousseau, J. and Mengersen, K. (2009). Bayesian goodness of fit testing with mixtures of triangular distributions. Scand. J. Stat. 36 337-354. Zentralblatt MATH: 1195.62053
Digital Object Identifier: doi:10.1111/j.1467-9469.2008.00620.x
· Zbl 1195.62053 · doi:10.1111/j.1467-9469.2008.00620.x
[31] Mukherjee, R. and Sen, S. (2018). Optimal adaptive inference in random design binary regression. Bernoulli 24 699-739. Zentralblatt MATH: 1387.62053
Digital Object Identifier: doi:10.3150/16-BEJ893
Project Euclid: euclid.bj/1501142460
· Zbl 1387.62053 · doi:10.3150/16-BEJ893
[32] Nickl, R. and Szabó, B. (2016). A sharp adaptive confidence ball for self-similar functions. Stochastic Process. Appl. 126 3913-3934. Zentralblatt MATH: 1348.62165
Digital Object Identifier: doi:10.1016/j.spa.2016.04.017
· Zbl 1348.62165 · doi:10.1016/j.spa.2016.04.017
[33] Nickl, R. and van de Geer, S. (2013). Confidence sets in sparse regression. Ann. Statist. 41 2852-2876. Zentralblatt MATH: 1288.62108
Digital Object Identifier: doi:10.1214/13-AOS1170
Project Euclid: euclid.aos/1387313392
· Zbl 1288.62108 · doi:10.1214/13-AOS1170
[34] Petrone, S., Rousseau, J. and Scricciolo, C. (2014). Bayes and empirical Bayes: Do they merge? Biometrika 101 285-302. Zentralblatt MATH: 06329732
Digital Object Identifier: doi:10.1093/biomet/ast067
· Zbl 1452.62148 · doi:10.1093/biomet/ast067
[35] Picard, D. and Tribouley, K. (2000). Adaptive confidence interval for pointwise curve estimation. Ann. Statist. 28 298-335. Zentralblatt MATH: 1106.62331
Digital Object Identifier: doi:10.1214/aos/1016120374
Project Euclid: euclid.aos/1016120374
· Zbl 1106.62331 · doi:10.1214/aos/1016120374
[36] Ray, K. (2017). Adaptive Bernstein-von Mises theorems in Gaussian white noise. Ann. Statist. 45 2511-2536. Zentralblatt MATH: 1384.62158
Digital Object Identifier: doi:10.1214/16-AOS1533
Project Euclid: euclid.aos/1513328581
· Zbl 1384.62158 · doi:10.1214/16-AOS1533
[37] Rivoirard, V. and Rousseau, J. (2012). Bernstein-von Mises theorem for linear functionals of the density. Ann. Statist. 40 1489-1523. Zentralblatt MATH: 1257.62036
Digital Object Identifier: doi:10.1214/12-AOS1004
Project Euclid: euclid.aos/1346850063
· Zbl 1257.62036 · doi:10.1214/12-AOS1004
[38] Rivoirard, V. and Rousseau, J. (2012). Posterior concentration rates for infinite dimensional exponential families. Bayesian Anal. 7 311-333. Zentralblatt MATH: 1330.62179
Digital Object Identifier: doi:10.1214/12-BA710
· Zbl 1330.62179 · doi:10.1214/12-BA710
[39] Robins, J. and van der Vaart, A. (2006). Adaptive nonparametric confidence sets. Ann. Statist. 34 229-253. Zentralblatt MATH: 1091.62039
Digital Object Identifier: doi:10.1214/009053605000000877
Project Euclid: euclid.aos/1146576262
· Zbl 1091.62039 · doi:10.1214/009053605000000877
[40] Rousseau, J. (2007). Approximating interval hypothesis: \(p\)-values and Bayes factors. In Bayesian Statistics 8 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.). <i/> Oxford Sci. Publ. 417-452. Oxford Univ. Press, Oxford. Zentralblatt MATH: 1252.62028
· Zbl 1252.62028
[41] Rousseau, J. and Szabo, B. (2017). Asymptotic behaviour of the empirical Bayes posteriors associated to maximum marginal likelihood estimator. Ann. Statist. 45 833-865. Zentralblatt MATH: 1371.62048
Digital Object Identifier: doi:10.1214/16-AOS1469
Project Euclid: euclid.aos/1494921959
· Zbl 1371.62048 · doi:10.1214/16-AOS1469
[42] Rousseau, J. and Szabo, B. (2020). Supplement to “Asymptotic frequentist coverage properties of Bayesian credible sets for sieve priors.” https://doi.org/10.1214/19-AOS1881SUPP.
[43] Rudelson, M. (1999). Random vectors in the isotropic position. J. Funct. Anal. 164 60-72. Zentralblatt MATH: 0929.46021
Digital Object Identifier: doi:10.1006/jfan.1998.3384
· Zbl 0929.46021 · doi:10.1006/jfan.1998.3384
[44] Scricciolo, C. (2007). On rates of convergence for Bayesian density estimation. Scand. J. Stat. 34 626-642. Zentralblatt MATH: 1150.62018
Digital Object Identifier: doi:10.1111/j.1467-9469.2006.00540.x
· Zbl 1150.62018 · doi:10.1111/j.1467-9469.2006.00540.x
[45] Serra, P. and Krivobokova, T. (2017). Adaptive empirical Bayesian smoothing splines. Bayesian Anal. 12 219-238. Zentralblatt MATH: 1384.62118
Digital Object Identifier: doi:10.1214/16-BA997
· Zbl 1384.62118 · doi:10.1214/16-BA997
[46] Sniekers, S. and van der Vaart, A. (2015). Adaptive Bayesian credible sets in regression with a Gaussian process prior. Electron. J. Stat. 9 2475-2527. Zentralblatt MATH: 1327.62300
Digital Object Identifier: doi:10.1214/15-EJS1078
· Zbl 1327.62300 · doi:10.1214/15-EJS1078
[47] Söhl, J. and Trabs, M. (2016). Adaptive confidence bands for Markov chains and diffusions: Estimating the invariant measure and the drift. ESAIM Probab. Stat. 20 432-462. Zentralblatt MATH: 1357.62198
Digital Object Identifier: doi:10.1051/ps/2016017
· Zbl 1357.62198 · doi:10.1051/ps/2016017
[48] Szabó, B. (2015). On Bayesian based adaptive confidence sets for linear functionals. In Bayesian Statistics from Methods to Models and Applications. Springer Proc. Math. Stat. 126 91-105. Springer, Cham.
[49] Szabó, B., van der Vaart, A. W. and van Zanten, J. H. (2015). Frequentist coverage of adaptive nonparametric Bayesian credible sets. Ann. Statist. 43 1391-1428. Zentralblatt MATH: 1317.62040
Digital Object Identifier: doi:10.1214/14-AOS1270
Project Euclid: euclid.aos/1434546205
· Zbl 1317.62040 · doi:10.1214/14-AOS1270
[50] Tsybakov, A. B. (2009). Introduction to Nonparametric Estimation. Springer Series in Statistics. Springer, New York. Zentralblatt MATH: 1176.62032
· Zbl 1176.62032
[51] van der Pas, S., Szabó, B. and van der Vaart, A. (2017). Uncertainty quantification for the horseshoe (with discussion). Bayesian Anal. 12 1221-1274. Zentralblatt MATH: 1384.62155
Digital Object Identifier: doi:10.1214/17-BA1065
· Zbl 1384.62155 · doi:10.1214/17-BA1065
[52] van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics 3. Cambridge Univ. Press, Cambridge. · Zbl 0910.62001
[53] van der Vaart, A. W. and van Zanten, J. H. (2008). Rates of contraction of posterior distributions based on Gaussian process priors. Ann. Statist. 36 1435-1463. Zentralblatt MATH: 1141.60018
Digital Object Identifier: doi:10.1214/009053607000000613
Project Euclid: euclid.aos/1211819570
· Zbl 1141.60018 · doi:10.1214/009053607000000613
[54] Verdinelli, I. and Wasserman, L. (1998). Bayesian goodness-of-fit testing using infinite-dimensional exponential families. Ann. Statist. 26 1215-1241. Zentralblatt MATH: 0930.62027
Digital Object Identifier: doi:10.1214/aos/1024691240
Project Euclid: euclid.aos/1024691240
· Zbl 0930.62027 · doi:10.1214/aos/1024691240
[55] Weimin Yoo, W. and van der Vaart, A. W. (2017). The Bayes Lepski’s method and credible bands through volume of tubular neighborhoods. ArXiv e-prints.
[56] Yang, Y. (1999). Minimax nonparametric classification. I. Rates of convergence. IEEE Trans. Inform. Theory 45 2271-2284. Zentralblatt MATH: 0962.62026
Digital Object Identifier: doi:10.1109/18.796368
· Zbl 0962.62026 · doi:10.1109/18.796368
[57] Yang, Y. (1999). Minimax nonparametric classification. II. Model selection for adaptation. IEEE Trans. Inform. Theory 45 2285-2292. Zentralblatt MATH: 0962.62027
Digital Object Identifier: doi:10.1109/18.796369
· Zbl 0962.62027 · doi:10.1109/18.796369
[58] Yoo, W. · Zbl 1338.62121 · doi:10.1214/15-AOS1398
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.