Reducing the computational cost of the ECF using a nuFFT: a fast and objective probability density estimation method. (English) Zbl 1506.62142
Summary: A nonuniform, fast Fourier transform can be used to reduce the computational cost of the empirical characteristic function (ECF) by a factor of 100. This fast ECF calculation method is applied to a new, objective, and robust method for estimating the probability distribution of univariate data, which effectively modulates and filters the ECF of a dataset in a way that yields an optimal estimate of the (Fourier transformed) underlying distribution. This improvement in computational efficiency is leveraged to estimate probability densities from a large ensemble of atmospheric velocity increments (gradients), with the purpose of characterizing the statistical and fractal properties of the velocity field. It is shown that the distribution of velocity increments depends on location in an atmospheric model and that the increments are clearly not normally distributed. The estimated increment distributions exhibit self-similar and distinctly multifractal behavior, as shown by structure functions that exhibit power-law scaling with a non-linear dependence of the power-law exponent on the structure function order.
MSC:
| 62-08 | Computational methods for problems pertaining to statistics |
| 62G07 | Density estimation |
| 65T50 | Numerical methods for discrete and fast Fourier transforms |
Keywords:
empirical characteristic function; ECF; kernel density estimation; histogram; nonuniform FFT; nuFFTReferences:
| [1] | Alba Fernández, V.; Gamero, J.; Muñoz Garcia, J., A test for the two-sample problem based on empirical characteristic functions, Comput. Statist. Data Anal., 52, 7, 3730-3748, (2008), URL: http://www.sciencedirect.com/science/article/pii/S0167947307004847 · Zbl 1452.62305 |
| [2] | Baringhaus, L.; Henze, P. D.N., A consistent test for multivariate normality based on the empirical characteristic function, Metrika, 35, 1, 339-348, (1988), URL: http://link.springer.com/article/10.1007/BF02613322 · Zbl 0654.62046 |
| [3] | Bernacchia, A.; Pigolotti, S., Self-consistent method for density estimation, J. R. Stat. Soc. Ser. B Stat. Methodol., 73, 3, 407-422, (2011) · Zbl 1411.62091 |
| [4] | Bilodeau, M.; Lafaye de Micheaux, P., A multivariate empirical characteristic function test of independence with normal marginals, J. Multivariate Anal., 95, 2, 345-369, (2005) · Zbl 1070.62045 |
| [5] | Bogenschutz, P. A.; Krueger, S. K.; Khairoutdinov, M., Assumed probability density functions for shallow and deep convection, J. Adv. Model. Earth Syst., 2, 10, (2010) |
| [6] | Cooley, J. W.; Tukey, J. W., An algorithm for the machine calculation of complex Fourier series, Math. Comp., 19, 90, 297-301, (1965) · Zbl 0127.09002 |
| [7] | Csörgő, S., Testing for independence by the empirical characteristic function, J. Multivariate Anal., 16, 3, 290-299, (1985) · Zbl 0585.62097 |
| [8] | Davis, A.; Marshak, A.; Wiscombe, W.; Cahalan, R., Multifractal characterizations of nonstationarity and intermittency in geophysical fields: observed, retrieved, or simulated, J. Geophys. Res., 99, D4, 8055-8072, (1994) |
| [9] | Davis, A.; Marshak, A.; Wiscombe, W.; Cahalan, R., Multifractal characterizations of intermittency in nonstationary geophysical signals and fields, (Treviño, G.; Hardin, J.; Douglas, B.; Andreas, E., Current Topics in Nonstationary Analysis. Proceedings of the Second Workshop on Nonstationary Random Processes and their Applications. San Diego, California 11-12 June 1995, (1996), Chires Associates Inc. Houghton MI) · Zbl 0944.60011 |
| [10] | Dutt, A.; Rokhlin, V., Fast Fourier transforms for nonequispaced data, SIAM J. Sci. Comput., 14, 6, 1368-1393, (1993) · Zbl 0791.65108 |
| [11] | Fan, Y., Goodness-of-fit tests for a multivariate distribution by the empirical characteristic function, J. Multivariate Anal., 62, 1, 36-63, (1997) · Zbl 0949.62044 |
| [12] | Feuerverger, A.; Mureika, R. A., The empirical characteristic function and its applications, Ann. Statist., 5, 88-97, (1977), URL: http://www.jstor.org/stable/10.2307/2958763 · Zbl 0364.62051 |
| [13] | Golaz, J.-C.; Larson, V. E.; Cotton, W. R., A PDF-based model for boundary layer clouds. part I: method and model description, J. Atmospheric Sci., 59, 24, 3540-3551, (2002), URL: http://journals.ametsoc.org/doi/abs/10.1175/1520-0469(2002)059 |
| [14] | Greengard, L.; Lee, J.-Y., Accelerating the nonuniform fast Fourier transform, SIAM Rev., 46, 3, 443-454, (2004) · Zbl 1064.65156 |
| [15] | Jiménez-Gamero, M.-D.; Alba-Fernández, V.; Muñoz-García, J.; Chalco-Cano, Y., Goodness-of-fit tests based on empirical characteristic functions, Comput. Statist. Data Anal., 53, 12, 3957-3971, (2009), URL: http://www.sciencedirect.com/science/article/pii/S0167947309002308 · Zbl 1453.62119 |
| [16] | Knight, J. L.; Yu, J., Empirical characteristic function in time series estimation, Econometric Theory, 18, 03, 691-721, (2002), URL: http://journals.cambridge.org/abstract_S026646660218306X · Zbl 1109.62337 |
| [17] | Liao, J.; Wu, Y.; Lin, Y., Improving sheather and Jones bandwidth selector for difficult densities in kernel density estimation, J. Nonparametr. Stat., 22, 1, 105-114, (2010), URL: http://www.tandfonline.com/doi/abs/10.1080/10485250903194003 · Zbl 1264.62029 |
| [18] | Mandelbrot, B. B., The fractal geometry of nature, (1983), Times Books · Zbl 0504.28001 |
| [19] | Murota, K.; Takeuchi, K., The studentized empirical characteristic function and its application to test for the shape of distribution, Biometrika, 68, 1, 55-65, (1981), URL: http://biomet.oxfordjournals.org/content/68/1/55.short · Zbl 0463.62041 |
| [20] | Nastrom, G.; Gage, K., A climatology of atmospheric wavenumber spectra of wind and temperature observed by commercial aircraft, J. Atmospheric Sci., 42, 9, 950-960, (1985) |
| [21] | Neale, R. B.; Hoskins, B. J., A standard test for AGCMs including their physical parametrizations: I: the proposal, Atmos. Sci. Lett., 1, 2, 101-107, (2000) |
| [22] | Neale, R., Richter, J., Conley, A., Park, S., Lauritzen, P., Gettelman, A., Williamson, D., Vavrus, S., Taylor, M., Collins, W., et al. 2010. Description of the NCAR Community Atmosphere Model (CAM 4.0), NCAR Technical Note, National Center of Atmospheric Research TN-485+STR. |
| [23] | O’Brien, T. A.; Li, F.; Collins, W. D.; Rauscher, S. A.; Ringler, T. D.; Taylor, M.; Hagos, S. M.; Leung, L. R., Observed scaling in clouds and precipitation and scale incognizance in regional to global atmospheric models, J. Clim., 26, 23, 9313-9333, (2013) |
| [24] | Pressel, K. G., Water vapor variability across spatial scales: insights for theory, parameterization, and model assessment, (2012), University of California Berkeley, (Ph.D. Thesis) |
| [25] | Rauscher, S. A.; Ringler, T. D.; Skamarock, W. C.; Mirin, A. A., Exploring a global multiresolution modeling approach using aquaplanet simulations\({}^\ast\), J. Climate, 26, 2432-2452, (2013) |
| [26] | Silverman, B. W., Density estimation for statistics and data analysis, vol. 26, (1986), CRC press · Zbl 0617.62042 |
| [27] | Skamarock, W. C., Evaluating mesoscale NWP models using kinetic energy spectra, Mon. Weather Rev., 132, 12, 3019-3032, (2004), URL: http://dx.doi.org/10.1175/MWR2830.1 |
| [28] | Skamarock, W. C.; Klemp, J. B.; Duda, M. G.; Fowler, L. D.; Park, S.-H.; Ringler, T. D., A multiscale nonhydrostatic atmospheric model using centroidal Voronoi tesselations and C-grid staggering, Mon. Weather Rev., 140, 9, 3090-3105, (2012), URL: http://dx.doi.org/10.1175/MWR-D-11-00215.1 |
| [29] | Srihera, R.; Stute, W., Kernel adjusted density estimation, Statist. Probab. Lett., 81, 5, 571-579, (2011) · Zbl 1209.62062 |
| [30] | Wand, M.; Jones, M., (Kernel Smoothing, Monographs on Statistics and Applied Probability, vol. 60, (1995), Chapman & Hall London) · Zbl 0854.62043 |
| [31] | Watson, G. S.; Leadbetter, M., On the estimation of the probability density, Ann. Math. Stat., 34, 2, 480-491, (1963) · Zbl 0113.34504 |
| [32] | Wilks, D. S., Statistical methods in the atmospheric sciences, vol. 100, (2006), Academic press |
| [33] | Wood, A. T.A.; Chan, G., Simulation of stationary Gaussian processes in [0, 1] d, J. Comput. Graph. Statist., 3, 4, 409-432, (1994) |
| [34] | Yang, Q.; Ruby Leung, L.; Rauscher, S. A.; Ringler, T. D.; Taylor, M. A., Atmospheric moisture budget and spatial resolution dependence of precipitation extremes in aquaplanet simulations, J. Climate, 27, 3565-3581, (2014) |
| [35] | York, D.; Evensen, N. M.; Martínez, M. L.; De Basabe Delgado, J., Unified equations for the slope, intercept, and standard errors of the best straight line, Amer. J. Phys., 72, 3, 367-375, (2004), URL: http://scitation.aip.org/content/aapt/journal/ajp/72/3/10.1119/1.1632486 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
