Asymptotics for Lasso-type estimators. (English) Zbl 1105.62357
Summary: We consider the asymptotic behavior ofregression estimators that minimize the residual sum of squares plus a penalty proportional to \(\sum| \beta_j|^{\gamma}\), for some \(\gamma>0\). These estimators include the Lasso as a special case when \(\gamma=1\). Under appropriate conditions, we show that the limiting distributions can have positive probability mass at 0 when the true value of the parameter is 0. We also consider asymptotics for “nearly singular” designs.
MSC:
| 62J05 | Linear regression; mixed models |
| 62E20 | Asymptotic distribution theory in statistics |
| 62J07 | Ridge regression; shrinkage estimators (Lasso) |
Software:
PDCOReferences:
| [1] | Anderson, P. K. and Gill, R. D. (1982). Cox’s regression model for counting processes: a large sample study. Ann. Statist. 10 1100-1120. · Zbl 0526.62026 · doi:10.1214/aos/1176345976 |
| [2] | Besag, J. (1986). On the statistical analysis ofdirty pictures (with discussion). J. Roy. Statist. Soc. Ser. B 48 259-302. JSTOR: · Zbl 0609.62150 |
| [3] | Chen, S. S., Donoho, D. L. and Saunders, M. A. (1999). Atomic decomposition by basis pursuit. SIAM J. Scientific Computing 20 33-61. · Zbl 0919.94002 · doi:10.1137/S1064827596304010 |
| [4] | Fan, J. and Li, R. (1999). Variable selection via penalized likelihood. Unpublished manuscript. |
| [5] | Fiacco, A. V. and McCormick, G. P. (1990). Nonlinear Programming: Sequential Unconstrained Minimization Techniques. SIAM, Philadelphia. · Zbl 0713.90043 |
| [6] | Frank, I. E. and Friedman, J. H. (1993). A statistical view ofsome chemometrics regression tools (with discussion). Technometrics 35 109-148. · Zbl 0775.62288 · doi:10.2307/1269656 |
| [7] | Fu, W. J. (1998). Penalized regressions: the Bridge versus the Lasso. J. Comput. Graph. Statist. 7 397-416. JSTOR: · doi:10.2307/1390712 |
| [8] | Fu, W. J. (1999). Estimating the effective trends in age-period-cohort studies. Unpublished manuscript. |
| [9] | Geyer, C. J. (1994). On the asymptotics ofconstrained M-estimation. Ann. Statist. 22 1993-2010. · Zbl 0829.62029 · doi:10.1214/aos/1176325768 |
| [10] | Geyer, C. J. (1996). On the asymptotics ofconvex stochastic optimization. Unpublished manuscript. |
| [11] | Kim, J. and Pollard, D. (1990). Cube root asymptotics. Ann. Statistics 18 191-219. · Zbl 0703.62063 · doi:10.1214/aos/1176347498 |
| [12] | Kupper, L. L., Janis, J. M., Karmous, A. and Greenberg, B. G. (1985). Statistical age-periodcohort analysis: a review and critique. J. Chronic Disease 38 811-830. |
| [13] | Linhart, H. and Zucchini, W. (1986). Model Selection. Wiley, New York. · Zbl 0665.62003 |
| [14] | Osborne, M. R., Presnell, B. and Turlach, B. A. (1998). On the Lasso and its dual. Research Report 98/1, Dept. Statistics, Univ. Adelaide. |
| [15] | Pflug, G. Ch. (1995). Asymptotic stochastic programs. Math. Oper. Res. 20 769-789. JSTOR: · Zbl 0847.90107 · doi:10.1287/moor.20.4.769 |
| [16] | Pollard, D. (1991). Asymptotics for least absolute deviation regression estimators. Econometric Theory 7 186-199. JSTOR: · doi:10.1017/S0266466600004394 |
| [17] | Sardy, S., Bruce, A. G. and Tseng, P. (1998). Block coordinate relaxation methods for nonparametric signal denoising with wavelet dictionaries. Technical Report, Dept. Mathematics, EPFL, Lausanne. |
| [18] | Srivastava, M. S. (1971). On fixed width confidence bounds for regression parameters. Ann. Math. Statist. 42 1403-1411. · Zbl 0224.62034 · doi:10.1214/aoms/1177693251 |
| [19] | Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. J. Roy. Statist. Soc. Ser. B 58 267-288. JSTOR: · Zbl 0850.62538 |
| [20] | van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes with Applications to Statistics. Springer, New York. · Zbl 0862.60002 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
