×

Second order correctness of perturbation bootstrap M-estimator of multiple linear regression parameter. (English) Zbl 1442.62091

Summary: Consider the multiple linear regression model \(y_{i}=\mathbf{x}'_{i}\boldsymbol{\beta}+\varepsilon_{i}\), where \(\varepsilon_{i}\)’s are independent and identically distributed random variables, \(\mathbf{x}_{i}\)’s are known design vectors and \(\boldsymbol{\beta}\) is the \(p\times1\) vector of parameters. An effective way of approximating the distribution of the M-estimator \(\bar{\boldsymbol{\beta}}_{n}\), after proper centering and scaling, is the perturbation bootstrap method. In this current work, second order results of this non-naive bootstrap method have been investigated. Second order correctness is important for reducing the approximation error uniformly to \(o(n^{-1/2})\) to get better inferences. We show that the classical studentized version of the bootstrapped estimator fails to be second order correct. We introduce an innovative modification in the studentized version of the bootstrapped statistic and show that the modified bootstrapped pivot is second order correct (S.O.C.) for approximating the distribution of the studentized M-estimator. Additionally, we show that the perturbation bootstrap continues to be S.O.C. when the errors \(\varepsilon_{i}\)’s are independent, but may not be identically distributed. These findings establish perturbation bootstrap approximation as a significant improvement over asymptotic normality in the regression M-estimation.

MSC:

62G09 Nonparametric statistical resampling methods
62E20 Asymptotic distribution theory in statistics
62G05 Nonparametric estimation
62J05 Linear regression; mixed models
PDFBibTeX XMLCite
Full Text: DOI arXiv Euclid

References:

[1] Allen, M. and Datta, S. (1999). A note on bootstrapping \(M\)-estimators in ARMA models. J. Time Series Anal.20 365-379. · Zbl 0956.62073
[2] Arcones, M.A. and Giné, E. (1992). On the bootstrap of \(M\)-estimators and other statistical functionals. In Exploring the Limits of Bootstrap (East Lansing, MI, 1990). Wiley Ser. Probab. Math. Statist. Probab. Math. Statist. 13-47. New York: Wiley. · Zbl 0842.62027
[3] Arlot, S. (2009). Model selection by resampling penalization. Electron. J. Stat.3 557-624. · Zbl 1326.62097 · doi:10.1214/08-EJS196
[4] Barbe, P. and Bertail, P. (2012). The Weighted Bootstrap. Lecture Notes in Statistics98. New York: Springer. · Zbl 0826.62030
[5] Beran, R. (1986). Discussion: Jackknife, bootstrap and other resampling methods in regression analysis. Ann. Statist.14 1295-1298.
[6] Bhattacharya, R.N. and Ghosh, J.K. (1978). On the validity of the formal Edgeworth expansion. Ann. Statist.6 434-451. · Zbl 0396.62010 · doi:10.1214/aos/1176344134
[7] Bhattacharya, R.N. and Rao, R.R. (1986). Normal Approximation and Asymptotic Expansions. New York: Wiley. · Zbl 1222.41002
[8] Bickel, P.J. and Freedman, D.A. (1981). Some asymptotic theory for the bootstrap. Ann. Statist.9 1196-1217. · Zbl 0449.62034 · doi:10.1214/aos/1176345637
[9] Chatterjee, S. and Bose, A. (2005). Generalized bootstrap for estimating equations. Ann. Statist.33 414-436. · Zbl 1065.62073 · doi:10.1214/009053604000000904
[10] Chatterjee, A. and Lahiri, S.N. (2013). Rates of convergence of the adaptive LASSO estimators to the oracle distribution and higher order refinements by the bootstrap. Ann. Statist.41 1232-1259. · Zbl 1293.62153 · doi:10.1214/13-AOS1106
[11] Cheng, G. (2015). Moment consistency of the exchangeably weighted bootstrap for semiparametric M-estimation. Scand. J. Stat.42 665-684. · Zbl 1360.62111 · doi:10.1111/sjos.12128
[12] Cheng, G. and Huang, J.Z. (2010). Bootstrap consistency for general semiparametric M-estimation. Ann. Statist.38 2884-2915. · Zbl 1200.62042 · doi:10.1214/10-AOS809
[13] Das, D. and Lahiri, S.N. (2017). Supplement to “Second order correctness of perturbation bootstrap M-estimator of multiple linear regression parameter”. DOI:10.3150/17-BEJ1001SUPP.
[14] Davidson, R. and Flachaire, E. (2008). The wild bootstrap, tamed at last. J. Econometrics146 162-169. · Zbl 1418.62183 · doi:10.1016/j.jeconom.2008.08.003
[15] Davidson, R. and MacKinnon, J.G. (2010). Wild bootstrap tests for IV regression. J. Bus. Econom. Statist.28 128-144. · Zbl 1198.62035 · doi:10.1198/jbes.2009.07221
[16] Efron, B. (1979). Bootstrap methods: Another look at the jackknife. Ann. Statist.7 1-26. · Zbl 0406.62024 · doi:10.1214/aos/1176344552
[17] El Bantli, F. (2004). M-estimation in linear models under nonstandard conditions. J. Statist. Plann. Inference121 231-248. · Zbl 1038.62026 · doi:10.1016/S0378-3758(03)00113-7
[18] Feng, X., He, X. and Hu, J. (2011). Wild bootstrap for quantile regression. Biometrika98 995-999. · Zbl 1228.62053 · doi:10.1093/biomet/asr052
[19] Freedman, D.A. (1981). Bootstrapping regression models. Ann. Statist.9 1218-1228. · Zbl 0449.62046 · doi:10.1214/aos/1176345638
[20] Fuk, D.H. and Nagaev, S.V. (1971). Probabilistic inequalities for sums of independent random variables. Teor. Verojatnost. i Primenen.16 660-675. · Zbl 0259.60024
[21] Haeusler, E., Mason, D.M. and Newton, M.A. (1991). Weighted bootstrapping of means. CWI Quarterly4 213-228. · Zbl 0745.62039
[22] Hall, P. (1992). The Bootstrap and Edgeworth Expansion. Springer Series in Statistics. New York: Springer. · Zbl 0744.62026
[23] Hlávka, Z. (2003). Asymptotic properties of robust three-stage procedure based on bootstrap for \(M\)-estimator. J. Statist. Plann. Inference115 637-656. · Zbl 1015.62027
[24] Hu, F. (1996). Efficiency and robustness of a resampling M-estimator in the linear model. J. Multivariate Anal.78 252-271. · Zbl 1057.62034 · doi:10.1006/jmva.2000.1951
[25] Hu, F. and Kalbfleisch, D.J. (2000). The estimating function bootstrap. Canad. J. Statist.28 449-499. · Zbl 0977.62045 · doi:10.2307/3315958
[26] Huber, P.J. (1981). Robust Statistics. Wiley Series in Probability and Mathematical Statistics. New York: Wiley. · Zbl 0536.62025
[27] Jin, Z., Ying, Z. and Wei, L.J. (2001). A simple resampling method by perturbing the minimand. Biometrika88 381-390. · Zbl 0984.62033 · doi:10.1093/biomet/88.2.381
[28] Karabulut, I.K. and Lahiri, S.N. (1997). Two-term Edgeworth expansion for M-estimators of a linear regression parameter without Cramer-type conditions and an application to the bootstrap. J. Aust. Math. Soc. A62 361-370. · Zbl 0883.62031 · doi:10.1017/S1446788700001063
[29] Kline, P. and Santos, A. (2012). A score based approach to wild bootstrap inference. J. Econometric Methods1 23-41. · Zbl 1279.62076 · doi:10.1515/2156-6674.1006
[30] Lahiri, S.N. (1992). Bootstrapping \(M\)-estimators of a multiple linear regression parameter. Ann. Statist.20 1548-1570. · Zbl 0792.62058 · doi:10.1214/aos/1176348784
[31] Lahiri, S.N. (1994). On two-term Edgeworth expansions and bootstrap approximations for Studentized multivariate M-estimators. Sankhya, Ser. A56 201-226. · Zbl 0834.62019
[32] Lahiri, S.N. (1996). On Edgeworth expansion and moving block bootstrap for Studentized M-estimators in multiple linear regression models. J. Multivariate Anal.56 42-59. · Zbl 0864.62028 · doi:10.1006/jmva.1996.0003
[33] Lahiri, S.N. and Zhu, J. (2006). Resampling methods for spatial regression models under a class of stochastic designs. Ann. Statist.34 1774-1813. · Zbl 1246.62117 · doi:10.1214/009053606000000551
[34] Lee, S.M.S. (2012). General M-estimation and its bootstrap. J. Korean Statist. Soc.41 471-490. · Zbl 1296.62074 · doi:10.1016/j.jkss.2012.02.004
[35] Liu, R.Y. (1988). Bootstrap procedures under some non-IID models. Ann. Statist.16 1696-1708. · Zbl 0655.62031 · doi:10.1214/aos/1176351062
[36] Ma, S. and Kosorok, M.R. (2004). Robust semiparametric M-estimation and the weighted bootstrap. J. Multivariate Anal.96 190-217. · Zbl 1073.62030 · doi:10.1016/j.jmva.2004.09.008
[37] Mammen, E. (1993). Bootstrap and wild bootstrap for high dimensional linear models. Ann. Statist.21 255-285. · Zbl 0771.62032 · doi:10.1214/aos/1176349025
[38] Mason, D.M. and Newton, M.A. (1992). A rank statistics approach to the consistency of a general bootstrap. Ann. Statist.20 1611-1624. · Zbl 0777.62045 · doi:10.1214/aos/1176348787
[39] Minnier, J., Tian, L. and Cai, T. (2011). A perturbation method for inference on regularized regression estimates. J. Amer. Statist. Assoc.106 1371-1382. · Zbl 1323.62076 · doi:10.1198/jasa.2011.tm10382
[40] Navidi, W. (1989). Edgeworth expansions for bootstrapping regression models. Ann. Statist.17 1472-1478. · Zbl 0694.62011 · doi:10.1214/aos/1176347375
[41] Qumsiyeh, M.B. (1990). Edgeworth expansion in regression models. J. Multivariate Anal.35 86-101. · Zbl 0705.62028 · doi:10.1016/0047-259X(90)90017-C
[42] Qumsiyeh, M.B. (1994). Bootstrapping and empirical Edgeworth expansions in multiple linear regression models. Comm. Statist. Theory Methods23 3227-3239. · Zbl 0823.62019 · doi:10.1080/03610929408831443
[43] Rao, C.R. and Zhao, L.C. (1992). Approximation to the distribution of \(M\)-estimates in linear models by randomly weighted bootstrap. Sankhya, Ser. A54 323-331. · Zbl 0773.62010
[44] Rubin, D.B. (1981). The Bayesian bootstrap. Ann. Statist.9 130-134.
[45] Wang, X.M. and Zhou, W. (2004). Bootstrap approximation to the distribution of M-estimates in a linear model. Acta Math. Sin. (Engl. Ser.) 20 93-104. · Zbl 1140.62317 · doi:10.1007/s10114-003-0246-6
[46] Wu, C.-F.J. (1986). Jackknife, bootstrap and other resampling methods in regression analysis. Ann. Statist.14 1261-1350. With discussion and a rejoinder by the author. · Zbl 0618.62072 · doi:10.1214/aos/1176350142
[47] You, J. and Chen, G. (2006). Wild bootstrap estimation in partially linear models with heteroscedasticity. Statist. Probab. Lett.76 340-348. · Zbl 1086.62059 · doi:10.1016/j.spl.2005.08.027
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.