×

Error variance estimation via least squares for small sample nonparametric regression. (English) Zbl 1244.62061

Summary: We explore statistical properties of some difference-based approaches to estimate an error variance for small samples based on nonparametric regression which satisfies a Lipschitz condition. Our study is motivated by T. Tong and Y. Wang [Biometrika 92, No. 4, 821–830 (2005; Zbl 1151.62318)], who estimated the error variance using a least squares approach. They considered the error variance as the intercept in a simple linear regression which was obtained from the expectation of their lag-k J. Rice estimator [Ann. Stat. 12, 1215–1230 (1984; Zbl 0554.62035)]. Their variance estimators are highly dependent on the setting of a regressor and weight of their simple linear regression. Although this regressor and weight can be varied based on the characteristic of an unknown nonparametric mean function, Tong and Wang have used a fixed regressor and weight in a large sample and gave no indication of how to determine the regressor and the weight.
We propose a new approach via local quadratic approximation to determine this regressor and weight. Using our proposed regressor and weight, we estimate the error variance as the intercept of simple linear regression using both ordinary least squares and weighted least squares. Our approach applies to both small and large samples, while most existing difference-based methods are appropriate solely for large samples. We compare the performance of our approach with other existing approaches using an extensive simulation study. The advantage of our approach is demonstrated using a real data set.

MSC:

62G08 Nonparametric regression and quantile regression
62J05 Linear regression; mixed models
65C60 Computational problems in statistics (MSC2010)
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Carter, C. K.; Eagleson, G. K., A comparison of variance estimations in nonparametric regression, Journal of the Royal Statistical Society B, 54, 773-780 (1992)
[2] Dette, H.; Munk, A.; Wagner, T., Estimating the variance in nonparametric regression—What is a reasonable choice?, Journal of the Royal Statistical Society B, 60, 751-764 (1998) · Zbl 0944.62041
[3] Gasser, T.; Sroka, L.; Jennen-Steinmetz, C., Residual variance and residual pattern in nonlinear regression, Biometrika, 73, 625-633 (1986) · Zbl 0649.62035
[4] Hall, P.; Carroll, R. J., Variance function estimation in regression: the effect of estimating the mean, Journal of the Royal Statistical Society B, 51, 3-13 (1989) · Zbl 0672.62053
[5] Hall, P.; Kay, J. W.; Titterington, D. M., Asymptotically optimal difference-based estimation of Variance in nonparametric regression, Biometrika, 77, 521-528 (1990) · Zbl 1377.62102
[6] Müller, H. G.; Stadtmüller, U., Estimation of heteroscedasticity in regression analysis, Annals of Statistics, 51, 3-13 (1987)
[7] Neumann, M. H., Fully data-driven nonparametric variance estimators, Statistics, 25, 189-212 (1994) · Zbl 0811.62047
[8] Rice, J. A., Bandwidth choice for nonparametric variance estimators, Annals of Statistics, 12, 1215-1230 (1984) · Zbl 0554.62035
[9] Seifert, B.; Gasser, T.; Wolf, A., Nonparametric estimation of residual variance revisited, Biometrika, 80, 373-383 (1993) · Zbl 0799.62038
[10] Tong, T.; Wang, Y., Estimating residual variance in nonparametric regression using least squares, Biometrika, 92, 821-830 (2005) · Zbl 1151.62318
[11] Wahba, G., 1990. Spline Models for Observational Data. CBMS-NSF Regional Conference Series in Applied Mathematics, vol 59. SIAM, Philadelphia.; Wahba, G., 1990. Spline Models for Observational Data. CBMS-NSF Regional Conference Series in Applied Mathematics, vol 59. SIAM, Philadelphia. · Zbl 0813.62001
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.