×

Nonlinear surface regression with dimension reduction method. (English) Zbl 1443.62111

Summary: This paper considers nonlinear regression analysis with a scalar response and multiple predictors. An unknown regression function is approximated by radial basis function models. The coefficients are estimated in the context of \(M\)-estimation. It is known that ordinary \(M\)-estimation leads to overfitting in nonlinear regression. The purpose of this paper is to construct a smooth estimator. The proposed method in this paper is conducted by a two-step procedure. First, the sufficient dimension reduction methods are applied to the response and radial basis functions for transforming the large number of radial bases to a small number of linear combinations of the radial bases without loss of information. In the second step, a multiple linear regression model between a response and the transformed radial bases is assumed and the ordinary \(M\)-estimation is applied. Thus, the final estimator is also obtained as a linear combination of radial bases. The validity and an asymptotic study of the proposed method are explored. A simulation and data example are addressed to confirm the behavior of the proposed method.

MSC:

62G08 Nonparametric regression and quantile regression
62-08 Computational methods for problems pertaining to statistics

Software:

ROBPCA; gamair; SemiPar; gss
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Alimadad, A; Salibian-Barrera, M, An outlier-robust fit for generalized additive models with applications to disease outbreak detection, J. Am. Statist. Assoc., 106, 719-731, (2011) · Zbl 1232.62142 · doi:10.1198/jasa.2011.tm09654
[2] Buja, A; Hastie, T; Tibshirani, R, Linear smoothers and additive models, Ann. Stat., 17, 453-555, (1989) · Zbl 0689.62029 · doi:10.1214/aos/1176347115
[3] Chen, X; Zhou, C; Cook, RD, Coordinate-independent sparse sufficient dimension reduction and variable selection, Ann. Stat., 38, 3696-3723, (2010) · Zbl 1204.62107 · doi:10.1214/10-AOS826
[4] Cook, RD; Weisberg, S, Discussion of ‘sliced inverse regression for dimension reduction” by K.C. Li, J. Am. Stat. Assoc., 86, 328-332, (1991) · Zbl 1353.62037
[5] Cook, RD, Principle Hessian directions revisited, J. Am. Stat. Assoc., 93, 84-100, (1998) · Zbl 0922.62057 · doi:10.1080/01621459.1998.10474090
[6] Cook, RD; Li, B, Dimension reduction for the conditional Mean in regression, Ann. Stat., 30, 455-474, (2002) · Zbl 1012.62035 · doi:10.1214/aos/1021379861
[7] de Boor, C.: A Practical Guide to Splines, revised edn. Springer, Berlin (2001) · Zbl 0987.65015
[8] Diaconis, P; Freedman, D, Asymptotics of graphical projection pursuit, Ann. Stat., 12, 793-815, (1984) · Zbl 0559.62002 · doi:10.1214/aos/1176346703
[9] Ganguli, B; Wand, MP, Feature significance in geostatistics, J. Comput. Gr. Stat., 13, 954-973, (2004) · doi:10.1198/106186004X12515
[10] Gu, C.: Smoothing Spline ANOVA Models. Springer, Berlin (2002) · Zbl 1051.62034 · doi:10.1007/978-1-4757-3683-0
[11] Green, P.J., Silverman, B.W.: Nonparametric Regression and Generalized Linear Models. Chapman and Hall, London (1993) · Zbl 0832.62032
[12] Hall, P; Li, KC, On almost linearity of low dimensional projections from high dimensional data, Ann. Stat., 21, 867-889, (1993) · Zbl 0782.62065 · doi:10.1214/aos/1176349155
[13] Hsing, T; Carroll, RJ, An asymptotic theory for sliced inverse regression, Ann. Stat., 20, 623-1145, (1992) · Zbl 0821.62019 · doi:10.1214/aos/1176348669
[14] Huber, P., Ronchetti, E.M.: Robust Statistics, 2nd edn. Wiley, Hoboken (2009) · Zbl 1276.62022 · doi:10.1002/9780470434697
[15] Hubert, M; Rousseeuw, PJ; Branden, KV, Robpca: a new approach to robust principal component analysis, Technometrics, 47, 64-79, (2005) · doi:10.1198/004017004000000563
[16] Koenker, R; Bassett, G, Regression quantiles, Econometrica, 46, 33-50, (1978) · Zbl 0373.62038 · doi:10.2307/1913643
[17] Lee, TCM; Oh, H, Robust penalized regression spline Fitting with application to additive mixed modeling, Comput. Stat., 22, 159-171, (2007) · Zbl 1194.62046 · doi:10.1007/s00180-007-0031-6
[18] Li, KC, Sliced inverse regression for dimension reduction, J. Am. Stat. Assoc., 102, 997-1008, (1991) · Zbl 1469.62300 · doi:10.1198/016214507000000536
[19] Li, KC, On principal Hessian directions for data visualization and dimension reduction: another application of stein’s lemma, J. Am. Stat. Assoc., 87, 1025-1039, (1992) · Zbl 0765.62003 · doi:10.1080/01621459.1992.10476258
[20] Li, B; Zha, H; Chiaromonte, F, Contour regression: a general approach to dimension reduction, Ann. Stat., 33, 1580-1616, (2005) · Zbl 1078.62033 · doi:10.1214/009053605000000192
[21] Li, B; Wang, S, On directional regression for dimension reduction, J. Am. Stat. Assoc., 33, 1580-1616, (2007) · Zbl 1078.62033
[22] Li, L, Sparse sufficient dimension reduction, Biometrika, 94, 603-611, (2007) · Zbl 1135.62062 · doi:10.1093/biomet/asm044
[23] Li, Y; Zhu, LX, Asymptotics for sliced average variance estimation, Ann. Stat, 35, 41-69, (2007) · Zbl 1114.62053 · doi:10.1214/009053606000001091
[24] Li, F; Villani, M, Efficient Bayesian multivariate surface regression, Scand. J. Stat, 40, 706-723, (2013) · Zbl 1283.62052 · doi:10.1111/sjos.12022
[25] Nychka, DW; Schimek, M (ed.), Spatial process estimates as smoothers, (2000), Heidelberg
[26] Pollard, D, Asymptotics for least absolute deviation regression estimators, Econom. Theory, 7, 186-199, (1991) · doi:10.1017/S0266466600004394
[27] Ruppert, D., Wand, M.P., Carroll, R.J.: Semiparametric Regression. Cambridge University Press, Cambridge (2003) · Zbl 1038.62042 · doi:10.1017/CBO9780511755453
[28] Wahba, G; Wang, Y; Gu, C; Klein, R; Klein, B, Smoothing spline ANOVA for exponential families, with application to the wisconsin epidemiological study of diabetic retinopathy, Ann. Stat., 23, 1865-1895, (1995) · Zbl 0854.62042 · doi:10.1214/aos/1034713638
[29] Wong, RK; Yao, F; Lee, TC, Robust estimation for generalized additive models, J. Comput. Gr. Stat., 23, 270-289, (2014) · doi:10.1080/10618600.2012.756816
[30] Wood, SN, Thin plate regression splines, J. R. Stat. Soc. B., 65, 95-114, (2003) · Zbl 1063.62059 · doi:10.1111/1467-9868.00374
[31] Wood, S.N.: Generalized Additive Models. An Introduction With R. Chapman & Hall/CRC, Boca Raton (2006) · Zbl 1087.62082
[32] Xiao, L; Li, Y; Ruppert, D, Fast bivariate \(P\)-splines: the sandwich smoother, J. R. Stat. Soc. B, 75, 577-599, (2013) · Zbl 1411.62109 · doi:10.1111/rssb.12007
[33] Zhu, L; Miao, B; Peng, H, On sliced inverse regression with high-dimensional covariates, J. Am. Stat. Assoc., 101, 630-643, (2006) · Zbl 1119.62331 · doi:10.1198/016214505000001285
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.