×

A unified approach to sufficient dimension reduction. (English) Zbl 1432.62119

Summary: Through investigating a recently introduced sufficient dimension reduction method with Hellinger index, this article shows that the generalized Hellinger index unifies three existing dimension reduction methods: kernel discriminant analysis, sliced regression and density minimum average variance estimation, with certain weight functions. The Hellinger index is then extended to regression models with multivariate responses. Furthermore, new algorithms based on Hellinger index to estimate the dual central subspaces and to enable variable selection for sparse models are proposed. Simulation studies and a real data analysis demonstrate the efficacy of the proposed approaches.

MSC:

62G08 Nonparametric regression and quantile regression
62H12 Estimation in multivariate analysis
62H20 Measures of association (correlation, canonical correlation, etc.)
62G07 Density estimation
62H30 Classification and discrimination; cluster analysis (statistical aspects)
62-08 Computational methods for problems pertaining to statistics
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Cook, R. D., Regression graphics: ideas for studying regressions through graphics, (1998), Wiley New York · Zbl 0903.62001
[2] Cook, R. D., Principal Hessian directions revisited (with discussion), J. Amer. Statist. Assoc., 93, 84-100, (1998) · Zbl 0922.62057
[3] Cook, R. D.; Ni, L., Sufficient dimension reduction via inverse regression: a minimum discrepancy approach, J. Amer. Statist. Assoc., 100, 410-428, (2005) · Zbl 1117.62312
[4] Cook, R. D.; Weisberg, S., Discussion of Li 1991, J. Amer. Statist. Assoc., 86, 328-332, (1991) · Zbl 1353.62037
[5] Eaton, M. L.; Tyler, D., The asymptotic distribution of singular values with applications to canonical correlations and correspondence analysis, J. Multivariate Anal., 50, 238-264, (1994) · Zbl 0805.62020
[6] Härdle, W.; Stoker, T. M., Investigating smooth multiple regression by method of average derivatives, J. Amer. Statist. Assoc., 84, 986-995, (1989) · Zbl 0703.62052
[7] Hernández, A.; Velilla, S., Dimension reduction in nonparametric kernel discriminant analysis, J. Comput. Graph. Statist., 14, 847-866, (2005)
[8] Hilafu, H.; Yin, X., Sufficient dimension reduction in multivariate regressions with categorical predictors, Comput. Statist. Data Anal., 63, 139-147, (2013) · Zbl 1468.62077
[9] Hristache, M.; Juditsky, A.; Polzehl, J.; Spokoiny, V., Structure adaptive approach to dimension reduction, Ann. Statist., 29, 1537-1566, (2001) · Zbl 1043.62052
[10] Hsing, T., Nearest neighbor inverse regression, Ann. Statist., 27, 697-731, (1999) · Zbl 0951.62034
[11] Iaci, R.; Sriram, T. N.; Yin, X., Multivariate association and dimension reduction: a generalization of canonical correlation analysis, Biometrics, 66, 1107-1118, (2010) · Zbl 1274.62422
[12] Iaci, R.; Yin, X.; Sriram, T. N.; Klingenberg, C. P., An informational measure of association and dimension reduction for multiple sets and groups with applications in morphometric analysis, J. Amer. Statist. Assoc., 103, 1166-1176, (2008) · Zbl 1205.62070
[13] Iaci, R.; Yin, X.; Zhu, L.-X., The dual central subspaces in dimension reduction, J. Multivariate Anal., 145, 178-189, (2016) · Zbl 1331.62289
[14] Kullback, S., Information theory and statistics, (1959), Wiley New York · Zbl 0149.37901
[15] Li, K. C., Sliced inverse regression for dimension reduction (with discussion), J. Amer. Statist. Assoc., 86, 316-342, (1991) · Zbl 0742.62044
[16] Li, K. C., On principal Hessian directions for data visualization and dimension reduction: another application of stein’s lemma, J. Amer. Statist. Assoc., 87, 1025-1039, (1992) · Zbl 0765.62003
[17] Li, K.C., 2000. High dimensional data analysis via the SIR/PHD approach. Lecture notes obtained from http://www.stat.ucla.edu/ kcli/sir-PHD.pdf; Li, K.C., 2000. High dimensional data analysis via the SIR/PHD approach. Lecture notes obtained from http://www.stat.ucla.edu/ kcli/sir-PHD.pdf
[18] Li, K. C.; Aragon, Y.; Shedden, K.; Agnan, C. T., Dimension reduction for multivariate response data, J. Amer. Statist. Assoc., 98, 99-109, (2003) · Zbl 1047.62059
[19] Li, B.; Wen, S.-Q.; Zhu, L.-X., On a projective resampling method for dimension reduction with multivariate responses, J. Amer. Statist. Assoc., 103, 1177-1186, (2008) · Zbl 1205.62067
[20] Li, L.; Yin, X., Sliced inverse regression with regularizations, Biometrics, 64, 124-131, (2008) · Zbl 1139.62055
[21] Li, B.; Zha, H.; Chiaromonte, F., Contour regression: a general approach to dimension reduction, Ann. Statist., 33, 1580-1616, (2005) · Zbl 1078.62033
[22] Liese, F.; Vajda, I., On divergence and information in statistics and information theory, IEEE Trans. Inform. Theory, 52, 4394-4412, (2006) · Zbl 1287.94025
[23] Luo, R.; Wang, H.; Tsai, C. L., Contour projected dimension reduction, Ann. Statist., 37, 3743-3778, (2009) · Zbl 1360.62184
[24] Ma, Y.; Zhu, L.-P., A semiparametric approach to dimension reduction, J. Amer. Statist. Assoc., 107, 168-179, (2012) · Zbl 1261.62037
[25] Ma, Y.; Zhu, L.-P., Efficiency loss and the linearity condition in dimension reduction, Biometrika, 100, 371-383, (2013) · Zbl 1284.62262
[26] Ma, Y.; Zhu, L.-P., Efficient estimation in sufficient dimension reduction, Ann. Statist., 41, 250-268, (2013) · Zbl 1347.62089
[27] Ma, Y.; Zhu, L.-P., On estimation efficiency of the central mean subspace, J. R. Stat. Soc. Ser. B, 76, 885-901, (2014) · Zbl 1411.62104
[28] Nierenberg, D. W.; Stukel, T. A.; Baron, J. A.; Dain, B. J.; Greenberg, E. R., Determinants of plasma levels of beta-carotene and retinol, Amer. J. Epidemiol., 130, 511-521, (1989)
[29] Samarov, A. M., Exploring regression structure using nonparametric function estimation, J. Amer. Statist. Assoc., 88, 836-847, (1993) · Zbl 0790.62035
[30] Setodji, C. M.; Cook, R. D., K-means inverse regression, Technometrics, 46, 421-429, (2004)
[31] Vajda, I., Theory of statistical inference and information, (1989), Kluwer Boston, Massachusetts · Zbl 0678.62035
[32] Wang, H.; Xia, Y., Sliced regression for dimension reduction, J. Amer. Statist. Assoc., 103, 811-821, (2008) · Zbl 1306.62168
[33] Wang, Q.; Yin, X.; Critchley, F., Dimension reduction based on Hellinger integral, Biometrika, 102, 95-106, (2015) · Zbl 1345.62068
[34] Xia, Y., A constructive approach to the estimation of dimension reduction directions, Ann. Statist., 35, 2654-2690, (2007) · Zbl 1360.62196
[35] Xia, Y.; Tong, H.; Li, W.; Zhu, L.-X., An adaptive estimation of dimension reduction space, J. R. Stat. Soc. Ser. B, 64, 363-410, (2002) · Zbl 1091.62028
[36] Yin, X., Canonical correlation analysis based on information theory, J. Multivariate Anal., 91, 161-176, (2004) · Zbl 1058.62049
[37] Yin, X.; Cook, R. D., Dimension reduction for the conditional \(k\)th moment in regression, J. R. Stat. Soc. Ser. B, 64, 159-175, (2002) · Zbl 1067.62042
[38] Yin, X.; Cook, R. D., Estimating central subspaces via inverse third moments, Biometrika, 90, 113-125, (2003) · Zbl 1035.62051
[39] Yin, X.; Cook, R. D., Direction estimation in single-index regressions, Biometrika, 92, 371-384, (2005) · Zbl 1094.62054
[40] Yin, X.; Li, B.; Cook, R. D., Successive direction extraction for estimating the central subspace in a multiple-index regression, J. Multivariate Anal., 99, 1733-1757, (2008) · Zbl 1144.62030
[41] Yin, X.; Sriram, T. N., Common canonical variates for independent groups using information theory, Statist. Sinica, 18, 335-353, (2008) · Zbl 1137.62040
[42] Zhu, Y.; Zeng, P., Fourier methods for estimating the central subspace and the central mean subspace in regression, J. Amer. Statist. Assoc., 101, 1638-1651, (2006) · Zbl 1171.62325
[43] Zhu, L.-P.; Zhu, L.-X.; Wen, S.-Q., On dimension reduction in regressions with multivariate responses, Statist. Sinica, 20, 1291-1307, (2010) · Zbl 1507.62267
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.