×

Learning rates for kernel-based expectile regression. (English) Zbl 1480.62067

Summary: Conditional expectiles are becoming an increasingly important tool in finance as well as in other areas of applications. We analyse a support vector machine type approach for estimating conditional expectiles and establish learning rates that are minimax optimal modulo a logarithmic factor if Gaussian RBF kernels are used and the desired expectile is smooth in a Besov sense. As a special case, our learning rates improves the best known rates for kernel-based least squares regression in aforementioned scenario. Key ingredients of our statistical analysis are a general calibration inequality for the asymmetric least squares loss, a corresponding variance bound as well as an improved entropy number bound for Gaussian RBF kernels.

MSC:

62G08 Nonparametric regression and quantile regression
60B12 Limit theorems for vector-valued random variables (infinite-dimensional case)
68T05 Learning and adaptive systems in artificial intelligence
62P20 Applications of statistics to economics

Software:

erboost; GURLS
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Abdous, B., & Remillard, B. (1995). Relating quantiles and expectiles under weighted-symmetry. Annals of the Institute of Statistical Mathematics, 47, 371-384. https://doi.org/10.1007/bf00773468. · Zbl 0833.62013 · doi:10.1007/bf00773468
[2] Adams, R. A., & Fournier, J. J. F. (2003). Sobolev spaces (2nd ed.). New York: Academic Press. https://doi.org/10.1016/s0079-8169(03)x8001-0. · Zbl 1098.46001 · doi:10.1016/s0079-8169(03)x8001-0
[3] Aragon, Y., Casanova, S., Chambers, R., & Leconte, E. (2005). Conditional ordering using nonparametric expectiles. Journal of Official Statistics, 21, 617-633.
[4] Bauer, F., Pereverzev, S., & Rosasco, L. (2007). On regularization algorithms in learning theory. Journal of Complexity, 23, 52-72. https://doi.org/10.1016/j.jco.2006.07.001. · Zbl 1109.68088 · doi:10.1016/j.jco.2006.07.001
[5] Bellini, F., Klar, B., Müller, A., & Gianin, R. E. (2014). Generalized quantiles as risk measures. Insurance: Mathematics and Economics, 54, 41-48. https://doi.org/10.1016/j.insmatheco.2013.10.015. · Zbl 1303.91089 · doi:10.1016/j.insmatheco.2013.10.015
[6] Blanchard, G., Bousquet, O., & Massart, P. (2008). Statistical performance of support vector machines. The Annals of Statistics,. https://doi.org/10.1214/009053607000000839. · Zbl 1133.62044 · doi:10.1214/009053607000000839
[7] Caponnetto, A., & De Vito, E. (2007). Optimal rates for the regularized least-squares algorithm. Foundations of Computational Mathematics, 7, 331-368. https://doi.org/10.1007/s10208-006-0196-8. · Zbl 1129.68058 · doi:10.1007/s10208-006-0196-8
[8] Chen, D., Wu, Q., Ying, Y., & Zhou, D. (2004). Support vector machine soft margin classifiers: Error analysis. Journal of Machine Learning Research, 5, 1143-1175. · Zbl 1222.68167
[9] Christmann, A., & Steinwart, I. (2007). How SVMs can estimate quantiles and the median. In: Advances in neural information processing systems (pp. 305-312).
[10] Cucker, F., & Smale, S. (2002). On the mathematical foundations of learning. Bulletin of the American Mathematical Society, 39, 1-49. https://doi.org/10.1090/s0273-0979-01-00923-5. · Zbl 0983.68162 · doi:10.1090/s0273-0979-01-00923-5
[11] De Vito, E., Caponnetto, A., & Rosasco, L. (2005). Model selection for regularized least-squares algorithm in learning theory. Foundations of Computational Mathematics, 5, 59-85. https://doi.org/10.1007/s10208-004-0134-1. · Zbl 1083.68106 · doi:10.1007/s10208-004-0134-1
[12] DeVore, R. A. (1998). Nonlinear approximation. Acta Numerica, 7, 51-150. · Zbl 0931.65007 · doi:10.1017/S0962492900002816
[13] DeVore, R. A., & Popov, V. A. (1988). Interpolation of Besov spaces. Transactions of the American Mathematical Society, 305, 397-414. · Zbl 0646.46030 · doi:10.1090/S0002-9947-1988-0920166-3
[14] DeVore, R. A., & Sharpley, R. C. (1993). Besov spaces on domains in \[\mathbb{R}^d\] Rd. Transactions of the American Mathematical Society, 335, 843-864. · Zbl 0766.46015
[15] Eberts, M., & Steinwart, I. (2013). Optimal regression rates for SVMs using Gaussian kernels. Electronic Journal of Statistics, 7, 1-42. https://doi.org/10.1214/12-ejs760. · Zbl 1337.62073 · doi:10.1214/12-ejs760
[16] Edmunds, D. E., & Triebel, H. (2008). Function spaces, entropy numbers, differential operators. Cambridge: Cambridge University Press. https://doi.org/10.1017/cbo9780511662201. · Zbl 1143.46001 · doi:10.1017/cbo9780511662201
[17] Efron, B. (1991). Regression percentiles using asymmetric squared error loss. Statistical Science, 1, 93-125. · Zbl 0822.62054
[18] Farooq, M., & Steinwart, I. (2017). An SVM-like approach for expectile regression. Computational Statistics & Data Analysis, 109, 159-181. https://doi.org/10.1016/j.csda.2016.11.010. · Zbl 1466.62064 · doi:10.1016/j.csda.2016.11.010
[19] Glasmachers, T., & Igel, C. (2006). Maximum-gain working set selection for SVMs. Journal of Machine Learning Research, 7, 1437-1466. · Zbl 1222.90040
[20] Guler, K., Ng, P.T., & Xiao, Z. (2014). Mincer-Zarnovitz quantile and expectile regressions for forecast evaluations under asymmetric loss functions. Northern Arizona University, The WA Franke College of Business Working Paper Series 14-01.
[21] Györfi, L., Kohler, M., Krzyzak, A., & Walk, H. (2002). A distribution-free theory of nonparametric regression. New York: Springer. · Zbl 1021.62024 · doi:10.1007/b97848
[22] Hamidi, B., Maillet, B., & Prigent, J. L. (2014). A dynamic autoregressive expectile for time-invariant portfolio protection strategies. Journal of Economic Dynamics and Control, 46, 1-29. https://doi.org/10.1016/j.jedc.2014.05.005. · Zbl 1402.91915 · doi:10.1016/j.jedc.2014.05.005
[23] Kim, M., & Lee, S. (2016). Nonlinear expectile regression with application to value-at-risk and expected shortfall estimation. Computational Statistics & Data Analysis, 94, 1-19. https://doi.org/10.1016/j.csda.2015.07.011. · Zbl 1468.62101 · doi:10.1016/j.csda.2015.07.011
[24] Koenker, R., & Bassett, G, Jr. (1978). Regression quantiles. Econometrica, 46, 33-50. https://doi.org/10.2307/1913643. · Zbl 0373.62038 · doi:10.2307/1913643
[25] Meister, M., & Steinwart, I. (2016). Optimal learning rates for localized SVMs. Journal of Machine Learning Research, 17, 1-44. · Zbl 1433.68368
[26] Mendelson, S., & Neeman, J. (2010). Regularization in kernel learning. The Annals of Statistics, 38, 526-565. https://doi.org/10.1214/09-aos728. · Zbl 1191.68356 · doi:10.1214/09-aos728
[27] Newey, W. K., & Powell, J. L. (1987). Asymmetric least squares estimation and testing. Econometrica, 55, 819-847. https://doi.org/10.2307/1911031. · Zbl 0625.62047 · doi:10.2307/1911031
[28] Nikol’skii, S. M. (2012). Approximation of functions of several variables and imbedding theorems (Vol. 205). Berlin: Springer. https://doi.org/10.1007/978-3-642-65711-5. · doi:10.1007/978-3-642-65711-5
[29] Schnabel, S., & Eilers, P. (2009). An analysis of life expectancy and economic production using expectile frontier zones. Demographic Research, 21, 109-134. https://doi.org/10.4054/demres.2009.21.5. · doi:10.4054/demres.2009.21.5
[30] Sobotka, F., & Kneib, T. (2012). Geoadditive expectile regression. Computational Statistics & Data Analysis, 56, 755-767. https://doi.org/10.1016/j.csda.2010.11.015. · Zbl 1241.62058 · doi:10.1016/j.csda.2010.11.015
[31] Sobotka, F., Radice, R., Marra, G., & Kneib, T. (2013). Estimating the relationship between women’s education and fertility in Botswana by using an instrumental variable approach to semiparametric expectile regression. Journal of the Royal Statistical Society Series C: Applied Statistics, 62, 25-45. https://doi.org/10.1111/j.1467-9876.2012.01050.x. · doi:10.1111/j.1467-9876.2012.01050.x
[32] Steinwart, I. (2007). How to compare different loss functions and their risks. Constructive Approximation, 26, 225-287. https://doi.org/10.1007/s00365-006-0662-3. · Zbl 1127.68089 · doi:10.1007/s00365-006-0662-3
[33] Steinwart, I. (2009). Oracle inequalities for support vector machines that are based on random entropy numbers. Journal of Complexity, 25, 437-454. https://doi.org/10.1016/j.jco.2009.06.002. · Zbl 1192.68540 · doi:10.1016/j.jco.2009.06.002
[34] Steinwart, I., & Christmann, A. (2008). Support vector machines. New York: Springer. https://doi.org/10.1007/978-0-387-77242-4. · Zbl 1203.68171 · doi:10.1007/978-0-387-77242-4
[35] Steinwart, I., & Christmann, A. (2011). Estimating conditional quantiles with the help of the pinball loss. Bernoulli, 17, 211-225. https://doi.org/10.3150/10-bej267. · Zbl 1284.62235 · doi:10.3150/10-bej267
[36] Steinwart, I., Hush, D., & Scovel, C. (2006). An oracle inequality for clipped regularized risk minimizers. In: Advances in neural information processing systems (pp. 1321-1328). · Zbl 1127.68090
[37] Steinwart, I., Hush, D. R., & Scovel, C. (2009). Optimal rates for regularized least squares regression. In: 22nd Annual conference on learning theory (pp. 79-93).
[38] Steinwart, I., Hush, D., & Scovel, C. (2011). Training SVMs without offset. Journal of Machine Learning Research, 12, 141-202. · Zbl 1280.68199
[39] Steinwart, I., Pasin, C., Williamson, R., & Zhang, S. (2014). Elicitation and identification of properties. In: M. F. Balcan & C. Szepesvari (Eds.), JMLR workshop and conference proceedings. Volume 35: Proceedings of the 27th conference on learning theory 2014 (pp. 482-526).
[40] Tacchetti, A., Mallapragada, P. K., Santoro, M., & Rosasco, L. (2013). GURLS: A least squares library for supervised learning. Journal of Machine Learning Research, 14, 3201-3205. · Zbl 1317.68191
[41] Takeuchi, I., Le, Q. V., Sears, T. D., & Smola, A. J. (2006). Nonparametric quantile estimation. Journal of Machine Learning Research, 7, 1231-1264. · Zbl 1222.68316
[42] Tartar, L. (2007). An introduction to sobolev spaces and interpolation spaces. Berlin: Springer. https://doi.org/10.1007/978-3-540-71483-5. · Zbl 1126.46001 · doi:10.1007/978-3-540-71483-5
[43] Taylor, J. W. (2008). Estimating value at risk and expected shortfall using expectiles. Journal of Financial Economics, 6, 231-252. https://doi.org/10.1093/jjfinec/nbn001. · doi:10.1093/jjfinec/nbn001
[44] van der Vaart, A. W., & van Zanten, J. H. (2009). Adaptive Bayesian estimation using a Gaussian random field with inverse gamma bandwidth. The Annals of Statistics, 37, 2655-2675. https://doi.org/10.1214/08-aos678. · Zbl 1173.62021 · doi:10.1214/08-aos678
[45] Wang, Y., Wang, S., & Lai, K. K. (2011). Measuring financial risk with generalized asymmetric least squares regression. Applied Soft Computing, 11(8), 5793-5800. https://doi.org/10.1016/j.asoc.2011.02.018. · doi:10.1016/j.asoc.2011.02.018
[46] Wu, Q., Ying, Y., & Zhou, D. X. (2006). Learning rates of least square regularized regression. Foundations of Computational Mathematics, 6, 171-192. https://doi.org/10.1007/s10208-004-0155-9. · Zbl 1100.68100 · doi:10.1007/s10208-004-0155-9
[47] Xu, Q., Liu, X., Jiang, C., & Yu, K. (2016). Nonparametric conditional autoregressive expectile model via neural network with applications to estimating financial risk. Applied Stochastic Models in Business and Industry, 32, 882-908. https://doi.org/10.1002/asmb.2212. · Zbl 1420.91531 · doi:10.1002/asmb.2212
[48] Yang, Y., & Zou, H. (2015). Nonparametric multiple expectile regression via ER-Boost. Journal of Statistical Computation and Simulation, 85, 1442-1458. https://doi.org/10.1080/00949655.2013.876024. · Zbl 1457.62124 · doi:10.1080/00949655.2013.876024
[49] Yang, Y., Zhang, T., & Zou, H. (2017). Flexible expectile regression in reproducing kernel Hilbert spaces. Technometrics, 1, 1-10. https://doi.org/10.1080/00401706.2017.1291450. · doi:10.1080/00401706.2017.1291450
[50] Yao, Q., & Tong, H. (1996). Asymmetric least squares regression estimation: A nonparametric approach. Journal of Nonparametric Statistics, 6, 273-292. https://doi.org/10.1080/10485259608832675. · Zbl 0879.62038 · doi:10.1080/10485259608832675
[51] Zhang, B. (1994). Nonparametric regression expectiles. Journal of Nonparametric Statistics, 3, 255-275. https://doi.org/10.1080/10485259408832586. · Zbl 1380.62182 · doi:10.1080/10485259408832586
[52] Ziegel, J. F. (2016). Coherence and elicitability. Mathematical Finance, 26, 901-918. https://doi.org/10.1111/mafi.12080. · Zbl 1390.91336 · doi:10.1111/mafi.12080
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.