×

A two stage \(k\)-monotone B-spline regression estimator: uniform Lipschitz property and optimal convergence rate. (English) Zbl 1392.62118

Summary: This paper considers \(k\)-monotone estimation and the related asymptotic performance analysis over a suitable Hölder class for general \(k\). A novel two stage \(k\)-monotone B-spline estimator is proposed: in the first stage, an unconstrained estimator with optimal asymptotic performance is considered; in the second stage, a \(k\)-monotone B-spline estimator is constructed (roughly) by projecting the unconstrained estimator onto a cone of \(k\)-monotone splines. To study the asymptotic performance of the second stage estimator under the sup-norm and other risks, a critical uniform Lipschitz property for the \(k\)-monotone B-spline estimator is established under the \(\ell_{\infty }\)-norm. This property uniformly bounds the Lipschitz constants associated with the mapping from a (weighted) first stage input vector to the B-spline coefficients of the second stage \(k\)-monotone estimator, independent of the sample size and the number of knots. This result is then exploited to analyze the second stage estimator performance and develop convergence rates under the sup-norm, pointwise, and \(L_{p}\)-norm (with \(p\in [1,\infty)\)) risks. By employing recent results in \(k\)-monotone estimation minimax lower bound theory, we show that these convergence rates are optimal.

MSC:

62G08 Nonparametric regression and quantile regression
62G20 Asymptotic properties of nonparametric inference
PDFBibTeX XMLCite
Full Text: DOI Euclid

References:

[1] Balabdaoui, F. and Wellner, J.A. (2010) Estimation of a \(k\)-monotone density: characterizations, consistency and minimax lower bounds., Stat. Neerl., 64, 45-70.
[2] Balabdaoui, F. and Wellner, J.A. (2007) Estimation of a \(k\)-monotone density: limit distribution theory and spline connection., Ann. Statist., 35, 2536-2564. · Zbl 1129.62019 · doi:10.1214/009053607000000262
[3] Beatson, R.K. (1981) Convex approximation by splines., SIAM J. Math. Anal., 12, 549-559. · Zbl 0458.41009 · doi:10.1137/0512048
[4] Beatson, R.K. (1982) Monotone and convex approximation by splines: error estimates and a curve fitting algorithm., SIAM J. Numer. Anal., 19, 1278-1285. · Zbl 0493.65002 · doi:10.1137/0719092
[5] Bellec, P. and Tsybakov, A. (2015) Sharp oracle bounds for monotone and convex regression through aggregation., J. Mach. Learn. Res., 16, 1879-1892. · Zbl 1351.62088
[6] Birke, M. and Dette, J. (2007) Estimating a convex function in nonparametric regression., Scand. J. Stat., 34, 384-404. · Zbl 1142.62019 · doi:10.1111/j.1467-9469.2006.00534.x
[7] Cator, E. (2011) Adaptivity and optimality of the monotone least squares estimator., Bernoulli, 17, 714-735. · Zbl 1345.62066 · doi:10.3150/10-BEJ289
[8] De Boor, C. (2001), A Practical Guide to Splines. Springer. · Zbl 0987.65015
[9] DeVore, C. (1977) Monotone approximation by splines., SIAM J. Math. Anal., 8 891-905. · Zbl 0368.41006 · doi:10.1137/0508068
[10] DeVore, R.A. and Lorentz, G.G. (1993), Constructive Approximation. Springer. · Zbl 0797.41016
[11] Dümbgen, L., Freitag, S., Jongbloed, G. (2004) Consistency of concave regression with an application to current-status data., Math. Methods Statist., 13, 69-81. · Zbl 1129.62033
[12] Eeckhoudt, L. and Schlesinger, H. (2013) Higher-order risk attitudes. In, Handbook of Insurance 41-57. Springer.
[13] Gao, F.C. (2008) Entropy estimate for \(k\)-monotone functions via small ball probability of integrated Brownian motions., Electron. Commun. Probab., 13, 121-130. · Zbl 1194.60027 · doi:10.1214/ECP.v13-1357
[14] Gao, F.C. and Wellner, J.A. (2009) On the rate of convergence of the maximum likelihood estimator of a \(k\)-monotone density., Sci. China Ser. A, 52, 1-14. · Zbl 1176.62031 · doi:10.1007/s11425-009-0102-y
[15] Giguelay, J. (2017) Estimation of a discrete probability under constraint of \(k\)-monotonicity., Electron. J. Stat., 11, 1-49. · Zbl 1357.62164 · doi:10.1214/16-EJS1220
[16] Golitschek, M.V. (2014) On the \(L_∞ \)-norm of the orthogonal projector onto splines: A short proof of A. Shardin’s theorem., J. Approx. Theory, 181, 30-42. · Zbl 1290.41003 · doi:10.1016/j.jat.2014.02.001
[17] Groeneboom, P., Jongbloed, F., and Wellner, J.A. (2001) Estimation of a convex function: Characterizations and asymptotic theory., Ann. Statist., 29, 1653-1698. · Zbl 1043.62027 · doi:10.1214/aos/1015345958
[18] Guntuboyina, A. and Sen, B. (2015) Global risk bounds and adaptation in univariate convex regression., Probab. Theory Related Fields, 63, 379-411. · Zbl 1327.62255 · doi:10.1007/s00440-014-0595-3
[19] Hu, Y.K. (1991) Convexity preserving approximation by free knot splines., SIAM J. Math. Anal., 22 1183-1191. · Zbl 0739.41009 · doi:10.1137/0522077
[20] Konovalov, V.N. and Leviatan, D. (2001) Estimates on the approximation of 3-monotone functions by 3-monotone quadratic splines., East J. Approx., 7, 333-349. · Zbl 1091.41008
[21] Konovalov, V.N. and Leviatan, D. (2003) Shape preserving widths of Sobolev-type classes of \(s\)-monotone functions on a finite interval., Israel J. Math., 133, 239-268. · Zbl 1038.41008 · doi:10.1007/BF02773069
[22] Kuipers, B.J., Chiu, C., Dalle Molle, D.T., and Throop, D.R. (1991). Higher-order derivative constraints in qualitative simulation., Artificial Intelligence, 51, 343-379. · Zbl 0751.68081 · doi:10.1016/0004-3702(91)90114-Y
[23] Lebair, T.M. (2016), Constrained Estimation and Approximation using Control, Optimization, and Spline Theory. Ph.D. Dissertation, University of Maryland, Baltimore County. http://pages.jh.edu/ tlebair1/Lebair_thesis.pdf
[24] Lebair, T.M., Shen, J., and Wang, X. (2016) Minimax lower bound and optimal estimation of convex functions in the sup-norm., IEEE Trans. Automat. Control, Vol. 62(7), pp. 3482-3487, 2017. · Zbl 1370.93258 · doi:10.1109/TAC.2016.2612543
[25] Mammen, E. (1991) Nonparametric regression under qualitative smoothness assumptions., Ann. Statist., 19, 741-759. · Zbl 0737.62039 · doi:10.1214/aos/1176348118
[26] Mammen, E. and van de Geer, S. (1997) Locally adaptive regression splines., Ann. Statist., 25, 387-413. · Zbl 0871.62040 · doi:10.1214/aos/1034276635
[27] Nemirovski, A. (2000) Topics in Non-parametric Statistics. In, Ecole d’Eté de Probabilités de Saint-Flour, Berlin, Germany: Springer, 1738, Lecture Notes in Mathematics. · Zbl 0998.62033
[28] Pal, J.K. (2008) Spiking problem in monotone regression: Penalized residual sum of squares., Statist. Probab. Lett., 78, 1548-1556. · Zbl 1325.62132 · doi:10.1016/j.spl.2008.01.015
[29] Pal, J.K. and Woodroofe, M. (2007) Large sample properties of shape restricted regression estimators with smoothness adjustments., Statist. Sinica, 17, 1601-1616. · Zbl 1133.62030
[30] Prymak, A.V. (2002) Three-convex approximation by quadratic splines with arbitrary fixed knots., East J. Approx., 8, 185-196. · Zbl 1331.41008
[31] Robertson, T., Wright, F.T., and Dykstra, R.L. (1988), Order Restricted Statistical Inference. John Wiley & Sons. · Zbl 0645.62028
[32] Sanyal, A.K., Chellappa, M., Ahmed, J., Shen, J., and Bernstein, D.S. (2003) Globally convergent adaptive tracking of spacecraft angular velocity with inertia identification and adaptive linearization., Proc. of the 42nd IEEE Conf. on Dec. and Contr., 2704-2709, Hawaii, 2003.
[33] Scholtes, S. (2012), Introduction to piecewise differentiable equations. Springer. · Zbl 1453.49002
[34] Shadrin, A.Y. (2001) The \(L_∞ \)-norm of the \(L_2\)-spline projector is bounded independently of the knot sequence: A proof of de Boor’s conjecture., Acta Math., 87, 59-137. · Zbl 0996.41006 · doi:10.1007/BF02392832
[35] Shen, J. and Lebair, T.M. (2015) Shape restricted smoothing splines via constrained optimal control and nonsmooth Newton’s methods., Automatica, 53, 216-224. · Zbl 1372.49037 · doi:10.1016/j.automatica.2014.12.040
[36] Shen, J. and Wang, X. (2010) Estimation of shape constrained functions in dynamical systems and its application to genetic networks., Proc. of Am. Control Conf., 5948-5953, Baltimore.
[37] Shen, J. and Wang, X. (2011) Estimation of monotone functions via \(P\)-splines: A constrained dynamical optimization approach., SIAM J. Control Optim., 49, 646-671. · Zbl 1232.41009 · doi:10.1137/090774513
[38] Shen, J. and Wang, X. (2012) Convex regression via penalized splines: A complementarity approach., Proc. of Am. Control Conf., 332-337, Montreal, Canada.
[39] Tsybakov, A.B. (2010), Introduction to Nonparametric Estimation. Springer. · Zbl 1176.62032
[40] Wang, X. and Shen, J. (2010) A class of grouped Brunk estimators and penalized spline estimators for monotone regression., Biometrika, 97, 585-601. · Zbl 1195.62041 · doi:10.1093/biomet/asq029
[41] Wang, X. and Shen, J. (2013) Uniform convergence and rate adaptive estimation of convex functions via constrained optimization., SIAM J. Control Optim., 51, 2753-2787. · Zbl 1280.41008 · doi:10.1137/120887837
[42] Woodroofe, M. and Sun, J. (1993) A penalized maximum likelihood estimate of \(f(0_+)\) when \(f\) is nonincreasing., Statist. Sinica, 3, 501-515. · Zbl 0822.62020
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.