Spline models for observational data.

*(English)*Zbl 0813.62001
CBMS-NSF Regional Conference Series in Applied Mathematics. 59. Philadelphia, PA: SIAM, Society for Industrial and Applied Mathematics. XII, 169 p. (1990).

Statisticians are generally interested in smoothing data of the form
\[
y_ i= f(x_ i)+ \varepsilon_ i, \qquad i=1,2,\dots, n,
\]
where \(\varepsilon_ i\) are random disturbances and \(f\) is only known to be “smooth”. A formal statement of the problem may be formulated as follows: find \(f\) in a given class of smooth functions \(W\) on the interval \((a,b)\) to minimize (for some \(\lambda>0\))
\[
{1\over n} \sum_{i=1}^ n (y_ i- f(x_ i))^ 2+ \lambda \int_ a^ b (f^{(m)} (x))^ 2 dx.
\]
All splines considered in the book are solutions to variational problems. The variational problems are treated from a unified point of view as optimization problems in reproducing kernel Hilbert spaces and it is assumed that the reader has a knowledge of the basic properties of Hilbert spaces.

Contents: 1. Background (1.1. Positive-definite functions, covariances, and reproducing kernels; 1.2. Reproducing kernel spaces on \([0,1]\) with norms involving derivatives; 1.3. The special and general spline smoothing problems; 1.4. The duality between reproducing kernel Hilbert spaces and stochastic processes; 1.5. The smoothing spline and the generalized smoothing spline as Bayes estimates).

2. More splines (2.1. Splines on the circle; 2.2. Splines on the sphere, the role of the iterated Laplacian; 2.3. Vector splines on the sphere; 2.4. The thin-plate spline on \(E^ d\); 2.5. Another look at the Bayes model behind the thin-plate spline).

3. Equivalence and perpendicularity, or What’s So Special About Splines? (3.1. Equivalence and perpendicularity of probability measures; 3.2. Implications for kriging).

4. Estimating the smoothing parameter (4.1. The importance of a good choice of \(\lambda\); 4.2. Ordinary cross validation and the “leaving- out-one” lemma; 4.3. Generalized cross validation (GCV); 4.4. Properties of the GCV estimate of \(\lambda\); 4.5. Convergence rates with the optimal \(\lambda\); 4.6. Other estimates of \(\lambda\) similar to GCV; 4.7. More on other estimates; 4.8. The generalized maximum likelihood estimate of \(\lambda\); 4.9. Limits of GCV).

5. Confidence intervals (5.1. Bayesian confidence intervals; 5.2. Estimate-based bootstrapping).

6. Partial spline models (6.1. Estimation; 6.2. Convergence of partial spline estimates; 6.3 Testing).

7. Finite-dimensional approximating subspaces (7.1. Quadrature formulae, computing with basis functions; 7.2. Regression splines).

8. Fredholm integral equations of the first kind (8.1. Existence of solutions, the method of regularization; 8.2. Further remarks on ill- posedness; 8.3. Mildly nonlinear integral equations; 8.4. The optimal \(\lambda\) for loss functions other than predictive mean-square error).

9. Further nonlinear generalizations (9.1. Partial spline models in nonlinear regression; 9.2. Penalized GLIM models; 9.3. Estimation of the log-likelihood ratio; 9.4. Linear inequality constraints; 9.5. Inequality constraints in ill-posed problems; 9.6. Constrained nonlinear optimization with basis functions; 9.7. System identification).

10. Additive and interaction splines (10.1. Variational problems with multiple smoothing parameters; 10.2. Additive and interaction smoothing splines).

11. Numerical methods.

12. Special topics (12.1. The notion of “high frequency” in different spaces; 12.2. Optimal quadrature and experimental design).

The bibliography contains more than three hundred items. The book is based on a series of 10 lectures at Ohio State University at Columbus in March 23-27, 1987.

Contents: 1. Background (1.1. Positive-definite functions, covariances, and reproducing kernels; 1.2. Reproducing kernel spaces on \([0,1]\) with norms involving derivatives; 1.3. The special and general spline smoothing problems; 1.4. The duality between reproducing kernel Hilbert spaces and stochastic processes; 1.5. The smoothing spline and the generalized smoothing spline as Bayes estimates).

2. More splines (2.1. Splines on the circle; 2.2. Splines on the sphere, the role of the iterated Laplacian; 2.3. Vector splines on the sphere; 2.4. The thin-plate spline on \(E^ d\); 2.5. Another look at the Bayes model behind the thin-plate spline).

3. Equivalence and perpendicularity, or What’s So Special About Splines? (3.1. Equivalence and perpendicularity of probability measures; 3.2. Implications for kriging).

4. Estimating the smoothing parameter (4.1. The importance of a good choice of \(\lambda\); 4.2. Ordinary cross validation and the “leaving- out-one” lemma; 4.3. Generalized cross validation (GCV); 4.4. Properties of the GCV estimate of \(\lambda\); 4.5. Convergence rates with the optimal \(\lambda\); 4.6. Other estimates of \(\lambda\) similar to GCV; 4.7. More on other estimates; 4.8. The generalized maximum likelihood estimate of \(\lambda\); 4.9. Limits of GCV).

5. Confidence intervals (5.1. Bayesian confidence intervals; 5.2. Estimate-based bootstrapping).

6. Partial spline models (6.1. Estimation; 6.2. Convergence of partial spline estimates; 6.3 Testing).

7. Finite-dimensional approximating subspaces (7.1. Quadrature formulae, computing with basis functions; 7.2. Regression splines).

8. Fredholm integral equations of the first kind (8.1. Existence of solutions, the method of regularization; 8.2. Further remarks on ill- posedness; 8.3. Mildly nonlinear integral equations; 8.4. The optimal \(\lambda\) for loss functions other than predictive mean-square error).

9. Further nonlinear generalizations (9.1. Partial spline models in nonlinear regression; 9.2. Penalized GLIM models; 9.3. Estimation of the log-likelihood ratio; 9.4. Linear inequality constraints; 9.5. Inequality constraints in ill-posed problems; 9.6. Constrained nonlinear optimization with basis functions; 9.7. System identification).

10. Additive and interaction splines (10.1. Variational problems with multiple smoothing parameters; 10.2. Additive and interaction smoothing splines).

11. Numerical methods.

12. Special topics (12.1. The notion of “high frequency” in different spaces; 12.2. Optimal quadrature and experimental design).

The bibliography contains more than three hundred items. The book is based on a series of 10 lectures at Ohio State University at Columbus in March 23-27, 1987.

Reviewer: R.Zielinski (Warszawa)

##### MSC:

62-02 | Research exposition (monographs, survey articles) pertaining to statistics |

62G07 | Density estimation |

65C99 | Probabilistic methods, stochastic differential equations |

65D07 | Numerical computation using splines |

65R20 | Numerical methods for integral equations |

62G20 | Asymptotic properties of nonparametric inference |

45B05 | Fredholm integral equations |