×

Optimal classification of Gaussian processes in homo- and heteroscedastic settings. (English) Zbl 1448.62209

Summary: A procedure to derive optimal discrimination rules is formulated for binary functional classification problems in which the instances available for induction are characterized by random trajectories sampled from different Gaussian processes, depending on the class label. Specifically, these optimal rules are derived as the asymptotic form of the quadratic discriminant for the discretely monitored trajectories in the limit that the set of monitoring points becomes dense in the interval on which the processes are defined. The main goal of this work is to provide a detailed analysis of such optimal rules in the dense monitoring limit, with a particular focus on elucidating the mechanisms by which near-perfect classification arises. In the general case, the quadratic discriminant includes terms that are singular in this limit. If such singularities do not cancel out, one obtains near-perfect classification, which means that the error approaches zero asymptotically, for infinite sample sizes. This singular limit is a consequence of the orthogonality of the probability measures associated with the stochastic processes from which the trajectories are sampled. As a further novel result of this analysis, we formulate rules to determine whether two Gaussian processes are equivalent or mutually singular (orthogonal).

MSC:

62R10 Functional data analysis
62H30 Classification and discrimination; cluster analysis (statistical aspects)
60G15 Gaussian processes
46E22 Hilbert spaces with reproducing kernels (= (proper) functional Hilbert spaces, including de Branges-Rovnyak and other structured spaces)
PDF BibTeX XML Cite
Full Text: DOI

References:

[1] Baíllo, A.; Cuevas, A.; Cuesta-Albertos, JA, Supervised classification for a family of Gaussian functional models, Scand. J. Stat., 38, 3, 480-498 (2011) · Zbl 1246.62155
[2] Baker, CTH, The Numerical Treatment of Integral Equations (1977), Oxford: Clarendon, Oxford
[3] Berlinet, A.; Thomas-Agnan, C., Reproducing Kernel Hilbert Spaces in Probability and Statistics (2004), Boston: Springer, Boston · Zbl 1145.62002
[4] Berrendero, JR; Cárcamo, J., Linear components of quadratic classifiers, Adv. Data Anal. Classif., 13, 2, 347-377 (2019) · Zbl 1459.62104
[5] Berrendero, J.R., Bueno-Larraz, B., Cuevas, A.: On Mahalanobis distance in functional settings (2018a). arXiv:1803.06550
[6] Berrendero, JR; Cuevas, A.; Torrecilla, JL, On the use of reproducing kernel Hilbert spaces in functional classification, J. Am. Stat. Assoc., 113, 523, 1210-1218 (2018) · Zbl 1402.68152
[7] Bollerslev, T.; Chou, R.; Kroner, KF, Arch modeling in finance: a review of the theory and empirical evidence, J. Econom., 52, 1-2, 5-59 (1992) · Zbl 0825.90057
[8] Cont, R., Empirical properties of asset returns: stylized facts and statistical issues, Quant. Finance, 1, 2, 223-236 (2001) · Zbl 1408.62174
[9] Cucker, F.; Smale, S., On the mathematical foundations of learning, Bull. Am. Math. Soc., 39, 1-49 (2002) · Zbl 0983.68162
[10] Cucker, F.; Zhou, DX, Learning Theory: An Approximation Theory Viewpoint (Cambridge Monographs on Applied & Computational Mathematics) (2007), New York: Cambridge University Press, New York
[11] Cuesta-Albertos, J.A., Dutta, S.: On perfect classification for Gaussian processes (2016). arXiv:1602.04941
[12] Cuevas, A., A partial overview of the theory of statistics with functional data, J. Stat. Plan. Inference, 147, 1-23 (2014) · Zbl 1278.62012
[13] Dai, X.; Müller, HG; Yao, F., Optimal Bayes classifiers for functional data and density ratios, Biometrika, 104, 3, 545-560 (2017) · Zbl 07072227
[14] Delaigle, A.; Hall, P., Defining probability density for a distribution of random functions, Ann. Stat., 38, 2, 1171-1193 (2010) · Zbl 1183.62061
[15] Delaigle, A.; Hall, P., Achieving near perfect classification for functional data, J. R. Stat. Soc. Ser. B (Stat. Methodol.), 74, 2, 267-286 (2012) · Zbl 1411.62164
[16] Delaigle, A.; Hall, P., Classification using censored functional data, J. Am. Stat. Assoc., 108, 504, 1269-1283 (2013) · Zbl 1288.62104
[17] Epifanio, I.; Ventura-Campos, N., Hippocampal shape analysis in Alzheimer’s disease using functional data analysis, Stat. Med., 33, 5, 867-880 (2014)
[18] Fama, EF, The behavior of stock-market prices, J. Bus., 38, 1, 34-105 (1965)
[19] Feldman, J., Equivalence and perpendicularity of Gaussian processes, Pac. J. Math., 8, 4, 699-708 (1958) · Zbl 0084.13001
[20] Ferraty, F.; Vieu, P., Nonparametric Functional Data Analysis: Theory and Practice. Springer Series in Statistics (2006), Secaucus: Springer, Secaucus · Zbl 1119.62046
[21] Galeano, P.; Joseph, E.; Lillo, RE, The Mahalanobis distance for functional data with applications to classification, Technometrics, 57, 2, 281-291 (2015)
[22] Hájek, J., A property of \(J\)-divergences of marginal probability distributions, Czechoslov. Math. J., 08, 3, 460-463 (1958) · Zbl 0082.34103
[23] Hastie, T.; Tibshirani, R.; Friedman, JH, The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Series in Statistics (2009), Berlin: Springer, Berlin · Zbl 1273.62005
[24] Hubert, M.; Rousseeuw, P.; Segaert, P., Multivariate and functional classification using depth and distance, Adv. Data Anal. Classif., 11, 3, 445-466 (2017) · Zbl 1414.62247
[25] Kailath, T., Some results on singular detection, Inf. Control, 9, 2, 130-152 (1966) · Zbl 0143.41702
[26] Kailath, T., RKHS approach to detection and estimation problems-I: deterministic signals in Gaussian noise, IEEE Trans. Inf. Theory, 17, 5, 530-549 (1971) · Zbl 0236.94005
[27] Kuelbs, J., Gaussian measures on a Banach space, Journal of Functional Analysis, 5, 3, 354-367 (1970) · Zbl 0194.44703
[28] Leng, X.; Müller, HG, Classification using functional data analysis for temporal gene expression data, Bioinformatics, 22, 1, 68-76 (2006)
[29] Lukić, MN; Beder, JH, Stochastic processes with sample paths in reproducing Kernel Hilbert spaces, Trans. Am. Math. Soc., 353, 10, 3945-3969 (2001) · Zbl 0973.60036
[30] Manton, JH; Amblard, PO, A primer on reproducing kernel Hilbert spaces, Found. Trends Signal Process., 8, 1-2, 1-126 (2015)
[31] Marks, S.; Dunn, OJ, Discriminant functions when covariance matrices are unequal, J. Am. Stat. Assoc., 69, 346, 555-559 (1974) · Zbl 0291.62073
[32] Martin-Barragan, B.; Lillo, R.; Romo, J., Interpretable support vector machines for functional data, Eur. J. Oper. Res., 232, 1, 146-155 (2014)
[33] Müller, HG, Peter hall, functional data analysis and random objects, Ann. Stat., 44, 5, 1867-1887 (2016) · Zbl 1349.62011
[34] Osborne, MFM, Brownian motion in the stock market, Oper. Res., 7, 2, 145-173 (1959) · Zbl 1414.91427
[35] Parzen, E.: Statistical inference on time series by Hilbert space methods. Technical report 23, Statistics Department, Stanford University (1959)
[36] Parzen, E., An approach to time series analysis, Ann Math Stat, 32, 4, 951-989 (1961) · Zbl 0107.13801
[37] Parzen, E.: Regression analysis of continuous parameter time series. In: Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics, University of California Press, Berkeley, California , pp 469-489 (1961b) · Zbl 0107.13802
[38] Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; Vanderplas, J.; Passos, A.; Cournapeau, D.; Brucher, M.; Perrot, M.; Duchesnay, E., Scikit-learn: machine learning in python, J. Mach. Learn. Res., 12, 2825-2830 (2011) · Zbl 1280.68189
[39] Ramos-Carreño, C., Suárez, A., Torrecilla, J.L., Carbajo Berrocal, M., Marcos Manchón, P., Pérez Manso, P., Hernando Bernabé, A.: scikit-fda: functional data analysis in Python (2019). 10.5281/zenodo.3468127
[40] Ramsay, JO; Silverman, BW, Functional Data Analysis. Springer Series in Statistics (2005), Berlin: Springer, Berlin
[41] Rasmussen, CE; Williams, CKI, Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) (2005), London: The MIT Press, London
[42] Rincón, M.; Ruiz-Medina, MD, Wavelet-RKHS-based functional statistical classification, Adv. Data Anal. Classif., 6, 3, 201-217 (2012) · Zbl 1254.62074
[43] Rossi, F.; Villa, N., Support vector machine for functional data classification, Neurocomputing, 69, 7, 730-742 (2006)
[44] Sato, H., On the equivalence of Gaussian measures, J. Math. Soc. Jpn., 19, 2, 159-172 (1967) · Zbl 0168.16104
[45] Shepp, LA, Radon-Nikodym derivatives of Gaussian measures, Ann. Math. Stat., 37, 2, 321-354 (1966) · Zbl 0142.13901
[46] Song, JJ; Deng, W.; Lee, HJ; Kwon, D., Optimal classification for time-course gene expression data using functional data analysis, Comput. Biol. Chem., 32, 6, 426-432 (2008) · Zbl 1158.62085
[47] Spence, A., On the convergence of the Nyström method for the integral equation eigenvalue problem, Numer. Math., 25, 1, 57-66 (1975) · Zbl 0297.65077
[48] Varberg, DE, On equivalence of Gaussian measures, Pac. J. Math., 11, 2, 751-762 (1961) · Zbl 0211.48002
[49] Wahl, PW; Kronmal, RA, Discriminant functions when covariances are unequal and sample sizes are moderate, Biometrics, 33, 3, 479-484 (1977) · Zbl 0371.62091
[50] Wang, JL; Chiou, JM; Müller, HG, Functional data analysis, Ann. Rev. Stat. Appl., 3, 1, 257-295 (2016)
[51] Zhu, H.; Brown, PJ; Morris, JS, Robust classification of functional and quantitative image data using functional mixed models, Biometrics, 68, 4, 1260-1268 (2012) · Zbl 1259.62056
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.