Robust and sparse multigroup classification by the optimal scoring approach. (English) Zbl 1433.68372

Summary: We propose a robust and sparse classification method based on the optimal scoring approach. It is also applicable if the number of variables exceeds the number of observations. The data are first projected into a low dimensional subspace according to an optimal scoring criterion. The projection only includes a subset of the original variables (sparse modeling) and is not distorted by outliers (robust modeling). In this low dimensional subspace classification is performed by minimizing a robust Mahalanobis distance to the group centers. The low dimensional representation of the data is also useful for visualization purposes. We discuss the algorithm for the proposed method in detail. A simulation study illustrates the properties of robust and sparse classification by optimal scoring compared to the non-robust and/or non-sparse alternative methods. Three real data applications are given.


68T05 Learning and adaptive systems in artificial intelligence
62H30 Classification and discrimination; cluster analysis (statistical aspects)
Full Text: DOI


[1] Alfons, A.; Croux, C.; Gelper, S., Sparse least trimmed squares regression for analyzing high-dimensional large data sets, Ann Appl Stat, 7, 1, 226-248 (2013) · Zbl 1454.62123
[2] Armanino, C.; Leardi, R.; Lanteri, S.; Modi, G., Chemometric analysis of tuscan olive oils, Chemom Intell Lab Syst, 5, 4, 343-354 (1989)
[3] Brodinova S, Ortner T, Filzmoser P, Zaharieva M, Breiteneder C (2015) Evaluation of robust PCA for supervised audio outlier detection. In: Proceeding of 22nd international conference on computational statistics (COMPSTAT) · Zbl 1414.62232
[4] Clemmensen L, Kuhn M (2012) sparseLDA: sparse discriminant analysis. R package version 0.1-6. https://CRAN.R-project.org/package=sparseLDA. Accessed 21 Oct 2015
[5] Clemmensen, L.; Hastie, T.; Witten, D.; Ersbøll, B., Sparse discriminant analysis, Technometrics, 53, 4, 406-413 (2012)
[6] Efron, B.; Hastie, T.; Johnstone, I.; Tibshirani, R., Least angle regression, Ann Stat, 32, 2, 407-499 (2004) · Zbl 1091.62054
[7] Hampel, F., The influence curve and its role in robust estimation, J Am Stat Assoc, 69, 346, 383-393 (1974) · Zbl 0305.62031
[8] Hampel, F.; Ronchetti, E.; Rousseeuw, P.; Stahel, W., Robust statistics: the approach based on influence functions (1986), Hoboken: Wiley, Hoboken
[9] Hastie, T.; Tibshirani, R.; Buja, A., Flexible discriminant analysis by optimal scoring, J Am Stat Assoc, 89, 428, 1255-1270 (1994) · Zbl 0812.62067
[10] Hastie, T.; Tibshirani, R.; Wainwright, M., Statistical learning with sparsity: the lasso and generalizations (2015), Boca Raton: CRC Press, Boca Raton · Zbl 1319.68003
[11] Hoffmann, I.; Filzmoser, P.; Serneels, S.; Varmuza, K., Sparse and robust PLS for binary classification, J Chemom, 30, 4, 153-162 (2016)
[12] Hubert, M.; Van Driessen, K., Fast and robust discriminant analysis, Comput Stat Data Anal, 45, 2, 301-320 (2004) · Zbl 1429.62247
[13] Hubert, M.; Rousseeuw, P.; Van Aelst, S., High-breakdown robust multivariate methods, Stat Sci, 23, 1, 92-119 (2008) · Zbl 1327.62328
[14] Johnson, R.; Wichern, D., Applied multivariate statistical analysis (2002), Upper Saddle River: Prentice Hall, Upper Saddle River
[15] A language and environment for statistical computing (2016), Vienna: R Foundation for Statistical Computing, Vienna
[16] Rousseeuw, P.; Van Driessen, K., A fast algorithm for the minimum covariance determinant estimator, Technometrics, 41, 3, 212-223 (1999)
[17] Tibshirani, R., Regression shrinkage and selection via the lasso: a retrospective, J R Stat Soc B, 73, 3, 273-282 (2011) · Zbl 1411.62212
[18] Todorov V (2016) rrcovHD: robust multivariate methods for high dimensional data. R package version 0.2-4. https://CRAN.R-project.org/package=rrcovHD. Accessed 17 Feb 2016
[19] Todorov, V.; Pires, A., Comparative performance of several robust linear discriminant analysis methods, REVSTAT Stat J, 5, 1, 63-83 (2007) · Zbl 05217607
[20] Vanden Branden, K.; Hubert, M., Robust classification in high dimensions based on the simca method, Chemom Intell Lab Syst, 79, 1, 10-21 (2005)
[21] Witten, D.; Tibshirani, R., Penalized classification using fisher’s linear discriminant, J R Stat Soc Ser B (Statistical Methodology), 73, 5, 753-772 (2011) · Zbl 1228.62079
[22] Wolke, R.; Schwetlick, H., Iteratively reweighted least squares: algorithms, convergence analysis, and numerical comparisons, SIAM J Sci Stat Comput, 9, 5, 907-921 (1988) · Zbl 0709.65130
[23] Wu, T.; Lange, K., Coordinate descent algorithms for lasso penalized regression, Ann Appl Stat, 2, 1, 224-244 (2008) · Zbl 1137.62045
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.