×

zbMATH — the first resource for mathematics

Incremental regularized least squares for dimensionality reduction of large-scale data. (English) Zbl 1338.65093

MSC:
65F10 Iterative numerical methods for linear systems
65F22 Ill-posedness and regularization problems in numerical linear algebra
65F50 Computational methods for sparse matrices
68T05 Learning and adaptive systems in artificial intelligence
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] T. Anderson, An Introduction to Multivariate Statistical Analysis, 3rd ed., Wiley, Hoboken, NJ, 2003. · Zbl 1039.62044
[2] H. Avron, P. Maymounkov, and S. Toledo, Blendenpik: Supercharging LAPACK’s least-squares solver, SIAM J. Sci. Comput., 32 (2010), pp. 1217–1236. · Zbl 1213.65069
[3] M. Baboulin, L. Giraud, S. Gratton, and J. Langou, Parallel tools for solving incremental dense least squares problems. Application to space geodesy, J. Algorithms Comput. Technol., 31 (2009), pp. 117–131. · Zbl 1226.65029
[4] D. P. Bertsekas, Incremental least squares methods and the extended Kalman filter, SIAM J. Optim., 6 (1996), pp. 807–822. · Zbl 0945.93026
[5] M. Blondel, K. Seki, and K. Uehara, Block coordinate descent algorithms for large-scale sparse multiclass classification, Mach. Learn., 93 (2013), pp. 31–52. · Zbl 1293.68216
[6] C. Burges, Dimension reduction: A guided tour, Found. Trends Mach. Learn., 2 (2009), pp. 275–365. · Zbl 1211.68126
[7] D. Cai, X. He, and J. Han, SRDA: An efficient algorithm for large-scale discriminant analysis, IEEE Trans. Knowledge Data Eng., 20 (2008), pp. 1–12.
[8] A. Cassioli, A. Chiavaioli, C. Manes, and M. Sciandrone, An incremental least squares algorithm for large scale linear classification, European J. Oper. Res., 224 (2013), pp. 560–565. · Zbl 1292.90198
[9] N. Cesa-Bianchi, Applications of regularized least squares to pattern classification, Theoret. Comput. Sci., 382 (2007), pp. 221–231. · Zbl 1127.68095
[10] W. Ching, D. Chu, L.-Z. Liao, and X. Wang, Regularized orthogonal linear discriminant analysis, Pattern Recognit., 45 (2012), pp. 2719–2732. · Zbl 1236.62067
[11] D. Chu and S. T. Goh, A new and fast orthogonal linear discriminant analysis on undersampled problems, SIAM J. Sci. Comput., 32 (2010), pp. 2274–2297. · Zbl 1215.93024
[12] D. Chu, S. T. Goh, and Y. S. Hung, Characterization of all solutions for undersampled uncorrelated linear discriminant analysis problems, SIAM J. Matrix Anal. Appl., 32 (2011), pp. 820–844. · Zbl 1229.62084
[13] D. Chu, L.-Z. Liao, M. K. Ng, and X. Wang, Incremental linear discriminant analysis: A new fast algorithm and comparisons, IEEE Trans. Neural Netw. Learn. Syst., 26 (2015), pp. 2716–2735.
[14] M. Dredze, K. Crammer, and F. Pereira, Confidence-weighted linear classification, in Proceedings of the 25th International Conference on Machine Learning, International Machine Learning Society, Madison WI, 2008, pp. 264–271. · Zbl 1432.68382
[15] R. Duda, P. Hart, and D. Stork, Pattern Classification, Wiley, New York, 2000.
[16] D. C.-L. Fong and M. Saunders, LSMR: An iterative algorithm for sparse least-squares problems, SIAM J. Sci. Comput., 33 (2011), pp. 2950–2971. · Zbl 1232.65052
[17] J. Friedman, Regularized discriminant analysis, J. Amer. Statist. Assoc., 84 (1989), pp. 165–175.
[18] G. Golub and C. Van Loan, Matrix Computations, 3rd ed., Johns Hopkins University Press, Baltimore, MD, 1996. · Zbl 0865.65009
[19] T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference and Prediction, 2nd ed., Springer, New York, 2009. · Zbl 1273.62005
[20] P. Howland, M. Jeon, and H. Park, Structure preserving dimension reduction for clustered text data based on the generalized singular value decomposition, SIAM J. Matrix Anal. Appl., 25 (2003), pp. 165–179. · Zbl 1061.68135
[21] T. Kim, B. Stenger, J. Kittler, and R. Cipolla, Incremental linear discriminant analysis using sufficient spanning sets and its applications, Int. J. Comput. Vis., 91 (2011), pp. 216–232. · Zbl 1235.68272
[22] T. Kim, S. Wong, B. Stenger, J. Kittler, and R. Cipolla, Incremental linear discriminant analysis using sufficient spanning set approximations, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Piscataway, NJ, 2007, pp. 1–8.
[23] F. la Torre, A least-squares framework for component analysis, IEEE Trans. Pattern Anal. Mach. Intell., 34 (2012), pp. 1041–1055.
[24] L. Liu, Y. Jiang, and Z. Zhou, Least square incremental linear discriminant analysis, in Proceedings of the 9th IEEE International Conference on Data Mining, IEEE, Piscataway, NJ, 2009, pp. 298–306.
[25] X. Meng, M. A. Saunders, and M. W. Mahoney, LSRN: A parallel iterative solver for strongly over- or underdetermined systems, SIAM J. Sci. Comput., 36 (2014), pp. C95–C118. · Zbl 1298.65053
[26] C. Paige and M. Saunders, LSQR: An algorithm for sparse linear equations and sparse least squares, ACM Trans. Math. Software, 8 (1982), pp. 43–71. · Zbl 0478.65016
[27] S. Pang, S. Ozawa, and N. Kasabov, Incremental linear discriminant analysis for classification of data streams, IEEE Trans. Syst. Man Cybern. A, 35 (2005), pp. 905–914.
[28] C. H. Park and H. Park, A relationship between linear discriminant analysis and the generalized minimum squared error solution, SIAM J. Matrix Anal. Appl., 27 (2005), pp. 474–492. · Zbl 1101.65009
[29] R. Polikar, L. Udpa, and V. Honavar, Learn ++: An incremental learning algorithm for supervised neural networks, IEEE Trans. Syst. Man Cybern. C, 31 (2001), pp. 497–508.
[30] B. S. and A. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, MIT Press, Cambridge, MA, 2002.
[31] C. Saunders, A. Gammerman, and V. Vovk, Ridge regression learning algorithm in dual variables, in Proceedings of the 15th International Conference on Machine Learning, Morgan Kaufmann, San Francisco, 1998, pp. 515–521.
[32] A. Tikhonov and V. Arsenin, Solutions of Ill-Posed Problems, V. H. Winston & Sons, Washington, DC, 1977. · Zbl 0354.65028
[33] TREC, Text Retrieval Conference, http://trec.nist.gov (1999).
[34] V. Vovk, Competitive on-line linear regression, in Advances in Neural Information Processing Systems 10, MIT Press, Cambridge, MA, 1998, pp. 364–370.
[35] J. Ye, Characterization of a family of algorithms for generalized discriminant analysis on undersampled problems, J. Mach. Learn. Res., 6 (2005), pp. 483–502. · Zbl 1222.62081
[36] J. Ye, Least squares linear discriminant analysis, in Proceedings of the 24th International Conference on Machine Learning, International Machine Learning Society, Madison, WI, 2007, pp. 1087–1094.
[37] J. Ye, Q. Li, H. Xiong, H. Park, R. Janardan, and V. Kumar, IDR/QR: An incremental dimension reduction algorithm via QR decomposition, IEEE Trans. Knowledge Data Eng., 17 (2005), pp. 1208–1222.
[38] J. Ye, T. Xiong, Q. Li, R. Janardan, J. Bi, V. Cherkassky, and C. Kambhamettu, Efficient model selection for regularized linear discriminant analysis, in Proceedings of the 15th ACM International Conference on Information and Knowledge Management, ACM, New York, 2006, pp. 532–539.
[39] Z. Zhang, G. Dai, C. Xu, and M. Jordan, Regularized discriminant analysis, ridge regression and beyond, J. Mach. Learn. Res., 11 (2010), pp. 2199–2228. · Zbl 1242.62067
[40] H. Zhao and P. Yuen, Incremental linear discriminant analysis for face recognition, IEEE Trans. Syst. Man Cybern. B, 38 (2008), pp. 210–221.
[41] A. Zouzias and N. M. Freris, Randomized extended Kaczmarz for solving least squares, SIAM J. Matrix Anal. Appl., 34 (2013), pp. 773–793. · Zbl 1273.65053
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.