Sparse principal component analysis via variable projection. (English) Zbl 1440.62231


62H25 Factor analysis and principal components; correspondence analysis
62J12 Generalized linear models (logistic models)
68W20 Randomized algorithms
Full Text: DOI arXiv


[1] G. Abraham and M. Inouye, Fast principal component analysis of large-scale genome-wide data, PloS ONE, 9 (2014), e93766.
[2] A. Aravkin and S. Becker, Dual smoothing and value function techniques for variational matrix decomposition, in Handbook of Robust Low-Rank and Sparse Matrix Decomposition: Applications in Image and Video Processing, Chapman and Hall/CRC Press, Boca Raton, FL, 2016, pp. 3.1-3.33.
[3] A. Y. Aravkin, D. Drusvyatskiy, and T. van Leeuwen, Efficient quadratic penalization through the partial minimization technique, IEEE Trans. Automat. Control, 63 (2018), pp. 2131-2138. · Zbl 1423.49020
[4] A. Y. Aravkin and T. van Leeuwen, Estimating nuisance parameters in inverse problems, Inverse Problems, 28 (2012), 115016. · Zbl 1253.49021
[5] T. Askham and J. N. Kutz, Variable projection methods for an optimized dynamic mode decomposition, SIAM J. Appl. Dyn. Syst., 17 (2018), pp. 380-416, https://doi.org/10.1137/M1124176. · Zbl 1384.37122
[6] H. Attouch, J. Bolte, P. Redont, and A. Soubeyran, Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the Kurdyka-Łojasiewicz inequality, Math. Oper. Res., 35 (2010), pp. 438-457. · Zbl 1214.65036
[7] H. Babaee and T. Sapsis, A minimization principle for the description of modes associated with finite-time instabilities, Proc. A, 472 (2016), 20150779. · Zbl 1371.34064
[8] A. Beck and M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci., 2 (2009), pp. 183-202, https://doi.org/10.1137/080716542. · Zbl 1175.94009
[9] G. Berkooz, P. Holmes, and J. L. Lumley, The proper orthogonal decomposition in the analysis of turbulent flows, in Annual Review of Fluid Mechanics, Vol. 25, Annual Reviews Inc., Palo Alto, CA, 1993, pp. 539-575.
[10] J. Bolte, S. Sabach, and M. Teboulle, Proximal alternating linearized minimization for nonconvex and nonsmooth problems, Math. Program., 146 (2014), pp. 459-494. · Zbl 1297.90125
[11] T. Bouwmans, A. Sobral, S. Javed, S. K. Jung, and E.-H. Zahzah, Decomposition into low-rank plus additive matrices for background/foreground separation: A review for a comparative evaluation with a large-scale dataset, Comput. Sci. Rev., 23 (2017), pp. 1-71. · Zbl 1398.68572
[12] M. Brand, Incremental singular value decomposition of uncertain data with missing values, in European Conference on Computer Vision, Lecture Notes in Comput. Sci. 2350, Springer, Heidelberg, 2002, pp. 707-720. · Zbl 1034.68580
[13] R. Bro, PARAFAC. Tutorial and applications, Chemom. Intell. Lab. Syst., 38 (1997), pp. 149-171.
[14] S. L. Brunton and B. R. Noack, Closed-loop turbulence control: Progress and challenges, Appl. Mech. Rev., 67 (2015), 050801.
[15] S. L. Brunton, J. L. Proctor, and J. N. Kutz, Discovering governing equations from data by sparse identification of nonlinear dynamical systems, Proc. Natl. Acad. Sci. USA, 113 (2016), pp. 3932-3937. · Zbl 1355.94013
[16] T. T. Cai, Z. Ma, and Y. Wu, Sparse PCA: Optimal rates and adaptive estimation, Ann. Statist., 41 (2013), pp. 3074-3110. · Zbl 1288.62099
[17] E. J. Candès, X. Li, Y. Ma, and J. Wright, Robust principal component analysis?, J. ACM, 58 (2011), 11. · Zbl 1327.62369
[18] K. Carlberg, Adaptive h-refinement for reduced-order models, Internat. J. Numer. Methods Engrg., 102 (2015), pp. 1192-1210. · Zbl 1352.65136
[19] P. L. Combettes and J.-C. Pesquet, Proximal splitting methods in signal processing, in Fixed-Point Algorithms for Inverse Problems in Science and Engineering, Springer, New York, 2011, pp. 185-212. · Zbl 1242.90160
[20] P. Comon, Tensors: A brief introduction, IEEE Signal Process. Mag., 31 (2014), pp. 44-53.
[21] C. Croux, P. Filzmoser, and H. Fritz, Robust sparse principal component analysis, Technometrics, 55 (2013), pp. 202-214.
[22] J. P. Cunningham and Z. Ghahramani, Linear dimensionality reduction: Survey, insights, and generalizations, J. Mach. Learn. Res., 16 (2015), pp. 2859-2900. · Zbl 1351.62123
[23] A. d’Aspremont, F. Bach, and L. E. Ghaoui, Optimal solutions for sparse principal component analysis, J. Mach. Learn. Res., 9 (2008), pp. 1269-1294. · Zbl 1225.68170
[24] A. d’Aspremont, L. E. Ghaoui, M. I. Jordan, and G. R. Lanckriet, A direct formulation for sparse PCA using semidefinite programming, in Advances in Neural Information Processing Systems, 2005, pp. 41-48.
[25] P. Drineas and M. W. Mahoney, RandNLA: Randomized numerical linear algebra, Comm. ACM, 59 (2016), pp. 80-90.
[26] B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, Least angle regression, Ann. Statist., 32 (2004), pp. 407-499. · Zbl 1091.62054
[27] N. B. Erichson, K. Manohar, S. L. Brunton, and J. N. Kutz, Randomized CP Tensor Decomposition, preprint, https://arxiv.org/abs/1703.09074, 2017.
[28] N. B. Erichson, S. Voronin, S. L. Brunton, and J. N. Kutz, Randomized Matrix Decompositions using R, preprint, https://arxiv.org/abs/1608.02148, 2016.
[29] M. Farazmand and T. P. Sapsis, Dynamical indicators for the prediction of bursting phenomena in high-dimensional systems, Phys. Rev. E, 94 (2016), 032212.
[30] G. Golub and V. Pereyra, Separable nonlinear least squares: The variable projection method and its applications, Inverse Problems, 19 (2003), pp. R1-R26. · Zbl 1022.65014
[31] J. C. Gower and G. B. Dijksterhuis, Procrustes Problems, Oxford Statist. Sci. Ser. 30, Oxford University Press, Oxford, 2004. · Zbl 1057.62044
[32] N. Halko, P.-G. Martinsson, and J. A. Tropp, Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions, SIAM Rev., 53 (2011), pp. 217-288, https://doi.org/10.1137/090771806. · Zbl 1269.65043
[33] J. H. Hong, C. Zach, and A. Fitzgibbon, Revisiting the variable projection method for separable nonlinear least squares problems, in Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2017, pp. 5939-5947.
[34] H. Hotelling, Analysis of a complex of statistical variables into principal components, J. Educ. Psychol., 24 (1933), pp. 417-441. · JFM 59.1182.04
[35] P. J. Huber, Robust statistics, in International Encyclopedia of Statistical Science, Springer, Berlin, 2011, pp. 1248-1251.
[36] M. Hubert, T. Reynkens, E. Schmitt, and T. Verdonck, Sparse PCA for high-dimensional data with outliers, Technometrics, 58 (2016), pp. 424-434.
[37] I. T. Jolliffe, Principal component analysis and factor analysis, in Principal Component Analysis, Springer, New York, 1986, pp. 115-128.
[38] I. T. Jolliffe, N. T. Trendafilov, and M. Uddin, A modified principal component technique based on the LASSO, J. Comput. Graph. Statist., 12 (2003), pp. 531-547.
[39] M. Journée, Y. Nesterov, P. Richtárik, and R. Sepulchre, Generalized power method for sparse principal component analysis, J. Mach. Learn. Res., 11 (2010), pp. 517-553. · Zbl 1242.62048
[40] E. Kaiser, B. R. Noack, L. Cordier, A. Spohn, M. Segond, M. Abel, G. Daviller, J. Östh, S. Krajnović, and R. K. Niven, Cluster-based reduced-order modelling of a mixing layer, J. Fluid Mech., 754 (2014), pp. 365-414. · Zbl 1329.76177
[41] J. N. Kutz, X. Fu, and S. L. Brunton, Multiresolution dynamic mode decomposition, SIAM J. Appl. Dyn. Syst., 15 (2016), pp. 713-735, https://doi.org/10.1137/15M1023543. · Zbl 1338.37121
[42] D. Lee, W. Lee, Y. Lee, and Y. Pawitan, Super-sparse principal component analyses for high-throughput genomic data, BMC Bioinf., 11 (2010), 296.
[43] J.-C. Loiseau and S. L. Brunton, Constrained sparse Galerkin regression, J. Fluid Mech., 838 (2018), pp. 42-67. · Zbl 1419.76205
[44] S. Ma and Y. Dai, Principal component analysis based methods in bioinformatics studies, Brief. Bioinform., 12 (2011), pp. 714-722.
[45] M. W. Mahoney, Randomized algorithms for matrices and data, Found. Trends Mach. Learn., 3 (2011), pp. 123-224. · Zbl 1232.68173
[46] K. Manohar, B. W. Brunton, J. N. Kutz, and S. L. Brunton, Data-driven sparse sensor placement for reconstruction, IEEE Control Systems Magazine, 38 (2018), pp. 63-86.
[47] R. Maronna, R. D. Martin, and V. Yohai, Robust Statistics, John Wiley & Sons, Chichester, 2006. · Zbl 1094.62040
[48] D. Meng, Q. Zhao, and Z. Xu, Improve robustness of sparse PCA by \(L_1\)-norm maximization, Pattern Recog., 45 (2012), pp. 487-497. · Zbl 1225.68202
[49] B. Mordukhovich, Variational Analysis and Generalized Differentiation. I, Grundlehren Math. Wiss. 330, Springer-Verlag, Berlin, 2006.
[50] B. R. Noack, K. Afanasiev, M. Morzynski, G. Tadmor, and F. Thiele, A hierarchy of low-dimensional models for the transient and post-transient cylinder wake, J. Fluid Mech., 497 (2003), pp. 335-363. · Zbl 1067.76033
[51] V. Ozoliņš, R. Lai, R. Caflisch, and S. Osher, Compressed modes for variational problems in mathematics and physics, Proc. Natl. Acad. Sci. USA, 110 (2013), pp. 18368-18373. · Zbl 1292.81024
[52] N. Parikh and S. Boyd, Proximal algorithms, Found. Trends Optim., 1 (2014), pp. 127-239.
[53] K. Pearson, On lines and planes of closest fit to systems of points in space, The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 2 (1901), pp. 559-572. · JFM 32.0710.04
[54] R. W. Reynolds, N. A. Rayner, T. M. Smith, D. C. Stokes, and W. Wang, An improved in situ and satellite SST analysis for climate, J. Climate, 15 (2002), pp. 1609-1625.
[55] R. T. Rockafellar and R. J.-B. Wets, Variational Analysis, Grundlehren Math. Wiss. 317, Springer-Verlag, Berlin, 2009. · Zbl 0888.49001
[56] T. P. Sapsis and P. F. Lermusiaux, Dynamically orthogonal field equations for continuous stochastic dynamical systems, Phys. D, 238 (2009), pp. 2347-2360. · Zbl 1180.37119
[57] H. Schaeffer, R. Caflisch, C. D. Hauck, and S. Osher, Sparse dynamics for partial differential equations, Proc. Natl. Acad. Sci. USA, 110 (2013), pp. 6634-6639. · Zbl 1292.35012
[58] H. Shen and J. Z. Huang, Sparse principal component analysis via regularized low rank matrix approximation, J. Multivariate Anal., 99 (2008), pp. 1015-1034. · Zbl 1141.62049
[59] C. D. Sigg and J. M. Buhmann, Expectation-maximization for sparse and non-negative PCA, in Proceedings of the 25th International Conference on Machine Learning, ACM, 2008, pp. 960-967.
[60] N. Simon, J. Friedman, T. Hastie, and R. Tibshirani, A sparse-group lasso, J. Comput. Graph. Statist., 22 (2013), pp. 231-245.
[61] K. Taira, S. L. Brunton, S. Dawson, C. W. Rowley, T. Colonius, B. J. McKeon, O. T. Schmidt, S. Gordeyev, V. Theofilis, and L. S. Ukeiley, Modal analysis of fluid flows: An overview, AIAA J., 55 (2017), pp. 4013-4041.
[62] K. Taira and T. Colonius, The immersed boundary method: A projection approach, J. Comput. Phys., 225 (2007), pp. 2118-2137. · Zbl 1343.76027
[63] K. Taira, A. G. Nair, and S. L. Brunton, Network structure of two-dimensional decaying isotropic turbulence, J. Fluid Mech., 795 (2016), R2. · Zbl 1359.76124
[64] R. Tibshirani, M. Saunders, S. Rosset, J. Zhu, and K. Knight, Sparsity and smoothness via the fused lasso, J. R. Stat. Soc. Ser. B Stat. Methodol., 67 (2005), pp. 91-108. · Zbl 1060.62049
[65] P. Tseng, Convergence of a block coordinate descent method for nondifferentiable minimization, J. Optim. Theory Appl., 109 (2001), pp. 475-494. · Zbl 1006.65062
[66] M. Udell and A. Townsend, Why are big data matrices approximately low rank?, SIAM J. Math. Data Sci., 1 (2019), pp. 144-160, https://doi.org/10.1137/18M1183480.
[67] M. Velegar, N. B. Erichson, C. A. Keller, and J. N. Kutz, Scalable diagnostics for global atmospheric chemistry using Ristretto library (version 1.0), Geosci. Model Dev., 12 (2019), pp. 1525-1539, https://doi.org/10.5194/gmd-12-1525-2019.
[68] D. M. Witten, R. Tibshirani, and T. Hastie, A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis, Biostatistics, 10 (2009), pp. 515-534.
[69] M. Yuan and Y. Lin, Model selection and estimation in regression with grouped variables, J. R. Stat. Soc. Ser. B Stat. Methodol., 68 (2006), pp. 49-67. · Zbl 1141.62030
[70] H. Zou and T. Hastie, Regularization and variable selection via the elastic net, J. R. Stat. Soc. Ser. B Stat. Methodol., 67 (2005), pp. 301-320. · Zbl 1069.62054
[71] H. Zou, T. Hastie, and R. Tibshirani, Sparse principal component analysis, J. Comput. Graph. Statist., 15 (2006), pp. 265-286.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.