# zbMATH — the first resource for mathematics

On the distribution of the largest eigenvalue in principal components analysis. (English) Zbl 1016.62078
Summary: Let $$x_{(1)}$$ denote the square of the largest singular value of an $$n\times p$$ matrix $$X$$, all of whose entries are independent standard Gaussian variates. Equivalently, $$x_{(1)}$$ is the largest principal component variance of the covariance matrix $$X'X$$, or the largest eigenvalue of a $$p$$-variate Wishart distribution with $$n$$ degrees of freedom and identity covariance. Consider the limit of large $$p$$ and $$n$$ with $$n/p=\gamma\geq 1$$. When centered by $$\mu_p=(\sqrt{n-1}+\sqrt p)^2$$ and scaled by $$\sigma_p=\break (\sqrt{n-1}+\sqrt p)(1/\sqrt{n-1}+1/\sqrt p)^{1/3}$$, the distribution of $$x_{(1)}$$ approaches the Tracy-Widom law [C.A. Tracy and H. Widom, J. Stat. Phys. 92, No. 5-6, 809-835 (1998; Zbl 0942.60099)] of order 1, which is defined in terms of a Painlevé II differential equation and can be numerically evaluated and tabulated by software.
Simulations show the approximation to be informative for $$n$$ and $$p$$ as small as 5. The limit is derived via a corresponding result for complex Wishart matrices using methods from random matrix theory. The result suggests that some aspects of large $$p$$ multivariate distribution theory may be easier to apply in practice than their fixed $$p$$ counterparts.

##### MSC:
 62H25 Factor analysis and principal components; correspondence analysis 62H10 Multivariate distribution of statistics 15B52 Random matrices (algebraic aspects) 33E17 Painlevé-type functions 33C45 Orthogonal polynomials and functions of hypergeometric type (Jacobi, Laguerre, Hermite, Askey scheme, etc.) 60F05 Central limit and other weak theorems
Full Text:
##### References:
  Aldous, D. and Diaconis, P. (1999). Longest increasing subsequences: from patience sorting to the Baik-Deift-Johansson theorem. Bull. Amer. Math. Soc. 36 413-432. · Zbl 0937.60001 · doi:10.1090/S0273-0979-99-00796-X  Anderson, T. W. (1963). Asymptotic theory for principal component analysis. Ann Math. Statist. 34 122-148 · Zbl 0202.49504 · doi:10.1214/aoms/1177704248  Anderson, T. W. (1996). R. A. Fisher and multivariate analysis. Statist. Sci. 11 20-34. · Zbl 0955.62504 · doi:10.1214/ss/1032209662  Bai, Z. D. (1999). Methodologies in spectral analysis of large dimensional random matrices: a review. Statist. Sinica 9 611-677. · Zbl 0949.60077  Baik, J., Deift, P. and Johansson, K. (1999). On the distribution of the length of the longest increasing subsequence of random permutations. J. Amer. Math. Soc. 12 1119-1178. JSTOR: · Zbl 0932.05001 · doi:10.1090/S0894-0347-99-00307-0 · links.jstor.org  Baker, T. H., Forrester, P. J. and Pearce, P. A. (1998). Random matrix ensembles with an effective extensive external charge. J. Phys. A 31 6087-6101. · Zbl 0912.15030 · doi:10.1088/0305-4470/31/29/002  Basor, E. L. (1997). Distribution functions for random variables for ensembles of positive Hermitian matrices, Comm. Math. Phys. 188 327-350. · Zbl 0905.47016 · doi:10.1007/s002200050167  Buja, A., Hastie, T. and Tibshirani, R. (1995). Penalized discriminant analysis. Ann. Statist. 23 73-102. · Zbl 0821.62031 · doi:10.1214/aos/1176324456  Constantine, A. G. (1963). Some non-central distribution problems in multivariate analysis. Ann. Math. Statist. 34 1270-1285. Deift, P. (1999a). Integrable systems and combinatorial theory. Notices Amer. Math. Soc. 47 631-640. Deift, P. (1999b). Orthogonal Polynomials and Random Matrices: A Riemann-Hilbert Approach. Amer. Math. Soc., Providence, RI. · Zbl 0123.36801 · doi:10.1214/aoms/1177703863  Dunster, T. M. (1989). Uniform asymptotic expansions for Whittaker’s confluent hypergeometric functions. SIAM J. Math. Anal. 20 744-760. · Zbl 0673.33003 · doi:10.1137/0520052  Dyson, F. J. (1970). Correlations between eigenvalues of a random matrix. Comm. Math. Phys. 19 235-250. · Zbl 0221.62019 · doi:10.1007/BF01646824  Eaton, M. L. (1983). Multivariate Statistics: A Vector Space Approach. Wiley, NewYork. · Zbl 0587.62097  Edelman, A. (1988). Eigenvalues and condition numbers of random matrices. SIAM J. Matrix Anal. Appl. 9 543-560. · Zbl 0678.15019 · doi:10.1137/0609045  Edelman, A. (1991). The distribution and moments of the smallest eigenvalue of a random matrix of Wishart type. Linear Algebra Appl. 159 55-80. · Zbl 0738.15010 · doi:10.1016/0024-3795(91)90076-9  Erdélyi, A. (1960). Asymptotic forms for Laguerre polynomials. J. Indian Math. Soc. 24 235-250. · Zbl 0105.05401  Forrester, P. J. (1993). The spectrum edge of random matrix ensembles. Nuclear Phys. B 402 709-728. · Zbl 1043.82538 · doi:10.1016/0550-3213(93)90126-A  Forrester, P. J. (2000). Painlevé transcendent evaluation of the scaled distribution of the smallest eigenvalue in the Laguerre orthogonal and symplectic ensembles. Technical report. www.lanl.gov URL: · www.lanl.gov  Geman, S. (1980). A limit theorem for the norm of random matrices. Ann. Probab. 8 252-261. · Zbl 0428.60039 · doi:10.1214/aop/1176994775  Gohberg, I. C. and Krein, M. G. (1969). Introduction to the Theory of Linear Non-selfadjoint Operators. Amer. Math. Soc., Providence, RI. · Zbl 0181.13504  Hastings, S. P. and McLeod, J. B. (1980). A boundary value problem associated with the second Painlevé transcendent and the Korteweg-de Vries equation. Arch. Rational Mech. Anal. 73 31-51. · Zbl 0426.34019 · doi:10.1007/BF00283254  Horn, R. A. and Johnson, C. R. (1985). Matrix Analysis. Cambridge Univ. Press. · Zbl 0576.15001  James, A. T. (1964). Distributions of matrix variates and latent roots derived from normal samples. Ann. Math. Statist. 35 475-501. · Zbl 0121.36605 · doi:10.1214/aoms/1177703550  Johansson, K. (1998). On fluctations of eigenvalues of random Hermitian matrices. Duke Math. J. 91 151-204. · Zbl 1039.82504 · doi:10.1215/S0012-7094-98-09108-6  Johansson, K. (2000). Shape fluctuations and random matrices. Comm. Math. Phys. 209 437-476. · Zbl 0969.15008 · doi:10.1007/s002200050027  Mardia, K. V., Kent, J. T. and Bibby, J. M. (1979). Multivariate Analysis. Academic Press, NewYork. · Zbl 0432.62029  Mar cenko, V. A. and Pastur, L. A. (1967). Distributions of eigenvalues of some sets of random matrices. Math. USSR-Sb. 1 507-536. · Zbl 0162.22501 · doi:10.1070/SM1967v001n04ABEH001994  Mehta, M. L. (1991). Random Matrices, 2nd ed. Academic Press, NewYork. · Zbl 0780.60014  Muirhead, R. J. (1974). Powers of the largest latent root test of = I. Comm. Statist. 3 513-524. · Zbl 0284.62026 · doi:10.1080/03610927408827154  Muirhead, R. J. (1982). Aspects of Multivariate Statistical Theory. Wiley, NewYork. · Zbl 0556.62028  Olver, F. W. J. (1974). Asymptotics and Special Functions. Academic Press, NewYork. · Zbl 0303.41035  Preisendorfer, R. W. (1988). Principal Component Analysis in Meteorology and Oceanogaphy. North-Holland, Amsterdam.  Riesz, F. and Sz.-Nagy, B. (1955). Functional Analysis. Ungar, NewYork. · Zbl 0070.10902  Soshnikov, A. (1999). Universality at the edge of the spectrum in Wigner random matrices. Comm. Math. Phys. 207 697-733. · Zbl 1062.82502 · doi:10.1007/s002200050743  Soshnikov, A. (2001). A note on universality of the distribution of the largest eigenvalues in certain classes of sample covariance matrices, Technical report, www.lanl.gov URL: · Zbl 1018.62042 · www.lanl.gov  Stein, C. (1969). Multivariate analysis I. Technical report, Dept. Statistics Stanford Univ., pages 79-81. (Notes prepared by M. L. Eaton in 1966.)  Szeg ö, G. (1967). Orthogonal Polynomials, 3rd ed. Amer. Math. Soc. Providence. · JFM 65.0278.03  Temme, N. M. (1990). Asymptotic estimates for Laguerre polynomials. J. Appl. Math. Phys. (ZAMP) 41 114-126. · Zbl 0688.33007 · doi:10.1007/BF00946078  Tracy, C. A. and Widom, H. (1994). Level-spacing distributions and the Airy kernel. Comm. Math. Phys. 159 151-174. · Zbl 0789.35152 · doi:10.1007/BF02100489  Tracy, C. A. and Widom, H. (1996). On orthogonal and symplectic matrix ensembles. Comm. Math. Phys. 177 727-754. · Zbl 0851.60101 · doi:10.1007/BF02099545  Tracy, C. A. and Widom, H. (1998). Correlation functions, cluster functions, and spacing distributions for random matrices. J. Statis. Phys. 92 809-835. · Zbl 0942.60099 · doi:10.1023/A:1023084324803  Tracy, C. A. and Widom, H. (1999). Airy kernel and Painlevé II. Technical report. www.lanl.gov solv-int/9901004. To appear in CRM Proceedings and Lecture Notes: ”Isomonodromic Deformations and Applications in Physics,” J. Harnad, ed. URL: · Zbl 1174.82309 · www.lanl.gov  Tracy, C. A. and Widom, H. (2000). The distribution of the largest eigenvalue in the Gaussian ensembles. In Calogero-Moser-Sutherland Models (J. van Diejen and L. Vinet, eds.) 461-472. Springer, NewYork.  Wachter, K. W. (1976). Probability plotting points for principal components. In Ninth Interface Symposium Computer Science and Statistics (D. Hoaglin and R. Welsch, eds.) 299-308. Prindle, Weber and Schmidt, Boston.  Widom, H. (1999). On the relation between orthogonal, symplectic and unitary ensembles. J. Statist. Phys. 94 347-363. · Zbl 0935.60090 · doi:10.1023/A:1004516918143  Wigner, E. P. (1955). Characteristic vectors of bordered matrices of infinite dimensions. Ann. Math. 62 548-564. JSTOR: · Zbl 0067.08403 · doi:10.2307/1970079 · links.jstor.org  Wigner, E. P. (1958). On the distribution of the roots of certain symmetric matrices. Ann. Math. 67 325-328. JSTOR: · Zbl 0085.13203 · doi:10.2307/1970008 · links.jstor.org  Wilks, S. S. (1962). Mathematical Statistics. Wiley, NewYork. · Zbl 0173.45805
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.