×

Spectrum estimation for large dimensional covariance matrices using random matrix theory. (English) Zbl 1168.62052

Summary: Estimating the eigenvalues of a population covariance matrix from a sample covariance matrix is a problem of fundamental importance in multivariate statistics; the eigenvalues of covariance matrices play a key role in many widely used techniques, in particular in principal components analysis (PCA). In many modern data analysis problems, statisticians are faced with large data sets where the sample size, \(n\), is of the same order of magnitude as the number of variables \(p\). Random matrix theory predicts that in this context, the eigenvalues of the sample covariance matrix are not good estimators of the eigenvalues of the population covariance.
We propose to use a fundamental result in random matrix theory, the V.A. Marčenko and L.A. Pastur equation [Mat. Sb., N. Ser. 72(114) 507–536 (1967, Zbl 0152.16101)], to better estimate the eigenvalues of large dimensional covariance matrices. The Marčenko-Pastur equation holds in a very wide generality and under weak assumptions. The estimator we obtain can be thought of as “shrinking” in a nonlinear fashion the eigenvalues of the sample covariance matrix to estimate the population eigenvalues.
Inspired by ideas of random matrix theory, we also suggest a change of point of view when thinking about estimation of high-dimensional vectors: we do not try to estimate directly the vectors but rather a probability measure that describes them. We think this is a theoretically more fruitful way to think about these problems. Our estimator gives fast and good or very good results in extended simulations. Our algorithmic approach is based on convex optimization. We also show that the proposed estimator is consistent.

MSC:

62H12 Estimation in multivariate analysis
15B52 Random matrices (algebraic aspects)
90C25 Convex programming
15A18 Eigenvalues, singular values, and eigenvectors
62H25 Factor analysis and principal components; correspondence analysis
62A09 Graphical methods in statistics

Citations:

Zbl 0152.16101

Software:

Mosek; PDCO
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Akhiezer, N. I. (1965). The Classical Moment Problem and Some Related Questions in Analysis . Hafner Publishing, New York. · Zbl 0135.33803
[2] Anderson, T. W. (1963). Asymptotic theory for principal component analysis. Ann. Math. Statist. 34 122-148. · Zbl 0202.49504 · doi:10.1214/aoms/1177704248
[3] Anderson, T. W. (2003). An Introduction to Multivariate Statistical Analysis , 3rd ed. Wiley, Hoboken, NJ. · Zbl 1039.62044
[4] Bai, Z. D. (1999). Methodologies in spectral analysis of large dimensional random matrices, a review (with discussion). Statist. Sinica 9 611-677. · Zbl 0949.60077
[5] Baik, J., Ben Arous, G. and Péché, S. (2005). Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices. Ann. Probab. 33 1643-1697. · Zbl 1086.15022 · doi:10.1214/009117905000000233
[6] Baik, J. and Silverstein, J. (2006). Eigenvalues of large sample covariance matrices of spiked population models. J. Multivariate Anal. 97 1382-1408. · Zbl 1220.15011 · doi:10.1016/j.jmva.2005.08.003
[7] Bickel, P. J. and Levina, E. (2004). Some theory of Fisher’s linear discriminant function, “naive Bayes,” and some alternatives when there are many more variables than observations. Bernoulli 10 989-1010. · Zbl 1064.62073 · doi:10.3150/bj/1106314847
[8] Bickel, P. J. and Levina, E. (2007). Regularized estimation of large covariance matrices. Ann. Statist. 36 199-227. · Zbl 1132.62040 · doi:10.1214/009053607000000758
[9] Böttcher, A. and Silbermann, B. (1999). Introduction to Large Truncated Toeplitz Matrices . Springer, New York. · Zbl 0916.15012
[10] Boyd, S. and Vandenberghe, L. (2004). Convex Optimization . Cambridge Univ. Press. · Zbl 1058.90049
[11] Burda, Z., Görlich, A., Jarosz, A. and Jurkiewicz, J. (2004). Signal and noise in correlation matrix. Phys. A 343 295-310. · doi:10.1016/j.physa.2004.05.048
[12] Burda, Z., Jurkiewicz, J. and Wacław, B. (2005). Spectral moments of correlated Wishart matrices. Phys. Rev. E 71 026111. · doi:10.1103/PhysRevE.71.026111
[13] Campbell, J., Lo, A. and MacKinlay, C. (1996). The Econometrics of Financial Markets . Princeton Univ. Press. · Zbl 0927.62113
[14] Chen, S. S., Donoho, D. L. and Saunders, M. A. (1998). Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20 33-61. · Zbl 0919.94002 · doi:10.1137/S1064827596304010
[15] Dozier, R. B. and Silverstein, J. W. (2007). On the empirical distribution of eigenvalues of large dimensional information-plus-noise-type matrices. J. Multivariate Anal. 98 678-694. · Zbl 1115.60035 · doi:10.1016/j.jmva.2006.09.006
[16] Durrett, R. (1996). Probability: Theory and Examples , 2nd ed. Duxbury Press, Belmont, CA. · Zbl 0709.60002
[17] El Karoui, N. (2007). Tracy-Widom limit for the largest eigenvalue of a large class of complex sample covariance matrices. Ann. Probab. 35 663-714. · Zbl 1117.60020 · doi:10.1214/009117906000000917
[18] Geman, S. (1980). A limit theorem for the norm of random matrices. Ann. Probab. 8 252-261. · Zbl 0428.60039 · doi:10.1214/aop/1176994775
[19] Geronimo, J. S. and Hill, T. P. (2003). Necessary and sufficient condition that the limit of Stieltjes transforms is a Stieltjes transform. J. Approx. Theory 121 54-60. · Zbl 1030.44003 · doi:10.1016/S0021-9045(02)00042-4
[20] Gibbs, A. L. and Su, F. (2001). On choosing and bounding probability metrics. Internat. Statist. Rev. 70 419-435. · Zbl 1217.62014 · doi:10.2307/1403865
[21] Gray, R. M. (2002). Toeplitz and circulant matrices: A review. Available at http://ee.stanford.edu/ gray/toeplitz.pdf.
[22] Grenander, U. and Szegö, G. (1958). Toeplitz Forms and Their Applications . Univ. California Press, Berkeley. · Zbl 0080.09501
[23] Hastie, T., Tibshirani, R. and Friedman, J. (2001). The Elements of Statistical Learning . Springer, New York. · Zbl 0973.62007
[24] Hiai, F. and Petz, D. (2000). The Semicircle Law, Free Random Variables and Entropy. Mathematical Surveys and Monographs 77 . Amer. Math. Soc., Providence, RI. · Zbl 0955.46037
[25] Johnstone, I. (2001). On the distribution of the largest eigenvalue in principal component analysis. Ann. Statist. 29 295-327. · Zbl 1016.62078 · doi:10.1214/aos/1009210544
[26] Jonsson, D. (1982). Some limit theorems for the eigenvalues of a sample covariance matrix. J. Multivariate Anal. 12 1-38. · Zbl 0491.62021 · doi:10.1016/0047-259X(82)90080-X
[27] Laloux, L., Cizeau, P., Bouchaud, J.-P. and Potters, M. (1999). Noise dressing of financial correlation matrices. Phys. Rev. Lett. 83 1467-1470.
[28] Lax, P. D. (2002). Functional Analysis . Wiley, New York. · Zbl 1009.47001
[29] Ledoit, O. and Wolf, M. (2004). A well-conditioned estimator for large-dimensional covariance matrices. J. Multivariate Anal. 88 365-411. · Zbl 1032.62050 · doi:10.1016/S0047-259X(03)00096-4
[30] Marčenko, V. A. and Pastur, L. A. (1967). Distribution of eigenvalues in certain sets of random matrices. Mat. Sb. (N.S.) 72 507-536. · Zbl 0152.16101
[31] Mardia, K. V., Kent, J. T. and Bibby, J. M. (1979). Multivariate Analysis . Academic Press, London. · Zbl 0432.62029
[32] MOSEK. (2006). MOSEK Optimization Toolbox. Available at http://www.mosek.com.
[33] Paul, D. (2007). Asymptotics of sample eigenstructure for a large dimensional spiked covariance model. Statist. Sinica 17 1617-1642. · Zbl 1134.62029
[34] Rao, N. R., Mingo, J., Speicher, R. and Edelman, A. (2008). Statistical eigen-inference from large Wishart matrices. Ann. Statist. 36 2850-2885. · Zbl 1168.62056 · doi:10.1214/07-AOS583
[35] Silverstein, J. W. (1995). Strong convergence of the empirical distribution of eigenvalues of large-dimensional random matrices. J. Multivariate Anal. 55 331-339. · Zbl 0851.62015 · doi:10.1006/jmva.1995.1083
[36] Silverstein, J. W. and Bai, Z. D. (1995). On the empirical distribution of eigenvalues of a class of large-dimensional random matrices. J. Multivariate Anal. 54 175-192. · Zbl 0833.60038 · doi:10.1006/jmva.1995.1051
[37] Wachter, K. W. (1978). The strong limits of random matrix spectra for sample matrices of independent elements. Ann. Probab. 6 1-18. · Zbl 0374.60039 · doi:10.1214/aop/1176995607
[38] Yin, Y. Q. (1986). Limiting spectral distribution for a class of random matrices. J. Multivariate Anal. 20 50-68. · Zbl 0614.62060 · doi:10.1016/0047-259X(86)90019-9
[39] Yin, Y. Q., Bai, Z. D. and Krishnaiah, P. R. (1988). On the limit of the largest eigenvalue of the large-dimensional sample covariance matrix. Probab. Theory Related Fields 78 509-521. · Zbl 0627.62022 · doi:10.1007/BF00353874
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.