×

Optimal rates of convergence for covariance matrix estimation. (English) Zbl 1202.62073

Summary: The covariance matrix plays a central role in multivariate statistical analysis. Significant advances have been made recently on developing both theory and methodology for estimating large covariance matrices. However, a minimax theory has yet been developed. We establish the optimal rates of convergence for estimating the covariance matrix under both the operator norm and the Frobenius norm. It is shown that optimal procedures under the two norms are different and consequently matrix estimation under the operator norm is fundamentally different from vector estimation. The minimax upper bound is obtained by constructing a special class of tapering estimators and by studying their risk properties. A key step in obtaining the optimal rate of convergence is the derivation of the minimax lower bound. The technical analysis requires new ideas that are quite different from those used in the more conventional function/sequence estimation problems.

MSC:

62H12 Estimation in multivariate analysis
62F12 Asymptotic properties of parametric estimators
62C20 Minimax procedures in statistical decision theory
65C60 Computational problems in statistics (MSC2010)
62G09 Nonparametric statistical resampling methods
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Assouad, P. (1983). Deux remarques sur l’estimation. C. R. Acad. Sci. Paris Sér. I Math. 296 1021-1024. · Zbl 0568.62003
[2] Bickel, P. J. and Levina, E. (2008a). Regularized estimation of large covariance matrices. Ann. Statist. 36 199-227. · Zbl 1132.62040 · doi:10.1214/009053607000000758
[3] Bickel, P. J. and Levina, E. (2008b). Covariance regularization by thresholding. Ann. Statist. 36 2577-2604. · Zbl 1196.62062 · doi:10.1214/08-AOS600
[4] Csiszár, I. (1967). Information-type measures of difference of probability distributions and indirect observation. Studia Sci. Math. Hungar. 2 229-318. · Zbl 0157.25802
[5] El Karoui, N. (2008). Operator norm consistent estimation of large dimensional sparse covariance matrices. Ann. Statist. 36 2717-2756. · Zbl 1196.62064 · doi:10.1214/07-AOS559
[6] Golub, G. H. and Van Loan, C. F. (1983). Matrix Computations . John Hopkins Univ. Press, Baltimore. · Zbl 0559.65011
[7] Fan, J., Fan, Y. and Lv, J. (2008). High dimensional covariance matrix estimation using a factor model. J. Econometrics 147 186-197. · Zbl 1429.62185 · doi:10.1016/j.jeconom.2008.09.017
[8] Furrer, R. and Bengtsson, T. (2007). Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants. J. Multivariate Anal. 98 227-255. · Zbl 1105.62091 · doi:10.1016/j.jmva.2006.08.003
[9] Huang, J., Liu, N., Pourahmadi, M. and Liu, L. (2006). Covariance matrix selection and estimation via penalised normal likelihood. Biometrika 93 85-98. · Zbl 1152.62346 · doi:10.1093/biomet/93.1.85
[10] Johnstone, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29 295-327. · Zbl 1016.62078 · doi:10.1214/aos/1009210544
[11] Johnstone, I. M. and Lu, A. Y. (2009). On consistency and sparsity for principal components analysis in high dimensions. J. Amer. Statist. Assoc. 104 682-693. · Zbl 1388.62174
[12] Lam, C. and Fan, J. (2007). Sparsistency and rates of convergence in large covariance matrices estimation. Technical report, Princeton Univ. · Zbl 1191.62101
[13] Muirhead, R. J. (1987). Developments in eigenvalue estimation. In Advances in Multivariate Statistical Analysis (A. K. Gupta, ed.) 277-288. Reidel, Dordrecht. · Zbl 0636.62039
[14] Ravikumar, P., Wainwright, M. J., Raskutti, G. and Yu, B. (2008). High-dimensional covariance estimation by minimizing l 1 -penalized log-determinant divergence. Technical report, Univ. California, Berkeley. · Zbl 1274.62190
[15] Rothman, A. J., Bickel, P. J., Levina, E. and Zhu, J. (2008). Sparse permutation invariant covariance estimation. Electron. J. Stat. 2 494-515. · Zbl 1320.62135 · doi:10.1214/08-EJS176
[16] Rudelson, M. and Vershynin, R. (2007). Sampling from large matrices: An approach through geometric functional analysis. J. ACM 54 Art. 21, 19 pp. (electronic). · Zbl 1326.68333 · doi:10.1145/1255443.1255449
[17] Saulis, L. and Statulevičius, V. A. (1991). Limit Theorems for Large Deviations . Springer, Berlin. · Zbl 0744.60028
[18] Wu, W. B. and Pourahmadi, M. (2009). Banding sample covariance matrices of stationary processes. Statist. Sinica 19 1755-1768. · Zbl 1176.62083
[19] Yu, B. (1997). Assouad, Fano and Le Cam. In Festschrift for Lucien Le Cam (D. Pollard, E. Torgersen and G. Yang, eds.) 423-435. Springer, Berlin. · Zbl 0896.62032
[20] Zhang, C.-H. and Huang, J. (2008). The sparsity and bias of the Lasso selection in high-dimensional linear regression. Ann. Statist. 36 1567-1594. · Zbl 1142.62044 · doi:10.1214/07-AOS520
[21] Zou, H., Hastie, T. and Tibshirani, R. (2006). Sparse principal components analysis. J. Comput. Graph. Statist. 15 265-286. · doi:10.1198/106186006X113430
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.