×

Gemini: graph estimation with matrix variate normal instances. (English) Zbl 1301.62054

The paper under review presents a theoretical framework for estimating the row and column covariance and inverse covariance matrices using only one matrix from matrix-variate normal distribution. The author establishes consistency and rates of convergence in the operator and the Frobenius norm of the covariance matrices and their inverses, proves large deviation results for the sample correlation estimators (proposed for estimating both the row an column correlation and covariance matrices given a single matrix or multiple replicates of the matrix-normal data), and provides conditions that guarantee simultaneous estimation of the graphs for both rows and columns. Provided simulation evidence and a real data example show that the methodology allows to recover graphical structures and also to estimate the precision matrices effectively.

MSC:

62H12 Estimation in multivariate analysis
62A09 Graphical methods in statistics
62F12 Asymptotic properties of parametric estimators
62F30 Parametric inference under constraints
62J07 Ridge regression; shrinkage estimators (Lasso)

Software:

glasso; UCI-ml
PDFBibTeX XMLCite
Full Text: DOI arXiv Euclid

References:

[1] Allen, G. I. and Tibshirani, R. (2010). Transposable regularized covariance models with an application to missing data imputation. Ann. Appl. Stat. 4 764-790. · Zbl 1194.62079 · doi:10.1214/09-AOAS314
[2] Banerjee, O., El Ghaoui, L. and d’Aspremont, A. (2008). Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. J. Mach. Learn. Res. 9 485-516. · Zbl 1225.68149
[3] Cai, T., Liu, W. and Luo, X. (2011). A constrained \(\ell_1\) minimization approach to sparse precision matrix estimation. J. Amer. Statist. Assoc. 106 594-607. · Zbl 1232.62087 · doi:10.1198/jasa.2011.tm10155
[4] Dawid, A. P. (1981). Some matrix-variate distribution theory: Notational considerations and a Bayesian application. Biometrika 68 265-274. · Zbl 0464.62039 · doi:10.1093/biomet/68.1.265
[5] Dutilleul, P. (1999). The MLE algorithm for the matrix normal distribution. J. Stat. Comput. Simul. 64 105-123. · Zbl 0960.62056 · doi:10.1080/00949659908811970
[6] Efron, B. (2009). Are a set of microarrays independent of each other? Ann. Appl. Stat. 3 922-942. · Zbl 1196.62138 · doi:10.1214/09-AOAS236
[7] Fan, J., Feng, Y. and Wu, Y. (2009). Network exploration via the adaptive lasso and SCAD penalties. Ann. Appl. Stat. 3 521-541. · Zbl 1166.62040 · doi:10.1214/08-AOAS215
[8] Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348-1360. · Zbl 1073.62547 · doi:10.1198/016214501753382273
[9] Friedman, J., Hastie, T. and Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9 432-441. · Zbl 1143.62076 · doi:10.1093/biostatistics/kxm045
[10] Gupta, A. K. and Varga, T. (1992). Characterization of matrix variate normal distributions. J. Multivariate Anal. 41 80-88. · Zbl 0745.62052 · doi:10.1016/0047-259X(92)90058-N
[11] Kalaitzis, A., Lafferty, J., Lawrence, N. and Zhou, S. (2013). The bigraphical lasso. In Proceedings of the 30 th International Conference on Machine Learning ( ICML- 13). JMLR W&CP 28 1229-1237. Atlanta, GA.
[12] Lam, C. and Fan, J. (2009). Sparsistency and rates of convergence in large covariance matrix estimation. Ann. Statist. 37 4254-4278. · Zbl 1191.62101 · doi:10.1214/09-AOS720
[13] Leng, C. and Tang, C. Y. (2012). Sparse matrix graphical models. J. Amer. Statist. Assoc. 107 1187-1200. · Zbl 1443.62194 · doi:10.1080/01621459.2012.706133
[14] Lu, N. and Zimmerman, D. L. (2005). The likelihood ratio test for a separable covariance matrix. Statist. Probab. Lett. 73 449-457. · Zbl 1071.62052 · doi:10.1016/j.spl.2005.04.020
[15] Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso. Ann. Statist. 34 1436-1462. · Zbl 1113.62082 · doi:10.1214/009053606000000281
[16] Peng, J., Zhou, N. and Zhu, J. (2009). Partial correlation estimation by joint sparse regression models. J. Amer. Statist. Assoc. 104 735-746. · Zbl 1388.62046 · doi:10.1198/jasa.2009.0126
[17] Ravikumar, P., Wainwright, M. J., Raskutti, G. and Yu, B. (2011). High-dimensional covariance estimation by minimizing \(\ell_1\)-penalized log-determinant divergence. Electron. J. Stat. 5 935-980. · Zbl 1274.62190 · doi:10.1214/11-EJS631
[18] Rothman, A. J., Bickel, P. J., Levina, E. and Zhu, J. (2008). Sparse permutation invariant covariance estimation. Electron. J. Stat. 2 494-515. · Zbl 1320.62135
[19] Tsiligkaridis, T., Hero, A. O. III and Zhou, S. (2013). On convergence of Kronecker graphical lasso algorithms. IEEE Trans. Signal Process. 61 1743-1755. · Zbl 1394.62095 · doi:10.1109/TSP.2013.2240157
[20] UCI (1999). UCI machine learning repository. Available at .
[21] Vershynin, R. (2012). Introduction to the non-asymptotic analysis of random matrices. In Compressed Sensing 210-268. Cambridge Univ. Press, Cambridge. · doi:10.1017/CBO9780511794308.006
[22] Weichsel, P. M. (1962). The Kronecker product of graphs. Proc. Amer. Math. Soc. 13 47-52. · Zbl 0102.38801 · doi:10.2307/2033769
[23] Werner, K., Jansson, M. and Stoica, P. (2008). On estimation of covariance matrices with Kronecker product structure. IEEE Trans. Signal Process. 56 478-491. · Zbl 1390.94472 · doi:10.1109/TSP.2007.907834
[24] Yin, J. and Li, H. (2012). Model selection and estimation in the matrix normal graphical model. J. Multivariate Anal. 107 119-140. · Zbl 1236.62058 · doi:10.1016/j.jmva.2012.01.005
[25] Yuan, M. (2010). High dimensional inverse covariance matrix estimation via linear programming. J. Mach. Learn. Res. 11 2261-2286. · Zbl 1242.62043
[26] Yuan, M. and Lin, Y. (2007). Model selection and estimation in the Gaussian graphical model. Biometrika 94 19-35. · Zbl 1142.62408 · doi:10.1093/biomet/asm018
[27] Zhang, X. L., Begleiter, H., Porjesz, B., Wang, W. and Litke, A. (1995). Event related potentials during object recognition tasks. Brain Res. Bull. 38 531-538.
[28] Zhang, Y. and Schneider, J. (2010). Learning multiple tasks with a sparse matrix-normal penalty. In Advances in Neural Information Processing Systems 23 (NIPS 2010) (J. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. S. Zemel and A. Culotta, eds.).
[29] Zhou, S. (2013). Supplement to “Gemini: Graph estimation with matrix variate normal instances.” .
[30] Zhou, S., Lafferty, J. and Wasserman, L. (2010). Time varying undirected graphs. Machine Learning 80 298-319. · Zbl 1475.62174 · doi:10.1007/s10994-010-5180-0
[31] Zhou, S., Rütimann, P., Xu, M. and Bühlmann, P. (2011). High-dimensional covariance estimation based on Gaussian graphical models. J. Mach. Learn. Res. 12 2975-3026. · Zbl 1280.62065
[32] Zou, H. and Li, R. (2008). One-step sparse estimates in nonconcave penalized likelihood models. Ann. Statist. 36 1509-1533. · Zbl 1142.62027 · doi:10.1214/009053607000000802
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.