×

zbMATH — the first resource for mathematics

Bayesian structure learning in graphical models. (English) Zbl 1308.62119
Summary: We consider the problem of estimating a sparse precision matrix of a multivariate Gaussian distribution, where the dimension \(p\) may be large. Gaussian graphical models provide an important tool in describing conditional independence through presence or absence of edges in the underlying graph. A popular non-Bayesian method of estimating a graphical structure is given by the graphical lasso. In this paper, we consider a Bayesian approach to the problem. We use priors which put a mixture of a point mass at zero and certain absolutely continuous distribution on off-diagonal elements of the precision matrix. Hence the resulting posterior distribution can be used for graphical structure learning. The posterior convergence rate of the precision matrix is obtained and is shown to match the oracle rate. The posterior distribution on the model space is extremely cumbersome to compute using the commonly used reversible jump Markov chain Monte Carlo methods. However, the posterior mode in each graph can be easily identified as the graphical lasso restricted to each model. We propose a fast computational method for approximating the posterior probabilities of various graphs using the Laplace approximation approach by expanding the posterior density around the posterior mode. We also provide estimates of the accuracy in the approximation.

MSC:
62H12 Estimation in multivariate analysis
62F12 Asymptotic properties of parametric estimators
62F15 Bayesian inference
Software:
glasso; HdBCS; huge
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Atay-Kayis, A.; Massam, H., A Monte-Carlo method for computing the marginal likelihood in nondecomposable Gaussian graphical models, Biometrika, 92, 2, 317-335, (2005) · Zbl 1094.62028
[2] Banerjee, O.; El Ghaoui, L.; d’Aspremont, A., Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data, J. Mach. Learn. Res., 9, 485-516, (2008) · Zbl 1225.68149
[3] Banerjee, S.; Ghosal, S., Posterior convergence rates for estimating large precision matrices using graphical models, Electron. J. Stat., 8, 2, 2111-2137, (2014) · Zbl 1302.62124
[4] Bickel, P.; Levina, E., Covariance regularization by thresholding, Ann. Statist., 36, 6, 2577-2604, (2008) · Zbl 1196.62062
[5] Bickel, P.; Levina, E., Regularized estimation of large covariance matrices, Ann. Statist., 36, 1, 199-227, (2008) · Zbl 1132.62040
[6] Boyd, S. P.; Vandenberghe, L., Convex optimization, (2004), Cambridge University Press · Zbl 1058.90049
[7] Cai, T.; Liu, W.; Luo, X., A constrained \(\ell_1\)-minimization approach to sparse precision matrix estimation, J. Amer. Statist. Assoc., 106, 494, 594-607, (2011) · Zbl 1232.62087
[8] Cai, T.; Zhang, C.; Zhou, H., Optimal rates of convergence for covariance matrix estimation, Ann. Statist., 38, 4, 2118-2144, (2010) · Zbl 1202.62073
[9] Castillo, I.; van der Vaart, A., Needles and straw in a haystack: posterior concentration for possibly sparse sequences, Ann. Statist., 40, 4, 2069-2101, (2012) · Zbl 1257.62025
[10] Curtis, S. M.; Banerjee, S.; Ghosal, S., Fast Bayesian model assessment for nonparametric additive regression, Comput. Statist. Data Anal., 71, 347-358, (2014)
[11] Dawid, A.; Lauritzen, S., Hyper Markov laws in the statistical analysis of decomposable graphical models, Ann. Statist., 21, 3, 1272-1317, (1993) · Zbl 0815.62038
[12] Dobra, A.; Hans, C.; Jones, B.; Nevins, J.; Yao, G.; West, M., Sparse graphical models for exploring gene expression data, J. Multivariate Anal., 90, 1, 196-212, (2004) · Zbl 1047.62104
[13] Friedman, J.; Hastie, T.; Tibshirani, R., Sparse inverse covariance estimation with the graphical lasso, Biostatistics, 9, 3, 432-441, (2008) · Zbl 1143.62076
[14] Ghosal, S., Asymptotic normality of posterior distributions for exponential families when the number of parameters tends to infinity, J. Multivariate Anal., 74, 1, 49-68, (2000) · Zbl 1118.62309
[15] Ghosal, S.; Ghosh, J. K.; Van Der Vaart, A. W., Convergence rates of posterior distributions, Ann. Statist., 28, 2, 500-531, (2000) · Zbl 1105.62315
[16] Guo, J.; Levina, E.; Michailidis, G.; Zhu, J., Joint estimation of multiple graphical models, Biometrika, 98, 1, 1-15, (2011) · Zbl 1214.62058
[17] Harville, D. A., Matrix algebra from a statistician’s perspective, (2008), Springer · Zbl 1142.15001
[18] Huang, J.; Liu, N.; Pourahmadi, M.; Liu, L., Covariance matrix selection and estimation via penalised normal likelihood, Biometrika, 93, 1, 85-98, (2006) · Zbl 1152.62346
[19] Karoui, N., Operator norm consistent estimation of large-dimensional sparse covariance matrices, Ann. Statist., 36, 6, 2717-2756, (2008) · Zbl 1196.62064
[20] Lam, C.; Fan, J., Sparsistency and rates of convergence in large covariance matrix estimation, Ann. Statist., 37, 6B, 4254, (2009) · Zbl 1191.62101
[21] Lauritzen, S., Graphical models, (1996), Clarendon Press Oxford · Zbl 0907.62001
[22] Ledoit, O.; Wolf, M., A well-conditioned estimator for large-dimensional covariance matrices, J. Multivariate Anal., 88, 2, 365-411, (2004) · Zbl 1032.62050
[23] Letac, G.; Massam, H., Wishart distributions for decomposable graphs, Ann. Statist., 35, 3, 1278-1323, (2007) · Zbl 1194.62078
[24] Liu, H.; Lafferty, J.; Wasserman, L., The nonparanormal: semiparametric estimation of high dimensional undirected graphs, J. Mach. Learn. Res., 10, 2295-2328, (2009) · Zbl 1235.62035
[25] Meinshausen, N.; Bühlmann, P., High-dimensional graphs and variable selection with the lasso, Ann. Statist., 34, 3, 1436-1462, (2006) · Zbl 1113.62082
[26] Park, T.; Casella, G., The Bayesian lasso, J. Amer. Statist. Assoc., 103, 482, 681-686, (2008) · Zbl 1330.62292
[27] Pati, D.; Bhattacharya, A.; Pillai, N.; Dunson, D., Posterior contraction in sparse Bayesian factor models for massive covariance matrices, Ann. Statist., 42, 3, 1102-1130, (2014) · Zbl 1305.62124
[28] Rajaratnam, B.; Massam, H.; Carvalho, C., Flexible covariance estimation in graphical Gaussian models, Ann. Statist., 36, 6, 2818-2849, (2008) · Zbl 1168.62054
[29] Rothman, A.; Bickel, P.; Levina, E.; Zhu, J., Sparse permutation invariant covariance estimation, Electron. J. Statist., 2, 494-515, (2008) · Zbl 1320.62135
[30] Rothman, A.; Levina, E.; Zhu, J., Generalized thresholding of large covariance matrices, J. Amer. Statist. Assoc., 104, 485, 177-186, (2009) · Zbl 1388.62170
[31] Roverato, A., Cholesky decomposition of a hyper inverse Wishart matrix, Biometrika, 87, 1, 99-112, (2000) · Zbl 0974.62047
[32] Wang, H., Bayesian graphical lasso models and efficient posterior computation, Bayesian Anal., 7, 4, 867-886, (2012) · Zbl 1330.62041
[33] Witten, D. M.; Friedman, J. H.; Simon, N., New insights and faster computations for the graphical lasso, J. Comput. Graph. Statist., 20, 4, 892-900, (2011)
[34] Yuan, M.; Lin, Y., Efficient empirical Bayes variable selection and estimation in linear models, J. Amer. Statist. Assoc., 100, 472, (2005) · Zbl 1117.62453
[35] Yuan, M.; Lin, Y., Model selection and estimation in the Gaussian graphical model, Biometrika, 94, 1, 19-35, (2007) · Zbl 1142.62408
[36] Zhao, T.; Liu, H.; Roeder, K.; Lafferty, J.; Wasserman, L., The huge package for high-dimensional undirected graph estimation in R, J. Mach. Learn. Res., 98888, 1059-1062, (2012) · Zbl 1283.68311
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.