×

zbMATH — the first resource for mathematics

A joint convex penalty for inverse covariance matrix estimation. (English) Zbl 06983941
Summary: The paper proposes a joint convex penalty for estimating the Gaussian inverse covariance matrix. A proximal gradient method is developed to solve the resulting optimization problem with more than one penalty constraints. The analysis shows that imposing a single constraint is not enough and the estimator can be improved by a trade-off between two convex penalties. The developed framework can be extended to solve wide arrays of constrained convex optimization problems. A simulation study is carried out to compare the performance of the proposed method to graphical lasso and the SPICE estimate of the inverse covariance matrix.

MSC:
62 Statistics
Software:
CHOMPACK; glasso
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Argyriou, A.; Evgeniou, T.; Pontil, M., Convex multi-task feature learning, Mach. Learn., 73, 3, 243-272, (2008)
[2] Bach, F., Consistency of trace norm minimization, J. Mach. Learn. Res., 9, 1019-1048, (2008) · Zbl 1225.68146
[3] Bach, F.; Jenatton, R.; Mairal, J.; Obozinski, G., Convex optimization with sparsity-inducing norms, (Optimization for Machine Learning, (2011), MIT Press) · Zbl 06064248
[4] Banerjee, O.; El Ghaoui, L.; d’Aspremont, A., Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data, J. Mach. Learn. Res., 9, 485-516, (2008) · Zbl 1225.68149
[5] Bertsekas, D. P., Incremental gradient, subgradient, and proximal methods for convex optimization, a survey, (Labaratory for Information and Decision Systems Report LIDS-P-2848, (2010), MIT)
[6] Bickel, P.; Levina, E., Regulatized estimation of large covariance matrices, Ann. Statist., 36, 199-227, (2008) · Zbl 1132.62040
[7] Candès, E.; Recht, B., Foundations of computational mathematics, (2009), Springer
[8] Dahl, J.; Vandenberghe, L.; Roychowdhury, V., Covariance selection for non-chordal graphs via chordal embedding, Optim. Methods Softw., 23, 4, 501-520, (2008), Methods and Software · Zbl 1151.90514
[9] Dempster, A., Covariance selection, Biometrika, 32, 95-108, (1972)
[10] Fazel, M., 2002. Matrix Rank Minimization with Applications. Elec. Eng. Dept, Stanford University (March).
[11] Friedman, J.; Hastie, T.; Tibshirani, R., Sparse inverse covariance estimation with the graphical lasso, Biostatistics, 9, 3, 432-441, (2007), 2008 Jul · Zbl 1143.62076
[12] Liu, H., Roeder, K., Wasserman, L., 2010. Stability approach to regularization selection (StARS) for high dimensional graphical models. In: Proceedings of the Twenty-Third Annual Conference on Neural Information Processing Systems (NIPS).
[13] Meinshausen; Bühlmann, P., High dimensional graphs and variable selection with the lasso, Ann. Statist., 34, 1436-1462, (2006) · Zbl 1113.62082
[14] Nesterov, Yu., Smooth minimization of non-smooth functions, Math. Program., 127-152, (2005) · Zbl 1079.90102
[15] Raguet, H., Fadili, J., Peyre, G., 2011. Generalized forward-backward splitting. Arxiv preprint arXiv:1108.4404.
[16] Rockafellar, R. T., Monotone operators and proximal point algorithm, SIAM J. Control Optim., 14, 5, (1976) · Zbl 0358.90053
[17] Rolfs, B., Rajaratnam, B., Guillot, D., Wong, I., Maleki, A., 2012. Iterative thresholding algorithm for sparse inverse covariance estimation. In: Adv. Neural Inf. Process. Syst. (NIPS), pp. 1583-1591.
[18] Rothman, A. J.; Bickel, P. J.; Levina, E.; Zhu, J., Sparse permutation invariant covariance estimation, Electron. J. Stat., 2, 494-515, (2008) · Zbl 1320.62135
[19] Sheena, Y.; Gupta, A., Estimation of the multivariate normal covariance matrix under some restrictions, Statist. Decisions, 21, 327-342, (2003) · Zbl 1041.62043
[20] Tomioka, R., Aihara, K., 2007. Classifying matrices with a spectral regularization. In: Proc. 24th Int. Conf. Machine Learning, pp. 895-902.
[21] Vandenberghe, L.; Boyd, S., Convex optimization, (2004), Cambridge University Press · Zbl 1058.90049
[22] Wainwright, M., Ravikumar, P., Lafferty, J.D., 2006. High-dimensional graphical model selection using \(L_1\)-regularized logistic regression. In: Proceedings of Advances in Neural in formation Processing Systems. · Zbl 1189.62115
[23] Won, J.; Lim, J.; Kim, S.; Rajaratnam, B., Condition number regularized covariance estimation, J. R. Stat. Soc. Ser. B, (2012)
[24] Yuan, M., Sparse inverse covariance matrix estimation via linear programming, J. Mach. Learn. Res., 11, 2261-2286, (2009) · Zbl 1242.62043
[25] Yuan, M.; Lin, Y., Model selection and estimation in the Gaussian graphical model, Biometrika, 94, 1, 19-35, (2007) · Zbl 1142.62408
[26] Zhou, S.; Rutimann, P.; Xu, M.; Buh¨lmann, P., High-dimensional covariance estimation based on Gaussian graphical models, J. Mach. Learn. Res., (2011), (in press)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.