×

Conjugate and conditional conjugate Bayesian analysis of discrete graphical models of marginal independence. (English) Zbl 1471.62152

Summary: A conjugate and conditional conjugate Bayesian analysis is presented for bi-directed discrete graphical models, which are used to describe and estimate marginal associations between categorical variables. To achieve this, each bi-directed graph is re-expressed by a Markov equivalent, over the observed margin, directed acyclic graph (DAG). This DAG equivalent model is obtained using the same vertex set or with the addition of some latent variables when required. It is characterised by a minimal set of marginal and conditional probability parameters. Hence compatible priors based on products of Dirichlet distributions can be applied. For models with DAG representation on the same vertex set, the posterior distribution and the marginal likelihood is analytically available, while for the remaining ones a data augmentation scheme introducing additional latent variables is required. For the latter, the marginal likelihood is estimated using Chib’s estimator. Additional implementation details including identifiability of such models are discussed. Moreover, analytic details concerning the computation of the posterior distributions of the marginal log-linear parameters are provided. The computation is achieved via a simple transformation of the simulated values of the probability parameters of the bi-directed model under study. The marginal log-linear parameterisation provides a straightforward interpretation in terms of log-odds ratios on specific marginals quantifying the associations between variables involved in the corresponding marginal. The proposed methodology is illustrated using a popular 4-way dataset.

MSC:

62-08 Computational methods for problems pertaining to statistics
62F15 Bayesian inference
62H17 Contingency tables
62H22 Probabilistic graphical models
PDFBibTeX XMLCite
Full Text: DOI Link

References:

[1] Barbieri, M. M.; Berger, J., Optimal predictive model selection, Annals of Statistics, 32, 870-897, (2004) · Zbl 1092.62033
[2] Bartlett, M. S., A comment on lindley’s statistical paradox, Biometrika, 44, 533-534, (1957) · Zbl 0081.13801
[3] Bartolucci, F.; Scaccia, L.; Farcomeni, A., Bayesian inference through encompassing priors and importance sampling for a class of marginal models for categorical data, Computational Statistics and Data Analysis, 56, 4067-4080, (2012) · Zbl 1255.62057
[4] Bergsma, W. P.; Rudas, T., Marginal log-linear models for categorical data, Annals of Statistics, 30, 140-159, (2002) · Zbl 1012.62063
[5] Buntine, W., Theory refinement on Bayesian networks, (D’Ambrosio, P. S.B.; Bonissone, P., Proceedings of the Conference on Uncertainty in Artificial Intelligence, (1991), Morgan Kaufmann), 52-60
[6] Chib, S., Marginal likelihood from the Gibbs output, Journal of the American Statistical Association, 90, 1313-1321, (1995) · Zbl 0868.62027
[7] Consonni, G.; Veronese, P., Compatibility of prior specifications across linear models, Statistical Science, 23, 332-353, (2008) · Zbl 1329.62331
[8] Coppen, A., The mark-nyman temperament scale: an English translation, British Journal of Medical Psychology, 39, 55-59, (1966)
[9] Cox, D. R.; Wermuth, N., Linear dependencies represented by chain graphs, Statistical Science, 8, (1993), (with discussion) 204-218, 247-277 · Zbl 0955.62593
[10] Dawid, A. P.; Lauritzen, S. L., Compatible prior distributions, (George, E. I., Bayesian methods with applications to science policy and official statistics. The Sixth World Meeting of the International Society for Bayesian Analysis, (2000)), 109-118
[11] Dellaportas, P.; Forster, J. J., Markov chain Monte Carlo model determination for hierarchical and graphical log-linear models, Biometrika, 86, 615-633, (1999) · Zbl 0949.62050
[12] Diebolt, J.; Robert, C. P., Estimation of finite mixture distributions through Bayesian sampling, Journal of the Royal Statistical Society B, 56, 363-375, (1994) · Zbl 0796.62028
[13] Drton, M.; Richardson, T. S., Binary models for marginal independence, Journal of the Royal Statistical Society B, 70, 287-309, (2008) · Zbl 1148.62043
[14] Drton, M.; Richardson, T. S., Graphical methods for efficient likelihood inference in Gaussian covariance models, Journal of Machine Learning Research, 9, 893-914, (2008) · Zbl 1225.62031
[15] Evans, R. J., Graphical mathods for ineuqality constraints in marginalised dags, IEEE International Workshop on Machine Learning for Signal Processing, 1-6, (2012)
[16] Evans, R. J.; Richardson, T. S., Marginal log-linear parameters for graphical Markov models, Journal of the Royal Statistical Society B, (2013), (in press)
[17] Frigyik, B. A.; Kapila, A.; Gupta, M. R., Introduction to the Dirichlet distribution and related processes. technical report. UWEETR-2010-0006, (2010), Department of Electrical Engineering. University of Washington
[18] Frühwirth-Schnatter, S., Markov chain Monte Carlo estimation of classical and dynamic switching and mixture models, Journal American Statistical Association, 96, 194-209, (2001) · Zbl 1015.62022
[19] Green, P. J., Reversible jump Markov chain Monte Carlo computation and Bayesian model determination, Biometrika, 82, 711-732, (1995) · Zbl 0861.62023
[20] Heckerman, D.; Geiger, D.; Chickering, D., Learning Bayesian networks: the combination of knowledge and statistical data, Machine Learning, 20, 194-243, (1995) · Zbl 0831.68096
[21] Jasra, A.; Holmes, C. C.; Stephens, D. A., Markov chain Monte Carlo methods and the label switching problem in Bayesian mixture modeling, Statistical Science, 20, 50-67, (2005) · Zbl 1100.62032
[22] Kang, C., Tian, J., 2012. Inequality constraints in causal models with hidden variables. arXiv:1206.6829.
[23] Khare, K.; Rajaratnam, B., Wishart distributions for decomposable covariance graph models, Annals of Statistics, 39, 514-555, (2011) · Zbl 1274.62369
[24] Knuiman, M. W.; Speed, T. P., Incorporating prior information into the analysis of contingency tables, Biometrics, 44, 1061-1071, (1988) · Zbl 0715.62106
[25] Letac, G.; Massam, H., Wishart distributions for decomposable graphs, The Annals of Statistics, 35, 1278-1323, (2007) · Zbl 1194.62078
[26] Lindley, D. V., A statistical paradox, Biometrika, 44, 187-192, (1957) · Zbl 0080.12801
[27] Lupparelli, M., 2006. Graphical models of marginal independence for categorical variables. Ph.D. Thesis. University of Florence.
[28] Lupparelli, M.; Marchetti, G. M.; Bergsma, W. P., Parameterization and Fitting of discrete bi-directed graph models, Scandinavian Journal of Statistics, 36, 559-576, (2009) · Zbl 1196.62074
[29] Marin, J.-M.; Robert, C. P., Approximating the marginal likelihood in mixture models, Bulletin of the Indian Chapter of ISBA, V, 1, 2-7, (2008), Also available as arXiv:0804.2414 at http://arxiv.org/abs/0804.2414
[30] Neal, R.M., 1998. Erroneous results in marginal likelihood from the Gibbs output (with associated comments written 16 March 1999). Technical Report. Department of Statistics and Department of Computer Science. University of Toronto. Canada. Available at http://www.cs.toronto.edu/ radford/chib-letter.html.
[31] Ntzoufras, I., Tarantola, C., 2012. Bayesian analysis of graphical models of marginal independence for three way contingency tables. Technical Report. Department of Political Economy and Quantitative Methods. University of Pavia.
[32] Papastamoulis, P.; Iliopoulos, G., An artificial allocations based solution to the label switching problem in Bayesian analysis of mixtures of distributions, Journal of Computational and Graphical Statistics, 19, 313-331, (2010)
[33] Pearl, J.; Wermuth, N., When can association graphs admit a causal interpretation?, (Cheesman, P.; Oldford, W., Models and Data, Artificial Intelligence and Statistics IV, (1994), Springer New York), 205-214 · Zbl 0828.05060
[34] Perks, W., Some observations on inverse probability including a new indifference rule, Journal of the Institute of Actuaries, 73, 285-334, (1947) · Zbl 0031.06001
[35] Richardson, T. S., Markov property for acyclic directed mixed graphs, Scandinavian Journal of Statistics, 30, 145-157, (2003) · Zbl 1035.60005
[36] Roverato, A.; Consonni, G., Compatible prior distributions for DAG models, Journal of the Royal Statistical Society B, 66, 47-61, (2004) · Zbl 1062.62050
[37] Roverato, A.; Lupparelli, M.; La Rocca, L., Log-mean linear models for binary data, Biometrika, (2013) · Zbl 1284.62465
[38] Rudas, T.; Bergsma, W. P., On applications of marginal models for categorical data, Metron, LXII, 1-25, (2004)
[39] Silva, R.; Ghahramani, Z., The hidden life of latent variables: Bayesian learning with mixed graph models, Journal of Machine Learning Research, 10, 1187-1238, (2009) · Zbl 1235.68191
[40] Silva, R.; Ghahramani, Z., Factorial mixture of gaussians and the marginal independence model, (Proceedings of the 12th International Conference on Artificial Intelligence and Statistics (AISTATS), (2009), Clearwater Beach Florida, USA)
[41] Sisson, S. A., Trans-dimensional Markov chains: a decade of progress and future perspectives, Journal of the Amarican Statistical association, 100, 1077-1089, (2005) · Zbl 1117.62428
[42] Stephens, M., Dealing with label switching in mixture models, Journal of the Royal Statistical Society B, 62, 795-809, (2000) · Zbl 0957.62020
[43] Tian, J.; Pearl, J., On the testable implications of causal models with hidden variables, (Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence, (2002), Morgan Kaufmann), 519-552
[44] Ver Steeg, G.; Galstyan, A., A sequence of relaxation constraining hidden variable models, (Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence, (2011), AUAI Press), 717-726
[45] Wermuth, N., Model search among multiplicative models, Biometrics, 32, 253-263, (1976) · Zbl 0339.62079
[46] Yao, W., Model based labeling for mixture models, Statistics and Computing, 22, 337-347, (2012) · Zbl 1322.62047
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.