×

A channel-based perspective on conjugate priors. (English) Zbl 1442.62054

Summary: A desired closure property in Bayesian probability is that an updated posterior distribution be in the same class of distributions – say Gaussians – as the prior distribution. When the updating takes place via a statistical model, one calls the class of prior distributions the ‘conjugate priors’ of the model. This paper gives (1) an abstract formulation of this notion of conjugate prior, using channels, in a graphical language, (2) a simple abstract proof that such conjugate priors yield Bayesian inversions and (3) an extension to multiple updates. The theory is illustrated with several standard examples.

MSC:

62F15 Bayesian inference
62N01 Censored data models

Software:

PRMLT
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Abramsky, S. and Coecke, B. (2009). A categorical semantics of quantum protocols. In: Engesser, K., Gabbay, D. M. and Lehmann, D. (eds.) Handbook of Quantum Logic and Quantum Structures: Quantum Logic, North-Holland, Elsevier, Computer Science Press, 261-323. · Zbl 1273.81014
[2] Ackerman, N., Freer, C. and Roy, D. (2011). Noncomputable conditional distributions. In: Dawar, A. and Grädel, E. (eds.) Logic in Computer Science, IEEE, Computer Science Press.
[3] Alpaydin, E. (2010). Introduction to Machine Learning, 2nd edn., Cambridge, MA, MIT Press. · Zbl 1191.68485
[4] Bernardo, J. and Smith, A. (2000). Bayesian Theory. Chichester, John Wiley & Sons. · Zbl 0943.62009
[5] Bishop, C. (2006). Pattern Recognition and Machine Learning. Information Science and Statistics. Chichester, Springer. · Zbl 1107.68072
[6] Blackwell, D. (1951). Comparison of experiments. In: Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, Springer/British Computer Society, 93-102. · Zbl 0044.14203
[7] Cho, K. and Jacobs, B. (2019). Disintegration and Bayesian inversion, both abstractly and concretely. In: Mathematical Structures in Computer Science. See also arxiv.org/abs/1709.00322.
[8] Cho, K., Jacobs, B., Westerbaan, A. and Westerbaan, B. (2015). An introduction to effectus theory. See arxiv.org/abs/1512.05813. · Zbl 1352.81011
[9] Clerc, F., Dahlqvist, F., Danos, V. and Garnier, I. (2017). Pointless learning. In: Esparza, J. and Murawski, A. (eds.) Foundations of Software Science and Computation Structures, Lecture Notes in Computer Science, vol. 10203, Berlin, Springer, 355-369. · Zbl 1486.68146
[10] Culbertson, J. and Sturtz, K. (2014). A categorical foundation for Bayesian probability. Applied Categorical Structures22(4) 647-662. · Zbl 1302.60011
[11] Diaconis, P. and Ylvisaker, D. (1979). Conjugate priors for exponential families. Annals of Statistics7(2) 269-281. · Zbl 0405.62011
[12] Faden, A. (1985). The existence of regular conditional probabilities: Necessary and sufficient conditions. The Annals of Probability13(1) 288-298. · Zbl 0593.60002
[13] Giry, M. (1982). A categorical approach to probability theory. In: Banaschewski, B. (ed.) Categorical Aspects of Topology and Analysis, Lecture Notes in Computer Science, vol. 915, Berlin, Springer, 68-85. · Zbl 0486.60034
[14] Jacobs, B. (2013). Measurable spaces and their effect logic. In: Logic in Computer Science, IEEE, Computer Science Press. · Zbl 1433.03162
[15] Jacobs, B. (2015). New directions in categorical logic, for classical, probabilistic and quantum logic. Logical Methods in Computer Science11(3). See https://lmcs.episciences.org/1600. · Zbl 1338.03117
[16] Jacobs, B. (2018). From probability monads to commutative effectuses. Journal of Logical and Algebraic Methods in Programming94200-237. · Zbl 1382.68073
[17] Jacobs, B. and Westerbaan, A. (2015). An effect-theoretic account of Lebesgue integration. In: Ghica, D. (ed.) Mathematical Foundations of Programming Semantics, Electronic Notes in Theoretical Computer Science, vol. 319, Amsterdam, Elsevier, 239-253. · Zbl 1352.81011
[18] Jacobs, B. and Zanasi, F. (2019). The logical essentials of Bayesian reasoning. In: Barthe, G. and Katoen, J.-P. and Silva, A.Probabilistic Programming, Cambridge, Cambridge University Press. See arxiv.org/abs/1804.01193. · Zbl 07311049
[19] Koopman, B. (1936). On distributions admitting a sufficient statistic. Transactions of the American Mathematical Society39399-409. · JFM 62.0611.03
[20] Panangaden, P. (2009). Labelled Markov Processes. London, Imperial College Press. · Zbl 1190.60001
[21] Russell, S. and Norvig, P. (2003). Artificial Intelligence. A Modern Approach. Englewood Cliffs, NJ, Prentice Hall. · Zbl 0835.68093
[22] Selinger, P. (2007). Dagger compact closed categories and completely positive maps (extended abstract). In: Selinger, P. (ed.) Proceedings of the 3rd International Workshop on Quantum Programming Languages (QPL 2005), Electronic Notes in Theoretical Computer Science, vol. 170, Amsterdam, Elsevier, 139-163. doi: http://dx.doi.org/10.1016/j.entcs.2006.12.018. · Zbl 1277.18008
[23] Selinger, P. (2011). A survey of graphical languages for monoidal categories. In: Coecke, B. (ed.) New Structures in Physics, Lecture Notes in Physics, vol. 813, Berlin, Springer, 289-355. · Zbl 1217.18002
[24] Stoyanov, J. (2014). Counterexamples in Probability, 2nd rev. edn. Chichester, Wiley.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.