×

Jeffreys versus Shtarkov distributions associated with some natural exponential families. (English) Zbl 1463.62024

Summary: Jeffreys and Shtarkov distributions play an important role in universal coding and minimum description length (MDL) inference, two central areas within the field of information theory. It was recently discovered that in some situations Shtarkov distributions exist while Jeffreys distributions do not. To demonstrate some of these situations we consider in this note the class of natural exponential families (NEF’s) and present a general result which enables us to construct numerous classes of infinitely divisible NEF’s for which Shtarkov distributions exist and Jeffreys distributions do not. The method used to obtain our general results is based on the variance functions of such NEF’s. We first present two classes of parametric NEF’s demonstrating our general results and then generalize them to obtain numerous multiparameter classes of the same type.

MSC:

62E15 Exact distribution theory in statistics
62B10 Statistical aspects of information-theoretic topics
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Bar-Lev, S. K.; Bshouty, D.; Enis, P.; Ohayon, A. Y., Compositions and products of infinitely divisible variance functions, Scandinavian Journal of Statistics, 19, 83-89 (1992) · Zbl 0753.60023
[2] Barron, A.; Rissanen, J.; Yu, B., The minimum description length principle in coding and modeling, IEEE Transactions on Information Theory, 44, 6, 2743-2760 (1998), (Special commemorative issue: Information Theory, 1948-1998.) · Zbl 0933.94013
[3] Bshouty, D., On a characterization of variance functions of natural exponential families, Mathematical Methods of Statistics, 4, 92-98 (1995) · Zbl 0831.62012
[4] Clarke, B.; Barron, A., Information-theoretic asymptotics of Bayes methods, IEEE Transactions on Information Theory, IT-36, 3, 453-471 (1990) · Zbl 0709.62008
[5] Clarke, B.; Barron, A., Jeffreys’ prior is asymptotically least favorable under entropy risk, Journal of Statistical Planning and Inference, 41, 37-60 (1994) · Zbl 0820.62006
[6] Cover, T.; Thomas, J. A., Elements of Information Theory (1991), Wiley · Zbl 0762.94001
[7] Grünwald, P., The Minimum Description Length principle (2007), MIT Press
[8] Grünwald, P.; Harremoës, P., Finiteness of redundancy, regret, Shtarkov Sums and Jeffreys integrals in exponential families, (Proceedings of the International Symposium for Information Theory, ISIT 2009 (2009), IEEE), 714-718
[9] Jeffreys, H., An invariant form for the prior probability in estimation problems, Proceedings of the Royal Statistical Society (London) Series A, 186, 453-461 (1946) · Zbl 0063.03050
[10] Letac, G.; Mora, M., Natural real exponential families with cubic variance functions, The Annals of Statistics, 18, 1-37 (1990) · Zbl 0714.62010
[11] Morris, C. N., Natural exponential families with quadratic variance functions, The Annals of Statistics, 10, 65-80 (1982) · Zbl 0498.62015
[12] Rissanen, J., Fisher information and stochastic complexity, IEEE Transactions on Information Theory, 42, 1, 40-47 (1996) · Zbl 0856.94006
[13] Shtarkov, Y. M., Universal sequential coding of single messages, Problems of Information Transmission, 23, 3, 3-17 (1987) · Zbl 0668.94005
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.