×

On typical encodings of multivariate ergodic sources. (English) Zbl 1513.94027

Summary: We show that the typical coordinate-wise encoding of multivariate ergodic source into prescribed alphabets has the entropy profile close to the convolution of the entropy profile of the source and the modular polymatroid that is determined by the cardinalities of the output alphabets. We show that the proportion of the exceptional encodings that are not close to the convolution goes to zero doubly exponentially. The result holds for a class of multivariate sources that satisfy asymptotic equipartition property described via the mean fluctuation of the information functions. This class covers asymptotically mean stationary processes with ergodic mean, ergodic processes, irreducible Markov chains with an arbitrary initial distribution. We also proved that typical encodings yield the asymptotic equipartition property for the output variables. These asymptotic results are based on an explicit lower bound of the proportion of encodings that transform a multivariate random variable into a variable with the entropy profile close to the suitable convolution.

MSC:

94A24 Coding theorems (Shannon theory)
94A29 Source coding
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Bassoli, R.; Marques, H.; Rodriguez, J.; Shum, K. W.; Tafazolli, R., Network coding theory: A survey., IEEE Commun. Surveys Tutor. 15 (2013), 1950-1978 · doi:10.1109/surv.2013.013013.00104
[2] Cover, T. M.; Thomas, J. A., Elements of Information Theory., John Wiley and Sons, 2012 · doi:10.1002/0471200611
[3] Gray, R. M.; Kieffer, J. C., Asymptotically mean stationary measures., Ann. Probab. 8 (1980), 962-973 · Zbl 0447.28014 · doi:10.1214/aop/1176994624
[4] Gray, R. M., Entropy and Information Theory., Springer Science and Business Media, 2011 · Zbl 1216.94001
[5] Kaced, T., Partage de secret et théorie algorithmique de l’information., PhD. Thesis, Université Montpellier 2, 2012
[6] Kieffer, J. C., A generalized Shannon-McMillan theorem for the action of an amenable group on a probability space., Ann. Probab. 3 (1975), 1031-1037 · Zbl 0322.60032 · doi:10.1214/aop/1176996230
[7] Matúš, F., Two constructions on limits of entropy functions., IEEE Trans. Inform. Theory 53 (2007), 320-330 · Zbl 1310.94052 · doi:10.1109/tit.2006.887090
[8] Matúš, F.; Csirmaz, L., Entropy region and convolution., IEEE Trans. Inform. Theory 62 (2016), 6007-6018 · Zbl 1359.94342 · doi:10.1109/tit.2016.2601598
[9] Matúš, F.; Kupsa, M., On colorings of bivariate random sequences., In: Proc. IEEE International Symposium on Information Theory 2010, pp. 1272-1275 · doi:10.1109/isit.2010.5513700
[10] Yeung, R. W., Information Theory and Network Coding., Springer Science and Business Media, 2008 · Zbl 1154.94001 · doi:10.1007/978-0-387-79234-7_1
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.