×

Expectation propagation for likelihood-free inference. (English) Zbl 1367.62063

Summary: Many models of interest in the natural and social sciences have no closed-form likelihood function, which means that they cannot be treated using the usual techniques of statistical inference. In the case where such models can be efficiently simulated, Bayesian inference is still possible thanks to the approximate Bayesian computation (ABC) algorithm. Although many refinements have been suggested, ABC inference is still far from routine. ABC is often excruciatingly slow due to very low acceptance rates. In addition, ABC requires introducing a vector of “summary statistics” \(s(y)\), the choice of which is relatively arbitrary, and often require some trial and error, making the whole process laborious for the user. We introduce in this work the EP-ABC algorithm, which is an adaptation to the likelihood-free context of the variational approximation algorithm known as expectation propagation. The main advantage of EP-ABC is that it is faster by a few orders of magnitude than standard algorithms, while producing an overall approximation error that is typically negligible. A second advantage of EP-ABC is that it replaces the usual global ABC constraint \(\|s(y) - s(y^{\star})\|\leq \varepsilon\), where \(s(y^{\star})\) is the vector of summary statistics computed on the whole dataset, by \(n\) local constraints of the form \(\|s_i(y_i) - s_i(y^{\star}_i)\|\) that apply separately to each data point. In particular, it is often possible to take \(s_i(y_i) = y_i\), making it possible to do away with summary statistics entirely. In that case, EP-ABC makes it possible to approximate directly the evidence (marginal likelihood) of the model. Comparisons are performed in three real-world applications that are typical of likelihood-free inference, including one application in neuroscience that is novel, and possibly too challenging for standard ABC techniques.

MSC:

62F15 Bayesian inference
62P10 Applications of statistics to biology and medical sciences; meta analysis

Software:

PRMLT
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] DOI: 10.1111/j.1467-9868.2009.00736.x · doi:10.1111/j.1467-9868.2009.00736.x
[2] Andrieu C., 44th IEEE Conference on Decision and Control, 2005 and 2005 European Control Conference. CDC-ECC’05 pp 332– (2005)
[3] Bazin E., Genetics 185 pp 587– (2010) · doi:10.1534/genetics.109.112391
[4] Beaumont M.A., Annual Review of Ecology, Evolution, and Systematics 41 pp 379– (2010) · doi:10.1146/annurev-ecolsys-102209-144621
[5] Beaumont M.A., Genetics 162 pp 2025– (2002)
[6] Bishop C., Pattern Recognition and Machine Learning (2006) · Zbl 1107.68072
[7] DOI: 10.1198/jasa.2010.tm09448 · Zbl 1390.62052 · doi:10.1198/jasa.2010.tm09448
[8] DOI: 10.1098/rstb.2007.2059 · doi:10.1098/rstb.2007.2059
[9] DOI: 10.1016/0304-4076(86)90063-1 · Zbl 0616.62119 · doi:10.1016/0304-4076(86)90063-1
[10] Boyen X., Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence pp 33– (1998)
[11] Boys R., Statistics and Computing 18 pp 125– (2008) · doi:10.1007/s11222-007-9043-x
[12] Brascamp J.W., Journal of Vision 6 pp 1244– (2006) · doi:10.1167/6.11.8
[13] Calvet L., Cahier de Recherche du Groupe HEC pp 947– (2011)
[14] DOI: 10.1080/01621459.1976.10480344 · doi:10.1080/01621459.1976.10480344
[15] DOI: 10.1214/11-STS376 · Zbl 1330.86021 · doi:10.1214/11-STS376
[16] Diggle P., Journal of the Royal Statistical Society, Series B 46 pp 193– (1984)
[17] Fearnhead P., Journal of the Royal Statistical Society, Series B 74 pp 1– (2012) · doi:10.1111/j.1467-9868.2011.01010.x
[18] Gelfand A., Journal of the Royal Statistical Society, Series B 56 pp 501– (1994)
[19] Gentle J., Random Number Generation and Monte Carlo Methods (2003) · Zbl 1028.65004
[20] Gerstner W., Spiking Neuron Models: Single Neurons, Populations, Plasticity (1st ed.) (2002) · Zbl 1100.92501 · doi:10.1017/CBO9780511815706
[21] DOI: 10.1021/j100540a008 · doi:10.1021/j100540a008
[22] Golightly A., Interface Focus 1 pp 807– (2011) · doi:10.1098/rsfs.2011.0047
[23] Jasra A., Statistics and Computing 22 pp 1223– (2012) · Zbl 1252.62093 · doi:10.1007/s11222-010-9185-0
[24] DOI: 10.1111/1467-937X.00050 · Zbl 0910.90067 · doi:10.1111/1467-937X.00050
[25] DOI: 10.1080/01621459.1994.10476469 · doi:10.1080/01621459.1994.10476469
[26] DOI: 10.1080/01621459.1992.10476265 · doi:10.1080/01621459.1992.10476265
[27] Liu S., Journal of Econometrics 10 pp 273– (1995) · doi:10.1002/jae.3950100305
[28] Luce D.R., Response Times: Their Role in Inferring Elementary Mental Organization (Oxford Psychology Series) (1991)
[29] DOI: 10.1073/pnas.0306899100 · doi:10.1073/pnas.0306899100
[30] Maybeck P. S., Stochastic Models, Estimation, and Control (1982) · Zbl 0546.93063
[31] Mengersen K.L., Proceedings of the National Academy of Sciences 110 pp 1321– (2013) · doi:10.1073/pnas.1208827110
[32] Menn C., European Journal of Operational Research 163 pp 201– (2005) · Zbl 1067.90120 · doi:10.1016/j.ejor.2004.01.009
[33] DOI: 10.1016/0301-0511(88)90013-0 · doi:10.1016/0301-0511(88)90013-0
[34] Minka T., Proceedings of Uncertainty in Artificial Intelligence 17 pp 362– (2001)
[35] DOI: 10.1016/S0927-5398(00)00019-0 · doi:10.1016/S0927-5398(00)00019-0
[36] Nolan J. P., Stable Distributions–Models for Heavy Tailed Data (2013)
[37] DOI: 10.1037/a0018924 · doi:10.1037/a0018924
[38] Opper M., On-line Learning in Neural Networks pp 363– (1999)
[39] Paquet U., Journal of Machine Learning Research 10 pp 1263– (2009)
[40] DOI: 10.1016/j.csda.2010.10.004 · Zbl 1255.62071 · doi:10.1016/j.csda.2010.10.004
[41] DOI: 10.1093/oxfordjournals.molbev.a026091 · doi:10.1093/oxfordjournals.molbev.a026091
[42] DOI: 10.1037/0033-295X.85.2.59 · doi:10.1037/0033-295X.85.2.59
[43] DOI: 10.1162/neco.2008.12-06-420 · Zbl 1147.68720 · doi:10.1162/neco.2008.12-06-420
[44] Rubin D., Journal of the American Statistical Association 12 pp 1151– (1984)
[45] DOI: 10.1111/j.1467-9868.2008.00700.x · Zbl 1248.62156 · doi:10.1111/j.1467-9868.2008.00700.x
[46] DOI: 10.1007/978-1-4612-4250-5 · doi:10.1007/978-1-4612-4250-5
[47] Tavaré S., Genetics 145 pp 505– (1997) · Zbl 0861.00030
[48] Titterington M., Mixtures: Estimation and Applications (Vol. 896, Chap. 1) (2011) · Zbl 1218.62003
[49] DOI: 10.1098/rsif.2008.0172 · doi:10.1098/rsif.2008.0172
[50] DOI: 10.1017/CBO9780511753930 · Zbl 1047.62098 · doi:10.1017/CBO9780511753930
[51] Varin C., Statistica Sinica 21 pp 5– (2011)
[52] Wang B., Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics pp 373– (2005)
[53] DOI: 10.3758/BF03194544 · doi:10.3758/BF03194544
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.