×

Bayesian parameter learning with an application. (English) Zbl 1394.62026

Summary: This paper deals with prior uncertainty in the parameter learning procedure in Bayesian networks. In most studies in the literature, parameter learning is based on two well-known criteria, i.e., the maximum likelihood and the maximum a posteriori. In presence of prior information, the literature abounds with situations in which a maximum a posteriori estimate is computed as a desired estimate but in those studies, it does not seem that the viewpoint behind its use is according to a loss function-based viewpoint. In this paper, we recall the maximum a posteriori estimator as the Bayes estimator under the zero-one loss function and criticizing the zero-one loss, we suggest the use of the general Entropy loss function as a useful loss when overlearning and underlearning need serious attention. We take prior uncertainty into account and extend the act of parameter learning for the case when prior information is polluted. Addressing a real world problem, we conduct a simulation procedure to study behavior of the proposed estimates. Finally, in order to seek the effect of changing hyperparameters of a chosen prior on the learning procedure, we carry out a sensitivity analysis w.r.t. some chosen hyperparameters.

MSC:

62F15 Bayesian inference
68T35 Theory of languages and software systems (knowledge-based systems, expert systems, etc.) for artificial intelligence
68T05 Learning and adaptive systems in artificial intelligence

Software:

TETRAD
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Berger, J.O.: Statistical Decision Theory and Bayesian Analysis, 2nd edn. Springer, New York (1985) · Zbl 0572.62008 · doi:10.1007/978-1-4757-4286-2
[2] Boratynska, A.: Posterior regret \[\Gamma\] Γ-minimax estimation of insurance Premium in collective risk model. Astin Bull. 38, 277-291 (2008) · Zbl 1169.91383 · doi:10.2143/AST.38.1.2030414
[3] Buntine, W.L.: Theory refinement on Bayesian networks. In: Proceedings of the Seventh conference on Uncertainty in Artificial Intelligence, pp. 52-60. Morgan Kaufmann Publishers (1991)
[4] Calabria, R., Pulcini, G.: An engineering approch to Bayes estimation for the Weibull distribution. Microelect. Reliab. 34, 789-802 (1994) · Zbl 0825.62167 · doi:10.1016/0026-2714(94)90004-3
[5] Cheng, J., Greiner, R., Kelly, J., Bell, D., Liu, W.: Learning bayesian networks from data: an information-theory based approach. Artif. Intel. J. 137, 43-90 (2002) · Zbl 0995.68114 · doi:10.1016/S0004-3702(02)00191-1
[6] Cooper, G.F., Herskovits, E.H.: A Bayesian method for the induction of probabilistic networks from data. Mach. Learn. 9(4), 309-347 (1991) · Zbl 0766.68109
[7] Cowell, R.G.: Conditions under which conditional independence and scoring methods lead to identical selection of Bayesian network models. In: Proceedings of the Seventeenth conference on Uncertainty in artificial intelligence, pp. 91-97. Morgan Kaufmann Publishers Inc. (2001)
[8] de Campos, L.M.: A scoring function for learning bayesian networks based on mutual information and conditional independence tests. J. Machine Learn. Res. 7, 2149-2187 (2006) · Zbl 1222.62036
[9] de Campos, C.P., Qiang, J.: Improving Bayesian network parameter learning using constraints. In: 19th International Conference on Pattern Recognition, pp. 1-4. IEEE (2008)
[10] Dey, D.K., Ghosh, M., Srinivasan, C.: Simultaneous estimation of parameters under entropy loss. J. Stat. Plan. Inference 15, 347-363 (1987) · Zbl 0609.62009 · doi:10.1016/0378-3758(86)90108-4
[11] Dey, D.K., Liu, P.L.: On comparison of estimators in a generalized life model. Microelect. Reliab. 32, 207-221 (1992) · doi:10.1016/0026-2714(92)90099-7
[12] Eaton, D., Murphy, K. P.: Exact Bayesian structure learning from uncertain interventions. In: International Conference on Artificial Intelligence and Statistics, pp. 107-114 (2007) · Zbl 1143.62002
[13] Fan, X., Malone, B., Yuan, C.: Finding optimal Bayesian network structures with constraints learned from data. In: Proceedings of the 30th Annual Conference on Uncertainty in Artificial Intelligence (2014)
[14] Grünwald, P.: The Minimum Description Length Principle. MIT Press, Cambridge (2007)
[15] Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: the combination knowledge and statistical data. Mach. Learn. 20(3), 197-243 (1995) · Zbl 0831.68096
[16] Heckerman, D.; Meek, C.; Cooper, G.; Glymour, C. (ed.); Cooper, GF (ed.), Computation, causation, and discovery, 141-166 (1999), Menlo Park
[17] Jozani, Jafari, Parsian, A.: Posterior regret \[\Gamma\] Γ-minimax estimation and prediction with applications on \[k\] k-Records data under entropy loss function. Commun. Stat.- Theory. Methods 37, 2202-2212 (2008) · Zbl 1143.62002
[18] Jensen, F.V.: An Introduction to Bayesian Networks. Springer-Verlag, New York (1996)
[19] Koski, T., Noble, J.M.: Bayesian Networks—An Introduction. John Wiley and Sons, NewYork (2011) · Zbl 1277.62022
[20] Lauritzen, S.L., Spiegelhalter, D.J.: Local computations with probabilities on graphical structures and their application to expert systems. J. R. Stat. Soc. Series B 50, 157-224 (1988) · Zbl 0684.68106
[21] Parsian, A., Kirmani, S.N.U.A.: Estimation under LINEX function. In: Ullah, A., Wan, A.T.K., Chaturvedi, A. (eds.), pp. 53-76 (2002) · Zbl 0639.62021
[22] Pearl, J.: Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Mateo (1988) · Zbl 0746.68089
[23] Ramoni, M., Sebastiani, P.: Bayesian methods. Intelligent Data Analysis. An Introduction, pp. 131-168. Springer, Berlin Heidelberg (2003)
[24] Silander, T., Roos, T., Myllymki, P.: Learning locally minimax optimal Bayesian networks. Int. J. Approx. Reason. 51(5), 544-557 (2010) · doi:10.1016/j.ijar.2010.01.012
[25] Sivaganesan, S.: Range of posterior measures for priors with arbitrary contaminations. Commun. Stat. -Theor. Methods 17, 1591-1612 (1988) · Zbl 0639.62021 · doi:10.1080/03610928808829700
[26] Sivaganesan, S., Berger, J.O.: Ranges of posterior measures for priors with unimodal contaminations. Ann. Stat. 17, 868-889 (1989) · Zbl 0724.62032 · doi:10.1214/aos/1176347148
[27] Soliman, A.: Estimation of parameters of life from progressively. IEEE Trans. Reliab. 54, 34-42 (2005) · doi:10.1109/TR.2004.842528
[28] Spirtes, P., Glymour, C., Scheines, R.: Causation, Prediction, and Search, 2nd edn. MIT Press, Cambridge (2000) · Zbl 0806.62001
[29] Tamada, Y., Imoto, S., Miyano, S.: Parallel algorithm for learning optimal Bayesian network structure. J. Machine Learn. Res. 12, 2437-2459 (2011) · Zbl 1280.68203
[30] Ueno, M. Learning networks determined by the ratio of prior and data. In Grunwald, P., Spirtes, P. (eds.). In: Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, pp 598-605 (2010)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.