×

Comparing probability measures using possibility theory: a notion of relative peakedness. (English) Zbl 1122.68133

Summary: Deciding whether one probability distribution is more informative (in the sense of representing a less indeterminate situation) than another one is typically done using well-established information measures such as, e.g., the Shannon entropy or other dispersion indices. In contrast, the relative specificity of possibility distributions is evaluated by means of fuzzy set inclusion. In this paper, we propose a technique for comparing probability distributions from the point of view of their relative dispersion without resorting to a numerical index. A natural partial ordering in terms of relative “peakedness” of probability functions is proposed which is closely related to order-1 stochastic dominance. There is also a close connection between this ordering on probability distributions and the standard specificity ordering on possibility distributions that can be derived by means of a known probability-possibility transformation. The paper proposes a direct proof showing that the (total) preordering on probability measures defined by probabilistic entropy refines the (partial) ordering defined by possibilistic specificity. This result, also valid for other dispersion indices, is discussed against the background of related work in statistics, mathematics (inequalities on convex functions), and the social sciences. Finally, an application of the possibilistic specificity ordering in the field of machine learning or, more specifically, the induction of decision forests is proposed.

MSC:

68T37 Reasoning under uncertainty in the context of artificial intelligence

Software:

C4.5
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Birnbaum, Z. W., On random variables with comparable peakedness, Annals of Mathematical Statistics, 19, 76-81 (1948) · Zbl 0031.36801
[2] Breiman, L., Random forests, Machine Learning, 45, 1, 5-32 (2001) · Zbl 1007.68152
[3] Delgado, M.; Moral, S., On the concept of possibility-probability consistency, Fuzzy Sets and Systems, 21, 311-318 (1987) · Zbl 0618.60003
[4] Dubois, D.; Prade, H., On several representations of an uncertain body of evidence, (Gupta, M. M.; Sanchez, E., Fuzzy Information and Decision Processes (1982), North-Holland: North-Holland Amsterdam), 167-181
[5] Dubois, D.; Prade, H., Unfair coins and necessity measures, Fuzzy Sets and Systems, 10, 1, 15-20 (1983) · Zbl 0515.60005
[6] Dubois, D.; Prade, H., A note on measures of specificity for fuzzy sets, International Journal of General Systems, 10, 279-283 (1985) · Zbl 0569.94032
[7] Dubois, D.; Prade, H., The principle of minimum specificity as a basis for evidential reasoning, (Bouchon, B.; Yager, R. R., Uncertainty in Knowledge-Based Systems (1987), Springer Verlag), 75-84
[8] Dubois, D.; Prade, H., When upper probabilities are possibility measures, Fuzzy Sets and Systems, 49, 65-74 (1992) · Zbl 0754.60003
[9] Dubois, D.; Prade, H.; Sandri, S., On possibility/probability transformations, (Lowen, R.; Roubens, M., Fuzzy Logic. State of the Art (1993), Kluwer Academic Publishers: Kluwer Academic Publishers Dordrecht), 103-112
[10] Dubois, D.; Nguyen, H. T.; Prade, H., Possibility theory, probability and fuzzy sets: misunderstandings, bridges and gaps, (Dubois, D.; Prade, H., Fundamentals of Fuzzy Sets. Fundamentals of Fuzzy Sets, The Handbooks of Fuzzy Sets Series (2000), Kluwer: Kluwer Boston, MA), 343-438 · Zbl 0978.94052
[11] Dubois, D.; Foulloy, L.; Mauris, G.; Prade, H., Possibility/probability transformations, triangular fuzzy sets, and probabilistic inequalities, Reliable Computing, 10, 273-297 (2004) · Zbl 1043.60003
[12] Fayyad, U.; Irani, K. B., Multi-interval discretization of continuous attributes as preprocessing for classification learning, (Proceedings of the 13th international Joint Conference on Artificial Intelligence (1993), Morgan Kaufmann), 1022-1027
[13] Friedman, J. H.; Kohavi, R.; Yun, Y., Lazy decision trees, (Proceedings AAAI-96 (1996), Morgan Kaufmann: Morgan Kaufmann Menlo Park, CA), 717-724
[14] Hardy, G. H.; Littlewood, J. E.; Polya, G., Inequalities (1952), Cambridge University Press: Cambridge University Press Cambridge, UK · Zbl 0047.05302
[15] Higashi, M.; Klir, G., Measures of uncertainty and information based on possibility distributions, International Journal of General Systems, 8, 43-58 (1982) · Zbl 0497.94008
[16] Hüllermeier, E.; Fürnkranz, J., Learning label preferences: ranking error versus position error, (Proceedings IDA05 6th International Symposium on Intelligent Data Analysis. Proceedings IDA05 6th International Symposium on Intelligent Data Analysis, LNCS, 3646 (2005), Springer-Verlag: Springer-Verlag Madrid), 180-191 · Zbl 1141.68568
[17] Hunter, D., Causality and maximum entropy updating, International Journal of Approximate Reasoning, 3, 1, 379-406 (1989)
[18] Klir, G., A principle of uncertainty and information invariance, International Journal of General Systems, 17, 249-275 (1990) · Zbl 0703.94026
[19] Lukasiewicz, T., Credal networks under maximum entropy, (Boutilier, C.; Goldszmidt, M., Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence (UAI 2000), July 2000 (2000), Morgan Kaufmann: Morgan Kaufmann Stanford, CA, USA), 363-370
[20] Marshall, A.; Olkin, I., Inequalities: A Theory of Majorization and its Applications (1970), Academic Press: Academic Press New York
[21] Maung, I., Two characterizations of a minimum-information principle in possibilistic reasoning, International Journal of Approximate Reasoning, 12, 133-156 (1995) · Zbl 0814.68121
[22] Mingers, J., An empirical comparison of selection measures for decision-tree induction, Machine Learning, 3, 319-342 (1989)
[23] Morales, D.; Pardo, L.; Vajda, I., Uncertainty of discrete stochastic systems: general theory and statistical theory, IEEE Transactions on System, Man and Cybernetics, 26, 1-17 (1996)
[24] Moulin, H., Axioms of Cooperative Decision Making (1988), Cambridge University Press: Cambridge University Press Cambridge, MA · Zbl 0699.90001
[25] Paris, J., The Uncertain Reasoner’s Companion (1994), Cambridge University Press: Cambridge University Press Cambridge, UK · Zbl 0838.68104
[26] Quinlan, J. R., C4.5: Programs for Machine Learning (1993), Morgan Kaufmann: Morgan Kaufmann San Mateo, CA
[27] Ramer, A., Possibilistic information metrics and distances: characterizations of structure, International Journal of General Systems, 18, 1-10 (1990) · Zbl 0726.94002
[28] Shore, E.; Johnson, R. W., Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy, IEEE Transactions on Information Theory, 26, 26-36 (1980) · Zbl 0429.94011
[29] Smets, P., Constructing the pignistic probability function in a context of uncertainty, (Henrion, M.; etal., Uncertainty in Artificial Intelligence, vol. 5 (1990), North-Holland: North-Holland Amsterdam), 29-39 · Zbl 0721.68065
[30] Yager, R. R., On the specificity of a possibility distribution, Fuzzy Sets and Systems, 50, 279-292 (1992) · Zbl 0783.94035
[31] Witten, I. H.; Frank, E., Data Mining: Practical Machine Learning Tools and Techniques (2005), Morgan Kaufmann: Morgan Kaufmann San Francisco · Zbl 1076.68555
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.