zbMATH — the first resource for mathematics

An introduction to model selection using probability theory as logic. (English) Zbl 0895.62004
Heidbreder, Glenn R. (ed.), Maximum entropy and Bayesian methods. Proceedings of the 13th international workshop, Santa Barbara, CA, USA, August 1-5, 1993. Dordrecht: Kluwer Academic Publishers. Fundam. Theor. Phys. 62, 1-42 (1996).
Summary: Probability theory as logic is founded on three simple desiderata: that degrees of belief should be represented by real numbers, that one should reason consistently, and that the theory should reduce to Aristotelian logic when the truth values of the hypotheses are known. Because this theory represents a probability as a state of knowledge, not a state of nature, hypotheses such as “The frequency of oscillation of a sinusoidal signal had value \(\omega\) when the data were taken,” or “Model \(x\) is a better description of the data than model \(y\)” make perfect sense. Problems of the first type are generally thought of as parameter estimation problems, while problems of the second type are thought of as model selection problems. However, in probability theory there is no essential distinction between these two types of problems. They are both solved by application of the sum and product rules of probability theory. Model selection problems are conceptually more difficult, because the models may have different functional forms. Consequently, conceptual difficulties enter problems that are not present in parameter estimation.
This paper is a tutorial on model selection. The conceptual problems that arise in model selection will be illustrated in such a way as to automatically avoid any difficulties. A simple example is worked in detail. This example, (radar target identification) illustrates all of the points of principle that must be faced in more complex model selection problems, including how to handle nuisance parameters, uninformative prior probabilities, and incomplete sets of models.
For the entire collection see [Zbl 0845.00049].

62B10 Statistical aspects of information-theoretic topics
62N99 Survival analysis and censored data
62A01 Foundations and philosophical topics in statistics
62P99 Applications of statistics