Hostname: page-component-7b7567c8c-7pwfn Total loading time: 0 Render date: 2026-11-16T10:30:42.322Z Has data issue: false hasContentIssue false

Simplicity and the Sub-Family Problem for Model Selection

Published online by Cambridge University Press:  25 May 2022

Alireza Fatollahi*
Affiliation:
Department of Philosophy, Princeton University, Princeton, USA
Kasra Alishahi*
Affiliation:
Department of Mathematical Sciences, Sharif University of Technology, Tehran, Iran
*
Rights & Permissions [Opens in a new window]

Abstract

Forster and Sober (1994) introduced the “sub-family problem” for model selection criteria that recommend balancing goodness-of-fit against simplicity. This problem arises when a maximally simple model (family of hypotheses) is artificially constructed to have excellent fit with the data. We argue that the problem arises because of a violation of the general maxim that balancing goodness-of-fit against simplicity leads to desirable inferences only if one is comparing models for the consideration of which one has a positive reason independently of the current data.

Information

Type
Article
Copyright
© The Author(s), 2022. Published by Cambridge University Press on behalf of the Philosophy of Science Association
Figure 0

Figure 1.T is the truth. L(K) is the likeliest hypothesis in K, which contains T. For a Gaussian distribution of error, L(K) is equally likely to fall on any point on the circle. T and L(K) are independent of the model. θ0 is the predictively most accurate hypothesis in the model (i.e., the closest member of the model to T).

Figure 1

Figure 2.Histogram of the Sub-family Error of {L(M4,y)}. Histogram of the Sub-family Error for M0:{–3x3+x2–1+N(0,1)}.

Figure 2

Figure 3.Histogram of the distance between M0 and L(M4,y). Histogram of the sub-family error for M0 only for those data sets for which ||M0-L(M4,y)||< 0.3 (left of the red line in 3a).

Figure 3

Figure 4.Histogram of the AIC difference between M0 and M4 for data sets for which ||M0-L(M4,y)||< 0.3 (left of the red line in 3a).