1. Introduction
The virtues of a good scientific theory, or simply theoretical virtues, are a central topic in the philosophy of science. Consistency, accuracy, simplicity, unifying power, and fertility figure prominently in debates about scientific realism and theory choice (Kuhn Reference Kuhn1977; van Fraassen Reference van Fraassen1980; McMullin 1982, Reference McMullin, Douven and Horsten1995; Okasha Reference Okasha2011; Douglas Reference Douglas2014; Schindler Reference Schindler2018). They have also been discussed as “explanatory virtues” in the rich literature on the Inference to the Best Explanation (Harman Reference Harman1965; Barnes Reference Barnes1995; Lipton Reference Lipton2004; Schupbach Reference Schupbach2011). The literature on value judgments in science includes discussions of the role of theoretical virtues (Longino Reference Longino1996; Douglas Reference Douglas2009; Steel Reference Steel2010). Finally, theoretical virtues have played a role in the literature on laws of nature, and, in particular, on Lewis’ best systems analysis (Lewis Reference Lewis1973; Woodward Reference Woodward2014b).
When philosophers ponder whether particular virtues are desirable, truth-conducive, or whether there are any trade-offs between the virtues, they have hitherto relied mostly on historical case studies, studies of bits of contemporary scientific practice, and formal work.Footnote 1 However, alternative ways of grounding one’s philosophical theorizing have recently emerged in the form of experimental methods.
Experimental methods have been employed in several subfields of philosophy since the early 2000s (Knobe and Nichols Reference Knobe, Nichols and Edward2017). Although there were early attempts to use experimental methods in the philosophy of science (Stotz and Griffiths Reference Stotz and Griffiths2004; Stotz Reference Stotz2009), experimental methods have been embraced more widely only in more recent years (Schupbach Reference Schupbach2011; Waskan et al. Reference Waskan, Ian Harmon, Spino and Clevenger2014; Douven and Schupbach Reference Douven and Schupbach2015; Steel et al. Reference Steel, Gonnerman and O'Rourke2017; Chall et al. Reference Chall, King, Mättig and Stöltzner2021; Mättig and Stöltzner Reference Mättig and Stöltzner2019; Robinson et al. Reference Robinson, Gonnerman and O’Rourke2019; Beebe and Dellsén Reference Beebe and Dellsén2020; Wilkenfeld and Lombrozo Reference Wilkenfeld and Lombrozo2020; Mizrahi, Reference Mizrahiforthcoming). This paper is a contribution to this recent wave in experimental philosophy of science.
The basic premise of the empirical study that was conducted for this paper is that scientists can be an important source of information about scientific practice in general and about theoretical virtues in particular. Scientists are dealing with theories on a day-to-day basis, they must make decisions whether or not to adopt a theory and for what reasons. They should therefore be most familiar with what considerations are important for choosing those theories that will best advance the goals of science. Of course, scientists are not infallible as informants about their own practices, and not all of scientists’ practices are good ones. Yet, if scientists are not terribly deceived about their own practices (the onus to prove the contrary would clearly be on philosophers, I think) and if scientific practice can be assumed to largely advance our knowledge (which would be foolish to deny), then there is really no reason why philosophers should not want to use scientists as an important (further) source of information in their quest to understand science. In fact, I think it would be a problem for philosophers if scientists do not think what philosophers think they ought to think.Footnote 2
It goes without saying that philosophers often disagree about what one ought to think about certain questions. The study therefore did not make any strong assumptions about the questions it posed to its participants. Instead, the nature of this study is exploratory; that is, the study sought to explore the views of scientists on a number of philosophical questions concerning theoretical virtues in order to then compare those views to some of the dominant or influential views in the philosophical literature (and to the views expressed by philosophers in the same study).
The paper will proceed as follows. Section 2 will provide motivations for the major hypotheses that the study tested. Section 3 will present the method of the study, section 4 the results, and section 5 contains the discussion of the results and conclusions.
2. Motivations and hypotheses
In one of the first philosophical publications on the topic of theoretical virtues, Kuhn (Reference Kuhn1977) gave what is now considered a standard list of theoretical virtues: consistency (internal and external), empirical accuracy, scope (or unifying power), simplicity, and fertility. Since then, the scholarship dedicated specifically to understanding the nature and role of theoretical virtues has been rather sparse, but there are a number of clarifications worth making about Kuhn’s original list.
First, it makes good sense to distinguish internal and external consistency (see Douglas Reference Douglas2014; Schindler Reference Schindler2018). Internal consistency is the absence of any contradictions within a theory, whereas external consistency is the absence of contradictions to other (empirically established) theories. Second, Kuhn did not have much to say about what he meant by the fertility of a theory.Footnote 3 There are different ways of interpreting fertility, but perhaps the most popular interpretation is that a theory is fertile when it has novel predictive success, that is, when the theory successfully makes predictions about new phenomena (Schindler Reference Schindler2018).Footnote 4 In what follows, fertility will be understood in this way. Third, there are two senses of scope or unification. According to one conception, a unifying theory is just more empirically accurate than one that does not unify the relevant phenomena. However, this “deflationary” conception of unification, as one might call it, simply reduces to the virtue of accuracy. But there is also another conception of unification, namely, unification as identifying principles underlying seemingly disparate phenomena (Morrison Reference Morrison2000; Schindler Reference Schindler2018). For example, Maxwell’s theory identified light as a form of electromagnetic radiation, and in the 1970s, high energy physicists unified electromagnetic and weak interactions in a precursor of the standard model, even when that did not yet translate to immediate empirical success (Schindler Reference Schindler2014). It is this “non-deflationary” sense of unification that will be used in what follows.
Although Kuhn thought that the list of virtues he came up with was fairly widely accepted as a universal standard, he stated that “two men fully committed to the same list of criteria for choice may nevertheless reach different conclusions [as to what theory to choose]” (Kuhn Reference Kuhn1977, 358), either because they interpret the virtues differently or because they weigh them differently. Kuhn gave several examples where different virtue preferences would lead to different theory-choices for the same pairs of theories, and he also emphasized “external” factors (such as ideology) that may shape scientists’ preferences for some virtues (e.g., Kuhn attributed Copernicus’s preference for simplicity to his Neoplatonism). All of these considerations led Kuhn to conclude that there is “no algorithm” for rational theory choice.Footnote 5
Kuhn is probably right that there is a lot of diversity of theory-choice preferences in the scientific community (and its subcommunities). However, if, despite this diversity, there was an aggregate preference order to be found in the scientific community, meaning that the majority of scientists would prefer some virtues over others, then theory choice could be determinate after all. The present study sought to test this question empirically in the form of the following hypothesis:Footnote 6
H1 Scientists have an aggregate preference order for theoretical virtues. (In what follows, we will refer to this hypothesis as virtue_preferences).
One of Kuhn’s examples that illustrates that virtues can be interpreted rather differently from one scientist to another is the apparent simplicity of the Copernican system as compared to the apparent complexity of the Ptolemaic system. As Kuhn points out, there was a sense in which the two systems were equally simple. Copernicus also employed a number of epicycles to accommodate planetary motions accurately. Many other philosophers have made similar points to illustrate the vagueness of simplicity and concluded that simplicity is a very problematic theory-choice criterion (McAllister Reference McAllister1999; Douglas Reference Douglas2014; Achinstein Reference Achinstein2018). Given these issues with simplicity, one may expect scientists to rank simplicity fairly low when forced to rank the virtues. The current study therefore tested the following hypothesis:
H2 scientists rank simplicity lower than other theoretical virtues. (simplicity_ranking).
A central focus of many philosophical discussions of theoretical virtues is the question of whether virtues like simplicity and unifying power are epistemic virtues, that is, whether they are truth-conducive. Famously, this question has been emphatically denied by the antirealist van Fraassen (Reference van Fraassen1980, Reference van Fraassen1989), who argued that simplicity and unifying power in particular are merely pragmatic virtues, that is, virtues that concern the use of theories, but not their truth content.Footnote 7 This “pragmatist” view of theoretical virtues, as one may call it, has been embraced by many others, even outside the realism debate—particularly with regards to simplicity (Hacking Reference Hacking1982; Barnes Reference Barnes1995; Douglas Reference Douglas2009, Reference Douglas2014; Achinstein Reference Achinstein2018; Wray Reference Wray2018). Where do scientists stand on this matter? The study tested the following hypothesis:
H3 scientists view simplicity / unification as merely pragmatic virtues. (simpl/unific_pragmatic).
As already mentioned, simplicity is a notoriously vague virtue; it can come in many shapes and forms. Two broad forms of simplicity have been identified, namely, syntactic and ontological parsimony (Baker Reference Baker and Edward2016). When a theory is syntactically parsimonious, it employs relatively few theoretical principles in explaining the phenomena. When a theory is ontologically parsimonious, it employs a relatively small number of basic entities in explaining the phenomena. There are many further questions and complications, such as “how few principles or entities must a theory employ in order to be simple?” and “what (sub)form of simplicity is relevant for a particular instance of theory choice?” (Baker Reference Baker and Edward2016; Achinstein Reference Achinstein2018; Schindler Reference Schindler2018). The broad distinction between ontological and syntactic parsimony, however, is clear enough.
There is yet a further, more technical, sense of simplicity that has been discussed in the philosophy of science literature. It concerns simplicity in terms of parameter freedom (Forster and Sober Reference Forster and Sober1994; Sober Reference Sober2015). A free parameter in a theory is a parameter whose value is not determined theoretically but has to be “fixed” on the basis of experiments. A well-known example is the standard model in particle physics, which does not predict many of the masses of the particles that figure in it (Friederich et al. Reference Friederich, Harlander and Karaca2014). Limited parameter freedom is widely regarded as a virtue, since it is harder to accommodate the phenomena in an ad hoc fashion (Forster and Sober Reference Forster and Sober1994; Hitchcock and Sober Reference Hitchcock and Sober2004; Worrall Reference Worrall2014; Sober Reference Sober2015; Schindler Reference Schindler2018). And the fewer the free parameters, the “simpler” the theory or model.
The study sought to determine whether scientists agreed that these aforementioned features (i.e., syntactic and ontological parsimony, and paucity of free parameters) would make a theory simpler and whether they would prefer any one of these features by testing the following hypothesis:
H4 scientists consider a theory with few free parameters / few basic principles / few basic entities simple. (simplicity_nature).
Accounts of unification often appeal to simplicity. For example, Kitcher’s influential (and still dominating) account states that a theory is unifying if it allows one to derive a large number of phenomena from a small number of argument patterns (Kitcher Reference Kitcher1981; see Woodward Reference Woodward and Edward2014a). In other words, the fewer argument patterns a theory employs in accommodating the phenomena, the more unifying it is. To check whether there is this kind of reciprocal relationship between unification and simplicity, the following hypothesis was tested:
H5 scientists subscribe to the view that the more unifying the theory, the simpler it is. (simpl_unific).
Finally, a more overarching hypothesis that this study tested was whether the views articulated by philosophers would actually reflect the views by practitioners working in the fields studied by philosophers. As mentioned in the introduction of this paper, it would be problematic for philosophers to articulate views about science that are unrecognizable or even in conflict with the views held by scientists about their own practices. To test whether or not this is the case, the study tested the following hypothesis:
H6 there is disagreement between scientists and philosophers. (:sci_vs_phil).
The study tested the aforementioned hypotheses by consulting the judgment of natural scientists, social scientists, and scholars from history and philosophy of science.
3. Methods
3.1 Participants
Subjects were recruited from three larger academic fields: natural science, social science, and history and philosophy of science (HPS). The study was advertised via email lists (such as Philos-L, HOPOS, PSA), social media (Twitter and Facebook), and by contacting science department heads via email. The current study also used an email list from another study.Footnote 8 The online platform used to implement the study was Qualtrics. Participation was incentivized with an optional amazon.com voucher lottery. The return was as follows: 86 philosophers, 214 natural scientists, and 278 social scientists.Footnote 9 The average age of these three groups was 38.6, 47.8, and 49.2, respectively (see also Appendix 1). The percentage male in these groups was 80%, 74%, and 71%, respectively, reflecting the unfortunate gender imbalance in these fields. See Appendix 1 for a detailed overview of the disciplines of the participating scientists. Subjects spent 782 seconds (ca. 13 min) on average on the questionnaire.
3.2 Study design
The questionnaire consisted of three main blocks of questions (“theories” [5 questions], “simplicity and unification” [5 questions], and “intuitions” [3 questions]) and background questions before (2) and after (5) the three main blocks. The first two background questions related to the participants’ research field (B1) and research orientation (theoretical or empirical; B2). The questionnaire’s thirteen substantive questions related to the purpose of theories (Q1.1), the ranking of theoretical virtues (Q1.2), prediction and ad hoc-ness (Q1.3-1.5), the epistemicity of virtues (Q2.2 and Q2.3), the nature of virtues (Q2.1, Q2.4), aesthetics and virtues (Q2.5), and intuitions in theoretical and empirical discovery (Q3.1–Q3.3). The answer options / statements of each question in this block were randomized. The background questions at the end of the questionnaire related to the gender, academic position, age, language skill, and residence of the participants (B3–B7). This paper focusses on a discussion of a subset of the substantive questions, namely on Q1.2, Q2.1, Q2.4, Q2.2, and Q2.3, under the consideration of the information provided in the background questions. The results of the remaining questions will be published elsewhere. The full questionnaire that was presented to participants is accessible at https://osf.io/pdscx/.
3.3 Materials
The first background question of the study (B1) identified participants as natural scientists, social scientists, or as scholars working in the history and philosophy of science and asked natural and social scientists to specify their academic discipline (natural scientists: Physics, Biology, Chemistry, Geology; social scientists: Political Science, Psychology, Sociology, Economics, Anthropology, Linguistics). The second background question of the study (B2) asked scientists about the orientation of their research (either theoretical or empirical):
How would you best describe your own work?
-
• Most of my work is of a theoretical nature.
-
• Most of my work is of an empirical nature.
Philosophers received an analogous forced choice question about their research interests:
What part of science that you study is of most interest to you?
-
• The theoretical part of science.
-
• The empirical part of science.
Philosophers were also given the following instruction:
-
• In what follows, please consider the scientific field that you know best (physics, biology, chemistry, etc.), and answer accordingly.
The first substantive question of the study (Q1.1) asked subjects about their epistemic attitudes toward the best theories in their academic field:
What do you think is the case of the best theories in your scientific field?
-
• They do a good job of summarizing and organizing the facts.
-
• They help us to get ‘behind’ the phenomena to the underlying causes or laws.
-
• They set constraints that the phenomena must obey.
Participants were asked to register their agreement with each of these statements on a 5-point Likert scale (5 = strongly agree; 1 = strongly disagree). The first statement was associated with an instrumentalist view and the second statement with a realist view. Obviously, statements 1 and 2 are not exclusive; one can hold both that a theory organizes the facts and that it latches onto something real. Yet, a negative answer to the second statement would indicate that participants hold antirealist attitudes regardless of subjects’ answers to statement 1. The third statement was motivated by more recent literature on “explanation by constraint” according to which theories explain by putting necessary constraints on the laws of nature (Lange Reference Lange2017).
In order to find out whether scientists have an aggregate preference order for theoretical virtues (H1:virtue_preference), subjects were asked to rank the five (plus one) Kuhnian virtues according to their perceived importance in Q1.2 of the study.
Please rank the following statements in the order of importance to you (1 = most important, 2 = 2nd most important, etc.). You can change the position by dragging and dropping the statements.
“A good scientific theory should. .. ”
-
• … be accurate with regard to all the relevant data.
-
• … be simple.
-
• … be externally consistent: it shouldn’t contradict what other established theories say.
-
• … be internally consistent: it shouldn't contain contradictions.
-
• … accurately predict phenomena that weren’t known before the theory was devised.
-
• … unify phenomena that prior to the theory appeared to be unconnected. Footnote 10
In Q2.1, the study sought to clarify whether subjects agreed that a theory with certain features is simple (H4:simplicity_nature). Subjects had to indicate their agreement on a 5-point Likert scale for each of the following three statements:
What makes a theory simple for you? A theory is simple if. . .
-
• . . . it has a small number of free parameters.
-
• . . . it uses a small number of basic principles.
-
• . . . the number of basic things which it postulates (e.g., particles) is small.
Questions Q2.2 and Q2.3 of the study explored whether subjects perceived unification and simplicity as epistemic or merely pragmatic (i.e., H3:simpl/unific_pragmatic). The first question (Q2.2) determined whether subjects’ confidence in a theory’s correctness would be increased if a theory was simple or unified:
Please indicate your (dis-)agreement with the following statements:
“My confidence that a theory is correct is increased (everything else being equal), when …”
-
• . . . a theory unifies phenomena previously thought to be unrelated.
-
• . . . a theory is simple.
Subjects had to indicate their agreement on a 5-point Likert scale for each of the two statements. The rationale was that if subjects agreed with these statements, this would indicate that they viewed unification and simplicity as epistemic virtues, that is, virtues that make the theory more likely to be true.
The second question regarding epistemicity (Q2.3) determined whether subjects would find it acceptable if a theory was conflicting with the data when it had other (philosophically controversial) theoretical virtues:
Please indicate your (dis-)agreement with the following statements:
“It’s acceptable for a theory to be in conflict with some of the relevant data…”
• . . . when a theory unifies phenomena previously thought to be unrelated.
• . . . when a theory is simple.
• It’s not acceptable for a theory to be in conflict with any of the relevant data.
Subjects had to indicate their agreement on a 5-point Likert scale for each of the three statements. The rationale behind this second question (Q2.3) was that if subjects believed that unifying power or simplicity were epistemic virtues, they should be more willing to accept conflicts between theories with these virtues and the data, because the virtues would then give them reasons to believe in the theory despite the theory not accommodating all the data (van Fraassen Reference van Fraassen1980; Schindler Reference Schindler2018). On the other hand, if the virtues were merely pragmatic, they should not be so inclined. Question Q2.3 also contained an optional subquestion, where subjects were invited to state further reasons or circumstances under which conflicts with the data were acceptable:
It’s acceptable for a theory to be in conflict with some of the relevant data, when … (optional)
Finally, question Q2.4 was designed to probe the relationship between unification and simplicity:
-
• Do you agree with the following statement? The more unifying a theory, the simpler it is.
Subjects had to indicate their agreement on a 5-point Likert scale. These were all of the questions used in the current study. Before we move on to the next section, where I will present the results of the study, I want to address a more general objection to a study like this one.
3.4 An objection to the study design
There is a view of theory-choice according to which theory-choice is a highly contextual matter: there are no theory-properties that could be regarded as universally “virtuous” or worth striving for in each and every context of theory choice. Instead, the way in which scientists rank virtues differs from one context to the other. On such a contextualist view, one may be concerned that it would not make much sense to ask scientists about their preferences in an unqualified way.Footnote 11
I am sympathetic to the contextualist view. However, I do not think it follows from this view that one cannot ask scientists about their overall preferences: even if scientists’ preferences depended on the particular theory-choice situations, all that is required by the current study is that scientists have a preference for certain virtues in most contexts, or for certain virtues in the most important contexts, or perhaps even in only certain ideal circumstances (after all, we are talking about desirable theory-properties). If even that was not the case, and scientists’ preferences were so context-dependent that it would be impossible for them to express any general preferences, one would expect—in a study like this one—that scientists neither agree nor disagree with the study prompts. But that is not borne out by the results, as we shall soon see.
4. Results
This section presents the results of the study. It is structured as follows. Sections 4.1 and 4.2 present the results of questions B2 and Q1.1, respectively. The data from these questions provided background information that was used to further analyze some of the results of the main questions of the survey. Sections 4.3 to 4.5 present the results pertaining to hypotheses H1—H6 in the following order: hypotheses H1:virtue_preferences and H2:simplicity_ranking are discussed in section 4.3, H3:simpl/unific_pragmatic in section 4.4, H4:simplicity_nature and H5:simpl_unific in section 4.5. H6:sci_vs_phil will be assessed in each section.
4.1 Professional orientation
The professional orientation of subjects (B2) was as follows (theoretical orientation vs. empirical orientation): Nat (36% vs. 66%), Soc (35% vs. 65%), and HPS (87% vs. 13%). It is quite apparent that the participants from both the natural and the social sciences have predominantly empirical research interests (about two thirds each). Among philosophers, there was a surprising dominance of interest in the theoretical (rather than the empirical) part of science.
4.2 Realist attitudes
In Q1.1, subjects were asked about their realist attitudes towards the best theories in their field. On average, all three groups indicated agreement (at about an equal level) with statement 1 (organization of facts): HPS = 3.92 (SD = 0.98); Nat = 4.23 (SD = 0.9); Soc = 4.05 (SD = 0.91). There were similar levels of agreements with the “realist” statement 2: HPS = 3.91 (SD = 1.23), Nat = 4.46 (SD = 0.84), Soc = 4.35 (SD = 0.98). The percentage of realist vs. antirealists was: Nat (91% vs. 6%), Soc (87% vs. 5%), HPS (76% vs. 7%).Footnote 12
This finding accords with what was found by Beebe and Dellsén (Reference Beebe and Dellsén2020) for natural scientists.Footnote 13 In contrast to the study by Beebe and Dellsén, the current study also found strong realist attitudes among social scientists (and among philosophers). This may have something to do with the particular form of the question that the two studies posed.Footnote 14
Regarding statement 3, that is, the setting of necessary constraints by theories, only the natural scientists agreed: Nat = 3.63 (SD = 1.38), Soc = 3.09 (SD = 1.4), HPS = 3.07 (SD = 1.35).Footnote 15 These results were not used in this paper.
4.3 Theoretical Virtues: preferences
In Q1.2 subjects had to rank theoretical virtues according to their relative importance. Based on subjects’ responses, the Condorcet winner was determined. The findings are summarized in table 1, with the ranks the virtues were assigned on the basis of the number of wins against the other virtues.
Results of a Condorcet winner analysis regarding the ranking of theoretical virtues. The number of pair-wise wins of a particular virtue against other virtues are in bracket;. (*) indicates a tie
| HPS | Nat | Soc | |
|---|---|---|---|
| 1st | internally consistent (5) | internally consistent (5) | internally consistent (5) |
| 2nd | accurate (4) | accurate (4) | accurate (4) |
| 3rd | predictive (3) | predictive (3) | predictive (2)* |
| 4 th | unifying (2) | unifying (2) | unifying (2)* |
| 5 th | externally consistent (1) | simple (1) | simple (1) |
| 6 th | simple (0) | externally consistent (0) | externally consistent (0) |
The results show that all three groups ranked internal consistency higher than any other virtue, followed by accuracy and predictive success. Unifying power was ranked fourth by philosophers and natural scientists, and joint third by social scientists. Subjects in all three groups ranked simplicity low (HPS: lowest, Soc and Nat: second lowest). The detailed win statistics show that these preference orders are quite unambiguous (see appendix 2 at https://osf.io/pdscx/).
From these results, we can conclude that there does exist an aggregate preference order for theoretical virtues among scientists (confirming H1:virtue_preferences). Furthermore, there is also clearly strong agreement between scientists and philosophers (disconfirming H6:sci_vs_phil). Given that scientists ranked simplicity low, H2:simplicity_ranking is confirmed as well.
4.3.1 Realism and ranking
The study investigated whether subjects with a “realist” attitude (as determined by Q1.1; see section 4.2) would be more likely to rank simplicity or unification higher than other subjects—perhaps because realists were more likely to view those virtues as epistemic. Order logit models were employed to test whether a higher degree of agreement (on the 5-point Likert scale) would correlate with simplicity and unification being ranked higher. That was true only for social scientists and the virtue of simplicity (p = 0.007; see appendix 3 at https://osf.io/pdscx/). However, the number of subjects exhibiting an antirealist attitude was very low (see section 4.3.1), affecting the power of the test. No robust conclusions could therefore be drawn from these results.
4.3.2 Professional orientation and ranking
In one background question (B2), subjects were asked about their professional orientation (theoretical or empirical, see section 4.1). Having a theoretical orientation in one’s research, one may speculate, may make it more likely to rank unification or simplicity higher than having an empirical orientation: theoreticians may find these “non-empirical” virtues more important than subjects working with more empirical problems. Order logit models were therefore constructed to test whether theorists would assign simplicity and unification a higher rank of importance than the experimentalists. However, that was not the case.
4.4 Epistemicity
Questions Q2.2 and Q2.3 of the study explored whether subjects perceived the philosophically controversial virtues of unification and simplicity as epistemic or merely pragmatic (this concerns hypothesis H3:simpl/unific_pragmatic).
4.4.1 Epistemicity: confidence
The first epistemological question (Q2.2) determined whether subjects’ confidence in a theory’s correctness would be increased if a theory was simple or unified. The results of this question are depicted in figure 1.
Responses to question Q2.2. Subjects’ views regarding whether their confidence in a theory’s correctness was raised by the theory unifying the phenomena or by being simple. The y-axis represents Likert scale ratings and the x-axis the two statements (unifying, simple). Error bars are standard errors. The lowest point of the y-axis corresponds to the lowest point on the Likert scale (=1).
Regarding the first statement of Q2.2 (unification), subjects’ confidence in a theory’s correctness is robustly increased for all three groups (HPS, Nat, Soc) when a theory unifies hitherto unrelated phenomena. The means on a 5-point Likert scale for the three groups are: HPS = 4.15 (SD = 0.86), Nat = 4.34 (SD = 0.77), Soc = 4.09 (SD = 0.9). A one-way ANOVA showed that there were significant differences between group means: F(2,577) = 5.399, p = 0.005. A post hoc Tukey test showed that the difference between the mean of the Nat group and the (lower) mean of the Soc group was significant (p = 0.003). In other words, natural scientists were significantly more likely than social scientists to agree that unifying power increases their confidence in a theory’s correctness. However, the effect size was small (Cohen’s d = 0.3).Footnote 16 There was no significant difference between philosophers and either science group.
As to the second statement in Q2.2 (simplicity), there was neither strong agreement nor disagreement for any of the three groups. The means for the three groups were: HPS = 2.97 (SD = 1.05), Nat = 3.36 (SD = 1.09), Soc = 3.29 (SD = 1.03). T-tests showed that the means of the Nat and the Soc group, but not the HPS group, were statistically different from the mid-point of the scale (HPS: t(85) = 0.03, p = 0.75, Nat: t(213) = 0.32, p = 0.000, Soc: t(279) = 0.29, p = 0.000. Both the means of the Nat and the Soc group were toward the higher end of the scale. A one-way ANOVA showed that there were significant differences between group means: F(2,577) = 4.354, p = 0.013. A post hoc Tukey test showed that the mean of the HPS group differed significantly from both (higher) means of the Nat and Soc groups (p = 0.011 and p = 0.032, respectively). The effect sizes were moderate and small (Cohen’s d = 0.36 for HPS-Nat and 0.32 for HPS-Soc). There was no significant difference between natural and social scientists. Hence, both natural and social scientists tend to agree that a theory being simple raises their confidence. Philosophers neither agreed nor disagreed with this view. See also figure 2.
Detailed plot for Q2.2, statement 2 (simplicity). There was a larger group of scientists who (strongly) agreed that the simplicity of the theory would raise their confidence. Nat: 50% agree vs. 20% disagree, Soc: 48% vs. 22%. Philosophers were more evenly divided: 33% vs. 30%.
In sum, there is evidence that subjects (from all groups) perceive unification as an epistemic virtue. There is some (but much weaker) evidence that natural and social scientists perceive simplicity as an epistemic virtue. Hence, H3:simpl/unific_pragmatic can be strongly rejected with regard to unification and, to some extent, also with regard to simplicity. H6:sci_vs_phil can be disconfirmed with regards to the question of whether unification raises one’s confidence but should be viewed as confirmed with regards to the question of whether simplicity raises one’s confidence.
4.4.2 Epistemicity: conflict
The second epistemological question (Q2.3) determined whether subjects believed that conflicts with the data are acceptable when the theory is simple or unifying. The results are depicted in figure 3.
Responses to Q2.3. Subjects’ views regarding the acceptability of conflict between theory and data, if the theory is unifying (left) or simple (middle), and their views regarding the unacceptability of conflicts between theory and data (right). The y-axis represents Likert scale ratings and the x-axis the three statements (unifying, simple, and unacceptable). The error bars are standard errors.
The means for the first statement in Q2.3 are (unification): HPS = 3.44 (SD = 1.12), Nat = 3.08 (SD = 1.24), Soc = 3.4 (SD = 1.11). T-tests determined that the means of the HPS and the social science group responses (but not the natural science responses) were statistically different from the midpoint of the scale (HPS: t(85) = 3.65, p = 0.000, Nat: t(213) = 0.06, p = 0.348, Soc: t(279) = 0.36, p = 0.000. One-way ANOVAs showed that there were statistically significant mean differences between the three groups: F(2,577) = 5.442, p = 0.005. Post hoc Tukey tests determined that the means of both the HPS and the Soc group were statistically different from the (lower) mean of the Nat group (p = 0.0039 and 0.008, respectively). The effect sizes were small (Cohen’s d = 0.27 for Soc-Nat and 0.31 for HPS-Nat). Hence, philosophers and social scientists, but not natural scientists, tend to agree with the statement that conflicts with the data are acceptable when the theory is unifying. The difference between the means of the HPS and the Soc group was not significant.
It is interesting to note that there was a quite sizable number of subjects in all three groups (including natural scientists) who agreed that conflicts between theory and data were acceptable when the theory is unifying (see figure 4).
Detailed plot for Q2.3, statement 1 (unification). A substantive number of subjects agrees that it would be acceptable for a unifying theory to conflict with the data: HPS = 63% agree, 26% disagree, Nat = 50% agree, 34% disagree, Soc = 59% agree, 24% disagree.
The means for the second statement in Q2.3 (simplicity) are: HPS = 2.73 (SD = 1.24), Nat = 2.54 (SD = 1.28), Soc = 3.21 (SD = 1.2). T-tests determined that each of these means is statistically different from the midpoint of the scale (HPS: t(85) = 0.22, p = 0.049, Nat: t(213) = 0.36, p = 0.000, and Soc: t(279) = 0,18, p = 0.003). One-way ANOVAs showed that there were statistically significant mean differences between the three groups: F(2,577) = 19.091, p = 0.000. Post hoc Tukey tests determined that there was no statistically significant difference between the means of the HPS and the Nat group (p = 0.431), but there was a difference between the means of those groups and the (higher) mean of the Soc group (p = 0.000 for Soc-Nat and p = 0.005 for Soc-HPS). The effect sizes were moderate (Cohen’s d = 0.39 for Soc-HPS and Cohen’s d = 0.55 for Soc-Nat). Hence, philosophers and natural scientists tend to disagree, and social scientists tend to agree, with the statement that conflicts with the data are acceptable when the theory is simple. The difference between the means of the HPS and the Nat group was not significant.
The means for the third statement in Q2.3 (conflict not acceptable) are: HPS = 2.64 (SD = 1.36), Nat = 3.1 (SD = 1.35), Soc = 2.55 (SD = 1.32). T-tests determined that the means of the HPS and the Soc group (but not the Nat group) differed significantly from the midpoint of the scale (HPS: t(85) = 0.26, p = 0.016, Nat: t(213) = 0.07, p = 0.289, Soc: t(279) = 0.34, p = 0.000). One-way ANOVAs showed that there were statistically significant mean differences between the three groups: F(2,577) = 10.476, p = 0.000. Post hoc Tukey tests determined that the means of the HPS and the Soc group were statistically different from the (higher, midpoint) mean of the Nat group (p = 0.021 and p = 0.000, respectively). The effect sizes were small for HPS-Nat (Cohen’s d = 0.34) and moderate for Soc-Nat (Cohen’s d = 0.41). Hence, philosophers and social scientists, but not natural scientists, tend to disagree with the statement that any data conflict is unacceptable. The difference between the means of the HPS and the Soc group was not significant.
It is interesting to note that the detailed response profile regarding the third statement revealed that respondents fell into two larger groups: those who, in general, reject that conflicts between theory and the data are unacceptable (overall the majority) and those who agree that any conflicts between theory and data are unacceptable. There was only a small minority who neither agreed nor disagreed. See figure 5.
Detailed responses to Q2.3, statement 3. There are two camps with regard to the question of whether conflicts between theories and the data are unacceptable: those who disagree (HPS: 58%, Nat: 44%, Soc: 60%) and those who agree (HPS: 37%, Nat: 48%, Soc: 29%). Very few subjects neither agreed nor disagreed.
In sum, the results for question Q2.2 indicate that unifying power tends to be perceived as an epistemic virtue by social scientists and at least a large subgroup of natural scientists. H3:simpl/unific_pragmatic is therefore to be rejected with regards to unification. The results of Q2.2 are less clear for simplicity: natural scientists tend not to accept data conflicts when the theory is simple, whereas social scientists tend to accept them. Philosophers, interestingly, tend to agree with social scientists that data conflicts are acceptable for theories that unify, and they tend to agree with natural scientists that data conflicts are not acceptable for theories that are simple.
A substantial number of subjects from all three groups answered the optional question Q2.3b (other circumstances in which conflicts are acceptable?): HPS (29%), Nat (37%), Soc (33%). The responses were coded manually. The three most frequent responses given were (in that order): D = the data may turn out to be unreliable or untrustworthy / there is reason to doubt data (30%), I = all theories are idealizations / can never explain all the data / reality is too complex (17%), and E = theory successfully explains important or most facts, and fails to accommodate only less important or fewer facts (12%). No other theoretical virtue was mentioned for which data conflicts may be acceptable. See appendix 4 at https://osf.io/pdscx/ for all responses.
4.4.3 Epistemicity and realism
An order logit model was employed to test whether a higher degree of commitment to the realist statement in Q1.1 (on the 5-point Likert scale) correlates with a higher degree of agreement with the statement of Q2.2 or Q2.3. The results are as follows (see appendix 5 for details).
Regarding Q2.2 (confidence raising), Soc was the only group for which a higher degree of agreement with the realist statement correlates with a higher degree of agreement with the statement that unifying power raises the confidence in a theory (p = 0.000). Nat was the only group for which a higher degree of agreement with the realist statement correlates with a higher degree of agreement with the statement that simplicity raises one’s confidence in a theory (p = 0.003).
Regarding Q2.3 (conflict with data), Soc was the only group for which a higher degree of agreement with the realist statement correlates with a higher degree of agreement with the statement that data conflicts are acceptable when the theory is unifying (p = 0.004). There were no other significant results, also regarding the statements that data conflicts are acceptable when the theory is simple, and with regard to the statement that data conflicts are not at all acceptable.
4.5 Nature of virtues
There were two questions in the study asking subjects about the nature of virtues (relating to H4:simplicity_nature). In one of these questions, the focus was on the nature of simplicity (Q2.1), and the other asked about the relationship between simplicity and unification (Q2.5).
4.5.1 Simplicity
The results for question Q2.1 are depicted in figure 6.
Responses to Q2.1: subjects’ views regarding the nature of simplicity. The y-axis represents Likert scale ratings and the x-axis the three statements (few free parameters, few principles, few things). Error bars are standard errors.
The means of agreement for the first statement (simplicity as parameter freedom) were: HPS = 3.45 (SD = 0.03), Nat = 3.99 (SD = 0.91), and Soc = 3.61 (SD = 0.88). They are all significantly different from the midpoint of the scale: HPS: t(85) = 0.49, p = 0.000, Nat: t(213) = 1.08, p = 0.000, Soc: t(279) = 0.7, p = 0.000. A one-way ANOVA determined that the means of the three groups were different: F(2,577) = 15.051, p = 0.000. Post hoc Tuckey tests showed that the means of the Nat group differed significantly from the (lower) means of both the HPS and the Soc group (p = 0.000, each). The effect sizes were moderate (Cohen’s d = 0.58 for Nat-HPS and Cohen’s d = 0.41 for Nat-Soc). There was no statistical difference between the means of the HPS and the Soc group (p = 0.316).
There was broad agreement with the second statement (simplicity as few basic principles) for each of the three groups. The means were: HPS = 4.14 (SD = 0.81), Nat = 4.08 (SD = 0.92), and Soc = 4.18 (SD = 0.77). A one-way ANOVA did not reveal any significant differences in the group means: F(2,577) = 0.844, p = 0.431.
The means for the third statement (simplicity as paucity of postulated entities) were: HPS = 3.41 (SD = 1.3), Nat = 3.4 (SD = 1.16), and Soc = 3.49 (SD = 0.98). They are all statistically significant from the midpoint of the scale: HPS: t(85) = 0.31, p = 0.005, Nat: t(213) = 0.35, p = 0.000, Soc: t(279) = 0.5, p = 0.000. In other words, all groups tended to agree (at a low level) that theories are simple when they postulate few entities. A one-way ANOVA did not reveal any significant group mean differences: F(2,577) = 0.443, p = 0.642.
In sum, scientists agreed that simplicity implies a small number of free parameters, a small number of principles, and a small number of postulated entities (even though the level of agreement differed). H4:simplicity_nature is confirmed. It is noteworthy that the level of agreement among all three groups was higher for the second statement (syntactic parsimony) than it was for the third statement (ontological parsimony).
The agreement between philosophers and scientists was broad: natural scientists differ significantly from philosophers only with regard to parameter freedom. H6:sci_vs_phil is thus mostly disconfirmed.
4.5.2 Simplicity and unification
Regarding Q2.4, namely, the question of whether subjects agreed that unification implies simplicity, the mean results were: HPS = 2.78 (SD = 1.12), Nat = 2.95 (SD = 1.04), and Soc = 2.79 (SD = 0.92). Only the mean of the Nat group was not statistically different from the midpoint of the 5-point Likert scale: Nat: t(213) = 0.05, p = 0.471, Soc: t(279) = 0.23, p = 0.000, HPS: t(84) = 1.19, p = 0.000. That is, social scientists and philosophers tended to slightly disagree that the more unifying a theory, the simpler it is. Natural scientists neither agreed nor disagreed. H5:simpl_unific is thus disconfirmed. A one-way ANOVA detected no significant differences between the means of the three groups: F(2,577) = 1.724, p = 0.179. H6:sci_vs_phil is thereby disconfirmed.
5. Discussion and conclusion
This paper presented results of an exploratory study into natural and social scientists’ views regarding theorizing and theoretical virtues. It compared these views to views held by researchers working in the field of “history and philosophy of science.”
The study asked scientists to sort the theoretical virtues in the order of their importance and produced a Condorcet winner analysis of the results. It was found among all three groups that internal consistency had the most wins against the other virtues, accuracy the second most wins, predictive success the third most wins, and unifying power the fourth most wins (for social scientists, unification had as many wins as predictive success). H1:virtue_preferences could therefore be confirmed. Simplicity, which is often considered the most problematic theoretical virtue in the philosophical literature, ranked low: among natural and social scientists, it ranked fifth (out of six ranks) with only one win. H2:simplicity_ranking is therefore confirmed. External consistency, however, ranked even lower (sixth with no wins). Given the low standing of simplicity in the philosophical literature, it was not surprising that simplicity had the fewest wins among philosophers (none).
What do these results mean for discussions of theory choice? There is a view—most prominently defended by Kuhn—that theory choice is a very subjective matter, and scientists tend to have diverse theory-choice preferences that make theory choice an indeterminate matter. However, the results produced by this study show that this view may be too pessimistic: scientists seem to prefer theories that are consistent, accurate, produce predictive success, unify the phenomena, are simple, and externally consistent (in this order).Footnote 17 Given this agreement, theory choice—contra Kuhn—does seem to be a determinate matter.
In order to test whether the lower ranking of simplicity had anything to do with how its epistemic status was viewed by scientists, the study tested whether scientists’ realist commitment would positively affect their ranking. It turned out that only social scientists with realist commitments ranked simplicity significantly higher than their colleagues. There were no differences in the ranking of simplicity or unification by “realists” among natural scientists and philosophers. However, given the low number of “non-realists” and the resulting ceiling effect, all of these results should be treated with caution.
Two questions in the study specifically and more directly sought to shed light on the potential epistemicity of simplicity and unifying power: a question relating to whether these virtues would raise subjects’ confidence in a theory and a question relating to whether subjects would find it acceptable that theories with such virtues conflict with the data. The rationale for the latter question was that if these theoretical virtues are merely pragmatic, then data conflicts should not be acceptable, because the virtues would then give scientists no reason to believe that the theory might be true despite the apparent conflict with the data. On the other hand, if these theoretical virtues were not just pragmatic but epistemic, then data conflicts should be acceptable, because the virtues would then give scientists reason to believe that the relevant theories are correct, despite conflicting with data (see van Fraassen Reference van Fraassen1980; Schindler Reference Schindler2018).
Regarding the first question, subjects from all three groups stated that their confidence in a theory’s correctness is raised when the theory is unifying (everything else being equal). Both social and natural scientists also agreed that simplicity raises their confidence in a theory (everything else being equal), albeit at a lower level of agreement. Philosophers did not share this view. This is a first point of significant disagreement between philosophers and scientists (confirming H6:sci_vs_phil).
Regarding the second question, social scientists and philosophers tended to agree that data conflicts are acceptable when the theory is unifying. Among all groups (even natural scientists), there was a sizable subgroup who even strongly agreed with this view. Natural scientists (and philosophers) tended to disagree (at a fairly low level) that data conflicts are acceptable when the theory is simple. Social scientists, in contrast, tended to agree with this view.
Realist commitments among social scientists correlate with viewing unification as confidence raising and with accepting conflicts with the data. Realist commitments among natural scientists correlate only with the view that simplicity raises one’s confidence in a theory. There were no significant results for philosophers. However, again, given the low number of subjects with non-realist commitment and the resulting ceiling effect, these results must be taken with a grain of salt.
Combining the results from the “confidence,” “conflict,” and “realist” questions, there is robust evidence that scientists view unifying power as an epistemic virtue, disconfirming H3:simpl/unific_pragmatic. Philosophers agreed with this view (in contrast to H6). Regarding simplicity, the picture is not quite as clear. The responses to the confidence question seem to indicate that both social and natural scientists view simplicity as an epistemic virtue, but the conflict question revealed that only social scientists tend to accept conflicts between simple theories and the data. Natural scientists tended to disagree that such conflicts would be acceptable. Clearly, social scientists have a more robust view of simplicity as an epistemic virtue than natural scientists. One can therefore conclude that natural scientists view simplicity also as an epistemic virtue (since they think simplicity raises confidence), but that they do not view it as strongly an epistemic virtue as unification (since they disagree that data conflicts with simple theories are acceptable). So overall, there is evidence for the disconfirmation of H3:simpl/unific_pragmatic also with regards to simplicity.
Philosophers were clearly the group most skeptical about the epistemicity of simplicity: they neither agreed nor disagreed that simplicity would raise their confidence in a theory, and they disagreed that conflicts with the data would be acceptable. Philosophers’ skepticism was thus at odds with social scientists on both counts (confidence and conflict) and at odds with natural scientists with regards to the confidence question. H6 is thus mostly confirmed.
Philosophers may want to draw the following lessons from these results. First, this study provides strong evidence that one of the most prominent positions in the realism debate is mistaken with regards to unifying power not being an epistemic virtue (van Fraassen Reference van Fraassen1980). The study even provides some evidence that scientists view simplicity as an epistemic virtue, which is at odds with large swaths of the literature (McAllister Reference McAllister1999; Douglas Reference Douglas2014; Achinstein Reference Achinstein2018; Wray Reference Wray2018). Arguments concerning, for example, discussions about the underdetermination of theories by evidence, which have presumed that theoretical virtues such as simplicity and unifying power are not epistemic, may have to be re-assessed (but see Tulodziecki Reference Tulodziecki2012). Second, philosophers have assumed that the status of a theoretical virtue as an epistemic or a pragmatic virtue holds across the board for all sciences. However, this assumption may be wrong. As this study shows regarding simplicity, social scientists do indeed have different views than natural scientists. There may even be further, more fine-grained differences on the level of scientific disciplines. Philosophers in the future may want to study the reasons for this difference. Third, philosophers have hitherto tended to view theoretical virtues as either epistemic or pragmatic. However, the results suggest that a more differentiated view of the potential epistemicity of theoretical virtues might be called for. On such a view, theoretical virtues are not necessarily either epistemic or just pragmatic, but they may be epistemic to one degree or another. For example, simplicity may also be an epistemic virtue (at least in some sciences), but not as strong an epistemic virtue as unifying power.Footnote 18
With regards to the nature of simplicity, the study found evidence that natural and social scientists agree that theories are simple if they: (i) have few free parameters, (ii) postulate few basic principles (corresponding to syntactic parsimony), or (iii) postulate few basic entities (corresponding to ontological parsimony). H4:simplicity_nature is therefore confirmed (for all three forms of simplicity). Natural scientists most strongly agreed with (i), perhaps owing to the higher degree of formalization of their theories. It is also noteworthy that the agreement among natural and social scientists for (ii) was much more pronounced than for (iii). Since much of the philosophical literature has hitherto focused on ontological parsimony (Nolan Reference Nolan1997; Baker Reference Baker2003; Jansson and Tallant Reference Jansson and Tallant2017), it may be time for philosophers to pay more attention to syntactic parsimony.
Interestingly, the philosophers of this study showed similar levels of agreement as scientists both with regards to ontological and syntactic parsimony (disconfirming H6). All the more reason, perhaps, for philosophers to turn their attention to syntactic parsimony.
The study also investigated the relationship between unification and simplicity. On some leading accounts of unification, the more unifying a theory, the simpler it is (in terms of basic principles, in particular). However, the study found no support for this view among scientists. Natural scientists were neutral on this issue, and social scientists slightly disagreed. Philosophers also slightly disagreed that unifying power and simplicity are related in this way (contra H6). At least as far as social scientists (and philosophers) are concerned, H5:simpl_unific is therefore disconfirmed: unifying power and simplicity may actually be independent virtues. Philosophers of explanation may therefore have to think of new ways of explicating the unifying power of theories; ways, that is, that do not appeal to the simplicity of theories.
In sum, the study has shown that some philosophical accounts of theory-choice are too pessimistic: there is an overall preference ranking for the standard theoretical virtues, rending theory-choice a much more determinate matter than previously assumed by some. Scientists view unifying power as a genuinely epistemic virtue, and they even view simplicity at least as a weakly epistemic virtue. Scientists prefer syntactic parsimony to ontological parsimony, and unifying power and simplicity are independent virtues. Philosophers can take comfort from several points of agreement with scientists. The most significant degree of disagreement probably concerns the question of whether or not simplicity is an epistemic virtue. This article suggested ways to amend this disagreement (e.g., allowing for degrees of epistemicity).
Overall, this study has provided valuable evidence for scientists’ views on theoretical virtues in theory choice and on the question of the epistemicity of theoretical virtues. It should therefore provide a better foundation for philosophical discussions of these issues.
Acknowledgments
I thank Christoffer L. Olesen and Florence So for doing the statistical analyses for this paper and for producing the figures. The paper profited from the extensive and constructive feedback of several anonymous referees for this journal. I also thank the audiences at the Experimental Philosophy of Science workshop at Aarhus University in 2019, the Bristol Centre for Science and Philosophy in 2021, the PSA 2021 poster forum, and the First European Experimental Philosophy conference in 2021 for their comments. The research for this paper was kindly supported by a DFF Sapere Aude grant (4180-00071).
