1. Introduction

Experimental philosophy has made great strides in several areas, including epistemology and ethics, but one area that has received less attention is philosophy of science. Of the scant experimental philosophy of science produced so far, the focus has been primarily on specific scientific concepts, entities, or properties, such as genes (Stotz, Griffiths, and Knight Reference Stotz, Griffiths and Knight2004; Griffiths and Stotz Reference Griffiths and Stotz2008) or innateness (Griffiths, Machery, and Linquist Reference Griffiths, Machery and Linquist2009; Knobe and Samuels Reference Knobe and Samuels2013; for a review, see Machery [Reference Machery, Sytsma and Buckwalter2016]). Already a clarion call has sounded for more experimental work in philosophy of science (Stotz Reference Stotz2009a, Reference Stotz2009b; Weinberg and Crowley Reference Weinberg and Crowley2009; Nagatsu Reference Nagatsu2013; Machery Reference Machery, Sytsma and Buckwalter2016). We concur and also believe that the scope of this work should be extended. To these ends, we endeavor to develop and extend experimental philosophy of science by engaging in philosophical cartography. Using data gathered by the Toolbox Dialogue Initiative (TDI),Footnote ¹ we attempt to illuminate some of the general contours of a map of the philosophical attitudes of practicing scientists and then use this map to contribute to current work in the philosophy of science on interdisciplinary integration.

An experimentally informed map of philosophical attitudes of scientists is philosophically interesting for several reasons. First, distinguishing between the natural and social sciences has been a topic of discussion in philosophy of science for decades. Hacking (Reference Hacking, Galison and Stump1996), for example, offers a complex distinction between the two, claiming that the classificatory practices of the social sciences interact with the world in a way not found in the natural sciences. Call something an electron, and it makes no differences to it; diagnosing people with narcissism can change how they behave. (For more on Hacking’s proposal, see Martínez [Reference Martínez2009].) Developing a map of the philosophical attitudes of scientists can inform projects like Hacking’s. For instance, if the classificatory practices of the social and natural sciences work as Hacking proposes, we should expect social scientists to have more antirealist sympathies than natural scientists. This is a testable prediction. If it were to bear out, and thus make an appearance on our collective map of scientists’ philosophical attitudes, Hacking’s proposal would gain a consideration in its favor. In this article, we develop the beginnings of such a map. We do so experimentally by basing our claims on a survey of scientists’ own reports of their attitudes regarding various issues, rather than by appeal to our own intuitions or a philosophical analysis of the literatures of various sciences. This survey-driven approach allows us to capture heterogeneity across different areas of sciences, something that can elude traditional intuition- and literature-based methods in philosophy of science (Faust and Meehl Reference Faust and Meehl2002; Griffiths and Stotz Reference Griffiths and Stotz2008; Machery and Cohen Reference Machery and Cohen2012; Steel, Gonnerman, and O’Rourke Reference Steel, Gonnerman and O’Rourke2017).Footnote ²

Second, experimental examination of scientists’ philosophical attitudes opens a new avenue into the existing discussion of interdisciplinary integration in philosophy of science. One prominent focus of philosophers of science has been on integration as a relation between various scientific objects, such as theories, fields, or specialties (Darden and Maull Reference Darden and Maull1977; Bechtel Reference Bechtel1993; Grantham Reference Grantham2004; Gerson Reference Gerson2013; Griesemer Reference Griesemer2013). Nagel’s (Reference Nagel1961) reductionist account of scientific theories can be seen as an early entry into this literature. More recently, as skepticism has grown about this reductive project, some (Mitchell et al. Reference Mitchell, Daston, Gigerenzer, Sesardic, Sloep, Weingart, Mitchell, Richerson and Maasen1997; Love Reference Love2008; Brigandt Reference Brigandt2013) have proposed a nonreductive conception of integration according to which the ideas and explanations of various disciplines are integrated “to yield an overall explanation of a complex phenomenon” (Brigandt Reference Brigandt2010, 297).

Drawing inputs from different disciplines provides reason to believe that integration of this sort is distinctly philosophical in character (O’Rourke, Crowley, and Gonnerman Reference O’Rourke, Crowley and Gonnerman2016). Disciplines engage with the world in different ways, foregrounding certain problems while leaving others in the background. This sort of engagement can be modeled in terms of a discipline’s research worldview, composed of a system of core beliefs, values, and practices (O’Rourke and Crowley Reference O’Rourke and Crowley2013). These worldviews differ in various ways, such as in the meaning of terms used to articulate them (Bracken and Oughton Reference Bracken and Oughton2006; Donovan, O’Rourke, and Looney Reference Donovan, O’Rourke and Looney2015). The task of interdisciplinary integration typically requires combining inputs that reflect these worldviews, and differences among worldviews can complicate this task, especially since they can be fundamental and hard to see. As Eigenbrode et al. explain, “Scientists collaborating within disciplines tend to share fundamental assumptions and values concerning the scientific process and, habitually, may discuss them little, but the failure to understand and address these fundamental differences can impede progress in cross-disciplinary efforts” (Reference Eigenbrode, O’Rourke, Wulfhurst, Althoff, Goldberg, Merrill and Morse2007, 56). Since the fundamental assumptions that compose disciplinary worldviews are partly metaphysical and epistemological in nature (O’Rourke and Crowley Reference O’Rourke and Crowley2013), philosophy can contribute to studies of interdisciplinarity integration by investigating these assumptions. By experimentally examining philosophical commitments or attitudes of practicing scientists, we gain a better understanding of the nature, mechanisms, and challenges of integration in interdisciplinary science.

This study explores a large data set generated by TDI (Eigenbrode et al. Reference Eigenbrode, O’Rourke, Wulfhurst, Althoff, Goldberg, Merrill and Morse2007; Donovan et al. Reference Donovan, O’Rourke and Looney2015). Since 2005, over 250 TDI (or “Toolbox”) workshops have been conducted worldwide to enhance mutual understanding, communication, and collaboration among members of interdisciplinary groups. After canvassing several issues in experimental philosophy of science in section 2, we offer detailed descriptions of these Toolbox workshops and the data they generate in section 3. In section 4 we present of our results. Then in section 5 we discuss those results, emphasizing what they mean for the distinction between the natural and the social sciences and for the challenge of interdisciplinary integration in the sciences.

2. Experimental Philosophy of Science

Our initial step toward developing experimental philosophy of science involves defending the validity and effectiveness of empirical methods in philosophy of science. Experimental philosophy is a recent and growing movement that employs the systematic collection and analysis of empirical data as means for philosophical investigation. A central tenet of this movement is that philosophical arguments often make, entail, or presuppose empirical claims, and, as such, the strength of these arguments can be judged, in part, by collecting and analyzing empirical data that bear on those claims. Experimental philosophers have moved into a wide range of philosophical debates in metaphysics, ethics, and epistemology (Knobe et al. Reference Knobe, Buckwalter, Nichols, Robbins, Sarkissian and Sommers2012; Sytsma and Livengood Reference Sytsma and Livengood2015; Machery Reference Machery2017). Yet almost ironically, the field of experimental philosophy of science has received little development (Machery Reference Machery, Sytsma and Buckwalter2016).

Stotz (Reference Stotz2009a, Reference Stotz2009b) and Weinberg and Crowley (Reference Weinberg and Crowley2009) outline some of the basic theoretical motivations for experimental philosophy of science. They assert that philosophy of science should be informed by how scientists conceive of and practice science. For this reason—as most contemporary philosophers would agree—philosophy of science that is conducted purely “from the armchair” with little awareness of actual scientists, scientific practices, and scientific findings is, at least prima facie, of dubious value. In our view, this common starting point for much of contemporary philosophy of science paves the way for the use of empirical data in philosophy of science.

The idea that philosophy of science can profit from empirical data about science is hardly novel. It is exemplified in the work of those who take what Bird (Reference Bird, Psillos and Curd2008) calls “the historical turn in the philosophy of science.” We also see it in works that use case studies to advance philosophical claims about science or its areas such as biology. A central distinction between experimental philosophy of science and these approaches, however, is the proposal that philosophy of science can also benefit from empirical analysis about how scientists think and behave. As Thagard notes, “Science is a human enterprise, and understanding the development of scientific knowledge depends on an account of the thought processes of humans” (Reference Thagard1988, 4). A central focus of experimental philosophy of science, then, should be the systematic collection and analysis of empirical data pertaining to scientific thought processes and practices. And where the existing empirical record about how scientists think and behave is too thin, philosophers should feel free to gather the data themselves, using appropriate methods, of course, and possibly in collaboration with practicing scientists.Footnote ³

In addition to data regarding the views and practices of actual scientists, another source of data could be relevant. Feist notes that “scientific thought and behavior are not limited to scientists per se but also encompass thought processes of children, adolescents, and adults who are simulating scientific problem solving and developing mental models of how the world works” (Reference Feist2006, 4). Experimental philosophy can gather and interpret data from either trained scientists or ordinary folk without specialized scientific training. This difference in data source corresponds to the difference between experimental philosophy of science narrowly or broadly construed. Both kinds of research are represented in what little experimental philosophy of science currently exists. As examples of narrow experimental philosophy of science, Stotz et al. (Reference Stotz, Griffiths and Knight2004) and Griffiths and Stotz (Reference Griffiths and Stotz2008) examine scientists’ conceptions of genes. Griffiths et al. (Reference Griffiths, Machery and Linquist2009) and Linquist et al. (Reference Linquist, Machery, Griffiths and Stotz2011), alternatively, engage in a broader form of experimental philosophy of science by examining views on innateness and human nature (respectively) held by nonscientists. Knobe and Samuels (Reference Knobe and Samuels2013) combine the two by examining the views on innateness held by both scientists and nonscientists.

In all of these manifestations, the commitment of experimental philosophy of science to the relevance of empirical data is fundamentally naturalistic. Despite debate about its details, the naturalist idea that the methods of philosophy are of a piece with the methods of science figures importantly into the background of these views (Papineau Reference Papineau and Zalta2009; Hartner Reference Hartner2013). That experimental philosophy of science is naturalistic should come as no surprise; what might be surprising, though, is the claim that philosophical aspects of science should be sensitive to data on the attitudes of scientists, including their philosophical attitudes. We contend that, without these data, philosophical claims about, for example, the roles of values in science or the epistemic standing of various methods will ignore a critical determinant of how values inflect actual scientific judgments, decisions, and behaviors and how methods are selected in examining specific research questions (Steel et al. Reference Steel, Gonnerman and O’Rourke2017). To some extent, data about these philosophical attitudes may be gleaned from a close acquaintance with scientists and their publications. But any such acquaintance is likely to be nonsystematic and nonrepresentative, thereby providing an insufficient basis from which to draw philosophical conclusions about science (Faust and Meehl Reference Faust and Meehl2002; Griffiths and Stotz Reference Griffiths and Stotz2008; Machery and Cohen Reference Machery and Cohen2012). A better way to uncover these attitudes is to deploy experimental methods, especially those of the social sciences. Of course, experimentally examining the philosophical attitudes of practicing scientists (specifically those relevant to philosophical reasoning about science) does not entail that philosophers should defer to the philosophical perspectives of scientists. Rather, the success of experimental philosophy of science requires combining appreciation for conceptual dimensions of science with details about how science actually works.

3. Method

We seek to expand experimental philosophy of science by engaging in it, specifically by experimentally probing for philosophical differences among scientific branches by examining the philosophical commitments of scientists who belong to those branches. To do this, we looked at 43 Toolbox workshops that were conducted with interdisciplinary research teams.Footnote ⁴ These workshops were conducted between March 2009 and October 2013; they had 346 participants (127 female), who ranged from graduate students to senior researchers with over 20 years of research experience.

At the start of these workshops, participants receive the STEM (science, technology, engineering, and mathematics) Toolbox instrument, which consists of six modules focusing on scientifically relevant philosophical issues and concludes with a page asking for demographic data. Each module contains four to five related prompts, 28 total (Looney et al. Reference Looney, Donovan, O’Rourke, Crowley, Eigenbrode, Rotschy, Bosque-Pérez, Wulfhorst, O’Rourke, Eigenbrode and Wulfhorst2013), in response to which participants either report agreement or disagreement on a five-point Likert scale ranging from 1 (disagree) to 5 (agree) or select “Don’t know” or “N/A.”Footnote ⁵ These last two options allow participants alternate responses to the Likert scale, thereby minimizing artificial clustering around the midpoint of the scale for cases of ignorance or nonapplicability. The demographic page includes four numbered spaces in which participants are asked to describe their disciplinary “identity.” By analyzing participants’ self-reported disciplinary identity in conjunction with their responses to these prompts, as we explain in section 4, we can look for philosophical differences across branches of science.Footnote ⁶

Two preliminary tasks were required before searching the data for philosophical differences. First, we needed to assign participants to academic disciplines. We began by examining the discipline(s) listed on the demographics page. Since these responses were open-ended, there was considerable variability in the discipline(s) listed. Some provided a single discipline; others gave several. Some responded quite generally (e.g., “biology”), while others responded very specifically (e.g., “fluvial geomorphology”). The question was how to categorize participants into disciplines and larger academic branches (groups of disciplines, such as engineering, arts and humanities, and life sciences) given this variability. This presented two challenges: selecting a disciplinary taxonomy to use in assigning participants to disciplines and interpreting participant responses in light of that taxonomy.

For the first challenge, we selected the Digital Commons Three-Tiered Taxonomy of Academic Disciplines (Bepress 2014) because the large degree of categorical resolution it offered made it easier to accommodate the wide variety of disciplinary self-identifications reported by workshop participants. The Digital Commons taxonomy is regularly updated based on user feedback and developed using entries from multiple sources, such as the University of California’s list of departments and the Taxonomy of Research Doctoral Programs from the National Academies. At the time of analysis, the Digital Commons taxonomy comprised over 900 unique disciplinary categories broken into three tiers that we refer to as academic branches, disciplines, and specializations. To illustrate, philosophy is a discipline in the academic branch of arts and humanities, with specializations including metaphysics, feminist philosophy, and philosophy of science. (The Digital Commons taxonomy’s third tier of specializations was especially useful for interpreting highly specific disciplinary identifications employed by Toolbox participants in their disciplinary self-identifications.) The top tier of academic branches for the Digital Commons taxonomy consists of the following categories: architecture, arts and humanities, business, education, engineering, law, life sciences, medicine and health sciences, physical sciences and mathematics, and social and behavioral sciences. For this article, we focused only on those researchers in the scientific branches: life sciences, physical sciences and mathematics, and social and behavioral sciences. As explained below, our focus on these branches was in part due to the number of participants in each branch.

The second challenge concerned interpreting participants’ disciplinary self-identifications in light of the Digital Commons taxonomy, a task complicated by demographic pages often containing multiple disciplinary specifications. In describing their disciplinary identity, most participants (276 of 346, 82.1%) provided at least two disciplines, such as “(1) algorithmics (2) microbial ecology (3) discrete math.” For the sake of categorizing participants into academic branches and disciplines, we only looked at the first discipline reported, for two reasons. First, we are interested in researcher’s primary disciplinary identity, and we operated on the assumption that participants entered this discipline first. Second, it was not possible to look beyond the first discipline for everyone, since 17.9% participants only listed one discipline. We thus opted to err on the side of caution by abstaining from drawing on secondary disciplines listed to inform judgments about disciplinary identity.

After meeting the challenges associated with the first preliminary task, we turned to the second, which was to clarify the nature of the differences that could be revealed by the STEM Toolbox instrument. This task was complicated by the number of dimensions of potential difference that the instrument was designed to reveal. The STEM instrument includes prompts such as “Objectivity implies an absence of values by the researcher” and “Validation of evidence requires replication.” It was designed in an explicitly philosophical fashion, with half of its prompts classified as epistemological and half as metaphysical (Eigenbrode et al. Reference Eigenbrode, O’Rourke, Wulfhurst, Althoff, Goldberg, Merrill and Morse2007; O’Rourke and Crowley Reference O’Rourke and Crowley2013). The epistemology prompts are divided into those that concern a scientist’s motivation for participating in research, those that concern the methods used in conducting research, and those that concern identifying when conclusions have been reached. The metaphysics prompts are organized into those that concern the nature of reality, the structure of reality (in particular, reductionism and emergence), and values.Footnote ⁷

The STEM instrument’s prompts also distinguish between how scientists view their own work and how they view science more broadly. In doing so, the prompts help isolate important determinants of a scientist’s identity and place them in the context of a broader construal of scientific practice. Of the 28 prompts that we examine, eight are reflexive prompts that express characteristics that some regard as fundamental to their own research (e.g., “My research typically isolates the behavior of individual components of a system”), and 20 are prompts that are not limited to one’s own research. Of these 20 prompts, the context of interpretation varies. For example, two focus specifically on cross-disciplinary research, six use the adjective ‘scientific’ explicitly to frame interpretation, and five explicitly mention research without modifying it with ‘scientific’. The balance talk about aspects of research without mentioning a context. Since this instrument is called the Scientific Research Toolbox Instrument and is deployed in workshop settings with interdisciplinary science teams, the expectation is that the participants will interpret the 20 nonreflexive prompts as pertaining to research that is scientific and that extends beyond one’s own practice.

Thematically, the eight reflexive prompts address aspects of science and scientific research practices that apply across multiple research projects, such as the methods one primarily uses or the ontological character of one’s subject matter, with one prompt addressing the collective project of the workshop participants (i.e., “The importance of our project stems from its applied aspects”). The 20 remaining prompts are more heterogeneous. Some offer provocative analyses of concepts that some take to be central to science, such as objectivity or advocacy. A second group orients workshop participants to certain visions of scientific explanation and scientific practice (e.g., “Value-neutral scientific research is possible”). Additional prompts express views about what must be true for a practice to count as science (e.g., “Scientific research must be hypothesis driven”), while still others address the values of particular scientists (e.g., “The principal value of research stems from the potential application of the knowledge gained”). This set of 20 prompts is unified in reflecting different ways of thinking about science, including its structure and function.

In indicating their agreement or disagreement with each prompt, participants reveal (in some cases perhaps only indirectly) how they practice science or how they think science should be practiced. All the prompts (but one) abstract away from specific research contexts and projects, allowing participants to understand them in the context of science as they practice it and view it. In responding to them, participants are asked to adopt the primary disciplinary perspective they assume in their research, and because of this, the responses can be understood as articulating commitments that frame one’s research approach, either affirmatively or negatively.Footnote ⁸ Together, the STEM instrument’s prompts can be seen as partly mapping an investigator’s research worldview (or at least the more cognitive aspects of it), which may be understood as a set of “more or less tacit beliefs held by researchers about what they are studying and how to study it, as well as views about the nature of the output of their inquiry” (O’Rourke and Crowley Reference O’Rourke and Crowley2013, 1938). When taken together, the maps that emerge for workshop participants display the topography of agreement and disagreement across a range of philosophical issues that some view as fundamental to scientific inquiry and that illuminate aspects of a scientist’s identity. Each map is systematic but skeletal, highlighting a few attitudes about the philosophical dimensions of science, sometimes fairly directly (as in the broader prompts about science) and sometimes more indirectly (as in the reflexive prompts). More technically, the prompts elicit, directly or indirectly, commitments about the metaphysical, epistemological, and axiological dimensions of science that often frame how one understands and practices science.

4. Results

Using participant self-reports of disciplinary identity, two of the authors served as raters to categorize participants into academic branches, using the Digital Commons Three-Tiered Taxonomy of Academic Disciplines (Bepress 2014).Footnote ⁹ The two raters first agreed to exclude nine participants for providing responses that were clearly not research disciplines (such as “grant administration”). After independently coding an academic branch for each participant, the two raters then compared their results. Cohen’s kappa determined a relatively high degree of correspondence ($\kappa =.82$), with the raters differing on 54 participants (out of 346). Of these, they resolved their disagreement for all but 13 participants, who were then excluded from further analysis.Footnote ¹⁰ The results are summarized in table 1.

Table 1.

Summary of Participants’ Academic Branches as Determined by Two Raters

As table 1 indicates, life sciences (LS), physical sciences and mathematics (PSM), and social and behavioral sciences (SBS) were well represented among Toolbox participants. Since we are interested in differences among scientific branches, we focused our attention on participants in these three fields, excluding participants from other academic fields from analysis. This left us with 264 participants (95 female).

For our analysis, we ran the Kruskal-Wallis rank sum test for each of the 28 Toolbox prompts.Footnote ¹¹ To contextualize this test, for each prompt, each of the three scientific branches had a median score. The Kruskal-Wallis test examines the three medians for each prompt for significant differences. The null hypothesis is that the medians are equal (i.e., Mdn_LS = Mdn_PSM = Mdn_SBS). The test includes a calculation of the probability (the p-value reported throughout this article) of the observed results (or more extreme results), given the null hypothesis. If the probability is low enough, then we reject the null hypothesis, meaning that at least one median of the three is significantly different from the others. A follow-up test reveals which median or medians are different. If, however, the probability is not sufficiently low, the null hypothesis is not rejected, indicating that responses to this prompt are statistically indistinguishable across the three academic branches. The results of the Kruskal-Wallis test for each prompt are summarized in table 2.Footnote ¹²

Table 2.

Results from Kruskal-Wallis Tests for Each of the Toolbox Prompts

* Significant after applying the Holm-Bonferroni method for correcting for family-wise error.

Because the STEM instrument was not designed to detect particular differences, we should be mindful of the family-wise error rate associated with multiple testing. In effect, we are not testing a single hypothesis but rather 28 hypotheses simultaneously. The danger in testing multiple hypotheses simultaneously is that it increases the probability of a false positive—a type I error. When testing a single hypothesis, the threshold for a significant p-value is often set at .05, representing a 5% chance for a type I error. If, however, the same threshold of .05 were applied separately for each of the 28 tests here, then the family-wise error rate (i.e., the likelihood of at least one false positive) is approximately .74. So some form of correction is called for; we employed the Holm-Bonferroni method (Holm Reference Holm1979), which is a stepwise algorithm that lowers the p-value required to reject a null hypothesis. Using this method, we find significant results for six of the prompts (marked with an asterisk in table 2): methodology 2, methodology 3, methodology 4, confirmation 3, reality 2, and reality 4. Three additional prompts were very close to significance after correction: reality 1, methodology 1, and values 3.Footnote ¹³ Since the Holm-Bonferroni method is a fairly conservative means of correcting for family-wise error, we have included these three for follow-up analysis along with the other six. But these results in particular should be taken tentatively.

The Kruskal-Wallis tests found significant or nearly significant differences among the three branches of science for nine prompts. To determine where the differences reside, for each of these prompts we conducted follow-up analyses using the Mann-Whitney-Wilcoxon (MWW) test. This is a pair-wise test, comparing the medians of two groups. In this case, for each prompt, three MWW tests are conducted, one for each comparison (LS and PSM, LS and SBS, and PSM and SBS). Results of all MWW tests for the nine prompts are summarized in table A1.

4.1. Reality

For the reality prompts, the initial Kruskal-Wallis tests found significant results for reality 2 and 4, and reality 1 was close. All of these results were borne out by the follow-up MWW tests (fig. 1).Footnote ¹⁴ Researchers in life sciences and physical sciences and mathematics were significantly were more likely to agree with reality 1 (“Scientific research aims to identify facts about a world independent of the investigators”) than those in social and behavioral sciences (LS and SBS: $U=\mathrm{4,776.5}$, $p=9.52\times {10}^{\mathrm{-04}}$, $r=.24$; PSM and SBS: $U=\mathrm{2,432.5}$, $p=.03$, $r=.20$). However, life sciences and physical sciences and mathematics were not statistically distinguishable. For reality 2 (“Scientific claims need not represent objective reality to be useful”), the difference was statistically significant only between life sciences and social and behavioral sciences ($U=\mathrm{2,364.5}$, $p=3.79\times {10}^{\mathrm{-04}}$, $r=.26$). As figure 1 shows, the mean for physical sciences and mathematics was between the other two, although its differences with the other two branches were not statistically significant. For reality 4 (“The subject of my research is a human construction”), life scientists and physical scientists and mathematicians were less willing to agree with this prompt than were social and behavioral scientists (LS and SBS: $U=\mathrm{1,698.5}$, $p=2.70\times {10}^{\mathrm{-09}}$, $r=.43$; PSM and LS: $U=875$, $p=2.90\times {10}^{\mathrm{-06}}$, $r=.42$). Again, life sciences and physical sciences and mathematics were statistically indistinguishable, which is a pattern repeated below.

Figure 1.

Summary of means by scientific branch for the reality 1, 2, and 4 prompts.

4.2. Methodology

Next we conducted follow-up analyses for methodology 1 (“Scientific research (applied or basic) must be hypothesis driven”), methodology 2 (“In my disciplinary research, I employ primarily quantitative methods”), methodology 3 (“In my disciplinary research, I employ primarily qualitative methods”), and methodology 4 (“In my disciplinary research, I employ primarily experimental methods”). For methodology 1, life scientists and physical scientists and mathematicians were more willing to agree that science must be driven by hypotheses than were social and behavioral scientists ($U=\mathrm{4,949}$, $p=.002$, $r=.22$ and $U=\mathrm{2,627}$, $p=.009$, $r=.23$ respectively).Footnote ¹⁵ Again, life sciences and physical sciences and mathematics were statistically indistinguishable (fig. 2).

Figure 2.

Summary of means by scientific branch for the prompts methodology 1–4.

Methodology 2 and 3 concern quantitative and qualitative methods, respectively, and so can be discussed together. Both exhibit the previously observed pattern, where there is no significant difference between life sciences and physical sciences and mathematics but both differed from social and behavioral sciences. Specifically, researchers in the social and behavioral sciences are less inclined than researchers in the other two branches to employ quantitative methods ($U=\mathrm{5,052}$, $p=1.10\times {10}^{\mathrm{-04}}$, $r=.28$ and $U=\mathrm{2,648.5}$, $p=3.99\times {10}^{\mathrm{-04}}$, $r=.31$) and more inclined to employ qualitative methods ($U=\mathrm{2,109.5}$, $p=3.89\times {10}^{\mathrm{-07}}$, $r=.36$ and $U=\mathrm{1,170}$, $p=5.91\times {10}^{\mathrm{-05}}$, $r=.35$). Methodology 4 is the only prompt for which all three branches are significantly different from one another. Life scientists were most likely to report employing experimental methods, followed by the physical scientists and mathematicians, and the social and behavioral scientists were the least likely to rely on experimentation.

4.3. Confirmation and Values

Finally, we consider the confirmation and values modules. Since only one prompt in each module was identified by the Kruskal-Wallis tests as requiring follow-up analysis, we present them together. As figure 3 shows, the pattern of results is similar for these two prompts. Confirmation 3 (“Validation of evidence requires replication”) suggests that the demand for replication in the social and behavioral sciences is significantly lower than in the life sciences ($U=\mathrm{5,309.5}$, $p=1.42\times {10}^{\mathrm{-05}}$, $r=.31$) or physical sciences and mathematics ($U=\mathrm{2,656.5}$, $p=.002$, $r=.27$). With respect to values 3 (“Value-neutral scientific research is possible”), social and behavioral scientists were less willing to endorse the possibility of value-neutral science than were life scientists ($U=\mathrm{4,639}$, $p=.002$, $r=.23$) and physical scientists and mathematicians ($U=\mathrm{2,586.5}$, $p=.006$, $r=.24$). Future research on this topic is particularly recommended. In both cases, life sciences and physical sciences and mathematics were statistically indistinguishable.

Figure 3.

Summary of means by scientific branch for the confirmation 3 and values 3 prompts.

5. Discussion

In this section, we dig deeper into our results by first addressing a concern about how we interpret participants’ responses. We then turn to the project of distinguishing the natural and the social sciences. Finally, we consider the implications these results have for the ongoing conversation within philosophy of science about interdisciplinary integration.

6. Conclusion

Our examination of Toolbox data revealed differences in how scientists from different branches of science reacted to philosophical claims about aspects of scientific practice. In total, these results suggest that distinctions among branches of science are correlated with interesting philosophical differences. These differences concern attitudes toward realism, objectivity, replication, values, hypotheses, and methods. Since our results are preliminary, further experimental research in the philosophy of science is warranted to explore the scope and nature of these differences, as well as the extent to which these differences are responsible for deeper differences among ways of practicing science.

These results make two contributions to the burgeoning field of experimental philosophy of science. First, reflection on our results, and especially their bearing on recent work on scientific integration, provides an argument by example for the validity and effectiveness of experimental philosophy of science. These results help to reinforce philosophical conclusions about scientific integration derived through nonexperimental means (e.g., the claim that integration across the natural/social divide can be especially challenging to achieve). In this way, experimental philosophy of science can provide a line of converging evidence in support of extant views in philosophy of science. Second, our results show that experimental philosophy of science can also suggest insights into science that may be harder to gain through nonexperimental means, for example, that philosophical differences across individuals may be more of an obstacle to interdisciplinary integration than cross-disciplinary differences.