Consider the following case, and the question that follows:
Bob has a friend, Jill, who has driven a Buick for many years. Bob therefore thinks that Jill drives an American car. He is not aware, however, that her Buick has recently been stolen, and he is also not aware that Jill has replaced it with a Pontiac, which is a different kind of American car.
Does Bob really know that Jill drives an American car, or does he only believe it? Footnote 1
This case (henceforth the Gettier Car Case) is a variant of classic Gettier cases, in which the subject purportedly has justified true belief but not knowledge. Since their emergence fifty years ago (Gettier Reference Gettier1963), Gettier-style cases have been used by epistemologists to refute the traditional conception of knowledge as justified true belief. As Timothy Williamson notes, “the refutation was accepted almost overnight by the community of analytic epistemologists. [The] thought experiments were found intrinsically compelling” (Williamson Reference Williamson2007: 180). In this way, Gettier-style cases have been seen as paradigmatic triumph of the method of cases, “which takes intuitions … as input and produces implicitly or explicitly normative claims as output” (Weinberg et al. Reference Weinberg, Nichols and Stich2001: 434).
In “Normativity and Epistemic Intuitions” (NEI), Weinberg et al. provide empirical evidence against the method of cases. Their data show significant differences in epistemic intuitions among people of different cultures and socioeconomic statuses. Among those differences documented in NEI,Footnote 2 the cross-cultural difference concerning the Gettier Car Case is of particular interest, given the symbolic significance of Gettier cases in epistemology. In their study, the authors of NEI conducted a survey in which Rutgers University undergraduate students of different ethnic backgrounds were asked to respond to the Gettier Car Case above. Whereas the majority of Westerners answered: “Bob only believes,” the majority of East Asians gave the unorthodox answer: “Bob really knows” (Weinberg et al. Reference Weinberg, Nichols and Stich2001: 443). We will refer to the discrepancy between the East Asian and Western participants' responses to this case as the Car Case Effect.Footnote 3
NEI argues that the Car Case Effect and other similar effects indicate the existence of systematic difference in epistemic intuitions. NEI mounts the following attack on the method of cases: since people's intuitions about cases are contingent on arbitrary factors such as ethnicity and socioeconomic status, intuitions should not be trusted as evidence that bears on the nature of knowledge.
NEI has generated a firestorm of debate regarding the philosophical implications of those demographic effects that it reported. Since publication, NEI has been cited 450 times according to Google Scholar.Footnote 4 This debate consists of questions that arise once we take those effects at face value: given that the majority of the East Asian participants do not share the orthodox Western intuition, what follows? Many philosophers accept NEI's conclusion that the method of cases is discredited by those demographic effects, while others are skeptical about it but for different reasons. One line of skeptical arguments against NEI amounts to a defense for the method of cases, as it finds fault with the method of surveys employed in NEI and so argues that NEI's results do not raise problems for the method of cases. For example, Kauppinen (Reference Kauppinen2007) argues that intuitions gathered in surveys among lay people are not the same sort of reflective intuitions that philosophers are able to tap into due to their training and reflection. In contrast, the other line of skeptical arguments against NEI is grounded on a denial of the method of cases. For example, Cappelen (Reference Cappelen, Booth and Rowbottom2014) argues that NEI among many other studies in experimental philosophy “has been engaged in an attack on a strawman,” because “philosophers don't rely on intuitions” (Cappelen Reference Cappelen, Booth and Rowbottom2014: 269) in the first place.
This paper does not aim to contribute to the philosophical debate concerning the implications of the purported demographic effects. Instead, it aims to examine whether or not those effects exist at all. The scope of our project is limited to the Car Case Effect, i.e., the purported difference of intuition concerning the Gettier Car Case between East Asian participants and Western participants.
Our project fits into a growing concern in social sciences over studies with flashy results failing to replicate (e.g., Doyen et al. Reference Doyen, Klein, Pichon and Cleeremans2012). In reading NEI, we noticed that the existence of the Car Case Effect was generalized from the responses of 23 East Asian participants. We think that it is worth seeing if the effect replicates with a larger sample size.
1. EXISTING REPLICATION STUDIES
Worries over the small sample size of the NEI study have prompted replication studies by Nagel et al. (Reference Nagel, San Juan and Mar2013) and Seyedsayamdost (Reference Seyedsayamdost2014). Neither found evidence of the Car Case Effect. In each study, the majority of both Westerners and East Asians gave the philosophically orthodox answer that, “Bob only believes.” Although these studies give us reason to doubt the existence of the Car Case Effect, they differ from the NEI study in ways that might affect the decisiveness of their results.
In Nagel et al. (Reference Nagel, San Juan and Mar2013), the authors conducted an online survey among Canadian undergraduate students which examined a wide range of controversial cases. They found that lay people, including East Asians, gave the orthodox answer to the Gettier Car Case, and thus failed to replicate the Car Case Effect. As a replication study, however, Nagel et al.'s study differs from the NEI's in two significant ways. The first concerns the large number of cases preceding the Gettier Car Case in their study. The Gettier Car Case was given to their participants after they responded to eight vignettes (Nagel et al. Reference Nagel, San Juan and Mar2013: 656). This fact rendered their results concerning this case vulnerable to substantial order effects. The second major difference lies in the answer choices that the authors presented to their participants. While the NEI study provided two answer choices, “Really Knows” and “Only Believes,” they provided three answer choices: “Knows,” or “Doesn't Know,” or “Unclear – not enough information provided in the story” (Nagel et al. Reference Nagel, San Juan and Mar2013: 656, 658).
In Seyedsayamdost (Reference Seyedsayamdost2014), the author attempted and failed to replicate almost all of the demographic effects presented in NEI, including the Car Case Effect. He surveyed the Gettier Car Case in three independent data sets (his data sets 1, 3 and 4), and participants in his study were mainly in Britain and included non-university students. As a replication study, there are again two ways in which Seyedsayamdost's study differs from NEI's. First, only the data set 1 was collected in person, which is the method employed in NEI, while the other two were collected online. Furthermore, he made the following changes to the case in his data set 4: the origin of the cars was changed from ‘American’ to ‘Japanese’, and the names of the cars were changed from ‘Buick and Pontiac’ to ‘Toyota and Honda,’ respectively (Seyedsayamdost Reference Seyedsayamdost2014: 6).Footnote 5
The goal of our study was to stay as close to the original NEI study as possible. Firstly, we used the same case and answer options as used in the NEI study. Secondly, our study used the same data collection method as the NEI study. Thirdly, like the NEI study, our participants were university students in the northeast region of the United States. If the NEI study picked up on an effect that really exists, ours should detect it as well.
Another advantage of our study was the large sample size of East Asians. There were 21 East Asians surveyed in Nagel et al. (Reference Nagel, San Juan and Mar2013), and 42 East Asians surveyed in Seyedsayamdost's data set 1, the only data set in his study that employed exactly the same case and method as those of the NEI study. Our sample includes 82 East Asians. An advantage of this larger sample size is that it satisfies the 2.5 sample size rule, which stipulates that replication studies should have samples that are 2.5 times the size of the original study (Simonsohn Reference Simonsohn2013).Footnote 6
2. METHODS AND RESULTS
2.1. Methods
2.1.1 Participants
The participants were 208 Yale University students (48% female). After removing individuals who did not complete the survey, we had a valid sample of 202 participants. 40% of participants self-identified as East Asian, and 28% as Caucasian. 70% of participants answered that English was their first language.
2.1.2 Materials and Procedure
Our survey consisted of two main components: a knowledge survey and a demographic questionnaire. Participants first completed the knowledge survey and then the demographic questionnaire. The knowledge survey was composed of a single vignette, the original Gettier Car Case given at the start of our paper. In addition to asking the knowledge question (Does Bob really know that Jill drives an American car, or does he only believe it?), we asked participants to rate, on a scale of 1 to 7, how confident they were about their answers.
Participants completed a demographic questionnaire that asked them to provide five categories of demographic data: (1) Yale college major/intended major, (2) gender, (3) first language, (4) ethnic background, and (5) number of philosophy courses taken. Question (4) asked participants to self-identify between six categories: East Asian, Caucasian, South Asian, African American, Latino, and Other. Participants who chose “Other” were asked to write in their own descriptions.
2.2. Results
Because we focused on comparison of East Asian (N = 82) and Caucasian (N = 58) participants' responses, all analyses were conducted on data from those two groups. Within those groups, 50.7% were female, 0.7% were philosophy majors. Participants in the Caucasian group had taken slightly more philosophy classes (M = 1.9, SD = 0.43) than participants in the East Asian group (M = 1.0, SD = 0.29), but the difference was not statistically significant, t (129) = 1.8, p = 0.07.
The percentage of the participants who answered “Bob only believes” in the two groups was strikingly similar: 85.4% for the East Asian group and 86.2% for the Caucasian group. This difference was not statistically significant, χ2 (1, N = 140) = 0.02, p = 0.89.
Following the method first developed by Starmans and Friedman (Reference Starmans and Friedman2012), we also calculated the “weighted knowledge ascription” for each participant and compared those of East Asian and Caucasian participants. We arrived at a participant's “weighted knowledge ascription” by multiplying one's response to the knowledge question (really knows = 1; only believes = −1) to his/her reported confidence regarding the response (1 to 7). Thus, weighted knowledge ascriptions ranged from −7 (fully confident knowledge denial) to 7 (fully confident knowledge ascription). The mean of the weighted knowledge ascription for our East Asian participants was −3.63, while for Caucasian participants it was −3.60. Here again, there was no significant difference between the two groups, t (137) = 0.04, p = 0.97.
We further looked at the 95% confidence interval of the weighted knowledge ascription, and found that the entire interval fell within the “only believes” region of the scale, both for Caucasians (−4.53 to −2.73) and for East Asians (−4.63 to −2.58). In other words, both groups strongly rejected the claim that Bob really knows.
We anticipated the worry that the East Asian participants in our study were likely to be second or even third generation immigrants had deeply adapted to Western culture, so that their answers became homogeneous with Westerners'. To start with, if this is a legitimate worry, it is not specific to our study; it also applies to NEI and with its sample of university students at Rutgers. But in any case, our study employed a precautionary step to address this concern. We asked for participants' first language and used it as a rough indicator of the depth of their Western assimilation. Among our East Asian participants, 87% of those who had English as their first language (N = 45) and 84% of those who did not have English as their first language (N = 37) answered that “Bob only believes.” This difference was not statistically significant, χ2 (1, N = 82) = 0.14, p = 0.71.
3. CONCLUSION
Our study, conducted using a sample size much larger than the NEI study's, resembled the original study in all relevant ways. We found no difference between self-identifying East Asian and Caucasian participants' responses to the Car Case. Both overwhelmingly rejected the claim that Bob knows: 85.4% of East Asians and 86.2% of Caucasian group denied knowledge.
As Figure 1 indicates, our results are in good company. The Car Case Effect does not exist in any sample other than NEI's sample of 23 East Asians. Residual worry that our failure to replicate is due to some peculiar property of our sample which is not shared by NEI's (for example, the socioeconomic status of Yale students) should be significantly mitigated given that all other studies, with different samples, also fail to replicate the Car Case Effect. In all other studies, the overwhelming majority of East Asians denied knowledge.
Percentage of participants in the NEI study and replication studies who chose “Bob really knows”.
The total body of evidence does not support the existence of a demographic effect on knowledge attribution. Indeed, there is cause for doubting the existence of newly discovered demographic effects beyond knowledge attribution.Footnote 7 The majority (85.0%) of experimental studies claiming any kind of demographic effect (non-epistemic cases included) fail to replicate, while only the minority (14.3%) of experimental studies not claiming a demographic effect fail to replicate.Footnote 8 Returning to knowledge attribution, we cannot rule out the possibility that there is some demographic effect yet to be discovered. If subsequent studies indicate the existence of such demographic effects, the issue of their philosophical implications warrants serious reconsideration.Footnote 9 But given the current absence of empirical evidence supporting the existence of such effects, this debate is best left tabled.Footnote 10
