1. Introduction
Much research in cognitive psychology and other related disciplines has explored mechanisms and processes underlying people’s understanding of written and spoken sentences (e.g., Christianson, Hollingworth, Halliwell, & Ferreira, Reference Christianson, Hollingworth, Halliwell and Ferreira2001; Duffy, Henderson, & Morris, Reference Duffy, Henderson and Morris1989; Gibson & Pearlmutter, Reference Gibson and Pearlmutter1998; McDonald, Pearlmutter, & Seidenberg, Reference MacDonald, Pearlmutter and Seidenberg1994; see Clifton, Reference Clifton, Crocker, Pickering and Clifton2000, for review). However, empirical research on what people do with the information that they understand or have understood is relatively limited (but see Bernicot & Laval, Reference Bernicot, Laval, Noveck and Sperber2004; Fusaro & Harris, Reference Fusaro and Harris2008). This is a problematic omission since people do things with language beyond just transmitting and comprehending information (Austin, Reference Austin1962; Clark, Reference Clark1996; Searle, Reference Searle1969; Semin, Reference Semin1995).
For example, more specific to our interest, informal observations of people involved in spoken and written communication suggests that people often respond to a written and spoken assertion (e.g., “Life is tough.”) with evaluative responses indicating whether they agree (or believe it to be true) or disagree (or believe it to be false). In addition, anecdotal evidence is often mentioned in the literature on child language acquisition that parents tend not to correct ungrammatical sentences produced by a child but to correct the truthfulness of the content of the sentence (e.g., Feist & Rosenberg, Reference Feist and Rosenberg2012). Thus, the evaluation of the truthfulness of the information conveyed by a sentence often appears to be an integral part of sentence processing (e.g., comprehending), indicating that doing things (i.e., presenting and evaluating arguments) with language may be inseparable from comprehension.
In this paper, we specifically focus on the evaluation of the truthfulness of an assertion produced by others. In particular, the paper explores these two specific descriptive questions related to one-sentence assertion that we often see in casual conversations (e.g., “Life is tough.”): (i) What is the relation between meta-cognitive and evaluative response – in particular, what kind of evaluative response (i.e., agree, disagree, or not sure) tends to be produced to a statement that people claim not to understand?; and (ii) whether the meta-cognitive (i.e., understand / do not understand)) or the evaluative (i.e., agree/disagree; true/false) statement is more prevalent as a response to a one-sentence assertive statement.
We believe these questions are important for at least two theoretical reasons across different levels of analysis/description. First, the questions are related to substantial theoretical issues on how people use and what people do with language, analyzed from the pragmatic perspective of linguistic/psycholinguistic research. Second, addressing these questions sheds lights on the nature of cognitive processes (e.g., the relation between acceptance and comprehension; see Sperber et al., Reference Sperber, Clement, Heintz, Mascaro, Mercier, Origgi and Wilson2010) underlying the generation of an evaluative and meta-cognitive response in reaction to an assertive statement. In the following sections, we will discuss how these questions shed light on the pragmatic and cognitive processes of assertive statements.
1.1. assertion as an argument: the pragmatics of how people infer a speaker’s meaning of an assertion
Recent psycholinguistic and linguistic work on pragmatics recognizes the importance of the recognition of a speaker’s intention as a goal of comprehension processes in which a recipient of a message uses all the available information to infer what the speaker intends to achieve or do with the message (e.g., Sperber & Noveck, Reference Sperber, Noveck, Noveck and Sperber2004; Wilson & Sperber, Reference Wilson, Sperber, Horn and Ward2011). The importance and appropriateness of this perspective in research on spoken language comprehension is also recognized and emphasized by cognitive psychologists working on psycholinguistic research (e.g., Tanenhaus & Brown-Schmit, Reference Tanenhaus and Brown-Schmidt2008). According to this approach, called the ‘inferential model’ (Wilson & Sperber, Reference Wilson, Sperber, Horn and Ward2011), so-called ‘linguistic meaning’ is only one part of the information that the recipient of a message uses to infer what a speaker/writer means with that message in that particular situation and at that particular time.
This perspective is more or less based on the general theoretical perspective paved by speech act theory (Austin, Reference Austin1962). Speech act theory attempts to illustrate aspects of communication that goes beyond strictly linguistic meaning by proposing three levels of description (analysis) of the effects (acts) that speech can theoretically (logically) achieve: locutionary acts, illocutionary acts, and perlocutionary acts. Locutionary acts refer to acts of performing an utterance with its ostensible meaning relying on the phonetic, lexical, syntactic, and semantic components of speech (Austin, Reference Austin1962). Thus, in a sense, locutionary acts can roughly be understood as surface meaning that can be decoded from a statement in isolation. In contrast, illocutionary act or illocutionary force refers to the actual or intended meaning or social action of that speaker in a specific context. Finally, perlocutionally acts are the utterances’ actual effects produced in the recipient of the speech on that particular occasion, such as persuading, convincing, scaring, and/or inspiring, whether the speaker intended such an effect or not. Thus, speech act theory emphasizes the need to analyze the effects of speech at many levels.
With regard to the processes of inferring a speaker’s meaning, which is illocution in speech acts theory, relevance theory (Sperber & Wilson, Reference Sperber and Wilson1995; Wilson & Sperber, Reference Wilson, Sperber, Horn and Ward2011), built upon Grice’s (Reference Grice, Cole and Morgan1975) work on how human language-based communication works as an inferential process, emphasizes the importance of relevance in describing/analyzing the nature of the conveyed meaning in (language-based) communicative situations. According to relevance theory, relevance represents and is characterized by people’s tendency or drive to achieve a maximum cognitive effect (i.e., the most informative meaning) given the speech, situation, recipient’s goal, and other factors operating in the communicative situation (Wilson & Sperber, Reference Wilson, Sperber, Horn and Ward2011). As such, people’s search for a speaker’s meaning is driven by their efforts to maximize the cognitive effect (e.g., informational value) that can be attained with minimum cognitive effort, given the contextual factors (Van der Henst & Sperber, Reference Van der Henst, Sperber, Noveck and Sperber2004).
Thus, the inferential model of communication emphasizes that language-based communication is fluid and dynamic, and involves multiple levels of meaning. As such, the recipient of a message needs to perform inferential processes to identify what a speaker’s intention is in a particular communicative situation, based on relevance. Assuming that there are multiple levels of meaning involved in communication using assertive statements, then, one naturally emerging empirical question is whether there are any systematic patterns of a speaker’s meaning that a recipient of a message identifies in a communicative situation involving a simple one-sentence assertive statement.
More specifically, our interest is in whether a speech recipient’s inferring a speaker’s meaning associated with an assertive statement remains conservative, such that they limit it to recognition of locution (i.e., linguistic surface meaning) or extend it to recognition of illocution (i.e., intended context-specific meaning), in particular, as advancement of an argument or making a claim about her belief about a subject as true. This proposal that people may recognize a speaker’s communicative intention associated with an assertive statement as active argument is in large part based on informal observation of people’s communicative exchanges involving assertion as described at the beginning of this paper. However, this proposal is also consistent with some theoretical perspectives which we will discuss shortly. We believe this issue is both interesting and important in understanding people’s use of language.
Consider what happens when a person hears or sees the statement “President Obama is a good president.” What thought or overt response might be elicited to this statement? One may just process the meaning of this sentence in the abstract and acknowledge that someone is trying to state that President Obama is a good president, unless the statement is followed by a question, such as “What do you think?” or “Don’t you think so?” It may depend on the specific situation. However, given the informal observation that people are generally judgmental, it is possible that a recipient perceives a statement without a follow-up question as a sign that the speaker is actively arguing that President Obama is truly a good president, even though there is no contextual information that positively signals this. If people tend to perceive the statement as an argument, they will respond with an evaluative response as opposed to a meta-cognitive response. This difference is significant, and potentially provides some useful information about how people determine the relevance of an utterance.
Now, turning to our specific proposal that people may have a tendency to perceive assertive statements as arguments, as opposed to, for example, self-expression, there is a proposal that a primary function of human reasoning is to build and evaluate arguments in communication (Mercier & Sperber, Reference Mercier and Sperber2011). Mercier and Sperber argue, based on various empirical data as well as their theoretical basis, that the main function of human reasoning is not so much about improving the quality of decisions but to produce an argument in support of one’s belief as well as to evaluate and detect the fallacy and weakness in an argument advanced by others. Hence, Mercier and Sperber’s proposal specifies one particular way in which relevance is instantiated in communicative situations by seeking the maximum informational value (e.g., epistemic truth) of communication in relation to what one believes as true.
We believe our proposal that people have a tendency to perceive an assertion as an argument is generally in line with Mercier and Sperber’s (Reference Mercier and Sperber2011) proposal, because their proposal assumes that people are biased to perceive communicative situations as an exchange of arguments in which reasoning ability serves to dissect and evaluate the argument’s truthfulness when and if it is necessary to do so. Our proposal also agrees with Perelman and Olbrecht-Tyteca (Reference Perelman and Olbrecht-Tyteca1969), who suggested that human communication is built around the presentation and evaluation of arguments.
Another related proposal is the theory of action identification in the field of social cognition. Vallacher and Wegner (Reference Vallacher and Wegner1985) proposed that people tend to identify the meaning (goal) of their actions at the highest possible level. For example, when people try to open a door to a bathroom, the action is most prevalently identified as going to the bathroom, as opposed to lower-level identifications, such as opening the bathroom door or turning the doorknob. Although the theory of action identification primarily focuses on one’s own action, the idea represented by the theory seems to be equally applicable to the interpretation and identification of any acts performed by others, such as speech or communicative acts. In a way, this theory is roughly in line with relevance theory (Sperber & Wilson, Reference Sperber and Wilson1995) because it points to the psychological tendency to identify any act at the most relevant level (i.e., the level at which the act has the largest cognitive effect on the recipient).
Based on these considerations, we hypothesize that people have a strong bias to perceive an assertion that a speaker/writer is making a claim that the content of an assertion is true. If the proposed hypothesis is correct, then the majority of assertive statements would elicit an evaluative response as opposed to a meta-cognitive response, even when only a minimal amount of on-site (contextual) information about the speaker’s/writer’s true intentions is available to the recipient. In addition, we expect that the evaluative response (i.e., agree/disagree) will be elicited faster compared to the meta-cognitive response (i.e., understand / do not understand).
1.2. cognitive processes underlying the comprehension and evaluation of a one-sentence assertion
The other issue relevant to the two research questions explored in this paper is the relation between comprehension and evaluation (acceptance) as analyzed/described from the cognitive processing perspective: What would be the cognitive processing underlying the evaluative and meta-cognitive response observed in the processing of a one-sentence assertion? We believe the examination of the relation between the meta-cognitive response (i.e., understand / do not understand) and the evaluative response (i.e., agree/disagree) will help us understand the nature of the cognitive processes underlying one-sentence assertive statements. In particular, we believe the types of evaluative response produced to a statement participants claim not to understand would reveal useful information in this regard.
For example, if producing a positive (i.e., agree) or negative (i.e., disagree) evaluative response to a statement, which means making a decision on whether one agrees or disagrees with it, is dependent on and conditional on the belief that one understood the statement to a sufficient degree, one should not commit to a positive or negative evaluative response to a statement that one claims not to understand. According to this (serial processing) perspective, the majority of evaluative responses to a statement that people claim not to understand should be neutral, such as “I am not sure”. This position is roughly in line with the Cartesian rationalist model where evaluating a statement as true (i.e., agree) or false (i.e., disagree) rationally follows upon the recognition of the meaning (i.e., comprehension) of the statement (Descartes, 1644).
Alternatively, it is at least logically possible that the evaluative response and meta-cognitive response may be completely unrelated and independent. This would happen if these two assumptions were both true: (i) the meta-cognitive process and the evaluative process are separate and unrelated; and (ii) the meta-cognitive response is exclusively based on a meta-cognitive process and the evaluative response is exclusively based on an evaluative process.
Yet another position is that people may temporarily accept a comprehended idea as true, even though they may modify that decision subsequently, based on some additional evaluative processing (Gilbert, Krull, & Malone, Reference Gilbert, Krull and Malone1990; Gilbert, Tafarodi, & Malone, Reference Gilbert, Tafarodi and Malone1993; see also Sperber et al., Reference Sperber, Clement, Heintz, Mascaro, Mercier, Origgi and Wilson2010, for detailed discussion on this position). However, this position does not generate any specific prediction about what kind of evaluative response would be more prevalent to a statement that people claim not to understand beyond the proposal based on the Cartesian rationalist model described above, because the proposal does not specify the relation between a lack of comprehension and evaluation.
Finally, it is also possible that there is a common underlying cognitive process that results in cues used for both evaluative and meta-cognitive responses. Note that this possibility assumes that there is single underlying cognitive process, but it generates different types of cue in the course of processing (e.g., Dunlosky, Rawson, & Hacker, Reference Dunlosky, Rawson, Hacker, Otero, León and Graesser2002; Koriat, Reference Koriat1993) that can be used for meta-cognitive and/or evaluative judgments. If this is the case, there should be some systematic relation between evaluative and meta-cognitive responses because the cues used for the judgments, although different in nature, are based on one common underlying cognitive process, and therefore must be related in a specific way. This position is the one we are currently inclined to suggest as a working hypothesis. In particular, we hypothesize that the meta-cognitive and the evaluative response are positively related, such that people tend to disagree with a statement they do not understand. Note that this prediction is counter-intuitive and directly in conflict with the Cartesian rationalist view described above. We will explain the basis of this prediction shortly.
Before explaining the basis of our proposal, we would like to discuss the findings by Voss, Fincher-Kiefer, Wiley, and Silfies (Reference Voss, Fincher-Kiefer, Wiley and Silfies1993), because their work is one of the foundations of our hypothesis. Voss et al. asked participants to make judgments of both meaningfulness and agreement for controversial informal arguments. The findings indicated that judgments of both agreement (i.e., true) and disagreement (i.e., false) can be made as fast as judgments of meaningfulness (i.e., comprehension). This finding appears to be in direct conflict with the proposal that comprehension equals agreement, and disagreement involves extra processing, as proposed by Gilbert et al. (Reference Gilbert, Tafarodi and Malone1993, Reference Gilbert, Krull and Malone1990). A similar finding was also reported by Thomsen, Lavine, and Kounios (Reference Thomsen, Lavine and Kounios1996), who examined the reaction time (RT) for truthfulness judgments using a sentence verification task. These findings suggest that the processes of evaluation may be related or similar to comprehension, since people can provide evaluative responses as fast as meta-cognitive responses, even when they disagree.
In our opinion, Voss et al.’s (Reference Voss, Fincher-Kiefer, Wiley and Silfies1993) and Thomsen et al.’s (Reference Thomsen, Lavine and Kounios1996) findings, together with research demonstrating automatic attitude activation upon perceiving an attitude object in social cognition (e.g., Bargh, Chaiken, Govender, & Pratto, Reference Bargh, Chaiken, Govender and Pratto1992; Fazio, Reference Fazio, Maio and Olson2000), provide important clues about the relation between the comprehension and evaluation processes involved in the processing of assertive sentences. An assertive statement begins with the subject and proceeds to the predicate. We believe that the fact that processing begins with the subject is a key to understanding the relation between comprehension and evaluation.
Given that a subject is presented first in assertive statements, the processing of the subject is likely to activate the person’s belief in, attitude to, or other cognitive or affective attributes of the subject matter relevant to the situation and the recipient’s goal (Fazio, Reference Fazio, Maio and Olson2000). This, in turn, implies that subsequent processing of the predicate of the sentence occurs in the light of one’s belief about the subject, leading to an incidental comparison between what one believes about the subject on the one hand, and what a sentence actually conveys about the subject with the use of a predicate, on the other hand (see Ozuru, Mock, Bowie, & Kaufman, unpublished observations, for detailed discussion about a possible model), often without additional processing. For example, on the one hand, when a predicate agrees (i.e., coheres) with one’s belief about the subject, people are likely to experience processing facilitation. On the other hand, when the predicate disagrees (i.e., does not cohere) with one’s belief about the subject, people are likely to experience response competition (i.e., inhibition). In our opinion, the experience of processing facilitation or competition (i.e., inhibition) of the predicate based on the processing of the subject can serve as a cue for generating an evaluative response when the degrees of facilitation and inhibition are sufficiently discriminating.
According to this line of reasoning, to the extent that the processing of a subject involves the activation of one’s belief and knowledge about the subject, people should tend to perceive assertive statements as an argument in which one makes a claim about the relation between a subject and a specific predicate. This further means that a recipient is likely to express disagreement, even when they believe that they do not understand the meaning of the statement, provided that they have some knowledge of the subject of the sentence. That is, failing to recognize the predicate as a potentially coherent attribute of the subject that they know (which signals a lack of understanding) also means that the meaning of a sentence as a whole contradicts what a person believes about the subject without additional processing.
As an informal demonstration of this hypothesis, imagine that one is presented with an assertion, such as “1 plus 1 equals 10”. If asked to evaluate one’s agreement with this statement, many people would ‘disagree’ with this statement rather than providing a neutral response such as “Not sure”, because this clearly contradicts what many people believe to be true about the subject, which is “1 plus 1”. How would they respond, though, when they are asked to indicate their level of understanding of this statement? People may indicate “I do not understand”, instead of “I’m not sure” or “I understand”, because they simply fail to recognize an objective phenomenon that can be adequately referred to by the semantic content of the statement. Evaluative responses, such as “I’m not sure” or “I agree”, may be produced when people have multiple possible referents such that they find it difficult to decide which of the possible referents the sentence refers to. In this particular sentence, “I’m not sure” may only be produced if one has knowledge of the binary number system which produces the possibility of an alternative interpretation.
Thus, the thought experiment above raises the intriguing possibility that people tend to disagree with a statement they do not understand. The occurrence of this type of phenomenon is a logical possibility given the sentence processing model that we hypothesized. Given this interesting possibility, it is worth investigating whether such a phenomenon can be produced consistently across multiple stimulus statements in a formal research setting; production of this type of response pattern would lend support to our hypothesis on how evaluation and comprehension are related in the cognitive processing of a one-sentence assertion.
1.3. brief overview of the studies
We conducted three studies to examine these two issues: (i) What is the relation between the evaluative and the meta-cognitive response – in particular, what kind of evaluative response tends to be most frequently produced to a statement that people claim not to understand?; and (ii) whether the meta-cognitive (i.e., understand / do not understand) or the evaluative (i.e., agree/disagree) response tends to be the more prevalent response to one-sentence assertive statements when people have a choice of producing either a meta-cognitive or an evaluative response.
The first two studies explored the question of the relation between the two types of response (i.e., meta-cognitive and evaluative responses) by presenting various one-sentence statements that vary in their understandability, and asked participants to perform a meta-cognitive judgment and an evaluative judgment on the same set of statements in a repeated measures design by counterbalancing the order of the tasks. The difference between Studies 1 and 2 are the response formats which will be described in the ‘Methods’ section. Most importantly for our questions, these two studies will reveal whether people tend to consistently produce a negative evaluative response (i.e., disagree/false) to a statement that they claim not to understand.
The third study explored the question of whether people are likely to respond with a meta-cognitive (i.e., understand / do not understand) or an evaluative response (i.e., agree/disagree) when they have a choice. Thus, in Study 3, we presented the statements used in Studies 1 and 2 to participants and asked them to choose the most appropriate response from these four options: understand, do not understand, agree, disagree. We monitored the frequency of each type of response in relation to the understandability (as measured by the meta-cognitive response in Study 2) to examine whether the meta-cognitive or the evaluative response was more prevalent as a function of understandability. In addition, we monitored the response time to explore potential traces of how these four types of response are produced.
2. Study 1
2.1. method
2.1.1 Participants
Thirty undergraduate students from the University of Alaska Anchorage participated in the study in exchange for extra credit for an introductory psychology course. Age and gender of the participants were not recorded. All participants were proficient English speakers.
2.1.2. Design and materials
In this study, the independent variable was the type of task (i.e., meta-cognitive judgment vs. evaluative judgment). The dependent variable was the judgment rating in each of the two judgment tasks.
Stimuli. The stimuli were 126 one-sentence statements about various topics. The statements were divided into three categories, depending on the verb/modal used. One-third (42) of the statements contained the verb be, one-third (42) contained the verb need, and one-third (42) contained the modal should. The forty-two statements containing the modal should contained different types of main verb, including the verb be, depending on the statement. Because the modal should is polysemic, the stimuli with should contained several different meanings across the statements, such as ‘duty/obligation’ as in “Women should wear dresses”, and expectation, as in “The universe should be dark”, as well as an ambiguous combination between obligation and expectation in nonsense statements, as in “Paintings should be smelled”. We believe that the variation in the use of the modal may have affected participants’ comprehension of the statements. However, given that our goal was not to make any claim about the function of a modal, but rather to investigate the more general psychological phenomenon of how people process a variety of sentences, the inclusion of various sentences is both appropriate and necessary to diversify the nature of the statements. Other semantic content of the statements was not controlled across the three types of statement. The sentence length varied from three to ten words; no sentence contained negation. Some of the sentences were nonsense statements (e.g., “Sound is color” or “History needs light”) in order to diversify the understandability of the statements. Some sentences were also designed to be politically, religiously, or culturally provocative (e.g., “Democracy is an empty word”), while others were kept relatively neutral (e.g., “Chocolate is sweet”) in order to diversify the agreeableness of the statements. We acknowledge that the selection of the stimuli is in large part intuitive. We did not conduct a specific pilot study to control and monitor the understandability and provocativeness of the stimulus statements. We did, however, monitor the understandability of the statement at the time of analysis (see Studies 2 and 3), as will be described later. The complete list of stimuli is provided in the ‘Appendix’.
Apparatus. The 126 stimuli were presented on a notebook computer using the program SuperLab. The statements appeared in the center of the screen. The font was Tahoma, size 16. Each statement was accompanied by the response options below the statement. The response options for the judgment of understanding task were 1 = understand, 2 = understand but not clearly, 3 = do not understand, and 4 = do not understand at all. The response options for the judgment of agreement task were 1 = strongly agree, 2 = somewhat agree, 3 = somewhat disagree, 4 = strongly disagree. The response options were also presented in Tahoma font, size 16. The order in which the statements were presented was randomized across participants.
2.1.3 Procedure
Participants were tested individually in a quiet room. First, participants were presented with instructions appearing on the notebook computer. The instructions informed participants that they were going to read various one-sentence statements on the notebook screen. Participants were told to imagine their friend had uttered the statements. This part of the instruction was aimed at providing a communicative context, although minimal, so that participants perceived a statement as a communicative act as opposed to a list of sentences. This operation of providing a constraint on communicative context remained the same across all three studies we report in this paper. We are certainly aware that if we were to provide a different context (e.g., “Imagine a University Professor uttered …”) the results may be different, for example, due to politeness (Brown & Levinson, Reference Brown and Levinson1987) and other factors (e.g., assumption on knowledge). However, we believe that the question of how this type of specific communicative context influences the evaluative and the meta-cognitive responses and their relation is perhaps a more distant goal. Instead, we presently focus on the description of communicative behaviors in the most basic or common communicative context here as a first step. In the judgment of understanding task, participants were asked to indicate the extent of their understanding of each statement using one of the four number keys (see above). In the judgment of agreement task, they were asked to indicate the extent of their agreement with each statement in the same way. Participants were also told that the RT for each statement was going to be recorded, so they would concentrate on the task. Once they had read the instructions, they were asked to begin the task. If they had any questions, the experimenter answered them. We did not specifically instruct them on what we meant by ‘understanding’ and ‘agreeing’ and on how they should make their judgments on the two tasks on their degrees of understanding and agreement. This is because we wanted to observe how they used their own understanding of the meaning of understanding and agreement whatever they meant to each participant on the assumption that the behaviors emerging in this study reflect how they use these two responses (understand and agree) in common communicative situations (i.e., with a friend) without philosophizing on the meaning of these two responses. Once the participants had begun the task, the experimenter let the participants perform the task at their own pace without any interference. After the participants indicated that they had finished the task, the experimenter immediately presented the second task. The second task was identical regarding overall structure and procedure, but the response options were different. If participants had performed the judgment of understanding task first, they were asked to perform the judgment of agreement task next, using the exact same stimuli and set-up, and vice versa. During the instructions for the second task, the experimenter told participants that (i) the statements they are going to see will be the same, but (ii) this will be a different task, and therefore they should not think about the responses they had provided in the previous task. We are aware that merely telling participants not to think about the response they gave on the previous task may not be sufficient to prevent participants from doing so. However, given that our main interest was to determine whether people tend to provide negative evaluative responses to statements they claim not to understand, participants’ being aware of and consciously thinking about their previous responses will work against our prediction by choosing a response option that makes the relation between the evaluative and the meta-cognitive response rational (i.e., not sure as evaluative response for a statement that they claim not to understand). In this sense, we believe that this procedure does not cause any problems with the interpretation of the findings. After finishing both tasks, participants were debriefed and dismissed. None of the participants indicated that either task was difficult or strange. Typically, the procedure took about 20 minutes.
2.2. results and discussion
Table 1 presents the average frequency of the four types of evaluative response (i.e., strongly agree, somewhat agree, somewhat disagree, strongly disagree,) separately for different levels of understanding (i.e., clearly understand, understand but not clearly, do not understand, do not understand at all) that a participant indicated. That is, Table 1 provides information on how often participants provided each of the four evaluative responses separately for items that received each of the four different types of meta-cognitive response.
Frequency of the different evaluative response types separately for the different meta-cognitive response types averaged across participants in Study 1
There is a clear relation between the types of response in the evaluative judgment and the meta-cognitive judgment tasks. According to sign tests, for items that participants judged as clearly understand, the strongly agree response was significantly more frequent than the somewhat agree ( p < .01), the somewhat disagree ( p < .001), or the strongly disagree response ( p < .001). For items that participants judged understand but not clearly, the somewhat agree response was significantly more frequent than the strongly agree ( p < . 05), the somewhat disagree ( p < .05), or the strongly disagree response ( p < .05). For items that participants judged do not understand, the somewhat disagree response was significantly more frequent than the strongly agree ( p < .05) or the somewhat agree ( p < .05) response, but not significantly more frequent than the strongly disagree response ( p = .48). Finally, for items that participants judged do not understand at all, the strongly disagree response was significantly higher than the strongly agree ( p < .001), the somewhat agree ( p < .001), or the somewhat disagree response ( p < .01).
In sum, the evaluative response and the meta-cognitive response appear to be systematically related such that people tend to agree with statements that they understand, and disagree with statements that they do not understand. Second, the most extreme level of negative evaluative response (i.e., strongly disagree) tended to occur when they expressed that they did not understood a statement at all. Thus, people appear to be irrational in making evaluative judgments, contrary to the Cartesian rationalist model. These two findings confirm our prediction.
One concern is that people may have responded with disagree to a statement they claimed not to understand because the four response options were rather confusing, in particular between the do not understand and the do not understand all, and the somewhat disagree and the strongly disagree response options. In addition, it is also possible that they may have pressed an unintended key because the keys representing the response options were next to each other (i.e., number keys 1, 2, 3, 4). In order to address these concerns and replicate these findings, we conducted the second study using the exact same stimuli but modifying the response options to three response options in the meta-cognitive and the evaluative judgment task.
3. Study 2
3.1. method
3.1.1. Participants
Forty-two undergraduate students from the University of Alaska Anchorage participated in the study in exchange for extra credit for an introductory psychology course. Age and gender of the participants were not recorded. All participants were proficient English speakers. None of the participants in this study had participated in Study 1.
3.1.2. Design and materials
The design and the materials of this study is exactly the same as Study 1 except for the following two aspects. First, we created two evaluative judgment tasks, judgment of truth and judgment of agreement, to explore whether the response patterns vary as a function of these two tasks, addressing the question of whether the process of evaluation can be treated in a general way. Also, the addition of a judgment of truth condition addresses the issue of whether people clearly differentiated the meaning of agree (evaluation/acceptance) and understand (comprehension) in Study 1 (see Section 2.1.3). Second, we used three response options to allow a neutral response in this study for both the meta-cognitive and the evaluative judgment task. Thus, the response options for the meta-cognitive judgment task were: 1 = understand, 3 = not sure, 5 = do not understand; the response options for the evaluative judgment task in the judgment of agreement condition were: 1 = agree, 3 = not sure, 5 = disagree; and in the judgment of truthfulness condition were: 1 = true, 3 = not sure, 5 = false. This aspect of the design addresses two concerns in Study 1: (i) the three response options are intuitively clearer and more straightforward than the four options used in the Study 1; and (ii) the designation of the response keys minimizes the mistake of pressing an unintended response option. Participants were randomly assigned to the judgment of agreement and the judgment of truthfulness conditions of the evaluative judgment task.
3.1.3. Procedure
The procedure was identical to Study 1. The only difference was the content of the written instructions, modified due to the nature of the different response options.
3.2. results and discussion
As a preliminary analysis, we analyzed if there were any differences in the participants’ responses between the two conditions of the evaluative judgment task (i.e., judgment of agreement vs. judgment of truthfulness). The results indicated no difference, whether analyzed separately or together by collapsing the two types of evaluative judgment. Therefore, we report the results based on the analysis performed on the data collapsed across the two conditions.
Table 2 presents the average frequency of positive (i.e., agree/true), neutral (i.e. not sure), or negative (i.e., disagree/false) responses for the evaluative judgment task separately for the different levels of understanding (i.e., understand, not sure, do not understand) expressed in the meta-cognitive judgment task. Similarly to Table 1, Table 2 indicates how often participants provided each of the three evaluative responses (agree/true, not sure, disagree/false) separately for items that received each of the three different types of meta-cognitive response (i.e., understand, not sure, do not understand).
Frequency of the different evaluative response types separately for the different meta-cognitive response types averaged across participants in Study 2
As in Study 1, there is a clear relation between the types of response in the evaluative judgment and in the meta-cognitive judgment task. According to sign tests, for items that participants judged understand, the frequency of agree/true responses was significantly higher than not sure responses (p = .000) and disagree/false responses (p = .000), indicating that participants were more likely to respond with agree/true as opposed to disagree/false or not sure (agree) to items that they claimed to understand.
For items they judged not sure, the frequency of the not sure was significantly higher than the agree/true response (p = .002). Also the frequency of the disagree/false response was significantly higher than the agree/true response (p = .000). This indicates that participants were more likely to respond with either disagree/false or not sure to items they judged not understand. Thus, they were somewhat biased to provide a negative evaluative response to an item they judged not sure (understand).
Finally, and most importantly for our research question, for items that they judged do not understand, the frequency of the disagree/false response was significantly higher than the not sure response (p = .000) and the disagree/false and the agree/true response (p = .000), indicating that participants were more likely to respond with disagree/false as opposed to agree/true or not sure. These findings are consistent with Study1 and confirm our prediction.
In addition, in order to examine the relation between the evaluative and the meta-cognitive judgments in more detail, we performed an item-based analysis of the relation between the evaluative and the meta-cognitive responses for each participant. In order to perform this analysis, we computed Cohen’s kappa and Spearman’s rank order correlation to assess the correspondence between the evaluative and the meta-cognitive judgments provided by a given participant. Cohen’s kappa represents the degrees of exact matching of the three categories of response between the evaluation judgment (i.e., agree/true, not sure, disagree/false) and the meta-cognitive judgment (i.e., understand, not sure, do not understand) for each participant. The mean Cohen’s kappa across participants was 0.44, indicating that the exact matching is rather poor. Yet the mean Spearman rank order correlation across participants is 0.56, and the correlation was 0.5 or higher among approximately 70% of the participants. These findings indicate that the meta-cognitive and the evaluative judgments are closely related, such that people tend to express agreement with a statement they claim to understand, and to express disagreement with a statement they claim not to understand, even though their exact matching is poor.
Overall, across the two studies, the results are consistent. First, the findings indicate that evaluative and meta-cognitive judgments may be related such that people tend to agree with a statement they understand and tend to disagree with a statement they do not understand. Second, people are evaluative in processing one-sentence statements. According to Study 1, people expressed either complete agreement or disagreement in the four-option judgment task approximately 55% (i.e., strongly agree for 45 statements; strongly disagree for 24 statements) of the time. According to Study 2, people expressed either agreement or disagreement in the three-option judgment task approximately 84% (i.e., agree for 65 statements; disagree for 41 statements) of the time when they are allowed to choose a neutral option. Importantly for our question, the two studies indicated that people tended to disagree with a statement or judge a statement to be false even when they indicated that they do not understand the statement. Thus, the Cartesian rationalist model appears to be a poor fit with what people actually do when making an evaluative judgment of a sentence that they do not understand well.
However, the findings obtained from the two studies are limited as evidence to support the proposal that people are biased to commit to evaluative processing of a one-sentence assertion. This is because both Studies 1 and 2 asked people to make evaluative and meta-cognitive judgments on separate occasions without providing a choice between these two forms of judgment. As we discussed in Section 1, in ordinary communicative situations people have the choice of responding either with an evaluative response (i.e., agree/disagree or true/false) or a meta-cognitive response (i.e., understand / do not understand); they even can ask clarification questions. Thus, it is important to examine which of these two types of response people are likely to provide to a statement when they are given a choice of responding with either an evaluative or a meta-cognitive response.
In order to address this question of whether people tend to produce an evaluative response (i.e., agree/disagree) or a meta-cognitive response (i.e., understand / do not understand), we conducted a third study. In Study 3, participants were asked to choose one of the following responses: agree, disagree, understand, and do not understand, when reading the same 126 stimuli presented on a computer screen.
4. Study 3
4.1. method
4.1.1. Participants
Thirty-two undergraduate students from the University of Alaska Anchorage participated in the study in exchange for extra credit in an introductory psychology course. Age and gender of the participants were not recorded. All participants were proficient English speakers. None of the participants in this study participated in Study 1 and/or 2.
4.1.2. Design and materials
Apparatus and stimuli were the same as in Studies 1 and 2. The only difference was the response options. Participants were asked to indicate which of the following four responses were most appropriate: U = understand, NU = do not understand, A = agree, and D = disagree, which were recorded using the number keys 1, 3, 5, 7, respectively. The goal of this study was to observe whether or not there is systematic pattern or bias in people’s tendency to respond to a one-sentence assertion with either a meta-cognitive or an evaluative response when they have both options available. As we hypothesized, we expect a higher frequency of the evaluative response compared to the meta-cognitive response. We would like to emphasize that the methods used in this study are not intended to examine what specific cognitive processes (i.e., comprehension and/or evaluation) people perform in processing one-sentence assertions; we cannot directly infer cognitive processes based on the response option chosen. However, the method is capable of detecting any systematic response bias, and, as such, it is possible to make inferences about how people perceive a communicative situation involving one-sentence assertions with fairly limited contextual information.
4.1.3. Procedure
Participants were tested individually in a quiet room. The instructions for Study 3 were very similar to those for Studies 1 and 2. Participants were told to imagine that their friend uttered the statements, and asked to choose the response they thought to be most appropriate from the four options by pressing the corresponding key. As in Studies 1 and 2, participants were told that their reaction time was going to be recorded to ensure that they would concentrate on the task.
4.2. results and discussion
Table 3 presents the proportion of each of the four types of response and the response time conditionalized to the number of words in a sentence (ms/word) averaged across participants. First, we discuss the results on frequency of the types of response.
Proportion of the different response types and reaction times associated with each response type averaged across participants in Study 3
It is obvious that the majority of responses are evaluative, and account for 76% of responses. As noted earlier, a limitation of this analysis is that the tendency to respond with agree/disagree as opposed to understand / do not understand may be due to the nature of the specific stimuli used in this study; in other words, stimuli may be provocative, and hence may elicit an evaluative response.
In order to address this issue, at least in part, we performed the same analysis after classifying the statements into three different classes that differ in understandability based on the data obtained from Study 2. That is, we first performed an item-based analysis of the 126 stimuli to classify them into high, low, and medium understandability based on the judgment of understanding response obtained in Study 2. The reason we used data from Study 2 as opposed to Study 1 is that the three response options (i.e., understand, not sure, do not understand) in Study 2 allowed a more straightforward classification of the items. We performed the same analysis using the data obtained in Study 1, which indicated similar findings; therefore, we report the analysis based on the data obtained in Study 2. An item was classified as a high understandability item when two-thirds of the participants (22 or more) indicated that they understood the item. An item was classified as a low understandability item when two-thirds of the participants (22 or more) indicated that they did not understand the item. The remaining items were classified as medium understandability items.
This classification resulted in 86 (68%) high understandability items, 23 (18%) medium understandability items, and 17 (14%) low understandability items. Examples of high understandability items are: “Cars are useful”, “Drivers should pay attention”, and “Fire needs fuel to burn”, Examples of medium understandability items are: “Communism is dead”, “A serial killer needs more blood”, and “The universe should be dark”. Examples of low understandability items are: “Fish are mechanical”, “Triangles should be squares”, and “Trees need paper”.
After classifying the 126 statements into three groups of statements with different levels of understandability, we performed three sets of non-parametric sign tests comparing the proportion of the evaluative response and the meta-cognitive response to examine whether the evaluative response was more frequent. The analyses indicated that the proportion of the evaluative response (i.e., agree/disagree) was significantly larger than the proportion of the meta-cognitive response across high understandability items (M = .77; p = .000), medium understandability items (M = .70; p = .008), and low understandability items (M = .70; p = .001). This pattern of results suggests that people tend to respond with an evaluative response as opposed to a meta-cognitive response, irrespective of the understandability of a statement. In other words, people’s tendency to produce an evaluative response present for items that are difficult to understand.
In addition, we also wanted to monitor whether the tendency to be judgmental by choosing an evaluative response over a meta-cognitive response was influenced by the verb or the modal (i.e., be without a modal, need without a modal, or should) used in the statement. Therefore, we analyzed the frequency of the evaluative response separately for each person as a function of the verb/modal type, and then performed three sets of paired sample t-tests. These analyses indicated that, even though the overall frequency of the evaluative response was similar across be-statements (M = 31.41, SD = 7.95), need-statements (M = 30.34, SD = 8.34), and should-statements (M = 32.97, SD = 7.48), the frequency of the evaluative response was significantly higher for should-statements than for need-statements (t(31) = 3.63, p < .01). No other differences were statistically significant. Note that this finding cannot be taken as evidence to suggest that people’s tendency to be evaluative is stronger for should-statements because the other content of the statements was not controlled for. Instead, it merely provides evidence that overall, people’s tendency to select the evaluative response is not specific to a statement with a certain modal because, across all the three types of statement, people responded with the evaluative response to more than 30 out of 42 statements, which is equivalent to more than 70% of the time.
Second, we were also interested in the question of which of the two negative responses (i.e., meta-cognitive: do not understand; evaluative: disagree) increased more significantly when a statement becomes less understandable. If people are rational, as entailed by the Cartesian rationalist model, they should choose do not understand more frequently as an item becomes less understandable. However, results from Study 1 and 2, as well as the earlier analysis of Study 3, indicate that people may choose the negative evaluative response (i.e., disagree) over the negative meta-cognitive response (i.e., do not understand), even when they do not understand a statement. In order to test this prediction, we performed a 2 (meta-cognitive: do not understand vs. evaluative: disagree) × 3 (high, medium, vs. low understandability) ANOVA on the proportion of negative responses (see second and fourth columns of Table 4).
Proportion of the four response types as a function of item understandability averaged across participants in Study 3
The ANOVA indicated a main effect of the understandability of a statement (F(2,62) = 301.66, MSE = .01, p = .000, partial η 2 = .97), a main effect of the response type (F(1,31) = 41.09, MSE = .09, p = .000, partial η 2 = .57), and an interaction between understandability and response type (F(2,62) = 14.93, MSE = .02, p = .000, partial η 2 = .58). The main effect of understandability indicates that both negative evaluative (i.e., disagree) and meta-cognitive (i.e., do not understand) responses increase as the understandability of a statement decreases. The main effect of the response type indicates that, across the levels of understandability, the negative evaluative response (i.e., disagree) is more prevalent than the negative meta-cognitive response (i.e., do not understand). The significant interaction is particularly interesting because it indicates that the increase of the negative evaluative response (i.e., disagree) appears to be larger than the increase of the negative meta-cognitive response (i.e., do not understand) as the understandability of a statement decreases.
Similarly to the analysis above, we also examined which one of the two positive responses (i.e., evaluative: agree or meta-cognitive: understand) changed more significantly when the understandability of statements increased. Therefore, we performed a 2 (meta-cognitive: understand vs. evaluative: agree) × 3 (high, medium, vs. low understandability) ANOVA on the proportion of positive responses (see first and third columns of Table 4). This analysis indicated a main effect of understandability (F(2,62) = 269.5, MSE = .005, p = .000, partial η 2 = .90), a main effect of response type (F(1,31) = 21.9, MSE = .06, p = .000, partial η 2 = .41), and an interaction between understandability and response type (F(2,62) = 73.6, MSE = .01, p = .000, partial η 2 = .70). These results suggest that while the proportion of both types of positive response (i.e., meta-cognitive: understand; evaluative: agree) increase as the understandability of a statement increases, the increase was more significant for the proportion of the positive evaluative response (i.e., agree). Together these analyses indicate that changes in the understandability of a statement consistently affect the frequency of the evaluative response more significantly than the frequency of the meta-cognitive response.
Finally, we analyzed whether the evaluative response was faster than the meta-cognitive response. So far, the findings indicate that people tend to have a bias of responding with an evaluative response. If this bias reflects a default assumption/perception about an assertive statement, the generation of an evaluative response should involve less processing time than the generation of a meta-cognitive response, which would only be produced when participants find it inappropriate to respond with an evaluative response. In order to test this possibility, we computed response time (i.e., ms/word) by dividing the RT for each statement by the number of words in a sentence for each participant, to make the RT comparable across statements of different length. The second row in Table 3 contains the average RT across participants for each of the four response categories: understand, do not understand, agree, and disagree.
A 2 (direction: positive vs. negative) × 2 (response type: evaluative vs. meta-cognitive) ANOVA indicated a significant main effect of response type (F(1,24) = 25.7, MSE = 225123, p < .01, partial η 2 = .52). No other effects were statistically significant. This means that the evaluative response is faster than the meta-cognitive response irrespective of the direction of the response (positive vs. negative). This finding is in line with our prediction that the evaluative response as opposed to the meta-cognitive response is a default response.
Overall, people tend to choose the evaluative response more frequently than the meta-cognitive response, and it takes less time to produce an evaluative as opposed to a meta-cognitive response. Together these findings from Study 3 converge to show a strong tendency of people to respond with an evaluative response over a meta-cognitive response regardless of whether it is positive (i.e., agree) or negative (i.e., disagree), as measured by both response frequency and response time.
5. General discussion and conclusion
We conducted three quasi-experimental studies to examine these two descriptive questions: (i) What is the relation between the evaluative and the meta-cognitive response that people provide to an assertive statement?; and (ii) What kind of response (i.e., evaluative or meta-cognitive) are people likely to provide to an assertive statement when they have a choice of responding with either an evaluative or a meta-cognitive response?
As for the first question, the results of Studies1 and 2 both indicated a systematic relation between the meta-cognitive and the evaluative response. That is, people tend to agree with a statement they understand, and tend to disagree with a statement they either do not understand or feel not sure about their understanding. This pattern of the relation between the two responses in the two studies directly contradicts the Cartesian rationalist position (Descartes, 1644) that the evaluative response is dependent on and follows the outcome of the meta-cognitive judgment (i.e., people are only judgmental when they feel that they clearly understand a statement).
Instead, the results indicated that people are judgmental to the extent to commit to a negative (i.e., disagree/false) as opposed to a neutral (i.e., not sure) evaluative response even when they claim not to understand a statement. This, then, leads to the second question of whether people are likely to provide an evaluative response, even when they have the option of choosing between the meta-cognitive and the evaluative response.
The results of Study 3 indicate that people have a strong tendency and bias to respond with an evaluative response as measured by both higher frequency of and shorter response time for the evaluative as opposed to the meta-cognitive response. Further, this tendency is observed across items with different levels of understandability (i.e., low, medium, and high) and verb/modal. As we noted in Section 4.1.3, we provided minimal contextual information (i.e., “Imagine your friend has uttered this statement”) in these studies. As such, we are certainly aware that people’s bias to respond with the evaluative response may change as a function of communicative situation, such as where it takes place (classroom vs. informal exchanges), speaker (friend vs. professor), and other factors such as perceived level of knowledge, as well as politeness (Brown & Levinson, Reference Brown and Levinson1987). Therefore, the effects of these contextual factors need to be explored in future research.
We believe that these findings provide some important insights into larger, more fundamental, theoretical issues related to pragmatics and cognitive processing. First, in relation to the perspective of pragmatics, we believe that the finding that people are generally biased to respond with an evaluative as opposed to a meta-cognitive response provide useful information on how people infer speakers’ meaning.
We are inclined to believe that the observed bias of responding with an evaluative response is indicative of people’s general tendency to perceive the meaning of an assertive statement as a speaker’s argument about their belief about the subject in a sentence. Given that we do not have any comparison conditions manipulating contextual factors (e.g., status of speaker, conversational situation, etc), we cannot make any claim on the extent to which this tendency generalizes across different conversational situations. Yet, given that the bias to respond with an evaluative response is relatively strong across the three types of statement with different verbs/modals, even though the bias is slightly stronger in should-statements, it seems reasonable to conclude that the bias is not just due to the nature of the specific stimuli used in this study. This conclusion is further buttressed by the finding that the bias is present across statements with different levels of understandability.
The other useful piece of information in Study 3 is the response time data that shows people’s evaluative response is produced faster than the meta-cognitive response irrespective of negative or positive response. According to this response time data, it is difficult to believe that people’s evaluative response, which we interpret to be indicative of people’s perception of the speakers’ meaning of statements, is produced as a results of extensive processing. Instead, it appears that people almost automatically opted to produce an evaluative response, and they only responded with a meta-cognitive response when they found the evaluative response inappropriate for some reason.
Hence, one interpretation of the overall findings of Study 3 is that people have a systematic tendency to see this type of assertive sentence as most relevant when it is perceived as an argument, even when they do not have much contextual information beyond task instruction. At a more specific level, this finding lends some support to the notion that people’s reasoning or cognition generally functions to evaluate an argument, supporting Mercier and Sperber (Reference Mercier and Sperber2011) as well as Perelman and Olbrechts-Tyteca (Reference Perelman and Olbrecht-Tyteca1969).
This, then, leads to the second issue, namely how evaluation (acceptance) and comprehension are related to each other in terms of the cognitive processing of assertive statements (e.g., Sperber et al., Reference Sperber, Clement, Heintz, Mascaro, Mercier, Origgi and Wilson2010). Although these three studies do not provide enough information about exactly how people perform evaluation and comprehension in the course of processing assertive statements, the results enable us to infer the type of processing which is likely or unlikely to underlie the generation of an evaluative and a meta-cognitive response to the assertive statements.
To the extent that generating an overt response in the form of an evaluative or meta-cognitive judgment requires some judgment and decision-making, it is reasonable to assume that such decision-making would involve the use of certain information or cues (e.g., Dunlosky et al., Reference Dunlosky, Rawson, Hacker, Otero, León and Graesser2002; Koriat, Reference Koriat1993), generated in the course of processing the meaning of a sentence. If the cues used in the two different types of judgment (i.e., evaluative and meta-cognitive) are different and unrelated, then there should be no relation between the outcomes of the two types of judgment.
However, the results indicate that the outcomes of the two types of judgment are closely related, such that participants are more likely to respond with agree or true when they understand a statement whereas they are more likely to respond with disagree or false when they do not understand a statement, or are unsure about the meaning of a statement. This pattern of results leads us to speculate that the decision criteria or cues used in these two types of judgment are closely related, if not identical. How are they related?
First, the data obviously contradicts the Cartesian rationalist account that the evaluation process depends and follows the comprehension process. That is, given that people often responded with a negative evaluative response (i.e., disagree/false) to a statement that they claim not to understand, the relation between evaluative and comprehension processes is unlikely to be a serial one in which evaluation is performed on the outcome of a successful comprehension process, as entailed in the Cartesian rationalist model; they expressed an outcome of evaluation even when they expressed comprehension failure.
What about the proposal that comprehension involves agreeing/believing in the content of the statement, at least momentarily (Gilbert et al., Reference Gilbert, Krull and Malone1990, Reference Gilbert, Tafarodi and Malone1993)? This proposal was also discussed in detail by Sperber et al. (Reference Sperber, Clement, Heintz, Mascaro, Mercier, Origgi and Wilson2010), leading them to suggest that people may have a bias to believe in the content of the understood statement in the absence of contrary information, even though people are still capable of being vigilant about the epistemic truth of the content. Sperber et al.’s proposal is based on the more fundamental assumption of relevance theory (Sperber & Wilson, Reference Sperber and Wilson1995), where recipients must have an assumption that a statement that is addressed to them is relevant (meaningful and informative) at some level in order for any processing of the statement to occur, and further that a statement is understandable. Indeed, the aspect of the findings that people tend to agree with a statement they understand in Study 1 and 2 appears to lend support to the view advocated by Sperber et al. (Reference Sperber, Clement, Heintz, Mascaro, Mercier, Origgi and Wilson2010) as well as Gilbert at al. (Reference Gilbert, Krull and Malone1990, Reference Gilbert, Tafarodi and Malone1993). However, we believe that there are two issues to be mentioned in this regard.
First, methodologically speaking, it is possible that, for some reason, understandable statements in this study may also be generally agreeable due to their specific content. That is, even though there may be some general positive relation between the understandability and agreeability of a statement in general (i.e., across all the statements in the world; Sperber et al., Reference Sperber, Clement, Heintz, Mascaro, Mercier, Origgi and Wilson2010), a psychological tendency as proposed by Sperber et al. (Reference Sperber, Clement, Heintz, Mascaro, Mercier, Origgi and Wilson2010), our stimuli may have biased representation of the statements to increase such bias. Thus, we need to note that this finding may be in part due to the nature of the specific statements included in these studies; more studies are needed to address this issue.
The second issue, more important for theoretical reasons, is the question of whether and to what extent the finding (i.e., people tend to agree with a statement that they understand) should be interpreted as a support for the specific model proposed by Gilbert et al. (Reference Gilbert, Krull and Malone1990, Reference Gilbert, Tafarodi and Malone1993). One possible problem is that Gilbert et al.’s model involves two processes that are partly serial in relation. That is, comprehension needs to occur before rejection (i.e., disagreement), even though acceptance can occur simultaneously to comprehension, according to this model. Contrary to this proposal, the data in Studies 1 and 2 indicate that people often disagree with (this means reject) a statement they claim not to understand, according to responses indicated in the meta-cognitive judgment task. Therefore, contrary to the Gilbert et al. proposal, people often reject the epistemic status of a statement as false, even when they feel that they do not understand the meaning of the statement. To the extent that the Gilbert proposal does not include a process in which rejection can occur as fast as comprehension (or detection of comprehension failure), Gilbert et al.’s model cannot be accepted as it is as a comprehensive model of processing that captures the relation between comprehension and evaluation (acceptance), in our opinion.
Instead, the findings indicate that the comprehension and evaluation processes (can) occur simultaneously in a parallel as opposed to a serial fashion, even when they reject the truthfulness of a statement. In fact, Sperber et al. (Reference Sperber, Clement, Heintz, Mascaro, Mercier, Origgi and Wilson2010) hinted at the possibility that comprehension and acceptance can be parallel, even though they did not specify how that might happen. It is not entirely clear from their writing whether they meant that only acceptance can occur in parallel, as proposed by Gilbert et al. (Reference Gilbert, Krull and Malone1990, Reference Gilbert, Tafarodi and Malone1993), or that the process of acceptance including rejection can occur as well, because they did not describe how that might occur.
We believe that the sentence processing model we postulated in the Section 1 of this paper provides a possible interpretation for the finding. During the processing of an assertion, which are the stimuli used in these studies, a subject is presented first, which means that the subject is processed before the predicate. This means, to the extent that a person has some knowledge about the subject (as in this study, in contrast with Gilbert et al., Reference Gilbert, Tafarodi and Malone1993), that the processing of the subject is likely to invoke activation of the person’s belief, evaluation, attitude, and other cognitive and affective attributes of the subject matter (e.g., Swinney, Reference Swinney1979; see also Bargh et al., Reference Bargh, Chaiken, Govender and Pratto1992; Fazio, Reference Fazio, Maio and Olson2000, for literature on automatic activation of attitude) in relation to larger contextual factors of communication, such as knowledge of the speaker, communicative situations, etc. This in turn implies that subsequent processing of the predicate of the sentence needs to occur in the light of one’s belief about the subject, which is still an active part of one’s mental representation when the predicate is processed, leading to an almost immediate comparison between one’s (working) belief about the subject on the one hand, and what a sentence actually says about the subject with the use of a predicate, on the other hand (see Ozuru et al., unpublished observations, for detailed discussion about a possible model).
We believe this comparison can occur in the form of a subjective experience of the degrees of processing facilitation of the predicate based on the subject (priming; e.g., Bloem & La Heij, Reference Bloem and La Heij2003; La Heij, Dirkx, & Kramer, Reference La Heij, Dirkx and Kramer1990). According to this perspective, additional evaluative processing may not be required to generate an evaluative response, at least in some circumstances when the degree of facilitation/inhibition provides a sufficiently discriminatory cue to afford an evaluative judgment (see Ozuru et al., unpublished observations).
One problem or question based on this model is that it seems that do not understand and disagree mean essentially the same thing. That is, according to this view, do not understand and disagree seem to be completely interchangeable, which, as we already know, is not true based on both informal observation and the results of Studies 1 and 2. In particular, Study 2 showed that matching (exact agreement) between the evaluative and the meta-cognitive response is at best moderate, as indicated by Cohen’s kappa. Also, the results indicated that people sometimes disagree with a statement even when they expressed that they understood the statement. More interestingly, the results of Study 2 indicated that participants sometimes agreed with a statement they claimed not to understand, even though the occurrence of such a case is very rare.
These findings (i.e., a strong systematic relation without exact matching) might imply that: (i) the cues used for a judgment of evaluation and a judgment of comprehension are not exactly the same, even though they are likely to be based on or related to the same cognitive process; and/or (ii) not all the evaluation processes occur simultaneously with comprehension processes, even though these two processes may occur at the same time in many instances. For the first possibility, this research is not capable of providing any specific information on how two or more possible cues are generated from a single process of comprehension that can be used for the judgment of comprehension and evaluation. More research needs to be conducted.
As for the second possibility, we certainly acknowledge that there are some circumstances in which comprehension and evaluation occur in a serial manner. For example, when the degrees of processing facilitation/inhibition by the subject on the predicate are not strong enough to discriminate between a positive or a negative evaluation, additional processing may need to be carried out to make a judgment on the extent to which one agrees with a statement. Under such circumstances, the relation between comprehension and evaluation may resemble either the Cartesian rationalist model (i.e., understanding first, evaluation second) or the Gilbert et al. (Reference Gilbert, Krull and Malone1990, Reference Gilbert, Tafarodi and Malone1993) model. Therefore, the important implication of our proposal is that the relation between comprehension and evaluation of one-sentence-assertion may differ depending on the person’s active knowledge on the subject of the sentence in relation to the predicate.
While we believe the proposed model is consistent with the literature on sentence processing (e.g., MacDonald et al., Reference MacDonald, Pearlmutter and Seidenberg1994) and some findings on argument processing (Thomsen et al., Reference Thomsen, Lavine and Kounios1996; Voss et al., Reference Voss, Fincher-Kiefer, Wiley and Silfies1993), we acknowledge that it is largely speculative at this stage. As such, it requires further empirical testing. Yet we believe that the findings based on these three studies offer interesting preliminary insight into the pragmatics of how people infer speakers’ intention at a behavioral level and the possible cognitive processes underlying such behavior that can be used for further empirical investigations. In particular, as our discussion in this section indicates, this preliminary proposal on the relation between evaluation and comprehension processes help us link and integrate the two levels of descriptions, namely pragmatic (behavioral) and cognitive processing levels of analysis of language comprehension and use, as implicated by Sperber et al. (Reference Sperber, Clement, Heintz, Mascaro, Mercier, Origgi and Wilson2010).
Complete list of stimulus statements
Democracy is an empty word.
Republicans are conservative.
Blue is hot.
Life is too long.
Money is evil.
Men are dogs.
Religion is harmful.
Science is wrong.
Communism is dead.
The universe is infinite.
Nature is dead.
Humans are machines.
Language is thought.
Politics is physics.
Color is light.
Fish are mechanical.
Cars are useful.
Airplanes are dangerous.
Running is healthy.
Statistics are an empirical science.
Psychology is art.
History is story.
Time is relative.
Light is made of particles.
Sound is made of particles.
The brain is meat.
Memory is inaccurate.
Thought is computation.
Chocolate is sweet.
Money is dirty.
Weight and mass are the same thing.
Coffee is poisonous.
Smoking is pollution.
The Constitution is just.
Government is complex.
Afghanistan is an enemy.
War is natural.
Sound is color.
Alaska is a nation.
Taste is visible.
Fighting is writing.
Cats are magic.
One plus one should be three.
Life should be longer.
Government should be simpler.
There should be more fish in the ocean.
Teachers should make more money.
Education should be useful.
People should work hard.
People should go to warm places in winter.
Everyone should have a computer.
The sun should rise tomorrow morning.
People should be worried about earthquakes.
We should act to save the polar bears.
Universities should hire more teachers.
There should be a professional baseball team in Anchorage.
Newspapers should report only the truth.
The government should increase taxes.
People should have health insurance.
Water should freeze below 32 degrees F.
The sky should be blue when it is clear.
The earth should be cooler.
Siblings should get along.
Criminals should be executed.
Drivers should pay attention.
People should grow their own food.
Women should wear dresses.
The financial industry should be regulated.
The oil should be untouched.
The mountain should be wild.
Money should be made.
God should save the earth.
The universe should be dark.
Thought should be visible.
Reason should prevail.
The president should be decisive.
Paintings should be smelled.
People should have tales.
Triangles should be squares.
Objects should fall up.
Democrats should vote for republicans.
Taxes should be free.
Men should give birth.
Rocks should grow.
We need more rain.
The earth needs clean energy.
The economy needs consumption.
Governments need taxes to run.
People need the mercy of God.
Nature needs balance.
Fire needs fuel to burn.
Men need women.
Evolution needs reproduction.
The moon needs the earth’s gravity to stay in orbit.
Extroverts need friends.
Students need computers.
Librarians need silence.
Teenagers need to get more sleep.
Women need to eat chocolate.
The homeless need to get jobs.
People need good morals.
We need to buy organic vegetables.
People need to lose more weight.
Life needs juice.
The world needs thoughts.
People need reasons.
History needs light.
Medicine needs more money.
A university needs students.
Roads need repairs.
Smokers need chewing gum.
Air needs oxygen.
A serial killer needs more blood.
Communication needs a medium.
Coffee needs milk.
Wine needs cheese.
Light needs darkness.
Women need more power.
People need wings.
Animals need shoes.
Governments need anarchy.
Space needs time.
Religion needs mathematics.
Politicians need saints.
Cars need ears.
Trees need paper.
