With some “increasingly rare” exceptions, the tasks used to investigate human thinking have multiple normative readings. Hence, one cannot establish whether they elicit correct or incorrect answers. This is the main argument used by Elqayam & Evans (E&E) against the normative evaluation of thinking performance. One problem with this argument is that some experimental paradigms for investigating human thinking have just one normative reading. E&E examine one class of these paradigms (i.e., the conditional syllogisms) and claim that it is actually subject to normative dispute. Thus, E&E conclude that sooner or later alternative normative systems will be proposed for any thinking paradigm.
There is reason to doubt this conclusion. Consider the typical estimation tasks wherein respondents judge the total or relative frequency of the items of a given class (e.g., Tversky & Kahneman Reference Tversky and Kahneman1973). Since there is an objective yardstick for enumerating the items (e.g., men in a list of people, words that begin with r), these tasks have one objectively correct answer. In some cases, respondents do not produce that answer. For example, Tversky and Kahneman's respondents judged that the class of men in a list was more numerous than the class of women, when in fact the list contained 19 names of men and 20 names of women. Accordingly, one has to conclude that these respondents erred. And such a conclusion will hold until some alternative account proves that judging a class of 19 items greater than a class of 20 items is normatively correct. E&E may be right in claiming that paradigms of this kind are becoming “increasingly rare” (target article, sect. 3, para. 3). The existence of such paradigms, however, shows that investigating human thinking is not destined to use tasks with competing normative readings, and that evaluating the normative status of a given judgment is not always open to dispute.
E&E correctly insist that one should understand what individuals are doing and why, rather than “discussing what they are not doing” (sect. 8, para. 2, authors' emphasis). The latter motivation may have driven some of the studies that have documented biases in human thinking. Other studies of this sort, however, have investigated the limits of human thinking processes in order to understand them. For example, consider Tversky and Kaheman's studies mentioned above. They were not aimed to simply reveal the faulty nature of respondents' estimations, but instead tested the hypothesis that the ease of recall of instances affects frequency and probability judgments in tasks for which there is a normative standard (e.g., judging the frequency of men in a list), as well as in tasks for which there is no such standard (e.g., judging the probability that a given depressed patient will commit suicide). Quite ironically, E&E contrast the controversial nature of thinking studies with the undisputed nature of memory studies, and indicate the “acute” problems derived from adopting memory paradigms to thinking research. Yet, as Tversky and Kahneman's studies show, memory search is the basis of many judgmental activities, and investigating recall processes may shed light on thinking processes.
E&E concede that in some cases one is entitled to evaluate thinking performance: “If your objective is to improve thinking (rather than to understand it), then you must have criteria for distinguishing good thinking from bad” (sect. 5.2, para. 5). E&E refer to individuals who behave against their interests, such as pathological gamblers. They argue that one has to help these individuals by modifying their wrong beliefs and teaching them the rules of probability calculus. According to E&E, such an instrumental approach is necessary in applied research but totally inappropriate in “basic theoretical research” (sect. 8, para. 2). The point is that even basic research has discovered individuals who need some normative help. Consider respondents who bet on the conjunction of events A&B, rather than on event A (Tversky & Kahneman Reference Tversky and Kahneman1983). If you follow E&E's recommendation, you should refrain from judging respondents' bets. Yet, these respondents behave against their interests. They miss a chance of winning the bet, the one in which the conjunction of events A¬-B occurs. Therefore, you should inform them that they have made a bad decision. In doing so, you employ a normative standard; that is, you inform respondents that they do not conform to basic norms of probabilistic calculus. In sum, even basic research may force you to evaluate respondents' performance and to improve it by means of normative guidance.
Besides applied domains, there is an entire domain of basic research, neglected by E&E, wherein evaluating thinking performance is inevitable. According to E&E, when respondents have to evaluate a posterior probability, in order to “get the problem right” (sect. 6, para. 5, emphasis added), respondents need to reason about frequencies or to learn Bayes' rule. This claim is inaccurate, since respondents, including preschoolers, may solve this problem without reasoning about frequencies (Girotto & Gonzalez Reference Girotto and Gonzalez2001; Reference Girotto and Gonzalez2008). Preschoolers, of course, are not familiar with the rules of probability calculus. Yet, at around the age of five, they solve this sort of problem. Before that age they fail to, and after that age their performance improves. This example is relevant because it concerns a problem for which even E&E accept that there is only one right solution. However, the entire investigation of the development of thinking processes speaks in favor of normative evaluation. If one does not use normative standards, how could one compare the answers produced by children of different ages? More generally, how could one claim that children's thinking processes improve (or, for that matter, worsen; see Noveck Reference Noveck2001), if one does not have normative standards to assess them?
E&E are probably right in claiming that some psychologists say, “Respondents should not think this,” in the same evaluative sense in which they say, “Poverty should not exist.” Yet, there are cases, like the ones mentioned above, in which psychologists are entitled to say, “Respondents should not think this,” in the same evaluative sense in which they say, “You should not eat this.”
With some “increasingly rare” exceptions, the tasks used to investigate human thinking have multiple normative readings. Hence, one cannot establish whether they elicit correct or incorrect answers. This is the main argument used by Elqayam & Evans (E&E) against the normative evaluation of thinking performance. One problem with this argument is that some experimental paradigms for investigating human thinking have just one normative reading. E&E examine one class of these paradigms (i.e., the conditional syllogisms) and claim that it is actually subject to normative dispute. Thus, E&E conclude that sooner or later alternative normative systems will be proposed for any thinking paradigm.
There is reason to doubt this conclusion. Consider the typical estimation tasks wherein respondents judge the total or relative frequency of the items of a given class (e.g., Tversky & Kahneman Reference Tversky and Kahneman1973). Since there is an objective yardstick for enumerating the items (e.g., men in a list of people, words that begin with r), these tasks have one objectively correct answer. In some cases, respondents do not produce that answer. For example, Tversky and Kahneman's respondents judged that the class of men in a list was more numerous than the class of women, when in fact the list contained 19 names of men and 20 names of women. Accordingly, one has to conclude that these respondents erred. And such a conclusion will hold until some alternative account proves that judging a class of 19 items greater than a class of 20 items is normatively correct. E&E may be right in claiming that paradigms of this kind are becoming “increasingly rare” (target article, sect. 3, para. 3). The existence of such paradigms, however, shows that investigating human thinking is not destined to use tasks with competing normative readings, and that evaluating the normative status of a given judgment is not always open to dispute.
E&E correctly insist that one should understand what individuals are doing and why, rather than “discussing what they are not doing” (sect. 8, para. 2, authors' emphasis). The latter motivation may have driven some of the studies that have documented biases in human thinking. Other studies of this sort, however, have investigated the limits of human thinking processes in order to understand them. For example, consider Tversky and Kaheman's studies mentioned above. They were not aimed to simply reveal the faulty nature of respondents' estimations, but instead tested the hypothesis that the ease of recall of instances affects frequency and probability judgments in tasks for which there is a normative standard (e.g., judging the frequency of men in a list), as well as in tasks for which there is no such standard (e.g., judging the probability that a given depressed patient will commit suicide). Quite ironically, E&E contrast the controversial nature of thinking studies with the undisputed nature of memory studies, and indicate the “acute” problems derived from adopting memory paradigms to thinking research. Yet, as Tversky and Kahneman's studies show, memory search is the basis of many judgmental activities, and investigating recall processes may shed light on thinking processes.
E&E concede that in some cases one is entitled to evaluate thinking performance: “If your objective is to improve thinking (rather than to understand it), then you must have criteria for distinguishing good thinking from bad” (sect. 5.2, para. 5). E&E refer to individuals who behave against their interests, such as pathological gamblers. They argue that one has to help these individuals by modifying their wrong beliefs and teaching them the rules of probability calculus. According to E&E, such an instrumental approach is necessary in applied research but totally inappropriate in “basic theoretical research” (sect. 8, para. 2). The point is that even basic research has discovered individuals who need some normative help. Consider respondents who bet on the conjunction of events A&B, rather than on event A (Tversky & Kahneman Reference Tversky and Kahneman1983). If you follow E&E's recommendation, you should refrain from judging respondents' bets. Yet, these respondents behave against their interests. They miss a chance of winning the bet, the one in which the conjunction of events A¬-B occurs. Therefore, you should inform them that they have made a bad decision. In doing so, you employ a normative standard; that is, you inform respondents that they do not conform to basic norms of probabilistic calculus. In sum, even basic research may force you to evaluate respondents' performance and to improve it by means of normative guidance.
Besides applied domains, there is an entire domain of basic research, neglected by E&E, wherein evaluating thinking performance is inevitable. According to E&E, when respondents have to evaluate a posterior probability, in order to “get the problem right” (sect. 6, para. 5, emphasis added), respondents need to reason about frequencies or to learn Bayes' rule. This claim is inaccurate, since respondents, including preschoolers, may solve this problem without reasoning about frequencies (Girotto & Gonzalez Reference Girotto and Gonzalez2001; Reference Girotto and Gonzalez2008). Preschoolers, of course, are not familiar with the rules of probability calculus. Yet, at around the age of five, they solve this sort of problem. Before that age they fail to, and after that age their performance improves. This example is relevant because it concerns a problem for which even E&E accept that there is only one right solution. However, the entire investigation of the development of thinking processes speaks in favor of normative evaluation. If one does not use normative standards, how could one compare the answers produced by children of different ages? More generally, how could one claim that children's thinking processes improve (or, for that matter, worsen; see Noveck Reference Noveck2001), if one does not have normative standards to assess them?
E&E are probably right in claiming that some psychologists say, “Respondents should not think this,” in the same evaluative sense in which they say, “Poverty should not exist.” Yet, there are cases, like the ones mentioned above, in which psychologists are entitled to say, “Respondents should not think this,” in the same evaluative sense in which they say, “You should not eat this.”