R2.1. Soft normativism and the is-ought problem

Supporters of normativist research programmes often argue that the normativist and descriptivist questions cannot be disentangled. We believe this arises mostly from equating norm with competence, which we discuss in section R2.2. However, we observe that a number of commentators, including some who focus on important psychological questions in their own research, support a mixed model in which descriptivism and normativism go side by side, each contributing usefully to the other (Achourioti, Fugard, & Stenning [Achourioti et al.]; Douven; Fuller & Samuels; McNair & Feeney; Nickerson; Quintelier & Fessler; Schurz; Stupple & Ball; Weinberg). We will dub this position “soft normativism.” According to this view, (a) normativism inspires research questions or whole paradigms that provide important descriptive findings, while (b) such findings feedback into the design and construction of normative theories. We deal with (a) later in section R7, but (b) still seems to leave us with the problem of is-ought inference. How can this be solved?

One suggestion, from Brase & Shanteau, is to adopt Platt's strong inference strategy (Platt Reference Platt1964). They claim that this method will “go a very long way towards avoiding the problems associated with prescriptive normativism” but they fail to explain how. Strong inference is, in our view, a very difficult research strategy to adopt, as people naturally focus on one hypothesis at a time in their hypothetical thinking (Evans Reference Evans2010b). But even if researchers can be persuaded to pursue multiple hypotheses, based on multiple normative systems, how will this help? If we end up adjudicating between normative systems on the basis of behaviour then we are certainly going from “is” to “ought” no matter how many we consider.

The two main types of solution proposed by our commentators are (a) to avoid is-ought inference by attempts to bridge the “Humean bifurcation” (Frankena Reference Frankena1939) between the “is” and the “ought,” or (b) to adhere to the Humean bifurcation, proceeding instead from given first principles or values. Schurz seems to support a first principles solution: If we accept a small number of fundamental norms as given extra-scientifically, we can then draw on empirical evidence to infer derived norms for a practically-applied psychology. He proposes truth conduciveness as such a fundamental epistemic norm. The solution is close to the one we offered for applied science in section 8 of the target article, and we welcome Schurz's emphasis on acknowledging the extra-scientific nature of fundamental norms. However, we have our doubts about truth-conduciveness as a candidate for a fundamental norm, as we discuss in section R3.2.

In contrast, bridging solutions to the is-ought problem identify constructs (concepts, principles, postulates) which combine both descriptive and evaluative/deontic elements, and allow evaluative conclusions to be derived from them (and them only). For example, as Searle (Reference Searle1964) famously argued, if Jones promised to pay Smith $5, then Jones ought to pay Smith $5. The promise, a deontic speech act, combines the descriptive linguistic fact with the deontic, value-laden social obligation. We cannot infer “ought” from pure “is,” but we can infer “ought” from “is-with-ought,” and bridge constructs supply us with the latter. Such solutions are as prevalent among our commentators (Nickerson; Pfeifer; Quintelier & Fessler; Sun & Wang; and, more implicitly, Stupple & Ball and perhaps Douven) as they are in the literature (e.g., Frankena Reference Frankena1939; Searle Reference Searle1964; Williams Reference Williams1985; but cf. Schurz Reference Schurz1997). The commentaries draw on a wide variety of bridge concepts, from “cannot implies not-is” (Quintelier & Fessler) to the relations between constitutive and regulative norms (Achourioti et al.)

Buckwalter & Stich discuss the influence of Goodman's (Reference Goodman1965) reflective equilibrium, brought by Cohen (Reference Cohen1981) into the rationality debate. The idea is that norms and psychological evidence justify each other by being brought into coherence, creating a “virtuous” circle of justification. We note that reflective equilibrium is itself a bridge solution – it presupposes that such coherence between norms and behaviours is possible. Anderson's six-step procedure of rational analysis (Hahn; Oaksford & Chater) is a derivative of reflective equilibrium: It seeks to establish coherence between behavioural observation and “optimal” (note the evaluative term) behavioural function. The Kantian “ought implies can” is a classic bridge construct, and so is its contrapositive “cannot implies not-ought” (Quintelier & Fessler). Similarly, Simon's (Reference Simon1982) bounded rationality (Gold, Colman, & Pulford [Gold et al.]; Hahn; Nickerson), the idea that rationality should be judged relative to biological and cognitive limitations, derives from “cannot implies not-ought” (and see sect. R2.4). Another bridge construct is Davidson's (Reference Davidson1973) “principle of charity” (Gold et al.; Hahn), the idea that behaviour should be interpreted in a way that makes the agent appear rational (i.e., can implies ought).

The viability of bridge solutions has been debated for the better part of a century, and we will not solve it here one way or the other. As we show in section R6.2, however, bridge solutions preclude universal or absolute normative evaluation, leaving the more radical type of normativism vulnerable to internal inconsistencies.

R2.2. Normative theory ≠ competence theory

Both Buckwalter & Stich and Fuller & Samuels agree that maintaining the distinction between competence theory and normative theory is important, but Buckwalter & Stich suggest that we have underestimated how difficult it could be to undermine the view that equates them. Alas, they are right. Despite our best efforts to the contrary (see sections 4 and 7 of the target article), many commentators persist either explicitly (Foss; Pothos & Busemeyer) or implicitly (Achourioti et al.; Pfeifer; Sun & Wang) in drawing the parallel. A special case of this is commentaries that propose or discuss Bayesianism as a normative system (Stanovich; McNair & Feeney), a position we discuss in section R5.2.

Buckwalter & Stich trace the origins of the normative–competence equation to reflective equilibrium (see sect. R2.1), and offer a persuasive counter-argument: Since psychological inference is subserved by two distinct systems (“minds” in our preferred terminology – see R4), there is no single coherent solution (and see R6). Another potential source of confusion seems to be that computational theories in reasoning and decision making often double as normative theories. One commenting strategy (McNair & Feeney; Pfeifer; Pothos & Busemeyer; Sun & Wang) is therefore to showcase a particular such theory or system, argue for its descriptive adequacy, and from that infer that normativism is indispensible. But this is not enough: such a strategy only demonstrates that the theory in question is a good computational-level (i.e., competence) theory.

The separation of norm and competence is, of course, crucial for our argument. To “de-normatise” competence theories, Buckwalter & Stich's argument from dual processing provides us a strong first step. We clarify again that both levels, competence-computational (what) and algorithmic-processing (how), are in our opinion indispensible to a descriptivist psychology of reasoning. A descriptivist psychology is not just about processing; and conversely, normativism does not hold a monopoly on computational analysis. On the contrary, we argue that it is theorists who insist on equating norm with competence who drive an unjustified wedge between computational and processing analysis. For us, the what stops short of the (evaluative) ought – that the system computes a specific function does not mean that this function is normatively justified. To claim otherwise is to invite an is-ought inference.

R2.3. The role of formal theories

Both Pfeifer and Nickerson express concerns about our depiction of the relation between formal and psychological theories, believing that we had shown an asymmetric relationship, in which the main flow appears to be top-down. Such was not our intention. On the contrary, we argued, following Marr (Reference Marr1982), that the main type of relationship, constraints on theory construction, goes both ways. We are more than happy to concede that the same goes for inspiration. However, we point out that inspiration equally holds when formal theories are construed as purely computational. By reclaiming computational analysis for descriptivism, all we have to lose by giving up normativism is the evaluative function (and, of course, several dubious inferences and a handful of research biases).

We also referred to “formal systems” or “formal theories” as the level relating to computational analysis. Whereas commentators tended to agree with our emphasis on the non-instrumental, fundamental, and universal evaluative nature of normativism, some challenged our assumptions about normative theories as formal systems (Schurz; Weinberg). Historically, psychological theories of higher mental processes have taken their normative standards from formal systems such as classical logic or Bayesianism, but we agree with Schurz and Weinberg that neither formality nor systematicity is crucial. [As an aside, this would obviate the need to define what a formal system means (Spurrett), which we have only done by extensional definition (i.e., by providing examples).] Instead, we propose that the crucial feature is the a priori nature of a principle (Pothos & Busemeyer; Schurz). The same argument we applied to formal systems is still relevant here: a priori principles can have a role in a computational level analysis, but cannot assume a role of fundamental value and yet be supported by empirical evidence. In practice, of course, computational level analyses will generally have to draw upon formal systems for their precision and detail.

R2.4. Distinguishing types of rationality

Lastly for this section, we reopen our classification of other types of rationality. Gigerenzer's ecological rationality, the idea that thinking must be adaptive, proved rather challenging to evaluate. In the target article we classified it as descriptive, albeit with some misgivings, noting its “is-ought” inference, and the weakness of avoiding computational-level theories. (Incidentally, the latter was why – contrary to Hrotic's evaluation – we never presented ecological rationality as the best way to conduct descriptivist research.) Thus, we can agree with Schurz that the “ought” of ecological adaptation seems to lie in-between an instrumental and evaluative ought, somewhere in the fuzzy chain connecting instrumental values to increasingly fundamental values (sect. R2); as Hrotic noted, Gigerenzer may simply be replacing one norm with another.

Commentators have agreed with us that, unlike other evaluative oughts in the literature, the ecological ought is informal rather than formal (Pothos & Busemeyer; Schurz; Weinberg); however, Schurz and Weinberg point out that this need not preclude its classification as normativist, and we have already conceded (in sect. R2.3) that formality is unnecessary to define normativism. Moreover, prototypical normativism tends to be absolute and universal, and, again, ecological rationality proves difficult to evaluate on that score. While Pothos & Busemeyer and Schurz emphasise the non-universal nature of individual heuristics, Weinberg argues, convincingly to our mind, that true instrumental rationality must be sensitive to the relative context of the agent and her goals, whereas ecological rationality authors often discuss “fit” or “success” in general and evaluative terms. All in all, the weight of evidence seems to point in favour of reclassifying ecological rationality as normativist. At least, as Schurz argues, it is no less normativist than rational analysis.

Brase & Shanteau argue that massive modularity is a better descriptivist system than hypothetical thinking theory (HTT), due to its sensitivity to evolutionary principles. This is beside the point of our article, which was not to promote any particular descriptive account: we referred to HTT only as an example of a descriptive theory which nevertheless still draws on formal theories for its computational level analysis. It is, however, debatable whether massive modularity, grounded in evolutionary rationality, is a truly descriptive account. As Weinberg discusses, ecological rationality cannot be seen as instrumental in the absence of sensitivity to the desires of individual agents, and genetic goals can dramatically clash with the goals of the individual (see Stanovich Reference Stanovich2004, for a detailed discussion). So the assumption on a priori principles that behaviour ought to be adapted to the environment in which an organism operates, is arguably normative, as noted above.

Last, we refer to Simon's (Reference Simon1982) bounded rationality, in itself an uncontroversial and widely accepted notion, which several commentators refer to as a potential basis for normative analysis (Gold et al.; Hahn; Nickerson). We have judged bounded rationality to be unproblematic in terms of normativism, and we see no reason to retract this. We see bounded rationality as open to a range of interpretations. With its emphasis on costs and benefits of cognitive effort, bounded rationality comfortably sits with instrumental rationality. Bounded rationality can also serve as a basis for soft versions of normativism discussed above. However, since bounded rationality means that relative considerations of cognitive and biological limits need to be taken into account, it can never fit with the more radical types of normativism, which rely on universal and absolute norms.

R3.1. Epistemic norms

Our critique focused specifically on the psychological study of higher mental processes (although some of it might be applicable to empirical philosophy as well). Some of the commentators, however, have addressed the broader issue of epistemic norms and their justification (Dutilh Novaes; Proust; Quintelier & Fessler; Schurz). To clarify, we never argued that epistemic norms are irrelevant for reasoning – only that it is fruitless for psychological science to focus on norms, especially when it actively interferes with the business of conducting research. As long as norms are kept apart from empirical evidence (e.g., by acknowledging, as Schurz proposes, their extra-scientific status), for us there is no issue.

Several commentators have also discussed the role of epistemic norms as constitutive rules justifying is-ought inference within the system they define, linking the constitutive function to the regulative one either explicitly (Achourioti et al.) or implicitly (Fuller & Samuels; Proust; Waldmann). Constitutive rules are definitional: they are what the system is about; regulative rules are deontic: they regiment behaviour. The classic example is the constitutive rules of chess, which define what chess is as much as they regulate it. If chess rules are violated, one is no longer playing chess. Table manners, in contrast, are purely regulative, as eating exists independently. Searle (Reference Searle1964) argued that inference from is to ought can be justified for constitutive rules only and Achourioti et al., at least, readily accept the moderate relativism it imposes (see also Stenning & van Lambalgen [Reference Stenning and van Lambalgen2008], and our discussion of bridge solutions in sect. R2.1). However, we cannot accept Achourioti et al.'s proposal that “without an ‘ought’ there can't be an ‘is’ in the first place” (the “ought” in question being constitutive ought). Constitutive rules define what an institution is, not what it ought to be. Again, we take our example from language. Word order is a constitutive rule; “kicked Jack ball the” is not a poor sentence, it is not a sentence at all. This is a descriptive fact, not an evaluative judgement. In contrast, double negation is a purely regulative rule: “I don't know nothing” is a sentence, albeit a non-normative one and condemned by linguistic purists. As this example clarifies, constitutive rules are competence rules, and therefore descriptive, not normative. Their “ought” is an epistemic ought rather than a deontic one.

R3.2. Epistemic rationality and normativism

A traditional view of epistemic rationality is that it is truth-seeking and several of our commentators believe that we need a normative account of it for this reason (Douven; Fuller & Samuels; Proust; Spurrett; Stanovich; Waldmann). As Dutilh Novaes observes, this reflects a Kantian tradition of logicist philosophy that has been very influential in psychology and still is, judging by some of the commentaries. Waldmann, for example, tells us that people distinguish between true and false beliefs, rejecting the false ones when evidence requires it – in the spirit of Popper, it seems. This truth-seeking notion of epistemic rationality needs to be complemented by a truth-preserving form of deductive inference (Schurz) in a traditional logicist theory of reasoning (and science). However, the latter has already been largely dispensed with by psychologists pursing the “new paradigm” psychology of reasoning (sect. R5), so why not the former? For example, if we think of people as Bayesian, then we might expect them to hold all propositions with an infinitely variable degree of belief. Beliefs may not rationally be held with certainty, however: a prior probability of 0 or 1 remains fixed and cannot be revised in the light of new evidence. Hence, Bayesian theorists require both a different normative and computational account of belief acquisition than do logicists – another example of the multiple-norm problem.

However, we reject the idea that we need a normative account of belief at all. Our view is that cognitive representations are viewed not as veridical, but as being fit for purpose. They reflect the evolutionary history and instrumental needs of the organism. Take the case of the visual system. It is fallacious to think that there is a “true” picture of the world which our eyes and brains deliver faithfully to us. There is a mass of information in light, which could be interpreted and constructed in many ways. In addition, our visual systems have clear limitations. Some snakes have evolved the ability to see infra-red light because they hunt at night and need to sense heat. We have no such facility. To take another example, our episodic memory system is designed not to preserve memories but to forget most of them rapidly after a relatively short interval unless they have great personal significance: This is adaptive, as generally we need access to the most recent memories which should not be confused with earlier ones. Our point is that cognitive representations are only veridical to the extent and in the manner required to serve our goals. Using the terms in a descriptive psychological sense, “epistemic rationality” is subservient to “instrumental rationality,” and an account of belief-generating systems is part of the computational account of human behaviour that cognitive science seeks. As Anderson (Reference Anderson1978) pointed out some years ago, all computational accounts of cognitive functions constitute representation-process pairs. Hence, the problem of cognitive representations is mopped up in the computational-level accounts that cognitive scientists provide for instrumentally rational behaviours.

R3.3. Self-knowledge and self-regulation

Proust argues that epistemic norms are used by reasoners to regulate their own performance; Waldmann similarly claims that people view themselves as rational agents and act accordingly. Achourioti et al. make a similar claim but on a relativist basis: There can be no single norm since participants switch between systems and goals. Achouriori et al. conclude that a descriptivist account without norms is thus doomed to failure. Similarly, some commentators (Gold et al.; Quintelier & Fessler; Waldmann) propose moral judgement as a test case in which the psychology of higher mental processing cannot proceed without normative judgement. But we believe there is a fundamental problem with these arguments. We can have a descriptivist account of how people pursue their own epistemic goals and norms. A psychological theory of moral reasoning, for example, may include a good computational description of the functions involved in rendering moral judgement, but this cannot (and should not!) justify the moral theory in question. Put simply, psychology may show how people think on “right” and “wrong”, but it cannot be used to decide what is right or wrong.

It is certainly part of our own cognitive theory that people pursue epistemic goals in the “new mind” (see sect. R4) and this may indeed involve moral reasoning, higher order preferences and motivation to be rational (Stanovich Reference Stanovich2008). We fail to see why a descriptivist account cannot include self-regulation by rules and systems if that is where the evidence leads us. But such systems need not be normative in the generally accepted sense. For example, Bonnefon describes a theory of his own in which he assumes that people follow folk axioms of decisions which are non-normative. There are, however, great difficulties in measuring the actual goals and rules that guide people's reasoning, even at the explicit level. Stanovich points us to evidence that when confronted with their behaviour and a normative explanation, people will often judge their own performances to be erroneous in hindsight. But does this really provide evidence for the psychological reality of norms?

Unfortunately, people are highly prone to confabulation and self-deception when it comes to the accounts they provide of their own thinking (Bargh Reference Bargh2006; Carruthers Reference Carruthers2009; Evans Reference Evans2010b; Wegner Reference Wegner2002; Wilson Reference Wilson2002). As an example, in one study the experimenters unkindly suggested to participants that one of several common incorrect (by the standard norm) solutions to the Wason selection task was the right answer (Evans & Wason Reference Evans and Wason1976). In all cases, participants confidently agreed that the proposed solution was correct; not one individual objected that it was wrong. So the explanation of the “right” answer offered only has to be plausible and not actually normative to get the approval of participants that Stanovich describes.

Thompson also refers us to some of her own recent work on meta-cognition showing that the feelings of rightness that people have in their own judgements and inferences are unrelated to their normative accuracy. The more fluently and quickly an intuitive response comes to mind, the more likely people are to be confident in it. But this fluency may be a function of a non-normative factor such as the believability of a syllogistic conclusion (Shynkarkuk & Thompson Reference Shynkarkuk and Thompson2006). A number of reasoning studies have allowed participants to give reasons for their choices, but to our knowledge no participant has ever declared a known cognitive bias, such as matching or belief bias, to be the cause of their decision. Instead, the evidence indicates that people give a reason which makes their choices appear rational in the context of the experimental instructions (Lucas & Ball Reference Lucas and Ball2005; Wason & Evans Reference Wason and Evans1975). So, even if people talk as though they were rationally motivated to follow their own norms and rules, it will be very difficult to establish that this is what they are actually doing. For example, a person may justify their preference to vote for a political party on the basis of belief in a principle, such as free market economics. Psychological study may, however, show that the true cause is something else, for example, identification with a social group the individual would like to join, and the stated reason no more than a theory for the behaviour supplied by the media. People are not lying in such cases, but they are confabulating.

R5.1. From deduction to probabilistic accounts of reasoning: The paradigm shift

Sternberg complains about the domination of the deduction paradigm in the psychology of reasoning, and its irrelevance to everyday reasoning, commenting that “few problems of consequence in our lives have deductive or even any meaningful kind of ‘correct’ solution.” In effect, he makes a case for one of our proposed research biases: researchers focus on tasks for which clear norms are available. He is right that the psychology of deductive reasoning has been dominant, although fortunately other kinds are now receiving more attention, including the study of inductive inference (Feeney & Heit Reference Feeney and Heit2007), causal inference (Sloman Reference Sloman2005) and counterfactual reasoning (Byrne Reference Byrne2005). What may be less apparent to those outside of the field, however, is that the deduction tradition itself has undergone a recent Kuhnian revolution and paradigm shift. One of the drivers for this was the desire of researchers to integrate theoretical ideas about reasoning and decision making. While Bonnefon discusses this in the recent context, theorists have been attempting to do this since the early 1990s (e.g., Evans & Over Reference Evans and Over1996; Manktelow & Over Reference Manktelow and Over1991; Oaksford & Chater Reference Oaksford and Chater1994).

The deduction paradigm has also collapsed under the weight of its own normative system – standard bivalent logic. As evidence accumulated that much reasoning was (a) logically erroneous and (b) highly sensitive to content and context amassed by use of the paradigm, the paradigm came under attack from within its own ranks. Researchers criticised logicism for its failure to deliver either an appropriate norm or a relevant description of human reasoning (Evans Reference Evans2002; Oaksford & Chater Reference Oaksford and Chater1998a). This eventually led a number of authors to declare recently that we have a “new” paradigm psychology of reasoning, freed from the constraints of logic and the deduction paradigm (see especially the two recent edited collections of Manktelow et al. Reference Manktelow, Over and Elqayam2011; Oaksford & Chater Reference Oaksford and Chater2010). The emphasis is no longer on truth and deduction but on probabilistic and Bayesian approaches, degrees of belief, and pragmatic factors. Participants are often asked to draw plausible or pragmatic inferences or to make direct judgements about the believability of conditional statements, the plausibility of conditional advice, and so on. And, as Bonnefon points out, researchers are free to focus upon how reasoning supports decision making, thus shifting the focus from truth and epistemic rationality towards action and instrumental rationality. Researchers are also becoming more interested in informal reasoning (e.g., Stanovich & West Reference Stanovich and West2007), thus hopefully taking us somewhat closer towards the everyday reasoning that concerns Sternberg.

The new paradigm is, however, divided on precisely the issue with which our target article is concerned. While some contemporary reasoning researchers (e.g., Thompson) support our critique of norms and evaluative approaches or advocate great caution in the use of norms (Stupple & Ball), others (e.g., Stanovich; and now less clearly, Oaksford & Chater) wish the new paradigm to adopt a normative framework, albeit no longer restricted to bivalent logic. By far the most popular alternative is Bayesianism, to which we now turn.

R5.2. Bayesianism – description or prescription?

Bayesian inference has two big advantages over logicism for the psychology of reasoning: (1) It permits people to hold and draw inferences about beliefs with variable degrees of uncertainty; and (2) it provides direct linkage between reasoning and decision making. People viewed as Bayesians would be able to draw upon all relevant sources of belief and knowledge in order to support consequential decision making. As McNair & Feeney illustrate, however, there seems to be some confusion as to whether the current focus on Bayesian theory is intended as a prescriptive normative account or merely a descriptive one.

Oaksford and Chater (Reference Oaksford and Chater2007) appeared explicitly to advocate Bayesianism as a normative system for human reasoning, as illustrated in detail in our target article. We, on the other hand, see Bayesianism as a broadly correct descriptive account of human reasoning and decision making, without any expectation of precise conformity to its computational rules (Evans Reference Evans2007; Evans et al. Reference Evans, Neilens, Handley and Over2008). Broadly speaking, the evidence indicates that people do represent degrees of belief, revise these beliefs in the light of evidence, and apply them in their decision making. Perhaps surprisingly, Oaksford & Chater (in contrast with Hahn) launch no strong defense of normativism in their commentary and take the stance that they are in fact descriptive rather than prescriptive Bayesians, thus avoiding is-ought inference. We are glad to hear it and note that we had made it quite clear in the target article that we have no objection in principle either to rational analysis or to the use of formal, computational theories in the manner they describe. However, we find it hard to reconcile the stance taken in their commentary with the numerous apparently evaluative uses of normative theory in their earlier writings.

In our view, the new paradigm will gain little from an approach that simply substitutes Bayesian norms for those of logic. But we also believe that researchers must go beyond demonstrations that Bayesianism is a good descriptive-computational account of aspects of human inference (Jones & Love Reference Jones and Love2011) for two reasons. First, an account of the cognitive processes that can produce such behaviour is needed at the algorithmic level, as there are many cognitive mechanisms and constraints to be studied. A major constraint, for example, is that people apparently focus on single rather than multiple hypotheses when engaged in hypothetical thinking and mental simulation (Evans Reference Evans2007). It should also be noted that there are areas of psychology where the Bayesian model fails spectacularly as a descriptive account. Descriptive Bayesianism can be hard to refute, since both beliefs and utilities can be assigned subjectively on an individual basis. However, if we examine a number of pieces of evidence in sequence, revising our beliefs after each one, it should make no difference at all to the order in which we see them (as an example, consider the evidence given to a jury during a courtroom trial). In contrast, research on belief updating shows marked primacy and recency effects on final belief judgements, as much research by social psychologists has shown. This has led researchers in this field to dismiss the Bayesian model out of hand and focus on alternative descriptive accounts (Hogarth & Einhorn Reference Hogarth and Einhorn1992).

Despite the fact that many philosophers find Bayesianism a satisfying or even compelling normative account of instrumental rationality (see Stanovich's commentary and our section R2), we still see little value in prescriptive Bayesianism in the psychology of decision making. One might argue (ought to is) that decision makers are shaped by evolution and learning to be instrumentally rational, therefore they must be Bayesian and understood accordingly. But this is an epistemic, “stands to reason”-type ought, not a deontic one; and moreover, there is no empirical force to it. Bayesian models fit approximately well to behaviour in some tasks studied by psychologists but very poorly in others, as we have mentioned. The only role that prescriptive Bayesianism can serve in such circumstances is to enable its advocates to declare behaviour irrational and in need of remediation. But this does not seem to advance theoretical understanding: primacy and recency effects in belief updating, for example, are descriptive facts that must be understood with reference to the relevant cognitive mechanisms. Instrumental rationality in its descriptive sense needs to account for the individual agent in her specific epistemic context (as Weinberg argues in his commentary and we discuss in R2.4) and her set of (often conflicting) goals. We fail to see how normative judgement assists this task in any way.

R6.1. Individual and cultural differences in reasoning

From a normative point of view, some people reason better than others, or make better judgements and decisions. There are a number of factors which underlie such individual differences. One is age: children's reasoning and decision making changes as they grow older. Performance on the raw score tests that underlie IQ projections, for example, increases up until about the age of 16 (actual IQs are age-corrected). Adults vary considerably in IQ, of course (and in working-memory capacity which is closely correlated); and this is one of several factors that have been implicated in the ability to produce normatively correct answers. Another important source of variation is culture, as an increasing number of research studies shows important differences in thinking styles; for example, between individuals in Western and Eastern societies (Buchtel & Norenzayan Reference Buchtel, Norenzayan, Evans and Frankish2009).

Stanovich argues that, since norms of rationality are culturally transmitted tools, the analogy we draw to language is “inapt.” But the cultural transmission thesis makes our analogy to language more apt, not less. Cognitive variability is a well-established fact in all areas of cognition, and in language more than anywhere else – otherwise, how can there be thousands of different languages in the world? Even language universals, these prototypical, hard core cognitive universals, vary in how they are instantiated in specific languages; and it has been suggested (Evans & Levinson Reference Evans and Levinson2009) that language diversity far exceeds any universal core. (For normativist discussions of the analogy to language, see, e.g., Cohen Reference Cohen1981; Stein Reference Stein1996; for relativist discussions, see Elqayam Reference Elqayam, Manktelow, Over and Elqayam2011; Stich Reference Stich1990.)

For a while, it seemed that normative assessment might provide a useful tool for dual-process research. In his earlier individual differences studies, Stanovich (Reference Stanovich1999; Stanovich & West Reference Stanovich and West2000a) found many examples of reasoning and decision tasks where participants of higher cognitive ability were more able to find correct answers by the standard norm applied to each task. This was attributed to their superior ability for System 2 reasoning. But as Thompson points out, there are multiple possible causes of correlations between cognitive capacity and normative performance. These include a disposition for rational thinking, and the presence of relevant education and training (Stanovich Reference Stanovich2009b; Reference Stanovich2010b). In recent research, identification of exceptions to the correlation of ability with normative performance has increased and we now know that many decision biases are fully independent of cognitive ability (Stanovich & West Reference Stanovich and West2008). Stanovich's most recent theory of the mind (Stanovich Reference Stanovich2010b) reflects these multiple requirements for normative responding. Ironically, in the developmental context, Stanovich (in press) has recently shown how simplistic and erroneous are some recent attempts to test dual-process theories of cognitive development, precisely because they assume that normativity reflects System 2 thinking. So why has Stanovich presented a staunch defence of normativism here? Because, it seems, his goal remains that of accounting for good and bad reasoning: he just thinks it is a complicated business. We ourselves have no mixed motives in critiquing the highly normative traditions of developmental psychology from Piaget onwards. This tradition is endorsed by Girotto who remarks that: “If one does not use normative standards, how could one compare the answers produced by children of different ages?” The answer is: by observing what the children actually do, as we discuss below.

A key feature of dual-process (system, mind) theories is that humans have a uniquely developed system for general purpose thinking and reasoning (objections of Brase & Shanteau noted). This is enabled by the acquisition of many beliefs, heuristics, and procedures in the new mind (or System 2) which are dependent upon education and cultural environment. So, while the object of cognitive psychology is to uncover the biological mechanism of the mind, we must always be aware of the culturally determined nature of much of its contents and sometimes even its processes. Hence, the importance of cross-cultural research in determining cognitive universals and the danger of approaching research with a priori normative framework which inevitably reflects the culture in which the researchers themselves were raised. Thinking about rationality, for example, has very different traditions in Eastern and Western cultures (Nisbett et al. Reference Nisbett, Peng, Choi and Norenzayan2001). We absolutely agree with Kim & Park that it is most important to study individual and cultural differences in a descriptive psychology of thinking, and to avoid application of universal norms in the process. However, we cannot accept at face value their proposal that cultural variations preclude any useful generalisation of cognitive architecture. On the contrary, we argue that cognitive variability (cultural, individual, historical) is essential to understanding universal cognitive architecture: without it, the picture portrayed by theorists may be too limited (Henrich et al. Reference Henrich, Heine and Norenzayan2010; Norenzayan & Heine Reference Norenzayan and Heine2005). Consider, for example, Buchtel and Norenzayan's (Reference Buchtel, Norenzayan, Evans and Frankish2009) proposal that contextualising, holistic thinking might be acquired in Eastern cultures as the preferred form of Type 2 thinking. If this is right, it contributes a powerful evidence against the received view that Type 2 processes are essentially abstract and decontextualised (Evans, in press).

From a descriptivist viewpoint, influence of age, intelligence, and culture are observable simply because people do different things. If this were not the case, the normative assessments advocated by Girotto and many others could not differ. The observable change is seen in what people do: the correctness of the responding is, as always, an externally imposed interpretation. Such differences in responding may be quantitative or qualitative. For example, as children grow older they are able to recall longer lists of words or digits, which is a quantitative change. They may also quite suddenly start to take account of another person's perspective in responding to a theory of mind task: a qualitative change. The same applies to individual differences in adults and between cultures. In both cases, we can observe what it is that people do differently without normatively assessing it. It is no coincidence that the descriptivist approach of cross-cultural research described by Kim & Park leads often to discovery of qualitative differences.

We agree with Thompson that a focus on normativity may lead us into reducing complex processes to binary measurements, and this danger is particularly strong when investigating individual and cultural differences. As a result, qualitative individual differences may be overlooked. Suppose we empirically observe four ways of responding, only one of which conforms with the norm we have in mind? Why is it any less important to understand how each of the other three arise? As an example, in a study of Bayesian reasoning, Evans et al. (Reference Evans, Handley, Perham, Over and Thompson2000) reported a number of qualitatively different responses; some participants gave the base rate, some the false positive rate, some, say, one minus the false positive rate, and so on. These variations have gone almost totally unnoticed in the large literature on the base rate fallacy, because authors time and again focus on normative responding or its absence. The right/wrong framing has dominated perceptions of findings in this literature, as in so many others.

R6.2. Normative relativism and why the bridge solution requires it

Normative relativism means that normative evaluation should be relative to specific agents with their specific goals and within a specific cultural and historical context (e.g., Elqayam Reference Elqayam, Manktelow, Over and Elqayam2011; Stich Reference Stich1990). Soft normativism (see sects. R2.1, R7) often leads to such normative relativism, as some commentators have readily acknowledged (Achourioti et al. [and see Stenning & van Lambalgen Reference Stenning and van Lambalgen2008]; Proust; Weinberg). In particular, normative relativism is inevitable where soft normativism attempts to bridge is and ought (sect. R2.2), drawing on constructs such as reflective equilibrium and constitutive norms.

The evaluative and deontic functions reflected by bridge constructs draw heavily on cultural and historical context (Elqayam Reference Elqayam, Manktelow, Over and Elqayam2011). Here are a few examples. First, as Stich (Reference Stich1990) famously argued (and Buckwalter & Stich remind us), reflective equilibrium can result in a radical, anything-goes relativism. Moreover, Searle's (Reference Searle1964) constitutive norms (sect. R3.1) are social institutions, and depend on cultural context. But the classic example is the Kantian “ought implies can,” perhaps the prototypical bridge construct, with its corollaries, “cannot implies not-ought” (bounded rationality), and “can implies ought” (principle of charity). Not for nothing did Nickerson (Reference Nickerson2008) classify bounded rationality under “relativity.” “Can” and “cannot” are fuzzy concepts. Ultimately, they rely on biology and cognitive architecture, but how cognitive architecture is instantiated in individual agents depends a great deal on cultural and historical context (Henrich et al. Reference Henrich, Heine and Norenzayan2010; Norenzayan & Heine Reference Norenzayan and Heine2005). Where “cannot” implies “not-ought,” the descriptive facts of cognitive variability should force the normativist's hand to adopt a position of normative relativism. Where theorists wish to bridge the “is” with the “ought,” yet avoid the relativist implications, they allow an inconsistency into their system.