Newell & Shanks (N&S) call for a more rigorous study of awareness using reliable, relevant, and sensitive measures that are administered when the relevant mental operation is taking place. In the current commentary, we extend N&S's arguments to implicit evaluations, which can be conceptualized as evaluative responses captured by performance-based measures, such as the Implicit Association Test (Greenwald et al. Reference Greenwald, McGhee and Schwartz1998), evaluative priming (Fazio et al. Reference Fazio, Jackson, Dunton and Williams1995), and various response interference tasks (for a review, see Gawronski et al. Reference Gawronski, Deutsch, Banse, Klauer, Voss and Stahl2011). Implicit evaluations are widely assumed to reflect unconscious attitudes that are held outside of awareness (for a review, see Gawronski et al. Reference Gawronski, Hofmann and Wilbur2006). Drawing on N&S's conceptual framework, we argue that lack of awareness in the domain of implicit evaluations is inferred from incomplete evidence that does not warrant the conclusion of unawareness.

Characterizations of implicit evaluations as reflecting unconscious attitudes are based on the finding that implicit evaluations typically show rather low correspondence to self-reported explicit evaluations of the same target object (for meta-analyses, see Cameron et al. Reference Cameron, Brown-Iannuzzi and Payne2012; Hofmann et al. Reference Hofmann, Gawronski, Gschwendner, Le and Schmitt2005). However, the conclusion that dissociations between implicit and explicit evaluations indicate unawareness of the former violates N&S's criteria of relevance and sensitivity.

Research and theorizing suggest that encountering an attitude object spontaneously activates evaluative associations in memory (De Houwer Reference De Houwer2009; Ferguson & Zayas Reference Ferguson and Zayas2009). Performance-based measures are assumed to capture these associations regardless of whether the person considers them valid. When a person is asked to report an explicit evaluation, activated associations are assessed for their (subjective) validity by propositional processes (Gawronski & Bodenhausen Reference Gawronski and Bodenhausen2006; Reference Gawronski and Bodenhausen2011). To the extent that the evaluation implied by activated associations is consistent with other salient propositions, it is typically regarded as valid and reported on measures of explicit evaluations. However, if the evaluation implied by activated associations is inconsistent with other salient propositions, consistency has to be restored before an explicit evaluation can be reported (Festinger Reference Festinger1957). In such cases, implicit and explicit evaluations often diverge, such that implicit evaluations reflect activated associations regardless of their perceived validity, whereas explicit evaluations reflect activated associations that are regarded as valid (e.g., Gawronski & LeBel Reference Gawronski and LeBel2008; Gawronski & Strack Reference Gawronski and Strack2004; Gawronski et al. Reference Gawronski, Peters, Brochu and Strack2008). From this perspective, an explicit evaluation is not a measure of a person's awareness of his or her implicit evaluation. Instead, it reflects the role of propositional processes in assessing the subjective validity of activated associations.

The inference that implicit evaluations reflect unconscious attitudes because they show low correspondence to explicit evaluations thus violates N&S's criteria of relevance and sensitivity. Low correspondence between implicit and explicit evaluations is not relevant for awareness of implicit evaluations, because explicit evaluations may differ from implicit evaluations for reasons other than lack of awareness. Moreover, low correspondence is not sensitive, because measures of explicit evaluation do not ask participants to merely report their evaluative associations, but to report the evaluative associations that they regard as valid.

To overcome these limitations, we have recently started a research project in which we asked participants to predict their implicit evaluations of multiple target groups before completing corresponding measures of implicit evaluation (Hahn et al., in press). We argue that predictions of implicit evaluations are both more relevant and more sensitive for inferences about awareness than correspondence to explicit evaluations. Predictions are more relevant, because they rule out cognitive inconsistency as a potential cause of diverging explicit evaluations. Moreover, predictions are more sensitive, because they directly capture participants' ability to report their implicit evaluations (e.g., “If we ran a computerized test, what would it show?”) rather than evaluations that they perceive as valid (e.g., “How much do you agree with the statement that group X is likeable?”). Our studies consistently showed that participants were highly successful in predicting their implicit evaluations, even in cases where they reported distinct explicit evaluations. In line with previous findings (e.g., Blair Reference Blair and Moskowitz2001; Hofmann et al. Reference Hofmann, Gawronski, Gschwendner, Le and Schmitt2005; Nosek Reference Nosek2005), implicit and explicit evaluations revealed correlations around 0.20. In contrast, participants' predictions showed mean correlations with implicit evaluations higher than 0.50 and median correlations of around 0.65.

Our research also led to some additional discoveries that highlight the benefits of studying awareness more rigorously. For example, high levels of accuracy in predicting implicit evaluations were found primarily when accuracy was determined within subjects (i.e., rank order of evaluations of different target groups for each participant). However, when accuracy was determined between subjects (i.e., rank order of evaluations of the same target group across participants), prediction accuracy was lower. In other words, although participants were able to predict their implicit evaluation of a given target group vis-à-vis other target groups (within-subjects analysis), their predictions were less accurate for identifying their implicit evaluations of a given target group vis-à-vis other participants (between-subjects analysis). Interestingly, participants also predicted lower levels of implicit evaluative bias against outgroups for themselves than for other participants. These findings suggest that, although people are aware of the evaluative quality of their implicit evaluations, they may not be aware of how their implicit evaluations compare to those of other people. That is, people seem to be aware of some aspects of their implicit evaluations (e.g., the fact that they hold more biased implicit evaluations against some groups than others) but not others (e.g., whether these biases are stronger than those of other people), and studying these two “kinds” of awareness requires different methods (i.e., within-subjects analysis vs. between-subjects analysis). Thus, rather than treating awareness as an all-or-none issue, our findings highlight the importance of more fine-grained analyses when studying conscious awareness.

In sum, we agree with N&S's concern that unawareness of psychological processes is often inferred from insufficient evidence. We argue that inferences of unawareness from dissociations between implicit and explicit evaluations violate the criteria of relevance and sensitivity. Research using more adequate measures indicates that implicit evaluations can be predicted with accuracy, suggesting that implicit evaluations do not reflect unconscious attitudes.