4.1.1. Case studies

To make the contrast between confirmatory and disconfirmatory predictions concrete, we conducted a series of studies (Firestone & Scholl Reference Firestone and Scholl2014b) inspired by an infamous art-historical reasoning error known as the “El Greco fallacy.” Beyond appreciating the virtuosity of his work, the art-history community has long puzzled over the oddly elongated human figures in El Greco's paintings. To explain these distortions, it was once supposed that El Greco suffered from an uncommonly severe astigmatism that effectively “stretched” his perceived environment, such that El Greco had simply been painting what he saw. This perspective was once taken seriously, but upon reflection it involves a conceptual confusion: If El Greco had truly experienced a stretched-out world, then he would also have experienced an equally stretched-out canvas, canceling out the supposed real-world distortions and thus leaving no trace of them in his reproductions. The distortions in El Greco's paintings, then, could not reflect literal perceptual distortions (Anstis Reference Anstis2002; Firestone Reference Firestone2013b).

We exploited the El Greco fallacy to show that multiple alleged top-down effects cannot genuinely be effects on perception. Consider, for example, the report that reflecting on unethical actions makes the world look darker (Banerjee et al. Reference Banerjee, Chatterjee and Sinha2012; Fig. 2O). The original effect was obtained using a numerical scale: After reflecting on ethical or unethical actions, subjects picked a number on the scale to rate the brightness of the room they were in. We replicated this effect with one small change: Instead of a numerical scale, subjects used a scale of actual grayscale patches to rate the room's brightness. According to the view that reflecting on negative actions makes the world look darker, this small change drastically alters the study's prediction: If the world really looks darker, then the patches making up the scale should look darker too, and the effects should thus cancel each other out (just as the alleged distortions in El Greco's experience of the world would be canceled out by his equally distorted experience of his canvas). However, the follow-up study succeeded: subjects still picked a darker patch to match the room after reflecting on an unethical action (Firestone & Scholl Reference Firestone and Scholl2014b, Experiment 5). This effect, then – like the distortions in El Greco's work – must not reflect the way the world actually looked to subjects.

This approach is in no way limited to the particulars of the morality/brightness study. Indeed, to apply the same logic more broadly, we also explored a report of a very different higher-level state (a subject's ability to act in a certain way) on a very different visual property (perceived distance). In particular, holding a wide rod across one's body (Fig. 2E) reportedly makes the distance between two poles (which form a doorway-like aperture) look narrower, as measured by having subjects instruct the experimenter to adjust a measuring tape to visually match the aperture's width. The effect supposedly arises because holding the rod makes apertures less passable (Stefanucci & Geuss Reference Stefanucci and Geuss2009). We successfully replicated this result, but we also tested it with one critical difference: Instead of adjusting a measuring tape to record subjects' width estimates, the experimenter used two poles that themselves formed an independent and potentially passable aperture. Again, the El Greco logic applies: If holding a rod really does perceptually compress apertures, then this variant should “fail,” because subjects should see both apertures as narrower. But the experiment did not “fail”: Subjects again reported narrower apertures even when responding with an aperture (Firestone & Scholl Reference Firestone and Scholl2014b, Experiment 2). Therefore, this effect cannot reflect a true perceptual distortion – not because the effect fails to occur, but rather because it occurs even when it shouldn't. (In later experiments, we determined the true, nonperceptual, explanation for this effect, involving task demands; see Pitfall 3.)

4.1.2. Other susceptible studies

As an example of testing disconfirmatory predictions, the El Greco fallacy applies to any constant-error distortion that should affect equally the means of reproduction (e.g., canvases, grayscale patches) and the item reproduced (e.g., visual scenes to be painted, bright rooms). The studies that fail to test such predictions are too numerous to count; essentially, nearly every study falls into this category. However, some studies of top-down effects on perception may have tested such predictions inadvertently – and, given their results, perhaps committed the El Greco fallacy.

Consider, for example, the report that after repeatedly viewing one set of red and violet letters and a second set of blue and violet numbers, subjects judged token violet letters to look redder than they truly were and token violet numbers to look bluer than they truly were (Goldstone Reference Goldstone1995; Fig. 2J). This effect was measured by having subjects adjust the hue of a stimulus to perceptually match the symbol being tested. However, the adjusted stimulus was a copy of that symbol! For example, after viewing a red “T,” a reddish-violet “L,” and a violet “E,” subjects judged the E to be redder – as measured by adjusting the hue of a second E. This commits the El Greco fallacy: if Es really look redder after one sees other red letters, then both the to-be-matched E and the adjustable E should have looked redder, and the effects should have canceled one another out. That such an effect was nevertheless obtained suggests it cannot be perceptual.

Similarly, consider the following pair of results, reported together: Subjects judged gray patches to be darker after reading negative (vs. positive) words, and subjects judged words printed in gray ink to be darker if the words were negative (vs. positive), as measured by selecting a darker grayscale patch to match the word's lightness (Meier et al. Reference Meier, Robinson, Crawford and Ahlvers2007; Fig. 2G). Here too is an El Greco fallacy: if, per the first result, reading negative words makes gray patches look darker, then the gray patches from the second result should also have looked darker, and the effects of one should have canceled out the other.

The El Greco fallacy may also afflict reports that linguistic color categories alter color appearance (Webster & Kay Reference Webster and Kay2012). For example, a color that is objectively between blue and green may appear either blue or green because our color terms single out those colors when they discretize color space, creating clusters of perceptual similarity. However, such studies use color spaces specifically constructed for perceptual uniformity, such that each step through the space's parameters is perceived as equal in magnitude. This raises a puzzle: If color terms affect perceived color, then such effects should already have been assimilated into the color space, leaving no room for color terms to exert their influence in studies using such color spaces. That these studies still show labeling effects suggests an alternative explanation.Footnote ²

4.1.3. A lesson for future research

To best determine the extent to which cognition influences perception, future studies should proactively employ both confirmatory and disconfirmatory research strategies; to do otherwise is to ignore half of the predictions the relevant theories generate. In pursuing disconfirmatory evidence, El Greco–inspired research strategies in particular have three distinct advantages. First, they can rule out perceptual explanations without relying on null effects and their attendant interpretive problems; instead, this strategy can disconfirm top-down interpretations through positive replications. Second, the El Greco strategy can fuel such implications even before researchers determine the actual (nonperceptual) culprit (just as we know that astigmatism does not explain El Greco's distortions, even if we remain uncertain what does explain them). Finally, this strategy is broadly relevant – being applicable any time a scale can be influenced just as the critical stimuli are supposedly influenced (e.g., in nearly all perceptual matching tasks).

4.2.1. Case studies

It has been reported that throwing a heavy ball (rather than a light ball) at a target increases estimates of that target's distance (Witt et al. Reference Witt, Proffitt and Epstein2004). One interpretation of this result (favored by the original authors) is that the increased throwing effort actually made the target look farther away, and that this is why subjects gave greater distance estimates. However, another possibility is that subjects only judged the target to be farther, even without a real change in perception. For example, after having such difficulty reaching the target with their throws, subjects may have simply concluded that the target must have been farther away than it looked.

A follow-up study tested these varying explanations and decided the issue in favor of an effect on judgment rather than perception. Whereas the original study asked for estimates of distance without specifying precisely how subjects should make such estimates, Woods et al. (Reference Woods, Philbeck and Danoff2009) systematically varied the distance-estimation instructions, contrasting cases (between subjects) asking for reports of how far the target “visually appears” with cases asking for reports of “how far away you feel the object is, taking all nonvisual factors into account” (p. 1113). In this last condition, subjects were especially encouraged to separate perception from judgment: “If you think that the object appears to the eye to be at a different distance than it feels (taking nonvisual factors into account), just base your answer on where you feel the object is.” Tellingly, the effect of effort on distance estimation replicated only in the “nonvisual factors” group, and not in the “visually appears” group – suggesting that the original results reflected what subjects thought about the distance rather than how the distance truly looked to them.

Similarly, it was reported that accuracy in throwing darts at a target affected subsequent size judgments of the target, which was initially assumed to reflect a perceptual change: Less-accurate throwing led to smaller target-size estimates, as if one's performance perceptually resized the target (Wesp et al. Reference Wesp, Cichello, Gracia and Davis2004; Fig. 2F). However, the same researchers rightly wondered whether this was genuinely an effect on perception or whether these biased size estimates might instead be driven by overt inferences that the target must have been smaller than it looked (perhaps to explain or justify poor throwing performance). To test this alternative, the same research group (Wesp & Gasper Reference Wesp and Gasper2012) replicated the earlier result – but then ran a follow-up condition in which, before throwing, subjects were told that the darts were faulty and inaccurate. This additional instruction eliminated the correlation between performance and reported size. With a ready-made explanation already in place, subjects no longer needed to “blame the target”: Rather than conclude that their poor throwing resulted from a small target, subjects instead attributed their performance to the supposedly faulty darts and thus based their size estimates directly on how the target looked.

Note that this is a perfect example of the kind of judgment that can only be described as “routine.” Even if other sorts of top-down effects on judgment more richly interact with foundational issues in perception research, blaming a target for one's poor performance is not one of them.

4.2.2. Other susceptible studies

Many alleged top-down effects on perception seem explicable by appeal to these sorts of routine judgments. One especially telling pattern of results is that many of these effects are found even when no visual stimuli are used at all. For example, factors such as value and ease of action have been claimed to affect online distance perception (e.g., Balcetis & Dunning Reference Balcetis and Dunning2010; Witt et al. Reference Witt, Proffitt and Epstein2005), but those same factors have been shown to affect the estimated distance of completely unseen (and sometimes merely imagined) locations such as Coney Island (Alter & Balcetis Reference Alter and Balcetis2011) or one's work office (Wakslak & Kim Reference Wakslak and Kim2015). Clearly such effects must reflect judgment and not perception – yet their resemblance to other cases that are indeed claimed as top-down effects on perception suggests that many such cases could reflect judgmental processes after all.

Other cases seem interpretable along these lines all on their own. For example, another study also demonstrated an effect of dart-throwing performance on size judgments – but found that the effect disappeared when subjects made their throws while hanging onto a rock-climbing wall 12 feet above the ground (Cañal-Bruland et al. Reference Cañal-Bruland, Pijpers and Oudejans2010). Though this phenomenon was interpreted as an effect of anxiety on action-specific perception, the finding could easily be recast as an effect on judgment instead: Subjects who performed poorly while clinging to the rock-climbing wall had an obvious explanation for performing poorly and so had no need to explain their misses by inflating target-size estimates.

In other cases, the inference to judgment rather than perception can be more straightforward. For example, politically conservative subjects rated darkened images of Barack Obama as more “representative” of him than lightened images, whereas liberal subjects showed the opposite pattern (Caruso et al. Reference Caruso, Mead and Balcetis2009), and this effect was interpreted as an effect of partisan attitudes on perceived skin tone. However, it seems more likely that darker photos (or darker skin tones) seemed more negative to subjects, and that conservatives deemed them more representative (and liberals less representative) for that reason – because conservatives think more negatively about Obama than liberals do. (By analogy, we suspect that conservative subjects would also rate a doctored image of Obama with bright red horns on his forehead as more “representative” than an image of Obama with a halo, and that liberals would show the opposite pattern; but clearly such a result would not imply that conservatives literally see Obama as having horns!) Other purported top-down effects that seem similarly explicable include effects on visually estimated weight (Doerrfeld et al. Reference Doerrfeld, Sebanz and Shiffrar2012), the estimated looming of spiders (Riskind et al. Reference Riskind, Moore and Bowley1995), and the rated anger in African-American or Arab faces (Maner et al. Reference Maner, Kenrick, Becker, Robertson, Hofer, Neuberg, Delton, Butner and Schaller2005).

4.2.3. A lesson for future research

The distinction between perception and judgment is intuitive and uncontroversial in principle, but it is striking just how few discussions of top-down effects on perception even mention judgmental effects as possible alternative explanations. (For some exceptions see Alter & Balcetis Reference Alter and Balcetis2011; Lupyan et al. Reference Lupyan, Thompson-Schill and Swingley2010; Witt et al. Reference Witt, Proffitt and Epstein2010.) Future work relying on subjective perceptual reports must attempt to disentangle these possibilities. It would of course be preferable for such studies to empirically distinguish perception from judgment – for example, by using performance-based measures in which subjects' success is tied directly to how they perceive the stimuli (such as a visual search task; cf. Scholl & Gao Reference Scholl, Gao, Rutherford and Kuhlmeier2013). Or, per the initial case study reviewed above, future work can at least ask the key questions in multiple ways that differentially load on judgment and perception.

At a minimum, given the importance of distinguishing judgment from perception, it seems incumbent on any proposal of a top-down effect to explicitly and prominently address the distinction, even if only rhetorically – because a shift from perception to judgment may dramatically reduce such an effect's potential revolutionary consequences. And at the same time, we note that certain terms may actively obscure this issue and so should be avoided. For example, many papers in this literature advert to effects on “perceptual judgment” (e.g., Meier et al. Reference Meier, Robinson, Crawford and Ahlvers2007; Song et al. Reference Song, Vonasch, Meier and Bargh2012; Storbeck & Stefanucci Reference Storbeck and Stefanucci2014), which can only invite confusion about this foundational distinction.

4.3.1. Case studies

Consider the effect of wearing a heavy backpack on slant estimates (Bhalla & Proffitt Reference Bhalla and Proffitt1999; Fig. 2D). One possibility is that backpacks make hills look steeper, and that the subjects faithfully reported what they saw. But another explanation is that subjects modified their responses to suit the experimental circumstances, in which a very conspicuous manipulation (a curiously unexplained backpack) was administered before obtaining a single perceptual judgment (regarding the hill's slant).

A recent series of studies shows that the experimental demand of wearing a backpack can completely account for the backpack's effect on slant estimates. When backpack-wearing subjects were given a compelling (but false) cover story to justify the backpack's purpose (to hold heavy monitoring equipment during climbing), the effect of heavy backpacks on slant estimation completely disappeared (Durgin et al. Reference Durgin, Baird, Greenburg, Russell, Shaughnessy and Waymouth2009; see also Durgin et al. Reference Durgin, Klein, Spiegel, Strawser and Williams2012). With a plausible cover story, subjects had very different expectations about the experiment's purpose (expectations that they articulated explicitly during debriefing), which no longer suggested that the backpack “should” modulate their responses. Similar explanations have subsequently been confirmed for other effects of action on perceptual reports, including effects of aperture “passability” on spatial perception (Firestone & Scholl Reference Firestone and Scholl2014b) and energy on slant perception (Durgin et al. Reference Durgin, Klein, Spiegel, Strawser and Williams2012; Shaffer et al. Reference Shaffer, McManama, Swank and Durgin2013). For example, no effect of required climbing effort is found without a transparent manipulation – such as when subjects estimate the slant of either an (effortful) staircase or an (effort-free) escalator in a between-subjects design (Shaffer & Flint Reference Shaffer and Flint2011).

Other studies have implicated task demands in very different top-down effects. For example, it has been reported that, when subjects can win a gift card taped to the ground if they throw a beanbag closer to the gift card than their peers do, subjects undershoot the gift card if it is worth $25 but not if it is worth $0 – suggesting (to the original authors) that more desirable objects look closer (Balcetis & Dunning Reference Balcetis and Dunning2010; Fig. 2P). However, in addition to the value of the gift card, the demands of the task differed across these conditions in an especially intuitive way: Whereas subjects may employ various throwing strategies in earnest attempts to win a $25 gift card, they may not try to “win” a $0 gift card (which is a decidedly odd task). For example, subjects who are genuinely trying to win the $25 gift card might undershoot the card if they believed it would be awarded to the closest throw without going over, or if they anticipated that the beanbag would bounce closer to the gift card after its first landing. However, they may not show those biases for the $0 gift card, which wouldn't have been worth any such strategizing. A follow-up study (Durgin et al. Reference Durgin, DeWald, Lechich, Li and Ontiveros2011a) tested these possibilities directly and found that slightly changing the instructions so that the challenge was to hit the gift card directly (rather than land closest) led subjects to throw the beanbag farther (perhaps because they were no longer worried that it would bounce or that they would be disqualified if they overshot), just as would be expected if differences in strategic throwing (rather than differences in actual perception) explained the initial results.

4.3.2. Other susceptible studies

Perhaps no pitfall is as generally applicable as demand and response bias, especially for studies relying entirely on observer reports. A great many reported top-down effects on perception use very salient manipulations and ask for perceptual judgments that either give subjects ample opportunity to consider the manipulation's purpose or make the “right” answer clear. For example, it has been reported that, when shown a range of yellow-orange discs superimposed on a traffic light's middle bulb, German subjects (for whom that light's color is called gelb, or yellow) classified more discs as “yellow” than did Dutch subjects (who call it oranje, or orange; Mitterer et al. Reference Mitterer, Horschig, Müsseler and Majid2009; Fig. 2Q). Though interpreted as an effect of language on perception – the claim being that the German subjects visually experienced the colored discs as yellower – it seems just as plausible that the subjects were simply following convention, assigning the yellow-orange discs the socially appropriate names for that context.

Many other studies use salient manipulations and measures in a manner similar to the backpacks and hills experiments (Bhalla & Proffitt Reference Bhalla and Proffitt1999). For example, similar explanations seem eminently plausible for reported effects of desirability on distance perception (e.g., the estimated distance of feces vs. chocolate; Balcetis & Dunning Reference Balcetis and Dunning2010), of racial identity on faces' perceived lightness (Levin & Banaji Reference Levin and Banaji2006), of stereotypes on the identity of weapons and tools (Correll et al. Reference Correll, Wittenbrink, Crawford and Sadler2015), of tool use on the perceived distance to reachable targets (Witt et al. Reference Witt, Proffitt and Epstein2005), of scary music on the interpretation of scary or nonscary ambiguous figures (Prinz & Seidel Reference Prinz and Seidel2012; Fig. 2H), and of fear of heights on perceived height (Clerkin et al. Reference Clerkin, Cody, Stefanucci, Proffitt and Teachman2009; Stefanucci & Proffitt Reference Stefanucci and Proffitt2009).

4.3.3. A lesson for future research

In light of recent findings concerning task demands in studies of top-down effects on perception (especially Durgin et al. Reference Durgin, Baird, Greenburg, Russell, Shaughnessy and Waymouth2009), it is no longer possible to provide compelling evidence for a top-down effect on perception without considering the experiment's social context. Yet, so many studies never even mention the possibility of demand-based effects (including several studies mentioned above, e.g., Mitterer et al. Reference Mitterer, Horschig, Müsseler and Majid2009; Prinz & Seidel Reference Prinz and Seidel2012). (For some exceptions, see Levin & Banaji Reference Levin and Banaji2006; Schnall et al. Reference Schnall, Zadra and Proffitt2010; Witt Reference Witt2011b.) This is especially frustrating because assessing demand effects is often easy and cost-free. In particular, although demand effects can be mitigated by nontransparent manipulations or indirect measures, they can also often be assessed by simply asking the subjects about the experiment – for example, during a careful postexperiment debriefing. For example, before the experiment's purpose is revealed, researchers can carefully ask subjects what they thought the experiment was about, what strategies they used, and so forth. These sorts of questions can readily reveal (or help rule out) active demand factors.

Such debriefing was especially helpful, for example, in the case of backpacks and reported slant, wherein many subjects explicitly articulated the experimental hypothesis when asked – and only those subjects showed the backpack effect (Durgin et al. Reference Durgin, Baird, Greenburg, Russell, Shaughnessy and Waymouth2009). In this way, we believe Durgin et al.'s Reference Durgin, Baird, Greenburg, Russell, Shaughnessy and Waymouth2009 report has effectively set the standard for such experiments: Given the negligible costs and the potential intellectual value of such careful debriefing, we contend that claims of top-down effects (especially in studies using transparent manipulations) can no longer be credible without at least asking about – and reporting – subjects' beliefs about the experiment.

4.4.1. Case studies

One especially compelling and currently influential top-down effect on perception is a report that Black (i.e., African-American) faces look darker than White (i.e., Caucasian) faces, even when matched for mean luminance, as in Figure 3A (Levin & Banaji Reference Levin and Banaji2006). This finding is today widely regarded as one of the strongest counterexamples to modularity (e.g., Collins & Olson Reference Collins and Olson2014; Macpherson Reference Macpherson2012; Vetter & Newen Reference Vetter and Newen2014) – no doubt because, in addition to the careful experiments reported in the paper, the difference in lightness is clearly apparent upon looking at the stimuli. In other words, this top-down effect works as a “demonstration” as well as an experiment.

Figure 3.

(A) The face stimuli (matched in mean luminance) from Experiment 1 of Levin and Banaji (Reference Levin and Banaji2006). (B) The blurred versions of the same stimuli used in Firestone and Scholl (Reference Firestone and Scholl2015a), which preserved the match in mean luminance but obscured the race information.

That last point is worth emphasizing given the prevalence of “demos” in vision science. In our field, experimental data about what we see are routinely accompanied by such demonstrations – in which interested observers can experience the relevant phenomena for themselves, often in dramatic fashion. For example, no experiments are needed to convince us of the existence of phenomena such as motion-induced blindness, apparent motion, or the lightness illusions depicted in Figure 1. Of course, that is not to say that demos are necessary for progress in vision science; most experiments surely get by without them. But effective demos can provide especially compelling evidence, and they may often directly rule out the kinds of worries we expressed in discussing the previous two pitfalls (i.e., task demands and postperceptual judgments).

In this context, the demonstration of race-based lightness distortions (Levin & Banaji Reference Levin and Banaji2006) is exceptional, insofar as it is one of the only such demos in this large literature. Indeed, it strikes us as an awkward fact that so few such effects can actually be experienced for oneself. For example, the possibility that valuable items look closer is testable not only in a laboratory (e.g., Balcetis & Dunning Reference Balcetis and Dunning2010) but also from the comfort of home: Right now you can place a $20 bill next to a $1 bill and see for yourself whether there is a perceptual difference. Similarly, knowledge of an object's typical color (e.g., that bananas are yellow) reportedly influences that object's perceived color, such that a grayscale image of a banana is judged to be more than 20% yellow (Hansen et al. Reference Hansen, Olkkonen, Walter and Gegenfurtner2006; Olkkonen et al. Reference Olkkonen, Hansen and Gegenfurtner2008); however, if you look now at a grayscale image of a banana (Fig. 2K), we predict that you will not experience this effect for yourself – even though the reported effect magnitudes far exceed established discrimination thresholds (e.g., Hansen et al. Reference Hansen, Giesel and Gegenfurtner2008; Krauskopf & Gegenfurtner Reference Krauskopf and Gegenfurtner1992). (You may notice that many of the top-down effects in Figure 2 are caricatured, e.g., with actual luminance differences for positive vs. negative words and smiling vs. frowning faces. This is because when the effects weren't caricatured in this way, readers could not understand the claims – because they could not experience the effect!) Figure 2.

All of this makes the reported lightness difference much more compelling: As you may experience in Figure 3A, the Black face truly looks darker than the luminance-matched White face. But is this a top-down effect on perception? Though the face stimuli were matched for mean luminance, there are of course many visual cues to lightness that are independent of mean luminance. For example, in many lightness illusions, two regions of equal luminance nevertheless appear to have different lightnesses because of depicted patterns of illumination and shadow (as in Fig. 1). Indeed, a close examination of the face stimuli in Figure 3A suggests that the Black face seems to be under illumination, whereas the White face doesn't look particularly illuminated or shiny – a difference that has long been known to influence perceived lightness (Adelson Reference Adelson and Gazzaniga2000; Gilchrist & Jacobsen Reference Gilchrist and Jacobsen1984). And the Black face has a darker jawline, whereas the White face has darker eyes. Of course, there must exist some low-level differences between the images, because otherwise they would be identical; nevertheless, the question remains whether such lower-level visual factors are responsible for the effect, rather than the meaning or category (here, race) that is correlated with that low-level difference. Figure 3.

To test whether one or more such low-level differences – rather than race, per se – explain the difference in perceived lightness, we replicated this study with blurred versions of the face stimuli, so as to eliminate race information while preserving many low-level differences in the images (including the match in average luminance and contrast) – as in Figure 3B (Firestone & Scholl Reference Firestone and Scholl2015a). After blurring, the vast majority of observers asserted that the two faces actually had the same race (or were even the same person). However, even those observers who asserted that the faces had the same race nevertheless judged the blurry image derived from the Black face to be darker than the blurry image derived from the White face. This result effectively shows how the lightness difference can derive from low-level visual features – which, critically, are present in the original images – without any contribution from perceived race. (And note that such results are unlikely to reflect unconscious race categorization; it would be a distinctively odd implicit race judgment that could influence explicit lightness judgments but not explicit race judgments.) And although the original effect (with unblurred faces) could of course still be explained entirely by race (rather than by the lower-level differences now shown to affect perceived lightness), it is clear that further experiments would be needed to show this – and so we conclude that the initial demonstration of Levin and Banaji (Reference Levin and Banaji2006) provides no evidence for a top-down effect on perception.Footnote ³

Many other effects that initially seemed to reflect high-level factors have been shown to reflect lower-level visual differences across conditions. Consider, for example, reports that categorical differences facilitate visual search with line drawings of animals and artifacts – e.g., with faster and more efficient searches for animals among artifacts, and vice versa (Levin et al. Reference Levin, Takarae, Miner and Keil2001). This result initially appeared to be a high-level effect on a fairly low-level perceptual process, given that efficient visual search is typically considered “preattentive.” However, on closer investigation (and to their immense credit), the same researchers discovered systematic low-level differences in their stimuli – wherein the animals (e.g., snakes, fish) had more curvature than the artifacts (e.g., chairs, briefcases) – which sufficiently explained the search benefits (as revealed by follow-up experiments directly exploring curvature).

4.4.2. Other susceptible studies

The possibility of such low-level confounds is a potential issue, almost by definition, with any top-down effect that varies stimuli across conditions. For example, size contrast is reportedly enhanced when the inducing images are conceptually similar to the target image – such that a central dog image looks smaller when surrounded by images of larger dogs versus images of larger shoes (Coren & Enns Reference Coren and Enns1993). However, dogs are also more geometrically similar to each other than they are to shoes (for example, the shoe images were shorter, wider, and differently oriented than the dog images; Fig. 2L), and size contrast may instead be influenced by such geometric similarity. (Coren and Enns are quite sensitive to this concern, but their follow-up experiments still involve important geometric differences of this sort.)

Other investigations of top-down effects on size contrast also manipulate low-level properties, for example contrasting natural scenes with different distributions of color and complexity (e.g., van Ulzen et al. Reference van Ulzen, Semin, Oudejans and Beek2008). Or, in a very different case, studies of how fear may affect spatial perception often involve stimuli with very different properties (e.g., a live tarantula vs. a plush toy; Harber et al. Reference Harber, Yeung and Iacovelli2011), or even the same stimulus viewed from very different perspectives (e.g., a precarious balcony viewed from above or below; Stefanucci & Proffitt Reference Stefanucci and Proffitt2009).

4.4.3. A lesson for future research

Manipulating the actual stimuli or viewing circumstances across experimental conditions is a perfectly viable methodological choice, but it adds a heavy burden to avoid low-level differences between the stimuli. Critically, this burden can be met in at least two ways. One possibility is to preserve the high-level factor while eliminating the low-level factor. (In other contexts looking at fearful stimuli, for example, images of spiders have been contrasted not with plush toys, but with “scrambled” spider images, or even images of the same line segments rearranged into a flower; e.g., New & German Reference New and German2015.) Another possibility – as in our study of race categories and lightness (Firestone & Scholl Reference Firestone and Scholl2015a) – is to preserve the low-level factor while eliminating the high-level factor. For top-down effects, this latter strategy is often more practical because it involves positively replicating the relevant effect; in contrast, the former strategy may require a null effect (which raises familiar concerns about statistical power, etc.). In either case, however, such strategies show how this pitfall is eminently testable.

4.5.1. Case studies

Attention has a curious status in the long-running debate about top-down effects. On the one hand, perhaps based on its prominence in previous discussions (especially Pylyshyn Reference Pylyshyn1999), the kinds of thoughts noted in the previous section are almost always recognized and accepted in most modern discussions of top-down effects – including recent discussions reaching very different conclusions than our own (e.g., two of the most recent literature reviews of top-down effects, which concluded that top-down effects have been conclusively established many times over; Collins & Olson Reference Collins and Olson2014; Vetter & Newen Reference Vetter and Newen2014). Nevertheless, these recent discussions happily agree that, to be compelling, top-down effects must not merely operate through attention. (Collins and Olson draw their conclusions based largely on contemporary top-down effects – including Levin and Banaji Reference Levin and Banaji2006 – that, unlike earlier results, “cannot be easily attributed to the influences of attention,” p. 843. And Vetter and Newen even define the question in terms of top-down effects obtaining “while the state of the sensory organs (in terms of spatial attention and sensory input) is held constant,” p. 64.)

On the other hand, given this prominence in theoretical discussions of top-down effects, it is curious that attention is almost never empirically explored in this literature – curious especially given that such effects are often straightforwardly testable. In particular, for a possible influence of intention on perception (for instance), it is almost always possible to separate intention from attention – most directly by holding one constant while varying the other. For example, to factor attention out, one can impose an attentional load, often by means of a secondary task (which is common in many other contexts, e.g., in exploring scene perception or feature binding without attention; Bouvier & Treisman Reference Bouvier and Treisman2010; Cohen et al. Reference Cohen, Alvarez and Nakayama2011). Or, in a complementary way, one can assess attention's role directly by actively manipulating the locus of attention while holding intention constant. This is what was done in one of the only relevant empirical case studies of attention and top-down effects.

Perhaps the most intuitively compelling evidence for intentional effects on perception comes from studies of ambiguous figures. For example, inspection of a Necker cube (Fig. 2R) reveals that one can voluntarily switch which interpretation is seen (in particular, which face of the cube seems to be in front), and such switching is also possible with many other such figures (such as the famous duck–rabbit figure). Several early defenders of top-down influences on perception essentially took such intuitions at face value and rejected the cognitive impenetrability of visual perception from the get-go. For example, Churchland (Reference Churchland1988) argued that one controls which interpretations of these ambiguous images one sees by “changing one's assumptions about the nature of the object,” and thus concluded that “at least some aspects of visual processing, evidently, are quite easily controlled by the higher cognitive centers” (p. 172).

However, several later studies showed that such voluntary switches from one interpretation to another are occasioned by exactly the sorts of processes that uncontroversially do not demonstrate top-down penetration of perception. For example, switches in the Necker cube's interpretation are driven by changes in which corner of the cube is attended (Peterson & Gibson Reference Peterson and Gibson1991; Toppino Reference Toppino2003), and the same has been found for other ambiguous figures (for a review, see Long & Toppino Reference Long and Toppino2004). (Such effects are driven by the fact that attended surfaces tend to be seen as closer.) In other words, though one may indeed be “changing one's assumptions” when the figure switches, that is not actually triggering the switches. Instead, the mechanism is that different image regions are selectively processed over others, because such regions are attended differently in relatively peripheral ways.

Evidence has recently emerged pointing to a similar explanation for certain effects of action on spatial perception. It has been reported that success in a golfing task inflates perception of the golf hole's size (Witt et al. Reference Witt, Linkenauger, Bakdash and Proffitt2008), and more generally that successful performance makes objects look closer and larger (Witt Reference Witt2011a). However, follow-up studies suggest that the mere deployment of attention may be the mediator of these effects. For example, diverting attention away from the hole at the time the golf ball is struck (by occluding the hole, or by making subjects putt around the blades of a moving windmill) eliminates the effect of performance on judged golf-hole size (Cañal-Bruland et al. Reference Cañal-Bruland, Zhu, van der Kamp and Masters2011), and the presence of the hole-resizing effect is associated with a shift in the location of subjects' attentional focus (e.g., toward the club vs. toward the hole; Gray & Cañal-Bruland, Reference Gray and Cañal-Bruland2015; see also Gray et al. Reference Gray, Navia and Allsop2014 for a similar result in the context of a different action-specific top-down effect). Moreover, the well-documented effects of action-planning on spatial judgments (e.g., Kirsch & Kunde Reference Kirsch and Kunde2013a; Reference Kirsch and Kunde2013b) also arise from the deployment of visual attention alone, even in the absence of motor planning (Kirsch Reference Kirsch2015). That visual attention may be both necessary and sufficient for such effects suggests that apparent effects of action on perception may reduce to more routine and well-known interactions between attention and perception.

4.5.2. Other susceptible studies

There have been fewer case studies of this sort of peripheral attention and its role in top-down effects, and as a result this pitfall is not on the sort of firm empirical footing enjoyed by the other pitfalls presented here. Nevertheless, it seems that such explanations could apply broadly. Most immediately, many recently reported top-down effects on perception use ambiguous figures but do not rule out relevant attention mechanisms. For example, it has been reported that scary music biased the interpretation of ambiguous figures (such as an alligator/squirrel figure; Fig. 2H) toward their scarier interpretation (Prinz & Seidel Reference Prinz and Seidel2012), and that subjects who are rewarded every time a certain stimulus is shown (e.g., the number 13) report seeing whichever interpretation of an ambiguous stimulus (e.g., a B/13 figure) is associated with that reward (Balcetis & Dunning Reference Balcetis and Dunning2006). However, such studies fail to measure attention, or even (in the case of Prinz & Seidel) to mention it as a possibility.

The effects of attention on appearance pose an even broader challenge. For example, findings suggesting that attended items can be seen as larger (Anton-Erxleben et al. Reference Anton-Erxleben, Henrich and Treue2007) immediately challenge the interpretation of nearly every alleged top-down effect on size perception, including reported effects of throwing performance on perceived target size (e.g., Cañal-Bruland et al. Reference Cañal-Bruland, Pijpers and Oudejans2010; Fig. 2F), of balance on the perceived size of a walkable beam (Geuss et al. Reference Geuss, Stefanucci, de Benedictis-Kessner and Stevens2010), of hitting ability on the perceived size of a baseball (Gray Reference Gray2013; Witt & Proffitt Reference Witt and Proffitt2005), of athletes' social stature on their perceived physical stature (Masters et al. Reference Masters, Poolton and van der Kamp2010), and even of sex primes on the perceived size of women's breasts (den Daas et al. Reference den Daas, Häfner and de Wit2013). In this last case, for example, if sex-primed subjects simply attended more to the images of women's breasts, then this could explain why they (reportedly) appeared larger. And this is to say nothing of attentional effects on other visual properties – such as the fact that voluntary attention perceptually darkens transparent surfaces (Tse Reference Tse2005), which could explain the increase in perceived transparency among thirsty observers (Changizi & Hall Reference Changizi and Hall2001; Fig. 2B) if nonthirsty observers simply paid more attention during the task (being less distracted by their thirst).

4.5.3. A lesson for future research

Attention is a rich and fascinating mental phenomenon that in many contexts interacts in deep and subtle ways with foundational issues in perception research. However, there also exist more peripheral sorts of attention that amount to little more than focusing more or less intently on certain locations (or objects, or features) in the visual field. And because such peripheral forms of attention are ubiquitous and active during almost every waking moment, future work must rule out peripheral forms of attention as mediators of top-down effects in order to have any necessary implications for the cognitive impenetrability of visual perception.

Given how straightforward such tests are in principle (per sect. 4.5.1), studies of attention could play a great role in advancing this debate – either for or against the possibility of truly revolutionary top-down effects. Some top-down effects might be observed even when attention is held constant or otherwise occupied, and this would go a long way toward establishing them as counterexamples to the modularity of perception. Or, it could be shown that the deployment of visual attention alone is insufficient to produce similar effects. For example, if it were shown that attending to a semitransparent surface does not make it look more opaque, this could rule out the possibility that attention drove thirst-based influences on perceived transparency (Changizi & Hall Reference Changizi and Hall2001). Similarly, if moving one's attention from left to right fails to bias apparent motion in that direction, then such attentional anticipation may not underlie alleged effects of language on perceived motion (as in Meteyard et al. Reference Meteyard, Bahrami and Vigliocco2007; Tse & Cavanagh Reference Tse and Cavanagh2000). Such studies would be especially welcome in this literature, given the seemingly dramatic disparity between how often attention is theoretically recognized as relevant to this debate and how seldom it is empirically studied or ruled out.

4.6.1. Case studies

Consider again the “moral pop-out effect” – the report that morally relevant words are easier to see than morally irrelevant words, supposedly because moral stimuli are “privileged” in the mind (Gantman & Van Bavel Reference Gantman and Van Bavel2014). Subjects were shown very briefly (40–60 ms) presented words and nonwords one at a time and had to decide whether each presented stimulus was a word or a nonword (see Fig. 4A). Some words were morally relevant (e.g., “illegal”), and some were morally irrelevant (e.g., “limited”). Subjects more accurately identified morally relevant words than morally irrelevant words. However, by virtue of being related to morality, the morally relevant words were also related to each other (e.g., the moral words included not only “illegal” but also “law,” “justice,” “crime,” “convict,” “guilty,” and “jail”), whereas the non-moral words were not related to anything in particular (including, in addition to “limited,” words such as “rule,” “exchange,” “steel,” “confuse,” “tired,” and “house”). In that case, it could be that the moral words simply primed each other and were easier to recognize for that reason rather than because of anything special about morality. Figure 4.

Figure 4.

(A) Experimental design for detecting “enhanced visual awareness” of various word categories. After fixation, a word or nonword appeared for 50 ms and then was masked by ampersands for 200 ms. (B) Results comparing “popout effects” for fashion, transportation, and morality (from Firestone & Scholl Reference Firestone and Scholl2015b).

Crucially, such semantic priming would not be a top-down effect on perception. For example, in more traditional lexical decision tasks (e.g., Meyer & Schvaneveldt Reference Meyer and Schvaneveldt1971), in which visual recognition of a word (e.g., “nurse”) is speedier and more accurate when that word is preceded by a related word (e.g., “doctor”) than by an unrelated word (e.g., “butter”), many follow-up experiments and modeling approaches have shown that this improvement in recognition occurs not because of any boost to visual processing per se, but rather because the relevant memory representations are easier to retrieve when evaluating whether a visual stimulus is familiar (e.g., because “doctor” activates semantically related lexical representations in memory, including “nurse”; Collins & Loftus Reference Collins and Loftus1975; Masson & Borowsky Reference Masson and Borowsky1998; Norris Reference Norris1995). Thus, just as “doctor” primes semantically related words such as “nurse,” words such as “illegal” may have primed related words such as “justice” (whereas words such as “limited” would not have primed unrelated words such as “exchange”).

One unique prediction of this alternative account is that, if the results are driven simply by semantic relatedness and its effect on memory, then any category should show a similar “pop-out effect” in similar circumstances – including completely arbitrary categories without any special importance in the mind. To test this possibility, we replicated the “moral pop-out effect” (Firestone & Scholl Reference Firestone and Scholl2015b), but instead of using words related to morality (e.g., “hero,” “virtue”), we used words related to clothing (e.g., “robe,” “slippers”). The effect replicated even with this trivial, arbitrary category: Fashion-related words were more accurately identified than non-fashion-related words (see Fig. 4B). (A second experiment replicated the phenomenon again, with transportation-related words such as “car” and “passenger.”) These results suggest that relatedness is the key factor in such effects, and thus that memory, not perception, improves detection of words related to morality. In particular, the work done by moral words in such effects (i.e., increased activation in memory) may be complete before any subsequent stimuli are ever presented – just as the spreading activation from “doctor” to “nurse” is complete before “nurse” is ever presented.

Similar investigations have implicated memory in other top-down effects. For example, labeling simple shapes reportedly improves visual detection of those shapes (Lupyan & Spivey Reference Lupyan and Spivey2008). When a certain blocky symbol () appeared as an oddball item in a search array populated by mirror images of that symbol (), subjects who were told to think of the symbols as resembling the numbers 2 and 5 were faster at finding a among s than were subjects who were given no such special instruction. However, it is again possible that such “labeling” doesn't actually improve visual processing per se but instead makes retrieval of the relevant memory representation easier or more efficient. Indeed, in similar contexts, the prevention of such labeling by verbal shadowing impairs subjects' ability to notice changes to objects' identities (e.g., a Coke bottle changing into a cardboard box) but not to their spatial configuration (e.g., a Coke bottle moving from one location in the scene to another), suggesting that the work done by such labels is to enhance contentful memories rather than visual processing itself (Simons Reference Simons1996).

In a follow-up, Klemfuss et al. (Reference Klemfuss, Prinzmetal and Ivry2012) reasoned that if memory – rather than vision – was responsible for improved detection of a among s, then removing or reducing the task's memory component would eliminate the labeling advantage. To test this account, subjects completed the same task as before, except this time a cue image of the search target was present on-screen during the entire search trial, such that subjects could simply evaluate whether the (still-visible) cue image was also in the search array, rather than whether the search array contained an image held in memory. Under these conditions, the label advantage disappeared, suggesting that memory was the culprit all along. (For another case study of perception vs. memory – in the context of action capabilities and spatial perception – see Cooper et al. Reference Cooper, Sterling, Bacon and Bridgeman2012.)

4.6.2. Other susceptible studies

Many top-down effects involve visual recognition and so may be susceptible to this pitfall. For example, under continuous flash suppression in a binocular-rivalry paradigm, hearing a suppressed image's name (e.g., the word kangaroo when a kangaroo image was suppressed) increased subjects' sensitivity to the presence of the object (Lupyan & Ward Reference Lupyan and Ward2013) – which they described in terms of the cognitive penetrability of perception. But this too seems readily explicable as an entirely “back-end ” memory effect: Hearing the name of the suppressed stimulus activates stored representations of that stimulus in memory (including its brute visual appearance; Léger & Chauvet Reference Léger and Chauvet2015; Yee et al. Reference Yee, Ahmed and Thompson-Schill2012), making the degraded information that reaches the subject easier to recognize – whereas, without the label, subjects may have seen faint traces of an image but might have been unable to recognize it as an object (which is what they were asked to report). (Note that this example might thus be explained in terms of spreading activation in memory representations, even with no semantic priming per se.) In another example, hungry subjects more accurately identified briefly presented food-related words relative to subjects who were not hungry (Radel & Clément-Guillotin Reference Radel and Clément-Guillotin2012); but if hungry subjects were simply thinking about food more than nonhungry subjects were, then it is no surprise that they better recognized words related to what they were thinking about, having activated the relevant representations in memory even before a stimulus was presented.

4.6.3. A lesson for future research

Given that visual recognition involves both perception and memory as essential but separable parts, it is incumbent on reports of top-down effects on recognition to carefully distinguish between perception and memory, in part because effects on back-end memory have no implications for the nature of perception. (If they did, then the mere existence of semantic priming would have conclusively demonstrated the cognitive penetrability of perception back in the 1970s, rendering the recent bloom of such studies unnecessary.) Yet, it is striking how many recent studies of recognition (including nearly all of those mentioned in sect. 4.6) do not even acknowledge that such an interpretation is important or interesting. (Lupyan & Ward [2013] attempted to guard against semantic priming by claiming that semantic information is not extracted during continuous flash suppression, but recent work now demonstrates that this is not the case; e.g., Sklar et al. Reference Sklar, Levy, Goldstein, Mandel, Maril and Hassin2012. Moreover, Lupyan and Ward worry about semantic priming, but they do not acknowledge the possibility of spreading activation among nonsemantic properties such as visual appearance; see Léger & Chauvet Reference Léger and Chauvet2015.) Even just highlighting this distinction can help, for example by making salient relevant properties such as relatedness (as in Firestone & Scholl Reference Firestone and Scholl2015b) when they would be obscure in a purely perceptual context (as in Gantman & Van Bavel Reference Gantman and Van Bavel2014). And beyond the necessity of highlighting this distinction, the foregoing case studies also make clear that this is not a vague theoretical or definitional objection but rather a straightforward empirical issue.