The main research program that aims at a beginning of an explanation of consciousness is based on a quest for the neural correlates of consciousness (NCCs): the minimally sufficient neural states that lead to a subject being conscious of a content (Crick and Koch Reference Crick and Koch1990; Chalmers Reference Chalmers and Metzinger2000).Footnote 1 Researchers typically rely on a comparison between two types of measurements for finding NCCs: measures of neuronal processes and measures of consciousness (Overgaard Reference Overgaard and Overgaard2015; Michel Reference Michel2017). By comparing measurement outcomes, one can judge which neural processes best correlate with consciousness. Establishing robust measures of consciousness is thus critical for this research program. Within this framework, the Perceptual Awareness Scale (PAS), a four-step scale used by subjects to report the degree of clarity of their experiences, has been widely used to assess participants’ levels of consciousness of contents (Ramsoy and Overgaard Reference Ramsoy and Overgaard2004). However, the development of the PAS in consciousness science faces the “problem of coordination,” namely, the problem of knowing whether a measurement procedure actually measures the quantity that it is intended to measure (Mach Reference Mach and Ellis1896; Chang Reference Chang2004; van Fraassen Reference van Fraassen2008).
Most critiques addressed against the PAS do so on the ground that it is an inaccurate measure of consciousness (Irvine Reference Irvine2012b; Wierzchoń et al. Reference Wierzchoń, Szczepanowski, Anzulewicz and Cleeremans2014). In this article, I argue that both proponents and critics of the PAS rely on the assumption that measures using the PAS are at least valid measures of consciousness, namely, that the PAS measures what it is intended to measure. On my view, the PAS fails to coordinate the quantity that it intends to measure (i.e., levels of consciousness) with the main quantity that it intends to measure consciousness with (i.e., reported levels of clarity of experiences). I conclude by showing how the use of the PAS could lead to systematic mistakes in the search for NCCs.
From the Response Criterion Problem to the PAS
Subjective and objective measures of consciousness are often distinguished (Irvine Reference Irvine2012a). A measure of consciousness is objective if the experimenter ascribes a conscious experience to a subject on the basis of her performance on a task. But, a measure of consciousness is subjective if the participant is required to produce a report about her current conscious experience. In this article, I focus on the latter kind of measures.
Subjective measurement procedures face a “response criterion problem.” When instructed to report whenever a stimulus is consciously visible, subjects must come up with a criterion to determine whether a subjective experience qualifies for the report. According to signal detection theory, a subject’s decision to report seeing a stimulus depends both on the accumulation of sensory evidence and on the subject’s response criterion determining whether to report seeing the stimulus (Macmillan and Creelman Reference Macmillan and Creelman2004). Importantly, categories such as “seen” or “unseen” are interpreted by subjects to set a response criterion depending on their own interpretations of these categories, such that stimuli that are only vaguely perceived might be reported either as “seen” or “unseen” (Merikle and Reingold Reference Merikle and Reingold1998). Accordingly, it is difficult for scientists to determine instructions that make sense for participants without introducing biases in the way in which they decide to report their experiences. Researchers must therefore make sure that subjects will report all the relevant experiences, and only those experiences, so that they can reliably infer the subjects’ conscious states from their reports. By doing so, scientists engage in a process that is similar to the process of calibration of measurement procedures observed in the natural sciences.
Calibration is the operation of “establishing a reliable relation between the indications of an instrument and relevant features of an object” or phenomenon (Tal Reference Tal2017, 33). In the cognitive neuroscience of consciousness, researchers aim at establishing a reliable relation between a subject’s report and the presence or absence of consciousness of a given content in that subject. For that purpose, calibration is typically accomplished by giving a set of instructions allowing researchers to make sure that subjects report all, and only, those features that are relevant for the experiment.
One of the main subjective measure of consciousness, PAS, was designed to solve the response criterion problem (Ramsoy and Overgaard Reference Ramsoy and Overgaard2004). As Sandberg and Overgaard (Reference Sandberg, Overgaard and Overgaard2015) explain: “Whereas the scientist has no external access to the contents of another person’s consciousness, one can still aim for a situation where reports stand in a ‘1∶1 relationship’ with the relevant inner states. Thus, even if the scientist cannot confirm such a relationship, a participant can inform the scientist that he/she can tell the difference between different degrees of visibility, and use this information to create the experimental categories for report. This can only be done, it seems, by involving the participants in the creation of those categories” (181). The PAS was developed by asking participants to design their own scale for rating properties of simple geometric shapes (shape, color, and precision), starting from “no experience at all” and ending with “a clear image” (although they were free not to follow these suggestions). Almost all subjects developed a scale with four categories: “no experience at all,” “brief glimpse,” “almost clear image,” and “clear image.” The “brief glimpse” category was used by subjects to report no awareness of form, color, or shape but only a vague experience of having seen something. Since subjects designed these categories themselves, one can thus suppose that the PAS avoids the confound of imposing response categories that subjects do not correctly understand or categories that would lead participants to rule out reports of some kinds of experiences.
The PAS is used not only to detect when subjects are conscious of given contents and when they are not but also to measure the level of consciousness of a given content. Indeed, it is assumed that subjects can be more or less conscious of some contents. Some contents are perceived “clearly and distinctly,” and are thus highly conscious, while other contents are perceived unclearly and in a somehow degraded way and are therefore less conscious (Overgaard et al. Reference Overgaard, Rote, Mouridsen and Ramsoy2006). As such, subjects introspect to produce indications, allowing experimenters to detect consciousness of given contents and to measure their levels of consciousness of those contents. In summary, a measurement outcome using the PAS is a statement that a subject has a given conscious experience, with a given level of consciousness, such that this statement is inferred from an indication, that is, a subjective report resulting from the subject’s introspection calibrated by PAS instructions.
The crucial point with this definition of a PAS measurement outcome is that measurement outcomes are different from indications. In order to get from an indication to a measurement outcome, a scientist has to interpret this indication as indicating something about what one intends to measure, usually by using a theory (Tal Reference Tal2017). For example, in order to measure temperature, one must interpret a thermometer indication, such as the height of a column of mercury in a thermometer, in light of a theory that relates the volume of mercury with temperature. An indication, by itself, does not tell us anything about the phenomenon we are interested in (van Fraassen Reference van Fraassen2008). Consequently, two steps must be distinguished in the production of a measurement outcome using the PAS. The first step is the use of introspection by a subject, given some instructions, to produce a report: this is the process that leads to a specific indication. The second step is the researcher’s interpretation of that indication as telling her something about the subject’s current experience and level of consciousness. It is only through this second, interpretative step that subjective reports become measurement outcomes.
Most debates focus on the first step of the production of measurement outcomes. Critics of the role of introspection in consciousness science argue that it is difficult to calibrate introspection such that it could provide reliable indications (Schwitzgebel Reference Schwitzgebel2011; Irvine Reference Irvine2012a, Reference Irvine2012b). This argument focuses on the accuracy of measurement procedures, defined as the closeness of agreement between measurement outcomes and the true value of the quantity that one intends to measure (JCGM 2008). On my view, these critiques are unnecessarily concessive to proponents of the PAS. The most important problem for the PAS is not a problem of accuracy but a problem of validity, or, as philosophers of science sometimes call it, a problem of coordination (Chang Reference Chang2004; van Fraassen Reference van Fraassen2008). A measurement procedure is valid if it measures the quantity that one intends to measure and not a different quantity. The problem of coordination thus touches on the second step of the measurement procedure: What entitles researchers to interpret reports as indicating something about conscious experiences or levels of consciousness? While the problem of accuracy presupposes that reports are indicating relevant aspects of consciousness, the problem of coordination questions the assumption that reports should be interpreted as indications of consciousness in the first place.
Consciousness Science and the Problem of Coordination
The Problem of Coordination
Empirical measurement procedures typically require the coordination of two quantities. For example, the height of a column of mercury in a thermometer, a quantitative property, can play the role of an indication that is then interpreted in light of a theory of temperature to generate a measurement outcome. To the extent that one can use a quantity such as the height of a column of mercury to measure another quantity, temperature, both quantities are coordinated.
Coordination requires some recognized regularities, so that one can discover the form of the function that links two quantities. The problem is that it is difficult to know whether variations in thermometer indications correlate with variations in temperature, and how so, before one is able to measure temperature (Chang Reference Chang2004; van Fraassen Reference van Fraassen2008). The problem of coordination, in other words, is that good measurement procedures require good theories, but good theories require good measurement procedures.
A first step toward solving this problem is the discovery of systematic correlations between variations in indications produced by measurement procedures and variations in the property that one intends to measure. This first step is crucial, since it is only through those correlations that one is able to assess the validity of the measurement procedure, that is, whether the procedure actually measures the property that one intends to measure (Cronbach and Meehl Reference Cronbach and Meehl1955). If variations in thermometer indications were to correlate with variations in properties other than temperature, such as, say, the color of measured objects, thermometers would not provide valid measures of temperature. By analogy, a procedure for measuring consciousness is valid only if variations in indications produced by that procedure correlate with variations in the presence or absence of consciousness, or with levels of consciousness, and do not correlate systematically with other properties that are independent from consciousness.
The various methods that aim at measuring and detecting consciousness all try to coordinate given quantities, such as confidence ratings (Norman and Price Reference Norman, Price and Overgaard2015) or visibility ratings (Sergent and Dehaene Reference Sergent and Dehaene2004), with the presence or absence of consciousness or with levels of consciousness. As noted above, the main quantity concept used in the PAS is the “level of clarity of an experience.” From this perspective, the main question is whether subjects categorize and report levels of clarity of their experiences depending on how conscious of these experiences they are. If a wide variety of factors other than levels of consciousness could produce variations in reported levels of clarity, it becomes at least open to question whether the PAS is a valid measure of consciousness.
I now argue that there are multiple plausible interpretations for all subjective reports resulting from the use of the PAS. As such, we have no reason to believe that the PAS is a valid measure of consciousness.
The PAS and the Phenomenological Fallacy
There are a variety of factors other than levels of consciousness that might lead to variations in reported levels of clarity (Irvine Reference Irvine2012b). Here, I suggest that reported levels of clarity of experiences do not vary depending on levels of consciousness of contents but depending on levels of intensity and precision of the contents of consciousness. Fazekas and Overgaard (Reference Fazekas and Overgaard2016, Reference Fazekas and Overgaard2018) argue that reported levels of clarity of experiences depend on two main characteristics of experiences: intensity and precision. The more intensely and precisely a content of consciousness is experienced, the higher the reported level of clarity of that experience. However, levels of intensity and precision of the contents of consciousness are not necessarily identical with levels of consciousness of those contents. Intensity and precision could be properties of the contents of consciousness and not properties of consciousness itself.
To understand this distinction, it is useful to go back to what Place (Reference Place1956) called the ‘phenomenological fallacy’: the error of concluding that properties of experienced contents are properties of experiences themselves. As Smart (Reference Smart1959) wrote against a similar mistake: “Trees and wallpapers can be green, but not the experience of seeing or imagining a tree or wallpaper” (151). The same thing goes for clarity, intensity, or precision: the contents of consciousness are clear, vague, intense, or precise. But consciousness of these contents itself is neither clear nor unclear; it is not intense or precise. To attribute intensity and precision to consciousness itself is to commit the phenomenological fallacy. Rosenthal (Reference Rosenthal2018) made this point clear: “Though variation in intensity of the content of a perception and in strength of awareness of the perception both affect overall experiential intensity, only variation that is due to the second source is strictly speaking a gradation of consciousness. It is plain that perceptual content can vary altogether independently of consciousness. Variation in intensity of perceptual content is not gradation of consciousness. One would think otherwise only on a theory that assimilates perceptual consciousness to the content of the perception, and so fails to distinguish subjective awareness of psychological states from the states themselves” (7). The phenomenological fallacy and the distinction between properties of contents and properties of consciousness are of fundamental importance if one is hoping to solve the problem of coordination. Indeed, measuring a representational property of the contents of consciousness is not the same as measuring levels of consciousness of those contents.
To be clear, I do not want to deny that there could be levels of consciousness, that levels of consciousness could contribute to experienced levels of clarity of contents, or that variations in reported levels of clarity could correlate partly with variations in levels of consciousness. All that is needed for my argument is that variations in reported levels of clarity do not correlate solely with levels of consciousness but also correlate with the way in which contents are built as clear or unclear by perceptual systems, independently of the fact that these contents are conscious or not.
Since variations in reported levels of clarity could correlate with both variations in properties of the contents of consciousness and variations in levels of consciousness, we do not know whether reported levels of clarity should be interpreted as delivering information about the former or the latter. Hence, for every subjective report using the PAS, two interpretations are available: either the report should be interpreted as indicating a given level of consciousness, or the report should be interpreted as indicating a given degree of clarity, precision, or intensity of a subject’s content of consciousness.
Validity and the PAS: An Inference to the Best Explanation
An alternative way to provide support for the validity of PAS measures could be to appeal to the fact that PAS measurement outcomes correlate with those of other procedures intended to measure consciousness. This strategy has been successful before. For instance, a measurement procedure relying on an electron microscope can be shown to be valid if its results correlate with those of a different procedure, such as a procedure using a light microscope, which one intends at measuring the same phenomenon (Hacking Reference Hacking1981). Here, correlations between outcomes of different measurement procedures are best explained by the fact that both instruments at least partly measure the same phenomenon.
Outcomes of PAS measures do not strictly correlate with outcomes of other measures, such as confidence ratings or postdecision wagering (Sandberg et al. Reference Sandberg, Timmermans, Overgaard and Cleeremans2010). Nonetheless, results of PAS measures correlate with so-called objective measures of consciousness, such as levels of performance in discrimination tasks. When subjects perform at chance level in sensory discrimination tasks, which is interpreted as indicating that they did not see the stimulus, they tend to use the lowest level on the PAS, and when they perform above chance, they tend to use higher levels (Overgaard et al. Reference Overgaard, Rote, Mouridsen and Ramsoy2006). However, to show that the PAS is valid on the basis of this correlation, one has to argue that the best explanation of the correlation between measurement outcomes using the PAS and other measures is that the PAS measures levels of consciousness.
On my view, the best explanation of the correlation between measurement outcomes of objective measures of consciousness and the PAS is not that the PAS is a valid measure of consciousness but that contents that are built by perceptual systems as being more precise and intense elicit better performances. Indeed, it does not really come as a surprise that when subjects are asked to discriminate between two stimuli with low contrast, low brightness, and low intensity, their performance is worse than when they have to discriminate between two stimuli with high contrast, high brightness, and high intensity. The way in which a content is built by perceptual systems as clear or unclear could lead to an increased or decreased performance and to a report using high- or low-level categories of the PAS. As long as we are not provided with a reason for thinking that performance is better on discrimination tasks in virtue of the fact that subjects are more conscious of stimuli, it is hard to see why a correlation between PAS ratings and performance should be interpreted as providing support for the validity of the PAS. Rather, the simplest explanation of this correlation is that contents that are clearer elicit better performances, independently of subjects’ levels of consciousness of these contents. Hence, one can explain the correlation between performance on discrimination tasks and PAS ratings without appealing to levels of consciousness.
Next, I show why we should care about the validity of measurement procedures in consciousness science by arguing that the uncritical use of the PAS might have led consciousness science astray in its quest for the neural bases of consciousness.
Neural Correlates of What?
In the search for NCCs, the minimally sufficient neural conditions for consciousness, scientists typically rely on a method called “contrastive analysis” (Baars Reference Baars1988). This method consists in comparing neural activity on trials in which a subject either consciously or unconsciously perceives a stimulus, for example, thanks to visual masking or binocular rivalry (Breitmeyer Reference Breitmeyer2015), while holding the stimulus constant. As such, it is thought that consciousness of a content can be manipulated as a variable, and one can thus record the neural states that best correlate with consciousness of contents and do not correlate with those contents remaining unconscious. As noted by Overgaard (Reference Overgaard2006), one obtains NCCs by matching two measures: a measure of the neural activity of the subject and a detection or measure of the subject’s consciousness of a content. My view is that the use of the PAS might have led consciousness researchers to mistake the correlates of the perceptual processes responsible for building contents as clear or unclear with the correlates of consciousness.
The search for the event-related potentials, or the measured electrophysiological activity of the brain, that best correlate with consciousness of contents provides a good example of the way in which the problem of coordination applies in consciousness science. There are currently two main candidates for the electrophysiological signature of consciousness: the N200, or Visual Awareness Negativity (VAN), a negative component that peaks at 200 milliseconds and is recorded in occipital and parietal areas (Koivisto and Revonsuo Reference Koivisto and Revonsuo2010; Rutiku, Aru, and Bachmann Reference Rutiku, Aru and Bachmann2016), and a late positive component, the P300 (or P3), which peaks at 300 milliseconds and is recorded in frontal areas (Sergent, Baillet, and Dehaene Reference Sergent, Baillet and Dehaene2005; Del Cul, Baillet, and Dehaene Reference Del Cul, Baillet and Dehaene2007).
In this debate, most studies using the PAS to investigate the NCCs bring support for the hypothesis that consciousness correlates with the VAN (Koivisto and Revonsuo Reference Koivisto and Revonsuo2010; Andersen et al. Reference Andersen, Pedersen, Sandberg and Overgaard2016; Tagliabue et al. Reference Tagliabue, Mazzi, Bagattini and Savazzi2016; Fu et al. Reference Fu, Liu, Dienes, Wu, Chen and Fu2017). The claim that the VAN correlates with consciousness is based on the observation that variations in the amplitude of the VAN correlate with variations of consciousness as measured by the PAS. For example, Andersen et al. (Reference Andersen, Pedersen, Sandberg and Overgaard2016) found that variations in PAS ratings could be predicted from magnetoencephalogram (MEG) signals originating in the occipital lobe in the VAN time range, while MEG signals originating in the prefrontal areas in the P3 time range could not be used to predict variations in PAS ratings. This result has been interpreted as indicating that “occipital activity seems a more likely candidate for a neural correlate of perceptual consciousness than does prefrontal activation” (2686).
However, this interpretation of the data assumes that variations in reported levels of clarity are best interpreted as indicating variations in subjects’ levels of consciousness. Yet, I argued that the different levels on the PAS might correspond not to several levels of consciousness but to several levels of clarity of the contents of consciousness. On this alternative interpretation of reported levels of clarity, the VAN does not correlate with consciousness itself. Rather, it is the best correlate of the clarity of the contents of consciousness. Hence, different interpretations of what the PAS actually measures lead to radically different interpretations of experimental results relying on the PAS. The view according to which the VAN correlates with a property of the contents of consciousness is perfectly compatible with the P3 component being the true NCC. Indeed, one could interpret the results as follows: first, a content is built as clear or unclear by perceptual systems, and this event correlates with the VAN; second, this content becomes conscious, and this event correlates with the P3 wave. Such an interpretation of the data is not available if one assumes that the PAS is a valid measure of levels of consciousness.
A neural correlate of the clarity of the contents of consciousness is not the same as a correlate of consciousness of those contents, and measuring the former does not amount to measuring the latter. Analyzing what we measure when using the PAS is of crucial importance in order to interpret data, as it determines what purported NCCs actually correlate with. Without measurement procedures that succeed at measuring consciousness, and demonstrably so, one cannot hope to establish correlations between neural states and consciousness, let alone between neural states and levels of consciousness. That is why measurement validity is so important.
Conclusion
While the accuracy of measures of consciousness is debated, their validity is often left unexamined. Here, I tried to lay the conceptual groundwork for analyzing measurement outcomes provided by measures of consciousness, focusing on the PAS. I argued that, as long as the problem of coordination is not solved, subjective reports can be interpreted in various ways. Consequently, I showed that experimental results that rely on the PAS for their measures of consciousness remain ambiguous. The problem of coordination should not be overlooked: assessing whether purported measures of consciousness actually measure consciousness is a crucial challenge for future research.
