2. Exploratory experiments, confounds, and control

Philosophical work on whether experiments in neuroscience generate knowledge has focused predominantly on experimental tests of causal hypotheses linking neural activity to behavior. Although this makes good sense given that most experiments in neuroscience aim to discriminate among competing causal hypotheses about phenomena of interest, the emphasis has served to obscure how knowledge production works in neuroscience and the fundamental role that exploratory practices (e.g., Steinle Reference Steinle1997) play in this endeavor. Relevant to my aims in this article are the confound probing functions that Jutta Schickore (Reference Schickore2019) attributes to exploratory experiments (EEs) at the frontiers of science.

Schickore (Reference Schickore2019) evaluates Charles Darwin’s research on digestion in insectivorous plants (e.g., bladderwort). As she explains, Darwin was interested in determining whether the presence of animal matter was necessary for protoplasm production inside plant bladders. Darwin used a solution to remove animal residue from a group of bladders and then ran a series of experiments exposing them to different substances (e.g., ammonia, urea). According to Schickore, these EEs served as diagnostic probes that enabled Darwin to gain an understanding of “factors that may have an impact on (disrupt, change)” or confound “the relationship between the independent and the dependent variable in an experimental situation” (ibid., 210). They allowed Darwin to construct what Schickore (ibid., 211) dubs a “confounder repertoire,” which she claims differs from Deborah Mayo’s (Reference Mayo1996) “error repertoires” because rather than unearthing “canonical” kinds of experimental errors like mistaking correlations for causal relations or misidentifying artifacts as real effects, “diagnostic probes” are devised to detect confounders idiosyncratic to specific experimental contexts. Investigators may struggle to determine what they are and how and where to probe for them. For example, Darwin had to hypothesize what kinds of factors may impede his experiments, what kinds of substances to test to determine their impact on digestion, and how best to clean bladders so no foreign substances remained.

Such exploratory work, however, does not end once potential confounders have been identified. Another type of EE, which Schickore dubs a “determinative probe” (Schickore Reference Schickore2019, 215), also must precede experimental tests of causal hypotheses. Investigators must determine whether potential confounders they have identified in the context of diagnostic probes have been adequately controlled in their experimental setups prior to running experimental tests of causal hypotheses. Darwin, for example, had to ensure in his experiments on protoplasm production that the experimental and control groups differed only with respect to the independent variable. The relevant notion of control, then, is the practice of ensuring that the effects of variables other than the independent variable on the variable of interest are minimized and, in the ideal case, completely absent from the experimental arrangement.

Importantly, Schickore emphasizes that diagnostic probes are rarely if ever exhaustive in unearthing potential confounds, leaving open the possibility that experimental tests of causal hypotheses occurring on their heels will fail to be appropriately “controlled” or sufficiently “severe.” Yet this means correctives must be in place to ensure that experimental tests of causal hypotheses eventually evolve in ways that increase control or “severity” (Mayo Reference Mayo1996). Schickore proposes her concepts of diagnostic and determinative probes could inform peer review in scientific journals by requiring manuscript authors to identify explicitly the types of probes used in their research studies. I believe that Schickore’s conceptual distinction also may be used as a guide for determining where correctives are and/or should be in place in the scientific process to improve control over scientific tools. Yet, I part ways with Schickore insofar as I take diagnostic and determinative probing always to be informed by a particular theoretical perspective or point of view (Sullivan Reference Sullivan2018) and therefore never “exploratory” in Steinle’s (Reference Steinle1997) sense of that term. In what follows, I use a historical case study from rodent behavioral neuroscience to demonstrate that, in some areas of neuroscience, perspectivally informed diagnostic and determinative probing are fundamental and routine parts of the research landscape and culture when novel tools for testing causal hypotheses are first developed and as they come to be used more widely within a research community.

3. Rodent touchscreens and early diagnostic probes

Rodents are widely used in neuroscience because they are inexpensive to procure and maintain, are regarded as representative models for the mammalian brain, and can be readily used in intervention experiments. Research in rodent behavioral neuroscience has historically involved the use of transient or permanent intervention techniques to alter brain activity and behavioral assessment tools to determine the effects of such interventions on behavior. In a typical experiment, an investigator aims to manipulate a single variable, such as neuronal activity in a single brain region, to determine the impact on behavioral performance in a task designed to assess a cognitive function. If differences in behavioral performance between the experimental and control groups are observed, the activity of neurons in the region that was the target of the intervention are interpreted as playing a causal role in behavioral performance on the task and in the psychological function the task is designed to measure. To establish such causal relationships, intervention techniques must act upon a single causal variable and behavioral assessment tools must individuate a single psychological function.

Prior to the early 1990s, behavioral tools for assessing cognition in rodents were limited to classical and operant conditioning chambers and different types of mazes. Given that rodents were assumed to be capable of more complex and human relevant forms of cognition than could be assessed using these tools, researchers were in pursuit of novel behavioral techniques. At that time, a common approach for studying the neural correlates of cognitive functions in brain-damaged human patients and brain-lesioned nonhuman primates was to use touchscreen-based tasks (e.g., Sahakian and Owen Reference Sahakian and Owen1992) with two-dimensional visual images presented on a computer screen. Subjects responded to visual images with a finger press to the screen and were provided feedback for correct and incorrect responses. Because rodents were thought to lack visual acuity to see and respond to computer-based stimuli, however, skepticism about the feasibility of developing a comparable touchscreen-based tool for assessing cognition in rodents was widespread.

Undeterred by such skepticism, in the early 1990s, Timothy Bussey and colleagues decided to place rats in a touchscreen operant chamber designed to investigate visual discrimination learning in pigeons to see what would happen (see Sullivan et al. Reference Sullivan, Dumont, Memar, Skirzewski, Wan, Mofrad, Ansari, Li, Muller, Prado, Prado, Saksida and Bussey2021) and found that rats “could attend and respond to [the] computer graphic visual stimuli (Bussey, Saksida, and Wilkie, unpublished observations)” (Bussey et al. Reference Bussey, Muir and Robbins1994). Bussey and his collaborators (ibid.) thus set out to build a species-specific apparatus for rats. In designing and constructing the apparatus (Figure 1), they needed to anticipate, explore, and test potential variables that might compromise its appropriateness for testing causal hypotheses about the role of specific brain areas in reward-based forms of visual discrimination learning. They engaged in “parametric analyses”—a type of diagnostic probing in which different mutable components of an experimental setup are systematically varied to determine their impact on the dependent variable (e.g., behavior). Mutable features of the original rodent touchscreen included the video-display unit (VDU) on which 2-D visual stimuli were presented and its features (e.g., touch or pressure sensitive surround to detect nose pokes), features of the visual stimuli (e.g., size, shape, color, luminance), type of rewards (e.g., sucrose pellets or chow), location of the reward-delivering food magazine, spatial arrangement of the apparatus and types of materials used for constructing it (e.g., wood, plexiglass), and features of those materials (e.g., sound attenuating, wipeable).

Figure 1.

The first rat touchscreen operant apparatus. Lister-hooded rat rearing and placing paws on plexiglass shelf (B) facing stimuli presented on video display unit (A) where computer graphic stimuli are presented in windows created by black plexiglass mask. Reward magazine at back of chamber (E), pressure-sensitive floor panel (D), and wooden sound attenuating box (F) (Bussey et al. Reference Bussey, Muir, Everitt and Robbins1997). Reprinted with permission.

Given that the apparatus was developed for use with different computer-automated learning tasks, it is relevant to note that each task and task protocol also had mutable features. For example, consider the delayed nonmatching to sample (DNMS) task, which was used in the original study to test working memory in the touchscreen. In the original touchscreen version of this task, a rodent learned to select a different stimulus from one it has just seen. During the training condition, an image appeared in one of three locations on the touchscreen. When the rat nose-poked the image, it disappeared, and two images appeared in the other two windows of the touchscreen-the image just seen and a different image. If the rat selected the correct image, the magazine light went on, the tone sounded, and it received a food reward; incorrect responses received a 5-second time-out. The DNMS task had variable features in addition to those that were part of the touchscreen apparatus including the location where the sample stimulus appeared on the screen, the number of training and testing trials and duration of training and testing sessions, the duration of intertrial intervals, and the time between training and testing sessions.

Bussey and colleagues knew to build standard “canonical” controls into the apparatus (Figure 1) and cognitive testing method in response to an established repertoire of potential confounds arising from historical work on operant conditioning in rodents. For example, to ensure rats were not distracted by extraneous stimuli (e.g., sounds) during training and testing, the testing chamber, which was plexiglass, was placed in a “sound attenuating” wooden box and a ventilation fan served to “mask extraneous noise” (Bussey et al. Reference Bussey, Muir and Robbins1994, 105). Rats were placed on a food restrictive diet prior to training and testing, a common strategy to ensure motivation to learn in reward-based learning experiments. Computerized automatization of touchscreen-based tasks like DNMS served to eliminate investigator errors and biases and reduced investigator-animal interactions, which could have unintended impacts on the animal’s behavior.

Other features of the apparatus and associated testing method resulted from parametric analyses testing different configurations of the apparatus while training rats in visual discrimination learning tasks. These “diagnostic probes” yielded novel information instrumental for improving the design of the apparatus for testing causal hypotheses. For example, sucrose reward pellets originally were delivered in food cups just below visual stimuli on the touchscreen. When Bussey and colleagues “observed that rats appear[ed] to orient toward the food cup rather than the stimuli,” making it difficult to determine “what,” if anything, was being learned, “subsequent configurations of the apparatus delivered food reward” using a food magazine in “the rear of the chamber” (Bussey et al. Reference Bussey, Padain, Skillings, Winters, Jennifer Morton and Saksida2008, 520) directly opposite the VDU.

Other parametric analyses were aimed at ensuring rats had full view of and were responding to all visual stimuli presented on the touchscreen and learning the intended stimulus relationships. These analyses prompted modifications to the design of the apparatus. For example, a pressure sensitive floor panel was installed in front of the food magazine directly opposite the VDU so that only when a rat was on the panel would the computer program initiate the next behavioral trial. The distance from the panel to the VDU was selected to ensure a rat would approach the screen “head on,” with all visual stimuli in full view. A plexiglass mask placed over the VDU, “allowing access to the stimuli only through windows just larger than the stimuli” (Bussey et al. Reference Bussey, Muir and Robbins1994, 109) served to focus a rat’s attention on those parts of the screen on which visual stimuli appeared. To ensure that a rodent’s nose-poke response to a stimulus was neither impulsive nor inadvertent, a plexiglass shelf was placed directly underneath the stimulus windows, “forc[ing] the animal to stop in front of the stimuli, rear [on its hind legs], and stretch toward a stimulus to make its response” (ibid.). The infrared touchscreen was designed to detect nose-pokes to an image without a rat having to physically nose-press the screen—a response that when repeated could cause discomfort that may impede learning. As rats were known to exhibit spatial response biases, if a rat subject “developed a spatial response bias (defined as >70% responses to the left or right window within a session)” during training, the investigators (ibid., 106) “[implemented] a correction procedure.”

Bussey and colleagues emerged triumphantly from these extensive parametric analyses with an innovative apparatus for testing causal hypotheses linking neural activity to behavior in rodents. The tool was also revolutionary insofar as it could be used to compare the performance of rodents with brain lesions and their brain-damaged human counterparts on similar kinds of psychological tasks, enabling cross-species comparisons linking neural activity to cognition and behavior. Yet, as I explain in the next section, during the next two decades after its introduction in the literature and the development of a similar apparatus for mice, concerns related to whether rodent touchscreens could be used to run adequately controlled tests of causal hypotheses linking neural activity to behavior began to emerge. As the community of rodent touchscreen researchers began to grow (Dumont et al. Reference Dumont, Salewski and Beraldo2021), hypothesis-driven experiments aimed at improving the knowledge-producing capacity of rodent touchscreens began to be undertaken in parallel with experiments using the tool to test causal hypotheses linking neural activity to behavior.

4. Evolution of rodent touchscreens

During the next decade, Bussey and colleagues began using the rat touchscreen operant chamber for hypothesis-driven experiments designed to determine the role of specific brain areas in different types of visual-based learning. One set of experiments compared the performance of rats with quinolinic acid-induced lesions in anterior and posterior cingulate and medial frontal cortices and “sham” (operated on surgically but not lesioned) rats on visual discrimination learning tasks (e.g., Bussey et al. Reference Bussey, Muir, Everitt and Robbins1997). In another study, Arjun Sahgal and Thomas Steckler (Reference Sahgal and Steckler1994) used a similar experimental arrangement (“TouchWindows”) to determine the impact of intraperitoneal amphetamine and serotonin agonist injections on performance in a task designed to assess visual attention.

By 2008, several other research groups (34 studies in total) had used rodent touchscreen operant chambers in some form to study behavior and/or test causal hypotheses linking brain lesions or pharmacological manipulations to behavior (see Bussey et al. Reference Bussey, Padain, Skillings, Winters, Jennifer Morton and Saksida2008; Dumont et al. Reference Dumont, Salewski and Beraldo2021). In part because the overall configuration of the rat touchscreen apparatus was not standardized across laboratories, with different researchers using different (a) types of touchscreens (b) types of food reward, (c) placements of reward magazines in the chamber, (d) types of visual images (e) types of learning tasks, (f) types of training and testing protocols, and (g) types of rat strains, novel questions began to emerge about additional variables that had not been previously considered that may confound tests of causal hypotheses the apparatus was used to run. In the context of causal hypothesis-driven experiments, researchers engaged in probative work, analyzing the behavior of their rodent subjects during training and testing and entertaining the possibility that observed changes in behavior that were taken as indicative of forms of visual discrimination learning may be due instead to other factors including biases for certain visual stimuli, poor visual acuity (making it difficult to assess the kind of stimulus relationships being learned), and differences in intensity between stimuli used in the context of the same experiment in which a choice between stimuli had to be made.

Bussey and colleagues (Reference Bussey, Padain, Skillings, Winters, Jennifer Morton and Saksida2008) engaged in a set of hypothesis-driven experiments to probe for these potential confounders, to systematically investigate those parameters that had come to be recognized as potentially impacting learning in the apparatus with an eye toward “optimizing” the touchscreen cognitive testing method for causal hypothesis-testing (ibid.). To take one representative example from this study, although it was known that rats exhibited stimulus preferences in experimental contexts in which three-dimensional objects were used, whether they preferred some two-dimensional touchscreen computerized visual stimuli over others had not been previously explored. If rats did have stimulus biases, an investigator would be unable to determine if changes in behavior following training were due to bona fide changes in cognition or merely preferences for certain stimuli. In intervention experiments using drugs or lesions to manipulate brain activity, then, she also would be unable to determine if the observed effects on behavioral performance in the apparatus were due to the intervention impacting cognition or instead “affecting … the expression of a stimulus bias” (ibid., 517).

In one experiment to probe for stimulus biases, Bussey and colleagues (ibid.) selected six pairs of then-standard touchscreen-based visual stimuli (e.g., plane–flower; spider–scissors) and presented them side-by-side in pairs in a single learning session to Lister Hooded male rats. Rats had to select one of two stimuli with a nose-poke; one stimulus was rewarded with a sucrose pellet, the other was unrewarded. To determine stimulus bias, they compared each rat’s performance on this first session to chance (50% correct). They determined that rats expressed a bias that was significantly different from chance with respect to one stimulus (the ribbon) in one pair of stimuli tested (ribbon–cross). This experiment served two functions. First, it identified a bias that could be eliminated by removing the bias-producing stimulus from the bank of touchscreen-based stimuli to be used in future experiments. Second, it specified a method for detecting spontaneous stimulus biases that might disrupt assessments of visual discrimination learning in the apparatus that could be used in future experiments aimed at establishing causal links between brain and behavior.

Yet, exploratory research to probe for previously unconceived potential confounds related to the touchscreen cognitive testing method for rodents did not end with Bussey and colleagues’ (Reference Bussey, Padain, Skillings, Winters, Jennifer Morton and Saksida2008) paper. Rodent touchscreen technology and associated tasks and testing methods have continued to evolve and been improved over the past two decades. In 2001, Bussey, Lisa Saksida, and Lawrence Rothblatt described a novel touchscreen operant apparatus to assess learning in mice using touchscreen-based tasks. With the introduction of cutting-edge technologies for creating genetically altered mice and for intervening in neural activity in awake behaving mice (e.g., chemogenetics, optogenetics), the past two decades have witnessed a steady increase in the use of rodent touchscreens and with it a growing community of researchers using them not only to test causal hypotheses but also working collaboratively to probe for confounders, improve experimental control over the behavioral apparatuses and associated testing methods, and gain a better understanding of the kind of causal knowledge they can be used to generate. I have space to consider only a few examples of this type of work here (but see Dumont et al. Reference Dumont, Salewski and Beraldo2021; Sullivan et al. Reference Sullivan, Dumont, Memar, Skirzewski, Wan, Mofrad, Ansari, Li, Muller, Prado, Prado, Saksida and Bussey2021 bibliographies).

Some hypothesis-driven studies aimed at diagnostic probing have sought to determine whether mice exhibit preferences for specific stimuli or stimulus locations in touchscreen-based tasks (e.g., Delotterie et al. Reference Delotterie, Chantal Mathis, Dorner-Ciossek and Anelise2014) or whether they have biases for specific images or their aspects (e.g., Belarde et al. Reference Belarde, Claire Chen, Yang and Troy2021). As new learning tasks have been proposed for use in touchscreens, probative work also has shaped their development and optimization (e.g., Delotterie et al. Reference Delotterie, Chantal Mathis, Dorner-Ciossek and Anelise2014). Another experimental study sought to assess whether different appetitive rewards used in mouse touchscreens (e.g., super-saccharin or strawberry milkshake) correlate with differences in task performance. That some rewards are more reinforcing than others may explain differences in behavioral results across labs using different reinforcers (e.g., Phillips et al. Reference Phillips, Christopher Heath, Bussey and Saksida2017).

In another example of a hypothesis-driven experiment aimed at diagnostically probing mouse touchscreens, Helene Richter and colleagues, who previously used touchscreens to assess cognitive flexibility in mice, sought to determine whether touchscreen training, which can “last several weeks until months” with “animals undergo[ing] daily training sessions of durations up to one hour,” causes a stress response detectable in the “physiology and behavior” of mice despite being reward-based (Krakenberg et al. Reference Krakenberg, Maximilian Wewer, Kaiser, Sachser and Helene Richter2021). In their study, compared to control mice, they found that mice trained in a paradigmatic version of the touchscreen Visuomotor Conditional Learning task exhibited significant increases in anxiety-like behavior and had increased levels of corticosterone metabolites in their fecal matter. Given these findings, which they took to indicate “profound effects” of touchscreen training on the physiology and behavior of their mice, they suggest that future “hypothesis-driven research” should be directed at clarifying “the influence of touchscreen training on the affective state of mice” and “disentangl[ing] the effects of different aspects of touchscreen training procedures” on behavior (Krakenberg et al. Reference Krakenberg, Maximilian Wewer, Kaiser, Sachser and Helene Richter2021).

This brief analysis of some of some stages in the evolution and history of rodent touchscreen operant chambers reveals some important features of the dynamics of behavioral tool development. Perspectivally informed exploratory experiments serve as initial probes to identify confounders and are fundamental for getting hypothesis-driven experiments linking the brain and behavior off the ground. However, these initial probes are rarely if ever exhaustive, nor can they be, because the experimental setups and the contexts in which they occur have many moving parts, and even when investigators due their due diligence to itemize and control for confounders, recognition of other possible confounders emerges only with time as the behavioral tool is used more regularly and more widely by investigators having different theorectical perspectives, as variables not previously considered are investigated and as conflicting results within or across research groups prompt further probative work to improve the tool for causal hypothesis-testing.

One implication of this, then, is that hypothesis-driven research using a behavioral tool serves as an important context for looking for and thinking about potential confounders that may later become the focus of hypothesis-driven experiments to probe for those very confounders, which may then yield findings fundamental for improving the behavioral tool and its appropriateness for testing causal hypotheses or shedding new light on the precise kind of causal knowledge the tool may be used to generate.