3.1. Simulation and Validity

To answer our central question, we must first ask how simulation helps to constrain the space of possible mechanisms. First, simulations can be used to test a model's completeness. Models with gaps or poorly specified steps cannot be “run” because a program built to the model's specifications could not get past the black or gray boxes. The machine is missing crucial parts and functions. If one builds a simulation according to the blueprint provided by a model and the simulation does not work at all (it does not run), then as long as the simulation is an implementationally verified realization of the model, the model does not accurately describe how the target mechanism works. So, verified implementation of a model favors that model relative to models that cannot be implemented or whose implementability remains a matter of conjecture. Many box and arrow diagrams in cognitive science await computer simulations because they are too imprecisely specified or gappy to be implemented. This was Marr's (Reference Mauk1981) central complaint about cognitive science.

Second, simulations can be used to test a model's phenomenal adequacy. The modeler compares the results of experiments performed on the simulation to the results of experiments performed on the target (see Parker Reference Pérez-Ramos2009). Experiments on simulations involve fixing the values of some parameters in the model and checking the effects of that intervention on the values of other variables. The modeler then manipulates and measures the target mechanism to see if it behaves similarly when it works or to see if it responds similarly when the experimenter intervenes to set the target variables to particular values. Simulation allows one effortlessly to perform laborious calculations. It is therefore an efficient tool for exploring a model's empirical commitments. There is no principled difference, however, between simulating a model on a computer and simulating it by putting pencil to paper. Huxley, for example, worked out the consequences of their model with an adding machine.

The general point is that, if one builds the simulation according to a blueprint provided by the model, the simulation is implementationally verified, and the simulation does not behave like the target, then the model is not phenomenally valid. It can be removed from the space of possible models.

A simulation might behave like the target under standard conditions or under all relevant conditions within the target system but fail to do so under nonstandard conditions. A model or simulation is narrowly phenomenally valid to the extent that its input-output relationship is relevantly similar to the target input-output relation under standard or “normal” input conditions. A model is widely phenomenally valid to the extent that it mimics the target's behavior when the inputs are outside of the standard range as well. Widely phenomenally valid simulations are required to test for mechanistic validity given that one expects the simulation of a mechanism to break and malfunction in ways that are relevantly similar to the conditions under which the target mechanism breaks and malfunctions. This difference is crucial for distinguishing how-possibly from how-actually models (Craver Reference Craver2007, chap. 4). It is also crucial for understanding how prostheses and simulations differ.

Turn now to a simulation's mechanistic validity. The space of implementable and phenomenally adequate models for a given phenomenon is typically very large. Phenomena are multiply realizable in lower-level mechanisms. Multiple realizability obstructs the inference from a model's phenomenal validity to its mechanistic validity. The space of phenomenally adequate simulations might well be too large and heterogeneous to provide any assurance that the mechanistic features of a phenomenally adequate simulation are relevantly similar to the mechanistic features of the target. As the set of phenomenally adequate models grows, the confirmatory value of phenomenal validity with respect to mechanistic validity diminishes.

For this reason, many simulations in neuroscience are intended to be more or less mechanistically valid or plausible. Researchers often use simulations to test a model's mechanistic validity. They compare the changes of internal variables in the simulation to the changes observed among the measurable variables in the target mechanism. Again, simulation per se adds no epistemic force that models do not already have. Simulation simply takes the math out of our hands.

Consider now the use of simulations to test affordance validity. For simulations, affordance validity amounts to phenomenal validity. One forms a hypothesis about the relevant input-output function exhibited by the target mechanism and builds a simulation to mimic that input-output function. As a result, simulations count as tests of affordance validity only on the assumption that one has correctly identified the relevant interfaces between the target mechanism and its containing system and that one's instruments reliably detect the relevant properties in the system.

Models and simulations almost always differ in some respects from their targets. The question of fidelity arises: How closely must they match to be judged phenomenally valid (see Weisberg Reference Weisberg2007)? Prosthetic models set a high bar for fidelity criteria. The epistemic act of comparing the simulation's input-output relation to the target's input-output relation is replaced in prosthetic modeling by a real-time causal interface between the simulation and the target.

3.2. Building a Prosthesis and Knowing How It Works

So now, what constraints do prosthetic models add to the space of possible mechanisms beyond those provided by simulations?

Consider mechanistic validity first. Prosthetic models at their most biologically realistic are engineered simulations. As such, they inherit the epistemic problem of multiple realizability. A prosthetic model might be affordance valid and phenomenally valid yet mechanistically invalid. Prosthetic runners’ legs do not work like typical biological legs. Heart and lung machines do not work like hearts and lungs. If so, then building a functional prosthesis that simulates a mechanistic model is insufficient to demonstrate that the model is mechanistically valid.

One might proceed from this point to compare the internal states and transitions of the simulation in the prosthesis with the internal states and transitions of the target, but this comparison is no more and no less difficult for prosthetic modeling than it is for simulation.

Next consider the model's phenomenal validity. The crucial difference between prosthetic and nonprosthetic simulations lies in the interfaces. To what extent should success in building a prosthesis increase confidence that one has correctly identified the target's working inputs and outputs?

Judging from current prosthetic devices, the epistemic problem of multiple realizability discussed above arises again at the level of inputs and outputs. On the input side, consider the use of brain-machine-interface (BMI) devices to drive a cursor around a computer screen, to play brain-PONG (a version of the 1972 video game made more exciting by the fact that it is played with electroencephalogram [EEG] waves), and to move a robotic arm (see Schwartz Reference Schwartz, Cui, Weber and Moran2004; Lebedev and Nicolelis Reference Lebedev and Nicolelis2006; Schwartz et al. Reference Velliste, Perel, Spalding, Whitford and Schwartz2006; Velliste et al. Reference Velliste, Perel, Spalding, Whitford and Schwartz2008). Brain signals are the inputs to these devices. Behaviors are the outputs. No currently available BMI device, to my knowledge, makes use of just those brain inputs that move limbs in typical animals. EEG interface systems use gross cortical waves; electrophysiological interface systems are driven by 15–200 neurons. The goal, rather, is to sample the electrical activity across the cortex to find signals that can be used or that subjects can learn to use to control a prosthesis.

For example, patients can be trained in a few days to use EEG signals to move a robotic arm or a paddle in brain-PONG. EEG signals, however, are widely believed to be epiphenomenal with respect to the mechanisms that produce motor output. They reflect synchronization and desynchronization of cortical activity that correlates with behavior, and one can be trained to voluntarily produce changes in EEG waves to obtain some outcome, but the EEG wave itself is not the functionally significant output of brain motor systems (or so we think). Furthermore, the EEG waves used as input to the prosthesis need not be recorded from the motor cortex: brain waves recorded from auditory cortex, for example, can also be commandeered for this purpose (Felton et al. Reference Felton, Wilson, Williams and Charles2007).

Miguel Nicolelis emphasizes that “precise knowledge of computations performed by brain circuits is not crucial for the construction of clinically relevant BMIs. Mostly BMI platforms take advantage of the well-known correlation between discharges of cortical neurons and motor parameters of interest, and perform a reverse operation: they predict motor parameters from patterns of neuronal firing. Generally, predictions of motor parameters do not signify a causal relationship between the neuronal activity and the generation of movements” (Lebedev and Nicolelis Reference Lebedev and Nicolelis2006, 540). Indeed, in electrophysiologically driven BMIs, the input signal is typically recorded from 15–200 neurons, which is certainly only an imperfect sample of the neurons that contribute to the behavior in the typical organism, if they contribute at all. The brain states that drive these prostheses need not be states that contribute to movement in subjects with full use of their limbs, and so building a prosthesis need not indicate that one has chosen the correct inputs.

On the output side, the prosthesis must produce outputs that can be used by downstream components in the system. Sensory prostheses must produce outputs that can be interpreted by the nervous system as meaningful signals. However, the lesson from the current generation of sensory prostheses is that the outputs need not be identical to, and in some cases do not even approximate, the signal used by typical biological sensory systems. Recent work on vision substitution, in which camera input is translated into mechanical output on the tongue or back, shows that the output of the prosthesis need not match the typical input to the brain. The substantial capacity of the brain to reorganize in the face of changing input signals further expands the space of affordance-valid outputs from prosthetic devices. The brain is extremely adaptive under the right circumstances. The set of affordance-valid prostheses is arguably much larger than the set of actual mechanisms exhibited in Homo sapiens at the present.

The lesson to be drawn from these examples is that to interface with the brain is not sufficient to demonstrate that one's model is phenomenally adequate. The space of functional inputs and outputs is larger than the space of functional inputs and outputs that development and evolution have thus far had occasion to exploit. This fact is liberating for engineers: they need not be shackled to the use of inputs and outputs that the standard, untrained, human brain happens to use. They are limited rather by what the brain can reasonably become in the allotted time and in available environments given the ethical, scientific, and technological limits of the age.

The situation is even more complicated for central systems involved in cognitive operations than for peripheral sensory and motor functions. In examples from the periphery, one of the interfaces is sufficiently well characterized that researchers can use it as firm ground from which to explore the other interfaces. In building a prosthetic brain region, both input and output are equally mysterious.

Take the long-range project of Berger and colleagues to build a prosthetic hippocampus. (For a fascinating review of progress to date, see Berger et al. [Reference Berger2005].) The dominant excitatory hippocampal trisynaptic loop projects from dentate gyrus (DG) to the CA3 region and then to the CA1 region. Their first major phase of the project is to create a replacement for the CA3 region. The first step of this phase is to do so in a 400-micrometer hippocampal slice. The prosthesis links DG output to CA1 input in a way that preserves the input-output pattern instantiated by the connections from DG to CA3 to CA1 in the intact rat hippocampus.

Brain regions transform spatiotemporal patterns of activity in a population of presynaptic neurons into spatiotemporal patterns of activity in a population of postsynaptic neurons. Populations of neurons transform the sequence of interspike intervals input to a neuron's dendrites into the sequence of interspike intervals output from the neuron's axon. To build the prosthesis, Berger et al. begin by constructing a mathematical model of the input-output function performed by the CA3 region. They stimulate the DG region in hippocampal slices at physiologically relevant intensities with randomly varying interspike intervals while determining experimentally (i) the interspike intervals and population spike amplitudes in CA3 and (ii) the timing and amplitude of excitatory postsynaptic potentials in CA1. The model constructed on the basis of these experiments is then implemented in a chip that interfaces via a multiunit electrode with both the dentate gyrus and CA1.

Even at this grain of description, it is clear that Berger et al. have chosen inputs and outputs that are different from those used by the typical hippocampus. First, limits of microchip and electrode technology prohibit the authors from modeling and interfacing with each individual neuron in the region of interest. Their prosthesis works on population measures of neuronal activity. If this simplifying strategy works (see Berger et al. Reference Berger2005), the prosthetic model will work on aggregates of the inputs and outputs used by the target mechanism. Second, the authors’ current prosthesis works in a razor-thin slice of hippocampus, removed from its context in the rest of the hippocampus and in the brain. The assumption that patterns in individual slices can be added up to produce the corporate behavior of thousands of adjacent slices remains to be tested. Last, in focusing on spike trains, Berger et al. intentionally remove other contextual factors (such as sleep-wake patterns, the endocrine environment of the hippocampus, and any other factors that might vary with aspects of the learning environment, including the content, the training, and other incidental factors).

Berger et al.'s model admirably captures the amplitude of the response in CA1 to trains of impulses delivered to the dentate gyrus. This is remarkable. Yet the authors recognize it is only a first step to demonstrating that such a prosthesis could work. In particular, it does nothing to demonstrate that the DG to CA1 input-output pattern beautifully replicated by the prosthesis is in fact the input-output pattern required for the hippocampus to play its role in memory encoding and storage.

That said, scores of computational models have been proposed for the hippocampus. For none of them is it currently possible to demonstrate that an animal would learn if the hippocampus worked as the model suggests. A successful prosthesis would answer this challenge definitively. To answer this challenge, however, is not to demonstrate that one understands how the target mechanism works. One can build a prosthesis by commandeering signals available in the brain for new purposes rather than by building a device that can take target inputs as prosthetic inputs and give target outputs as prosthetic outputs.

This point is driven home when one considers the well-documented plasticity of the brain (see Nicolelis Reference Parker2003, 421). Just as brain systems recover and rewire in response to brain damage and disease, they also reorganize to accommodate new devices (Lebedev et al. Reference Marr and Haugeland2005). This is one of the fundamental lessons driven home by BMI research. For example, subjects can learn reliably to produce motor output linked to the behavior of as few as 15 cortical cells. Indeed, BMI research is now routinely directed at understanding how the brain changes to accommodate these devices.

These considerations show that it is helpful to distinguish phenomenal validity from affordance validity. Phenomenal validity is a matter of whether the model instantiates the input-output relation by which the target interfaces with its environment. Affordance validity, in contrast, is a matter of whether the model instantiates an input-output relationship that the brain could use to replace the target's function. A prosthetic model demonstrates that the brain can be made to work with a simulation, even if it falls short of demonstrating that the brain works like the simulation.

Affordance validity is a matter of degree. A prosthetic with no affordance validity cannot be used to replace or restore function. A prosthetic with maximal affordance validity performs optimally. Qualitatively, a “commandeering index” to reflect affordance validity would depend on (1) the capacity of the target to accept inputs, (2) the capacity of the target to adapt to new inputs, (3) features of the subject's social and physical environment, (4) the available interface technology, and (5) the mechanistic and phenomenal details concerning the implemented model. It remains to be seen whether phenomenally and mechanistically valid interfaces are, in fact, optimal (achieving the highest value of affordance validity) or whether (as seems likely) it is possible to alter nature's handiwork and find new ways to do old things with brains.