Newly minted scientists will never read everything published in their subfield; there is just too much science done these days. Yet they must still design useful new experiments, despite their limited view of what has already been done. That is a tall order. Computers are good at mapping and visualizing large data sets. By harnessing their power, one might see where the ground has already been well trodden and where gaps remain.
Such is the suggestion in Engineering the Next Revolution in Neuroscience by Alcino J. Silva, Anthony Landreth, and John Bickle. As the authors note, in molecular neuroscience “the experimental record now grows at a rate that outstrips the human capacity to integrate it” (124). Silva (a prominent molecular neuroscientist), Landreth (an entrepreneur and former member of Silva’s lab), and Bickle (well known to philosophers of science) outline a hypothetical method for computerized mining of research results. They focus on one subfield, studies of long-term potentiation (LTP), but the method is meant to be generally useful for neuroscientists. After introducing the project (chaps. 1 and 2), the authors taxonomize both experiments and analyses in molecular neuroscience (chaps. 3 and 4). They use this taxonomy to develop a theory about experimental reliability (chaps. 5 and 6). This is then integrated into a hypothetical system for visualizing the experimental landscape in molecular neuroscience (chaps. 7–9). Interspersed throughout, ostensibly by way of illustration, is a historical narrative about the challenges of studying LTP.
I will start with the big vision and work backward. In a representative passage, the authors envision “a glowing map of every phenomenon studied in neuroscience, laid out in constellations of hypothesized causal interactions. Surveying its landscape, we find each tentative causal connection appended with a dependability score, based on the outcomes of the different forms of Integration analyses. … We type into the search bar: ‘highlight all causal paths involving CA1 and CaMKIII with dependability scores higher than n’ and a legion of phenomena and their connections fade into the background, leaving glowing only the part of the network to which we limited our search” (117).
The ‘glowing map’ represents collections of experiments, each designed to test a hypothesized causal linkage between phenomena. These “NEXes” (“Network of EXperiments”) are inspired by the causal graphs of Pearl and Glymour (121) but with an important twist. Each node represents something that molecular neuroscientists might care about: receptors, proteins, low-level processes, or high-level cognitive functions (132). Edges represent the results of experiments designed to test causal relationships between nodes (127). While causal information might be extracted from NEXes (129), the edges do not themselves represent causal relations—most obviously because edges can also carry information about experiments that failed to find a causal relationship between nodes. (This is ambiguous in the text. Sometimes the authors represent observed absence of effect by a broken arrow [172]; other times, by no arrow at all [125, 130]. Only the former convention would distinguish between experiments that found no effect and the mere absence of any relevant experiment.)
As there are multiple different types of experiments, we can test the richness and robustness of connections between nodes by examining the number and diversity of the experiments that connect them (132). We could also look for gaps in the experimental record, places where there are no or ambiguous relationships between phenomena, and use this to “estimate the impact of specific experiments on our uncertainty” (132).
The utility of a NEX obviously depends on the information carried by the edges. Although the authors’ taxonomies distinguish a variety of experiments, only three receive detailed discussion. These are positive manipulations, which increase the probability of one phenomenon and measure the effect on another; negative manipulations, which handle decreases; and non-intervention experiments, which track changes in the absence of direct intervention (16–17). So-called Convergent 3 analyses test to see whether the results of each of these three kinds of experiments converge with one another (20). If they do, then we can trust the putative causal linkage (60). If they do not converge, or if legs of the tripod are missing, then we have an indication of where more experiments ought to be done.
Such is the vision. As a compact representation of what is on PubMed (say), NEXes would offer obvious advantages. But for NEXes to provide “the tools and clarity needed for engineering the next revolution in neuroscience” (13), they would need to do more than graphically index experiments. If scientists still had to read everything and form their own opinion of the experimental landscape, having NEXes would not leave them much better off than before. So a NEX would need to provide—as the authors hope—some sense of which experiments are reliable guides to the actual causal connections. But I doubt that NEXes can play this role, at least in the form envisioned by the authors.
For starters, consider the information coded in the edges of a NEX. The diversity of the experimental record (or the lack thereof) is only one indicator of reliability. For science is not just about filling in gaps in knowledge—it is also about testing putative causal relationships revealed by experiments and seeing whether they actually hold up to scrutiny. A few examples from the authors themselves illustrate the point.
Evidence for the role of α-CaMKII in LTP comes in part from impaired spatial memory in knockout mice. The mechanism behind this linkage remained unclear for a time, and there was a worry that knockout mice were simply developmentally disadvantaged—α-CaMKII is a common synaptic protein and also expressed outside of the hippocampus (81). These worries were ultimately put to rest by the use of pharmacologically inducible mutations in adult mice, bypassing worries about development (84). Or take the case of AP5, which blocks the action of NDMA receptors. Initial tests showed that AP5 injections into the hippocampus impaired rats in the Morris water maze. That suggested a role for hippocampal NDMA receptors in LTP (37). Yet drugged rats turned out to have difficulty with the water maze even when the normally hidden platform was visible. That suggested that AP5’s effect was due to its broad sensorimotor effects, not to any specific effect on memory (Donald P. Cain, Deborah Saucier, Jeff Hall, Eric L. Hargreaves, and Francis Boon, “Detailed Behavioral Analysis of Water Maze Acquisition under APV or CNQX: Contribution of Sensorimotor Disturbances to Drug-Induced Acquisition Deficits,” Behavioral Neuroscience 110 [1996]: 86–102).
In both cases, there was a manipulation experiment that suggested a causal connection. In both, there were doubts about whether this causal connection was real. Those doubts came from a familiar source: they were worries about the possibility of confounds in complex, redundant biochemical processes (Richard G. M. Morris and Mary B. Kennedy, “The Pierian Spring,” Current Biology 2 [1992]: 511–14, 513). The possibility of confounds is not ruled out by looking for other legs of the Convergent 3 (despite Silva, Landreth, and Bickle’s claim on 79). It is ruled out by thinking hard about what the confounds might be, and testing them, and being satisfied with the results. The problem was not that the above experiments did not show enough. It is that what these experiments did show could not be taken at face value.
It is hard to see how NEXes would help here. Indeed, given the complexity of biochemical systems and the combinatorial explosion of variables that could, logically speaking, act as confounds, all a NEX would show is that there are many, many experiments left to do. We already knew that. Part of the problem is a relative lack of engagement with contemporary philosophy of science. One of the hot topics in philosophy of neuroscience over the past decade has been intervention-based explorations of mechanisms. The authors do not use the resources from that literature. Interlevel and intralevel causal relations are mixed without comment (e.g., 132). Topics like the robustness and specificity of causal relationships, or the difference between background conditions and causes proper, are treated superficially if at all (see, e.g., 55, 103).
Perhaps the biggest inadvertent problem for the authors, though, is the historical narrative that runs throughout. Readers interested in LTP, or in experimental method in neuroscience more generally, will find a lot to interest them in these historical interludes. The story that emerges is rich and nuanced—so much so, you might have noticed, that it cuts against the overt proposals that it is meant to support.
Experts often underestimate how difficult it is to do what they do. I have no doubt that, for a skilled scientist like Silva, something like a NEX might be a handy research tool. I also suspect that the next revolution in neuroscience will require something more. Silva, Landreth, and Bickle’s proposal is nevertheless a useful contribution in building the tools of the future.
