2.1. Background philosophical methodology: Robustness analysis

In this discussion, I focus on robustness analysis (RA): The identification of conclusions or results that converge over a variety of independent models, measurements, or derivations (Wimsatt Reference Wimsatt2007, 43). Robustness analysis has been detailed extensively in scientific modeling, experimentation, and evidence (Levins Reference Levins and Sober1966; Glymour Reference Glymour1980; Horwich Reference Horwich1982; Hacking Reference Hacking1983; Franklin Reference Franklin1986; Sober Reference Sober1989; Cartwright Reference Cartwright1991; Culp Reference Culp, Hull, Forbes and Burian1994; Trout Reference Trout1998; Staley Reference Staley2004; Woodward Reference Woodward2006; Weisberg Reference Weisberg2006; Wimsatt Reference Wimsatt2007; and Stegenga Reference Stegenga2009). While there are a number of important considerations at the intersections of epistemology and methodology in RA (Woodward Reference Woodward2006; Weisberg Reference Weisberg2006; Wimsatt Reference Wimsatt2007; Stegenga Reference Stegenga2009; Schupbach Reference Schupbach2018; Basso Reference Basso2017), my focus is on identifying key factors and relationships in measurement results, procedures, conditions, and assumptions. As such, my focus will be on MR, which consists of using different measurement procedures to measure the value of some quantity (Woodward Reference Woodward2006; Basso Reference Basso2017; Karaca Reference Karaca2022). In this section, my aim is to selectively attend to key details of MR in order to analyze the case study in the next section.

According to Woodward (Reference Woodward2006) the success of measurement robustness depends on the use of “independent” procedures, operating according to different design, principles, and assumptions:

In this discussion, I pay particular attention to the roles of procedures and assumptions in measurement robustness. In MR, different procedures are used for the purpose of increasing our confidence that we have measured the quantity “accurately” (ibid.). But the relationship between procedures and results is quite a bit more complicated.

Basso (Reference Basso2017) details the methodology behind MR, where results converge under varying procedures, populations, and/or background conditions. According to Basso (Reference Basso2017), one might assume that the success of MR lies in the following inferential pattern: Independent measurement procedures corroborate individual results (ibid., 64). Procedures play a necessary function in MR (ibid.). The convergence between “expected” and “actual” results indicates the reliability of procedures (ibid., 60). For Basso, expected results require the “idealized definition of the quantity under measurement” (ibid.). Actual results are individual measurement outcomes from measurement procedures (ibid.). According to Basso: “By comparing different measurement procedures, scientists evaluate how well the outcomes of an empirical procedure approximate the idealized definition, and hence how well a procedure measures what it is meant to measure” (ibid.). So, MR maintains a methodological function of investigating procedure reliability (ibid.). Basso frames this nicely in terms of a symmetry between procedures and results: “according to which the convergence of independent procedures confirms the correctness of the robust result delivered by these procedures, if and only if it confirms the reliability of the procedures for delivering the robust result” (ibid., 59). To summarize, the inferential pattern of MR requires an analysis of fit between expected and actual results, thus providing an assessment of measurement procedures.

MR indicates that both results and procedures are robust and that measurement procedures adequately measure what is intended to be measured (Basso Reference Basso2017; Karaca Reference Karaca2022). But what can we conclude about robust results in the context of partial procedure robustness? Moreover, what can we conclude about measurement robustness in the context of divergent measurement results and standards? The case study that follows will test our ability to apply MR at the result and procedure levels—compelling us to go upstream in the measurement process to discover other factors relevant to measurement robustness.

2.2. Robustness analysis applied: From shallow failure to deeper problem

A strain of bacteria from the Halomonadaceae family of Gammaproteobacteria was found in the arsenic-rich waters of Mono Lake (Wolfe-Simon et al. Reference Wolfe-Simon, Switzer Blum, Kulp, Gordon, Hoeft, Pett-Ridge, Stolz, Webb, Weber, Davies, Anbar and Oremland2011). Designated “GFAJ-1,” the strain was carefully preserved in an artificial medium and moved to a laboratory—a noteworthy detail that foreshadows the uncertainties that will soon unfold in this long-disputed case study.Footnote ¹

Initially, the laboratory measurements were considered groundbreaking (NASA 2010). Specifically, the measurements put into question the necessity of phosphorous for cellular function in living organisms. Briefly, carbon, hydrogen, nitrogen, oxygen, sulfur, and phosphorus are used by living organisms as nutrients; however, arsenic is not (Wolfe-Simon et al. Reference Wolfe-Simon, Switzer Blum, Kulp, Gordon, Hoeft, Pett-Ridge, Stolz, Webb, Weber, Davies, Anbar and Oremland2011, 1163). Additionally, there are no known cases of organisms incorporating arsenic into the DNA backbone (ibid.). While arsenic-tolerance has been extensively documented (Benner et al. Reference Benner, Bains and Seager2013), lab tests reveal that GFAJ-1 both uses arsenate as a nutrient and, more importantly, incorporates arsenate into its DNA backbone. According to Wolfe-Simon the discovery is paradigm-shifting for the life sciences, shedding light on a unique evolutionary pathway (Wolfe-Simon et al. Reference Wolfe-Simon, Switzer Blum, Kulp, Gordon, Hoeft, Pett-Ridge, Stolz, Webb, Weber, Davies, Anbar and Oremland2011).

If the Wolfe-Simon et al. (Reference Wolfe-Simon, Switzer Blum, Kulp, Gordon, Hoeft, Pett-Ridge, Stolz, Webb, Weber, Davies, Anbar and Oremland2011) procedures are valid, then this case study can serve as encouragement to expand measurement into new territories and question foundational claims and measurement practices (Marcheselli Reference Marcheselli2020). The methodological details are paramount for assessing the procedures. First, when isolated in an arsenate-rich and phosphate-absent environment, GFAJ-1 grows at 60 percent of its growth rate (Wolfe-Simon et al. Reference Wolfe-Simon, Switzer Blum, Kulp, Gordon, Hoeft, Pett-Ridge, Stolz, Webb, Weber, Davies, Anbar and Oremland2011, 1164). Relevant to robustness analysis, two independent measurement processes—scanning electron microscope and transmission electron microscope—are used to track the growth (ibid.). Second, the team uses multiple mass-spectrometry techniques to identify “incorporated [arsenic] into the DNA backbone of GFAJ-1 in place of phosphorus, with an estimated 4% replacement of P by As” (ibid., 1164–65). Notice that multiple measurement procedures are used to identify both the consumption effect and DNA bonding effect. The details behind the measurement outputs are important: Nucleic acids extracted from arsenate-rich and phosphate-absent samples contain increased arsenate and decreased phosphate compared to extracted nucleic acid from the arsenate-absent and phosphate-rich samples (ibid., 1165). The interpretation is that this shows that arsenic is incorporated into the DNA backbone in place of phosphorus (with an estimated 4 percent replacement) (ibid., 1164–65).

Reaves et al. (Reference Reaves, Sinha, Rabinowitz, Kruglyak and Redfield2012) question both the use of arsenate as nutrient and incorporation of arsenic into the DNA backbone. Moreover, to perform an adequate replication, they use samples from Wolfe-Simon et al. (Reference Wolfe-Simon, Switzer Blum, Kulp, Gordon, Hoeft, Pett-Ridge, Stolz, Webb, Weber, Davies, Anbar and Oremland2011). While phosphorous is treated as being absent because it is “insufficient to elicit growth in the control” (ibid., 1164), Reaves et al. (Reference Reaves, Sinha, Rabinowitz, Kruglyak and Redfield2012) show that the samples contain a basal level of phosphorous (3–4 μM), which has been documented to support moderate growth (p. 4). Moreover, they note that the arsenate dependence observed by Wolfe-Simon et al. (Reference Wolfe-Simon, Switzer Blum, Kulp, Gordon, Hoeft, Pett-Ridge, Stolz, Webb, Weber, Davies, Anbar and Oremland2011) may reflect the presence of a contaminant in their sample that is creating a metabolic effect—especially given the fact that the purity and supplier of the sample are unknown (Reaves et al. Reference Reaves, Sinha, Rabinowitz, Kruglyak and Redfield2012, 2). Next, Reaves et al. (Reference Reaves, Sinha, Rabinowitz, Kruglyak and Redfield2012) report that there is no covalently bound arsenate in the DNA backbone. They take extra preparatory procedure steps to purify the DNA—ensuring that no arsenate merely “stuck” to the DNA. The preparatory procedure details consist of a series of steps. They use cesium chloride density-gradient centrifugation, separating DNA from impurities based on density (ibid., 2). Because cesium chloride density-gradient centrifugation can create salt by-products, they also remove excess salts (ibid., 3). Next, they separate DNA into nucleotide blocks and use liquid chromatography-mass spectrometry (LC/MS) to analyze the blocks (ibid.). This is important because it uses polarity to physically separate the nucleotides and affords an analysis of their mass. Contrary to Wolfe-Simon et al. (Reference Wolfe-Simon, Switzer Blum, Kulp, Gordon, Hoeft, Pett-Ridge, Stolz, Webb, Weber, Davies, Anbar and Oremland2011), Reaves et al. (Reference Reaves, Sinha, Rabinowitz, Kruglyak and Redfield2012) find that when phosphate is limiting, arsenate does not contribute to GFAJ-1 growth; and they find that purified DNA from arsenate-abundant and phosphate-limited environments does not show detectible amounts of covalently bound arsenate—only free arsenate (2–3).

The simple resolution is that the original arsenic consumption and bonding effects are not genuine phenomena, but rather they result from inadequate preparatory procedures. In other words, once the procedures are reconfigured to measure what is intended to be measured, the results are corrected. In methodological terms, MR indicates that the “relevant phenomenon has been genuinely and correctly detected” (Basso Reference Basso2017, 59). MR can indicate that because independent procedures are used—with instruments operating according to different causal principles and data interpretation guided by different assumptions (Woodward Reference Woodward2006, 234)—that a given robust measurement remains relatively independent of the errors and idealizing assumptions behind each measurement procedure, and that the robust measurement is thus not an artifact of some form of measurement dependence between procedures (Woodward Reference Woodward2006; Wimsatt Reference Wimsatt2007). In contrast, results that are not robust are considered “disputable” (Basso Reference Basso2017, 59), and they can indicate that a given measurement outcome is compromised by the errors and idealizing assumptions behind a given measurement procedure (Basso Reference Basso2017; Woodward Reference Woodward2006; Wimsatt Reference Wimsatt2007). So, because of unsuccessful MR, both the arsenic consumption and bonding effects are better characterized as “artifacts” or results dependent on the particular measurement conditions of each preparatory procedure.Footnote ² For example, the bonding artifact results from a lack of adequate DNA purification. Moreover, they are artifacts of the preparatory procedures—with other measurement procedures (e.g., multiple mass-spectrometry techniques) performing their intended functions.

However, this is but a shallow resolution to the methodological story of GFAJ-1. After the initial NASA press conference fireworks (NASA 2010), Wolfe-Simon became locked in a back-and-forth about replication, which was eventually shelved by the scientific community until the debate reignited fifteen years later with news about the retraction (Offord Reference Offord2025). But perhaps there is something more that we can learn from this case study—especially about its informativeness for seeking new paradigms in scientific methodology (Marcheselli Reference Marcheselli2020). Is the resolution of this case study just about substituting an adequate procedure for an inadequate one, or is there anything methodologically deeper? In an often-overlooked response to the Reaves et al. (Reference Reaves, Sinha, Rabinowitz, Kruglyak and Redfield2012) research, as well as the demand for replication, Wolf-Simon brings up a fascinating methodological point:

Simply put, the method may compromise the phenomenon. We come back to the aforementioned distinction between actual measurement result and expected result. In the International Vocabulary of Metrology, there is a qualification that: “The measurement, including the measuring system and the conditions under which the measurement is carried out, might change the phenomenon, body, or substance such that the quantity being measured may differ from the measurand as defined. In this case, adequate correction is necessary” (JCGM 2012, 2.3 Note 3; my emphasis). Wolfe-Simon is drawing our attention to the fact that there may be disagreement about what counts as the “adequate correction.” For Reaves et al. (Reference Reaves, Sinha, Rabinowitz, Kruglyak and Redfield2012), the initial standards used by Wolfe-Simon et al. (Reference Wolfe-Simon, Switzer Blum, Kulp, Gordon, Hoeft, Pett-Ridge, Stolz, Webb, Weber, Davies, Anbar and Oremland2011) result in mischaracterizations of the arsenic consumption and bonding effects under study. But for Wolfe-Simon (Drahl Reference Drahl2012), the Reaves et al. (Reference Reaves, Sinha, Rabinowitz, Kruglyak and Redfield2012) standards risk changing the phenomena under study.

We can reframe this as a problem of methodological standards: By using lax methodological standards, the risk is to mischaracterize an artifact as a genuine phenomenon; but by using stringent standards, the risk is to denature the genuine phenomenon. Notice that this problem requires an assessment of how we represent a given phenomenon as well as the extent to which we can manipulate that phenomenon without destroying it. In other words, there is an interesting clash between representation and intervention, which we come back to in section 3.2. Related to this, notice that this problem of methodological standards extends beyond measurement procedures. In other words, it requires the recruitment of other key factors in measurement methodology. Otherwise, we risk an impasse at the measurement procedure level, resulting in uncertainty about whether the preparatory procedures really perform their intended function. As I discuss in the next section, to find clear solutions in murky waters, we must go all the way upstream to track down other key factors in the measurement process.

3.1. Measurement robustness between disciplines

Resolving the aforementioned problem of methodological standards requires qualification: Whose standards? This case study comes from a complicated field, astrobiology. Benner et al. (Reference Benner, Bains and Seager2013) argue that because of the cross-disciplinary nature of astrobiology, experiments can produce evidence that is “sufficient proof for one scientific discipline but sufficient only to create further controversy for another”—meaning that, at their extremes, standards from each field can lead to either flawed science or the rejection of genuine innovation (Benner et al. Reference Benner, Bains and Seager2013, 1). For instance, for the original Wolfe-Simon et al. (Reference Wolfe-Simon, Switzer Blum, Kulp, Gordon, Hoeft, Pett-Ridge, Stolz, Webb, Weber, Davies, Anbar and Oremland2011) paper, reviewers from geology and physics found that the evidence strongly supports the arsenate-bonding effect (Kaku Reference Kaku2010). While Benner et al. (Reference Benner, Bains and Seager2013) focus on standards in the sense of “expectations of [a given] community,” standards can also be framed in terms of discipline-specific background knowledge that streamlines discipline-specific assumptions. As I detail in this section, those assumptions can then guide methodological choices. That is, the live science provides a fascinating illustration of how those assumptions moderate research questions and methodological choices.

How can reviewers from geology and physics find that the evidence strongly supports the arsenate-bonding effect (Kaku Reference Kaku2010), while chemists see the very same data as “disproof” of the effect (Drahl 2010; Benner et al. Reference Benner, Bains and Seager2013)? Details from each field are important. For geologists, arsenate in many arsenate minerals is often substituted by phosphate. So, the arsenate-bonding effect is the simplest explanation for the data reported (Benner et al. Reference Benner, Bains and Seager2013, 2). In contrast, as documented in chemistry databases of molecular structures, all known arsenate esters hydrolyze rapidly in water at any temperature where water is liquid (ibid., 1). This leads chemists to expect that all arsenate esters will hydrolyze rapidly in water, even those not yet examined (ibid.). Putting all this together, there is divergence between the Wolfe-Simon et al. (Reference Wolfe-Simon, Switzer Blum, Kulp, Gordon, Hoeft, Pett-Ridge, Stolz, Webb, Weber, Davies, Anbar and Oremland2011) results and foundational knowledge in chemistry. Data collected from GFAJ-1 is interpreted as evidence that its DNA contains arsenate esters (ibid.). The processed DNA is separated from other components by procedures performed in water and resolved by electrophoresis in a gel containing water (Benner et al. Reference Benner, Bains and Seager2013, 2). All known arsenate esters hydrolyze rapidly in water (ibid., 1). Yet, somehow, the arsenate-bound DNA does not hydrolyze (ibid., 2). In other words, how could DNA with arsenic replacing phosphorus in its backbone possibly be stable, given foundational knowledge about the instability of arsenate esters? It is true that the specific molecule proposed by Wolfe-Simon et al. (Reference Wolfe-Simon, Switzer Blum, Kulp, Gordon, Hoeft, Pett-Ridge, Stolz, Webb, Weber, Davies, Anbar and Oremland2011) is not in the database of known arsenate esters—meaning that its instability in water has never been measured. Marcheselli (Reference Marcheselli2020) brings up an interesting point about GFAJ-1 and other anomalies. To adequately measure and model anomalies, our instruments and assumptions have to change dramatically (Marcheselli Reference Marcheselli2020).

Before we jump paradigm, it’s important to bring in biological foundations. GFAJ-1 is reported to be closely related to bacteria that do not have arsenate-bound DNA (Wolfe-Simon et al. Reference Wolfe-Simon, Switzer Blum, Kulp, Gordon, Hoeft, Pett-Ridge, Stolz, Webb, Weber, Davies, Anbar and Oremland2011), which are well-documented in biology databases (Benner et al. Reference Benner, Bains and Seager2013, 2)—calling into question the arsenate-bonding effect. Moreover, GFAJ-1 cells do not appear to be markedly different from those of other arsenic-tolerant bacteria (ibid.)—calling into question the arsenate-consumption effect.

Amalgamating foundational knowledge from chemistry and biology gives us reason to doubt that GFAJ-1 is a paradigm-shifting organism. To summarize, robustness analysis within and between each discipline, coupled with the Reaves et al. (Reference Reaves, Sinha, Rabinowitz, Kruglyak and Redfield2012) results, call into question the arsenate-bonding and arsenate-consumption effects. This is fascinating because it shows a kind of meta robustness between disciplines, where divergence leads to a correction of assumptions and, as we will see shortly, a reconfiguration of measurement methodology. Chemistry is used to question the DNA bonding effect, while biology is used to limit the reference class to arsenic-tolerant strains.

The story gets even worse for GFAJ-1. Combine the aforementioned conclusions with follow-up robust discoveries: GFAJ-1 needs a low level of phosphate (1.6 µM) to grow at all (Erb et al. Reference Erb, Kiefer, Hattendorf, Günther and Vorholt2012; Reaves et al. Reference Reaves, Sinha, Rabinowitz, Kruglyak and Redfield2012); Both enzyme degradation methods and newly applied plasma methods show no arsenate in the DNA backbone (Erb et al. Reference Erb, Kiefer, Hattendorf, Günther and Vorholt2012; Reaves et al. Reference Reaves, Sinha, Rabinowitz, Kruglyak and Redfield2012; Drahl Reference Drahl2012). Importantly, plasma methods appease Wolfe-Simon’s original concern about denaturing the genuine phenomenon (Drahl Reference Drahl2012). The punchline is that intradisciplinary and interdisciplinary robustness analyses reveal the original arsenate-consumption and arsenate-bonding effects as artifacts, dependent on the particular measurement conditions in the preparatory procedures used by Wolfe-Simon et al. (Reference Wolfe-Simon, Switzer Blum, Kulp, Gordon, Hoeft, Pett-Ridge, Stolz, Webb, Weber, Davies, Anbar and Oremland2011).

But this is where the methodology becomes even more complicated and interesting. Even though Benner et al. (Reference Benner, Bains and Seager2013) conclude that it is an “unexceptional” organism (p. 4), it is not the end of the story for GFAJ-1. As we will see in the next section, the actual punchline is that intradisciplinary and interdisciplinary robustness analyses lead to a modification of the research question, which subsequently leads to a revolutionary discovery about strains of bacteria like GFAJ-1. A new set of research questions emerges: How can GFAJ-1 be arsenate-tolerant even though it is phosphate-dependent? Does arsenate play an important causal role in GFAJ’s cellular growth? Additionally, a new methodological direction emerges, consisting of intervention-based methods to test phosphate-dependence (Basturea et al. Reference Basturea, Harris and Deutscher2012).

3.2. Manufactured monstrosity: Lab artifact as genuine phenomenon

While Reaves et al. (Reference Reaves, Sinha, Rabinowitz, Kruglyak and Redfield2012) show that arsenate is not incorporated into the DNA backbone, it is not clear why GFAJ-1 is so successful in arsenic-rich environments. Additionally, many research teams continue to attempt making sense of the original GFAJ-1 findings (Benner et al. Reference Benner, Bains and Seager2013, 4). So, the research inquiry can be reframed to be about how arsenate can play an important causal role in GFAJ’s cellular growth—even if it’s not incorporated into the DNA. Addressing this new direction for research requires a complex intervention before a measurement procedure can even be applied (Keyser Reference Keyser2017). Recall Wolf-Simon’s concern (Drahl Reference Drahl2012) with the methods used by Reaves et al. (Reference Reaves, Sinha, Rabinowitz, Kruglyak and Redfield2012)—that the manipulation involved in the measurement procedures will destroy the genuine phenomenon. I propose that, contrary to Wolf-Simon’s concern, manipulation is the key to salvaging the scientific importance of GFAJ-1. A lab manufactured effect is required to reveal something groundbreaking about GFAJ-1 and other arsenic-tolerant bacteria.

Philosophers have addressed the creation of instructive effects (Hacking Reference Hacking1983; Radder 2003; Heidelberger Reference Heidelberger and Radder2003; van Fraassen Reference van Fraassen2008; Carusi Reference Carusi2016a, Reference Carusi, Whitehead and Woods2016b; Keyser Reference Keyser2017). For instance, there are “mimetic effects,” which consist of carefully controlled experimental conditions, used by scientists in the context of theory, to resemble natural phenomena (van Fraassen Reference van Fraassen2008, 94–96). There are even “engines of creation,” where scientists produce or manufacture new phenomena (ibid.). A given instructive effect occurs as an important part of the measurement process (Keyser Reference Keyser2017). However, instructive effects rarely make their way into discussions about robustness analysis. Recall that in the International Vocabulary of Metrology, measurement can “change the phenomenon, body, or substance,” which can require adequate correction (JCGM 2012, 2.3, Note 3). In this case study, the change to the phenomenon serves an instructive methodological function, which I illustrate next.

To study arsenic-induced cellular growth in GFAJ-1, a strain of E. coli is manufactured and used in place of GFAJ-1.Footnote ³ It is a laboratory-produced organism (MG1655 (seq)* I⁻) (Basturea et al. Reference Basturea, Harris and Deutscher2012). Once the organism is manufactured, the measurement conditions and results unfold. In a growth medium without both arsenate and phosphate, MG1655 (seq)* I⁻ does not grow (ibid., 28816). However, upon the addition of 40 μM of arsenate into the growth medium (and absence of phosphate), there is a lag of about 80 h, then positive growth—characterized by a twenty-fold increase in cell number over six days (ibid.). It is quite odd that only with the addition of 40 μM arsenate does growth occur. Recall that a phosphate “absent” medium still has negligible amounts of phosphate, expected to serve as a growth factor (Reaves et al. Reference Reaves, Sinha, Rabinowitz, Kruglyak and Redfield2012). However, Basturea et al. (Reference Basturea, Harris and Deutscher2012) point out that without the addition of arsenate, no growth occurs—the implication being that the small levels of phosphate do not stimulate growth. In the language of procedure robustness (Basso Reference Basso2017), the actual result diverges from the expected result, extrapolated from Reaves et al. (Reference Reaves, Sinha, Rabinowitz, Kruglyak and Redfield2012). Additionally, when MG1655 (seq)* I⁻ cells are placed in various media—with composition described by Wolfe-Simon et al. (Reference Wolfe-Simon, Switzer Blum, Kulp, Gordon, Hoeft, Pett-Ridge, Stolz, Webb, Weber, Davies, Anbar and Oremland2011) (Basturea et al. Reference Basturea, Harris and Deutscher2012, 28817)—the results seem to converge with results from Wolfe-Simon et al. (Reference Wolfe-Simon, Switzer Blum, Kulp, Gordon, Hoeft, Pett-Ridge, Stolz, Webb, Weber, Davies, Anbar and Oremland2011) (Keyser Reference Keyser2017). Specifically, in both studies there is the disappearance of 16 S and 23 S rRNA bands in the presence of arsenate and absence of phosphate (Basturea et al. Reference Basturea, Harris and Deutscher2012, 28817). This robust result may be one of the most profound thus far—smoothing out the discrepancies between Wolfe-Simon et al. (Reference Wolfe-Simon, Switzer Blum, Kulp, Gordon, Hoeft, Pett-Ridge, Stolz, Webb, Weber, Davies, Anbar and Oremland2011) and later studies. According to Basturea et al. (Reference Basturea, Harris and Deutscher2012), the disappearance of 16 S and 23 S rRNA bands is a reliable indicator of ribosomal degradation (pp. 28817–18). Mysteriously, in the arsenate present and phosphate absent medium, cells initially die—with a subpopulation surviving and, subsequently, growing.

The missing explanatory link emerges because the research question is now a question about the mechanism behind arsenic tolerance. According to Basturea et al. (Reference Basturea, Harris and Deutscher2012), arsenate induces ribosome degradation by releasing free bases from RNA, which provides a phosphate source for the remaining cells (ibid., 28818). The resolution to this case study is unexpected. By manipulating a strain of bacteria to create a mimetic effect, we find that the cells do thrive in arsenic-rich environments (Keyser Reference Keyser2017), returning to the excitement of the original GFAJ-1 findings. But the new explanation embeds part of the results from both Wolfe-Simon et al. (Reference Wolfe-Simon, Switzer Blum, Kulp, Gordon, Hoeft, Pett-Ridge, Stolz, Webb, Weber, Davies, Anbar and Oremland2011) and Reaves et al. (Reference Reaves, Sinha, Rabinowitz, Kruglyak and Redfield2012), while correcting the mechanism (Keyser Reference Keyser2017). Combining the production-based results (Basturea et al. Reference Basturea, Harris and Deutscher2012) with other results from Erb et al. (Reference Erb, Kiefer, Hattendorf, Günther and Vorholt2012) and Reaves et al. (Reference Reaves, Sinha, Rabinowitz, Kruglyak and Redfield2012) (where no new organisms are manufactured), we can infer that GFAJ-1 is arsenate-tolerant but phosphate-dependent.

Our philosophical analysis has extended vastly beyond the substitution of one measurement procedure for another. To take seriously if measurement changes the genuine phenomenon into an artifact (i.e., the problem of methodological standards presented in section 2), divergence prompts the re-examination of assumptions and research questions, a new production-based methodological avenue is explored, and a mimetic effect is manufactured to demonstrate a genuine phenomenon. As such, there is a new type of measurement robustness analysis on the horizon—intersecting discovered phenomena and manufactured effects. This exciting philosophical territory lies at the intersection of representation and intervention and requires treatment in a new discussion.