3.1 “Rival Spaces” – The Logic of the Typology

The exercise of “casting the net widely for alternative explanations” is analogous to a foundational concept in probability and set theory: building a sample space.Footnote ⁶ The difference is trivial: in probability theory, sample spaces are populated by a set of potential outcomes; in process tracing, sample spaces—or what we might call “rival spaces”—would be populated by a set of competing hypotheses explaining a phenomenon.

Examining the properties of the sample space reveals what process-tracing research is missing: in many cases, statisticians cannot accurately compute the likelihood of one event relative to another without specifying the relationship among events, which can take numerous forms (e.g. exclusivity, independence, and dependence).Footnote ⁷ If a statistician does not know the relationships between events (or, alternately, assumes that one relationship is true all of the time), any attempt to calculate probability is liable to generate inaccurate estimates. Recasting competing hypotheses in these terms makes clear that without proper specification of relationships, process-tracing conclusions are susceptible to the same errors. The following sections draw on this insight to elaborate the variety of relationships and derive the corresponding implications for inference.

3.2 Mutually Exclusive Hypotheses

The first relationship researchers may encounter is mutual exclusivity, in which two explanations cannot simultaneously be valid in a single case. Mutual exclusivity comes in three flavors. Outcome mutual exclusivity occurs when two theories make consistently divergent predictions on the outcome, whereas evidentiary mutual exclusivity occurs when one theory requires evidence that would undermine the other (i.e. two theories operate by mutually exclusive paths). Finally, two theories may exhibit conditional mutual exclusivity, in which a scope condition or an idiosyncrasy of the case forces two explanations to be mutually exclusive in some instances but not in others.Footnote ⁸ This relationship is the only one in which evidence in favor of one hypothesis logically subverts an alternative hypotheses.

Mutual exclusivity is a strong modeling assumption for which the burden of proof is on researchers. In social science phenomena, mutual exclusivity is rare and difficult to establish. It is not the default state of the world, notwithstanding the wealth of substantive literature operating on this assumption. In the literature on rebellion, for instance, greed and grievance are often posited as “rival” explanations of insurgent participation (Collier and Hoeffler Reference Collier and Hoeffler2004), as though someone cannot be both sad and acquisitive.

The most straightforward case of mutual exclusivity is the null hypothesis. After all, if one’s theory is that $X$ affects $Y$ , it would be impossible for $X$ to simultaneously not affect $Y$ . Brady’s (Reference Brady, Brady and Collier2010) examination of Lott’s (2000) argument regarding the effect of the “early call” of the 2000 U.S. presidential race is a clear case of using process tracing to evaluate one hypothesis against the (mutually exclusive) null. Lott argues that when the Florida media called the race in favor of Al Gore after the polls closed in the main peninsula, potential Republican voters in the Florida panhandle (where polls were still open) were dissuaded from voting. In response to Lott’s claims that the media suppressed approximately 100,000 votes, Brady systematically performs a series of hoop tests, providing evidence against each one in turn. In effect, Brady’s analysis lends support to the null.

Assuming mutual exclusivity when it does not hold can prove detrimental for both present and future research. First, treating evidence in favor of $H_{M}$ as though it necessarily counts against $H_{R}$ can lead researchers to hastily rule out explanations that may work in conjunction with the main hypothesis. Scholars drawing on this flawed research are then susceptible to discounting the theoretical value of an explanation that was eliminated on the basis of an assumption rather than empirical evidence.Footnote ⁹

3.3 Coincident Hypotheses

The second possible relationship between hypotheses is coincidence. Akin to statistical independence, explanations are coincident when the validity of one has no impact on that of the other. In substantive terms, coincident theories operate via sufficiently different mechanisms that both explanations could independently contribute to the outcome. As such, evidence in favor of one has no effect on the other. Consequently, the process of eliminating a coincident rival hypothesis involves collecting additional evidence that specifically invalidates the causal process of that explanation.

A prime example of coincident explanations comes from the democratic peace literature. In seeking an explanation for the phenomenon that democracies exhibit some immunity from fighting one another, scholars have proposed two models (Maoz and Russett Reference Maoz and Russett1993). The normative model posits that peaceful methods for intrastate conflict resolution are externalized reliably enough to shape interstate conflict resolution. The structural approach contends that democratic institutions constrain leaders in a way that makes waging war difficult. Though both explanations arrive at the same outcome, they posit different causal stories. Indeed, as Maoz and Russett explicitly note “these two explanations are not mutually exclusive” (Reference Maoz and Russett1993, 626). Thus, a piece of evidence corroborating a normative process in democratic decision-making does not preclude structural influences from playing a comparably strong role when explaining why two democracies did not go to war.

3.4 Congruent and Inclusive Hypotheses

The third possible relationship among “rival” explanations is congruence. Under congruence, hypotheses make similar predictions or operate via sufficiently similar mechanisms that evidence in favor of one also corroborates the other. In such cases, we would expect to find similar evidence under each theory. This is not to say that the two theories are indistinguishable—they may be congruent on some pieces of evidence, and just coincident on others. Alternately, the two theories may describe different levels of analysis, or one theory might describe an outcome that is sequentially prior to another. In cases of congruence, researchers must be especially careful to specify which pieces of evidence simultaneously support both theories, and what would be necessary to eliminate the rival from consideration.

Finally, inclusive hypotheses represent a special case of congruence in which one hypothesis represents a novel extension of an existing theory. Scholarship typically proceeds by first demonstrating the inadequacy of existing theories, and then proposing a novel explanation to fill the gaps. Sometimes, however, what begins as a separate theory may instead constitute a new dimension of an existing one—even when that framework is initially presented as a “competing” explanation. Either an assumption of mutual exclusivity or just a habit of dismissing alternative explanations can lead researchers to overlook cases in which new theories represent useful extensions of existing frameworks. The RAR approach provides guidance for finding evidence that suggests theoretical integration.

Researchers are potentially dealing with inclusive hypotheses when three conditions are met. First, the hypotheses must make similar or otherwise complementary predictions. In other words, the same input cannot generate contradictory output. Second, the hypotheses cannot operate via contradictory assumptions. If assumptions across two theories are dissimilar, scholars should ask if they can be reconciled; perhaps the original set of assumptions is less empirically valid than the new ones, so the latter should supplant the former. Third, hypotheses are potentially inclusive if the evidence presented in favor of one either supports or has no impact on the other and if no invalidating evidence is found for either hypothesis.

3.5 Guidance for Identifying Relationships among Rivals

The typology introduced above raises two important questions. How can scholars establish which relationship is present, and what are the implications for evaluating evidence? Since the relationship between two hypotheses affects both evidence gathering and interpretation, researchers need a clear guide to assess which relationship is present. Figure 1 presents a decision tree to guide researchers through the process of identifying relationships and drawing inferences.

Figure 1.

Identifying relationships among rivals.

The first set of questions addresses broad theoretical considerations by asking whether the competing explanations make divergent predictions on the outcome. If one theory expects outcome $Y$ , and another expects no $Y$ (or high versus low levels of $Y$ ), they may be candidates for mutual exclusivity, but even predictive divergence is not sufficient to ensure it.

Answering “yes” to the first question then requires a scope check—asking whether the theories make different predictions only at distinct levels of analysis or under certain conditions. If divergence does not seem to be a function of other variables or scope conditions, then the two explanations are good candidates for outcome mutual exclusivity. If, however, a scope condition is driving the divergence, the two theories may exhibit conditional mutual exclusivity, in which they are exclusive under some conditions, but not others. Since mutual exclusivity may be contingent on scope, researchers should search for additional variables or conditions that affect a theory’s validity in a given case, and they must be cautious when generalizing about the overall relationship between theories. Finally, if two hypotheses are conditionally mutually exclusive, researchers should note the driving factor and the boundaries of each theory.

The next set of questions concerns the nature of evidence required under each explanation. Researchers should start by asking whether the two theories call for divergent types of evidence. Crucially, this question is not about asking whether researchers would expect to find different pieces of evidence from one theory to another, but rather whether they would expect to find incompatible pieces of evidence.Footnote ¹⁰ For example, if one theory is supported by the presence of $E$ , yet another requires its absence, they would also be good candidates for mutual exclusivity (even if they make the same predictions about the outcome).

If two theories do not have contradictory requirements, researchers should then ask whether evidence in favor of one hypothesis has any effect on the validity of the other. If not, the two theories are likely coincident: they require different pieces of evidence both for corroboration and invalidation. Explanatory factors $A$ and $B$ may together bring about an outcome; consequently, verifying one does not invalidate the other. As such, to rule out a coincident alternative (say, theory $B$ ) researchers must search for additional evidence that specifically addresses $B$ . Alternately, if the evidence in favor of one theory does have an effect on the other theory, researchers must then ask whether the effect is undermining or corroborating. In the former case, the two hypotheses become good candidates for evidentiary mutual exclusivity; in the latter case, the two hypotheses likely exhibit congruence (at least on that piece of evidence).

Finally, if two theories are congruent, researchers should ask whether they are candidates for inclusiveness. Does integrating them into a single theoretical framework provide more analytic leverage than either one separately? If two explanations make predictions about the same outcome and one explanation represents a new dimension or channel through which the prior may operate, the theories are good candidates for inclusiveness.Footnote ¹¹ If their predictions or mechanisms are too different to reconcile into a single framework, then they remain congruent.

The framework introduced in this section provides researchers with a tool for explicitly modeling and testing relationships among competing hypotheses. By disaggregating competing theories into their constituent mechanisms and assumptions, researchers can identify the precise dimensions on which two theories differ, thereby providing a more precise guide to the type of evidence needed to support or undermine a given explanation. While this framework represents a crucial advancement to the method, many previous innovations have given rise to internecine debates over how process tracing is best conducted—some of which go so far as to advocate one approach to the exclusion of others. Thus, contributions notwithstanding, the introduction of the RAR framework raises two key questions: (1) What are the implications for existing process-tracing approaches? and subsequently, (2) how should this framework be incorporated into the research process more broadly? The next two sections address these questions in turn.

4.1 Analytic Narratives

The classical approach to process tracing—constructing analytic narratives on the basis of systematically collected evidence—must be expanded beyond Collier’s (Reference Collier2011) framework to reflect the diverse implications of evidence when hypotheses are nonexclusive. Thus, while the general principles of this approach hold, when adjudicating among competing explanations, researchers should follow the inferential procedures laid out in Figure 1. Other than incorporating a more nuanced conception of rivals into the research procedure, no specific changes to the approach itself are required.

4.2 Crisp-Set-Theoretic Process Tracing

The set-theoretic approach understands process tracing as a search for “necessary, sufficient, and jointly sufficient factors for an outcome” (Blatter and Haverland Reference Blatter and Haverland2012, 24). Consequently, Goertz and Mahoney maintain that inferences must follow the rules of set-theoretic logic (Reference Goertz and Mahoney2012, 13). Integrating RAR into this approach adds inferential nuance in two key areas. First, insights about the rarity of mutual exclusivity suggest that researchers should not be looking for the ideal “doubly decisive” evidence in all cases. Doubly decisive tests entail finding a single piece of evidence that simultaneously confirms $H_{M}$ while undermining $H_{R}$ . For doubly decisive evidence to exist, three rare conditions must align at once: (a) the two theories must be mutually exclusive; (b) researchers must find a piece of evidence sufficiently strong to confirm the main hypothesis; (c) the piece of evidence must be on the same dimension that defines the exclusivity of the two theories.

The second implication for the crisp-set-theoretic approach is that the nonexclusive RAR (coincidence, congruence, and inclusiveness) dovetail with the logic of INUS (jointly sufficient) conditions. Set-theoretic scholars routinely acknowledge that multiple conditions may work jointly to bring about an outcome (Mahoney and Goertz Reference Mahoney and Goertz2006; Mahoney Reference Mahoney2008), yet this insight remains analytically separate from discussions of rival hypotheses.Footnote ¹³ The diverse set of RAR explanations can help scholars in the set-theoretic camp further parse out the relationships among INUS conditions, which are simply nonexclusive rivals. RAR thus provides those using set-theoretic process tracing with a guide to identifying jointly sufficient conditions and a more nuanced way of describing how these conditions work together.

4.3 Bayesian Process Tracing

The Bayesian innovation represents the frontier of process-tracing research. Bayesian process tracing asks a simple analytic question for each piece of evidence: What is the probability that our main hypothesis ( $H_{M}$ ) is correct, given that we searched for and found evidence $K$ : $P(H_{M}|K=1)$ ? Footnote ¹⁴ Formally, this question takes the following form:

(1) $$\begin{eqnarray}P(H_{M}|K=1)=\frac{P(H_{M})P(K=1|H_{M})}{P(H_{M})P(K=1|H_{M})+P(\neg H_{M})P(K=1|\neg H_{M})},\text{ where},\end{eqnarray}$$

• ∙ $P(H_{M})$ represents the estimated probability that our hypothesis is correct before new evidence is found (i.e. the prior),

• ∙ $P(K=1|H_{M})$ represents the probability of observing that piece of evidence when our hypothesis is correct (i.e. the likelihood),

• ∙ $P(\neg H_{M})$ represents the probability that our hypothesis is incorrect (this quantity is computed as $(1-H_{M})$ ),

• ∙ $P(K=1|\neg H_{M})$ represents the probability of finding that piece of evidence under a different hypothesis, and finally

• ∙ $P(H_{M}|K=1)$ represents the updated (i.e. posterior) probability that our hypothesis is correct, given that we found evidence $K$ .

Thus, for each piece of evidence ( $K$ ), scholars must first surmise and justify these probabilities, and then input them into Bayes’ rule to compute the updated confidence in $H_{M}$ in light of the evidence. Footnote ¹⁵

Due in part to its pivotal role in the refinement of qualitative inference and in part to the formalized procedures on which Bayesian process tracing relies, this approach warrants special consideration in light of the RAR framework. Specifically, since Bayes’ rule includes terms that presume a disjoint sample space (i.e. comprising $N$ mutually exclusive possibilities), it requires extensive modification when dealing with nonexclusive theories. Although Bayesian inference is sensitive to misspecification of relationships among competing explanations, the challenge of expanding Bayes’ rule to accommodate nonexclusivity is practical, rather than conceptual.

Expanding Bayes’ Rule to Accommodate NonExclusivity.

This section explores the mathematical implications of computing posterior probabilities in cases of nonexclusivity. To motivate this discussion, consider the two rival spaces depicted in Figure 2. Computing the posterior probability $P(H_{1}|K=1)$ in Figure 2a—where the rival space is clean and disjoint—follows the procedures outlined above. Complications arise, however, when researchers move into the more common, nonexclusive territory illustrated in Figure 2b. If, for example, $H_{1}$ and $H_{2}$ are congruent (i.e. the same evidence is expected under both), researchers must now evaluate the probability that the case is explained by the dark gray area, which represents the intersection $H_{1}\cap H_{2}$ . Using the equation above to compute $P(H_{1}|K=1)$ when $H_{1}$ and $H_{2}$ are congruent would compromise every term in Bayes’ rule. First, the analysis would be performed on the wrong area, thus rendering the prior incorrect. Second, the likelihood of finding $K$ under $H_{1}$ is most likely lower than that of finding $K$ in the intersection ( $H_{1}\cap H_{2}$ ), thus rendering the likelihood incorrect. Third, the probability of finding $K$ under $H_{2}$ , $P(K=1|H_{2})$ , wrongfully acts as a penalty term, thus rendering the denominator incorrect.

Figure 2.

The intuition behind nonexclusivity.

Thus, if a researcher believes a case is best explained by $H_{1}$ and $H_{2}$ together, Bayes’ rule must be modified. For each piece of evidence, she must evaluate $P(H_{1}\cap H_{2}|K=1)$ . Each term in the expanded equation requires numerous additional considerations, which I address in turn. The modified equation takes the following form:

(2) $$\begin{eqnarray}P(H_{1}\cap H_{2}|K=1)=\frac{P(H_{1}\cap H_{2})P(K=1|H_{1}\cap H_{2})}{[P(H_{1}\cap H_{2})P(K=1|H_{1}\cap H_{2})]+[(1-P(H_{1}\cap H_{2}))P(K=1|\overline{H_{1}\cap H_{2}})]}.\end{eqnarray}$$

Rival Space and Priors. First, researchers are tasked with specifying a more complex rival space than the examples in the introductory literature. In addition to specifying priors $P(H_{1})$ and $P(H_{2})$ , the burden is on the researcher to specify the probability of co-occurrence: $P(H_{1}\cap H_{2})$ (which is not necessarily $P(H_{1})\times P(H_{2})$ ).Footnote ¹⁶ Rather, this quantity depends upon the researcher’s beliefs about the frequency with which the two hypotheses work together. $P(H_{1}\cap H_{2})$ could be as small as 0, if the two are mutually exclusive; or it could be as large as the smaller probability if one is completely subsumed under the other. As with any quantity, the point the researcher chooses for $P(H_{1}\cap H_{2})$ must be justified theoretically and is subject to debate. The range is defined as follows:

(3) $$\begin{eqnarray}0\leqslant P(H_{1}\cap H_{2})\leqslant \min (P(H_{1}),P(H_{2})).\end{eqnarray}$$

Likelihoods. After deriving the priors— $P(H_{1})$ , $P(H_{2})$ , and $P(H_{1}\cap H_{2})$ —researchers must specify three likelihoods. First, they must estimate $P(K=1|H_{1})$ : the probability of finding evidence $K$ if $H_{1}$ is true. Second, they must estimate the same quantity for $H_{2}$ . Third, they must estimate the probability of finding $K$ in the intersection, $P(K=1|H_{1}\cap H_{2})$ . How $P(K=1|H_{1}\cap H_{2})$ is calculated depends on the brand of nonexclusivity. If $H_{1}$ and $H_{2}$ are coincident, and a given piece of evidence is likely under $H_{1}$ , but effectively zero in $H_{2}$ , $P(K=1|H_{1}\cap H_{2})$ is given by

(4) $$\begin{eqnarray}P(K=1|H_{1}\cap H_{2})=P(K=1|H_{1})P(H_{1}\cap H_{2}).\end{eqnarray}$$

If, however, the two hypotheses are congruent and evidence $K$ is expected with some positive probability under both $H_{1}$ and $H_{2}$ , $P(K=1|H_{1}\cap H_{2})$ is given by,

(5) $$\begin{eqnarray}P(K=1|H_{1}\cap H_{2})=1-[P(K=0|H_{1})P(K=0|H_{2})].\end{eqnarray}$$

Finally, the last term in the denominator, $P(K=1|\overline{H_{1}\cap H_{2}})$ , represents the probability of observing $K$ anywhere in the rival space except for the intersection of $H_{1}$ and $H_{2}$ . Computing this term is cumbersome in its own rite. Building on our intuition from Figure 2b, we must compute the probability of observing $K$ under each other alternative in the rival space and weight each by the area of the respective hypothesis (i.e. by the probability that hypothesis is true). Thus, $P(K=1|\overline{H_{1}\cap H_{2}})$ is given by,

(6) $$\begin{eqnarray}\begin{array}{@{}rcl@{}}P(K=1|\overline{H_{1}\cap H_{2}}) & = & \left[P(H_{1})-P(H_{1}\cap H_{2})\right]P(K=1|H_{1})\\ & & +\left[P(H_{2})-P(H_{1}\cap H_{2})\right]P(K=1|H_{2})\\ & & +\left[1-P(H_{1}\cup H_{2})\right]P(K=1|\overline{H_{1}\cup H_{2}}).\end{array}\end{eqnarray}$$

For $n$ hypotheses, the final term in Equation 6 is computed as,

(7) $$\begin{eqnarray}[1-P(H_{1}\cup H_{2})]P(K=1|\overline{H_{1}\cup H_{2}})=\mathop{\sum }_{i=3}^{n}P(K=1|H_{i})p(H_{i}).^{17}\end{eqnarray}$$

The key takeaway from this discussion is that Bayesian procedures can be expanded to accommodate nonexclusivity, but researchers will encounter a tradeoff between rigor and transparency on the one hand, and added complexity on the other. The Bayesian approach still has the benefit of forcing researchers to be explicit about their assumptions (Bennett Reference Bennett2015; Humphreys and Jacobs Reference Humphreys and Jacobs2015). However, the number of quantities the researcher must posit and justify theoretically grows considerably as more hypotheses are thrown into the mix. The insights derived here will enable researchers to assess the tractability of Bayesian process tracing for a given project and to employ this approach more accurately should they choose to.

5.1 Step 1: Build the Rival Space

Researchers should begin process tracing by constructing the rival space: an exhaustive set of alternative explanations for the phenomenon.Footnote ¹⁸ This suggestion echoes Platt’s (Reference Platt1964) exposition on the procedure of Strong Inference, in which he urges researchers to “devise and write down our alternatives every day” (348). Whether represented graphically (as something akin to a Venn diagram as in Figure 3) or as a list, working explicitly with rival spaces prior to analysis can reveal potential problemsFootnote ¹⁹ and can guide evidence collection and interpretation in a more directed way. In accordance with the RAR framework, the rival space should include (1) the main explanation and the alternatives, (2) the constituent hypotheses and assumptions that comprise each theory, and (3) the relationships among them. In a graphical depiction of the rival space, this task involves asking, Do  $H_{M}$ and  $H_{R}$ overlap? If so, by how much? And on which pieces of evidence?

Figure 3.

Graphical representation of the rival hypothesis space.

5.2 Step 2: Devise Tests

In light of the RAR framework, researchers must devise two categories of tests. First, researchers must think about the type of evidence required to corroborate or invalidate each competing explanation. On this point, Waldner (Reference Waldner, Bennett and Checkel2015) makes a compelling case for standardizing the use of directed causal graphs to specify mechanisms and to help researchers identify the constituent pieces of a theory for which they must find evidence.Footnote ²⁰ This particular exercise forces researchers to think about (and then test) the underlying mechanisms and assumptions that motivate a theory’s predictions.

Second, researchers should devise tests of the posited relationships among explanations. What sort of evidence would suggest that $H_{M}$ and $H_{R}$ are working in conjunction? In the context of Figure 3, we might ask how we knew we were operating in the darker overlap areas. Alternately, we might ask what sort of evidence would suggest mutual exclusivity. Thinking a priori about how to test relationships among competing explanations can both guide evidence collection and promote more nuanced interpretation.

5.3 Step 3: Collect and Evaluate Evidence

In this template, the process of evidence collection and evaluation is explicitly catholic in nature: researchers may opt to construct analytic narratives based on the evidence, they may draw on the crisp-set-theoretic approach, or they may use explicit Bayesian inference. Each tool represents a modeling choice, the appropriateness of which is left to the researcher to justify. Furthermore—and in the spirit of avoiding assumptions of mutual exclusivity on all fronts—researchers may use different approaches for different pieces of evidence within the same project.Footnote ²¹ For example, if for one part of a project, a researcher is adjudicating between two exclusive hypotheses for which the overall balance of evidence is unclear, Fairfield and Charman (Reference Fairfield and Charman2015) argue that Bayesian computation is especially useful. Yet for another hypothesis, the balance of evidence may be so clearly tipped in one direction that the benefits of computation become negligible, in which case a simple narrative presentation may be preferable.

6.1 Main Hypothesis: Affirming the “Confirmatory Effect”

Schultz’s main hypothesis is that French decision-makers backed down once they realized that the British government had support from both the majority and opposition parties in parliament. This hypothesis is an application of a more general proposition known as the confirmatory effect: “the idea that a signal sent by two actors with opposing interests is more informative than a signal sent by one actor with known incentives to misrepresent its preferences” (2001, 162). Schultz validates his theory by first providing evidence that passes a hoop test, then providing strong confirmatory evidence in favor of this explanation, which in effect satisfies a smoking-gun test.

H1:

Support from the opposition party in Britain is a requirement for the confirmatory effect.

To satisfy this hoop test, Schultz provides primary source evidence indicating the Liberal Party’s resolve to back the governing party. The most convincing piece comes from a speech made by the Liberal Party leader in which he states, “Behind the policy of the government is the united strength of the nation $\ldots \,$ . The nation will make any sacrifice and go any length to sustain them” (188).

H2:

France backed down due to unified British resolve.

On this point, Schultz finds evidence that nearly constitutes a smoking gun for his hypothesis: a letter between key French decision-makers noting that “the Liberals [in London] have come out as much if not more intransigent than the partisans in the government” (190). Schultz concludes from this exchange that “the support of British opposition groups was not lost on French diplomats” (190).

Schultz is able to supply the sort of confirmatory evidence of which most researchers only dream. Yet his treatment of competing hypotheses could be strengthened given a more nuanced conceptualization of their relationship to his main. Reevaluating Schultz’s evidence in light of the new framework reveals key oversights regarding his alternative explanations.

6.2 Neorealism: Establishing Coincident Rivals

Turning to competing explanations, Schultz first explores the neorealist balance-of-power theory. Since Britain had a decisive military advantage, neorealists would argue that “France backed down because Britain was stronger” Schultz (Reference Schultz2001, 177). Despite the valid prediction on the outcome, Schultz dismisses balance-of-power theory on the grounds that it fails to explain the onset of the crisis in the first place, and why the conflict escalated to near war (Reference Schultz2001, 177).

However, taking Schultz’s evidence through the RAR framework reveals that while power asymmetries alone do not account for all aspects of the crisis, this explanation should not be entirely ruled out. I first examine the relationship between Schultz’s confirmatory effect hypothesis and balance-of-power theory. Drawing on insights from the relationship, I demonstrate that evidence provided to rule out balance-of-power theory was inadequate.

Q1:

Do balance of power and the confirmatory effect predict divergent outcomes?

No—neither would have predicted an escalation to all-out war.

Q2:

Do the theories require divergent evidence?

No—the confirmatory effect hypothesis requires evidence about threat perception and the credibility of threats from unified governments. Balance-of-power theory requires evidence of a power asymmetry and actors’ knowledge of the power distribution.

Q3:

Does evidence in favor of one have any effect on the other?

No, the two theories operate via distinct, but noncontradictory pathways. Balance of power does not acknowledge substate behavior, and the confirmatory effect is not affected at all by the actual power distribution.

It follows that confirmatory effect and balance of power are coincident hypotheses: they operate through distinct pathways to make similar predictions about the outcome at Fashoda. In the broader context of the conflict, it is likely that a combination of British resolve and military advantage worked together to convince France to back down. After all, if France had evidence of Britain’s resolve to go to war, yet the power distribution favored the French, retreat would have been less likely. Since the rival explanations are coincident, ruling out balance-of-power theory requires finding explicit evidence that Britain’s military advantage did not factor into French decision-making. The following summarizes Schultz’s evaluation of the neorealist explanation.

H1:

Preexisting knowledge of the power distribution is a requirement for avoiding war.

To address this test, Schultz first turns to a statement made by the British Admiralty claiming that “in the event of war, France would not have ‘a ghost of a chance”’ (177). He argues that “the French had to know from the outset” that they could not withstand a confrontation, though he does not provide explicit evidence of their knowledge.

H2:

Collusion with third parties to tip the balance is a requirement for the weaker state to act aggressively.

On this point, Schultz provides evidence that Russia was neither willing nor able to assist France in the crisis, and that French leaders did not harbor optimistic expectations of Russian assistance (2001, 180).

Schultz provides convincing evidence that balance-of-power theory cannot explain French aggression or the duration of the crisis. Insights from RAR, however, reveal a critical gap between the evidence needed and the evidence used to eliminate the neorealist explanation. Merely demonstrating that a theory provides an incomplete explanation is an insufficient basis to eliminate it from consideration. The evaluation of balance-of-power theory could have been stronger by acknowledging its partial role in explaining the outcome at Fashoda.

6.3 Institutional Constraints: Insights from Congruent Rivals

Schultz’s second alternative explanation derives from the institutional democratic peace proposition (hereafter, IDPP). He draws on the underlying mechanism to derive a set of monadic expectations about the behavior of British leaders during the crisis.Footnote ²⁴ The central tenet of this theory is that democratic institutions—specifically, a legitimated opposition party and the public’s ability to sanction leaders via elections—exert constraints on democratic leaders’ behavior. In short, since democratic leaders usually face multiple and varied sources of opposition to war, it is difficult for them to take unilateral military action, thus impeding hawkish behavior (Reference Schultz2001, pp. 13, 18, 182, 183). In light of Britain’s steadfast belligerence, Schultz quickly rules out the IDPP as “problematic” (Reference Schultz2001, 182).

Since the institutional constraints argument typically explains why conflict is less likely, this explanation indeed appears inappropriate. To evaluate the validity of his conclusion, I reanalyze his evidence through the RAR framework: first by considering the relationship between the IDPP and the confirmatory effect, then by retesting the IDPP in light of the insights revealed. I demonstrate that a more systematic evaluation of evidence in line with the prescriptions of RAR would have (1) generated the opposite conclusion regarding the role of institutional effects in explaining the Fashoda crisis, and (2) revealed an interesting and counterintuitive prediction regarding conflict behavior.

Q1:

Do the theories predict divergent outcomes?

No—largely because they make predictions about different aspects of conflict. Neither would predict war, and their respective logics stem from different places.

Q2:

Do the theories require divergent evidence?

The evidence required to test the confirmatory effect is (1) a unified opposition, (2) a threat, (3) the target observing that the government is unified behind the threat.

The evidence required to test whether institutional constraints shape leader behavior is (1) a specification of the variety of opinions across the polity (the governing party, the opposition party, and the public), (2) an indication that the leader’s behavior was a function of those opinions.

Since the IDPP does not deal with threats, the last two conditions for the confirmatory effect (threat existence and perception of unanimity) can be ruled out as possible triggers of mutual exclusivity. We are then left asking how evidence of a unified government would bear on the evidentiary requirements of IDPP. Since one requirement of demonstrating institutional constraints is to specify the range of opinions across the polity, evidence of a unified government represents one possible configuration of opinions out of many. Unification does not controvert IDPP; rather, it describes a rare condition in which the governing party and opposition party agree. As such, we can rule out evidentiary mutual exclusivity as well.

Beyond ruling out mutual exclusivity, the rarity of a unified government raises an important question: what would the institutional constraints argument predict under this condition? This question highlights a crucial and testable assumption on which IDPP predictions are based. IDPP assumes that the more people to whom a leader is accountable, the more likely she is to encounter a wide variety of opinions pulling in opposing directions. Schultz and others predict restraint and pacifism because navigating countervailing opinions leaves the leader no choice but to be balanced and circumspect. However, when all of the constraining opinions are exerting force in the same direction, the theory of institutional constraints is also supported if a leader is consequently forced to move excessively far in an undesirable policy direction (e.g. going to war). Testing that assumption reveals a novel insight.

A1:

The presence of multiple factions in government leads to a variety of countervailing opinions about going to war.

As Schultz previously stated, the nation was “unified”, and the Liberals “have come out as much if not more intransigent than the partisans in the government” (2001, 190). Thus, the assumption on which IDPP predictions are based does not hold.

At this point, Schultz overlooks an interesting and unprecedented outcome of the IDPP, which in turn leads to an inconsistency in his argument. The falseness of the assumption does not undermine the theory, rather, it changes the prediction. Since all actors were pushing for war, the IDPP would, ironically, predict that Salisbury was constrained into adopting a hawkish stance, which is precisely what Schultz’s evidence shows. Thus, the evidence Schultz provides to counter the IDPP actually corroborates it. Consequently, the confirmatory effect and the institutional constraints argument are congruent: they make similar predictions and are corroborated by similar evidence.

In accordance with the RAR framework, an analysis of the relationship between IDPP and the confirmatory effect hypothesis reveals not only their congruence, but also a critical assumption on which the institutional constraints argument relies. Rather than undermining IDPP, Schultz’s corroborating evidence for the confirmatory effect—unified government opinion—unearths a rare political situation that fundamentally alters the predictions of the theory. This insight would have allowed Schultz to exploit an interesting opportunity in IR theory: predictions about conflict proneness change as a function of government unification. Schultz has the evidence to demonstrate that when all constraining actors happen to agree on a policy issue, democratic leaders may be forced away from pacifism and toward belligerence. This insight suggests that some of the IR literature broadly conflates institutional constraint and institutional restraint.