How effective are intelligence agencies at assessing and anticipating the nuclear intentions of foreign countries? More specifically, can they reliably detect and predict when other countries will seek to acquire nuclear weapons?
Correct assessments of this sort are a crucial precursor for successful nonproliferation efforts.Footnote 1 According to one recent study, the vast majority of nonproliferation deals are struck early, when a country is only considering a nuclear weapons program or has just begun one. If this window is missed, deals tend to be struck late, which requires correspondingly greater concessions to the proliferator.Footnote 2 But achieving these optimal early deals requires accurate forecasting and early detection of countries’ nuclear intentions—a process we have little understanding of. Other prominent research argues that uncertainty about whether a country has or will soon invest in a nuclear weapons program can generate mistaken preventive wars, as in the ill-fated 2003 invasion of Iraq.Footnote 3 Yet we have little empirical evidence on how accurate such intelligence assessments are on average and therefore how uncertain policymakers should be.
While the ability of states to assess other countries’ nuclear intentions may not tell us whether they can discern other sorts of intentions—for example, to go to war—it nonetheless is of clear policy relevance and establishes a baseline for assessing intentions to acquire powerful military technologies. Moreover, the nuclear domain presents many of the difficulties identified in the literature as making judgment of intentions especially challenging: there is a high degree of ambiguity, in part due to the dual-use nature of nuclear facilities; there are usually efforts at deception; and key decisions are often made by a small set of leaders, who can have widely divergent preferences. Finally, most of the assessments analyzed here are efforts to predict future intentions, which is generally considered harder than making judgments about the present.Footnote 4
In the realm of nuclear proliferation, there have been four major works examining the accuracy of intelligence assessments. Yusuf surveyed proliferation predictions from US intelligence agencies and nongovernmental analysts since the onset of the Cold War, finding a significant bias toward alarmism and over-prediction of the extent of proliferation.Footnote 5 Looking at US estimates of countries with active nuclear weapons programs, Montgomery and Mount likewise find a poor track record, with a bias toward overestimation.Footnote 6 Bollfrass quantitatively examined the accuracy of US and European assessments during the Cold War. In contrast to the prior work, Bollfrass found a strikingly high accuracy rate for US and British assessments—over 90 percent—and a tendency toward underestimating rather than overestimating programs.Footnote 7 Most recently, Schneider qualitatively analyzed US assessments from the 1950s to 1990s and found a bias toward overestimating the number of countries that would acquire nuclear weapons, which he attributes to a focus on technological capacity rather than motives.Footnote 8
The goal of this research note is to build on this literature by examining how assessment accuracy evolves over time. Specifically, I analyze close to 200 US assessments of countries’ nuclear intentions, drawn from National Intelligence Estimates (NIEs) produced between 1957 and 1966. I find a high success rate overall, with close to 80 percent of testable assessments in the NIEs correctly differentiating between countries that would and would not proliferate. However, this number obscures wide variation in accuracy over time, as assessments improved from highly inaccurate in the late 1950s to highly accurate in the 1960s. Drawing on additional evidence and analysis, I conclude that learning from past failures played a major role in this improved performance.
This research note begins by describing the data used and the methodology. After analyzing the accuracy of US assessments overall, I explore why the assessments improved. I conclude with a discussion of the implications of the findings.
Data and Methods
Analysts in the US government have long sought to forecast the likelihood and consequences of the spread of nuclear weapons. In the late 1950s the intelligence community (IC) began producing NIEs that systematically tackled this problem, identifying which countries were currently seeking nuclear weapons and which were likely to do so over the next decade. The data for this research note are drawn from the NIEs that have been declassified, covering the period from 1957, when the first NIE was completed, to 1966, when basic agreement was reached between the United States and the Soviet Union on the Nonproliferation Treaty.Footnote 9
This research builds on the prior work discussed but differs in a few key respects. Instead of examining estimates of the technical progress of a country's nuclear program and its likely date of acquisition—the subject of many of the estimates Montgomery and Mount and Schneider analyze—it zeroes in on estimates of a country's current and future intentions to develop nuclear weapons. This narrows the scope of inquiry but also broadens the aperture to include estimates for countries that never went on to seek nuclear weapons but were considered candidates for proliferation at the time. Thus the findings shed light on whether the IC was accurate in predicting who was likely to be next in seeking nuclear weapons, and the proportion of false positives—countries that were predicted to seek nuclear weapons but did not.
The analysis here also differs from Bollfrass's work in several ways. It restricts the analysis to the US IC, which has the most declassified estimates, enabling longitudinal analysis. Moreover, instead of treating the country-year as the unit of analysis and using a global sample, it uses the individual assessment of a given country, as explained later. This avoids an inflated sample size which might bias the analysis toward finding a high success rate and statistically significant results.Footnote 10 Finally, this research focuses on whether the quality of assessments changes over time, and also assesses more fine-grained process assessments—that is, not just whether a country would seek nuclear weapons but whether a given event would lead them to debate the nuclear option, or a particular arms control policy would affect their calculus.
I focus on NIEs, as opposed to the wide range of other estimates produced by the US government, for two main reasons. First, NIEs are widely regarded as the most important and influential. As one intelligence scholar puts it, “The NIE is arguably the highest form of intelligence art…The authors of these documents…are considered to be the best analysts in the US Intelligence Community.”Footnote 11 According to Sherman Kent, who led the production of NIEs in the 1950s and 1960s, NIEs were intended “to command recognition and respect throughout the Government as the best available and presumably the most authoritative estimate.”Footnote 12 NIEs were produced by the CIA, but only after extensive consultation with the government's other intelligence agencies, whose leaders read and approved the final text.Footnote 13 Second, by examining a series of NIEs, rather than a wide range of estimates produced by different agencies and through different processes, we have a more homogeneous set of data, as NIEs followed mostly consistent procedures over time with respect to the drafting process.Footnote 14 This supports “apples to apples” comparisons across assessments and over time.
Coding the Accuracy of Assessments
The unit of observation is an individual assessment of a country's current or future intention to seek nuclear weapons.Footnote 15 I exclude assessments of countries that have already acquired nuclear weapons and leave out duplicate assessments from the same NIE. I omit vague assessments—words like “might,” “could,” and “is possible”—whose accuracy cannot be coded. I also exclude assessments that are conditional on an antecedent condition that did not occur—for example, “If Japan acquires nuclear weapons, South Korea will follow.” Applying these coding rules to the seven declassified NIEs on proliferation under study—from 1957, 1958, 1960, 1961, 1963, 1964, and 1966—yields 199 distinct observations.Footnote 16
One limitation of this study is that it relies on declassified assessments. Several of the NIEs have redacted portions, although these are relatively small compared to the length of the documents and may reflect information about sources and methods rather than the conclusions themselves. In any case, I draw on the assessments that are declassified as a first cut.
I assess the accuracy of the assessments in various ways, drawing on recent research on political forecasting.Footnote 17 First, I assess the discrimination of the assessments, or whether they accurately differentiate events that do occur from those that do not. Following Mandel and Barnes, I use a simple measure and code assessments as correctly classified if one of two conditions holds: greater than 50 percent odds are attached and the event does in fact occur in the specified time frame; or less than 50 percent odds are attached and the event does not occur in the specified time frame. Conversely, assessments are coded as incorrect if less then 50 percent odds are attached and the event does occur in the time frame, or greater than 50 percent odds are attached and the event does not occur in the time frame.Footnote 18
Second, I code false positives and false negatives. A false negative is when an assessment predicted no proliferation but proliferation did in fact occur. A false positive is when an assessment predicted proliferation but it did not occur.
Third, I assess the overall accuracy of forecasts using Brier scores, which range between 0 and 1 and capture the difference between forecasted probabilities and actual outcomes.Footnote 19 This metric accounts not just for discrimination but also the calibration of the assessments: in other words, whether events forecasted as 75 percent likely, for instance, actually occur around 75 percent of the time. The further the forecasted probabilities are from the actual outcomes (meaning a less accurate forecast), the closer the Brier score is to 1. To calculate Brier scores, it is necessary to translate the qualitative probabilities in the NIEs into quantitative terms. I draw on research by Mandel, who conducted an experiment to gauge the numerical probabilities people associate with a range of linguistic probability terms.Footnote 20
Finally, I assess calibration separately by calculating a calibration index, which ranges between 0 and 1, where 0 is perfect calibration. This statistic captures the mean square differences between the forecasted probabilities and the actual frequency of events, averaged over each of the probability categories and weighted by the number of observations in each category.Footnote 21
Coding whether an NIE accurately assessed that a country would seek nuclear weapons poses a few challenges. It can be ambiguous when the weapons program started or stopped. Moreover, the dividing line between exploration and a determined pursuit of nuclear weapons can be blurry.Footnote 22 When the language in NIEs refers to the outcome of interest for this study, deciding to seek nuclear weapons, I rely on Singh and Way's coding, which draws on historical sources to cover all countries from 1945 to 2009 and capture instances where there is a “political decision by cabinet-level officials, movement toward weaponization, or development of single-use, dedicated technology.”Footnote 23 As a robustness check, I use Bleek's coding of nuclear pursuit, which uses the same definition as Singh and Way but draws on different historical sources.Footnote 24 When it comes to more fine-grained process assessments—for example, whether a country would seek to keep its nuclear options open, debate developing nuclear weapons, seek nuclear weapons from a third country, or whether a particular US policy would sway their decisions—I rely on secondary sources for countries’ nuclear histories. About 45 percent of the assessments in the data set are about process.
Another potential concern is that the process and outcome assessments are not always independent of each other and that this could artificially inflate the accuracy rate. For example, one assessment may predict that India will seek nuclear weapons, and another in the same year may predict a specific event that will trigger this decision and how India is likely to respond to a US nonproliferation initiative. Logically, if the former assessment is correct, the latter are more likely to be as well. To address this concern, I divide the sample into process and outcome observations and assess whether including process assessments drives up the accuracy rate.Footnote 25
A final challenge in determining accuracy is the possibility of self-defeating prophecies, whereby the assessors are correct that a country would seek nuclear weapons but it was dissuaded by successful policy interventions. To some extent, this problem is insurmountable, since we cannot test the alternate history where a given policy intervention did not take place. On the other hand, the assessors were sensitive to this issue and explicitly addressed the possibility that US or other nonproliferation efforts would affect a state's calculus. Thus, if an NIE predicted that US policies would affect a state's choices, I code this as accurate if the policy occurred and historical evidence suggests it swayed the country's nuclear decisions. In other words, the assessments under study include those that predict the impact of US policies.
Analysis and Results
Pooled Sample
So, how accurate were US assessments of other countries’ nuclear intentions? Starting with discrimination, 78 percent were correctly classified, which seems decent if one's prior is that it is highly challenging to judge current or future intentions.Footnote 26 This rate is lower, however, than the over 90 percent recorded in Bollfrass’s research, or the 94 percent found in Mandel and Barnes’s recent analysis of Canadian intelligence estimates for the Middle East and Africa.Footnote 27 If we exclude observations about process and look only at outcomes, accuracy increases to 83.3 percent. This suggests that including process assessments is actually driving down the accuracy rate rather than leading to an artificially high rate.
But is 78 percent actually impressive, given the type of events being forecast? After all, if seeking nuclear weapons is a rare event, and assessors always (or almost always) predicted no proliferation, they would have a high accuracy rate but also be unhelpful in predicting the rare cases where proliferation does occur. However, proliferation is not a rare outcome in this data set, since the assessments focus on countries deemed to be relatively high proliferation risks. And predictions of proliferation and nonproliferation were both reasonably accurate (Table 1). While predictions that countries would not seek (or move toward) nuclear weapons were better (85.3 percent), predictions that they would were still mostly on target (68.7 percent). Indeed, if an assessor simply predicted no proliferation in every instance—the modal outcome—they would only have been correct 63 percent of the time, as opposed to the 78 percent we actually observe using the NIE data, a statistically significant difference.
Discrimination of intelligence assessments
A stronger counterargument to this optimistic take is that the analysis excludes vague assessments, whose accuracy could not be judged. This probably biases in favor of finding a high success rate, since vague estimates could indicate a lack of confidence. Indeed, about 20 percent of assessments were too vague to be included in the data set, which highlights an important limitation to the utility of the assessments.
With respect to false positives versus false negatives, of the forty-three incorrect assessments, 60.4 percent were false positives and 39.6 percent were false negatives. This ratio, about 1.5 to 1, is what one might expect in a situation where the costs of inaction (failing to prevent proliferation) are perceived to be high, thus incentivizing false positives.Footnote 28
Turning to the Brier score, which helps account for both discrimination and calibration, the result is 0.176. This is far from a perfect score of 0 (no error in forecasts) but better than the 0.23 score of predicting the base rate probability of proliferation in each instance, a difference that is statistically significant at the 95 percent level.
The calibration index for the full sample is 0.026, which is not as impressive as the results of previously published analyses of expert forecasts in a variety of different issue areas.Footnote 29 In the online supplement (Table A7), I provide raw data on calibration, that is, the observed frequency of events for each forecasted probability. These data show that the forecasts tended to be overconfident. For example, events that were deemed to have no chance of occurring actually occurred about 8 percent of the time, and events forecast as certain only occurred about 78 percent of the time.
Change over Time
These results suggest that the NIEs’ assessments of intentions were reasonably accurate as a whole, though by no means perfect. But how did this change over time? Using the basic measure of discrimination, Figure 1 breaks up the sample by year and reveals a striking improvement between the initial assessments in 1957–58 and the later assessments of 1960–66, a statistically significant difference. Figure 2 demonstrates that this change is driven by a downward trend in the rate of false positives, while there is no clear trend in the rate of false negatives. The difference between the 1957–58 and 1960–66 periods in the rate of false positives is statistically significant; the difference in the rate of false negatives is not.
Performance by year
Error type, by year
Table 2 shows Brier scores calculated separately for the 1957–58 and 1960–66 periods, again demonstrating a dramatic improvement between the 1950s and 1960s. The score for 1957–58 is worse than what would be achieved by predicting the base rate probability of proliferation for this period. Starting in the 1960s, this trend reverses, with Brier scores showing that the NIEs outperform the predictions using base rates. The difference in scores between the 1957–58 and 1960–66 periods is statistically significant at the 95 percent level.
Brier scores over time
Calibration likewise dramatically improved across the two periods, from 0.189 in the 1950s to 0.012 in the 1960s. The 0.012 figure for the later period is on par with the performance of expert analysts in other published studies of forecasting.Footnote 30 The raw data on calibration (Tables A8 and A9) tell a similar story. In the 1950s, observed frequencies were way off the mark of forecast probabilities, and the analysts had a striking bias toward incorrectly estimating events as highly likely. For instance, of events forecast with 75 percent probability, only 35 percent occurred; of events forecast to be certain, only 43 percent occurred. In the 1960s, calibration dramatically improved. For example, the events characterized as 75 percent likely occurred 76 percent of the time in actuality, and “certain” predictions came to pass 94 percent of the time.
In the online supplement, I show that the improvement over time is robust to accounting for the composition of countries and measurable characteristics like regime type and relation to the United States, as well as substituting Bleek's coding of nuclear programs. To test whether the results still hold after the 1960s, I coded a similar 1974 NIE and find the improvement was sustained. The results also hold if we restrict the data set to outcome assessments.
Accounting for the Improved Accuracy
The evidence demonstrates that estimates of proliferation intentions significantly improved over time—in particular by reducing the rate of false positives—but it does not demonstrate how or why this improvement occurred. One possibility is that the improvement was the result of learning from the mistakes of early assessments. Jack Levy defines learning as “a change of beliefs (or the degree of confidence in one's beliefs) or the development of new beliefs, skills, or procedures as a result of the observation and interpretation of experience.” Learning “is not a passive activity” and involves effort to gather the information needed to evaluate ideas, as well as trial-and-error policy changes to assess whether outcomes improve. Learning can be either causal, that is, changing beliefs about cause and effect, or diagnostic, that is, changing beliefs “about the definition of the situation or the preferences, intentions, or relative capabilities of others.” As a general rule, learning occurs at the individual level; organizations only “learn” when individuals take “action to change organizational procedures, which leads to a change in organizational behavior.”Footnote 31
Using this definition, if learning is responsible for the IC's improved performance, we would expect to find evidence that (1) the IC gathered information to assess its performance; (2) this information led to a change in skills, procedures, or beliefs that was institutionalized at the organizational level; and (3) these changes led to the improved performance we observe.
On the first score, declassified documents make clear that the IC was gathering information on the accuracy of its NIEs. In September 1958, two months after that year's NIE was completed, the US embassy in Sweden cabled Washington to register its dissent from how the estimate had characterized Sweden's nuclear intentions, arguing that it was “not supported by present facts” and recommending that the estimate lower the likelihood of Swedish proliferation.Footnote 32 The following month, the State Department cabled its embassies in Western Europe to ask for more information on French and possible cooperative European nuclear weapons programs, which it noted had been a weak point in the 1958 NIE. Rather than approaching French or European officials directly on this question, it suggested indirect approaches, as well as seeking to glean insight from third parties.Footnote 33
A few weeks later, a postmortem on the 1958 NIE was produced, expressly to assess its quality. The report identified improvements over the 1957 installment, but nonetheless judged that “in many respects the available information did not permit confident judgments of the timing and extent of future programs, to the degree that we believe information could be obtained with a higher priority effort.”Footnote 34 As targets for greater collection going forward, the report singled out France, Italy, West Germany, and other European nations, noting that “it should be possible to increase the scale of appropriate operation to obtain fuller and more accurate information.” In a separate recommendation, the postmortem called for more reporting on the “the political factors in the possible decisions of fourth countries to initiate nuclear weapons programs,” which “could take greater account of the underlying forces, often not openly expressed, that may influence the making and shape of such decision.”Footnote 35
The postmortem thus suggests that at least two types of learning were taking place: development of new procedures, such as gathering more information and collating information already available from different sources for a specific set of Western European countries; and possibly a change in causal beliefs, highlighting the importance of political factors in affecting states’ nuclear decisions.
There is also evidence of an effort to institutionalize this learning. In addition to the memo sent to Western European embassies in September 1958, the entire postmortem was transmitted in December 1958 to the US embassies in Switzerland, West Germany, Belgium, the Netherlands, the United Kingdom, Luxembourg, France, Italy, Sweden, and Japan. In delivering the report, the State Department again emphasized the importance of political factors, both domestic and international: “Analysis of political factors, such as, for example, the interplay of party orientation and rivalries, the effect of scheduled elections, the implications of national aims and roles, and a country's relations with and attitudes toward other powers is of material importance to an assessment of ‘fourth power’ intentions in the nuclear field.”Footnote 36 As later noted in a 1961 postmortem on Israel's nuclear program, these recommendations were not communicated to all US embassies abroad, possibly contributing to US failure to detect Israel's nuclear program in a timely fashion.Footnote 37 But it wasn't just US embassies that were receiving new instructions. The postmortem also called on the Atomic Energy Commission and the Joint Atomic Energy Intelligence Committee to work on gathering more information, and directed the US Intelligence Board, which helped oversee the production of NIEs, to ensure greater emphasis on political factors.Footnote 38
But did this actually lead to the improvement we observe? If so, we would expect to find that assessments improved especially for countries that were identified as high-priority targets, where more information and effort were requested, and less so for other countries. And if learning is a result not just of gathering more information but also of the IC changing its type of analysis, we should also expect to find that initial assessments of countries were more accurate after 1958. If this latter implication holds, it helps rule out the possibility that the improved performance was entirely due to having observed a given country's behavior for longer, as opposed to new information-gathering procedures or a focus on different causal factors.
The data confirm both observable implications. In countries identified as targets in 1958, there is a massive improvement (Figure 3). Brier scores show the same pattern, with assessments of countries designated as targets changing from highly uninformative in the 1950s to highly informative in the 1960s (Table A5). For countries not identified as high-priority targets, there is only very modest, nonsignificant improvement after 1958 (Figure 4). Regression models that account for country fixed effects and potential confounders and use the basic measure of discrimination as the dependent variable similarly show significantly better performance in the 1960s for countries identified as targets and nonsignificant improvement for other countries (Table A6).
Performance by year, intel targets
Performance by year, non-targets
Finally, if we look only at countries that are being assessed for the first time, accuracy also improved dramatically—a statistically significant difference (Figure 5). This suggests that the IC was successful not only at improving accuracy with specific target countries but also at improving its performance with new countries going forward.
Accuracy of first assessments
Taken together, the qualitative and quantitative evidence strongly suggests that the IC learned from its early failures and took effective steps to improve assessments. To increase confidence in this finding, I turn to a short case study of Sweden to tease out how exactly assessments improved after 1958.
The Swedish Case
One case that illustrates the improvement of US assessments is Sweden, one of the countries where US officials were directed to step up their collection and analysis efforts after 1958. In both the 1957 and 1958 NIEs, the IC was overly alarmist in predicting that Stockholm would develop nuclear weapons. According to the 1957 NIE, “Although a vocal minority opposes equipping Sweden's armed forces with nuclear weapons, public opinion in general appears to support the government view that nuclear weapons are essential to Sweden's defense.” The NIE concluded that “in the absence of substantial progress in disarmament, Sweden will initiate production of nuclear weapons as the necessary materials become available.”Footnote 39 The following year's NIE came to an identical conclusion based on slightly different logic and evidence. The report acknowledged that in Sweden there was “strong opposition to nuclear weapons on moral grounds” as well as “uncertainty as to whether Sweden's traditional neutrality is better served by foregoing or by producing atomic arms,” given that developing nuclear weapons may in fact trigger an attack. Nevertheless, the NIE judged, “the major political parties and most leaders are inclined to compromise [on] this issue by supporting the manufacture of tactical and air defense atomic weapons, while avoiding—at least initially—the more ‘provocative’ longer range missiles.”Footnote 40
Based on what we know now, there are several things the 1957 and 1958 NIEs got right, even though they erred in predicting that Sweden would ultimately go nuclear. They were correct that a plurality of the public supported acquiring nuclear weapons and that there was also strongly felt opposition, in segments of both the populace and the elite. They were also right that these divisions would incline the Swedish government to seek a compromise.Footnote 41 However, the estimates exaggerated the support for nuclear weapons in the political leadership and got wrong the form that this compromise would take.
In July 1958, the Swedish parliament declined to fund a program to develop nuclear weapons, instead authorizing “protection research,” which allowed research into how to defend against nuclear weapons.Footnote 42 A few months later, prime minister Tage Erlander of the Social Democratic Party (SDP) established a committee to study the issue and help resolve the debate. The committee, which included strong opponents of nuclear weapons, recommended that the government continue to follow the line set out by parliament: that is, no development of nuclear weapons but continued research into defense against them, which would allow Sweden to advance its nuclear expertise without taking the provocative step of building the bomb.Footnote 43 As Arnett explains, “Erlander claimed to be sympathetic with the pro-nuclear side, but did not want to risk dividing the party over the issue.” The antinuclear position was strengthened by the outcome of the 1958 election, which compelled the SDP into an alliance with the strongly antinuclear Communist Party.Footnote 44 Cole likewise concurs that the SDP “sought to avoid a split that would seriously weaken the party,” leading to a decision to defer producing nuclear weapons.Footnote 45
As noted before, the US embassy in Sweden had criticized the 1958 NIE as alarmist. It specifically took issue with the idea that Swedish leaders would “compromise” by producing short-range nuclear weapons, arguing that this was based on a misreading of Swedish politics. According to the memo:
Of the four non-Communist parties, only the leadership of the opposition Conservative Party has pleaded the case for atomic weapons as a modern defensive necessity while the other three parties, including the governing Social Democrats, have not yet taken definitive positions and are not likely to do so in the near future. In these circumstances the distinction between tactical and strategic atomic weapons has not been of much significance as of yet. The Embassy therefore recommends that the sentence be revised to read as follows: “The question of domestic production of these weapons is still a matter of party and public debate, with party opinion apparently lagging somewhat behind some segments of public opinion in favor of Sweden's acquiring them.”
The memo recommended that the NIE's prediction for Sweden be revised to include more uncertainty: Sweden would “probably” produce nuclear weapons in the absence of disarmament measures, as opposed to Sweden “will” do so.Footnote 46
By 1960, the US assessment of Sweden's intentions had changed, in line with the feedback from the embassy and the 1958 postmortem's recommendation to focus more on political factors. The 1960 NIE especially emphasized domestic politics as a major inhibitor of any decision to develop nuclear weapons, describing “strong opposition within the SDP,” and noting that “many Swedes, and particularly those in the Social-Democratic leadership, continue to hope that the international climate will develop so as to obviate the need to make a decision.” Whereas the previous NIEs had estimated that “substantial progress in disarmament” would be needed to deter Swedish proliferation, by 1960 the IC had determined that “a nuclear test ban or positive steps toward disarmament—or even a reasonable hope that one or the other would soon come about—would probably be sufficient.”Footnote 47 This assessment was right on the mark. Erlander wanted to avoid tearing his party apart over the nuclear issue. Moreover, beyond domestic politics, the Social Democrats felt the international situation gave them the latitude to wait and see. As Jonter describes, “The committee of the Social Democratic Party Council took the view that the future security policy situation should determine whether or not Sweden should acquire nuclear weapons,” and that “Sweden had a breathing space at least until the mid-1960s, when international developments should provide an answer.”Footnote 48
A 1961 NIE came to similar conclusions; the 1963 NIE was vaguer in its predictions—“whether the Swedes will actually go on to produce nuclear weapons is uncertain;”Footnote 49 and the 1964 NIE returned to domestic politics as a key inhibiting factor:
Swedish capabilities for nuclear weapons production have increased in the past year, but public opposition to a weapons program probably has also increased. The view is widely held in Sweden, and has increased in strength since the signing of the partial test ban treaty, that nothing should be done to encourage proliferation of nuclear weapons, but that Sweden instead should use its influence to get the present nuclear powers to agree on further controls over these weapons.
Moreover, the ruling SDP was “on record as wishing to postpone a decision on nuclear weapons as long as possible.” For all these reasons, the NIE concluded, “the chances that the Swedish Government will decide upon a nuclear weapons program are less than even during the next decade.”Footnote 50 Two years later, the NIE highlighted similar factors—public opinion and the government's desire for international arms control—as making it likely that Sweden would “probably continue to postpone a decision for the next several years.”Footnote 51
These assessments from the mid-1960s were on point, accurately focusing on domestic opposition and the status of international arms control. A 1962 government study, for example, concluded that “international developments must determine a decision for or against in Sweden,” namely the success of arms control efforts.Footnote 52 Sweden welcomed and signed the 1963 Partial Test Ban Treaty, and a faction of Social Democrats hoped that this “would be the first of several arms control measures that would permit Sweden to reject nuclear weapons” once and for all. Nonetheless, as accurately judged by the NIEs, Swedish leaders continued to defer a final decision.Footnote 53 With respect to domestic politics, by the early 1960s public opinion had decisively turned against Swedish acquisition of nuclear weapons.Footnote 54 As Erlander later explained in his memoirs, his government ultimately decided against nuclear weapons for both international and domestic reasons. The prime minister “did not want to divide the party or destabilize the policy of non-alignment, issues of domestic politics directly related to Swedish regulatory and constitutive norms.” Moreover, Erlander wanted to arrest nuclear proliferation internationally and believed Sweden would not use nuclear weapons; therefore, it would be better off if it would “campaign instead for complete nuclear disarmament.”Footnote 55 These factors ultimately combined to lead Sweden to renounce nuclear weapons and sign the Nonproliferation Treaty in 1968. In line with the quantitative findings earlier, US assessments of Sweden significantly improved after 1958. And they improved in the way the 1958 postmortem called for: by providing more detailed information and more focus on political rather than technical dynamics.Footnote 56
Alternative Explanations
In the online supplement, I assess a variety of alternative explanations for improvement over time—including changes in technology, changes in the nonproliferation regime, and turnover in who is doing the assessments—and find that none are as persuasive as learning. I also assess the quality of reasoning in the assessments in two additional cases and find that the analysis was mostly right for the right reasons.
Conclusion
This research note has offered new evidence on US assessments of foreign nuclear intentions. The results suggest that testable US assessments were quite accurate overall and dramatically improved over the period studied. A combination of qualitative and quantitative evidence suggests that the improvement in performance was largely driven by a process of learning, whereby the IC recognized past failures in assessments, took steps to address them, and subsequently performed better.
These findings have important implications for the study of intelligence and nuclear proliferation. With respect to intelligence, the results cast doubt on the notion that assessing the intentions of foreign countries is a fool's errand and suggest that at least in this issue area, intelligence agencies are capable of learning the “right” lessons rather than reflexively maladapting and flipping to the opposite errors from before—a pathology often discussed in the intelligence literature.Footnote 57 They also should help temper the pessimism expressed by many practitioners and scholars of intelligence analysis, as in Lowenthal's claim that “No one has yet come up with any methodologies, machines, or thought processes that will appreciably raise the Intelligence Community's” accuracy.Footnote 58 Indeed, the findings support the idea that systematic reviews of intelligence forecasts may improve rather than erode the credibility of intelligence agencies.Footnote 59 They also show that learning takes place in two ways: changing organizational procedures to gather more information in key countries, and changing the analysis to give greater weight to political rather than technical factors.
The findings also deepen our understanding of the nonproliferation process. The prevalence of nonproliferation bargains early in the proliferation process has been established.Footnote 60 The findings in this paper suggest that accurate US intelligence assessments of nuclear intentions may be partly responsible for success in containing nuclear programs early, and also that this accuracy was not automatic but evolved over time. The findings also imply that US decision makers perhaps should not be so uncertain about intelligence assessments of countries’ nuclear intentions when deciding whether to go to war or implement other nonproliferation tools. While it is not perfect, the overall track record is quite impressive in the assessments under study here. The failure of the IC in the case of Iraq should not blind us to its broader track record of success.
Data Availability Statement
Replication files for this research note may be found at <https://doi.org/10.7910/DVN/BI4FJW>.
Supplementary Material
Supplementary material for this research note is available at <https://doi.org/10.1017/S0020818321000345>.
Acknowledgments
I thank Mark Bell, Alex Bollfrass, Jeffrey Friedman, Guy Laron, Julia Macdonald, Rolff Mowatt-Larsen, Jeremi Suri, Ben Valentino, Arne Westad, participants in Tobin Project workshops, and the editors and reviewers for their helpful comments and feedback.
