WHAT IS THE PUBLIC VALUE OF SOCIAL SCIENCE RESEARCH?

Decision makers often face complex challenges. On topics ranging from security, health, and the economy, many decision makers in the public and private sector have incentives to seek the best available information in their quests for greater effectiveness and efficiency. By providing needed information in a timely and usable format, public investments in credible forms of data collection and analysis can be crucial to the success of a wide range of public and private decision makers.

Social scientists examine a wide range of phenomena about which policy makers and private sector decision makers care. Social science has developed a range of powerful methods for collecting and analyzing many kinds of data (see, e.g., Prewitt, Schwandt, and Straf Reference Prewitt, Schwandt and Straf2012). Social science has produced methods that many people can use to rigorously assess the plausibility and accuracy of competing explanations of critical events. When individuals and institutions around the world have questions about the validity and reliability of causal claims in social and behavioral domains, social science provides much of the knowledge base for the best available answers.Footnote ²

In many cases, however, decision makers have access to more information than they can possibly use. When decision makers can choose from multiple sources of information, the quality of their decisions can depend on which information they choose to believe and which information they choose to ignore (see, e.g., de Sola Pool and Kochen Reference De Sola Pool and Kochen1978; Lupia and McCubbins Reference Lupia and McCubbins1998; Gigerenzer, Todd, and ABC Research Group Reference Gigerenzer, Todd and Group1999). Those whose goals and ambitions involve working for a government agency, moreover, receive information from various constituents and interest groups who offer wide-ranging opinions about the consequences of certain actions and about what government should do. These opinions give our democracy great energy, but when it comes to developing and implementing effective ways to serve the public, knowing which claims to believe can be the difference between success and failure.

One attribute that distinguishes social scientific explanations from others is that social science offers an increased capacity for honesty in attempts to characterize social and behavioral phenomena. For some readers, this description may seem odd. After all, some social scientific analyses are so complex that they complicate more than they clarify. As Mark Twain (Reference Twain1906) once said, “There are three kinds of lies: lies, damned lies, and statistics.”

What gives social science its distinctive potential to expand our capacity for honesty is its norm of procedural transparency. I contend that strong answers to questions about the public value of social science can be built on information about how its claims are produced. Social science produces descriptive inferences, causal inferences, and interpretations of events that are more than just data collections. Credible social science conclusions are drawn from methods grounded in rigorous introspection about what individuals can and cannot claim to know about the world. When researchers are transparent about the procedures that they use to produce and evaluate their conclusions, they give these conclusions a meaning that others can inspect for themselves. Such processes give social scientists the ability to tell decision makers in the public and private sector things about the world that they could not have discovered on their own or are unlikely to have been able to rigorously verify on their own.Footnote ³

Social science's increased capacity for honesty comes from researchers' willingness and ability to publicize the path from a particular set of observations to a particular conclusion about the social world. Indeed, the focal expectation among many social science research traditions is that the meaning of a conclusion should not depend on who is making the claim or on irreproducible procedures. As King, Keohane, and Verba (Reference King, Keohane and Verba1994, 6–9) describe, the hallmarks of scientific research are that “the goal is inference”, “the procedures are public”, “the conclusions are uncertain”, and “the content is the method.” So, if recipients of social scientific analyses view the procedures as legitimate, then they have a basis for valuing the resulting claims.

This capacity for honesty is particularly valuable when vested interests seek to interpret important events in self-interested ways. Consider, for example, the many interpretations that follow monthly releases of economic data. The current government's supporters often claim the data as evidence of their success, while the government's opposition alleges that the data signals the government's failure (C. Wilson Reference Wilson2013). At such moments, societies benefit from being able to differentiate false stories from explanations that are consistent with transparent logic applied to the best available evidence. Social science can provide such benefits.

While social science can provide an understanding of important social phenomena that is precise and credible, it does not always do so. Social scientists are paying increasing attention to questions of research transparency (see, e.g., Lupia and Elman Reference Lupia and Elman2014). Across the social sciences, there have been many cases in which research claims cannot be replicated and, in a few cases, instances of fraud (Bartlett Reference Bartlett2012). Scholars who seek to have social science fully leverage the legitimacy that comes from being transparent about how knowledge claims are produced are pursuing a series of projects that make sharing data and information about scientific procedures easier and more appealing to scholars. The Reproducibility Project in psychology (Open Science Collaboration 2013) and Data Access and Research Transparency (DA-RT) in political science (Elman and Kapieszewski Reference Elman and Kapieszewski2014; Lupia and Alter Reference Lupia and Alter2014) are examples of broad collaborations that can help scholars document how they know what they claim to know.

Social science not only lays claim to having valuable procedures for generating knowledge claims with high truth content, it is also willing and able to lay those procedures bare. Transparency, in turn, generates self-correcting mechanisms, where replication uncovers errors, and the possibility of replication encourages rigor. This openness distinguishes science from many other sources of information about social and behavioral topics (e.g., news reports, interest group claims). Social science gives decision makers the ability to make better decisions about what kinds of claims to believe. When social science provides such opportunities in the context of important social decisions, it provides value to society by helping the public and private sectors evaluate their feelings and beliefs with respect to credible data and replicable logic.

Government support of social science provides opportunities for its citizens to convert a wide range of information about the world into actionable insights. Today many people in the private and public sector use such knowledge to improve productivity and efficiency in many socially valuable domains. In addition to informing government activity, social science helps a diverse range of nongovernmental organizations, faith-based institutions, community groups, and educational endeavors achieve their missions more effectively. These organizations use science to understand numerous complex issues. In cases where intuition or misreadings of past events can cause us to make bad decisions, social science can help us evaluate these beliefs with logic and evidence.

Consider, for example, the role of social science in what we can learn about military strategy from the war in Iraq. In 2003, troops from the United States and other nations invaded Iraq. Within a few weeks, these military forces assumed control of Baghdad and Iraq's president Saddam Hussein was removed from power. While many people hoped that the invasion would produce a relatively quick and stable transition of power in Iraq, the months and years following the invasion became increasingly characterized by insurgency and a wave of sectarian violence. This was a tragic sequence in so many ways.

In 2007, the United States embarked on a new military strategy in Baghdad. The foundation of the strategy was a “surge”—a massive increase in the number of troops on the ground in Iraq supplemented by increased training of Iraqi security forces. In 2008 and 2009, violence in Baghdad and other important areas of Iraq decreased significantly. As elected officials, members of the military, and the public attempted to understand the sequence of events in Iraq, questions lingered about the relationship between the surge and the fall in violence. Although some leaders claimed that the surge caused the decline, others pointed to a different factor. Much of the violence in Baghdad was the result of Sunnis attacking Shias and vice versa. As these attacks escalated a migration pattern commenced. Whereas before the invasion Sunnis and Shias often lived in close proximity to one another, the violence produced significant sectarian unmixing. Sunnis moved out of neighborhoods where they were outnumbered. Shias did the same. Some analysts claim that it was not the surge that reduced violence, but rather the rise of homogeneous ethnic enclaves that, by reducing sectarian contact, reduced sectarian violence.

A social science study conducted by Nils B. Weidmann and Idean Salehyan and funded in part by the Air Force Office of Scientific Research sheds important light on military strategy in response to ethnic strife (Weidmann and Salehyan Reference Weidmann and Salehyan2013). From a range of different sources, Weidmann and Salehyan accumulated detailed data on the ethnic composition of Baghdad neighborhoods, changes in settlement patterns across these neighborhoods over time, and changes in the number of violent acts in each neighborhood over time. Weidmann and Salehyan used a computational model to examine correlations between the timing and nature of violent acts and the movement of various populations within and across Baghdad. This evaluation led them to identify violence-migration correspondences that were consistent with the best available data. With such results in hand, Weidmann and Salehyan simulated the effects of various policing strategies. These simulations allowed them to produce and evaluate estimates of whether and how a surge could be effective.

Their findings shed important light on Baghdad before and after the surge. Their data showed that most attacks were against nearby sectarian rivals. If a person was among a sectarian minority within neighborhoods where attacks were occurring, they searched for safety by moving to areas where their sect was in the majority. Examined many different ways, their analysis makes clear that violence-induced sectarian migration played a significant role in reducing Baghdad violence.

Their findings also provided an important insight about the surge. Timing matters. Their work supports the hypothesis that if a surge is going to be pursued, it is much more effective when it is done in the early stages of a conflict. The intuition is that if a surge is delayed, then individuals and groups that are prone to violence will accumulate a rationale for violent action (revenge for attacks against their members) and will have made efforts to organize their resources for increased effectiveness at violence. When groups have such rationales and resources, the troops have to work much harder to reduce the level of violence.

Findings such as those of Weidmann and Salehyan, while deeply rooted in research, are not simply academic matters. Military actions entail considerable risk for the servicemen and women who are sent to implement a strategy. When the sacrifices of our servicemen and women are at stake, it is not enough to rely on intuition or wishful thinking about the effectiveness of the mission on which they are to embark. At these moments, these individuals and our nation benefits from distinguishing false stories about cause-and-effect from explanations that are consistent with precise logic and the best available evidence. This is the type of activity that social science is exceptional at identifying and supporting.

Social science is also proving useful in informing the public and private sectors about the most efficient and effective ways to improve public health. One such example, in which I was involved, shows how even a small amount of funding can have a large effect. In 2001, NSF funded an interdisciplinary infrastructure project titled Time-Shared Experiments for the Social Sciences (TESS http://www.tessexperiments.org/). Diana Mutz and I were the founding principal investigators.

TESS invited researchers to run innovative experiments on large national respondent samples. At that time most social science experiments were conducted on undergraduates or residents of college towns. TESS gave investigators an opportunity to evaluate social scientific research hypotheses on a much more diverse group of people (Loftis and Lupia Reference Loftis and Lupia2008).

In the project's early days, I was approached by a young doctor who wanted to know whether these methods could be used to help physicians across the country better understand and more effectively serve their patients. We discussed a variety of options and soon he sent a proposal to TESS. The proposal went through TESS' review process and within months we were fielding the study. The question pertained to the best way to make newly approved vaccines available to patients (Davis and Fant Reference Davis and Fant2005).

At the time of the study, employer-sponsored health plans were the dominant source of health insurance for children and adults. A typical plan was the result of a negotiation between an employer and a health insurance provider. The employer and the provider would agree on a contract that set rates and benefits for a fixed period of time—often several years.

During the contractual period, the Centers for Disease Control and Prevention would regularly identify new vaccines with public health benefits. These new vaccines were rarely named in the existing contracts. As a result, most Americans would have to wait several years before their insurance would cover the cost of newly approved vaccinations.

The TESS study by Matthew M. Davis and Kathryn Fant examined the public health consequences of this situation. Their study compared citizens' willingness to pay out of pocket for new vaccines to an alternate method of providing the benefit. In this alternate method, citizens were offered what economists call an “options contract.” David and Fant asked people if they would be willing to pay a small additional amount per month ($3–$6) for immediate access to newly approved vaccines.

The study revealed important information. While fewer than half of the participants were willing to pay out of pocket for new vaccines, more than three-quarters of respondents would pay for the options contract. This finding is noteworthy not just for the public health benefits arising from increasing vaccination rates, but also because of the actual costs to insurance companies of providing such coverage. The additional premium of $3–$6 per month that was offered to participants in the study was an intentional overestimate of the amount that it would cost an insurance provider to offer the option. In fact, the best available estimate of the maximum impact of a new vaccine on a monthly premium was 1/12 the cost of the options contract that three-quarters of the participants accepted.Footnote ⁴ In other words, the study shows that many people would be willing to pay health insurance providers $3–$6 per month for a service that would significantly improve public health and cost the providers pennies per month to provide.

This study, while deeply rooted in research, is not simply an academic matter. Finding ways to provide health care in efficient and cost-effective ways is critical to the health of many companies and, many would argue, to the nation as a whole. When the health of our citizens is at stake, it is not enough to rely on intuition or wishful thinking about the effectiveness of various health provision strategies. At these moments, these individuals and our nation benefits from distinguishing false stories about cause-and-effect from explanations that are consistent with precise logic and the best available evidence. NSF has been exceptional at identifying and supporting this type of activity. Moreover, the doctor who ran this TESS study continues to integrate social scientific methods and public health issues in his work. He now leads the National Poll on Children's Health (http://mottnpch.org/) that is helping many doctors and patients better understand one another as they work together to improve childrens' health outcomes.

SHOULD CONGRESS FUND SOCIAL SCIENCE RESEARCH?

In this final section, I want to address a topic that I have been asked about many times in recent months. As you may know, questions have been raised about how and whether Congress should fund social science research. In recent months, I have been told that a bill had been drafted that would eliminate the entire social and behavioral sciences directorate at NSF. At the same time, the State Department, the Department of Homeland Security, the Department of Defense, and the Department of Energy, to name a few, are all seeking social science expertise to help them accomplish their missions. So there are clearly different points of view about the value of social science research to American taxpayers. With such differences in mind, let us address this question: What is the role of Congress in funding social science research? While the answer that I will give to this question can be applied to other scientific activity, I will focus on social science because that is where the recent debate is focused.

A productive way to think about the question is to first remind ourselves about the role of Congress and then to think about whether and how funding social science research helps Congress do its job. Congress' job, as stated in the Constitution is to “establish Justice, insure domestic Tranquility, provide for the common defence, promote the general Welfare, and secure the Blessings of Liberty to ourselves and our Posterity.” To understand the role of Congress in funding social science research, we must understand when and how does scientific research help Congress achieve the Constitution's goals.

Science plays an important role in American life. Scientific discoveries help individuals, firms, and governments improve the lives of people at home and abroad. Science makes our factories more efficient, our farms more bountiful, our military stronger, and our doctors better able to save lives.

For many decades, America has been the global leader in science. America's science leadership did not happen by accident. America's science leadership owes a huge debt to congressional foresight. In particular, Congress' creation of the NSF presented America with widely admired abilities to produce research in the public interest. Through rigorous peer-review processes, NSF brings wisdom to complex topics by supporting research that evaluates competing scientific claims. In many cases, NSF chooses which proposals to fund through highly competitive processes. In each competition, it puts forward a question of national interest and asks researchers to bring the best available scientific methods to bear on them. NSF does this because it knows that knowledge often advances faster when scientists are free to rigorously evaluate competing ideas. Freedom and competition provides the foundation from which scientific innovation and leadership emerge.

As the previous sections show, social scientific research has provided great value to the nation. But should Congress pay for it? Let me propose that Congress is not obligated to spend a single cent on scientific research. Its obligation is to the American people under the framework of the Constitution. So the question we have to ask is whether Congress choosing to fund social science research is essential to its obligation to the American people.

It is possible, for example, that social science is valuable, but that Congress need not fund it. This proposition merits serious consideration. We have to recognize that a term like “political science” sounds odd. To the general public, much of government and policy seems anything but scientific. Many of us, however, have seen that using science to understand critical elements of how modern societies organize ourselves to meet important challenges, and why we sometimes fail to do this, is serious business. Ample evidence suggests that supporting social science is critical to any modern society that wants to become or remain effective and great.

However, some people claim that you do not need science to understand government and policy because people can explain these things without science. That's partially right. Americans explain politics, policy, and government in many different ways. However, people who tell such stories often emphasize the world as they want to see it and are less interested in objective evaluations of whether their stories are true. When people's lives and livelihoods are at stake, our nation benefits from distinguishing false stories from explanations that are consistent with precise logic and the best available evidence.

Other people claim that the private sector will fund basic research on science and government. It is true that many firms use the social sciences to inform and advance their operations. But most firms have very specific interests and lack a mandate to support broadly beneficial science. Economists refer to such situations as “free rider problems.” Free rider problems explain why advanced societies use government (rather than waiting around for voluntary contributions) to fund justice systems, interstate highways, and soldiers. As a public good, basic social science is unlikely to be provided in sufficient quantity by the private sector. So, when it comes to promoting science of broad public value, leading nations have chosen to combat free rider problems by empowering national science foundations—and, for decades, our nation has led the way in this regard. But should Congress continue such funding?

Given that social science can be expensive to produce, we should expect those who pay for it to ask questions about the return on their investments. To those questions, we must be able to explain the many ways in which social science provides insights that are useful for governance. It helps communities, cities, states, and nations better serve their citizens, and it helps us advance multiple national priorities, like homeland security and job creation. Social science does all of this by increasing our ability to operate from knowledge, rather than being captive to untested stories, when designing and evaluating government programs. This is why social science plays an increasingly critical role in how America “establishes Justice, insures domestic Tranquility, provides for the common defence, promotes the general Welfare, and secures the Blessings of Liberty to ourselves and our Posterity.”

At the same time, we should prepare for continuing controversy. Social scientists often research topics about which others have strong opinions. This is particularly true of social scientists who study politics and policy (Lupia Reference Lupia2000)—as many people in these domains do not like to be told that data and evidence support other ways of viewing topics about which they are passionate. In the face of such opinions, social scientists can expect questions about whether science describes social phenomena better than media commentators, interest groups, and politicians. While societies often benefit from the exchange of views that come from these and other sources, societies also benefit by basing policy decisions on the best available logic and evidence. The ability to offer such logic and evidence in a credible, legitimate, and rigorous manner is what social science is built to do. Honest, empirically informed, and technically precise analyses of the past provide the strongest foundation for knowledge and can significantly clarify the future implications of current actions. For this reason, supporting social science is critical to any nation that seeks to achieve its greatest aspirations with ever-increasing effectiveness and efficiency.

ACKNOWLEDGMENTS

This essay is based on the 2013 Ithiel de Sola Pool Lecture, which was delivered at the APSA Annual Meeting (Chicago, Illinois. August 30, 2013). I thank Henry E. Brady and Samuel L. Popkin for graciously introducing me at the lecture. I thank John H. Aldrich, Cheryl Boudreau, John G. Bullock, James N. Druckman, Colin Elman, Adam Seth Levine, Spencer Piston, and Gisela Sin for insightful comments on a previous draft of this essay.