Public administration scholars have both scholarly and practical interests in the protection of human subjects of research by Institutional Review Boards (IRBs). As scholars, the implementation of public policy and the bureaucratic pathologies that occur are core academic questions. We should be able to offer insight about why the IRB process generates such controversy and produces what many perceive as suboptimal results. On the practical side, we also do research that can involve human subjects and we want to educate IRBs on how public administration research and much of social science research differs from the research models IRBs use to regulate the protection of human subjects.
HUMAN SUBJECTS PROTECTION: THE SCHOLARLY VIEW
The protection of human subjects is a noble goal. The Tuskegee syphilis study and perhaps the Public Health Service Guatemala syphilis study, which involved the deliberate infection of prisoners, soldiers, and mental patients, are well known (McGreal Reference McGreal2010). In health policy, catalytic events often define the problem and set into motion a bureaucratic process that shapes policy long after the event. The thalidomide scare did this for drug regulation (Quirk Reference Quirk1981); the New York Willowbrook State School Experiment on children generated informed-consent regulations (Field et al. Reference Field and Behrman2004); and the Tuskegee syphilis study created a path-dependent process in human-subjects research (Pierson Reference Pierson2000). The Tuskegee study involved medical research; the legislative hearings and the resulting law and regulations focused on preventing future Tuskegees by using a medical model for research.
Implementation problems occur because the range of research involving human subjects is vast, and the variation across these studies is as broad as the variation across intellectual inquiry. The policy model adopted for human subjects protection (HSP) consists of uniform national regulations with delegation to local units (in this case, universities and other research institutions) for implementation. The strength of this model—also used in environmental policy, unemployment insurance, and health care delivery—is that it allows for variation in implementation to deal with specific conditions. Whether this model is optimal for HSP, however, is open to question because it is unclear that research ethics do or should vary by location in the United States.
Local implementation also generates variation in local expertise or local values that can undermine the implementation of public policy (Lieberman Reference Lieberman1998). For the specific case in point, a national implementation of human subjects clearly could afford to hire social scientists and address variation across research areas. Local IRBs may or may not have a social scientist or someone who does qualitative or survey research. The local IRB expertise problem is exacerbated by chronic understaffing and high levels of personnel turnover. Our experience is that the in-take personnel are clerical employees with little training who are unable to provide any guidance to a researcher seeking to protect human subjects. This problem is exacerbated by the convoluted language of federal regulations with double negatives, multiple exceptions, and exceptions to exceptions.
The local implementation of national policy generates extensive variation in the application of research protection standards, particularly in social science areas. A study approved for expedited review at one university might be subject to full review at another. Variation is found not only across universities’ IRBs but also within a single university’s IRB process. For example, our annual Texas Middle Managers Survey has the same research focus, design, and subjects every year because the goal is to study the management process over time. In every case in which we submitted the approved IRB application from the previous year, the IRB required additional changes. The variation both within and across universities prompts the question of how local IRBs implement national guidelines.
Local implementation, of necessity, will be influenced by organizational incentives. Universities must generate revenues to survive, and research funds are highly prized. Because of the relative importance of federal funds, the ability to attract them is paramount. One of the largest sources of research money is the National Institutes of Health (NIH); social science research funding in the United States, as is well known, is essentially trivial. Universities are highly risk averse, and an adverse action involving human subjects could lead to a withdrawal of federal funds. Given the central role of the NIH, this implies that a rational university will apply a medical model in IRB regulations and use the logic of the worst-case scenario. This logic then leads directly to the over-regulation of social science research.
Universities are highly risk averse, and an adverse action involving human subjects could lead to a withdrawal of federal funds. Given the central role of the NIH, this implies that a rational university will apply a medical model in IRB regulations and use the logic of the worst-case scenario. This logic then leads directly to the over-regulation of social science research.
The effort to create a high-reliability system (LaPorte and Consolini Reference LaPorte and Consolini1991) through stringent regulation focusing on the worst-case scenario must recognize that designing perfect systems is impossible (e.g., airplanes, nuclear power plants, and space shuttles). In any system, errors are possible. Even the best-designed medical experiments can go awry (e.g., unexpected adverse drug reactions and failure of the subjects to follow the protocols) with negative effects. A rational organization facing this situation will seek to avoid individual harms and also will seek to protect its access to federal research dollars. Rational universities will respond to an adverse event by demonstrating that they followed all necessary procedures—and even went beyond the required procedures. Therefore, the adverse event was the result of bad luck rather than any malfeasance or nonfeasance of the university. In short, IRB processes are designed to create a paper trail for post-hoc justification. The result is a legal model grafted on to the medical model of research.
The result of this process is an organizational pathology: goal displacement (Blau Reference Blau1955). Goal displacement occurs when organizations lose sight of objectives (i.e., outcomes) and focus on procedures that may not be directly tied to achieving their goals. Goal displacement leads to suboptimization; that is, because IRBs maximize compliance with procedures, researchers spend more time on compliance procedures. Because human subjects’ risks vary greatly across research areas, projects that pose little risk have a higher than optimal cost–benefit ratio. Even the IRB suffers from goal displacement based on the medical–legal model because it spends more time than it should on low-risk projects; as a result, it has less time available for high-risk projects. This misallocation of resources then increases the possibility of an adverse event affecting human subjects.
Inefficiency is not the only consequence of the current IRB implementation. Research scholars recognize that goal displacement exists. They are asked to implement procedures that are not clearly linked to any possible harm to humans. (IRBs are notoriously unwilling to specify exactly what harm a given procedure is designed to eliminate.) The relationship between researchers and the IRB becomes adversarial rather than cooperative. A system that works best on trust and shared goals degenerates into a system with distrust and goal conflict. This creates incentives for scholars to downplay risks or to not disclose all information. (Hypothetically, a field experiment may not provide the exact postcard used in the experiment.) Alternatively, researchers seek to feed the paper trail that the IRB wants rather than design their research to minimize human subjects’ risk.
PRACTICAL ISSUES IN PUBLIC ADMINISTRATION
Our view of HSP implementation is greatly influenced by the practical problems it creates for a particular form of public administration scholarship. Our units of analysis are generally organizations (e.g., schools, local governments, and hospitals) and, to the extent that individuals are examined, we study elite decision making (primarily organizational management). Our research raises interesting questions that IRBs generally do not or will not recognize.
1. What Is a Human Subject?
Our surveys of school districts (Meier and Rutherford Reference Meier and Rutherford2014) are required to go through the Texas A&M IRB process. These surveys ask for public information: What is the process for selecting members to the board (e.g., election procedures), what is the racial composition of the board, the student enrollment, and the district administrative and teaching staff? The surveys are not confidential; school districts are identified so that these data can be merged with other public datasets for analysis. The question posed to our IRB was: “Who are the human subjects in this research and what risks do they incur that we should be prepared to mitigate”? The answer was the people who filled out the form are human subjects and they might be subject to retaliation if it were revealed how they filled it out. Insofar as we do not know exactly who fills out this form (it is likely a staff person designated to respond to information requests), the idea that an employee would be subject to retaliation for providing public information—information often posted on the district website—strikes us as an argument that is absurd on its face.
2. What Is a Public Official?
Public officials are specifically not covered by federal regulations for HSP [i.e., CFR § 46.101(b)3i]; clearly, this lack of coverage is consistent with the legislative history of HSP. The regulation states that research in which “the human subjects are elected or appointed public officials” are “exempt from this policy.” No one believes that the purpose of the policy is to protect the Public Health Service employees who designed the Tuskegee and Guatemala syphilis studies. As students of bureaucracy, we teach that bureaucrats are public officials subject to different laws and regulations for no other reason than that they work for the government. We could contend that, at a minimum, any appointed officials in a policy-making position (e.g., school superintendents, university presidents, and agency heads) are public officials and, therefore, research on them is not covered by federal regulation. The Texas A&M IRB rejected this position and insists that surveys of such officials are subject to review and regulation.
3. What Does Informed Consent Mean?
The issue of informed consent has many dimensions when there is risk to subjects, when the treatment is intrusive, when the subjects may or may not be capable of informed consent, and when coercion is involved. None of these cases applies to management surveys sent to school superintendents or other public-organization managers. We survey highly educated, fully capable adults (i.e., 95% of Texas school superintendents have a Master’s degree and 50% have a doctorate). We have no ability to coerce them in any way; if we inadvertently ask a question that might prove embarrassing if made public, our respondents are politically astute enough to not respond. Because researchers focus on elites, we are cognizant of the time pressures and seek to limit the time costs of participating. An informed-consent form at the beginning of a survey adds additional time cost to the respondent and likely increases refusal rates. Participating in such a survey per se should be considered informed consent.
4. Why Create a Redundant System?
In some cases, government agencies have existing HSP (primarily in terms of confidentiality) that should supersede local IRB regulation. The federal regulations clearly recognize this in exempting research on data gathered by government agencies that do not permit identification of individuals (i.e., CFR § 46.101(b)3). Agencies also provide confidential data for research purposes subject to their own regulations; the Department of Education has a licensure system for several such datasets, as do other federal agencies. The licenses come with penalty provisions and personal liability for breaches of confidentiality. Local IRBs must recognize this governmental regulation and not put additional burdens on scholars using these data.
The basic problems of IRBs and elite research result from what is a lack of understanding of the research process.
The basic problems of IRBs and elite research result from what is a lack of understanding of the research process. Elite research requires the cooperation of highly intelligent subjects. Because we build panel datasets, our relationship with our respondents is long term and cannot be jeopardized by violations of research ethics. In particular, the norm of confidentiality is absolute. Respondents will be less willing to respond to our surveys—especially on controversial policy issues—if their answers appear in the local newspaper. In many cases, this norm and the need to maintain trust have other implications. We include survey experiments in online surveys, but we have a rigid rule of no deception. As policy scholars who write frequently on contemporary issues of education, we have a reputation for credibility. Inadvertently providing misinformation could lead policy makers to act on that misinformation.
An illustration of how IRB actions conflict with this rigid protection of confidentiality is in order. Several years ago, the Project for Equity, Representation, and Governance conducted a study on Latino high school dropouts. The study began with a large quantitative study of school districts to determine how well they were performing relative to their resources and constraints. The second part of the study selected 10 districts based on regression diagnostics— five districts doing exceptionally well and five doing poorly—for in-depth study. We promised confidentiality to the districts in return for access. The Texas A&M IRB requested the district names; we refused. We are confident in our own confidentiality procedures; the ability of our IRB to protect confidentiality was unknown and its implementation problems did not inspire optimism concerning its ability to protect confidentiality. Footnote 1
A SIMPLE POLICY PROPOSAL
The current problems of HSP policy implementation have generated a policy that focuses too much time on research that is unlikely to adversely affect human subjects—time that, by implication, could be spent on projects in which human subjects are exposed to real risks. The result is that IRB processes are inefficient and likely have a greater risk of failure than most would consider acceptable. The implementation of HSP, we believe, would be improved greatly by asking two questions that focus on the underlying problem.
First, how vulnerable is the population being studied? The federal regulations clearly incorporate this dimension. Public officials clearly are not considered vulnerable and therefore are not protected under the regulation. Similarly, the regulations have specific provisions for children, pregnant women, and prisoners. Second, how intrusive is the research; that is, what are the risks that it entails? Again, a commonsense continuum with medical research at one pole and elite surveys at the other appears to cover the range. Again, this dimension is recognized in the federal regulations with its clear statements that observable public behavior and the use of existing data (if publically available or if not confidential) are not covered by HSP policy.
The current problems of HSP policy implementation have generated a policy that focuses too much time on research that is unlikely to adversely affect human subjects—time that, by implication, could be spent on projects in which human subjects are exposed to real risks.
IRBs could use this two-by-two continuum to advise researchers and perhaps have hypothetical guidelines that would provide illustrations of research that would be exempt from review, subject to expedited review, or subject to full review. An even better policy would result if such a matrix and guidelines were adopted by the US Department of Health and Human Services in the first step to eliminating the extensive local variation in HSP policy implementation.
Figure 1 illustrates a more efficient and effective IRB process. The horizontal axis represents the level of target-population vulnerability, with low-vulnerability populations shown on the left. The vertical axis represents the level of intrusion to the population that is created by the research, with the highest level of intrusion at the bottom (e.g., medical research).
IRB Human Subjects
To illustrate how the continuum between vulnerability and intrusion should trigger different IRB reviews, the following illustrations serve as examples. Research conducted by most social science involves the study of questions applied to a population of low vulnerability with little to no intrusion (see the top left corner in figure 1) and should be exempt from IRB review. The individuals studied are fully capable of understanding and voluntarily participating in the research. Additionally, the research does not create any environment of harm or risk for those subjects who want to participate.
As vulnerability or intrusion increases, so does the discretion by local IRBs when deciding whether research applications should undergo a full or expedited review. Falling into this gray area, a highly intrusive research design with a low-vulnerability population has the possibility of causing high risk or harm to a population that is fully capable of understanding the informed consent. For example, the Stanford Prison Experiment (Zimbardo Reference Zimbardo1999) used college-age males with “no prior record of criminal arrests, medical conditions, or psychological disorders” to simulate a prison environment. The lack of training and full immersion created an environment for a high level of risk and generated negative consequences.
Additionally, within the gray area of full or expedited review is research on highly vulnerable populations with low intrusiveness, illustrated by the Tennessee Class Size Project (Mosteller Reference Mosteller1995). This research studied the effects of class size on Kindergarten through third-grade student achievement. Although little to no foreseeable risk was involved, the population’s vulnerability was a cause for IRB concern. To avoid any possible adverse consequences, similar studies would undergo either expedited or full IRB review contingent on the goals and guidelines used by the local IRB.
The bottom right corner in figure 1 characterizes a research design of high intrusiveness on a highly vulnerable population that should always undergo full IRB review. For example, the three year research study conducted by New York State Psychiatric Institute affiliated with Columbia University’s Queens College and Mount Sinai School of Medicine (Hilts Reference Hilts1998) tested how low chemical levels in the brain of children between the ages of six and ten would reduce criminal or violent behavior. To complete the experiment, researchers used injections on the children. As a protected population, the children were highly vulnerable to coercion because their parents were offered monetary compensation for participation. Furthermore, the levels of intrusion present created the ability for subjects to experience harmful mental and physical consequences. To protect human subjects from harmful or long-term consequences, experiments and research similar in nature require the full IRB review process.
CONCLUSION
Protection of human subjects is an important policy objective, but the current process is characterized by extensive local variation and, in many cases, generates a process that wastes resources on low-risk research and depletes resources that could be used to focus on research with high potential risks. This article views the HSP policy through both academic and practical lenses. On the academic side, it discusses how path dependence based on extreme cases and the incentives of universities have resulted in goal displacement—the process is as much about creating a paper trail to protect the university as it is about protecting human subjects. This creates an environment that is characterized by mistrust and gaming rather than cooperation for common goals. On the practical side, we illustrate how social science research is forced onto a Procrustean bed that imposes significant costs for little or no benefit. We propose a simple two-dimensional decision matrix that would lead to a more effective and more efficient process of protecting human research subjects.
