1 Introduction
The use of analysis and the use of public participation are two of the most important tools used in bureaucratic decision-making. Both have been touted as solutions to the problem of accountability among unelected agency officials. Both are also supposed to make policy decisions “better.” Finally, both (1) the systematic analysis of government agency decisions prior to their issuance (“analysis”), and (2) the requirement that these decisions incorporate the input of the communities affected by them (“participation”) have been accused of not living up to their potential (Shapiro, Reference Shapiro2016).
Public participation in government decision-making has long been a goal for scholars of public administration attempting to improve the accountability of decisions (Davis, Reference Davis1969). Myriad works have been written on how we can improve the responsiveness and accountability of government decisions, particularly those made by unelected officials in the bureaucracy. In the regulatory arena, public comment, electronic rulemaking, negotiated rulemaking, and other techniques have been tried with at best mixed success. Regulatory decisions are still too often derided as ignoring the concerns voiced by the public.Footnote 1
Meanwhile, analytical requirements have also been increased on regulatory agency decisions. While these requirements are often implemented with the stated goal of rationalizing regulatory decisions, their advocates have long touted the transparency and participation benefits of good analysis (Sunstein, Reference Sunstein2002; Greenberg, Reference Greenberg2013). But whether analysis actually facilitates participation is often ignored. This article attempts to begin to fill this gap by looking at participation in the context of analysis. Does analysis encourage public participation or stifle it? How can analytical requirements be designed in order to better involve the public?
I spoke with 48 analysts spread across three specialties (benefit-cost analysis, environmental impact assessment, and risk assessment). They shared with me both their perceptions on how their analysis affected the participation of the public and how it was shaped by this participation. After learning from these analysts the importance of conducting analysis at an early stage of the decision-making process and the challenges of making analysis accessible to outside parties, I conducted a case study of a requirement that is intended to encourage both early and simpler analysis. This requirement is the statutorily required use of panels of small business representatives to provide input early on certain regulatory decisions by the Occupational Safety and Health Administration (OSHA) and other agencies.
In order for analysis to reach its potential, academics and practitioners must pay greater attention to the conditions under which analysis can facilitate participation. If transparency is a goal of analysis, analytical requirements should be designed with participation in mind and should be structured so as to best take advantage of the public input. Hopefully this article will further this goal articulated in several NAS studies (NRC, 1996, 2008).
This article proceeds as follows. I briefly review the literature on analysis, participation, and the interaction between them in the next section. Following that, I describe my interview methodology. The results of the interviews are discussed in the subsequent section followed by a description of the small business panel case study. I conclude with observations about the interaction between analysis and participation and a discussion of the need for more attention to this interaction. In particular, the use of panels of affected individuals to comment on analysis, movement toward simpler clearer analysis, and the placement of analysis earlier in the decision-making process may all lead to more useful public input on policy decisions.
2 Analysis and participation
There are many requirements for agencies to engage in thorough analysis of their decisions and to solicit some degree of public participation in these decisions. Both of these tools have been the subject of vast literatures to which I cannot do justice here. To best frame the empirical work that follows, I very briefly review the concerns that participation in regulatory policy has been one-sided or insufficient. Then I discuss the literatures on the three specific forms of “comprehensive-rational” analysis that this paper examines (benefit-cost analysis, risk assessment, and environmental impact assessment), particularly as those literatures consider issues of participation and transparency.
2.1 Participation in regulatory policy
Notice and comment rulemaking requires agencies to publish (with some exceptions)Footnote 2 a proposed rule in the Federal Register. The proposed rule describes the regulation that the agency is contemplating, and gives a detailed rationale for the proposal. The agency also gives the public a chance to comment on the rule (usually 30, 60, or 90 days). Upon receiving public comments, the agency must consider the comments, and either change the regulation or explain its rationale for not adopting the suggestions of commenters. The agency issues a final rule upon completion of this process.
Agency regulations can be overturned by courts, according to the APA, for being “arbitrary or capricious.” Over the years courts have determined that agencies must (among other things) give reasoned responses to public comments in order to not be considered arbitrary or capricious. The notice and comment process is intended to imbue the promulgation of executive branch agency regulation with a sense of democratic accountability (Davis, Reference Davis1969). Regulations are a form of policy-making that takes place outside the normal U.S. structure specified in the Constitution, which involves Congress passing a law and the President signing it. As such, there has long been concern about whether the agencies that issue regulations to set public policy are responsive to public preferences. By forcing agencies to solicit public comment and then respond to it, the hope was that rulemaking, would like lawmaking, reflect to some degree the popular will (Davis, Reference Davis1969). Mission driven regulatory agencies (Wilson, Reference Wilson1989) however have incentives to pursue their preferred policies regardless of public comment.
How well has the notice and comment process worked? Academic studies have produced mixed results. Several scholars have concluded that agencies pay little attention to public comment and it has been derided as “Kabuki theater” (Elliott (Reference Elliott1992); see also Golden (Reference Golden1998), and Shapiro (Reference Shapiro2007)). Others have found that agencies do respond to public comments. In particular, Yackee (Reference Yackee2006), who has conducted the most sophisticated studies of the public comment process, concluded that, “interest group comments can and often do affect the content of final government regulations.” In other work she concluded that business interests were the groups most likely to have their concerns addressed by regulatory agencies (Yackee & Yackee, Reference Yackee and Yackee2006). This finding has been echoed by others (Wagner, Barnes & Peters, Reference Wagner, Barnes and Peters2011; Wagner, Reference Wagner2010).
The debated influence of participation on rulemaking, and the generally supported idea that participation favors powerful interests has cast doubt on the existing process for securing input on regulatory decisions. Advocates for various forms of analysis have long touted the potential of analysis to improve public participation in decisions made by the unelected bureaucracy. In this article, I focus on three forms of “comprehensive-rational analysis,” a term coined by Lindblom (Reference Lindblom1959). Comprehensive-rational analysis embodies the idea that all aspects of a policy decision can be analyzed and in doing so, the optimal decision will be reached.
Comprehensive-rational analysis has its roots in the Progressive Era and the idea that policy problems had optimal solutions. It flowered in the 1960s and 1970s in the forms of environmental impact assessment, benefit-cost analysis (BCA), and risk assessment. Each of these three forms of analysis have been continually used in regulatory decision-making for over 35 years (Shapiro, Reference Shapiro2016). And as I detail below, broad claims have been made about the potential for each of these forms of analysis (and for policy analysis more broadly (Fisher, Reference Fisher, Hajer and Wagenaar2003) and program evaluation (Cousins & Whitmore, Reference Cousins and Whitmore1998)) to improve public participation in agency decisions by laying out all the effects of those decisions.
The National Academy of Sciences (NAS) has been particularly forceful in its advocacy for the potential of participation and analysis to work together to improve policy decisions on complex questions. In Understanding Risk (NRC, 1996), they introduce the idea of the “analytical-deliberative” process saying, “Analysis and deliberation can be thought of as two complementary approaches to gaining knowledge …deliberation is important at each step of the process that informs risk decisions such as deciding which harms to analyze, and how to describe scientific uncertainty and disagreement,” (p. 5). They expand on this idea for environmental decision-making in a 2008 report noting, “environmental decisions present very complex choices among interests and values so that the choices are political, social, cultural, and economic at least as much as they are scientific and technical” (NRC, 2008, pp. 7–8).
The NAS acknowledges that in practice, creating an analytic-deliberative process will be challenging. They note in particular that BCA and risk assessment make “value judgments opaque and inaccessible for political debate” (NRC, 2008, p. 143). The success of an analytical-deliberative process depends on whether participants absorb and understand the complexities that are often imbued in the analysis (Fischhoff, Reference Fischhoff2015). Indeed analysis is inherently technocratic and therefore exclusive (Rayner, Reference Rayner2003). How have three of the most technocratic types of policy analysis stimulated and interacted with participation and deliberation? Has an analytic-deliberative process manifested itself in the use of any of these types of analysis?
2.2 Cost benefit analysis and participation
The form of analysis with the longest and most analyzed history in the regulatory process is BCA. Executive branch agencies are required to conduct a Regulatory Impact Analysis (RIA) when issuing a regulation that has an impact of more than $100 million in any given year.Footnote 3 As part of this RIA, the agency must calculate the benefits and costs of their proposed action. This requirement has been in place since the Reagan Administration and has been endorsed by all six presidents from Reagan to Trump. The literature on BCA is considerable and I focus here only on the literature that discusses the impact of analysis on participation in regulatory decisions.
Benefit-cost analysis has long been touted as promoting transparency, and thereby abetting participation by outside parties in regulatory decision-making (Sunstein, Reference Sunstein2002). The theory is that by making clear the implications of an agency’s regulatory actions, an analysis published as part of a proposed rule, can help the public comment on the regulation in a more informed way. It has long been the hope from advocates of BCA that analysis could be used as a lever to open up the black box of bureaucratic decision-making.
It has not always worked out that way. Over time, the RIAs which contain benefit-cost analyses have become increasingly dense, growing from 31,000 words to 128,000 words between 2000 and 2012 (Carrigan & Shapiro, Reference Carrigan and Shapiro2016). Sinden (Reference Sinden2014) describes this as a movement toward “false formality” in BCA. She argues that agencies have increased the density of benefit-cost analyses while decreasing the ability to use the analysis to aid in decision-making either within the agency or outside it. In this view, the requirement for agencies to conduct BCA has made regulatory decision-making less transparent rather than more so. Keohane (Reference Keohane2009) recommends reframing cost benefit results in simpler terms that highlight the tradeoffs to individuals.
The trend toward increased complexity of BCA may have to do with the incentives faced by regulatory agencies. Agencies have often been criticized for conducting their RIAs after they make their decision on preferred policies (Carrigan & Shapiro, Reference Carrigan and Shapiro2016). If this is indeed the case, we can infer that the analysis must be used to justify the decision before an eventual reviewing court rather than to inform the public or their elected representatives (or facilitate their participation).Footnote 4 An analysis that is transparent and clear is more likely to raise questions about the agency’s preferred policy choice, and therefore agencies have an incentive to produce a document that is hard for outside readers to penetrate.
To date much of the treatment of the interaction between BCA and participation has had a “if you build it, they will come,” quality. Agencies are instructed to do an RIA and to put it out for public comment. This is in contrast to the analytical-deliberative process described by NAS. Agencies have reacted to this requirement, and to the legal climate in which they operate by producing analyses that are often inscrutable to anyone besides those with a great deal of expertise and resources. For these reasons the potential for BCA to increase transparency cited in the literature (Sunstein, Reference Sunstein2002) does not appear to have been realized.
2.3 Risk assessment and participation
Risk assessments often take place earlier in the regulatory process than benefit-cost analyses (particularly if their results are a necessary input for the measurement of benefits). They are often used in priority setting for agencies rather than in the choice of a solution to a policy problem. Risk assessment has its origins in concerns about exposure to nuclear radiation and statutory requirements on food safety. The Environmental Protection Agency (EPA) and OSHA, prompted in part by executive order, and in part by the Supreme Court,Footnote 5 began to institutionalize risk assessment in the regulatory process. The practice of risk assessment evolved rapidly in the 1970s. The NAS (National Research Council, 1983) defined risk assessment as “the characterization of the potential adverse health effects of human exposure to environmental hazards.”
In various reports commissioned by Congress and EPA over the years, the NAS has repeatedly called for increased transparency in the risk assessment process. The NAS reports trumpet the potential for a mutually beneficial interaction between participation and risk assessment. Indeed the analytical-deliberative process described above was developed in NAS work on improving risk assessment (NRC, 1996). However, their 2009 report also acknowledges that the potential has not been reached. “In recent years, a number of federal agencies have raised concerns about EPA risk assessments of contaminants …Those agencies and other public and private stakeholders often assert that they are inadequately involved in EPA processes” (NRC, 2009, p. 17). NAS does acknowledge that the “inherent complexity of many of the questions addressed by risk assessment make transparency a challenge” (NRC, 2009, p. 66).
Risk assessments, particularly at EPA, have long been criticized for taking a long time, sometimes decades. There is an inherent tension between the steps to address timeliness and transparency. Increasing participation slows down assessments. More participation often means taking more time to reach a policy decision (De Marchi, Reference De Marchi2003). And participation in the evaluation of a complex document may be limited even if the intentions of the agency producing the document are to spur transparency. There are only so many outside parties with the expertise necessary to understand the document and make contributions to the policy decision it analyzes (Shapiro, Reference Shapiro2016). The amount of time that a risk assessment takes also depends on the acceptable level of uncertainty, the more precise estimates that are required, the longer the analysis will likely take.
There seems to be widespread agreement that more participation in the creation and critique of risk assessments would lead to better outcomes. But aside from one NAS proposal for EPA to fund the participation of under-represented groups (NRC, 2014), most of the recommendations for increasing participation (NRC, 1996) has produced suggestions that are more general than specific. As a result, the same criticisms are voiced repeatedly.
2.4 Environmental impact assessment and participation
Environmental impact analysis has been a part of the federal landscape since 1970. Congress passed the National Environmental Policy Act (NEPA) in January of 1970. Coming amidst a wave of environmental statutes, the legislation was intended to upgrade the minimal environmental emphasis in federal decisions ranging from timber leasing, to dam building, to nuclear site approval to regulation (Clark & Canter, Reference Clark and Canter1997). NEPA required the production of environmental impact statements (EISs) for government decisions (including regulatory decisions) that have a significant impact on the environment. For decisions that have a less significant impact, agencies are still required to produce an “environmental assessment (EA).”
The role of participation is far more fundamental and ingrained with environmental impact assessment than it is with risk assessment and BCA. The basis for this emphasis though is the same, the provision of analysis to affected communities will help them understand the implications of these decisions. Unlike BCA and risk assessment, in the case of an EIS, the need for public input is written into the implementing regulations. Agencies are required to get public input on their EIS and academics have seized on the potential of this process for improving policy decisions (Greenberg, Reference Greenberg2013).
While some (Greenberg, Reference Greenberg2013) have noted successful examples of participation spurred by EIS, others have been less confident that the hope of spurring participation has been realized (Walker, Reference Walker2014). These observers have noted that public comment is a blunt tool for participation, and differences in risk perception between the public and experts may make meaningful individual participation impossible (Eckerd, Reference Eckerd2014). Others have noted that key questions regarding the use of participation (including the meaning of participation itself) in the EIS context remain unresolved (Glucker, Driessen, Kolhoff & Runhaar, Reference Glucker, Driessen, Kolhoff and Runhaar2013). The problem of complexity has plagued EIS as well. The EISs are excessively detailed and this detail conveys the false impression of precision, while obscuring the ability of outsiders to evaluate the accuracy of the analysis (Karkkainen, Reference Karkkainen2002).
A brief summary of the three types of comprehensive-rational analysis is in Table 1.
Three forms of analysis.
The scholarly community and NAS seem certain that analysis has the potential to increase participation in policy-making. But the experience with BCA, risk assessment, and environmental impact assessment makes clear that this potential is often not realized. Analysis seeks primarily to rationalize decision-making processes. Participation seeks responsiveness and public accountability. While these may work together there are many times when they may also come into conflict.
The literature above that touts the virtues of comprehensive-rational analysis largely predicts that analysis will facilitate participation. The NAS reports are less naïve in that they outline the conscious need to design analysis in a way that will reap the benefits of meaningful deliberation. The empirical work cited above is much more cautionary. The experience with notice and comment rulemaking broadly and the few studies of how BCA, risk assessment, and environmental impact assessment impact participation, have shown that encouraging participation involves more than simply creating a document and hoping affected parties will react. Indeed the comprehensive-rational nature of these forms of analysis may in fact deter participation and the steps necessary to create an analytical-deliberative process are rarely taken.
By gaining a greater understanding of the nature of participation in analysis in practice, we can learn whether and when the forms of comprehensive-rational analysis encourage public participation. With the qualitative approach to understanding how analysis and participation currently interact, I cannot provide definitive answers to the questions of how to design analytical processes to facilitate participation but I hope that shedding light on the ways that analysis facilitates and frustrates participation in practice will improve the use of both analysis and participation in policy-making.
3 Methodology
This work is part of a larger project on the role of analysis in decision-making. As previously noted, this paper considers the results of two attempts to improve this understanding. The first is a series of interviews with agency economists who conducted BCA, agency risk assessors, and environmental impact assessors both for the government and for government contractors. Collectively my interviewees conducted or reviewed thousands of analyses. I spoke to a total of 48 individuals all with at least 5 years of experience in the field (and most with more than 20) and many of whom worked at high levels at federal agencies, supervising other analysts in their field after a career of doing analyses themselves.
I initially identified the economists and risk assessors through personal networking. I identified the environmental impact analysts via a posting in a “LinkedIn” group for environmental impact assessment. With both types of analysts, I then identified further interview subjects through snowball sampling. The interview subjects were promised confidentiality – hence, interviewees’ names do not appear in this article. Such guarantees of confidentiality are standard in good qualitative research in order to ensure that interview subjects are comfortable speaking freely (Rubin & Rubin, Reference Rubin and Rubin2011). They are particularly important in this context, given that subjects were asked about government decisions made during their tenure with which they may have disagreed.
Interviews lasted between 30 and 60 minutes. Each interview had a standard Institutional Review Board approved interview protocol but follow-up questions were asked as appropriate. The interview protocol is attached in Appendix 1. I transcribed interviews and grouped themes were open coding.
The economists had spent time at eleven different agencies.Footnote 6 As a group they had conducted or reviewed more than 700 economic analyses of regulations. Because of the emphasis in the risk assessment literature on the EPA, ten of my interview subjects for risk assessment were current or former employees of this agency. However, I did not want to ignore risk assessment in other regulatory agencies, so I also spoke with risk assessors in the Departments of Agriculture, Health and Human Services, Homeland Security, and Labor. Collectively my interview subjects have worked on more than 1000 risk assessments. The environmental impact assessors had also worked on or reviewed more than 1000 EISs or EAs. They had also worked at a wide variety of agencies.Footnote 7 I spoke with sixteen individuals in each category of analysis for a total of 48 interviews.
Among the questions asked of interview subjects were requests for examples of where analysis made a difference in a policy decision and when it did not.Footnote 8 It quickly became clear that one of the factors that varied between these two categories was the extent to which the analysis spurred outside participation. I added follow-up questions to my interviews to better understand the interaction between the work of the analysts and the input of outside individuals or groups.
In my description of the interviews below I highlight several cases of the public reacting to an analysis. These examples are both positive and negative. I make no claim that the examples cited below are representative of all analyses. Rather they are illustrative of the beneficial effects that analysis can have in making decisions more participatory …and of the ways that analysis can stifle participation.
The second part of this project was sitting in on a “Small Business Regulatory Enforcement Fairness Act (SBREFA) panel” conducted by OSHA. After conducting the interviews and understanding that among the key variables that affected the ability of analysis to spur participation were clarity, simplicity, and timing, I decided to do a case study of the SBREFA process. I had known about the SBREFA process from other work and felt that it had a number of the characteristics that my interviews were revealing as helpful to improve participation in regulatory decisions.
Notes were taken during the phone calls but there was no transcription. Summaries of the calls also appear in the SBREFA documentation issued by OSHA. Participants were promised confidentiality during the calls by OSHA. The calls proceeded quite similarly however indicating that the main themes were not dependent on the participants on the line. The SBREFA case may or may not be representative of these small business panels generallyFootnote 9 but it should be read as an illustration of how analysis can help with participation at least among a particular affected community.
4 Results
4.1 Interviews
4.1.1 Analysis spurring participation
The clearest examples of the act of analysis abetting participation in government decision-making came from the world of environmental impact assessments. Whether it is because of the long history of environmental impact assessments (first required in the early 1970s), the fact that EISs are published as freestanding documents for public comment (unlike benefit-cost analyses or many risk assessments), or because participation is required by the implementing regulations for EIAs, practitioners readily produced examples of policy decisions being affected by participation that was motivated by agency analysis. They described instances of local communities engaged in the EIS process and providing valuable information to agencies.
A representative of the Forest Service said, “The Forest Service invested quite a bit in collaboration since the mid late 1990s. That is the way we promote doing our big projects, through collaborative processes. With the internet and our ability to post all of our documents and citizens ability to read and engage this has been really enhanced as well.” An interview subject from NOAA said, “We’ve had a lot of public involvement from the get go. Even in the EA process where we are not required to have public comment,Footnote 10 we still provide that opportunity so the people in that community have some say in how we do restoring habitats.”
Perhaps the best example from my interviews of an EIA spurring participation which in turn affected a policy decision, is a case from NOAA on the importation of beluga whales from Russia. NOAA conducted an EA under NEPA for the decision to grant a permit to the Georgia Aquarium (NOAA, 2012) to import the whales. The agency published the EA for public comment in August 2012 and held a public hearing on October 12, 2012. The EA was a mere 19 pages and considered only two alternatives; granting the permit and refusing it.
The public response was overwhelming. NOAA received more than 9000 comments which were overwhelmingly against the approval of the permit. Many of the letters were from individuals and provided little additional information. However, major environmental organizations also commented. One letter from 64 nongovernmental organizations cited the tremendous stress that would be placed on the whales, and also more broadly discussed the long-term impacts of the decision, “We are concerned that any international trade in these animals, including the proposed import of belugas by the Georgia Aquarium, will increase demand by the public display industry, with a resultant impact on wild populations targeted by live capture operations” (Animals Asia Foundation et al., 2012). As a result of the outcry, NOAA reversed itself and denied the permit. This is an example of analysis and participation interacting in the way envisioned by designers of this requirement and academics who have touted the mutual benefits of these tools.
Environmental impact assessors also had other examples of participation affecting their work. They readily produced examples of projects changed as a result of the participation following their analyses. One interview subject said, “One project there were a lot of different alternatives; we put out an EIS without identifying a particular alternative because we wanted community groups to provide input and pros and cons and we did take that into account. That helped a lot in making the determination.” From a former project manager in the military,
I have always found it beneficial to learn what the critical issues were upfront and then do everything you can to avoid impacts. One of the first NEPA projects I worked on, neighbors told us there would be a historical cemetery site where we were planning on building. We found it and avoided impacts by fencing it off.
And a third subject told the following story
Due to ongoing work with the affected tribes, we eventually modified the flight plans ultimately selected to coexist with the traditional ceremonies and locations. By working with Tribal elders and their council we determined that particular sites were of significance only at certain times of year and day. We worked out an arrangement with the Tribe to ensure a “no fly” area of several miles during the specified times/days provided to us.
The policy changes are not huge but they are arguably the types of changes that some advocates of EIS were hoping for when designing participation requirements into the EIS regulations.
From the world of BCA comes an encouraging example from the Transportation Security Administration (TSA). Congress required the TSA to ensure the security of aircraft repair stations. TSA issued a proposed regulation requiring these stations to, “adopt and carry out a standard security program.” The bulk of the regulation specified the components required in such a security program (TSA, 2009). In its regulatory impact analysis, the TSA clearly and concisely went through the individual security program requirements, assumed the costs of those requirements and multiplied the sum by the number of stations affected. The range of ten-year costs was calculated at US$241–296 million. They also estimated that if the regulation prevented one “minimal attack” every 1.1 years, one “target attack” every 32.1 years, or a “severe attack” once every 92.7 years, then the benefits of the regulation would exceed the costs.
The TSA received 177 comments on the proposed rule. Many of the commenters complained that the costs of the rule were high, and that the likelihood of achieving the benefits was remote. Numerous commenters accused the TSA of not recognizing the diverse nature of aircraft repair stations and the resultant diverse security risks associated with them. Owners of smaller stations in particular were concerned that they would have to shut their doors if forced to develop a security program as envisioned by the TSA. The final regulation was very different than the proposed rule (TSA, 2014) as was the accompanying analysis. The TSA abandoned the idea of requiring all stations to adopt a security program. The highlighting of the costs associated with the proposed rule gave small business owners a clear picture of the burdens that the regulation would impose. Would the TSA have modified the proposed rule, without a BCA? The answer is uncertain but the individuals that I spoke with pointed to the comments on the analysis as a significant factor in driving the TSA toward a much more lenient final regulation.
These examples all had analyses that were relatively simple to understand. While a mere 19 page environmental impact assessment may seem insufficient, it was enough to highlight to interested parties that their input was necessary. The RIA for the TSA regulation was also written in plain English and was nothing like the dense several hundred page documents that are often typical for regulatory agencies (Carrigan & Shapiro, Reference Carrigan and Shapiro2016). Examples of simplicity in risk assessment were rarer. Many of my interview subjects expressed a desire for public input but did not have examples of risk assessment spurring participation for anyone beyond a few well-funded scientifically expert parties.
4.2 Analysis deterring participation
In the world of risk assessment, analysts clearly see one of their purposes as increasing transparency and participation. A risk assessor I interviewed emphasized the importance of transparency, “That’s why the Council on Environmental Quality called for risk analysis to have more of a link with health education and policy-making. To engender a galvanized will.” In the case of risk assessment, practitioners largely mirrored the scholars who praise participation. Everyone loved the idea of making their analyses more transparent and getting public input on them but there were few examples of this actually happening. The motivations of those performing the analysis do not seem to be at fault when participation is lacking.
The complexity of the analyses is a primary culprit in limiting participation to those with considerable pre-existing expertise in the subject matter. A NAS study of the risk assessments conducted under the Integrated Risk Information System (IRIS)Footnote 11 described this problem. The NAS found that the EPA had increased transparency but that in response, participation in IRIS reviews was uneven. Industry participated much more often than public interest groups and was able to contribute much more to the scientific debate underlying risk assessments. The NAS suggested that the EPA should perhaps provide technical assistance to underfunded groups in order to facilitate their more effective participation and increase their attendance at professional meetings (NRC, 2014).
Industry participation in IRIS decisions has led to a process that stretches out over decades. As one EPA employee said, “They are typically chemicals where the stakes are high. They’ve been out for a long time or there are big legacy contamination issues and applying the IRIS value would imply lots of money and remediation. There is a huge incentive to push back.” Because IRIS values are used by other parts of EPA to justify regulations of often widely used chemicals, the producers of those chemicals have tremendous incentives to fight the conclusions of risk assessments. As a result, industry continually finds fault with risk assessments in IRIS and the agency is stymied in attempts to complete them. This is analysis and participation working together in a way that leads to indecision and paralysis.
The world of BCA provided examples of dense analyses largely ignored by the public. Some economists blamed the public with one saying, “Then it gets into will people listen? You can move horse to water but can’t make him drink.” Others acknowledged the growing complexity of analyses as a factor in deterring participation. Analyses by the Department of Energy of the impact of energy efficiency standards regularly reach a thousand pages. One rarely sees consumers who will pay higher prices as a result of these standards participating in these decisions.
While environmental impact assessments did produce heartening cases of analysis leading to public involvement, as in the case of the beluga whales (and cases where participation occurred at the early stages of analysis as described by some of the interview subjects above), the problem of complexity was present in EIAs as well. One representative of the military said, “NEPA is too structured too rulebound. This is your chance to get input and if not we are moving on without you in 30 days. It is more of a disclosure law.” If an agency treats a participation requirement that way, then the participation is unlikely to be useful. My interview subjects also pointed out numerous instances of EISs that were thousands of pages long and very difficult for the public to understand.
In addition, the requirement for EISs to solicit public participation has spurred interest groups to engage in mass letter-writing campaigns to attempt to impact the analysis and the underlying policy decision. The effect however has been to make analysts cynical about outside participation. One interview respondent said,
“Does it bring people in? Yes. Does the government use it? No, it gets junk. 99% of participation is junk. The project goes out under a draft. We get 50,000 postcards, that’s not useful. It’s not democratic and it didn’t become democratic. The myth that is perpetrated, we crave your input. But then we ignore your input …Where you get interesting pushback, is where you have two or three academics vs. 50,000 names on a petition, what’s more important? For the technical guys the two professors are more important.”
As a result EISs, while sometimes spurring participation, also have two other less healthy interactions with participation. Many EISs are so dense that the public cannot meaningfully engage with them. Others prompt such extensive letter-writing campaigns that the volume of participation drowns out any public input that may actually affect the calculus of decision-makers (Shapiro, Reference Shapiro2016).
The story of the interaction between analytical requirements and requirements for public participation is therefore very mixed. There are heartening examples of analysis spurring public awareness of the unintended consequences of a policy decision and resulting in changes to that decision. This is how the synergy between analysis and participation is supposed to work. These examples show that there is considerable potential for the hope voiced by scholars that analysis and participation will combine to effectively influence policy.
But there are also many examples of analyses so complex that public participation is deterred and of interest groups using analysis as a lever to either delay decisions or spur letter-writing campaigns that do little to affect decision-making. There are examples of participants suggesting flaws with the analysis being ignored by agencies. In Table 2, I summarize the examples for the three types of comprehensive-rational analysis.
Examples of success and failure in analysis and participation from interviews.
What differentiates these examples? The biggest factor seems to be the complexity of the analysis. Analysis, designed to encourage rationality in government decisions, is not naturally suited to assisting the lay public in understanding those decisions. This inherent tension has been overcome by some analysts and agencies who have worked to make their analyses simpler and more comprehensible to a broader audience. Another key factor is timing, if the analysis is produced and made public before a decision has been finalized, then any public participation that the analysis incents may affect the final decision. Indeed, advocates of the “analytical-deliberative” perspective argue that participation and analysis should be interwoven from the beginning of the deliberative process (NRC, 1996). If the analysts and agencies have become cynical about public participation, they are less likely to be interested in tailoring their work to a broader audience.
In the next section I describe an example of a type of participation that attempts to grapple with both of these issues. SBREFA panels lead to agencies producing analyses of regulatory impact that can be understood by small business owners. The panels are also conducted early in the regulatory process. They introduce their own problems, particularly the one-sided nature of the panelists, but they may hold useful lessons for future applications of analytical requirements.
4.3 The hybrid: SBREFA panels
In 1996, Congress passed the Small Business Regulatory Enforcement Fairness Act (SBREFA)Footnote 12 as an amendment to the Regulatory Flexibility Act.Footnote 13 The panel requirements apply to the Environmental Protection Agency (EPA), the Occupational Safety and Health Administration (OSHA), and the Consumer Financial Protection Bureau (CFPB).Footnote 14 One innovative provision in SBREFA is the creation of the small business panel process. These agencies must convene small business panels to review regulations and the supporting analyses of impacts on small businesses that are expected to have a significant impact on small businesses. Agencies must convene these panels well before formally proposing their regulations. The panels operate under the joint supervision of OIRA, the Office of Advocacy within the Small Business Administration and the regulating agency.Footnote 15
The process works like this: a group of small business owners likely be affected by the rule is assembled. The regulating agency sends them materials on the need for a regulation, possible provisions in the regulation, and an analysis of the expected impacts of the regulation on small businesses. After receiving this material from the regulating agency, the small business owners have a series of meetings (also attended by representatives of the regulatory agency, OIRA, and the Office of Advocacy) during which the business owners provide feedback on the agency proposals. The three government agencies then take that information and provide a report to the administrator of the regulating agency with recommendations for the eventual proposed rule (these recommendations may include considering alternatives to regulation).
The SBREFA panel has many of the various properties that the interviews above identified as critical in ensuring that analysis encourages outside input. The regulatory agency must produce an analysis early in the rulemaking process for small business owners to comment on. The analysis must be comprehensible to these noneconomists or else the process will grind to a halt. This combination of timing and simplicity makes the SBREFA process worth examining for its potential to effectively combine analysis and public participation.
One recent study of the SBREFA process highlighted some significant concerns. The Center for Effective Government (CEG) (a group that generally supports public health regulation and opposes analytical requirements in the regulatory process)Footnote 16 looked at the SBREFA process and cast doubt on its impartiality (Weatherford, Reference Weatherford2014). The CEG interviewed staff members at the Office of Advocacy, used Freedom of Information Act (FOIA) requests to get information on panels, and looked at the impact of the recommendations of the panels. They found that some of the small business owners who took part in the panels were “representatives, board members, lawyers, or consultants for trade associations and did not own or operate a small entity likely to be affected by the rule under development.” They also found that the Office of Advocacy briefed the small business representatives prior to the panel meetings advising them to raise particular concerns.
Keeping in mind both the potential advantages of the panel structure and the biases described in the CEG report, I observed the operation of a SBREFA panel for a potential proposed regulation by OSHA. I listened in on the panel meetings, reviewed the documentation given to the SERs, and read the final report. The issue involved was the regulation of the transmission of infectious diseases in the workplace, primarily in healthcare facilities.
The rationale for the possible regulation being considered by the SBREFA panel is outlined in a background document written by OSHAFootnote 17 supplied to small business owners at the onset of the panel process. The document explained that while OSHA already has a “Bloodborne Pathogens” standardFootnote 18 which is intended to protect health care workers from diseases transmitted via bloodborne routes, the standard does not protect such workers from illnesses transmitted via contact, droplet, or breathing. The background document also described the Centers for Disease Control and Prevention (CDC) (Siegel, Rhinehart, Jackson & Chiarello, Reference Siegel, Rhinehart, Jackson and Chiarello2007) guidance for workers potentially exposed to infectious diseases. This guidance is nonbinding but came up repeatedly in the small business panel discussions.
OSHA supplied to small business representatives a “framework” for a potential regulation.Footnote 19 The framework, according to OSHA,
represent(s) all of the provisions the Agency believes, at this point, would constitute the best, most protective rule while providing the most flexibility and minimizing the burden on affected entities. While this framework represents OSHA’s initial thinking, the Agency is still considering a number of alternatives and options.
The core of the framework is a requirement that affected workplaces develop a “Workplace Infection Control Plan (WICP),” which contains a number of required elements.
One requirement that would prove particularly controversial in the framework was the “Medical Removal Protection (MRP)” provision. This provision, also present in a number of other OSHA workplace health standards, would require employers to ensure that workers who were forced to miss work due to infectious disease exposure would not lose their wages for the time that was missed.Footnote 20 Employers would have to continue compensating the workers while they recovered. The purpose of this provision is to ensure that workers feel comfortable reporting illnesses because they do not fear losing pay as a result of coming forward.
Fifty SERs were chosen to participate in the discussions with the SBREFA panel. The SERs came from a variety of businesses including hospitals, doctors’ and dentists’ offices, ambulance services and funeral parlors. The SERs represented businesses of various sizes and (as detailed in the CEG report) some were representatives of trade associations that represented large businesses as well as small ones.
The SBREFA panel held three conference calls with the SERs on November 12, 13, and 14, 2014. I was on the telephone for the entirety of the calls on the 12th and 14th, and for part of the call on the 13th. I remained silent throughout the call. The SBREFA panel promised the SERs that they would not be quoted on the calls and I will adhere to that promise. What follows is a general summary of the concerns (much of which is also reported in the final report of the SBREFA panel described below).
The small business owners repeatedly questioned the need for an infectious disease standard. Many said that they are already subject to numerous requirements that address the issues in the standard. They mentioned the OSHA bloodborne pathogen regulation, state regulations (Cal/OSHA the California occupational safety and health agency was mentioned specifically), and the CDC guidelines. While the SERs acknowledged that the CDC guidelines were not mandatory, at least five of the SERs argued that they were in compliance with the guidelines.
The two provisions that caused the most apprehension for the SERs were the vaccination and the MRP provisions. MRP in particular came in for a great deal of criticism. Many SERs argued that this provision alone could drive them out of business. They voiced the concern that if one employee in a small business became sick, and the business had to pay both that employee and their replacement, the cost would be prohibitive. They also wondered why MRP was necessary when their businesses were governed by state workers compensation laws.
With regards to vaccinations, SERs were concerned that being required to vaccinate their employees would be costly. Many of the SERs maintained that if employees ask to be vaccinated, the employers generally provide the vaccines. However, the OSHA requirement would shift the cost for vaccines from insurance companies to the employers and would lead to employers purchasing vaccines that would eventually go unused.
Finally, SERs disputed the cost estimates for the regulation provided by OSHA. This disagreement extended across multiple provisions of the regulation and some SERs even voiced their belief that the regulation would put them out of business. SERs also submitted written comments to the SBREFA panel. These written comments largely echoed the comments made during the telephone conferences. The comments are publicly available as part of the SBREFA panel’s final report.
This report was issued to the OSHA Administrator on December 22, 2014.Footnote 21 The report, jointly issued by the representatives of the three agencies, was filled with recommendations that OSHA reconsider the need for a standard and the scope of the standard. As reflected in the verbal and written comments of the SERs, the report’s first recommendation was, “The panel recommends that OSHA not proceed with issuing a proposed rule until it assesses available data on risk to address the need for the rule for each potentially covered task and work setting.”
The report goes on to suggest ways in which OSHA should limit the standard if it decides to proceed with regulating. The panel recommended that the agency considers limiting the scope of the standard to exclude businesses where the risk of catching an infectious disease is minimal (such as funeral parlors and laundromats – both businesses represented by SERs that argued that the standard should not apply to them). The panel also recommended that OSHA consider exempting certain businesses from certain portions of the standard and reassess the need for certain provisions (such as the MRP and vaccine provisions). Finally, the panel recommended that OSHA revisit its estimates of the costs of the standard.
The panel report reflected the views of the small business owners who participated in the SBREFA process. As this article went to press, OSHA had not yet issued a proposed rule on the transmission of infectious diseases. Given the election of President Trump, a regulation in the near future is very unlikely.Footnote 22 If OSHA does issue a regulation, the cost estimates will be higher and the standard will likely be less strict than envisioned in the framework issued by OSHA. If this is not the case, OSHA will have difficult questions to answer about why it ignored a report which its own staff had signed off on.Footnote 23
Placed early in the regulatory process, and given an institutional supporter in the Office of Advocacy, which has a voice in the management of the panels, the SBREFA panels have engendered meaningful participation and had impacts on policy. This is evidenced both by the experience with the infectious disease panel and by the panels detailed in the CEG (2014) report. But this impact is not on behalf of “better” regulation in the sense often envisioned by advocates of the more comprehensive forms of analysis discussed earlier in this article. Instead it is on behalf of a particular constituency, small businesses. Regulations that have gone through the SBREFA process are more accommodating of the needs of small businesses (and possibly larger businesses who through trade groups appear to also have a voice in the process).
SBREFA panels explicitly combine analysis and participation in the attempt to influence policy. The analyses produced by the regulating agency inform the views of the panel participants. The participants then critique the analysis (along with the underlying policy proposal). Their comments are likely to improve the analysis and perhaps lead to modifications in the policy decision before it goes out to a wider audience. The panelists are hardly an unbiased audience, they represent one politically favored sector amidst the universe of those affected by the regulation in question. But the effectiveness of the analysis/participation combination could be a model for combining these two procedural mechanisms in other contexts.
The SBREFA model could be extended to other populations. For policy issues that affect labor, one could do a “labor impact analysis” and ask panels made up of worker representatives to evaluate the analysis and the policy proposal. New university research regulations could be analyzed and then vetted by a panel of researchers (NAS 2015).
5 Conclusion
Scholars examining analysis requirements faced by agencies often mention the possible nexus between analysis and increased transparency (Sunstein, Reference Sunstein2002; Greenberg, Reference Greenberg2013). Majone (Reference Majone1989, p. 7) said,
Good policy analysis is more than data analysis or a modeling exercise; it also provides standards of argument and an intellectual structure for public discourse. Even when its conclusions are not accepted, its categories and language, its criticism of traditional approaches, and its advocacy of new ideas affect – even condition – the policy debate.
This is echoed in the NAS reports on improving risk analysis and environmental impact assessment (NRC, 1996, 2008). Despite these recommendations, the actual interplay between analysis and participation has been largely theoretical. In practice, the tension between creating an analytical document that comprehensively examines a policy question and using that document to involve the public, who may be turned off by overly technical explanations has been hard to overcome.
In speaking with dozens of high level analysts at federal regulatory agencies, it became clear that public participation is often on their minds as well. In the cases of both BCA and environmental impact assessment, it was not difficult to find examples of analysis and participation being used together to develop real improvements in policy. This synergy was observed in areas as varied as the importation of beluga whales and the security of aircraft repair stations.
But there were also cases such as the risk assessments carried out under the IRIS program where powerful interests continually found fault with analysis and used these faults to delay decisions that might impose significant costs upon them. Interview subjects also brought up numerous examples of benefit-cost analyses and EIS cases where analyses spurred massive letter-writing campaigns which gave decision-makers no new information and increased their cynicism about participation. Analyses of all three types examined here routinely stretched into the hundreds or even thousands of pages, and in doing so deterred all but the most determined and most well-trained from looking at them.
So as repeatedly noted by the NAS and scholars, there is considerable potential for requirements for analysis to both spur and be intertwined with public involvement and in doing so, improve public policy. But that potential is not always realized and on occasion analysis deters participation or gives special interests a lever to hobble agency decision-making. This work points to several characteristics of analysis that make it more likely to lead to public participation. Most importantly, analysis needs to be simple in order to be accessible. The EA in the beluga whale case was 19 pages long. This is in no way typical of EISs and this lack of typicality shows one of the big problems with analysis as a tool to involve the public. Not all policy decisions can be dealt with in a mere 19 pages, but few require the thousands of pages that have become increasingly prevalent in benefit-cost analyses and EISs.
Second, analysis should be done and made public early in the decision-making process (a point also made in Schwartz (Reference Schwartz, Dunlop and Radaelli2016)). Indeed where feasible, it should be done with public input. If done later, the analysis will almost inevitably be used to justify decisions rather than inform them. The public quickly becomes aware of this and learns to ignore the analysis. Risk assessment is the one type of analysis examined here that actually tries to affect agenda setting. It is also the type where engaging the public has been the most challenging. The NAS Report, Understanding Risk, recommends a number of mechanisms similar to SBREFA panels. These include citizen advisory committees, citizen juries (a random selection of informed citizens), and focus groups. While recommended by NAS in 1996, these methods have been rarely used in the context of the types of policy analysis discussed in this article.
The SBREFA panels combine specialized individuals along with simplicity and early involvement. These panels have, on numerous occasions, including the case of infectious disease control discussed here, had a significant influence on policy. While that influence is decidedly one-sided (in favor of small businesses), that does not mean that the model of the panels cannot be expanded to other areas. Putting a clear analysis of a policy proposal in front of parties who will be affected by that policy and then asking for their input in a carefully moderated setting may move us closer to realizing the goals held by those who have advocated for better analysis and for more public input in policy decisions.
In addition, the SBREFA panels highlight the importance of clarity and concision in analysis. The analysis produced by OSHA (and others used in SBREFA panels) is produced with the knowledge that it must be accessible to nonexperts (small business owners). This clarity and concision could be expanded to non-SBREFA contexts (Carrigan & Shapiro, Reference Carrigan and Shapiro2016). If done in a more clear concise manner, analysis can help improve accountability by making the impacts of policy decisions and which alternatives to those decisions the regulatory agency considered clear to the public.
Supplementary material
For supplementary material accompanying this paper, please visit https://doi.org/10.1017/bca.2018.9.
