“. . .those kind of informal, [. . .] official contacts—they go on almost incessantly.”
Former congressional aide,
interview with author
Monitoring unelected officials implementing public policy is a chief concern in democratic government. By overseeing that process, elected officials aim to prevent shirking, corruption, performance failures, and policy drift in bureaucracies. Not surprisingly, questions about the efficacy of this oversight has generated a vast body of theoretical and empirical research.Footnote 1 I advance this work by analyzing records of over 50,000 oversight contacts between members of Congress and bureaucratic agencies. These new data reveal activity long-theorized about, but rarely observed. In an influential article on bureaucratic accountability, McCubbins and Schwartz (Reference McCubbins and Schwartz1984) argue members of Congress use administrative procedures to set up “fire alarms” for agency misbehavior, which allow them to avoid performing costly “police-patrols.”Footnote 2 While scholars have analyzed published oversight hearings (Aberbach Reference Aberbach1990; McGrath Reference McGrath2013; MacDonald and McGrath Reference MacDonald and McGrath2016; Kriner and Schickler Reference Kriner and Schickler2016), this study documents informal comments and inquiries conducted by individual legislators.
This type of oversight is both more frequent than formalized hearings and concerns consequential public policy (Lowande, Reference Lowandeforthcoming). Prior to the 2014 Veterans Affairs (VA) wait-time scandal, legislators made inquiries about VA health system record-keeping procedures that prefigured the problems to come. Their inquiries eventually led to a de facto nationwide review of these procedures. After the enactment of the Dodd-Frank Act, legislators forwarded hundreds of concern letters to the financial regulators required to approve the controversial Volcker Rule. When amendments to the Plutonium Management Disposition Agreement between the US and Russia to dispose of 68 metric tons of weapons-grade material were signed in 2010, Congress did not weigh in with legislation. Instead, legislators contacted the Department of Energy (DOE) to inquire about the construction of disposal facilities, the implementation timeline, and verification procedures. In a policymaking environment with executives increasingly at the center, legislative representatives rely on contacts like these to police agency behavior.
These data provide leverage to test a conventional proposition in existing research: that ideological disagreement with agencies drives the oversight behavior of legislators. The assumption that agencies have preferences of their own that sometimes differ from those of their political principals hardly bears repeating in political science research. It is central to studies of political appointments, delegation, judicial review, rule-making, and oversight. However, to test this proposition, studies have focused almost exclusively on public auditing activities like committee hearings and investigations. Theories of oversight have long argued this behavior represents a fraction of all legislative oversight. This study attempts to address this limitation by analyzing the oversight relationships of individual legislators and executive agencies.
Analyzing informal communication also presents an opportunity to improve estimation of the effect of ideological disagreement, since the dyadic relationship between a congressional median and the bureaucracy limits the variation explored by past empirical studies. My strategy for estimating the causal effect of ideological disagreement is to leverage the transition in presidential administration from George W. Bush to Barack Obama. What is sacrificed in scope is compensated for in internal credibility. Presidential transitions provide critical variation in agency preferences driven by turnover in both appointed and career personnel (Chen and Johnson Reference Chen and Johnson2014; Bolton, de Figueiredo, and Lewis Reference Bolton, de Figueiredo and Lewis2017; Dahlström and Holmgren Reference Dahlström and Holmgren2017), whereas the preferences of legislators within that narrow window are likely to be stable.Footnote 3 It also provides an important secondary contribution, since agency ideology is often thought of as a function of both an agency’s “mission” and personnel. Thus, whereas past work is unclear about the source of interbranch disagreement, this design isolates ideological variation driven by persons rather than statutes.
Contrary to the intuitive notion that ideological disagreement increases oversight, I find robust evidence that it has a negligible effect. Across a variety of measurement specifications, the substantive effect of disagreement is near zero with high precision. The estimates’ 95% confidence intervals never include an effect size greater in magnitude than a two percentage point difference in the probability of oversight—and more plausible counterfactual scenarios suggest effects that are much smaller. In analyses of the number of oversight contacts, the interval never includes an effect greater than a tenth of a contact. These effects contrast those of district characteristics and committee roles, which have strong and substantively significant associations with the likelihood of oversight.
These results offer initial evidence that legislative oversight is driven by policy valence, rather than ideological disagreement. While existing studies analyze the valence dimension of bureaucratic policymaking (e.g., Ting Reference Ting2011), this study provides rare evidence of its relative prominence vis-à-vis ideology. I argue this oversight is not systematically driven by ideological or partisan forces because it is informal and most often unobserved by reelection audiences. Without the opportunity to publicly discredit ideologically divergent agents, legislators’ oversight may instead be motivated by the quality of administration. This implies that existing research may over-emphasize the importance of ideological disagreement for legislative-executive interactions since this broader universe of cases may be driven by policy quality or valence. I present case studies of the Department of Veterans Affairs and the DOE that suggest these concerns are operative.
The results also suggest other important implications for future research on congressional oversight and the separation of powers, more broadly. First, the ubiquity of informal requests suggests that theoretical models of oversight which treat Congress as unitary would be misapplied to most oversight observed in practice. In addition, evidence from this study suggests that past concerns about the overall efficacy of congressional oversight are well warranted. Whereas existing scholarship suggests the multiplicity of legislative principals creates collective actions problems that reduce oversight (Gailmard Reference Gailmard2009; Clinton, Lewis, and Selin Reference Clinton, Lewis and Selin2014), this study suggests that private communications may pose additional obstacles and lead to coordination failures. Thus, evidence that informal oversight is comparatively free of ideological and partisan considerations is not sufficient to guarantee that legislatures effectively police shared goals in policy implementation.
CONGRESSIONAL OVERSIGHT AND BUREAUCRATIC ACCOUNTABILITY
This study attempts to build on some common limitations of past work on oversight. Researchers most often treat Congress as a unitary actor because they are typically interested in questions about political control of the bureaucracy meant to analyze whether policy will deviate from the congressional median.Footnote 4 This means that past work does not generally address the research question of who in Congress performs oversight. The exception is, studies tend to argue, generally, that oversight is conducted by-in-large in committee (e.g., Weingast and Moran Reference Weingast and Moran1983; Ogul and Rockman Reference Ogul and Rockman1990), with more recent work investigating the consequences of overlapping jurisdictions (Gailmard Reference Gailmard2009; Clinton, Lewis, and Selin Reference Clinton, Lewis and Selin2014; Rezaee, Wood, and Gailmard Reference Rezaee, Wood and Gailmard2015).
In general, most empirical research investigates patterns of legislative oversight hearings. But there are numerous reasons to suggest that hearings and investigations may exhibit unique oversight patterns. First, they are subject to congressional gatekeepers or often require some degree of coalition-building. In addition, norms of bipartisanship and committee rules vary across committees and time. So, investigating theories based on the individual-level incentives of legislators to police the bureaucracy in this institutional context is a conceptual mismatch. Second, public hearings and investigations often involve goals beyond policy and ideological disagreement. A common example is the 33 hearings held by the House Select Committee on Benghazi, which Democrats widely criticized as an attempt to discredit Hillary Clinton’s 2016 presidential campaign (Lillis Reference Lillis2015). Studies of investigations suggest that the public signal they generate may drive oversight patterns, so it is difficult to isolate the theoretical motivations behind public “police patrols” (Kriner and Schwartz Reference Kriner and Schwartz2008; Kriner and Schickler Reference Kriner and Schickler2014; Lowande and Peck Reference Lowande and Peck2017).
Research confined to public oversight hearings is also typically unclear about what variation drives the results it presents. The key “treatment” is often interbranch ideological disagreement. Measures of agency preferences in these studies are either static or taken to be presidents or chief executives. Some designs rely on agency cross-sections, which is problematic because of the numerous unobserved differences that may drive interbranch contact. Others rely on the implicit assumption that over-time changes in the preferences of Congress is exogenous. This may not be warranted, since most of this variation is a function of elections, and elections bring about policy changes that plausibly influence interbranch contact.
Inquiries made directly by legislators differ from public oversight in important respects. They are comparatively unilateral. Until recently, there was little question that individual legislators could make inquiries without “clearance” from copartisans, committee chairs, party leaders, or any other official. They are also comparatively private, and less likely to be in service of non-policy-related goals. This point warrants some discussion, because legislators do occasionally “claim credit” for inquiries by posting them on their public web pages. One legislative assistant interviewed indicated that recently, it was also more common for congressional staff to forward inquiries to media outlets and interest groups: “Offices want to make sure their bosses are the one that gets the attribution in the press.”Footnote 5 However, an analysis of the universe of such correspondence is not subject to this selection issue.
The prevalence of informal contact suggests this is a critical institutional setting in which to evaluate questions about bureaucratic accountability. The mere threat of legislation, investigations, hearings, and subpoenas is often used as leverage to compel the production of documents and compliance with Congress’s demands.Footnote 6 Thus, as research on oversight has long-acknowledged, the absence of these events does not imply that Congress is powerless—and might suggest the opposite. Informal inquiries, on the other hand, are not neither “carrot” nor “stick.” They are as close to sincere attempts at oversight as researchers are likely to obtain.
Chuck Grassley (R-IA) highlighted this basic point in May 2017, after the release of an opinion from the White House Office of Legal Counsel (Office of Legal Council, White House 2012). The opinion claimed the Trump administration had the authority to restrict the individual oversight inquiries of members of Congress after months of reports that agencies had stopped responding to letters from Democrats (Elliot Reference Elliot2017). In a letter addressed to the President, Senator Grassley’s response was pointed: “This is nonsense.”Footnote 7 He went on to write that such inquiries were included among “mechanisms that support the legislative check and balance of the executive power. All members participate in deciding whether, when, and how Congress will exercise these authorities.” This makes them ideal for testing long-held notions about Congress and the bureaucracy.
Models of Oversight
Who polices the administrative state? I assess a proposition found in numerous models of oversight: ideological preference divergence between principal and agent should increase oversight.Footnote 8 Following McCubbins and Schwartz (Reference McCubbins and Schwartz1984), I define oversight as actions to “detect and remedy executive-branch violations of legislative goals” and consider the incentives of individual legislators to police particular agencies (165). I take as given that there is some opportunity cost to oversight, so that members have to make strategic decisions about which agencies to allocate attention to. Broadly speaking, there are two conceptually distinct categories of oversight. Efforts to reveal violations are typically thought of as “auditing,” whereas communicating explicit demands is thought of as “lobbying.” Congressional oversight includes both activities, and under minimal assumptions, the divergence hypothesis applies to both.Footnote 9
Members audit to reveal information about governance tasks performed by agencies. This information is valuable to members for a variety of reasons. It may improve their ability to make policy adjustments through legislation. It may reveal “drift” in policy implementation. It can also reveal potential wrongdoing or maladministration. Benchmark models of delegation show that “drift” is more likely among ideologically distal agents. This result typically leads to the “ally principle,” which suggests that principals rely more heavily on agents with proximate preferences. However, delegation decisions are made by the legislative median, while decisions to audit an agency informally are made by the individual legislator. This means that the severity of drift and the value of revealing wrongdoing vary by legislator. Put differently, even an agency strictly loyal to the congressional median will make enemies in Congress.
This implies “enemies” will be more likely to oversee agencies for two reasons. First, the likelihood of policy drift is higher. That is, they are more likely to disagree with agency choices. Moreover, their preferences tend to reflect those of constituents who are more likely to be injured by agency choices. Second, these enemies have more to gain from revealing wrongdoing. The logic here, again, is fairly straightforward. A conservative legislator like Senator Ted Cruz (R-TX) benefits from portraying the Environmental Protection Agency (EPA) as corrupt or incompetent—and preferably, both.Footnote 10 Moreover, there is anecdotal evidence that the mere act of auditing can tap agency resources. Responding to congressional inquiries often requires time-consuming document production unrelated to an agency’s core mission. This logic underlies studies that see investigations as efforts to discredit the opposition and reap electoral gains.
In other attempts at oversight, members lobby to communicate preferences. Work on lobbying typically sees legislators as the target of such communications. In this case, statutory discretion and unforeseen circumstances guarantee that agencies make decisions after enactment that are politically important to legislators and warrant explicit communication (Ritchie and You Reference Ritchie and You2018). Lobbying can be valuable for two reasons. First, it may influence policy outcomes. In a review of major congressional investigations from 1947-2002, Kriner and Schickler (Reference Kriner and Schickler2016) find that about 80% produced “tangible policy change” (171). Moreover, a recent study by Ritchie and You (Reference Ritchie and You2018) suggests that informal contacts are associated with the approval of trade adjustment assistance applications at the Department of Labor. In an analysis of the Federal Aviation Administration, however, Mills, Kalaf-Hughes, and MacDonald (Reference Mills, Kalaf-Hughes and MacDonald2016) find such oversight may have little to no impact. So it is important to note that lobbying can be valuable if it has no impact on policy. In some cases, oversight attempts offer the opportunity to “credit claim” (Mayhew Reference Mayhew1974). Exerting effort on behalf of a constituent may be enough to win their support. This differs from the “grandstanding” opportunities found in congressional hearings because of its focus on serving an individual constituent.
The logic behind lobbying divergent agents follows from these goals. Again, delegation decisions are made collectively, whereas informal oversight is performed by individual legislators. This means there will always be some subset of overseers who would benefit from a shift. From the legislator’s perspective, distal agents are more likely to “drift,” so swaying their choices is particularly valuable.Footnote 11 Likewise, if it is instead assumed that oversight is totally ineffectual, distal agents present the best opportunity to demonstrate to the individual constituent that legislators are serving as faithful advocates.
Not surprisingly, the explicit notion that preference divergence increases oversight has been critical to studies of formal hearings (e.g., Dodd and Schott Reference Dodd and Schott1979; Kriner and Schwartz Reference Kriner and Schwartz2008; McGrath Reference McGrath2013; Fowler Reference Fowler2015; Rezaee, Wood, and Gailmard Reference Rezaee, Wood and Gailmard2015; Kriner and Schickler Reference Kriner and Schickler2016). As McGrath writes, “congressional committees conduct oversight hearings primarily as a response to the extent to which agencies have different policy preferences” (McGrath Reference McGrath2013, 349). More broadly, attention to spatial disagreement is at the core of models of delegation, with some studies even assuming congressional oversight helps set the boundaries of agency discretion.Footnote 12
There is also no shortage of reasons this notion should apply to informal congressional inquiries. As recent work by Wiseman and Wright (Reference Wiseman and Wright2015) highlights, the time elapsed between delegation decisions and agency policymaking is sometimes decades. This may coincide with turnover in both principal (via elections) and agent (via career decisions). In the narrower period of concern for this study, the basic fact is that legislators inherit the delegation contracts of prior congresses and observe changes in agency preferences. This presents the risk of deviation identified in past work. It is also consistent with McCubbins and Schwartz’s model, since changes in agency ideology may result in changes in policy behavior that—in turn—result in more complaints from constituents.
Nonetheless, there are key features of informal oversight which suggest that these conventional arguments may not generalize. First, they are typically unobserved by reelection audiences. Informal contacts allow members to seek the redress of grievances with a more limited audience. News coverage of these contacts have been historically muted. They lack the apparent “grandstanding” opportunity that allows members to distinguish themselves from the opposition, which is often cited as a key driver of the patterns of oversight under divided and unified government (e.g., Kriner and Schwartz Reference Kriner and Schwartz2008; Lowande and Peck Reference Lowande and Peck2017). Moreover, unlike congressional hearings and subpoenas, these contacts are almost entirely unregulated by legislative and executive branch rules. This gives agencies a freer hand in responding and may limit the efficacy of ideologically charged inquiries. Requests with implications for the ideological orientation of policy are zero-sum.
This, naturally, raises the question of what the null hypothesis implies about the broader universe of congressional oversight. I present and begin to assess one argument, that oversight is motivated by “valence” or policy quality-related concerns that are largely orthogonal to ideological disagreement. Broadly speaking, valence is often modeled as a dimension of policy over which all legislators agree.Footnote 13 Studies that consider its analytical properties point to a variety of examples. As Ting (Reference Ting2011) notes, determining the tax rate (10% v. 50%) is ideological, whereas whether the IRS extracts the correct rate from all those it applies to is a question of capacity or implementation quality. In other words, uneven application of the rate would violate a shared goal of avoiding arbitrary exceptions and policy uncertainty. Similarly, Hitt, Volden, and Wiseman (Reference Hitt, Volden and Wiseman2017) argue that legislators have shared preferences over the expected benefit-to-cost ratio of a policy (e.g., minimizing budgetary impact while maximizing the number of beneficiaries). This does not supplant ideological concerns; it simply suggests that policy outcomes vary both by ideological content and perceived quality.
In the context of informal oversight, this implies that legislators would be motivated primarily by concerns over how well policies are implemented. Audits for information and lobbying for policy changes should be driven by the potential for policy failures that negatively impact constituents and could draw broad-based criticism. The lack of public observation means members have far less to gain from lobbing ideologically motivated attacks. Moreover, the lack of rules governing the process gives agencies more discretion in handling requests and induces members to conduct oversight that is not zero-sum. Policy valence-based explanations are featured in some accounts of high profile investigations (Mayhew Reference Mayhew1991; Light Reference Light2014) and they are the dominant subtext for studies of oversight in public administration and public policy. For example, Wood and Lewis (Reference Wood and Lewis2017) leverage a Freedom of Information Act (FOIA) program audit by Representative Darrell Issa (R-CA) to measure responsiveness to private inquiries. In this case, concerns over FOIA performance motivated the government-wide audit. In summary, standard models of oversight that emphasize ideological cleavages may have less purchase absent incentives generated by public observation and formalized communication.
Valence is orthogonal to ideology by definition, but the absence of evidence for the explanatory power of ideology is not evidence of the explanatory power of valence concerns. I defer a more complete discussion of measurement to the following section, but past work has suggested a number of possible markers of valence-related oversight. The most obvious is that oversight would be conducted by-in-large by members of jurisdictionally relevant committees (Ogul and Rockman Reference Ogul and Rockman1990). The committee system allows members to acquire expertise that may make them more effective at detecting the problems. Moreover, whistleblowers informed of Congress’s distribution of labor may target committee members and leadership with complaints. Generally, I also expect members to respond to external evaluations of quality that reveal issues. Many quality concerns are episodic—so that oversight would be associated with the idiosyncratic features of particular congresses and legislative districts. Thus, I conclude this study with several case studies illustrative of valence concerns at work in congressional oversight.
MEASURING OVERSIGHT WITH CONGRESSIONAL CORRESPONDENCE
To reexamine the divergence hypothesis, I analyze records of correspondence between bureaucratic agencies and members of Congress. Since the vast majority of contact records are not publicly available, these records were collected through a series of FOIA requests to 72 agencies in October 2014 (Mills, Kalaf-Hughes, and MacDonald Reference Mills, Kalaf-Hughes and MacDonald2016; Lowande, Reference Lowandeforthcoming).Footnote 14 Though agencies are required by law to respond to these inquiries, their responses do not always provide information usable for analysis. For that reason, the sample of agencies and time series is constrained by data availability.
This raises a key issue: the particular subsample of agencies included in the analysis. Broadly speaking, this analysis over-samples more structurally independent agencies. This is confirmed in Figure 7 presented later, which plots the structural independence of the sample and population as measured by Selin (Reference Selin2015). However, the structural features that render agencies more or less likely to receive demands are invariant within agencies during the period analyzed. So the internal validity of the results should not be biased by such structural features—given the modeling approach discussed in the next section. On the other hand, the broader generalizability of the findings should be taken in careful context. I discuss this issue at length during the discussion of the results by characterizing the relative influence of these agencies on the key findings.
Figure 1 provides an overview of the sample, as well as a comparison with the oversight studied in past work. Specifically, the figure provides a side-by-side look at the number of days agency officials spent testifying in oversight hearings and the number of informal inquiries over the same period. Several descriptive findings are worth noting. First, the number of requests dwarfs that of witness testimony. This, I argue, provides important descriptive support of McCubbins and Schwartz’s notion that most oversight is conducted outside formal hearings. Second, these measures of oversight are not highly correlated. Some reasons will be obvious. A few agencies only contact with legislators is fielding the complaints of the constituents they provide services. But the differences also highlight some of the arguments in the previous section. In general, informal inquiries are not subject to the same institutional constraints. So it is unsurprising to find diverging patterns of oversight. I return to these differences in my discussion of the key findings.
Oversight Activity in the 110th and 111th Congresses
Figure 1 also raises questions about the inquiries themselves. First, the variety of contact begs the question of what does and does not constitute congressional oversight. Many of these inquiries are mundane aspects of daily governance. That is, they are inquiries on behalf of particular constituents with specific grievances—what those in Congress often refer to as “casework.” Others are not broad criticisms of programs—but instead, specific concerns about the details of implementation left to the discretion of agencies. This includes requesting federal funds be spent a particular way—a practice known as “letter-marking”(Mills, Kalaf-Hughes, and MacDonald Reference Mills, Kalaf-Hughes and MacDonald2016). However, the conventional perspective on oversight previously described makes no distinction between the mundane, the important, or the distributive. McCubbins and Schwartz (Reference McCubbins and Schwartz1984) write, for example, “as part of the fire-alarm system, district staff and casework help individuals and groups—some of them otherwise powerless—to raise and redress grievances against decisions by administrative agencies” (173). Moreover, in interviews with current and former congressional staff, most considered casework “low level” oversight.Footnote 15 In short, members of Congress attempting to gather information and influence any public policy may qualify as oversight of agencies.
However, it is important to distinguish between the direct service of constituents (casework) and more general policy-related concerns (policy). First, legislators most often frame the content of oversight hearings as general policy concerns. This facilitates more direct comparison of this study with prior studies of hearings. Second, there are theoretical reasons to believe that casework would be only weakly associated with ideology. Members of Congress are not ideological clones of their median constituent (Tausanovitch and Warshaw Reference Tausanovitch and Warshaw2013). Though constituent grievances are likely to be correlated with member preferences, a strong association would imply members were systematically disregarding complaints that went against those preferences. There is no evidence that members “politicize” their casework operation. In contrast, general policy-related inquiries are more likely to be revealed member positions—since failing to communicate them does not mean actively disregarding the voice of a constituent. In sum, the subset of oversight requests dealing with general policy concerns should be most likely to follow the expectations laid out in the previous section.
Moreover, as the previous section indicates, there is a clear conceptual distinction between oversight that requests information and those that advocate for particular policies (lobbying). Though I have argued that both should follow similar patterns, recent work by MacDonald and McGrath (Reference MacDonald and McGrath2016) suggests lobbying might be negatively associated with ideological divergence. More generally, it is not difficult to construct alternative models of oversight as lobbying that produce this reversed expectation.Footnote 16
Accordingly, though past research on oversight does not distinguish between hearings that fall into the above categories, I take the additional step of coding all informal oversight. Each contact is labeled either “casework” or “policy” related. I also label the contacts that are cases of “lobbying.” Complete definitions, coding rules, and diagnostics can be found in Online Appendix A2 of the Supplementary Information (SI). The contacts were classified with supervised machine learning to reduce reliance on human coders and enhance replicability. I have also removed several classes of correspondence that fall outside the scope of the analysis. These include contacts about committee business (including scheduling, questions for the record, and contacts related to appointments), as well as thank you or congratulatory letters that do not require responses. Excluding committee business is important, since otherwise, the effect of committee roles discussed later might be driven entirely by uninteresting logistics contacts. I discuss several examples of oversight contact below, and report examples for every agency and category in Table A4 in the SI.
Examples of casework abound. Rep. Joseph Pitts (R-PA) contacted the DOE on behalf of constituents with “unsightly” natural gas regulators placed near their homes. Constituents endured this externality because UGI Utilities Inc., an energy provider in western Pennsylvania, was required by agency rules. Members of Congress also make frequent requests that federal funds be spent in a particular way—usually to benefit some specific constituent. For example, Senator Russ Feingold (D-WI) contacted USAID in February of 2007 on behalf of Marquette University (his constituent) to help secure an additional $50,000 for the African democracy training program run by the Les Aspin Center.
Policy audits are also common. Senator Barbara Boxer (D-CA), for example, contacted the VA after the broadcasting of an All Things Considered episode that described the families of veterans “tricked” out of death benefits by MetLife, Prudential, and other life insurance providers.Footnote 17 Other policy contacts appear to be attempts to detect wrongdoing or policy deviation. Then-Senator Joseph Biden (D-DE), for example, contacted the U.S. Agency for International Development in 2007 requesting all internal policy evaluations of funded programs. In Figure 2, Susan Davis audits the DOE’s efforts to implement President Bush’s executive order designed to reduce fuel consumption in the federal vehicle fleet.
Policy Audit (Dept. of Energy)
Members of Congress also lobby agencies to take specific actions. Many of these requests demand extensions to notice-and-comment periods in federal rule-making or request specific changes to proposed rules. Other policy requests are more general and indicative of members’ ideological orientation toward an agency’s mission. In October of 2014, Senator Ted Cruz sent a letter to the EPA alleging “collusion” and “corruption” in the agency’s Office of Transportation and Air Quality. In March of 2009, Senator Dianne Feinstein urged the Interior Department to maintain its ban on concealed carried weapons in national parks. Rep. Steve King (R-IO), who once wrote that the United States could not be “restored [. . .] with somebody else’s babies,” contacted the Department of Homeland Security in 2008 about its efforts to reduce legal immigration.Footnote 18 In short, these requests are both substantively meaningful and contain concerns similar to those raised in other kinds of oversight activities.
RESEARCH DESIGN
To isolate the effect of ideological divergence on oversight, I compare changes in oversight of a particular agency by a particular legislator during a period that includes the transition from the presidency of George W. Bush to Barack Obama. I rely on a generalized difference-in-difference design implemented with the linear probability model belowFootnote 19:
$$\begin{equation*}
Contact_{ijt} = \alpha + \gamma _{i} + \delta _j + \phi _t + \beta Distance_{ijt} + \boldsymbol{\zeta }\boldsymbol{X} + \epsilon _{i},
\end{equation*}$$
where γi represents legislator fixed effects, which accounts for differences in legislative constituencies and congressional office culture, δj denotes agency fixed-effects accounting for the fact that some agencies perform functions inherently more likely to be overseen by Congress,Footnote 20 and ϕt is Congress fixed-effects, accounting for exogenous shocks common across each Congress.Footnote 21 
$\boldsymbol{\zeta }$ represents the effect of time-varying characteristics within districts and agencies I discuss later in this section, and εi represents residuals clustered by legislator.
This research design has a variety of appealing features. First, all time-invariant characteristics of agencies and legislators will be accounted for by intercept shifts. Second, it allows me to isolate the over-time variation in agency-ideology attributable to a change in the presidency. As I describe below, this will primarily be a function of staff turnover. Third, since Democrats maintained majorities in the House and Senate, all committee turnover will be unrelated to partisan changes in the executive branch. This approach is most similar to recent work by Berry and Fowler (Reference Berry and Fowler2015), Powell and Grimmer (Reference Powell and Grimmer2016), and Fouirnaies and Hall (Reference Fouirnaies and Hall2018), which employs similar designs to study district spending and campaign contributions, respectively.
Nonetheless, restricting the analysis to 2007-2010 raises the concern that the results may be unique to presidential transitions. To investigate this, I reestimated the model above with an unbalanced panel of additional agencies and congresses, which I report in Table B7 of the SI. The results are robust to this specification. I also examine the volume and content of requests over time, to determine if transitions are associated with unique temporal trends. I plot several of these time trends in Figure D1 of the SI. In general, presidential transitions do not appear to be unique in the content of informal oversight, but some agencies exhibit clear changes in the volume of oversight during these periods. Thus, it is important to note that without additional data, I cannot assess the relative uniqueness of the transition in question or presidential transitions, more generally. Footnote 22
The key independent variable is Distance, a continuous measure of the ideological divergence between the legislator and the agency. This is the absolute Commonspace DW-NOMINATE distance between the agency and legislator in a given Congress. Legislator ideal points come from Carroll et al. (Reference Carroll, Lewis, Lo, McCarty, Poole and Rosenthal2015). Agency ideal points were estimated by Chen and Johnson (Reference Chen and Johnson2014). This provides a time-variant measure of disagreement on an interpretable and often-used scale. Chen and Johnson use campaign donations and staff composition to place legislators and executive agencies on a common scale. That is, agency personnel make campaign contributions used to scale their preferences, and agencies encounter staff turnover across administrations that change their overall ideological composition. The key measurement assumption is that campaign contributions represent sincere endorsements of political candidates.Footnote 23 Alternative measures of agency ideology based on surveys of scholars (Clinton and Lewis Reference Clinton and Lewis2008) and bureaucrats (Richardson, Clinton, and Lewis Reference Richardson, Clinton and Lewis2018) produce similar results. An additional concern is that including the preferences of career bureaucrats may be inappropriate, since legislators are most focused on Senate-confirmed appointees. To address this concern, I also replicate the main results with mean appointee scores from Bonica, Chen, and Johnson (Reference Bonica, Chen and Johnson2015) and legislator scores from Bonica (Reference Bonica2013).Footnote 24 The results are robust to this measure.
Figure 3 plots the key variation for each agency: shifting ideal points following the transition from the 110th to the 111th Congress. Most agencies shift left, which is consistent with recent work by Bolton, de Figueiredo, and Lewis (Reference Bolton, de Figueiredo and Lewis2017), which finds that the beginning of new presidential terms lead to the exit of senior employees in agencies with views that diverge from those of the new president. This highlights the two sources of variation: voluntary departure by career officials, and political appointments.Footnote 25 The expectation is that this shift left should result in conservative members of Congress increasing their oversight of the administration—while we would expect liberal members to do the opposite.Footnote 26
Agency Ideology and the Transition from Bush to Obama
This approach still requires accounting for factors that vary within agencies and legislators over time. Most importantly, I include dichotomous indicators for whether the legislator is a member, ranking minority member, or chair of a jurisdictionally relevant oversight committee for the agency in question. This provides an important check on the validity of the dependent variable as a measure of oversight, since the scholarly consensus suggests that committee members should be most likely to perform oversight. That is, being assigned to (removed from) an agency’s oversight committee should increase (decrease) the likelihood of informal oversight. It is also critical to reiterate that contacts explicitly related to committee business have been removed. Relevant oversight committees were determined through agency staff responses to the 2014 Survey on the Future of Government Service (Richardson, Reference Richardsonforthcoming). An insignificant number of respondents reported more than one relevant committee, and these committee jurisdictions do not meaningfully differ from oversight plans released by Congress.
I also control for agency budget. Agency budgets account for legislative changes that surely promote oversight. This is particularly important for the time series considered, since it includes the enactment of the American Recovery and Reinvestment Act (2009). For example, among other provisions, the stimulus funded renewable energy research grants in the DOE, which could plausibly influence Congress’s collective propensity to oversee. Release of these funds drew some controversy, as Republican Senators eventually released a list of “wasteful” spending under the stimulus.Footnote 27 In supplemental analyses reported in Table B9 of the SI, I control for agency staff as a robustness check. Note that because of the restricted time series, there are no changes in majority party within legislator. Though there are changes in presidential copartisanship, including it as a covariate may lead to attenuation bias because it is correlated with Distance. The results presented later, however, are not sensitive to including it.
FINDINGS
A descriptive look at the data foreshadows the key finding that follows. Figure 4 plots the Commonspace DW-NOMINATE distribution of contacts in eight agencies, overlaid with the distribution of all legislators in both congresses. If the policing hypothesis was correct, we might expect them to differ. Liberal (conservative) agencies would have more conservative (liberal) contact lists. That is not the case. KS tests suggest that none of the 16 distributions significantly differ. This provides preliminary evidence against the hypothesis, which the full models largely confirm. Across all model specifications, preference divergence has a negligible relationship with oversight activity, whereas committee roles and idiosyncratic differences across agencies and legislators explain most variations.
Ideological Composition of Oversight
Oversight and Ideology
Table 1 reports the results across all measures of oversight contact. Overall, the estimates are consistent across specifications and run counter to the divergence hypothesis. Ideological distance appears to have a negligible effect on oversight, whereas committee membership increases the likelihood of oversight. By negligible effect, I do not mean the absence of statistical significance (Rainey Reference Rainey2014). In model 3, the upper bound of the 95% confidence interval would suggest that a standard deviation increase in distance (0.26) is associated with a 1.4 percentage-point increase in the probability of oversight. This is substantively small, given the unconditional probability of contact, 0.36. Across all measures, this is the largest (in magnitude) effect size within a confidence interval, suggesting the effect of distance is near zero. Moreover, even this overstates the substantive effect, since the within-legislator standard deviation of ideological distance (0.14) is much lower (see Figure B2). It implies the largest magnitude effect within a confidence interval is a 0.7 percentage point change in the probability of oversight (Mummolo and Peterson Reference Mummolo and Peterson2018).
Oversight and Ideology
Linear probability estimates of dichotomous indicator for informal legislator oversight of agency; robust standard errors clustered by legislator in parentheses; all models include over-time controls (agency budget and legislator seniority); models (1) and (2) exclude the FDIC due to missing contact descriptions.
One concern might be that by focusing on within-legislator variation, this approach “stacks the deck” against finding support for the divergence hypothesis. But it is worth noting that even models that exclude legislator fixed-effects (and thus, leverage cross-sectional variation in Congress) return similar results. I report these in Appendix B2 of the SI. Moreover, given conventional concerns about nonsignificant study results, it is important to stress that this finding is not a function of other model specification decisions. As indicated earlier, modeling the count of contacts in each category does not meaningfully change the result. Table B2 of the SI reports the marginal effect of distance on the number of contacts. Here again, the largest magnitude effect within a 95% confidence interval is about a seventh of a contact. These findings are also robust to narrowing the definition of oversight to other varieties of contact, including omitting casework and requests that merely request information (Table B1). Moreover, they are consistent across three alternative measures of ideological distance (Tables B3, B4, and B5). Thus, I argue the results provide strong evidence that preference divergence does not drive oversight.
Oversight and Committee Roles
Another potential concern is that the data analyzed could differ so markedly from the concept scholars have tried to capture in the past that it is unsurprising it would not hew to conventional ideas about oversight. In keeping with those notions, however, committee membership is consistently associated with oversight. More specifically, membership on a relevant oversight committee results in a six-percentage-point increase in the probability of oversight. There are also intuitive gradations by role in committee. Though there are fewer leadership switches—and thus, less variation to estimate their effect—chair and ranking member status may result in an additional boost in oversight. For chairs, the effect for policy related requests is likely positive and around 11 percentage points.
Figure 5 also suggests some additional nuance by the type of oversight. Of particular interest is that the effect of being appointed to or removed from a relevant committee is stronger for policy related requests than for constituent casework. This provides additional evidence of the validity of the dependent variable, a legislators’ constituencies remain constant—so the “demand” for casework should be relatively invariant. However, legislators are more likely to raise more general policy-related concerns once on a relevant committee. This is consistent with a myriad of points made about committees and oversight. Among these are arguments about the distribution of labor, specialization, and the potential for increased contact with relevant interest groups (e.g., Powell and Grimmer Reference Powell and Grimmer2016). The latter is particularly important, given longstanding arguments about interest group influence in government. These results provide initial evidence of a possible mechanism for that influence. That is, if legislators assigned to committees are targeted by interest groups, those legislators may attempt to serve those interests in the context of informal communications rather than roll call votes. This raises important prospects for future research to consider.
Oversight and Committee Roles
Scope Conditions
The results provide consistent evidence that the divergence hypothesis holds much less explanatory power than previously thought. These data also present the opportunity to understand the precise scope conditions of that evidence. Thus, I take the additional step of explicitly characterizing the generalizability of these findings with respect to factors of substantive interest to researchers in executive and legislative politics. Specifically, I show which legislators and agencies are contributing the identifying variation-driving estimates of the effect of ideological distance and committee membership. Following work by Aronow and Samii (Reference Aronow and Samii2016), I residualize distance and membership by the remaining controls, using the square of these residuals as multiple regression weights. Figures 6 and 7 plot the mean weight by legislator and agency by several variables of interest. This has two utilities. First, it directly addresses questions about external validity by describing the subpopulations driving the observed effects. Second, and relatedly, it suggests where future research is (or is not) likely to find support for the divergence hypothesis.
Ideology of Effective Legislator Sample
Structural Independence of Effective Agency Sample
Figure 6 plots the effective sample of legislators by Commonspace DW-NOMINATE score. For the effect of committee membership, the distributions are mostly indistinguishable since there is near parity in partisan committee membership (with slight majorities for Democrats during both congresses). Figure 6(a), however, demonstrates that the ideological distance results are driven disproportionately by more conservative legislators. This is partly a function of the agencies studied, which lean moderately left. This means results are more heavily reliant on conservative legislators (who mostly see agencies moving away from them after the presidential transition). However, this also places the negligible effect of disagreement in an important context. Conservatives are typically thought of as more skeptical of central state authority, and disproportionally responsible for polarization. These are the precise legislators driving the negligible effect of disagreement.
Understanding the relative influence of each agency also reveals some additional findings of interest. Figure 7 plots the mean sample weight by agency by structural independence. Selin (Reference Selin2015) develops a model of latent independence based on a vector of time-invariant agency characteristics. The x-axis plots latent independence of the agency’s decision makers, which represent how much control principals have over hiring and firing leadership. The y-axis plots characteristics related to opportunities principals have to review and reverse agency decisions. Higher values indicate more independence on each dimension. Not surprisingly, the effective sample for the effect of committee membership is not distinct from the observed.
However, Figure 7(a) suggests some noticeable differences for ideological distance. That is, the effect relies on variation observed in more independent agencies. These are the agencies that—according to Figure 3—saw the largest changes in personnel ideology attributable to the presidential transition. This is in keeping with stylized accounts of large departments, which are difficult to change by virtue of their size. This suggests, counterintuitively, that ideological changes in agencies more independent (by design) do not provoke policing efforts on the part of members of Congress. Moreover, it also defines scope conditions of this result. Since large shifts were not observed in the less insulated agencies, future work should search for changes in these agencies.
POLICY VALENCE AND OVERSIGHT
I have found no evidence for the proposition that preference divergence drives oversight. Initial evidence presented in this section suggests that informal oversight most often reflects the quality of administration, rather than concerns driven principally by ideology. Systematic measurement of policy quality presents research challenges because of the diversity of functions performed by agencies and the lack of common scale for quality. Therefore, I present two case studies in this section as evidence. The 2014 wait-time scandal at the VA illustrates a rapid (and preemptive) response to an apparent policy failure. I also present a cross-sectional analysis of offices in the DOE and show that lower performance ratings were associated with more oversight in the 110th Congress. These cases illustrate an important theoretical point: public observation may be a necessary condition for the ideological patterns of oversight observed by past research. They also raise questions about the efficacy of unilateral oversight attempts.
VA Wait-Time Scandal
In April of 2014, CNN reported that bureaucrats in the Veterans Health Administration had deliberately fabricated appointment wait times—and that at least 40 veterans had died while waiting to receive medical care (Bronstein and Griffin Reference Bronstein and Griffin2014). The response from Congress was dramatic: The House and Senate oversight committees convened special hearings, subpoenaed VA officials, demanded resignations, funded a criminal investigation, and enacted sweeping legislative reforms. Records show, however, there were numerous informal oversight contacts in advance of the 2014 wait-time scandal. In 2010, then-Senator Jim Webb (D-VA) contacted the Department of Veterans Affairs after receiving complaints about the Washington DC VA Medical Center. General complaints about claim processing delays are common in the VA log. Between 2010 and April 2014, ten legislators made inquiries about delays in medical care. These inquiries are not all constituent “casework”—as some voice general concerns about Department procedures. According to the log, in a 2011 inquiry, Senator Patty Murray (D-WA)—Chair of the Senate Veterans’ Affairs Committee, voiced the following:
Concerns about an internal VA memorandum indicating that “gaming strategies” are used at VA facilities to improve internal scoring measures at the expense of Veterans seeking VA health care. Request from VA how the VA has eliminated these improper scheduling practices . . .
Murray refers to an April 2010 memo that indicates VA management was aware of the record-keeping procedures that led to the 2014 crisis. So the key Senator tasked with overseeing the department raised concerns about the exact problem a full three years in advance of the first CNN report. This oversight by individual legislators was accompanied by Government Accountability Office reports in 2000, 2004, and 2013 that indicated reported wait times were unreliable (Government Accountability Office 2013). The release of the CNN report led to a flood of informal contacts from legislators with the same basic aim: determining how the scandal affected veterans in their districts. Though veterans’ benefits are often considered uniquely nonpartisan, the motivation behind these contacts is particularly noteworthy. The key concern is policy quality. It was not a question of how generous VA benefits would be—it was about whether the Department was consistently providing the level of care specified by Congress.
It is important to note, here, however that collective action—despite the action of individual legislators—was largely absent. The lone exception is a hearing on March 14, 2013 in the House Veterans’ Affairs Subcommittee on Oversight and Investigations, where the problems with reporting wait times were voiced by VA officials and acknowledged by legislators. I do not mean to suggest that oversight could have prevented policy failure in this instance, or that legislators knew about policy failure and did not act. In fact, legislators engaged in considerable oversight. But, much of this activity is in service of the narrow concerns of a few constituents. Collectively, this activity suggests the more widespread problems later revealed. But addressing those problems required coordination among legislators that was eventually generated by the salience of the scandal—not foreknowledge of the problems that generated it. I return to this point in the Discussion.
Policy Performance at the Department of Energy
The VA scandal provides some evidence that policy valence drives oversight. To address the apparent limitations of this anecdote, I also analyze oversight and performance in offices within the DOE. The DOE is uniquely suited to within-agency analysis because its offices are ideologically diverse. It includes a conservative defense agency, the National Nuclear Security Administration, along with liberal offices in charge of funding renewable energy projects. During the George W. Bush administration, many of its programs were evaluated using a common metric: the Office of Management and Budget’s Program Assessment Rating Tool (PART).Footnote 28 Finally, its logs are sufficiently detailed so that member contacts can be matched to specific programs and offices.
To assess the proposition that valence concerns should drive policy oversight, I subsetted the DOE log to the 110th Congress. The final PART evaluation of DOE programs was conducted from 2006-2007, since the rating system was discontinued by the Obama administration. I matched legislator correspondence to one of 49 programs rated by the OMB, which were run by the eight offices rated by Richardson, Clinton, and Lewis (Reference Richardson, Clinton and Lewis2018).Footnote 29 These measures are based on the 2014 Survey on the Future of Government Service. Since this analysis leverages cross-sectional variation, it is important to note that to be valid, the ordinal ideological ranking of these subagencies must remain roughly constant from 2008 to 2014. Since such rankings do occasionally change from administration to administration (see Figure 3), this may not be the case. For this reason, the inclusion of this control should not be seen as a “horse-race” between valence and ideology. Of the roughly 1,100 contacts to the DOE in the 110th Congress, 255 explicitly reference PART-evaluated programs. They run the gamut of contact subjects described in this study: from grant support to member concerns and explicit program audits.
I regress member oversight on the mean results score by office.Footnote 30 Table C11 in the SI reports these estimates. The models control for two confounders: the number of programs rated within an office and the ideological distance between the legislator and office. Since measures of legislator (Carroll et al. Reference Carroll, Lewis, Lo, McCarty, Poole and Rosenthal2015) and office preferences (Richardson, Clinton, and Lewis Reference Richardson, Clinton and Lewis2018) are on difference scales, I cut the scores into quintiles (1–5, with higher values indicating greater conservatism), then take the absolute difference.
The results suggest that poor program performance in an office was associated with greater oversight. More specifically, a standard deviation decrease in the PART results score (0.12 or about a full letter grade difference) results in a two-percentage-point increase in the probability of contact, holding the other covariates at their means. The unconditional probability of any single legislator contacting a particular office is quite low (~2%), so this is the equivalent of doubling the likelihood of contact. The effect of ideological distance is near zero—a standard deviation increase in divergence is associated with a 0.3-percentage-point increase in oversight. These two effects should not be interpreted as the result of a competition between hypotheses.Footnote 31 Assessing the relative explanatory power of each is difficult, since both vary at different levels (office and legislator-office, respectively). In addition, the lack of over-time variation means relying on a single cross-section of offices.
Though these estimates should be treated as purely descriptive, the case suggests that many efforts to oversee bureaucracies are driven by concerns about the quality of administration. In other words, they suggest critical empirical scope conditions of the ideological divergence hypothesis. Legislators may be only conditionally motivated to systematically pursue agencies with whom they disagree. In fact, the apparent ideological conflict might be an artifact of political signals meant for reelection audiences. But in private, it is perhaps reassuring that their concerns most often reflect questions about the details of implementation rather than reproducing the partisan conflicts evident in congressional chambers.
DISCUSSION
The scope and effectiveness of legislative oversight of the bureaucracy is a perennial issue in modern government. Agencies are increasingly tasked with addressing complex problems that require delegated authority and policy discretion. Since the publication of McCubbins and Schwartz’s (Reference McCubbins and Schwartz1984) work, political scientists have acknowledged that much of legislative oversight of the bureaucracy occurs outside legislative chambers. But without a record of those communications, we cannot test basic empirical predictions about this larger universe of interactions between Congress and the bureaucracy. This information permits analysis of individual-level incentives of members of Congress to conduct oversight.
This study suggests congressional oversight is far more diffuse and ubiquitous than previously thought. Hearings, resolutions, and statutory changes are exceptions to the oversight that occurs day-to-day. In light of that fact, I have modeled factors influencing legislators likelihood of performing that oversight. Contrary to a proposition put forth by past work, I find that ideological disagreement has a negligible impact on oversight. By focusing on public auditing activities, existing oversight research may oversell the importance of preference divergence relative to other explanations like policy valence. Typical ideological divisions of Congress do not seem to map onto private oversight activity by virtue of the fact that they are informal and go unobserved. That implies that public observation is a necessary condition for oversight driven primarily by preference divergence.
These findings and arguments should, of course, be placed in the context of several caveats. My approach is designed to isolate ideological variation driven by personnel changes. These results do not preclude disagreement over agency missions as a possible driver of oversight. The problem, as I have noted, is that this often requires making inferences based on cross-sections of agencies. It is also possible that the “shock” of a single presidential transition is not large enough to reveal meaningful effects. But that point, as well, raises the question of how large a shift would be required to provoke a response. This study suggests that those during the 2008-2009 transition had little impact (and they are fairly typical of other transitions in magnitude). Finally, this analysis relies on measures of agency ideology which place all actors on a common scale. Though I argue this is appropriate in this context, and should capture the changes of interest, it is possible that measures specific to policy area would find meaningful effects.Footnote 32 But notably, any policy-area specific measure would approach many of the valence-related issues for which I have provided examples—so the basic points about the content of most oversight stand.
These findings also imply some important considerations for future work. First, oversight is often informal and conducted by individual legislators. This means that the collective action problems identified by past work are likely understated, since all legislators have the opportunity to oversee the bureaucracy (Gailmard Reference Gailmard2009; Clinton, Lewis, and Selin Reference Clinton, Lewis and Selin2014). It is also worth reiterating that the frequency of informal contacts implies models of oversight that treat Congress as a unitary actor are not well suited to describe most oversight observed in practice.
Finally, the findings—coupled with a brief look at several cases—suggest that past concerns about the overall efficacy of congressional oversight are warranted. Whereas existing scholarship suggests the multiplicity of legislative principals creates collective actions problems that reduce oversight, this study suggests that private communications may pose additional obstacles and lead to coordination failures or poor performance. This is a qualitatively different collective action problem than the one pointed out by Gailmard (Reference Gailmard2009). Whereas that work focused on redundancy as generating suboptimal level of oversight induced by the free-rider issues, this is primarily a coordination problem. Because the vast majority of oversight is conducted by individual legislators privately, the institution is may miss systemic problems in administration (as in the case of the VA). This naturally raises questions about what could prevent such failures. One possibility is more reliance on collective requests. Another is institutional reforms to make interbranch communication more transparent and available to the public. It is, of course, difficult to determine how such reform would alter congressional behavior, so these issues remain for future work to consider.
SUPPLEMENTARY MATERIAL
To view supplementary material for this article, please visit https://doi.org/10.1017/S0003055418000497.
Replication files can be found on Dataverse at: https://doi.org/10.7910/DVN/LFQZVU.
Comments
No Comments have been published for this article.