Nelson Aldrich (R, R.I.) dominated the Senate at the turn of the last century, known, by his admirers as much as his critics, as the “General Manager of the United States.”Footnote 1 He held three formal positions of consequence—he chaired the Senate Rules Committee, 1887–93 and 1895–99; he chaired the Senate Finance Committee, 1899–1911; and he sat on the Republican steering committee, 1893–1911—but he never served as caucus chairman or as elected floor leader. Indeed, at least on the Republican side of the aisle, the position of elected floor leader had yet to be invented during Aldrich’s time as a senator. When he announced plans to retire from the Senate, Aldrich told his friends that he had grown tired of being a “pack-horse” for his colleagues. In the recent battle over revisions to the tariff, Aldrich had “labored incessantly, day and night,” the New York Times reported in the spring of 1910. “There were innumerable conferences, in which he took part, at his committee rooms, at the White House, and elsewhere. He was always on duty, and the strain was tremendous.”Footnote 2
Majority Leader Joe Robinson (D, Ark.) in May 1937. He served as Democratic leader for almost 14 years.
But the power of any senator in those years, even of Aldrich, was limited. “The public now and again picks out a Senator who seems to act and to speak with true instinct of statesmanship and who unmistakably merits the confidence of colleagues and of people,” Woodrow Wilson wrote in Reference Wilson1885 (213), in his classic book Congressional Government. “But such a man, however eminent, is never more than a Senator. No one is the Senator.” Aldrich’s leadership was at its greatest on financial issues—even the Boston Herald, which appears to have popularized the notion of Aldrich as “General Manager,” confined the epithet to the matters of “trade and commerce and finance”Footnote 3—and he was reminded often of the constraints on his ability to shape outcomes in other realms. In 1890–91, when Democrats launched a historic filibuster to frustrate the Republican effort to pass the Federal Elections Bill, Aldrich announced plans to use majority cloture to end the debate and bring the bill to a vote.Footnote 4 Thomas Reed, speaker of the House of Representatives, had already acted decisively to reform his chamber’s rules and to pass the landmark bill protecting voting rights for Black men in the South, and Republican senators, who controlled the Senate, were on record supporting the bill. Aldrich faltered in the effort, and the Federal Elections Bill failed.Footnote 5 A decade later, in 1900, in a news story meant to demonstrate his skill in managing the Senate, the Washington Post inadvertently exposed the weakness of Aldrich’s position. Titling its story “Mr. Aldrich’s Clever Move,” the Post reported how he took advantage of a nearly empty Senate chamber to secure a “‘unanimous’ agreement” that the gold standard bill would be considered to the exclusion of all other business until it came to a vote eight days later.Footnote 6 That Aldrich was most effective when he had the chamber largely to himself suggests the challenges he faced in managing currency legislation, let alone the rest of the Senate’s business.
In the 2020s, the power wielded by the Senate majority leader is, in contrast, immense and in public view. It is visible in historic procedural decisions. First Harry Reid (D, Nev.) in 2013, then Mitch McConnell (R, Ky.) in 2017, mobilized their caucuses to end the filibuster and adopt majority cloture for judicial and executive branch confirmation votes, in the face of determined minority obstructionism. That power is visible in unprecedented actions taken to block or confirm nominees to the Supreme Court. When Justice Antonin Scalia unexpectedly died in February 2016, McConnell, on vacation with his wife in the Caribbean, immediately announced that the Senate would not consider any nominee by President Barack Obama. “I had members all over the country and all over the world. I didn’t want to deal with 52 different opinions when we got back about how we were going to handle it,” McConnell explained later. “And so I decided to lay down a marker and hope that people would fall in line. And with few exceptions, they did.”Footnote 7 Four years later, in 2020, when Justice Ruth Bader Ginsburg died six weeks before the November elections, McConnell also acted swiftly, declaring that the Senate majority would confirm a nominee by President Donald Trump. And that power is visible in legislation—as Reid in 2009 worked “behind closed doors,” in his office, to determine the final shape of the Affordable Care Act and build support from the 60 Democrats whose votes he needed to pass the bill without minority support; as McConnell in 2017 brought a handful of senators into his office and helped manage the negotiations that led to massive changes in the tax code, with the bill passing by a 51–48 vote; and as Charles E. Schumer (D, N.Y.), “the cordial collaborator who always keeps his flip phone nearby to start a new discussion toward sealing the deal,” negotiated, over many months and in secret, with Joe Manchin (D, W.Va.) to secure passage of a historic climate package, the Inflation Reduction Act, in the summer of 2022 by a 51–50 vote.Footnote 8
Leadership of the Senate was an afterthought at the Constitutional Convention. It took more than a century’s time before senators began to find an enduring solution to that problem. The Constitution, of course, provided the Senate with a leader—“The Vice President of the United States shall be President of the Senate”—but “president” in this case simply meant presiding officer, and it was a presiding officer without a vote (except in cases of ties) and a leader imposed on the Senate from the outside, neither chosen by senators nor accountable to them. The framers of the Constitution assigned presiding officer duties to the vice president to give vice presidents something to do with their abundant free time, not to empower the Senate to govern itself. For its first hundred years, as we will show, the Senate struggled with a variety of organizational challenges and experimented with various institutional solutions: granting powers to the presiding officer, inventing party caucuses, organizing an array of ad hoc caucus committees, and establishing powerful campaign committees, committees on committees, and steering committees. Only in 1890 did senators in the Democratic caucus create a position that would become the ancestor of the modern posts of majority and minority leaders, and it was a position that was not adopted by the Republican caucus until 1913. Republicans, after all, had Aldrich and the Republican steering committee. Elected floor leadership developed late in the Senate’s history, fitfully, in the shadows.
Our account in this book is largely new and original. The course of Senate party development—including the origins of caucuses, steering committees, other caucus committees, and floor leadership—has long been obscure. Before we began this research, no accurate lists of early floor leaders or caucus chairs existed. The standard narrative identified the 1920s as the moment when recognized floor leaders appeared in the Senate. Until recently, in fact, when it drew on early drafts of this book to update its records, even the Senate Historical Office reported on its website and in its reports that “the positions of party floor leaders … developed gradually in the 20th century,” identifying 1920 (for Democrats) and 1925 (for Republicans) as the date when each party first named a formally designated leader.Footnote 9 But that timeline was off by a generation. We will show that the origins of party floor leadership, including formal caucus designation, came in the 1890s—when Maryland senator Arthur Pue Gorman led the minority Democrats in their successful effort to defeat the landmark Federal Elections Bill of 1890, the last serious effort by Congress, until the modern civil rights era, to protect Black voting rights in the South.
The majority and minority leaders of the twentieth and twenty-first centuries are, first and foremost, party leaders. Unlike the president, elected by a nation-spanning electoral college, and the speaker, chosen by the full membership of the House, the Senate leaders are selected in caucus, designated by members of their party to lead them on the chamber floor. In the early years that this position was created, few understood its significance. It was simply one among many party institutions with which senators were experimenting. But by the 1930s, it was well established that the majority and minority leaders of the Senate were consequential, and the significance of these positions has only grown in recent decades.
Our main concern is to understand Senate party development. What are the problems that individual legislators encounter in the absence of leadership? How do they set out to solve problems of coordination and collective action? How do they assign members to committees, organize campaigns, and set legislative agendas? At which moments do individual legislators decide to delegate some of their powers to a collective group of legislators and, ultimately, to a single leader? How and when, over time, do they decide to grant that single leader greater authority? Because the invention of entirely new party structures in the Senate stretched over a century’s time, from 1789 to the 1890s—and the powers of floor leaders have then been elaborated over the course of more than another century, from the 1890s to the 2020s—the U.S. Senate offers an outstanding laboratory for answering these questions. Time after time, senators have turned to new institutions and new forms of party organization, which gives us multiple opportunities to ask what distinguishes those moments.
Our answer, and our central argument, focuses on three factors: party competition, factionalism, and entrepreneurship. In the Senate, where leadership and institutional organization rest in the two parties rather than in the presiding officer, members adopt innovative structures when parties are most closely balanced. It has been at those moments when parties are battling for control of the Senate, often just after a shift in majority control, that one party (or both) adopts organizational innovations. New institutions are the products of tenuous majorities and hopeful or disappointed minority parties. But these innovations are adopted not only in response to moments of party competition but to address factionalism within parties. Factions often seek organizational and procedural changes within parties to gain an advantage. Moreover, divided parties and unified parties demand different forms of organization. And entrepreneurship matters. Whether it was Thomas Hart Benton (D, Mo.) and Henry Clay (W, Ky.) in the 1840s, Aldrich and Gorman in the 1890s, Charles Curtis (R, Kans.) and Joseph Robinson (D, Ark.) in the 1920s, Robert Taft (R, Ohio) and Lyndon B. Johnson (D, Tex.) in the 1950s, or Schumer and McConnell in the 2020s, individual entrepreneurs are the ones who take the lead in constructing new institutions in response to competitive parties or factionalized coalitions.
The Understudied Senate
The longstanding individualism and informality of Senate proceedings have limited scholarly interest in the development of Senate party leadership and organization. Both the contemporary Senate and the historical Senate have been underexamined by scholars. Most major studies of Congress have been studies of the House of Representatives, and the House speakership alone has been the subject of several books. Traditionally viewed as a less rules-bound body, with weaker leadership and more powerful individual members, the Senate has languished in congressional scholarship from the nineteenth century until the present day. In the 1970s, Jones (Reference Jones1976, 19–20), reflecting on the modern Senate, made the uncontroversial observation that “strong substance-oriented policy leadership by party leaders is neither possible nor desirable in the United States Senate.” The rules of the Senate, it has been commonly observed, protect the rights of senators to debate and offer proposals on the floor of the Senate and grant few special privileges to its presiding officer or to the leader of the majority party. And Senate rules rarely change from Congress to Congress.
The few existing studies of Senate party development present competing accounts of when and how centralized parties emerged. A set of studies from the 1960s and 1970s, three published and two unpublished, offer outstanding examples of this scholarship. The three published accounts have had a lasting impact on congressional studies, yet their main features are difficult to reconcile with one another. Rothman’s Politics and Power (Reference Rothman1966), the most important book-length study of the Senate’s institutional development, focuses on the collective governance structures, dominated by senior senators, that emerged in both parties in the late 1890s—institutions that Rothman regards as the foundation of modern, disciplined Senate party organization—but Rothman’s account focuses on the turn of the last century, failing to note that these oligarchical institutions did not survive the early 1910s. Munk, in a dissertation (Reference Munk1970) and published article (Reference Munk1974), emphasizes an entirely different institution, the position of floor leader. In her dissertation, she lays out a nuanced, gradual process of institutional change, examining the ascendancy and collapse of the Republican oligarchy in the 1890s, 1900s, and 1910s, alongside the steady development of the floor leadership position from the 1890s to the 1960s—remarkably similar to the account in Kravitz (ca. Reference Kravitz1971), an unpublished book manuscript. However, in refining her argument for publication, Munk argues that floor leadership remained a primitive position until 1913, when John W. Kern (D, Ind.) became the Senate’s first widely recognized majority leader. “The existence of majority and minority floor leaders,” Munk (Reference Munk1974, 23) contends in her article, “can be traced with assurance only back to the second decade of the twentieth century.” That crisp contention, rather than the subtle argument of her dissertation, had lasting resonance for subsequent scholarship. Meanwhile, Riddick (Reference Riddick1971), drawing directly on surviving minutes of the two Senate caucuses, which exist for the Democrats since 1903 and the Republicans since 1911, concluded from his research that designated floor leaders emerged only in the 1920s.
The tensions between these accounts create obvious challenges for generating and testing theories of institutional change in the Senate. These scholars, each a careful student of Senate history, disagreed on which institutions, whether the Republican oligarchy of the 1890s or the floor leaders of the first decades of the twentieth century, best reflected a centralized structure of Senate party organization. For those emphasizing the role of floor leaders, they disagreed on timing—whether this was a gradual process of development, a dramatic break in 1913, or a new institution in the 1920s. Consequently, as they and subsequent political scientists sought to explain the rise of party institutions in the Senate, their efforts were hobbled by the elusiveness of their dependent variable.
Careerism, for example, has been cited by scholars as critical to Senate party development, but they disagree about the role careerism has played. Rothman (Reference Rothman1966) contends that the centralization of Senate parties in the late nineteenth century resulted from the increasing number of senators who regarded Senate service as a career and owed their advancement to state party organizations. Yet Ripley (Reference Ripley1969b) insists that, until the 1880s, careerism slowed the elaboration of Senate party organization and central leadership posts because career-minded senators, unlike less career-oriented members of the House, were unwilling to tolerate strong central leaders. And Brady et al. (Brady, Brody, and Epstein Reference Brady, Brody and Epstein1989; Brady and Epstein Reference Brady and Epstein1997) assert that it was the dominance of “noncareerists” in the Senate that facilitated the centralization of power in the late nineteenth century in the Aldrich oligarchy.
The Senate’s workload and size are natural candidates as factors that shaped party organization. McConachie (Reference McConachie1898, 313–21) observes in the 1890s how the Senate’s increasing workload compelled adjustments in procedure and practice. Similarly, Baker and Davidson (Reference Baker, Davidson, Baker and Davidson1991, 2) argue that the independent power of committee chairs eroded sharply in the 1910s when the Senate reorganized its affairs to respond to the Great War and to an increasingly powerful president. Looking at the development of new party leadership positions in the Senate in this era, they conclude that the new conditions “necessitated a coordinated Senate leadership quite beyond the capacity of individual committee chairmen.”Footnote 10 A war effort, domestic emergencies, a large policy agenda, or even new forms of press coverage may create new demands for coordination, particularly from within the majority party, which is likely to be blamed by outsiders if the Senate fails to act on desired legislation.
The Senate’s size, too, may create coordination and collective action problems that can be addressed by party organization, including the creation of formal leadership positions. For decades, social scientists have emphasized the importance of group size as a factor that influences the severity of the collective action problem and the effect of individual efforts to address it. The larger the group, the smaller the contribution of the individual, the greater the incentive to be a “free rider,” the greater the transaction costs, and the more inefficient the pursuit of collective interests. Members of small groups tend to recognize the importance of their individual contributions to their personal interests and make an effort to achieve collective goals (Olson Reference Olson1965; Frohlich, Oppenheimer, and Young Reference Frohlich, Oppenheimer and Young1971). In the case of the Senate, the addition of states to the Union led the Senate to grow from 48 seats in 1821 to 66 seats in 1857, to 88 seats in 1889, and to 96 seats in 1911. It may seem more than coincidental that leadership emerged when the Senate approached 90 members.
In practice, we seldom find that increases in workload and size, by themselves, stimulate organizational innovation. While size and workload surely set the context for the activities of Senate parties, we rarely find that increases in size and workload per se are mentioned by senators as their motivation for organizational innovations. Nor does either workload or size change sharply in the short term. Instead, when more directly partisan or political factors motivate senators to improve their organizational effectiveness, workload and size may condition the form that innovation takes. For example, a sizable workload makes effective floor leadership important, particularly for the majority party, but the emergence of floor leadership occurred first in the minority party and was a by-product of the need for a coordinated strategy rather than an increase in the Senate’s legislative workload or size. Moreover, the free-rider problem is not severe in Senate parties. They are neither very large nor very small. They are large enough to make building coalitions and other legislative activities expensive in time and other resources. But the Senate remains small enough that the efforts of individual senators and factions can make a difference.
In her account, Munk argues that modern Senate floor leadership was the product of President Wilson’s need for a lieutenant to push his program through a factionalized, filibuster-prone Senate (Munk Reference Munk1974). With a small Democratic majority behind him in 1913 and a large legislative program, Wilson recognized that his chances of success were undermined by having a party divided between progressive and conservative forces. In the House, the speaker was the recognized party leader and a ready-made champion of the president’s cause. But the Senate majority party, without a central leader, promised to be ineffective. Wilson’s obvious solution, according to Munk, was to endorse a senator for a leadership role who shared his views on key issues and then rely on that senator to represent administration interests. Subsequent presidents then followed Wilson’s example, giving the floor leadership a distinctive role as a partner to the president. Relations with the president are one of the main responsibilities of Senate floor leaders, but, as we contend below, it is one of several duties attached to this position, and a focus on this one aspect of the position fails to account for the substantial development of this position in the years before the Wilson presidency and in realms beyond executive relationships.
Other scholars point out that an influx of new members can drive institutional change (Davidson and Oleszek Reference Davidson and Oleszek1977; Evans and Oleszek Reference Evans and Olezsek1997; Fenno Reference Fenno1997). According to this perspective, new members have no vested interest in the existing institutional arrangements and may even perceive a mandate to change the way their chamber and party operate. Large numbers of new members, too, can upset preexisting balances within or between the parties. As we lay out our own theory below, we draw on this insight too, considering how changes in the membership of the two parties can give rise to new forms of party organization.
Finally, a large body of legislative research argues that the influence of central party leaders over the policymaking process is related to the extent of ideological or policy differences between the parties (Cooper and Brady Reference Cooper and Brady1981; Sinclair Reference Sinclair1983, Reference Sinclair1995; Brady Reference Brady1988; Brady, Brody, and Epstein Reference Brady, Brody and Epstein1989; Smith and Deering Reference Smith and Deering1990; Rohde Reference Rohde1991; Aldrich and Rohde Reference Aldrich and Rohde1997, Reference Aldrich and Rohde1998; Brady and Epstein Reference Brady and Epstein1997; Aldrich, Berger, and Rohde Reference Aldrich, Berger, Rohde, Brady and McCubbins2002). This perspective holds that congressional parties are preeminently policy coalitions of varying degrees of cohesiveness and polarization (Aldrich and Rohde Reference Aldrich and Rohde1997, Reference Aldrich and Rohde1998). When intraparty cohesiveness is high and the distance between the parties is great, legislators are eager to license aggressive party leaders who will coordinate strategy in their interest. Members of internally divided parties, in contrast, tend to distrust strong leaders and prefer a more decentralized, committee-oriented policymaking process. The argument, usually termed “conditional party government,” was developed for the House of Representatives but has been extended to the Senate.
This attention to variation in party polarization influences our approach, but we diverge in emphasizing the creation and development of party institutions—caucuses, steering committees, leadership positions, and other forms of partisan organization—rather than how leaders make use of their powers, as is the focus in studies grounded in the conditional party government perspective. We also, of course, are examining a legislative body that lacks the majoritarian features of the House. In the House of Representatives, the speaker is selected by the full chamber and a simple majority controls the agenda, so, as power in the majority party becomes more concentrated, power within the House becomes more concentrated. But in the Senate changes within a party caucus, even within the majority party caucus, do not translate automatically to shifts of power within the chamber.
A theory of Senate institutional development requires understanding how Senate parties and leaders help solve the problems of collective action and coordination for their membership, problems long recognized by social scientists as basic to understanding human organizations and leadership (Frohlich, Oppenheimer, and Young Reference Frohlich, Oppenheimer and Young1971; Calvert Reference Calvert1987; Rohde and Shepsle Reference Rohde and Shepsle1987). Senators require means to coordinate activities such as setting the agenda, drafting legislation, mobilizing majorities, assigning members to committees, and winning elections. When senators’ political interests are differentiated by party, as they were from the early years of the Congress, solutions will be organized by party. But what form those party-based solutions take—and at what moments in time—is what needs elaboration.
Individual and Collective Goals
In developing a theory of institutional change in the Senate, we begin with the premise that legislators have goals not only as individuals—reelection, good public policy, and influence (Fenno Reference Fenno1973; Bullock Reference Bullock1976; Smith and Deering Reference Smith and Deering1990)—but also as members of an organized party. Each of the individual goals has a correlative collective goal:
1. As individuals value reelection, the party values a favorable reputation.
2. As individuals value good public policy, the party values the ability to shape legislative outcomes.
3. As individuals value influence, the party values majority control of the chamber.
The collective goals are grounded in the individual-level goals. Legislators seeking to enact certain policies are advantaged if their party wins majority control of Congress, its committees, and scheduling mechanisms. Legislators seeking reelection are advantaged if the public has a favorable view of their party’s legislative record and their party’s reputation. For parties, these collective goals are central to their creation and development, they justify the elaboration of party organizations, and they motivate the strategies of party leaders.
There is no one-to-one correspondence between individual and party goals. To be sure, we would expect changes in the relative importance of reelection and policy goals among party members to be reflected in a party leader’s priorities. Critically, however, the electoral and policy goals of the party are interdependent. A party’s reputation influences its electoral success, its electoral success influences its legislative record, and its legislative record influences its popularity. Winning elections helps a party create the coalitions necessary for passing or blocking legislation, and a legislative record helps generate the desired reputation essential to winning elections. The interdependence of party goals, whatever their origin in senators’ goals, requires that party leaders be attentive to a full array of party goals. In this way, the emergence of new party institutions, like the caucus and steering committees, and the creation of leadership positions all represent non-incremental changes in Senate party organization.
Interdependence of goals means that parties will pursue all three goals even if one of the personal goals that underpins them loses some of its importance to rank-and-file senators. Conversely, it means that party leaders will continue to pursue all three goals even if one of the personal goals gains greater importance for senators. In fact, an even stronger argument can be made: leaders will pursue all three party goals even if rank-and-file senators are motivated exclusively by one of the personal goals. Thus, the priority given to any of the party goals by party leaders will show more stability than variation in the importance or compatibility of individual goals might suggest. Even if we treat legislators as individually motivated by the desire to see their policy preferences reflected in law, fellow partisans would share an interest in both enacting their common policy interests and in maintaining or securing majority party status. They would care about the electoral fortunes of other party members. Consequently, they might be willing to exchange some of their policy aspirations for the electoral benefit of their party as a whole. It is precisely such exchanges that each party must address collectively, doing so by taking into account the broad electoral environment and expected behavior of the other party.
Obstacles to Achieving Party Goals
Party efforts to acquire public goods—that is, to achieve collective party goals—are subject to collective action and coordination problems (Baumol Reference Baumol1952; Coase Reference Coase1960; Olson Reference Olson1965; Frohlich, Oppenheimer, and Young Reference Frohlich, Oppenheimer and Young1971). The collective action problem arises when the successful achievement of party goals is not greatly influenced by the actions of any individual legislator. Consequently, the incentive for individuals to contribute a fair share to collective party efforts is small (Olson Reference Olson1965). With respect to passing or blocking legislation, the party needs a majority of legislators’ votes to win, so the effort is necessarily a collective effort. But the leadership necessary for corralling votes is concentrated in a small number of members. Considering their own reelection, the ordinary legislator cannot reasonably expect to materially affect the reputation of their party and would therefore focus their energies on campaigning at home rather than working on Capitol Hill to enhance the party reputation (Mayhew Reference Mayhew1974). A legislator might even reasonably expect a net gain in votes from criticizing or standing apart from their own party.
Senate parties must also address coordination and transaction costs. The party’s members must somehow coordinate their strategies or, alternatively, create a mechanism that will produce a choice from one of a few acceptable strategies. Coordination is costly. Meetings must be organized, responsibilities delegated, information disseminated, and leaders held accountable. Indeed, high transaction costs, relative to the expected gains from coordination, can prevent a collective choice and the realization of benefits from a public good (Coase Reference Coase1960; North Reference North1990). Organizational innovations, such as arranging for regular meetings of committees to make decisions for the larger party caucus, may reduce transaction costs and improve the collective efforts of a party.
Transaction costs vary over time. In a world in which the Senate has a modest workload and senators have time to spare, party caucus meetings can be held frequently and issues discussed at length without taking away from other efforts. As the volume of legislative business that is processed by committees and on the floor increases and as legislative and electoral activities take up more of senators’ time, members are likely to seek relief by relying on party leaders, committees, and staff to manage some matters on their behalf. Over the span of many decades, increasing transaction costs create incentives for senators to propose organizational responses to address them.Footnote 11 Moreover, the need for quick action by a party to assert or protect its interests may lead party leaders to take more initiative and suppress demands for full deliberations of the caucus. Institutional innovation is costly—in time and money, but also in transferring powers from individual members to collective party organizations and leaders. For senators to acquiesce in this delegation, they must be convinced of its necessity to further their own personal goals.
Leaders
If legislators operated independently, party goals would be pursued inefficiently. At least in the long run, then, competitive pressures from the other party encourage legislators to invent ways of organizing their party for collective action, and successful innovations in one party are readily adopted by the other. Parties create committees, task forces, and leadership positions, and they assign responsibility for achieving collective party goals to the legislators appointed or elected to those posts. Legislators need to be motivated to pursue these efforts, but the same mix of goals that motivate others—additional influence over policy outcomes, visibility useful in the pursuit of reelection or higher office, the sheer satisfaction of being in the middle of important events—may motivate at least a few legislators to attend to the collective interests of the party.
Of course, legislators may not fully trust their leaders and party committees. After all, power delegated to leaders and committees to pursue party interests might instead be used in pursuit of personal interests (Ross Reference Ross1973; Mitnick Reference Mitnick1975). Like all organizations, congressional parties address this threat—called the principal-agent problem or, more simply, the agency problem—in a variety of ways, such as by defining the responsibilities and jurisdictions of committees and leaders, assigning some decisions to multi-member committees or leadership groups rather than to individuals, subjecting leaders to periodic election, and requiring leaders to justify their strategies at caucuses or other party meetings (Kiewiet and McCubbins Reference Kiewiet and McCubbins1991). All these venues give the party rank-and-file an opportunity to define how party goals should be defined and pursued. As in other settings, though, leaders acquire advantages that give them considerable influence over the direction of party strategy.
Variation in individuals’ electoral and policy objectives and uncertainty about how events will unfold give rise to squabbles about party strategy. These arguments are articulated in contests for leadership positions, party meetings, and everyday conversations. The responsibility of party leaders is to further party goals while minimizing the severity of the trade-offs that are required. Giving priority to some legislation over other measures, at least until after the next election, is a common strategic choice of parties. This type of strategic choice is manifested in the activities of party leaders—managing the party organization, coordinating with the president or leaders of the other chamber, speaking on behalf of the party for the media and other audiences, setting staff priorities, managing floor activity, negotiating legislation within the chamber and with leaders of the other policymaking institutions, and even taking the lead in writing legislation and building majorities.
Why Parties Innovate
This brings us to our central question: What political conditions are associated with the elaboration of Senate party organizations? The general condition that promotes innovation in Senate party organizations is keen interparty competition in the legislative and electoral arenas (Gamm and Smith Reference Gamm, Smith and Loomis2000, Reference Gamm, Smith and Oppenheimer2002a, Reference Gamm, Smith, Brady and McCubbins2002b; Smith and Gamm Reference Smith, Gamm, Dodd and Oppenheimer2001, Reference Smith, Gamm, Dodd and Oppenheimer2020; Lee Reference Lee2016; Koger and Lebo Reference Koger and Lebo2017). The payoff for organizational innovation is likely to be greatest when the two parties are close to parity—when the parties are of nearly equal size and when party strength, due to the parties’ relative cohesiveness, is nearly equal. These are the times, we contend, that a party is most likely to make innovations in party organization, rules, or leadership to gain advantage over the opposing party. For a minority party, especially one with reasonable hope of attaining majority status, the drive to overcome its limitations with more effective legislative and electoral strategies stimulates innovations. For a majority party clinging to power, or having just come to power, we predict similar attention to organizational change.
Within the Senate, the most important obstacle to achieving party goals is the other party. Party competition is at the root of most major organizational developments in congressional parties. The incentive to innovate often is greatest in the minority party, which looks to improve its ability to compete in the legislative and electoral arenas and gain majority status. But beyond challenges from the other party, legislators’ goals often entail trade-offs that must be resolved or managed. Not all members of a legislative party share the same political interests, and a party’s policy and electoral goals may require setting priorities.Footnote 12 Even partially conflicting interests and priorities often generate factionalism, and resolving the conflicts entails collective action and coordination (Smith Reference Smith2007). Evolving challenges in pursuing party goals determine the functions that party organizations and leaders perform.
The focus on party competition and factions is not new, of course. Historians of American political parties have long made the interplay of the parties and the maneuvering of factions a central feature of their narratives. Holt’s magisterial history of the Whigs, to cite just one prominent study, begins with the observation that “no political party can be fully understood in terms of its own beliefs, actions, and internal quarrels. Its relationships with rival parties must also be incorporated into the analysis” (Holt Reference Holt1999, ix). Moreover, the Whigs, even more than mid-twentieth-century Democrats, struggled with factionalism that eventually made them the only mass major party to disappear. We, too, make interparty and factional relations central to our story.
Individual senators matter, too. Entrepreneurs—senators motivated by ambition or who simply take joy from deep engagement in the legislative process—attain leadership positions and, at times when the party is ready to accept organizational changes, champion innovations that advance both party and personal interests. Opportunism underlies many innovations in Senate party organization. A senator, or perhaps a small group of senators, recognizing that their party interests align with their personal or factional interests, invents new organizational forms, often (at least in modern times) advocating for these reforms in campaigns to win leadership elections. These entrepreneurs exploit occasions in which their party colleagues are open to redistributing power within the party caucus in order to serve collective goals. We are not arguing that great senators have, on their own, molded Senate parties. On the contrary, party competition and factionalism create opportunities and constraints that shape the incentives for individuals to promote their own electoral, policy, and career interests. The greater the opportunities and the looser the constraints, the more room for senators’ varied interests, backgrounds, and skills to affect organizational choices.
Organizational innovation in Senate parties is not continuous because it is costly. Structural change, after all, often creates winners and losers among the members and factions of a party, may take considerable time and effort to work out, and may harm interpersonal relations. It is most likely to occur when the short-term payoff, in terms of the goals of majority party control and winning legislative battles, is substantial. In fact, as we will show, competition in the legislative and electoral arenas often are closely connected and pursued in tandem. A party seeking electoral victories may seek to get credit, or avoid blame, with the electorate by passing legislation, blocking bills, or keeping issues off the Senate agenda entirely. It also may pursue its legislative goals by persuading the electorate to support its legislative initiatives.
There is nothing automatic about parties becoming organized to perform these tasks, in parties becoming “cartels” (Cox and McCubbins Reference Cox and McCubbins1993, Reference Cox and McCubbins2005). When the majority party is united, it can control the agenda-setting processes of a legislative body, produce legislation, and choose the officers and staff of the chamber. Jenkins and Stewart (Reference Jenkins and Stewart2013) demonstrate that it took decades for House majority parties to do this. Seemingly simple matters like the choice of a House speaker or a printer were not resolved within meetings of the majority party until the mid-nineteenth century. Control of the House floor by a recognized leader of the majority party, the speaker, was not established until the end of that century.
As we examine 235 years of Senate history, we find that the development of its party organizations has tended to be cumulative. There have been dead ends, to be sure—the effort to invest committee appointment powers in the presiding officer, as we will show, or the concentration of chamber leadership in the Republican steering committee at the turn of the last century. Complete party failure was experienced by the Federalists and the Whigs. But, most often, as new challenges generate new institutional arrangements, these are built upon inherited structures (Schickler Reference Schickler2001). The floor leadership position, invented in obscurity in the 1890s, has proved to be remarkably durable. As long as an innovation continues to serve the party’s interest, it will survive long beyond the life span of the senators who initially were most advantaged by it and well past the competitive circumstances of its origins. And imitation is a Senate party’s highest form of flattery. As parties have competed through the years, successful innovations initiated by one party are embraced and extended by the other party—a coevolutionary process. Competition between the parties, over many generations, has created in the U.S. Senate a remarkable hothouse of institutional tinkering and experimentation, and neither party has held a patent on its best discoveries.
The earliest photograph (a daguerreotype) of the Capitol. Taken in 1846, it shows the building before the new House and Senate wings were constructed during the 1850s and the new dome was completed in the 1860s.
Overview
The book proceeds chronologically, which allows us to emphasize the context, sequence, and timing of important developments in Senate party organization. In each chapter, our focus is on identifying those developments, providing detailed evidence, much of it newly discovered, to document each innovation. Crucially, given our theoretical approach, we specify when each new institution is created. Chapter by chapter, we examine, too, how institutional innovation relates to periods of competitive parties and, where relevant, the roles played by factions and entrepreneurs. The generally cumulative nature of party organizational development quickly becomes obvious. The political context evolves and leaders adapt, but some important elements of modern Senate party organization—above all, the primacy of the caucus, over both the floor and the rostrum, for solving collective action problems—can be traced back to the first half of the nineteenth century.
Senate party organization emerged and has changed in response to the challenge of securing common goals. For most of the nineteenth century, that meant simply achieving a measure of cooperation within the caucus. But, over time, solving collective action and coordination problems meant developing institutions that could provide active leadership on the Senate floor and extend the reach of the caucus beyond the chamber entirely. Since the middle of the twentieth century, modern Senate leadership has been defined by a set of discrete functions or activities (Ripley Reference Ripley1969a, Reference Ripley1969b):
Party management. Each party has conference meetings that must be organized and led; a staff, budget, and office space that must be managed; and party committees that must be appointed.
Floor management. Each party has assigned responsibilities for planning a floor schedule, making motions on behalf of the party, and overseeing floor activity.
Intermediary with the president and House. Each party is represented in regular and irregular meetings at the White House and in relations with other administration officials, House leaders, and others.
Party spokespersonship. Each party’s leaders serve as spokespersons in public and media venues and maintain a public relations operation, sometimes in the form of a party committee or task force.
Coalition building. Each party’s leaders seek to build coalitions to pass or block legislation, often in alliance with the president, interest groups, and others.
These activities are how leaders promote the party’s collective goals of enhancing the party’s reputation, gaining or holding majority status, and pursuing a legislative agenda. As the book proceeds, we are attentive to these different functions. In early chapters, our emphasis, like that of senators through most of the nineteenth century, is on the tasks of party management and coalition building, primarily in the legislative realm. Starting in the 1890s, however, senators created leadership positions that began to assume responsibility for all five functions and expanded their reach to elections and the party brand, so, in examining Senate organization since the late nineteenth century, we focus on how new institutions came to perform these activities.
We begin our account with the presiding officer, in Chapter 2. The failure of the vice president—even more, the failure of the president pro tempore, who unlike the vice president is elected by the Senate—to become a party leader is a puzzle. Perhaps the weakness of the vice president was preordained by the inability of senators to hold him accountable, but, even so, this development took decades. However, as someone chosen by their fellow senators—selected, like the speaker of the House, and eventually by the majority party’s caucus and ratified by the full chamber—it is not immediately clear why the president pro tempore did not consolidate in the Senate powers comparable to those of the House speaker. Indeed, in the late 1830s and early 1840s, a full half-century after the first Congress, the president pro tempore, like the speaker, was routinely exercising the power to name the chairs and members of all standing committees; elections for president pro tempore were contested and closely watched; and no seniority rule existed. But, as we show in Chapter 2, the influence of the president pro tempore over the Senate’s affairs reached its zenith in the early 1840s. Although senators continued to refine the office’s responsibilities for maintaining order and enforcing rules, the position of the Senate’s presiding officer never fulfilled its early promise.
In its place, the party caucus emerged. As we show in Chapter 3, senators met in caucus only infrequently in the first half-century of the chamber’s history. From the 1790s through the 1830s, caucus meetings were controversial affairs, seen by many as illegal assemblies of public officials, and meetings rarely focused on legislative business. The most notable use of early caucuses was by Republicans in the two chambers, who gathered on a quadrennial basis to nominate their presidential candidate. But, in 1841–45, led by the Whigs, the modern Senate party caucuses were born. In that brief period of time, senators began utilizing caucuses on a regular basis, both to conduct legislative business and to organize the membership of the Senate’s standing committees. In later years, first in 1856–62, then in the 1870s, Republicans and Democrats gave new structure and permanence to their caucuses. They created formal positions in the caucus, such as a regular caucus chairman, and caucus committees, including committees on committees, campaign committees, and ad hoc committees on the order of business. From time to time, individual senators emerged as de facto leaders in the chamber, but these senators—people like Thomas Hart Benton (Mo.), Henry Clay (Ky.), John C. Calhoun (S.C.), and Daniel Webster (Mass.)—did not serve their caucuses in any formal, elected capacity.
The emergence of the Senate caucus in 1841–45 coincided with the rise of a competitive two-party system, both in the electorate and in the Senate. Whigs gained their first Senate majority with the 1840 elections, and, as they organized the Senate in 1841, they relied heavily on the caucus as an institution. As Figure 1.1 shows, the early and mid-1840s was a time of intense party competition between the nation’s first two broad-based political parties, the Democrats and the Whigs. Between 1839 and 1847, the majority party never held more than 34 seats and the minority party never had fewer than 22 seats. The Democrats controlled the Senate until 1841, then the Whigs were in the majority for four years, and then the Democrats regained control in 1845. From 1845 through the late 1850s, the Senate was dominated by the Democratic party. Opposition parties—whether Whigs, Know-Nothings, Free Soilers, or early Republicans—represented a small minority in those years. But, with the invention of the Republican party in the mid-1850s, Democrats came to face a new threat. Republicans, who held one seat in the 34th Congress (1855–57), held 20 seats in the 35th Congress (1857–59), 26 seats in the 36th Congress (1859–61), and, with the secession of southern states, a large majority starting in 1861.
Size of Senate parties, 1789–1889
Figure 1.2 shows the relative size of Senate parties from the organization of the Republican party in the 1850s through the present day. Several periods, scattered over time, exhibited near parity in party strength. The first came in the aftermath of Reconstruction, from the mid-1870s through the mid-1890s. Then, in the 1911–21 decade, came a second era of close competition, with three Democratic majorities and two Republican majorities, all of them narrow. Small Republican majorities in 1927–29 and 1931–33 were surrounded by larger majorities in the 1920s and large Democratic majorities in the 1930s and early 1940s. Republican election victories in 1946 then initiated a period of intense competition and small majorities lasting through the 1950s. Democrats controlled the chamber by large margins in the 1960s and 1970s. The 1980 election proved to be a dramatic turning point: for more than 40 years since then, the parties have regularly alternated control of a Senate in which the majority party has never exceeded 60 seats and averaged fewer than 54 seats.
Size of major Senate parties, 1857–2025.
As Chapter 4 discusses, the closely divided Senates between the late 1870s and early 1890s was a period of remarkable innovation in the Senate. In 1881, each party controlled the same number of seats, something that had never before occurred in Senate history. It was in that Senate, in 1882–83, that Republicans invented a wholly new institution, which they gave the name “steering committee.” As we show, the Republican steering committee built on precedents established by both parties, but the institution that emerged in 1882–83 was truly novel. Over the next decade, both parties, but especially Republicans, regularly turned to steering committees to manage their caucus’s business. Then, in 1892–93, the Republican steering committee was strengthened and redefined, making it the dominant institution of the dominant Senate party for the next two decades. Facing the prospect of losing their majority following the 1892 elections, Republican senators mobilized, first to try to influence the state legislatures picking new senators and then to manage the legislative process. In Chapter 4, we detail the reorganization of the steering committee in 1892–93 as a permanent, ongoing standing committee of the Republican caucus. Created by a competitive party suddenly in the minority, the committee came to eclipse the caucus itself once Republicans regained majority status in 1895. During these years, Republican leadership in the Senate was centered in a handful of men: Nelson Aldrich (R, R.I.), William Allison (R, Iowa), Orville Platt (R, Conn.), Eugene Hale (R, Maine), and John Spooner (R, Wisc.). The group functioned as an interlocking directorate of Republican committee and party leaders, with Allison and Hale serving as caucus chairmen for most of this period and all but Platt serving regularly on the steering committee.
The Senate parties had been competitive since the 1870s, but, in the early 1890s, the intensity of competition was exacerbated by the waning battle by Republicans to secure voting rights for Black men in the South. For years, white southerners had worked to erect new structures of white supremacy and racist oppression, undermining Reconstruction-era gains for civil and political rights for southern Blacks. The Federal Elections Bill of 1890–91 represented the last attempt, for the next two-thirds of a century, to protect the right to vote. Chapter 5 examines this battle and its transformative impact for the Senate. Faced with the prospect of reenfranchised Black voters, an overwhelmingly Republican group, and thus the end of Democratic hegemony in the South, Senate Democrats regarded the battle against the bill as an existential fight. Leading them in battle was Arthur Pue Gorman (D, Md.), who, in the process of defeating the Federal Elections Bill, honed the filibuster as a weapon of minority obstruction and transformed the position of caucus chairman, at least for the minority Democratic party, into a position of elected floor leadership. Modern party leadership in the Senate, we show, traces its birth to Gorman and to this battle.
With the 1912 elections, 18 years of Republican control of the Senate came to an abrupt end. In Chapter 6, we examine the institutional inventions of the 63rd Congress (1913–15), when, in a newly competitive world, the Democratic floor leader—John W. Kern (D, Ind.), a progressive senator closely allied with the newly elected president Woodrow Wilson—became the first person widely regarded as an elected majority leader of the Senate, with responsibility for devising and implementing party strategy. The Republicans, now in the minority, created their own position of elected floor leader in 1913, following the generation-old Democratic model, and both parties invented the position of whip. Other developments, such as the emergence of the modern use of unanimous consent agreements, the creation of party floor staff, and, for the Democrats, the practice of entrusting committee assignments to their leadership, were accomplished in the 1910s and 1920s.
Between the mid-1920s and mid-1940s, the two Senate parties strengthened the position of floor leadership, building on the foundations laid by Gorman in the 1890s and on the innovations of the 1910s. This era of consolidation—a period first of Republican dominance, then of extended Democratic dominance, interrupted by a short period of competitiveness—is the subject of Chapter 7. Charles Curtis (R, Kans.), who became Republican leader in 1924, elaborated leadership posts as he navigated factionalism within his party. The New Deal elections left the Republican conference depleted—down to 16 members in 1937—so they minimized their formal organization in the 1930s and waited until 1944 to reinvent it. Democrats in this era were led by Joe Robinson (D, Ark.) and Alben Barkley (D, Ky.), who centralized power in the floor leadership position and wielded it effectively but otherwise made few organizational changes.
Informed observers of the Senate may view the 1945–80 period as the doldrums of Senate party history, a period of stasis, the unchanging “textbook Congress” (Shepsle Reference Shepsle, Chubb and Peterson1989). The study of Congress in these years has long emphasized the role of weak parties and a decentralized policymaking process dominated by committees and their chairmen. However, as we discover in Chapter 8, the parties under Robert Taft (R, Ohio) and Lyndon Johnson (D, Tex.) were dynamic, establishing organizational foundations that shaped future developments and patterns of leadership behavior. We emphasize four developments, none of which have been emphasized in other accounts: evolving new venues for collective decision-making, expanding staff resources, adopting new committee assignment practices, and establishing more consequential campaign committees. These developments were responses to the parties’ electoral challenges, factional problems, and personal interests of key leaders.
The 1970s and 1980s proved to be transitional years. At first, it appeared that individualism had taken hold in the Senate, undermining the influence of both leaders and standing committees, but, as Chapter 9 shows, partisan conflict intensified during the 1980s and senators began demanding more effective party organizations and leadership. The Democratic leader, Robert Byrd (D, W.Va.), was challenged for Democratic floor leader in 1986 and two years later gave up his post because he was not meeting his colleagues’ expectations as a team leader and party spokesman. By the mid-1990s, party leaders not only served as their parties’ chief strategists on floor procedure but increasingly guided the tandem tasks of writing important legislation and fashioning media strategies. Procedural maneuvering intensified on the Senate floor, which placed floor leaders in the middle of every legislative battle. By the 2020s, Senate policymaking was remarkably centralized in the two floor leaders, Mitch McConnell and Chuck Schumer. The Senate reached 2024 with stronger central party leaders and more elaborate organizations than ever before.
Conclusion
The ability of Senate floor leaders to marshal legislative, public relations, and campaign resources on behalf of their parties has never been greater than it is today. The two floor leaders are the central players in performing each of the major functions of their parties, and the organizational apparatus of each party centers on them. No earlier senator, not Henry Clay or Nelson Aldrich or Lyndon Johnson, had an array of resources comparable to those available to Reid, McConnell, and Schumer. How modern floor leaders use their resources and perform those functions remains contingent and circumscribed, conditioned in large part by the demands placed on them by party colleagues. In recent decades, intensifying partisan polarization and political competition has produced demands for effective party strategies that leaders are expected to devise and implement, which has prompted leader-generated legislative agendas, leader-led negotiations on major legislation, and leader-centered public relations efforts, all backed by enlarged leadership staffs.
Party organization and leadership roles have a cumulative character. In this regard, we diverge from the literature on conditional party government, which suggests that powers accumulate when parties are strong and polarized, and then dissipate when partisan coalitions overlap. As we argue instead, leaders rarely give up resources that they have inherited. They use what they are given and make incremental improvements, and then, occasionally, exhibit a willingness to remodel party organization and resources when their colleagues allow or even demand it. Today’s Senate leaders, as a consequence, benefit not only from an era of minimal factionalism and high levels of party competition but also from the accretion of earlier innovations. On the House side of the Capitol, party leadership rests in the speakership, in a position defined in the Constitution of 1787 and well-developed as a party institution by the time of the Civil War. Senate party organization and leadership, in contrast, are endogenous to the Senate itself. The floor leaders of the 2020s are inheritors of 235 years of tinkering, of dead ends long abandoned, of lessons learned and remembered, and of a multitude of useful scraps that are now part of the institution’s warp and woof.
