2.1. The public choice approach

Important developments in science frequently occur at the boundaries of disciplines when scholars from two or more fields address old questions in new ways. Important scholarly works at the borders of economics and political science included the publication of Kenneth Arrow's Social Choice and Individual Values in 1951, Anthony Down's An Economic Theory of Democracy in 1957, Duncan Black's The Theory of Committees and Elections in Reference Black1958, James Buchanan and Gordon Tullock's The Calculus of Consent in 1962, and Mancur Olson's The Logic of Collective Action in 1965. This generated a new approach – called public choice – to be developed by a group of economists, political scientists, and sociologists who have used methods originally developed in economics to examine the constitution of order in the public sector.

An organizing question underlying work in the public choice tradition has been: What incentives do actors face when making decisions in the public sector? Having identified these incentives, public choice theorists predict how different individuals will act and how individual behavior will aggregate into collective outcomes (see, for example, Shepsle, Reference Shepsle1979, Reference Shepsle and Weisberg1986; Weingast, Reference Weingast1989; Weingast, Shepsle, and Johnsen, Reference Weingast, Shepsle and Johnsen1981; Krehbiel, Reference Krehbiel1988; Diermeier, Reference Diermeier1995; Acemoglu and Robinson, Reference Acemoglu and Robinson2006, as a small sample of this important work). Incentives result from the structure of a situation that is affected by the type of goods involved, combined with attributes of a community and the rules used for making decisions about allocation, production, distribution, and consumption of those goods.

The early work of public choice theorists challenged the notion that order in the public sector stemmed only from central direction. Instead of prejudging the performance of complex organizations in metropolitan areas, Vincent Ostrom, Charles Tiebout, and Robert Warren proposed in 1961, for example, that the multiplicity of local jurisdictions in a metropolitan area be conceived as a ‘polycentric political system’.

This brought a different perspective to the study of governance. Instead of presuming the existence of only two kinds of order – the market and the government – political economists have come to recognize that order can be achieved in public economies where large, medium, and small governmental and nongovernmental enterprises engage in both competitive and cooperative relationships.

The public choice or ‘rational choice’ approach was quickly adopted by many political scientists who rapidly developed and applied it to the study of many different types of institutional arrangements, including legislatures (Riker, Reference Riker1962; Shepsle, Reference Shepsle1989), voting rules (Rae, Reference Rae1969), and bargaining (Riker, Reference Riker1967) (for an excellent overview of rational choice theory as of the early 1980s, see Barry and Hardin, Reference Barry and Hardin1982). The approach was strongly criticized by other political scientists, however, because it was thought to portray all human behavior as narrowly self-interested and shortsighted (Green and Shapiro, Reference Green and Shapiro1994).

2.2. The tragedy of the commons

Another important interdisciplinary development is the analysis by both ecologists and economists of ‘the tragedy of the commons’ (Hardin, Reference Hardin1968; Gordon, Reference Gordon1954). This theory, like that of Mancur Olson, challenged the earlier ‘group theorists’ (e.g. Truman, Reference Truman1951; Bentley, Reference Bentley1935) who presumed that individuals were motivated to contribute to efforts to solve common problems when it was in their long-term interest to do so. Gordon and Hardin saw individuals as focused on maximizing their immediate short-term benefits. This meant that they were helpless to do anything else but overharvest when they jointly used a resource system that was not privately owned or the property of a governmental unit. The prediction that individuals would destroy the very resources on which they depended was consistent with that of noncooperative game theory models of one-shot or finitely repeated dilemma settings where everyone pursuing their own short-term benefits ended up achieving far less individually and together than was feasible if they had found a way of cooperating with one another.

This work opened up a new body of theoretical and empirical work to challenge the presumption that individuals were forever trapped in a remorseless tragedy (McCay and Acheson, Reference McCay and Acheson1987; Berkes, Reference Berkes1985; National Research Council, 1986). Many common-property institutions around the world were documented where individuals had overcome the tragedy. No ‘sure cures’ for the problem were found, however, as failure occurred in regard to private property, government property, and common property. Overharvesting was assured whenever a resource was effectively an open-access resource. Gordon and Hardin, and the myriad of scholars and policymakers from multiple disciplines who accepted this theory as a general theory, were thus correct in identifying a challenging problem, but their analysis was incomplete.

2.3. ‘New’ institutional economics

The ‘new’ institutional economics field has been a major challenge to both economists and political scientists (even though two of its key progenitors – Coase and North – received the Nobel Prize in Economics).Footnote ² Ronald Coase started the first foray in Reference Coase1937 with his article on ‘The Nature of the Firm’. By asking ‘Why do firms exist?’, he asked an embarrassing question for economists. Why should one find firms existing in the midst of highly competitive markets for strictly private goods? He challenged the presumed dichotomy of the world into markets for production, allocation, and distribution of private goods and hierarchies for the production, allocation, and distribution of collective goods. Coase answered his own question by pointing to the costs that are associated with all forms of organization that had not been included in economic models of the market. These transaction costs could be substantial and would lead entrepreneurs to try out alternative forms of organization even when they were dealing with private goods.

Influenced by Coase, Douglass C. North (Reference North1981, Reference North1990a, Reference North1990b) initiated a series of studies of institutional change that challenged the static focus of both political science and economics. He examined the growth of the transaction sector in the American economy over century-long periods and demonstrated that transaction costs were a more important factor than production costs in a modern economy. Then he and colleagues turned to the evolution of property rights in the Middle Ages and how the evolutionary process was affected by the need to reduce uncertainty in relationships and the costs of transactions (Milgrom, North, and Weingast, 1990). They demonstrated that a variety of self-organized institutions had guaranteed the property rights of merchants long before the creation of a state. He and other scholars in this tradition are currently trying to expand the model of human behavior used to explain institutional behavior so as to be broadly consistent with important developments in cognitive science (Denzau and North, Reference Denzau and North1994; North, Reference North2005). Oliver Williamson (Reference Williamson1975, Reference Williamson1985) also took up Coase's challenge to examine a diversity of internal mechanisms within firms to keep agents accountable and to reduce transaction costs. Williamson provides one of the clearest presentations in the literature of the dysfunctions of a strict hierarchy. Research in political science on principal-agent theory (Moe, Reference Moe1984; Miller, Reference Miller1992) owes a great deal to the pathbreaking work of the transaction cost economists.

2.4. Behavioral game theory

By the 1960s, game theory became one of the standard tools used by many political economists, while game theory was frequently dismissed by some political scientists as just another form of rational choice theory. While some economists had dismissed game theory in its early days as irrelevant given the theoretical tools already developed to explain market behavior, it has become one of the accepted tools used by most economists.

That one can use the same set of tools to analyze a game of tennis, the decision of when to run for office, how much to trust a stranger, and how much to contribute to a public good or to sustain a common-pool resource, makes this one of the most important analytical tools available to all of the social sciences. What has made game theory such an important tool for institutional analysts, however, is not just the theory but the development of an experimental tradition that enables scholars to test predictions under controlled settings that can be replicated by others and modified to examine the impact of specific rules or other important elements that affect decisions and outcomes in complex settings. Behavioral game theorists have slowly developed an extremely careful set of methods that have greatly improved the reliability and veracity of their results over time. Vernon Smith (Reference Smith1976) first developed an early ‘rule book’ for how experiments should be run (to see how the method has evolved, see Davis and Holt, Reference Davis and Holt1993; Kagel and Roth, Reference Kagel and Roth1995; Camerer, Reference Camerer2003; Holt, Reference Holt2007).

Behavioral game theory has evolved into a useful empirical body of research that has supported many propositions derived from noncooperative game-theoretical models as well as challenging others. Because experimentalists have consistently found game theory predictions to hold in regard to many market-like settings (see, for example, Smith and Walker, Reference Smith and Walker1993), behavioral game theory has not been a major challenge to the core of economic theory related to markets. On the other hand, very simple games that generate crystal-clear predictions have not been supported in a large number of experimental settings around the world – including the extension of trust and the division of benefits in settings that lack external enforcement, the provision of public goods, and the appropriation from common-pool resources (Henrich et al., Reference Henrich, Boyd, Bowles, Camerer, Fehr and Gintis2004; E. Ostrom, Gardner, and Walker, Reference Ostrom, Gardner and Walker1994). Findings from multiple experiments have been inconsistent with a theory that presumes individuals maximize short-term, objective payoffs to self alone. Thus, findings from these experiments challenged the core assumptions of microeconomics as well as those of public choice and rational choice theorists in political science.

The last half of the twentieth century was a time of challenge. Political scientists and economists have reacted in multiple ways to these challenges. Some have continued on their own paths, undaunted by the calls for change. On the other hand, many have attempted to build on the core of the challenges to craft an interdisciplinary approach to the study of the governance of public economies. New frameworks, theories, and models have been developed and now underpin considerable empirical research in both the field and the experimental laboratory.

Slowly, a new foundation for the social sciences – and hopefully for public policy – is emerging. It is not yet broadly accepted, but many scholars are independently drawing on, improving, and extending this work. In Section 3, I will provide a brief overview of some of the conceptual tools that have cumulated during the last several decades of the study of self-governance and public economies that are the foundation for the empirical results presented in Section 4.

3.1. An action arena at the core of analysis

The core analytical unit of institutional analysis is an ‘action arena’ in which participants (e.g. individuals, families, firms, voluntary associations, governmental units) interact in a structure of incentives generated by the characteristics of the goods involved, the rules-in-use, and the attributes of the community of participants involved (E. Ostrom, Reference Ostrom2005). Participants possess varying levels of information, potentially diverse preferences and norms of behavior, and diverse time horizons. The arena may be represented as a formal game, an agent-based model, a detailed case study, or an analytical narrative.

An action arena is affected by three clusters of variables that are treated as exogenous for any specific analysis of the incentives and likely behavior within an arena, but become endogenous variables when analyzing institutional change. These variables are the characteristics of goods, the rules-in-use, and the attributes of the community of participants. Performance in regard to efficiency, equity, adaptation, and robustness is not only an attribute of the human system of relationships that is generated but how that system is related to specific biophysical and social domains.Footnote ³

3.2. Characteristics of goods

Two basic attributes are now widely used to distinguish among goods and services in regard to their provision and production: difficulty of exclusion and subtractability (V. Ostrom and E. Ostrom, Reference Ostrom, Ostrom and Savas1977).

3.3. The difference between the provision and production of goods

A crucial problem in obtaining public goods and common-pool resources relates to the provision of the good rather than to the production of the good. If large groups of individuals wish to consume collective goods, it is difficult to rely on voluntary arrangements over long periods of time to (1) obtain revenue in a fair and equitable manner, (2) articulate demands, (3) allocate the goods and services to some individuals and exclude others, (4) regulate the patterns of use among the community of users, and (5) monitor the performance of producers. These five activities relate to the provision side of a public economy and not to the production side. The production side involves the transformation of input units (land, labor, and capital) into particular goods and services.

In a private market, provision activities take place as individual buyers engage in quid pro quo relationships with sellers of goods and services. When exclusion is low-cost to the supplier, preferences for the amount and quantity of a good are revealed as a result of many quid pro quo transactions. Producers learn about preferences through the consumers’ willingness to pay for various goods offered for sale. Where exclusion is difficult, designing mechanisms that honestly reflect beneficiaries’ preferences and their willingness to pay is difficult and complex. In very small groups, those affected are usually able to discuss their preferences and constraints on a face-to-face basis and to reach a rough consensus. In larger groups, provision decisions are apt to be made through mechanisms such as voting or the delegation of authority to public officials. The extensive literature on voting systems demonstrates how difficult it is to translate individual preferences into collective choices that adequately reflect individual views (Arrow, Reference Arrow1951; Shepsle, Reference Shepsle1979; Buchanan and Tullock, Reference Buchanan and Tullock1962).

When citizens establish an organization with the authority to use sanctions against those who do not contribute resources toward the provision of a collective good, they constitute a provision or collective consumption unit. Many provision units have the formal status of a government established at a local, regional, or national scale. Governmental units may be general-purpose or organized as a special district or regime for the purpose of providing one or a limited range of collective goods. Private associations that sanction, or even expel, those who do not contribute their share of resources to provide for a collective good are also collective consumption units. Sports leagues and housing condominiums are two types of private associations that provide collective goods for their members.

Once a collective consumption unit is established, how production is organized is an entirely separate question. A collective consumption unit is faced with at least six different institutional arrangements for arranging for the supply of local public goods. These include: (1) establishing and operating its ‘own’ production unit, (2) contracting with a private firm, (3) contracting with another governmental unit, (4) obtaining some services from its own production unit and other services from other governmental or private producers, (5) establishing standards of service that must be met by authorized producers and allowing each consumer to select a private vendor and to procure services from an authorized supplier, and (6) issuing vouchers to families and permitting them to purchase service from any authorized supplier. All of these arrangements are used by collective consumption units at a local, regional, national, or international level to arrange for the production of particular collective goods (Gibson et al., Reference Gibson, Andersson, Ostrom and Shivakumar2005). Given both the diversity of collective consumption units and the diversity of mechanisms each can use to arrange for production, institutions for the governance of public economies and other regions could be expected to derive significant advantage through complex patterns of organization.

Provision systems often do not resemble the textbook versions of either a government or a strictly private-for-profit firm, especially when participants have constituted their own self-governing units. Thus, scholars drawing on traditional conceptions of ‘the market’ and ‘the state’ have not recognized them as potentially viable forms of organization and have either called for their consolidation into a centralized government (as metropolitan reformers continue to do) or ignored their existence (as many resource economists have done). It is a bit ironic that many vibrant self-governed institutions have been misclassified or ignored in an era that many observers consider one of ever-greater democratization.

3.4. Further attributes of collective goods affecting the organization of provision and production units

To explain why citizens and their officials have organized such diverse sets of enterprises in local, regional, and international systems, one needs to examine attributes of collective goods, in addition to those of exclusion and jointness, which affect the costs and benefits of provision and production activities. Goods that are capital-intensive, for example, can achieve economies of scale in larger production units, while labor-intensive services can be produced at lower average costs in smaller production units. In addition to economies of scale in production, other attributes of goods may affect how provision and production activities are accomplished.

Whether a good needs to be coproduced or not is an important factor leading to different outcomes depending on the way public economies are organized (see Parks et al., Reference Parks, Baker, Kiser, Oakerson, Ostrom, Ostrom, Percy, Vandivort, Whitaker, Wilson and McGinnis1999; E. Ostrom, Reference Ostrom1996). Education and other public services cannot be fully produced by a production unit entirely by itself. The teacher in the classroom is an essential aspect of the production of education but so is the effort made by students and their parents, siblings, and friends. Coproduction of services is more difficult to maintain when provision units are extremely large and heterogeneous and production units are isolated from the input of those for whom the service is intended.Footnote ⁴

Given that the size of the group jointly consuming a good may vary from a small neighborhood to an entire nation or larger, one should expect to find citizens creating small, medium, and large collective consumption units and not relying on a single scale of organization. Unless appropriate boundaries can be established, effective means for gaining information about citizen preferences can be devised, and appropriate means for generating revenue can be designed, instruments of coercion can be used to create more harm than good. This is particularly a problem when individuals have little choice about whether they consume particular local collective goods or not.

3.5. Commonly understood and enforced rules

A key finding of empirical field research is the multiplicity of specific rules-in-use found in operational settings related to the provision and production of collective goods. An important type of rules is boundary rules, which determine who and what is in and out of a provision organization. The important attribute of boundary rules is the degree of match between the provision organization and the local situation rather than the specific rule used (E. Ostrom, Reference Ostrom2005, Reference Ostrom and Sabatier2007).

Some provision units face considerable biophysical constraints when the good is a natural common-pool resource such as a groundwater basin or a river. Such resources have their own geographic boundary. Creating a match between the boundary of those who benefit and are required to contribute and the boundary of the resource may be impossible in a highly centralized regime. Further, common-pool resources may themselves be nested in an ever-larger sequence of resource units. A micro watershed is nested in a system of ever-larger watersheds that eventuates into a major river system such as the Rhine or the Mekong River (Myint, Reference Myint2003). On the other hand, the biophysical world does not have a strong impact on the efficacy of using diverse boundaries for the provision of public education or police response services. One is constrained more by the transaction costs of reaching decisions and monitoring performance.

Once basic boundary rules define who is a legitimate beneficiary and who must contribute to the provision of a collective good, provision units frequently create rules related to the information that must be made public or kept secret, to the actions that must or may be taken or are forbidden, and the outcomes (and resulting benefits and costs) to be achieved and distributed. To be effective, rules must be generally known and understood, considered relatively legitimate, and thus in general to be followed and enforced. Written legislation or contract provisions that are not common knowledge do not affect the structure of a particular action situation unless someone involved in the situation invokes the rule and finds someone to enforce it. Thus, a problem with doing research on the effect of diverse institutional arrangements is sorting out the rules that exist on paper but are not used by participants as contrasted with rules that are common knowledge of the participants and enforced locally but not part of the formal legal structure. When scholars refer to a system as having a ‘rule of law’, they mean that most of the rules used in practice are consistent with the rules-in-form detailed in legislation, court decisions, and administrative procedures (Sproule-Jones, Reference Sproule-Jones1993).

In the interests of space, and since I have discussed rules in considerable detail elsewhere, the reader is referred to Crawford and Ostrom (Reference Crawford, Ostrom and Ostrom2005) where detailed analyses of rules are presented. The crucial point is that rules affect the structure of the situation under analysis. One should expect to see differences in the incentives and likely behavior when one configuration of rules is used versus another (see also Aoki, Reference Aoki2001, for a good discussion of the diversity of rules needed to cope with the array of problems humans face).

3.6. Attributes of a community

Many attributes of a community are also likely to affect provision activities, including the size of the group affected, the homogeneity or heterogeneity of interests, the patterns of migration into or out of a community, and the discount rate used by individuals in ongoing situations. The specific attributes of a community that might affect outcomes could be very large. An institutional analyst, however, needs to know answers to the following set of questions to begin an analysis of an action arena:

1. • Is there general agreement on the rules related to who is included as a member with both benefits and responsibilities?

2. • Do participants have a shared understanding of what their mutual responsibilities are as well as the formulae used for distribution of benefits?

3. • Are these rules considered legitimate and fair?

4. • How are the rules transmitted from one generation to the next or to those who migrate into the group?

A diversity of community attributes affects the answers to these questions.

3.7. Polycentric-linked systems

Once one recognizes that the organization of the provision of collective goods is the crucial step in obtaining or sustaining collective goods, one can begin to build a different theoretical explanation of political orders. One form of political order is created when a central power uses a monopoly of force to impose its central will on its subjects. The work of Margaret Levi (Reference Levi1988) convincingly demonstrates how costly and unstable it is to rely strictly on a central monopoly of force to govern effectively.

Other forms of order at all levels of organization exist and need serious study. Robert Putnam (Reference Putnam2000; Putnam et al., Reference Putnam, Leonardi and Nanetti1993) has demonstrated the importance of networks in the creation of social capital that enables complex governance systems to evolve without all links being planned from the top (see also E. Ostrom and Ahn, Reference Ostrom and Ahn2003). Robert Keohane (Reference Keohane1984, Reference Keohane2001) views political order at the international level without depending on the hegemony of one state dominating the entire system.

We need to include in our analyses institutional arrangements that are not encountered in basic economics and political science textbooks that tend to focus exclusively on markets or the state. In addition to general-purpose governments, a host of other organizations may undertake the provision of collective goods. Sometimes these will be bowling leagues. Other times they will be groups of lobster fishermen (Acheson, Reference Acheson2003). They may be international regimes (McGinnis and Williams, Reference McGinnis and Williams2001). And, given that the appropriate scale for organizing the provision side is not always the same scale as for organizing the production side of any collective good, one should expect to find messy polycentric systems of provision and production units operating at multiple scales in regard to almost any collective good.

4.1. Public economies in metropolitan areas

In regard to the organization of governments serving a metropolitan area, scholars in the public choice and new institutional economics tradition have analyzed why complex systems perform better than simple and highly centralized systems. The optimal scale of organization for the production and provision of goods and services differs radically for different types of collective goods and services. This leads to the prediction that public economies in metropolitan areas with many governmental enterprises organized at diverse scales will perform more effectively than either one large-scale (metropolitan-wide) government or a large number of governments organized only at a small scale.

Considerable research on local public economies, and multi-level systems more generally (see Houghe and Marks, Reference Houghe and Marks2001), has demonstrated that effective social order can emerge from polycentric systems and need not be imposed by a centralized system (Poel, Reference Poel2000; McGinnis, Reference McGinnis1999). Individuals are not able to engage in a wide diversity of independent quid pro quo relationships with any vendor they choose, but rather receive collective goods from the producer chosen by a provision unit. Unlike markets, however, thick relationships exist among entities in a local public economy. One must examine structure and performance at an interorganizational level of analysis as well as the level of a single firm or governmental units.

As an illustration of the research undertaken in this vein, allow me to briefly present the core results of a 15-year research program that examined the impact of the structure of public economies related to the provision of police in metropolitan areas.

4.2. Studying common-pool resources

Since the dramatic metaphor created by Hardin (Reference Hardin1968), many theoretical studies of common-pool resources have continued to analyze simple systems using relatively similar assumptions. Users are assumed to be short-term, profit-maximizing actors who have complete information and are homogeneous in terms of their assets, skills, discount rates, and cultural views. In this theory, anyone can enter the resource and take resource units and overharvesting results.

4.3. Testing predictions of the standard model of the individual

In addition to experimental research that has examined the level of cooperation achieved in stark collective-action settings that has challenged the validity of the standard theory, research in the behavioral game theory tradition, focusing on ultimatum, dictator, and trust games, examines whether individuals behave as the neoclassical model of individual choice predicts.

Based on repeated experiments conducted in Germany, Indonesia, Israel, Japan, Slovenia, Slovakia, and the United States (Güth, Schmittberger, and Schwarze, Reference Güth, Schmittberger and Schwarze1982; for a review, see Camerer and Thaler, Reference Camerer and Thaler1995; Camerer, Reference Camerer2003), subjects in these experiments do not behave as classical game theory predicts. Modest differences do occur in the average sum offered or rejected in different cultures (Roth et al., Reference Roth, Prasnikar, Okuno-Fujiwara and Zamir1991; Slonim and Roth, Reference Slonim and Roth1998). The overwhelming finding is, however, that Proposers in ultimatum games tend to offer around half of the sum and Responders tend to reject low but positive offers. This is even true when the amount of payoff is very high. When a sample of Indonesian subjects was assigned a provisional sum of three-months wages (200,000 Rp), 56% of the Proposers allocated between 40 and 50% of this substantial sum to the Responder (Cameron, Reference Cameron1995).

Evidence from field and experimental research thus challenges the basic underlying model of individual behavior used in neoclassical economics, public choice theory, and game theory. In some settings, individuals do contribute to public goods, do restrict their use of a resource, and do trust one another contrary to theoretical predictions. This evidence has generated another period of turmoil as scholars have tried to understand why predictions derived from the theory of individual rational behavior are supported in some settings and not in others.