1. Introduction
In recent decades the diverse and complex relationships between economics and political science has challenged the interdisciplinary field of institutional analysis (Ostrom, Reference Ostrom2007), particularly regarding the need for scientifically well-founded ex-ante policy assessment from an institutional perspective, which cannot be accomplished with basic economic or political science approaches. This is particularly true where agricultural and environmental policies are concerned.
Currently, however, institutional policy analysis focuses mainly on ex-post policy impact studies to evaluate past policy performance. While a vast amount of institutional ex-post case studies and indicator databanks exists, institutional economists have not yet developed standardized procedures for using this information to make predictions on policies’ institutional feasibility. Similarly, there is a need for these standardized procedures to be easily linked with environmental and agricultural models widely used for policy impact assessment. Both issues illustrate the substantial theoretical and methodological challenges inherent in analysing institutions. Such critical aspects encompass the question of how to capture not only the incentive structure faced by individuals in different decision-making contexts, but also bounded rational behaviour, informal institutions that form an important part of the institutional environment, and the complexity of transactions related to nature (Hagedorn, Reference Hagedorn2008). These problems become even more exaggerated when developing tools for a standardized ex-ante institutional policy analysis. Ex-ante impact assessment has become an integral and systematic part of the political decision-making processes of the European Commission, but also at the national level in European Union (EU) Member States (Bäcklund et al., Reference Bäcklund, Macombe and Zemek2007). Here, the analysis of likely social, economic, and environmental impacts is increasingly complemented by an assessment of the institutional dimension (EC, 2005).
In this paper, we introduce a formalized procedure for ex-ante modelling institutional aspects for policy implementation: the ‘Procedure for Institutional Compatibility Assessment’ (PICA).Footnote 1 PICA has been developed as an explorative and flexible, yet systematic methodology to assess the compatibility between policy options and various institutional contexts. We conceive of policy options as sets of policy instruments that a policy-maker intends to implement to reach policy objectives.
In Section 2, we relate the procedure to a brief digest of prominent approaches for policy assessment. Thereafter, we outline the basic theoretical assumptions underlying the concept of institutional compatibility and its use for ex-ante institutional policy assessment. In Section 4, we elaborate on PICA's four distinct steps. In Section 5, we apply PICA to the implementation of the EU Nitrate Directive in the département Allier in France to illustrate its practical methods and to validate the intermediate results wherever possible. In the concluding Section 6, we discuss the importance of PICA as an explorative tool within the policy-making process, as well as present options for methodological improvements.
2. Institutional aspects in policy assessment
Policy analysis guides the process of selecting appropriate policy options to be put into practice. The analysis is commonly subdivided into two categories: ex-ante and ex-post analysis. Ex-post policy analysis is designed to evaluate past policy performance, for example in terms of effectiveness, transparency, and distributional fairness to reach policy objectives, and includes a wide range of methods, such as surveys, case studies, etc. The ability of these approaches, however, is limited since they do not provide for a way for evaluating the effects of policies prior to their implementation. In contrast, experiences with ex-ante evaluations are still less numerous and their results less reliable (Blazek and Vozab, Reference Blazek and Vozab2006; Todd and Wolpin, Reference Todd and Wolpin2006).
Furthermore, all ex-ante political evaluation studies discussed in the following, including PICA, face inherent risks, as they are usually part of the political process. Those risks include, for example: (a) providing purposefully misleading information to the interviewer; (b) conveying certain expectations to the evaluators of the public authority who commissioned the assessment; (c) influencing the objective tool choice, and (d) using the studies as political instruments to justify preconceived legislative actions (De Ridder et al., Reference De Ridder, Turnpenny, Nilsson and von Raggamby2007; Turnpenny, Reference Turnpenny2008; Bohne, Reference Bohne2008).
The set of approaches described in the following is not comprehensive. Rather, we pick prominent examples of approaches and tools for policy assessment in order to highlight the respective comparative advantages of PICA.
First, there are approaches which do not explicitly consider institutional aspects at all, that is the characteristics of the institutions and governance structures that facilitate or hinder policy implementation. The Cost of Policy Inaction (COPI) method, for instance, is often used as an ex-ante evaluation tool to identify and roughly quantify the environmental damage that would occur if no new policy is designed to address the underlying (environmental) problem or if the existing policies are not revised accordingly. COPI is not suitable, however, for comparing and choosing between different policy options, or for judging the efficiency of policies (Bakkes et al., Reference Bakkes, Bräuer, Brink, Görlach, Kuik and Medhurst2006).
In cases where it is possible to quantify costs and benefits in monetary terms, a major tool for ex-ante impact assessment is the Cost–Benefit Analysis (CBA). In contrast to COPI, CBA has a narrower and more concrete focus and tends to work with data that are more specific. Despite its widespread use, it has many practical and conceptual difficulties associated with monetizing costs and, in particular, benefits of a proposed policy (Kirkpatrick et al., Reference Kirkpatrick, Parker and Zhang2003).
Other supporting valuation methods try to capture likely policy impacts in general and the problem of monetizing environmental benefits and costs in particular. These methods can be – and actually often are – embedded into CBA, as they provide a basis for monetizing public goods that do not possess a market value. Prominent approaches are the contingent valuation method (Wagner, Reference Wagner2000), choice modelling (Morrison and Bennet, Reference Morrison and Bennet2004), the travel cost method (Hanley and Splash, Reference Hanley and Splash1993), and hedonic pricing (Freeman, Reference Freeman1993).
Second, some policy assessment approaches consider institutional aspects, yet they are limited and somewhat fixed in the sets of variables considered; they incorporate only a pre-defined selection of institutional aspects and do not allow for discovering and assessing additional institutional constraints. However, variables incorporated in an assessment should not be restricted to certain areas of institutions, such as the existence of corresponding legislation, because it may be exactly those other institutional aspects that are key to the success or failure of a particular policy in a specific institutional context.
A first group subsumes various methods of ex-ante impact assessment, including environmental and social impact assessments (Becker, Reference Becker2001). Modelling approaches for ex-ante impact assessment that explicitly incorporate institutional issues as single model input variables have been applied, for example by Takasaki (Reference Takasaki2007) who presents a household model of forest clearing subject to certain land and labour market institutions, and by Capello and Spairani (Reference Capello and Spairani2004) who use scenario building methodology to estimate growth and spatial distribution of the Gross Domestic Product in alternative scenarios for communication and infrastructure policies.
More comprehensive assessments are integrated impact assessments, of which sustainability impact assessments are a specific form comprising a range of processes that follow the aim of integrating sustainability concepts into decision-making (Pope, Reference Pope2006). Integrated assessments not only incorporate ex-ante appraisal and ex-post evaluation, they also provide for some horizontal integration, bringing together economic, environmental, and social impacts into a single, overall assessment (Lee, Reference Lee2006). As shown by Noble (Reference Noble2008), who provides a critical review of strategic environmental assessment systems and practices in Canada, specific characteristics of the institutional environment may be included in these integrated assessments, though they cover only a pre-defined section of the overall institutional context.
The second group of policy assessment approaches that considers institutional aspects includes the various strands of multicriteria analysis (Figueira et al., Reference Figueira, Greco and Ehrgott2005), in particular social multicriteria evaluation (Munda, Reference Munda2004). Those portions of policy implementation's institutional constraints that are related to hidden interests, lobbying groups, power relations, social participation, ecological awareness, and cultural constraints, which can be expressed in terms of different actors' values and preferences, can be incorporated into these models. Costs and frictions of policy design and implementation, however, are not addressed, not least because they are difficult to estimate and quantify ex-ante.
A third group is constituted by experimental approaches, which are more comprehensive and much more flexible in their choice of variables, yet do not entail a systematic search process for discovering relevant institutional aspects. This group implements alternative versions of a policy in an experimental situation and compares their relative impacts and effects to form ex-ante predictions of the likely impacts of this policy. Despite the fact that such an experimental approach is often too costly and time-consuming to be feasible for policy design purposes, in some cases experimental data have been used to successfully validate forecasting model outputs (Todd and Wolpin, Reference Todd and Wolpin2006).
This brief categorization demonstrates the need to supplement existing ex-ante policy analysis from an institutional perspective. The PICA approach for ex-ante institutional policy assessment has been designed to mitigate some of the flaws in existing assessment approaches. First of all, it explicitly considers institutional aspects of policy implementation. Further, PICA uses a theory-induced yet flexible raster of categories of policy-specific institutional aspects, which is updated and refined constantly to incorporate new institutional aspects that hinder or facilitate policy implementation. Further, PICA features an inherent search process to discover those institutional aspects that are particularly relevant either for the policy option under scrutiny or for the particular regional institutional context. Thus, PICA offers a systematic yet comprehensive way to assess the compatibility between a given policy option and a particular regional institutional context.
3. The concept of institutional compatibility
According to Boettke and Coyne (Reference Boettke and Coyne2005), human interaction models based on neoclassic economic theory often have problems and limitations in real social settings. Similarly, Searle (Reference Searle2005) states that economics is often treated like a natural science, assuming economic theories as facts, taking the institutional setting for granted and neglecting that these facts ‘can only exist given certain human institutions’. The same is true for most agri-environmental models used for policy assessment. Although aware of the oversimplification, those models usually assume that with the implementation of a new policy, the institutional arrangements conducive for that policy will be perfectly in place, or that a sub-optimal institutional arrangement will change automatically towards perfection at once and with no costs. In addition, it is often assumed that all targeted actors will comply with the policy.
However, plenty of empirical evidence exists demonstrating often substantial discrepancies between the intended effects of a new policy – usually a set of new or modified institutions – and the outcomes of this policy – its institutional performance in terms of effectively guiding actors’ behaviour (e.g. Damania et al., Reference Damania, Fredriksson and Mani2004; Theesfeld, Reference Theesfeld2008). Institutional economic and other economic theories and concepts offer quite a few explanations why these discrepancies or institutional incompatibilities may emerge in some situations, but not – or to a lesser extent – in others. Instead of trying to provide a comprehensive overview of all potentially relevant theories and concepts – which would undoubtedly futile – we highlight four broad categories of determinants for institutional policy options’ (in-)compatibilities:
1. Formal and informal rules shape the incentives that actors have and thus determine their behaviour in a particular situation. If a new policy is introduced, actors may face a new or modified action rule. However, whether actors actually change their behaviour accordingly, that is whether the institution becomes effective, depends on the overall incentive structure that actors face. There may be countervailing rules or policies (designed at other administrative levels) that provide incentives detrimental to the objectives of the newly introduced policy. These countervailing rules may also be informal rules that are deeply embedded in religious beliefs or cultural traditions. Thus, it is possible that the newly implemented policy may fail to substitute or override the institutions already in place and thereby change actors’ incentive structures according to its objectives.
2. Suitable governance structures are necessary to make new rules effective, that is to supervise actors and to sanction actors’ non-compliance. Governance structures intended to monitor and coordinate the implementation of a new policy, however, may not have the necessary capacities or the appropriate design to reduce information asymmetries sufficiently, or may even be absent in some regions. Further, when it comes to agri-environmental problems, governance structures other than markets, hierarchies, or hybrids (Ménard, Reference Ménard2004) may prove to be more effective. This includes, for example, horizontal non-market coordination (e.g. cooperation and collective action – Ostrom, Reference Ostrom1990), formal and informal networks such as knowledge and information systems, methods and infrastructure for measuring, monitoring, and evaluating environmental damages and benefits, and also conflict resolution mechanisms.
3. Institutional incompatibilities may occur if the design of the institutional arrangements implemented with the new policy does not correspond with actors’ characteristics. For example, targeted actors’ values and beliefs may not be in line with the policy objectives, or actors may not have the necessary resources, competencies, and knowledge to comply with the rules. Further, actors’ prevailing interdependencies need to be taken into account, that is the fact that one actor's choice may influence the choices of others (Paavola and Adger, Reference Paavola and Adger2005).
4. The performance of institutional arrangements induced by the new policy is closely related with the characteristics of the transactions targeted by this policy. Relevant features of transactions are asset specificity, uncertainty, and frequency (Williamson, Reference Williamson, van Huylenbroeck, Verbeke and Lauwers2004). In particular, if agricultural, environmental, and rural policies are concerned, the specificities of nature-related transactions also deserve attention (Hagedorn, Reference Hagedorn2008).
Thus, the effectiveness and cost-effectiveness of new policies depend on the compatibility between the characteristics of their institutional design and the institutional arrangements already in place at the time of policy implementation, also taking into account the characteristics of actors and transactions targeted by the policy. Given specific attributes of actors and transactions and appropriate institutional arrangements increase the likelihood of actually achieving the policy objectives, that is they increase the likelihood of actors’ compliance and intended change of behaviour, or they ensure that these policy objectives are achieved at reasonable costs. An adequate understanding of the institutional configuration and the situational logic of the institutional environment in which a policy is to be implemented is a necessary precondition for assessing the balance between the intended and unintended consequences of that policy (Aligica, Reference Aligica2005; Esty et al., Reference Esty, Levy, Srebotnjak and de Sherbinin2005: 11).
Given the plethora of institutional economic and social theories underlying and populating these four groups of determinants, the task at hand is to organize a systematic search process to identify those institutional factors that are likely to hamper or facilitate the implementation of a new policy in an existing institutional environment. The procedure provides indications on whether or not a new policy will be able to effectively change the existing institutional arrangements – or establish new ones – that guide actors’ behaviour in such a way that the policy objectives can be reached. The presented approach can predict neither the precise extent of effective institutional change that may be induced by the new policy, nor the shape of the then effective institutional arrangements. Rather, it only considers those institutional arrangements, actors’ characteristics, and features of transactions that can be observed prior to, or at the time of, policy implementation. In this sense, it takes a static approach. Yet we are aware that the overall institutional context, including actors’ characteristics such as knowledge and problem awareness, and the features of the transactions may change over time in response to newly implemented policies (Field, Reference Field2007: 269).
4. The procedure for institutional compatibility assessment (PICA)
In the previous section, we argued that ex-ante policy assessment must be linked to a comprehensive examination of the institutional contexts in which individuals and groups seek to act on their preferences and shared understandings (Bickers and Williams, Reference Bickers and Williams2001: 234). Hence, the institutional assessment has been conceptualized to reveal in which country or region a policy option would be compatible with the existing institutional structures, and where an institutional misfit likely to hamper policy implementation can be expected.
The method proposed is the Procedure for Institutional Compatibility Assessment (PICA), which comprises four distinct working steps:
Step 1: The policy options are clustered to identify the generic structure of a policy option.
Step 2: Each policy type is characterized by a specific set of crucial institutional aspects (CIA).
Step 3: Indicators help to evaluate the potential of respective CIA to constrain or foster the implementation of a policy option.
Step 4: The information provided by the indicators is used for a qualitative assessment of each identified CIA, is further aggregated, leading to qualitative statements about the probable effectiveness of a policy option and the institutional fit or lack of fit between policy options and institutional contexts.
Assumptions made within the PICA steps, such as the categories in which to cluster the policy options, as well as assumptions underlying the CIA or the explanatory power of the indicators, are informed by institutional economic and social theories, as well as the policy assessment literature, in particular empirical ex-post policy assessment studies. However, all four PICA steps allow for the integration of stakeholders’ and policy-makers’ opinions: they can suggest, for example, considering (or not) certain CIA for the assessment, and whether to have certain indicators included or excluded. They also play an important role in the assessment of some of the indicators used and, finally, in the ranking and grouping of CIA. In this sense, this open concept takes into account the findings by Page (Reference Page2007) that the recognition of diverse preferences and perspectives proves to be better at problem solving.
4.1. PICA step 1: classification of policy options
The systematic classification system used in this step identifies the generic structure of a policy option linked to agriculture, the environment, or rural development. The particular type of intervention, together with the area of intervention, provides the basic information to describe a certain policy type. An additional dimension used to classify policy options is possibly induced property rights changes. The objective of this specification of policy types is to provide a suitable yet formalized structure to identify CIA that are of particular importance for the policy option under scrutiny. It is assumed that the policy types represented in Table 1 are decisive for the range and kind of CIA that can be expected to be conducive or detrimental to the implementation of a policy option. This typology allows us to limit the number of CIA that need to be reviewed when evaluating the new policy. In the absence of this classification or filter, all identified CIA relevant for agricultural, environmental, and rural development policies would have to be processed every time a policy option is assessed.
Policy type matrix
In the following, the dimensions of the classification system will be explained in more detail. The types of intervention, that is the policy instruments, are inscribed in the respective rows in Table 1 and describe how and by which means the impact of a policy will be reached:
• Regulatory or command-and-control instruments (compulsory): laws, regulations, specific protection targets, and designations of areas for protected habitats or species.
• Economic instruments that often use financial (dis-)incentives: taxes, subsidies, grants, loans, and tradable pollution permits.
• Advisory/voluntaryFootnote 2 instruments: codes of good practice, extension services, informative measures, and environmental audits.
This classification is based on the work of Stone (Reference Stone2002) who distinguishes between five general mechanisms for changing or coordinating actors’ behaviour: (1) inducements, that is changing people's behaviour with (often financial) rewards and punishments – here named economic instruments; (2) rules, that is commands to act in certain ways, or commands that determine permissions and entitlements; (3) rights, that is strategies that allow individuals, groups or organizations to invoke government power on their behalf; (4) powers, that is shifting the power of decision-making to different people (the last three are here subsumed under regulatory instruments); and (5) facts, that is strategies that rely principally on persuasion (here named advisory/voluntary instruments). Stone also stresses that these instruments are idealized types and that no policy option ever relies purely on one type of instrument. A similar distinction is made by Moskowitz (Reference Moskowitz, May and Wildavsky1978: 65ff.), and corresponds with distinctions made by environmental economists (e.g. Stavins, Reference Stavins, Blume and Durlauf2004).
The area of intervention points to the governance structures a policy is supposed to have an impact on. More precisely, a policy aims to influence real-world transactions such as pesticide use or protection of species by changing existing or creating new governance structures that co-ordinate these transactions so that, for example, their results are internalized by the actors. The differentiation used in PICA follows, to a large extent, the widely used categories of governance structures (hierarchy, market, and hybrid) suggested by Williamson (Reference Williamson, van Huylenbroeck, Verbeke and Lauwers2004). Though the boundaries between market, hybrid, and hierarchy are often blurred (Ménard, Reference Ménard2004), it can be assumed that almost every governance structure can indeed be seen as some hybrid form between the polar cases of market and hierarchy. Thus, in the respective columns of Table 1, those areas of intervention that are closer to either market or hierarchy are subsumed. Drawing on collective action theory in relation to natural resource management, the third column, self-organized network is specified. This specific hybrid form of governance structure is of particular interest in the context of agricultural, environmental, and rural development policies (e.g. Ostrom, Reference Ostrom1990; Baland and Platteau, Reference Baland and Platteau1996; Agrawal and Ostrom, Reference Agrawal and Ostrom2001).
The property rights change column is the third dimension used to describe a policy type and accounts for changes in private and collective property rights, in particular in relation to the effect on natural resources likely to be induced by the policy option (Schlager and Ostrom, Reference Schlager and Ostrom1992). This dimension covers an important institutional specificity of environmental policies. Undoubtedly, most policy options will imply some changes in property rights. However, here it is defined in a more narrow sense, indicating the effects of changes in the property rights of actors, for example farmers, on natural resources needed for production, such as land and water. Contemporary agricultural policy in the industrialized world is predicated upon the property rights structure of land that – transmitted through the political process – created a presumptive entitlement for the favoured treatment of land owners (Bromley and Hodge, Reference Bromley and Hodge1990). Also important in this respect is the endowment effect, which describes the significant disparity between demanding a payment to give up a property right, which was allocated to somebody in the first place, rather than being prepared to pay to acquire it (Sunstein, Reference Sunstein1993). Thus, introducing a policy option that will cut back on the previously allocated property rights of an actor will meet stronger resistance than assigning limited property rights in the first place.
To sum up, the three dimensions necessary to comprehensively describe a policy type are illustrated as a three-dimensional graphic in Figure 1. The x-axis describes the area of intervention, the y-axis the type of intervention, and the z-axis the dimension of property rights change. Each cuboid in the space represents a certain policy type. To illustrate, the establishment of the European Food Safety Authority, a policy that intervenes at hierarchies using regulatory instruments and does not directly influence farmers’ property rights, can be assigned to the policy type of the dark grey cuboid, whereas the case of restrictions on land use implemented by means of a regulatory instrument is illustrated by the white cuboid.
Three dimensions of a policy type
4.2. PICA step 2: crucial institutional aspects
In the process of developing PICA, we have compiled an initial library of 42 CIA linked to respective policy types in agriculture, environment, and rural development. For each CIA, we formulated detailed assumptions on how it relates to a respective policy type, that is we describe what effects the existence of this CIA in a particular regional institutional context would have on policy implementation. These CIA and the assumed relationships are derived from a broad range of institutional economic and social theories as well as from policy assessment literature, in particular empirical ex-post policy assessment studies. In many cases, the empirical and the theoretical literature point to similar relationships.
For example, we take the CIA ‘Bargaining power of farmers’ associations’, which we found to be primarily related to regulatory policies intervening in markets.Footnote 3 It is assumed that this CIA may have a detrimental effect on policy implementation such that farmers’ associations might be able to weaken mandatory restrictions or obtain exception clauses if they foresaw economic disadvantages when the policy option was implemented. This assumption is based on political economy studies focusing on classic public choice approaches, such as Peltzmann (Reference Peltzman1976), Becker (Reference Becker1983), Miller (Reference Miller1991), and Swinnen (Reference Swinnen1994), which understand agricultural policies to be the result of political bargaining among various agrarian and non-agrarian groups for income redistribution. Other studies highlighting the effect of the distribution of bargaining power among different agrarian groups within a country are Lynggaard (Reference Lynggaard2001), Wislon et al. (Reference Wilson, Petersen and Holl1999), and Valentinov (Reference Valentinov2007).
Another example is the CIA ‘Information asymmetry between state and firms’, which is related to several policy types in Table 1. The presence of information asymmetries may induce high costs for establishing a monitoring system to ensure actors’ compliance with a policy. Or, given a fixed low monitoring intensity, actors may find it safe to defect, thus reducing the effectiveness of the implemented policy. This relationship is based on Principal-Agent Theory (Arrow, Reference Arrow, Pratt and Zeckhauser1985), but also features prominently in many empirical ex-post assessment studies (e.g. McCann and Colby, Reference McCann and Colby2005). Valentinov (Reference Valentinov2007), and highlights information asymmetry in the agricultural production process and focuses on the monitoring advantage of family farms.
4.3. PICA step 3: indicators
In PICA Step 3, indicators are used to evaluate the extent of those CIA that have been selected in the previous step. Indicators are variables and proxies that are used as input to the institutional assessment within PICA. About 100 indicators have been compiled so far by the authors. The linkages between a CIA and the respective suggested sets of indicators are based, again, on a broad range of institutional economic and social theories, as well as on relevant empirical ex-post studies. For instance, ‘membership in farmers’ associations’ is suggested as one possible indicator for assessing the extent of the CIA ‘Bargaining power of farmers’ associations’. Here, it is assumed that a high number of members increase the influence of the farming organizations within the political decision-making process. This relation is explained by the economic theory of democracy. Similarly, Rabinowicz and Swinnen (Reference Rabinovicz, Swinnen and Swinnen1997) use ‘share of agricultural votes’ to explain land policy reforms in Central and Eastern European countries, based on public choice considerations.
4.4. PICA step 4: aggregating information on institutional compatibility
In this final step, the expert team that runs PICA is using the information provided by the indicators for a qualitative assessment of the institutional compatibility between the policy option under scrutiny and the regional institutional context. This entails first compiling the various indicator values available for every single CIA selected in PICA Step 2 and assessing the relative level of each indictor value, that is relative to a reference value derived by comparing the indicator value for the region under scrutiny with the respective values of the same indicator at higher geographical scales, or in other regions at the same geographical scale. For example, the level of corruption can be determined for every country where the policy option is to be implemented, thus providing insights in the relative – country-wise – likelihoods for ineffective policy implementation. Second, the assessments of all indicator values related to a single CIA are used to arrive at a qualitative statement on the extent of this CIA. Third, the PICA team defines thematic categories of institutional compatibility to group the CIA and the respective qualitative statements. While it is certainly helpful to use science-driven categories such as property rights compatibility or embeddedness compatibility, policy-makers who commission an assessment might prefer different or additional categories. Each thematic category draws on information from at least one CIA. At this stage, focus groups with regional stakeholders and scientific experts may be used to assess both the relative importance of all CIA within one thematic category, and of the respective thematic categories vis-à-vis each other.
Finally, these categorized and ranked qualitative statements on the compatibility of the policy option are presented to the policy-maker who has commissioned the policy assessment. Here, an interactive form of communication is preferred since this provides the opportunity to discuss the results and, perhaps, the introduction of complementary policy instruments if – according to the PICA results – the implementation is likely to be substantially hampered.
5. Applying and validating PICA
There are several ways to validate the methodology and the results of the PICA procedure. First, assuming various applications of PICA over the years, policies might have been implemented for which ex-ante assessments had been made. In these cases, a validation of the PICA outputs could be made by comparing them to the outcomes of respective ex-post impact studies. Since PICA was developed recently, such validation studies could not yet have been carried out. Second, PICA can be applied to hypothetical policy options with the main purpose of testing and refining the methods used within each PICA step. This approach was taken in two testing applications of PICA in two regions in France (Amblard et al., Reference Amblard, Aznar, Mann, Schleyer, Theesfeld and Hagedorn2008; Schleyer et al., Reference Schleyer, Theesfeld, Hagedorn, Amblard, Aznar and Mann2007). Yet since there is no ex-post information available, this approach does not allow for a validation of PICA results. Third, PICA can be applied to a policy option that has already been implemented and for which ex-post impact studies are available. Here, the ex-ante situation can be simulated by strictly reconstructing the institutional context that existed shortly before the actual implementation of the policy. This implies that only empirical information that had been available at the time prior to the implementation can be used, thus ignoring experience with the policy after its implementation. Despite being aware of the methodological difficulties of this validation approach, such a test case was conducted in the French département Allier in the Auvergne, taking the EU Nitrate Directive that was implemented in the region in 1994 as a policy option. Apart from gaining important insights for the further development of the practical methods used within each PICA step, the comparison between the results of the simulated ex-ante assessment and the ex-post evaluation of the implementation of the Directive in Allier shed light on the ability of PICA to predict those institutional aspects that indeed turned out to be crucial for the implementation of this policy in this particular regional institutional context (Schleyer et al., Reference Schleyer, Theesfeld, Hagedorn, Amblard, Aznar and Mann2007).
In the remainder of this section, we elaborate on the theoretical and conceptual underpinnings of each individual PICA step and apply PICA to the implementation of the EU Nitrate Directive in Allier to illustrate the practical methods used within each step and to validate the intermediate results wherever possible.
5.1. Applying PICA step 1: classification of the policy option
The EU Nitrate Directive (Council Directive 91/676/EEC) (EC, 1991), adopted in 1991, can be seen as a prominent and typical example of an EU environmental policy addressing water pollution. One of the core elements of the Directive states that, when implementing the EU Nitrate Directive, Member States have to draw up and implement action programmes in designated vulnerable zones that shall consist of mandatory rules. These rules determine, for example, periods when the application of certain types of fertilizer is prohibited, as well as limitations of the application rates of fertilizers, taking into account soil conditions, soil type, slope, land use, and agricultural practices. Furthermore, Member States have to establish suitable monitoring and enforcement systems to ensure actors’ compliance with the rules.
While the EU Nitrate Directive comprises various policy elements that can be combined in diversified ways, only the ‘uncompensated and mandatory production restrictions’ in previously defined vulnerable zones were considered for this application. The PICA team that carried out the policy assessment focused on this element of the Directive and treated it as a single policy instrument and thus avoided distorting effects due to the other – certainly interdependent – policy elements that would also be implemented if the Directive were to be introduced.
Using all available information on the concrete form and content of the policy option, the PICA team categorized this element of the EU Nitrate Directive – according to the matrix of policy types (see Table 1) – as a regulatory type of policy having effects on markets. Effectively, only national regulations determine the precise limits of restrictions. These restrictions also induce severe changes in and constraints on private property rights of farmers with respect to the production factor land. Further, it is assumed that no compensations are paid to cover the costs induced by these restrictions. These uncompensated restrictions have an impact on farmers’ production costs (e.g. because yields decrease due to restrictions in fertilizer use) and, thus, on their market position. More precisely, farmers might be forced to offer their products at a higher price, resulting in a decrease in demand for those products, or they might keep the price and accept reduced profits. The respective impact levels of the restrictions on the farmers’ production costs, however, depend on a variety of factors, such as farming practices before implementation of the restrictions, size and type of the agricultural enterprise, soil fertility, and share of land farmed by the agricultural firm affected by the restrictions. Clearly, in some cases the restrictions may not impose any further costs, for example because the regulated fertilizers were not used in the agricultural firm.
5.2. Applying PICA step 2: crucial institutional aspects
At first, only those CIA that potentially hamper or foster the effective implementation of the type of policy option identified in the previous PICA Step 1 were extracted from the overall library of CIA. Thereafter, the PICA expert team assessed the relevance of these pre-selected CIA by consulting literature and documents that had been available prior to the implementation of the policy (1994) and by carrying out several interviews with stakeholders and policy-makers that had been involved in the pre-implementation phase of the EU Nitrate Directive in Allier. During this process, some of the pre-selected CIA were not considered crucial for the policy option under scrutiny – the EU Nitrate Directive – or for the regional institutional context in Allier, and were therefore skipped. Conversely, informed by the empirical information gathered, some CIA that had not been pre-selected were added. As a result, the PICA team identified four CIA of high relevance for the policy option and the regional institutional context: (1) ‘Bargaining power of farmers’ associations’; (2) ‘Information asymmetry state vs. firm’; (3) ‘Compliance costs for farmers’; and 4) ‘Level of information on policy’. These four CIA were assessed in the subsequent PICA steps.
The ex-post evaluation of the implementation of the EU Nitrate Directive in Allier confirmed that these four crucial institutional aspects were important constraints to effective implementation. Three of the most crucial ex-post institutional aspects were: (1) ‘Bargaining power of farmers’ associations’; (2) ‘Information asymmetry state vs. firms’; and (3) ‘Level of information on policy’. However, while PICA was able to correctly predict these CIA as being important, two additional institutional aspects also played an important role in the real implementation process: (1) ‘Attitude of farmers towards ecological considerations’; and (2) ‘Bargaining power of environmental groups’. Overall, the filter function of the current typology, in combination with the ability of the empirical methods used to identify additional CIA relevant for the particular policy option and regional institutional context, can be regarded as satisfactory.
5.3. Applying PICA step 3: indicators
For each of the four CIA identified in the previous PICA step, indicators were selected from the existing indicator lists, revised, and validated by the PICA expert team and by using the results of a scientific expert workshop. During the revision process, the relevance, or explanatory power of each indicator with respect to a particular CIA in the specific policy and regional institutional context, was critically assessed. Further, the availability of reliable qualitative or quantitative data played an important role in selecting appropriate institutional indicators. For some CIA, new proxies had to be identified to compensate for missing data or for indicators that were considered less meaningful in the given context. Thereafter, problem-centred interviews with regional stakeholders and scientific experts were used to determine those qualitative and/or quantitative institutional indicator values that were not available in statistical databases.
Table 2 contains a sample selection of those indicators that were used to assess the extent of the selected CIA ‘Bargaining power of farmers’ associations’.
Indicators for assessing the CIA ‘Bargaining power of farmers’ associations’
5.4. Applying PICA step 4: aggregating information on institutional compatibility
The assessment and compilation of indicator values was done by the PICA team using reference values and a weighting system (Amblard et al., Reference Amblard, Aznar, Mann, Schleyer, Theesfeld and Hagedorn2008: 22pp.). The relative importance of each CIA within a thematic category, its manner of influence (fostering or hindering), as well as the relative importance of each category with regard to the process of implementation of the EU Nitrate Directive in Allier were likewise assessed through a focus group workshop with regional stakeholders. Subsequently, the PICA team grouped the respective CIA into three thematic categories of institutional compatibility:
1. Communication capacity
• ‘Bargaining power of farmers’ associations’ (CIA)
2. Governance structures compatibility
• ‘Information asymmetry state vs. firm’ (CIA)
3. Actor characteristics
• ‘Compliance costs for farmers’ (CIA)
• ‘Level of information on policy’ (CIA).
The application of PICA in a simulated ex-ante situation using the already implemented EU Nitrate Directive in one region of France as an exemplary policy option has shown that PICA is able to produce meaningful and fairly precise indications for potential institutional incompatibilities. The quality of the PICA results is, however, closely related to the time and human resources available to conduct the assessment, as well as to the availability of meaningful empirical data for determining the indicator values.
Figure 2 summarizes the four steps of PICA.
Scheme of the procedure for institutional compatibility assessment
6. Conclusion
PICA's methodological and theoretical conceptualization is innovative in that it adopts the perspective of a policy-maker who intends to influence the behaviour of actors when designing and implementing new policies. Perhaps unlike a scientist, the actor may not be interested in a detailed analysis of the institutional dynamics on the ground. Instead, he wants to know if a potential policy is likely to change actors’ behaviour in such a way that the policy objectives can be reached. Consequently, PICA has been designed as an explorative tool that is able to identify those main institutional incompatibilities that might act against policy implementation; it can thus be considered an early warning system for institutional incompatibilities. However, PICA results point to potential institutional incompatibilities without providing detailed insights into the concrete causalities that lead to these institutional incompatibilities; thus, further empirical analysis would be necessary to design adequate policy instruments to overcome or mitigate those incompatibilities. Here, the overall importance of PICA within the policy-making process reaches well beyond exploring institutional incompatibilities, since the PICA results allow for a more focused design of such a subsequent institutional analysis. Furthermore, PICA provides a flexible structure within the four steps that can be adapted very easily to all possible agricultural, environmental, and rural policy options and regional institutional contexts. Indeed, it allows for low-cost and time-saving research and the results are easy to communicate to policy-makers. In this respect it complements other ex-ante policy assessments.
PICA can also play an important role in integrated modelling frameworks that have been developed for an ex-ante assessment of policy impacts on environmental, economic, and social systems. Here, the economic and environmental models often assume that appropriate and required institutions are in place for resource governance towards sustainability, or that those institutions can be implemented at no cost. PICA can be seen as a method that qualifies those underlying modelling assumptions by raising awareness that actors do not automatically comply with policies, that is the institutional change induced by the implemented policy does not necessarily become effective.
Despite being an explorative tool, all PICA steps can build on a solid basis derived from theoretical insights and empirical institutional analysis. Yet PICA is still a work in progress. Neither the current CIA library nor the lists of CIA linked to a particular policy type can be seen as static, but need to be continually revised to improve the accuracy of the predictions. Regarding validation, it is essential that the experience made and insights gained during every PICA application are used systematically and carefully to make the empirical basis of PICA more comprehensive. Thus, the CIA library can be seen as an ever-growing source of information. The same applies to the library of institutional indicators used in PICA Step 3. Here, the experience gained through testing applications has shown that the methods used within PICA Steps 2 and 3 facilitated the integration of case-specific CIA and institutional indicators.
Further main avenues for improvement would include putting the ability of policy option typology, which actually filters the CIA, properly to the test. For example, is the current typology indeed able to account for those essential characteristics of a policy option that determine the range of CIA relevant for this policy? Is this filter really useful for excluding those CIA that are not relevant for this policy option? Does the filter exclude CIA that may turn out to be relevant in PICA Step 2 (too exclusive)? Does the filter include many CIA that may turn out to be irrelevant or less relevant (too inclusive)? How to incorporate the probability that the CIA themselves change due to policy implementation? Similarly, we may ask if the typology covers those main features of a policy option that are important for the success of policy implementation. An important feature not yet addressed by the current typology, for example, is the dimension of the bio-physical system that is addressed by a policy. Here, some crucial institutional aspects can stem from the fact that the characteristics of a natural resource addressed – or, more precisely, the attributes of a nature-related transaction that is induced or influenced by the policy – might call for specific institutional arrangements to make a policy option effective (Hagedorn, Reference Hagedorn2008). For instance, policies addressing water quality often have to deal with non-point pollution from agriculture, which constitutes challenges for adequate forms of monitoring and sanctioning. In contrast, soil conservation measures address an immobile resource. Further, policies for the protection of biodiversity or specific rare species face particular incentive problems, not least because the future value of these rare species is uncertain and the benefits of protection cannot only be reaped by the one protecting it. In addition, the geographical dimensions of resources can also be important. Thus, distinct institutional aspects for each of the natural resources addressed can be expected. This clearly illustrates that PICA needs to be further developed and the resulting concept tested as a valid and innovative tool to capture the institutional dimension in ex-ante policy assessment.
