1. Introduction
Advocated across the international community for more than 15 years, the Extractive Industries Transparency Initiative (EITI) has become widely recognised as a hallmark anti-corruption scheme in the extractive sector. While the Organization for Economic Co-operation and Development identifies the extractive industries as the world's most corrupt economic sector (OECD, 2014), it is thought that the EITI will help to alleviate the sector's corruption problems by unveiling financial and contractual discrepancies in public agreements and enhancing public accountability (Rustad et al., Reference Rustad, Le Billion and Lujala2017; Van Alstine, Reference Van Alstine2017). In practice, the EITI requires its member countries to abide by financial and contractual disclosure standards and maintain a public feedback mechanism in the form of a national multi-stakeholder group comprising of private, public and civil society representatives.Footnote 1 The EITI also uses audits to verify members’ compliance with its disclosure standards and ensure that these requirements are upheld properly (Sovacool et al., Reference Sovacool, Walter, Van De Graaf and Andrews2016).Footnote 2
Fifty-five countries are currently publicly committed to implementing the EITI's standards and the initiative has contributed to the disclosure of more than US$3 trillion of public revenues from the extractive industries worldwide (EITI, 2020). Some suggest that these commitments may have helped members to attract foreign aid and investment (Lujala, Reference Lujala2018). However, discontent has grown among many in the sector as questions have continued to arise about the EITI's ability to induce meaningful changes among its members. While some critics point towards issues concerning the EITI's implementation (such as members’ incomplete and insufficient reporting of information) (Öge, Reference Öge2016), a review by Rustad et al. (Reference Rustad, Le Billion and Lujala2017) adds to the growing disillusionment surrounding the initiative. The review identifies a broad range of studies offering mixed conclusions as to whether a relationship exists between EITI membership and countries’ progress in tackling corruption.
Bickham (Reference Bickham, Eads, Mitchell and Paris2009) argues that the ultimate test of the EITI must be whether it has stimulated the systemic effects expected at its inception and whether this has contributed to improvements in EITI countries’ levels of corruption. Nonetheless, little attention has been paid to the methodologies applied in this literature to measure countries’ progress. In particular, these studies often examine changes in corruption indicators (such as the World Governance Indicators’ Control of Corruption Index) that are more indicative than consistent in their measurement of corruption over time. Such issues may confound the results of existing studies as the corruption scores in one year are not necessarily comparable to the next year. Also, studies that draw comparisons between countries’ outcomes often neglect the endogeneity issues caused by countries’ self-selection into the initiative.
Reflecting on these limitations, this study provides a re-assessment of the relationship between the EITI and countries’ progress in tackling corruption using a ‘state-of-the-art’ indicator called the Bayesian Corruption Indicator. Compared to other common corruption indicators, this measure benefits from characteristics such as improved comparability over time and reduced demands to impute data during its construction. Meanwhile, to address the endogeneity issues caused by self-selection, this study combines an entropy balancing approach with a difference-in-difference framework. This approach compares changes in EITI members’ corruption scores to an adjusted (weighted) control group of non-EITI countries. The weights are created such that they minimise the difference in baseline characteristics between member and non-member countries. The approach draws on research showing that minimising the baseline differences between compared samples may significantly decrease an observational estimator's bias (Glazerman et al., Reference Glazerman, Levy and Myers2003; Jaciw, Reference Jaciw2016).
Contrary to the findings of many leading studies, this study shows that corruption scores among EITI member countries have improved significantly relative to non-member countries after adjusting for the differences in their baseline characteristics. In particular, this evidence is strongest among countries considered compliant with the initiative's international standards. The results of this cross-country assessment of the EITI offers important insights into suggestive empirical regularities concerning changes in corruption outcomes across the globe. While a degree of cynicism (or at least scepticism) has built up in recent years around the EITI, this study paints a more optimistic picture of the progress made by its members. Nevertheless, even though its members do appear to be making promising progress, it is clear that even the initiative's most ardent supporters would not claim it is a silver bullet. Critics raise important issues about the implementation of the EITI and recent changes to the EITI's standards (for example, increasing the scope of its standards and its compliance verification process) have only just begun to address these issues.
This paper continues in section 2 with a more in-depth review summarising the key theories, debates and empirical evidence concerning the EITI and corruption. Section 3 then provides details of this study's methodology. This includes a description of the data and estimation strategy it uses. Section 4 presents the results of the analysis. Finally, section 5 further discusses the study's findings and provides the study's concluding remarks concerning policy and future research.
2. The EITI and corruption: literature review
The EITI has become a widely-recognised policy instrument to help combat corruption in the extractives sector. However, since its inception, its implementation has also been controversial and subject to a polarised debate. This section provides a brief overview of some of the key arguments underlying the debates concerning the EITI. It starts with a summary of policy arguments underpinning the EITI's inception and then describes some of the key criticisms the initiative has faced through its implementation. Following this, it examines some of the existing empirical evidence on the relationship between the EITI and corruption.
2.1 Shining a light on corruption: arguments underpinning the EITI
The policy arguments underlying the EITI's creation generally portray the initiative as a multi-pronged and complex intervention. The EITI is not thought of as subscribing to one single channel or mechanism delivering change in the extractive industries. Rather, its approach considers that tackling corruption may require systematic changes supported by several simultaneous mechanisms combining transparency, deliberation, demand/capacity-building, and support for the policy environment. However, the first of these mechanisms concerning information and transparency has most clearly defined the growth of the EITI as an anti-corruption policy. Here it is conceived that increased disclosure in the extractives sector will enable better identification of corrupt activity. This relates to the idea that transparency may help to detect and reduce public malfeasance through its ‘sunshine effect’ (Wilson, Reference Wilson2014). An example of this is described by McDevitt (Reference McDevitt2017) who highlights that company and government reporting to the EITI uncovered nearly US$10 billion of missing tax and royalty payments in Nigeria.
The second route through which it is thought the initiative may help to tackle corruption is through its effect on accountability and deliberation in the sector. The EITI's proponents argue that the initiative dilutes information asymmetry and thus may help to empower citizens to hold public officials accountable for the incidence of financial discrepancies (Van Alstine, Reference Van Alstine2017). This may also help to alter the balance of incentives (i.e., it may increase the risks) associated with officials using their position for private gains. While some suggest this improved accountability may deter illicit acts of corruption or embezzlement from occurring in the first place (Gillies and Heuty, Reference Gillies and Heuty2011), it is also thought that the EITI's multi-stakeholder groups provide another way through which the initiative can contribute to improving accountability too. These groups intend to create a legitimate space for civil society to participate in the management of the extractive sector.
A third way the EITI may contribute to helping build stronger institutions and combatting corruption is through its capacity-building role. For example, more than 1,000 targeted participants from multi-stakeholder groups, national secretariats, government, civil society, industry, parliament, state-owned enterprises and national audit institutions participated in EITI peer learning and capacity-building schemes in 2016 alone (EITI, 2020). This function is considered important for developing an understanding of the information created by the EITI and to stimulate demand for better practices for managing the extractive activity. Building on this point, Bickham (Reference Bickham2015) describes the ‘viral’ (or systemic) effects it is also expected the EITI may have on the broader governance of natural resources. This considers that the initiative may act as an ‘entry point’ or stimulant for broader reforms (Fenton Villar, Reference Fenton Villar2021). Bebbington et al. (Reference Bebbington, Arond and Dammert2017) offer similar insights highlighting the influence of the initiative on the policy environment. They find that the politics concerning transparency in the extractive sector has been more stable and less susceptible to political U-turns due to changes in the national political landscape among EITI members. Numerous examples also exist indicating the noticeable spill-over effects the EITI has had on its members’ legislative and governing environments. For instance, Reinfeldt (Reference Reinfeldt2018) highlights some of the EITI's achievements supporting broader economic and institutional reforms helping to combat corruption in Ukraine.
2.2 Is the EITI sufficient?
However, the EITI has not grown without criticism. Embodied by theories of ‘mock compliance’, critics often argue that the EITI enables governments to appease the international community by mimicking compliance with global norms without inducing meaningful changes (Öge, Reference Öge2016). This could also reflect countries’ historically slow progression to reaching compliance. Lujala (Reference Lujala2018) reports that it has taken countries, on average, 17 months to become formal candidates (to join the formal process of applying to join the initiative) after making a public commitment to implement the EITI's transparency standards, and a further four years to progress their implementation to fully comply with them. Here it is suggested that some countries may simply intend to remain associated with the EITI without inducing the desired changes. This is an issue that, in part, motivated major updates to the EITI compliance validation system in 2016. This included introducing a more disaggregated validation system with specific timeframes that members must adhere to. Failure to comply with these timeframes can result in a country becoming delisted (or expelled) from the initiative.
Beyond this, the Institute for Multi-Stakeholder Initiative Integrity has criticised the EITI because governments are allowed to select the members that constitute the national multi-stakeholder group. This freedom may enable governments to simply appoint favourable representatives to this group and circumvent the initiative's intended accountability mechanism.Footnote 3 Maconachie et al. (Reference Maconachie, Srinivasan and Menzie2015) further argue that even where meaningful representation exists, it may not be reasonable to consider that civil society's inclusion in decision making is meaningful. For example, in some cases, civil society representatives have been invited to stakeholder meetings too late to be able to influence the agenda of those meetings. Also, while many governments appear willing to partially increase their levels of disclosure, recommendations from the initiative's audits have not necessarily materialised in action despite serious irregularities being noted in some instances. This relates to critics’ warnings about the initiative's limited legal mandate in many implementing countries (Kasekende et al., Reference Kasekende, Abuka and Sarr2016).
Further concerns are also voiced about the shortcomings of the EITI's relatively limited focus or scope (Vadlamannati and De Soysa, Reference Vadlamannati and De Soysa2016). The initiative's disclosure standards historically focused very narrowly on the resource revenues received by governments and not the deployment (or expenditure) of revenues or other contractual factors. Reflecting on this issue, the EITI has continued to modify and expand its standards. For example, updates have expanded the EITI's standards remit since 2016 to include issues such as the disclosure of the distribution of revenues, environmental payments, and beneficial ownership information. Nevertheless, despite the widened remit of its disclosure standards, the appeal of the initiative to ordinary citizens remains a contentious issue. McDevitt (Reference McDevitt2017) describes how the technical nature of the disclosed information may render it incomprehensible to the public. This also explains the growing concerns that the EITI is failing to engage and empower local populations.
2.3 The EITI and corruption: existing evidence
From the arguments summarised above, it is clear that some scholars emphasise the potential positive aspects of EITI membership while others remain more sceptical. A review of existing evidence shows that the empirical literature on the EITI and corruption also offers no clear indication of the progress made in tackling corruption among EITI members. Evidence provides mixed conclusions about the direction and significance of changes to corruption outcomes (Rustad et al., Reference Rustad, Le Billion and Lujala2017). However, limited discussion exists about the various approaches adopted to examine this complex intervention. This is particularly important in this context, given that many existing studies offer new and competing evidence based on methodological debates.
For instance, studies using time-series based approaches have rarely offered positive conclusions concerning the effects of the EITI (e.g., see Öge, Reference Öge2016). These approaches often examine graphical trends in corruption outcomes or use a single group interrupted time-series analysis. Yet, both of these methods impose stringent assumptions about changes in corruption scores in the absence of the EITI. They assume corruption scores would not have changed had members not joined the initiative or that they would change at a linear rate (a rate often inferred from as few as one or two years of outcome data). Furthermore, like other research discussed in this review, these time-series studies also use common corruption indicators, such as the World Governance Indicators’ Control of Corruption Index (CCI) and Transparency International's Corruption Perceptions Index (CPI). It is well-known that such measures often lack a degree of comparability over time. For example, the World Bank cautions against comparing the CCI's scores across different years because the scale changes annually.Footnote 4 This issue may confound any inferences drawn from an analysis doing so. Öge (Reference Öge2016) justifies the position that in the absence of alternative indicators, such indicators remain insightful. Nevertheless, this point highlights a broader underlying limitation evident in this evidence.
Some evidence also exists using a ‘counterfactual’ based approach: using a control group of non-EITI countries to determine changes in corruption outcomes that would occur in the absence of the EITI. For example, Ölcer (Reference Ölcer2009) compares EITI members’ CCI scores for 2007 to non-EITI countries’ scores. According to his analysis, perceptions indicate that the incidence of corruption is, on average, higher in EITI countries compared to non-EITI countries. However, Pitlik et al. (Reference Pitlik, Frank and Firchow2010) and Lujala (Reference Lujala2018) highlight that countries’ decision to join the initiative is not random. This raises concerns of an endogeneity problem caused by selection bias (i.e., this is effectively like comparing apples with oranges). One method used to address this issue of selection bias includes controlling for variables correlated with countries’ EITI membership in a regression specification (see Papyrakis et al., Reference Papyrakis, Rieger and Gilberthorpe2017). Nevertheless, a discussion by Kasekende et al. (Reference Kasekende, Abuka and Sarr2016) emphasises that this approach remains biased if unobserved variables jointly determine both the outcome variable (i.e., corruption) and the selection process (i.e., the decision to join and progress with EITI).
To purge any potential confounding correlation between these factors, Kasekende et al. (Reference Kasekende, Abuka and Sarr2016) adopted a two-stage treatment effects model to analyse the effects of the EITI. However, numerous practical limitations also exist in this application. One example is that the year when countries joined the EITI varies. While they try to estimate the likelihood of a country joining the EITI over time, the two-step econometric estimator they employ does not allow variables to simultaneously determine the outcome variable and EITI membership status. This represents a problem because the reasons why a country may join the EITI (e.g., to improve aid commitments or FDI) may also be factors that EITI membership enhances (Lujala, Reference Lujala2018). They circumvent this issue by including lagged values (from before the inception of the EITI) for a selection of variables. Beyond the subjectivity of some of their decision to lag (or not lag) particular variables, this also risks introducing a dynamic form of panel bias to this estimator design for static econometric models (as the authors also note).Footnote 5
Other strategies for dealing with this problem include using a Synthetic Control Method (SCM). At an individual country-level, Fenton Villar and Papyrakis (Reference Fenton Villar and Papyrakis2017) show perceptions of corruption improved in Zambia following its commitment to the EITI. López-Cazar et al. (Reference López-Cazar, Papyrakis and Pellegrini2021) further replicated Fenton Villar and Papyrakis’ (Reference Fenton Villar and Papyrakis2017) SCM approach, showing that, while corruption scores did not improve in Colombia, Guatemala, Honduras or Peru following their commitment to the EITI, they did improve marginally in Trinidad and Tobago. However, a broader challenge of the SCM approach exists in creating well-behaved synthetic comparisons for each member of the EITI to replicate the analysis across the globe.
Alternatively, Sovacool et al. (Reference Sovacool, Walter, Van De Graaf and Andrews2016) also introduced an approach that defines a group of EITI countries that joined the initiative any time before 2014 and a ‘control group’ that did not join the EITI before this date. Measuring corruption using the CCI, the study then compares the average differences in changes in corruption scores between the EITI's inception in 2002 and 2014. They find that joining the EITI has not been associated with statistically significant changes in corruption. The key drawback of such a strategy is that it may provide a conservative estimate of the true effect of the EITI if the initiative's effects grow with time. This conservatism grows innately with the degree of late adopters in the defined group of EITI countries.Footnote 6
3. Methodology
Reflecting on the limitations of studies in this existing literature, this study examines the progress made by EITI countries in improving corruption indicators using a contemporary indicator known as the Bayesian Corruption Indicator (BCI). Consistent with Sovacool et al. (Reference Sovacool, Walter, Van De Graaf and Andrews2016), this study also creates a comparative analysis comparing changes in EITI members’ corruption scores to changes in non-EITI countries’ scores. To address the endogeneity issues caused by self-selection, it also explores combining an entropy balancing approach with a difference-in-difference framework to minimise the difference in baseline characteristics between members and non-member countries. This section continues by describing the details of the variables and then provides further information on the estimation strategy.
3.1 Data
In this study, we examine the relationship between the EITI and changes in countries’ corruption outcomes using the BCI developed by Standaert (Reference Standaert2015). The BCI is a composite index of the perceived overall level of public corruption. It combines information from 17 international surveys and 110 different survey questions covering perceptions of corruption. The values of the BCI variable range between 0 and 100 (with higher values given to countries perceived to have a higher level of corruption).
Due to insufficient quality data on the actual level of corruption, this corruption measure leans on well-established arguments that perceptions provide valuable insights into the incidence of corruption (Charron, Reference Charron2016).Footnote 7 Of course, perceptions may still deviate to some degree from actual levels of corruption. However, perceptions of corruption are also important outcomes in their own right in this context. Perceptions are considered an important factor from a development perspective because they also directly matter for many outcomes (Kaufmann et al., Reference Kaufmann, Kraay and Mastruzzi2006). For example, some highlight that citizens’ perceptions of these issues have further exacerbated local conflicts in many contexts and increased demand for consumption (Collier, Reference Collier2017). Perception outcomes may also capture instances where societal improvements occur due to a reduction in misconceptions about corruption, which may have previously been prevalent due to the lack of clear information. This last point reflects many members’ motivation to join the EITI. Zambia's public officials, for instance, expressed a desire to join the EITI to address misconceptions about corruption and restore public confidence in the government (Fenton Villar, Reference Fenton Villar2020).Footnote 8 It is ambiguous whether we should expect actual or perception-based corruption measures to be more responsive to the EITI, but this poses an interesting issue which future research might seek to examine further (data permitting).
Ideally, we would also have a corruption measure focused on the extractive industries, but such international measures do not currently exist. The application of the BCI here is based on the understanding that changes in corruption in extractive industries are also likely to affect cross-sector indicators (even if the extractives sector does not entirely determine the score and it will likely create a conservative estimate of the true effect). It is worth noting that the selection of this type of cross-sector measure is consistent with the type of measures used in existing EITI studies. This creates an interesting exercise that examines whether the use of this new indicator draws similar conclusions to commonly-used indicators.
To further justify why this measure makes a particularly interesting indicator compared with other corruption indicators already featuring in the EITI literature (such as the CCI and CPI), it is important to understand the relative methodological strengths of the BCI. The BCI keeps the scaling of its index and its model parameters estimating its index constant across time, for instance. This enables it to provide a greater deal of comparability between estimated corruption scores over time than these other alternative indicators. Also, even though the BCI draws information from multiple sources of information (in fact it is the same sources as the CCI), the BCI's aggregation approach averts the need for additional data manipulations during the computation of the indicator (such as imputation and sub-level aggregation).
The interested reader may see Standaert (Reference Standaert2015) for a detailed description of the composition of the BCI and an empirical review comparing it with other available corruption indicators mentioned in this text. Some of the key findings from the article show that the between-correlations (the correlation between the mean values for each country) between the BCI and other comparable international corruption indicators are generally very high (above 0.9). The within-correlations (between the demeaned values) are, however, much lower (below 0.5). This means that, while the choice of indicator might not have a large effect on the results in a cross-sectional study, the differences between indicators may be significant in a study using time-series or panel data (such as this study). These potentially important differences help to motivate the interest in this study in re-examining the relationship between the EITI and changes in corruption scores using the methodologically more robust BCI measure.
This analysis measures changes in corruption scores between 2002 (the year before the first set of public commitments made to the EITI by participating countries) and 2016 (the year of a major overhaul in the EITI's standards).Footnote 9 The period examined is largely recognised as the initiative's inception period before a significant change in the scope of EITI's standard, and the way members’ progress was measured, occured. As noted in the conclusions below, future research might consider the implications and experiences of members transitioning to the new standards when more data for the BCI is made available and sufficient time has lapsed for their effects to have convincingly transpired.
This study focuses on changes in corruption outcomes of developing countries identified by their eligibility for Official Development Assistance. This reflects that the motivations of the limited sample of developed economies, such as the U.K., Norway and Germany, for joining the EITI have been intrinsically different from those of developing countries (which the initiative originally targeted). For example, the U.K. joined the initiative primarily because of the role that its Department for International Development had in brokering the EITI's inception and Norway joined because the EITI Secretariat is in Oslo and also due to the country's leadership role in the industry. Similarly, Germany is a major consumer of raw materials and aid donor among developed economies, and so the country's membership was intended to reflect its role as a ‘role model’ to aspiring nations. Hence, it largely represents a notional commitment (e.g., to ‘lead by example’), as opposed to a domestic development one per se (Short, Reference Short2014; von Klencke, Reference von Klencke2016).Footnote 10
Member countries participating in the EITI go through different stages. The first stage is for the government of the participating country to publicly announce its commitment to the EITI. After this, it needs to develop a work plan that sets concrete objectives (regarding ways to improve transparency in the extractive sector) and establish a multi-stakeholder group together with companies and civil society. Once these steps have been carried out, the country moves to the second stage by formally applying to the initiative to become a candidate country. Candidate countries are then required to work towards fulfilling the initiative's various standards (requiring full and timely disclosure of financial and contractual information stipulated in its transparency standards, a continuous and effective functioning multi-stakeholder group, and so forth). The EITI uses audits to assess when countries reach the third stage. In the third stage, countries are validated that they are compliant with the EITI's standards (Papyrakis et al., Reference Papyrakis, Rieger and Gilberthorpe2017).
To verify members’ compliance with its standards and ensure the requirements are upheld properly, the initiative's international secretariat's validation team reviews information provided by each member according to its standards. Here the onus is on the country multi-stakeholder group and supporting national bodies to provide evidence of compliance with the standard. After reviewing the information provided by each country, the secretariat's validation team will offer the multi-stakeholder group the opportunity to discuss preliminary findings at a teleconference and may also undertake targeted virtual stakeholder consultations, consult the international secretariat's country team or draw on external experts to seek further information at this stage. In exceptional cases, especially if there are severe concerns related to stakeholder engagement, the validation team may visit the country to undertake in-person consultations and seek further information. The EITI board in the international secretariat make the final assessment determining the compliance status of each country based on the evidence and recommendations from the validations committee and other appointed external experts.Footnote 11
In this analysis, a binary variable represents a country's EITI status. This equals 1 if a country has made a public commitment to the EITI, and 0 otherwise. Data for countries’ histories concerning the EITI derives from the online EITI country index (EITI, 2020).Footnote 12 This analysis also examines the difference in changes of perceptions of corruption among a subgroup of those countries that have complied with the intended intervention (i.e., among just countries that are verified compliant EITI members). The binary variable used for the subgroup analysis equals 1 if country i is verified an EITI-compliant member country, and 0 if country i did not commit to the EITI before 2016. Non-compliant EITI-committed countries are not included in the sub-group analysis (i.e., this compares compliant and non-committed countries only). The analysis does not conduct a subgroup analysis comparing members that reach candidate status to non-member countries because very few members in the sample period did not graduate to candidate status (see online appendix 1). Some further analysis does, however, consider interacting these EITI variables with the length of time each country has been a committed or compliant member. Appendix 1 lists information on the sample of 78 countries included in the analysis and their EITI status. This includes a sample of 33 countries committed to the EITI which have sufficient data available for the analysis, 21 of which were verified compliant with the EITI standard by 2016.Footnote 13
For data pre-processing purposes described below, the data compiled for this analysis also includes information on country characteristics related to both the corruption outcome and the EITI self-selection process. Variable selection is informed by relevant studies by Pitlik et al. (Reference Pitlik, Frank and Firchow2010) and Lujala (Reference Lujala2018) who examine factors correlated with EITI membership. This includes variables for GDP per capita, as well as the relative economic size of natural resource rents, trade, FDI and aid to GDP. Other variables include state polity, the incidence of conflict, freedom of the press, and a measure of each country's pre-EITI corruption score (measured by the BCI). Note that a broader literature also justifies that the inclusion of pre-intervention outcomes can improve the efficacy of observational estimators (see Jaciw, Reference Jaciw2016; Fenton Villar and Waddington, Reference Fenton Villar and Waddington2019). Finally, this list of variables includes the interaction between corruption scores and the economic importance of resource rents. This reflects the discussion by Lujala (Reference Lujala2018) that countries with high rents and corruption may be intrinsically less likely to join the EITI. Online appendix 2 further provides detailed definitions and information on each of the variables included in this analysis and online appendix 3 presents a table of descriptive statistics.
3.2 Identification strategy
This analysis starts by adopting a difference-in-difference (DiD) approach, which is analogous to the approach by Sovacool et al. (Reference Sovacool, Walter, Van De Graaf and Andrews2016). A DiD approach calculates the average effect by simply taking the differences in the observed changes in outcomes between countries in the EITI intervention group and those not in the EITI intervention group (a control group). The purpose of the control group is that it measures what would have happened to beneficiaries in the absence of the intervention, thereby controlling for secular trends in the outcome variable.
To formally outline this estimator using a common language to express observations’ potential outcomes, here we further define some basic notation. We denote the EITI intervention variable using a simpler term D, where d ${\in} \{ 0,\,1\}$
. EITI countries remain represented by the value 1 (as described above). T defines a variable representing two time periods, where t ${\in} \{ 0,\,1\}$
. Period zero indicates the intervention baseline year, 2002, and period one denotes the year 2016. Also, X is a matrix of J exogenous pre-intervention characteristics from ${t_0}$
such that ${X_{ij}}$
then denotes the value of the jth characteristic for country i and ${X_i}$
= $[{X_{i1}}, \ldots ,\; {X_{ij}}]$
. With this basic notation, we may index observations’ potential outcomes by the potential states of the intervention variable, where $Y_t^d$
denotes the outcome that would be realized for a specific value of d in period t.
Using the potential outcomes notation defined above, the DiD estimator's average effect is formally described by the notation $E[Y_{t + 1}^1 -Y_{t0}^1\,|\,D = 1)] - E[Y_{t + 1}^0 - Y_{t0}^0\,|\,D = 1)$
. In a regression framework, the DiD approach using panel data is equivalent to estimating the following equation:
where $\mathrm{\Delta }Y$
represents the change in the corruption outcome variable between 2002 and 2016 for country i. D is the binary EITI variable described above. The parameter α estimates the time-trend (i.e., the average change in corruption outcomes) observed among control countries. β is the estimated average difference in changes in corruption scores between EITI members and the control group. Finally, ε is the error term.
This estimator infers a common trend which assumes that in the absence of the EITI intervention, the difference between treatment and control groups’ outcomes would remain the same over time (i.e., that $\Delta Y_t^d \bot {D_t}\; |\; D = 0$
, where $E[Y_{t + 1}^0 - Y_{t0}^0\,|\,D = 1)$
= $E\textrm{[}Y_{t + 1}^0 - Y_{t0\; }^0\textrm{|}\; D = 0]$
). It is important to note that this does not mean that there is no trend in the outcome variable in the counterfactual state (just that the trend is analogous across the treatment and control groups). It also does not mean that it requires the level of the outcome variable for the two groups to be the same in the pre-treatment era. However, empirical assessments support that minimising the baseline differences in compared sample characteristics can help to improve the efficacy and reduce the bias associated with observational DiD estimators (see Glazerman et al., Reference Glazerman, Levy and Myers2003; Jaciw, Reference Jaciw2016). In other words, this indicates that comparing groups that are observationally similar at baseline can increase the plausibility of the common trends assumption.
To do justice to the portion of the methodological literature that supports using data pre-processing approaches to minimise the baseline differences between compared sample characteristics, this analysis also considers adjusting the simple DiD framework using Hainmueller's (Reference Hainmueller2012) method of entropy balancing.Footnote 14 The entropy balancing approach involves creating an adjusted control group of non-EITI countries using a re-weighting procedure that minimises the baseline inequalities between member and non-member countries’ characteristics.Footnote 15 More formally, here the counterfactual outcome is denoted $E[Y_t^0|\mathop{D}\limits^{\frown}1] = ({\sum _{\{ i|D = 0}}{Y_i}{\omega _i}/{\sum _{\{ i|D = 0}}{\omega _i})$
. The outcome variable in the difference-difference regression framework described above is simply adjusted using the procedures estimated weights $(\omega )$
:
Hainmueller (Reference Hainmueller2012) discusses in greater detail the similarities and advantages of entropy balancing compared with alternative pre-processing approaches available, such as the better-known propensity score matching method. In particular, they highlight the practical advantages caused by its approach directly measuring the balance of covariates (as opposed to, say, a propensity score). This approach prevents the need for the researcher to manually iterate between modelling the propensity score and checking whether pre-specified covariates are stochastically balanced and also ensures pre-specified covariates are balanced (which indirectly matching on propensity scores does not guarantee).Footnote 16 Through simulations and empirical evidence from within-study comparisons, they also highlight entropy balancing's appealing finite sample properties and both demonstrate the estimator's efficacy (ability to mitigate bias) relative to other common matching techniques. The efficacy of the entropy balancing estimator has also recently been confirmed in further empirical testing provided by Matschinger et al. (Reference Matschinger, Heider and König2020) and Wang (Reference Wang2019).
4. Results
We now examine the results of this analysis. First, panel A in table 1 compares the baseline characteristics (from 2002) of the group of EITI-committed countries to the control group of non-EITI-committed countries. The comparison shows that the characteristics of the EITI and non-EITI countries are statistically similar in some respects, but also considerably different in others. For example, while the standardised mean differences (SMDs) are very small for covariates related to countries’ polity (−0.035), FDI (−0.07) and press freedom (−0.058), SMDs for factors such as log GDP (0.810), aid (−0.815) and corruption (−0.749) are particularly large and statistically significant. Panel B in table 1 provides details of this comparison limited to a sub-group of EITI countries which progressed to reach compliant status during the study period. We see the same pattern also exists here: log GDP, aid, and countries’ corruption scores remain significantly different in EITI-compliant countries compared to non-EITI-committed countries.
Summary of country characteristics in 2002
Notes: SD = Standard deviation. SMD = Standardised mean difference. SMD is calculated by dividing the difference between treatment and control group mean values by the pooled standard deviation.
*** corresponds to a 1% level of significance from a t-test with robust standard errors.
A degree of selection bias between these groups at baseline is not necessarily an issue, given that a DiD estimator may intrinsically account for this type of bias where the assumption of common trends holds. However, as noted in the description of our identification strategy, increasing the similarity of observed groups may help to improve the plausibility of this estimator. The column headed ‘Adjusted Control’ (Adj. Control) in table 1 provides the results of re-weighting the control group using the entropy balancing procedure. The comparison shows that the entropy balancing procedure works well in creating an alternative control group that is observationally comparable to the EITI-committed group at baseline. The adjusted control group's SMDs are negligible across the included covariates, and this is also the case when performing the same re-weighting procedure to balance the control group with the group of EITI-compliant countries (see panel B). Hence, we continue further reflecting on the findings inferred from both unadjusted and adjusted control groups (as well as examining the common trends assumptions below).
However, before doing so, it is also interesting to note some of the differences between countries that have progressed to compliant status during this period in our sample. Comparing the means in panels A and B of table 1, we see that compliant EITI countries are more aid-dependent and less democratic than the average EITI-committed country. We also find, on average, compliant countries derive from countries with lower incomes but where foreign investment appears relatively more important. Finally, albeit the values are somewhat similar, the perceived levels of the incidence of corruption, the relative economic importance of natural resource rents, and their interaction are also higher in compliant countries. These findings are in line with the expectation and previous evidence (see Lujala, Reference Lujala2018) that the countries which are hypothesised to benefit the most from implementing the EITI are also the ones most likely to progress with its implementation. This implies that a simple comparison of the size of the average relative change in corruption scores between EITI-committed countries and the sub-group of EITI-compliant countries may not be directly attributable to countries’ progression in the EITI.
Further attempts to address this issue using the entropy balancing technique to minimise the baseline inequalities between EITI-committed and EITI-compliant countries were, though, unsuccessful. This reflects a known limitation of this re-weighting procedure, which is that it may not converge to a balanced solution. The universe of countries here is very small (only 12 EITI-committed countries in the sample may serve as a control for the group of compliant EITI members). This may explain why limited overlap exists between these two samples. The analysis, therefore, continues reporting and discussing the differences between the results of EITI-committed countries, and a sub-group of compliant members, to each group's respective adjusted control group of non-EITI countries. However, it should be cautioned that this only provides intuitive evidence of the difference in the effects between committed and fully compliant with the EITI. As noted above, it is not clear that the difference in effects can be attributed to countries’ progression with the implementation of the EITI.
Table 2 presents the analysis examining the changes in corruption scores according to the BCI. The changes reported are scaled using the pooled standard deviation of the level of corruption in the baseline year (2002). When using an unadjusted control group (the one not using entropy balancing), the results in panel A in table 2 show that between 2002 and 2016 the difference in changes in corruption scores between EITI-committed and non-EITI countries was small. Here, while the coefficient labelled change (Chg) shows perceptions of corruption improved (i.e., scores decreased) by 0.070 standard deviations in non-EITI countries during this period, the DiD coefficient shows the average decrease in corruption scores was only slightly larger (0.025 standard deviations) in EITI-committed countries relative to non-EITI countries. Limiting the analysis to the sub-group of EITI-compliant countries, the estimates in panel B in table 2 show similar but slightly larger improvements in EITI countries. The DiD estimate indicates the change in corruption scores was 0.128 standard deviations lower in EITI-compliant countries relative to non-EITI countries. However, in neither instance are the DiD coefficients statistically significant.
Changes in Bayesian Corruption Indicator score
Notes: ‘Chg’ provides the change in the corruption scores in the control group measured in standard deviations (i.e., it is the parameter α in the DiD regression equation in section 3.2). ‘DiD’ reports the corresponding difference-in-differences between the EITI and control group (i.e., it corresponds to the β coefficient in the DiD regression equation in section 3.2). The results are estimated using OLS regressions. ‘Control’ refers to the estimates using the unweighted control group and ‘Adj. Control’ the estimates using the weighted control group; weights are derived from the entropy balancing approach described above. ‘Obs.’ is the number of countries included in the analysis.
*, **, *** correspond to a 10%, 5% and 1% level of significance, respectively. Robust standard errors are reported in parentheses.
In contrast to the estimates from the unadjusted control group, looking at the estimates from the adjusted control group obtained from the entropy balancing procedure, we find that the relative improvements in perceptions of corruption measured by the BCI are statistically significant. The results show that, after adjusting for the baseline differences in country characteristics, the average change in corruption scores between 2002 and 2016 was 0.347 standard deviations lower in EITI-committed countries relative to non-EITI countries. This estimate is significant at a 5 per cent confidence level. The results also indicate improvements in perceptions of corruption were slightly better in the sub-group of EITI-compliant countries. The estimates show that the average change in corruption scores was approximately 0.391 standard deviations lower in EITI-compliant countries relative to the adjusted control group. In this instance, the DiD coefficient is significant at a 1 per cent confidence level.
Much like it is not simple to comprehend the practical meaning of a point change in the underlying BCI index, comprehending the magnitude of the size of this change reported in standard deviations is not very simple either. However, one benefit of examining the results measured in standard deviations is that this figure can be easily transformed (or converted) into an intuitive and well-known improvement index based on Cohen's U3 index (Cohen, Reference Cohen1988).Footnote 17 Concerning the results reported above, an effect size decreasing the average corruption score by 0.347 (0.391) standard deviations in the treatment group indicates that 64 per cent (65 per cent) of the treatment group score lower than the mean score in the control group. In other words, it corresponds to a 14 per cent (15 per cent) relative improvement. This does not appear to be a transformative improvement but it still appears to indicate a marked improvement which is by no means meagre.
Estimations using both the unadjusted and adjusted control group infer that perceptions of the incidence of corruption have improved in EITI countries. The discrepancy concerning the significance of these findings does raise a point for concern though. One explanation for the difference in these findings might be that the pre-treatment differences in the unadjusted control group are creating non-parallel outcome dynamics (which would bias the DiD estimator). To examine this issue further, here we assess the validity of the common trends assumption associated with the DiD estimator using an ‘In-time Placebo Test’. This placebo test re-applies the same analysis to outcomes before countries were exposed to the EITI intervention. It is expected that before the inception of the EITI, the estimated difference in the changes in corruption between the control group and EITI group should not be larger than what we might expect to occur by chance. Empirical analysis has shown this type of test to be a useful method for detecting poor-performing observational estimators which are more susceptible to bias (Glazerman et al., Reference Glazerman, Levy and Myers2003).
In this application, we examine a placebo test for the period 1997 to 2002. This covers the period when BCI data for the full sample of included treatment and control countries is available. The results of this robustness test are presented alongside the main estimates in table 2. They indicate that changes in corruption were significantly different between EITI-committed countries and the unadjusted control group even before the EITI's inception. The DiD coefficient shows that, in the period between 1997 and 2002, changes in corruption scores were 0.038 standard deviations higher in EITI-committed countries relative to non-EITI countries. This difference is significant at the 1 per cent confidence level. Similarly, focusing on the analysis for the sub-group of EITI-compliant countries in panel B in table 2, we see that changes in corruption outcomes were also significantly higher (approximately 0.037 standard deviations) in EITI-compliant countries before its inception. Again, this finding is also significant at the 1 per cent confidence level. This undermines the plausibility of the DiD estimator's assumptions for the results using the unadjusted control group. The placebo test results using the adjusted control group, on the other hand, show the DiD coefficients in the pre-EITI period are not significant. This, therefore, increases our confidence in the main findings using the adjusted control group: that the perceptions of corruption have improved significantly (and corruption scores decreased) in EITI countries relative to non-EITI countries.
To further examine the robustness of the results from the estimator using entropy balancing, another type of test considers a natural extension of Rosenbaum's sensitivity analysis (a test that is widely used for matched observational studies) (see Rosenbaum, Reference Rosenbaum2002). Adapted by Soriano et al. (Reference Soriano, Ben-Michael, Bickel, Feller and Pimentel2021) for the entropy balancing approach, this test assesses the degree to which the estimates would change due to ‘hidden bias’ caused by unobserved confounding variables. The test's results provide a critical value of 2.45 for the estimates using the EITI-committed country sample and 2.65 for the model using the sub-sample of EITI-compliant countries. In other words, these unobserved factors would need to more than double the likelihood of being an EITI country to overturn the main findings. This test cannot rule out the existence of some degree of unobserved confounding in the current estimates. However, these test results do indicate that the reported estimates are reasonably insensitive to this potential type of bias based on common thresholds used to interpret the test's critical value.Footnote 18
Some further analysis also considered examining whether progress among EITI members is related to countries’ characteristics. This included, for instance, interacting the EITI variable with the length of time each country has been a committed or compliant member, as well as other baseline characteristics (such as natural resource rent dependence, aid dependence, and the level of corruption). The coefficients of the interaction terms testing such effects were consistently small and statistically insignificant.Footnote 19 This may suggest that changes in corruption outcomes are relatively abrupt rather than gradually growing over time (similar to the trajectory of the effects reported by Fenton Villar and Papyrakis (Reference Fenton Villar and Papyrakis2017) in Zambia). However, one issue here is that the time variable may be a poor indicator of the maturity of the initiative. The discussion above highlighted that several countries were seen to be ‘dragging their heels’ with the implementation of the EITI's standards. Some countries progressed very slowly despite having been members for several years while others adapted to the standards very quickly. This suggests a more accurate representation of maturity may well simply be each member's EITI status (as seen in the results above).
Finally, additional results available in online appendix 4 compare these findings to those using three corruption indicators more conventionally adopted in the cross-country literature on the EITI and corruption. This includes examining the estimated outcomes using the Political Risk Services (PRS) Group corruption indicator, the World Governance Indicators’ Control of Corruption Index and Transparency International's Corruption Perceptions Index (online appendix 2 reports further variable descriptions).Footnote 20 As discussed previously, the additional results from these indicators should be interpreted with a degree of caution. The CCI and CPI indicators inherently host limitations concerning their measurement of corruption over time and the PRS indicator creates annual scores only using a single set of experts' subjective opinions and offers little transparency about the consistency in their measurement. Also, in two instances the entropy balancing procedure did not converge to a consistent weighting solution. This explains why the adjusted control results are not reported for the PRS indicator when using the sample of all committed EITI countries or the CPI for the sub-sample of EITI-compliant countries.
From these additional results, we generally see that the estimates obtained from these alternative corruption indicators are largely aligned with those using the more contemporary BCI outcome variable. In particular, the DiD estimates are negative for both the adjusted and unadjusted control groups.Footnote 21 However, the lack of correspondence concerning the statistical significance of the results also highlights that the headline conclusions drawn from existing flagship EITI studies using these alternative indicators (such as Sovacool et al., Reference Sovacool, Walter, Van De Graaf and Andrews2016) may be sensitive to the type of EITI member and the corruption measure used, as well as the estimator applied. For example, if we examine the results of the CCI indicator in appendix 4 using the unadjusted control group, we see that the results show that the relative changes in corruption scores among EITI members are only significant among the sub-sample of compliant countries (indicating conclusions are temperamental depending on the definition of EITI member type, as also recently shown by Sovacool (Reference Sovacool2020)). These results also contrast with the results from the more robust BCI indicator (which is based on the same sources of information as the CCI). The results from estimates using the BCI and the unadjusted control group indicate that the changes have not been significant in the EITI-compliant group. This emphasises that the corruption indicator used is also an important factor determining the conclusions of existing studies.
5. Conclusion
The advent of the EITI has been a major step forward towards generating a more transparent extractive sector in countries across the globe. However, interest in the initiative in recent years has stimulated a vigorous debate questioning its sufficiency to tackle the sector's corruption problem. While its proponents highlight the potential benefits of the EITI to the governance of the extractives sector, critics often draw attention to the common limitations surrounding its implementation. The question, therefore, remains whether its members have witnessed any improvements in their scores from international assessments of the prevalence of corruption. This study uses a state-of-the-art corruption indicator – the Bayesian Corruption Indicator – combined with an estimation strategy using a DiD model and an entropy balancing technique, to address the measurement and self-selection issues prevalent in existing studies on the progress of EITI members. It finds that, on average, corruption scores have improved significantly in EITI countries compared to non-EITI countries after adjusting for baseline differences between these groups of countries’ characteristics.
Further analysis also looks at the relationship between the EITI and corruption among a sub-group of members compliant with the international standards. The results provide even stronger evidence of the relationship between EITI membership and improvements in corruption outcomes. However, a limitation of this study remains that it is unable to generalise whether changes have occurred more strongly during a particular stage of implementation (e.g., when countries are joining the initiative or after they become compliant). This is largely due to the differences in the ‘expected potential benefits’ profile of compliant members. It may be that those countries that have already progressed with the initiative stand to benefit the most from its implementation. If this is the case, these countries may naturally have benefited more from commitment to the EITI than the slow and late adopters. Nevertheless, reflecting that on average more corrupt and resource-dependent countries with lower incomes have been likely to progress with the implementation of the initiative (the type of country the intervention was most keenly intended for at the EITI's inception), it is intuitively encouraging to see from these findings that progress has been strongest among the sub-group of compliant members (even if the results are only slightly stronger).
Concerning the policy implications of this research, this evidence does not advocate the EITI as the policy panacea that will lead to the eradication of corruption in the extractive industries. It is clear from discussions on its implementation that the initiative must continue to strengthen its standards and increase its stringency and local outreach to ensure it remains relevant (particularly as its membership continues to mature). Nonetheless, this evidence supports the positive role that the EITI may contribute in helping to develop the policy environment, infrastructure, and capacity required to stimulate better governance of the extractive industries. As highlighted by Van Alstine (Reference Van Alstine2017), the initiative may use transparency as a necessary ‘entry point’ to help build a better mutual understanding between stakeholders and stimulate changes in public governance and management. With this in mind, the recent package of changes to further support the EITI's standards and verification process remains relatively fresh, and so their full implications may take some time to transpire. Considering the implications of transitioning to new standards on EITI members’ experiences may pose an interesting line for future research. Furthermore, examining how institutional changes to the EITI's standards have caused differences in early and late adopters’ experiences with its uptake and implementation also offers another interesting avenue for future research.Footnote 22
Cross-country assessments of the EITI, such as this study, offer important insights into suggestive empirical regularities involving changes in corruption outcomes among members across the globe. Nevertheless, as this study also highlights, methodological challenges and limitations inevitably exist with this approach. Further research might consider exploring sub-national variations in citizens’ interactions with public officials and the EITI as an alternative approach for identifying the distribution of the EITI's effects. Consideration should also be given to research into understanding how different modes of information provision and stakeholder deliberation may help to maximise the benefits of increased transparency.
A final point for discussion concerns whether this type of mechanism is relevant or could be expanded to other sectors. This has been a lively and interesting issue even among stakeholders in the EITI itself. For instance, in Ghana, extensive work has taken place over the past 10 years to try to incorporate natural-resource-based industries beyond the mining sector into the remit of its EITI scheme. This included a bill put to its parliament in 2012 to expand the scope of the EITI to cover other sectors such as the forestry and fishery sectors. Expanding the EITI's scope to the forestry sector has also been a prominent issue in other countries, such as Tanzania.Footnote 23 This highlights that the EITI model does indeed appear highly relevant to other sectors too. Important factors that typically distinguish the EITI from many other (often local) transparency schemes include that its mechanism is based on a set of internationally-developed technical standards and that it has a validation process run by an independent international secretariat (which also provides leadership through capacity-building and training activities). Other related topics that are drawing considerable advocacy towards the need to improve transparency concern land deals and land registration.Footnote 24 Whether an EITI-type multi-stakeholder initiative with an international validation and membership scheme could help to support efforts on such issues warrants further research.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S1355770X21000383
Conflict of interest
The author(s) declare none.
