Study Context

Our study covers 11 provinces in the densely populated Red River Delta region, with Vietnam’s capital, Hanoi, at their center.^{Footnote 3} From 1990, when market reforms shifting the country away from central planning began in earnest, to 2013, Vietnam was one of the fastest growing economies in the world (World Bank 2017a). Following the 1999 passage of a new company law that led to rapid growth in the number of domestic private firms, through 2013, no region grew faster than the Red River Delta (Vietnamese General Statistics Office Multiple Years). Development of government institutions, however, significantly lagged economic growth. Vietnam’s regulatory system remains among the world’s most cumbersome, corrupt, and opaque (Transparency International 2017; World Economic Forum 2017). Worsening overall transparency in the drafting of new regulations was even highlighted in VCCI’s annual report on ministerial efficiency (VCCI 2014).

Vietnam’s domestic private SMEs are spread throughout the country, hard to reach, and have limited technological capacity. This makes them exactly the type of firms for which regulatory compliance is hardest to achieve (MOLISA 2016) and contributes to the prevalence of industrial accidents. They also see the policy environment as unfair: in a 2016 survey of a representative sample of domestic private firms, 61% said the state is biased in favor of large, elite, private firms, particularly in regard to firm entry, land access, and procurement (Malesky Reference Malesky2016). This underlines how the chief beneficiaries of any initiative to expand access to the policy-making process would be politically unconnected SMEs.

Vietnam’s Law on the Promulgation of Legal Normative Documents, beginning in 2008, formally mandated that all ministries publicly post all draft regulations for a public comment period of at least 60 days.^{Footnote 4} Even with this requirement placed by the government on itself, however, adherence by ministries has been poor and inconsistent (Online Appendix A shows variation across ministries in formal rules on the posting of draft documents, the frequency of posting, and the length of delays along the way). Consequently, our study essentially involves experimenting with implementation of an insufficiently utilized government policy.

Within this context, the first task in our research design was to identify an appropriate not-yet-completed draft regulation on which to conduct our experiment.^{Footnote 5} We arranged a national workshop to explain our needs to key officials responsible for designing business regulations and learn about ministerial plans for regulations to be drafted in the coming months. We settled on a planned regulation by the Ministry of Labor, Invalids and Social Affairs (MOLISA)’s Worker Safety Department, which aimed to introduce protections for workers dealing with hazardous chemicals.

Experimental Sample and Design

We created an initial sampling frame of 18,701 firms^{Footnote 6} from a national firm list, which VCCI accessed from Vietnam’s General Department of Taxation (GTD). However, consultation with a professional survey firm led us to conclude that it was necessary to first screen the list to ensure firms were legitimate, active, and operating in sectors that used dangerous chemicals. In Vietnam’s highly dynamic market, firms frequently go out of business, change operations, or simply disappear without notifying the GTD. There are also “ghost firms” with tax codes and contact information, but no actual operations, which may be fronts for illicit activities, such as money laundering. Such problems with sampling frames are, in fact, relatively common in emerging economies. As a result, sending interviewers to firms from the GTD list without screening would have been extremely inefficient, wasting valuable time and financial resources.

Our screening, performed primarily by phone, bore out the above concerns. More than 11,000 firms were eliminated because they were no longer active. A further 3,550 firms had to be dropped because they were incorrectly listed as operating in sectors that used chemicals or refused to answer questions about the sector in which they operated. We were left with 2,635 firms verifiably operating in chemical sectors in our target provinces. Of these, 1,200 agreed to participate (an acceptance rate of 46%).^{Footnote 7} This satisfied our target of 300 firms per treatment group.^{Footnote 8}

For the baseline round, our research teams visited sample firms over a three-month treatment period (October 2014 to January 2015). Visits closely followed our receipt of a draft version of the hazardous chemical regulation from MOLISA, dated September 12, 2014. All visits involved a tablet-based survey with 37 questions about the CEO, firm size and performance, and feelings about government’s regulatory legitimacy. Blocking on available data regarding firm size, two-digit industry codes, and the CEO’s gender, we assigned the 1,200 firms across our three treatment groups. For all firms, we insisted on meeting the CEO. We were successful 64% of the time. In cases where we could not meet the CEO, which generally occurred because they were located in a different province or country, we met with the highest ranking onsite manager.

We illustrate the key differences across treatment groups in Figure 1. The first group (henceforth, the Control) received our placebo treatment and consisted of 388 firms at baseline.^{Footnote 9} Control firms were shown a placebo video presentation about VCCI efforts to mobilize input from firms on draft regulations. The video was shown on the tablet and lasted six minutes and 48 seconds.

FIGURE 1.

Experimental Treatment Conditions

The second group (T1) consisted of 295 firms at baseline and was designed to test the information mechanism (M2 above). To this end, the invitation letter mailed to T1 CEOs ahead of baseline round interviews included a copy of the draft regulation and a distinctively blue-colored form that summarized 11 key clauses identified by chemical safety experts, hired by our project to provide advice and firm auditing, as particularly likely to require firm-level investments of time, effort, and money. Further, after completing the baseline survey, T1 firm representatives were shown a video on the labor protection aims of the target draft regulation and the operational effects of the 11 clauses. As with the placebo video, this video was tablet-based and lasted about six and a half minutes (6:24).

The third and final group (T2) included 517 firms at baseline and tested the legitimacy mechanism.^{Footnote 10} After receiving the entire T1 treatment experience, these firms were asked to respond to a tablet-based series of open- and closed-ended questions on the costs, quality, and need for improvement for each clause. According to our chemical safety experts, 28% of these firms offered comments that were of sufficient substance to have potential use for altering the regulation. All others only answered the close-ended questions or offered feedback on the regulation that lacked clear enough policy implications for regulators to respond.

All T2 firms subsequently received a report that described results of the participation exercise. This was sent to firms through the mail in late April 2015, more than three months after our final baseline visit.^{Footnote 11} The report included information on all changes made by the government’s drafting committee, as of its April 13 revision, and responses to a subset of comments that our chemical safety experts identified as particularly salient. To test whether this additional round of contact with government had an influence on compliance, we randomly assigned 97 T1 firms to also receive the report. We analyze the effect of the response report in “Sensitivity Tests” below.

Table 1 lists the key 11 clauses highlighted in the T1 video and how these clauses had been revised as of the mailing of the report to firms. Overall, few changes could be connected to firm comments.^{Footnote 12}

TABLE 1.

Clauses in Original Draft and Final Draft of Hazardous Chemical Regulation

Bolded words depict changed language.

An area of potential concern relating to our treatments is the extent to which they were absorbed by sample firms. In the case of T1, it is important to determine whether firms were actually better informed about the hazardous chemicals regulation. In the case of T2, absorption implies that firms were convinced that their participation was meaningful.

Figure 2 presents responses to three questions on awareness, understanding, and perceptions of quality in the endline round survey that speak to treatment absorption. Assessments of each were extremely low in the Control (20%, 1.2, and 1.21, respectively). These numbers were all significantly higher in T1 (45%, 1.54, and 1.69, respectively) and higher still in T2 (58%, 1.8, and 2.0, respectively).^{Footnote 13} The high share of firms that did not remember hearing of the regulation may reflect the regulatory environment’s lack of transparency and the preponderance and constantly shifting nature of regulatory red tape in Vietnam’s transition economy. Furthermore, the absorptive capacity of SMEs may be particularly limited.^{Footnote 14} The large difference in quality assessments points toward our legitimacy mechanism. Participating firms overwhelmingly believe the regulation benefitted from their input. In a separate question in the endline survey, 91% of firms in T2 agreed that “providing comments improved their opinion of the regulation’s quality.” Notably, T1 firms were also significantly more likely than the placebo to acknowledge the regulation’s quality.

FIGURE 2.

Manipulation Checks

Note: Range bars represent 95% confidence intervals; Awareness measured using question: Have you ever heard of this Draft before? (No = 0, Yes = 1) from endline survey; Understanding measured using question: If Yes, could you please rate your understanding of the Draft on the scale from 1 to 5? (5. Fully; 4. Well; 3. Average; 2. Slightly; 1. Not at all) Quality measured using question: How do you rate the quality of this draft regulation relative to the other regulation that you have opportunities to read or give comments on? (5. Much higher; 4. Higher; 3. Similar; 2. Lower; 1. Much lower)

Compliance Monitoring

The endline round began in November 2015 and finished in March 2016. This meant an average of roughly 13 months between treatment and the endline. Requests for return visits to perform compliance audits were framed as a free business support service by VCCI, including technical advice on how to most effectively invest into complying with the target regulation.

To provide this service, we hired a set of auditors with substantial professional experience judging the chemical safety conditions of factories in Vietnam. This experience equipped them to engage firm managers in serious discussions about what constructive and cost-efficient steps could be taken to maximize the odds of being judged to be in compliance by government regulators. Importantly, auditors were not informed of the study’s hypotheses or experimental design. They were simply asked to provide a standard inspection for all firms.

The advanced technical expertise of our auditors was of heightened importance because of the fundamentally low quality of both the original and final draft versions of the target regulation. First, the low quality increased the degree to which well-intentioned firms really did need expert advice in interpreting what the government wanted them to do in regard to chemical safety. Second, it meant that we ultimately had to rely on auditors’ subjective judgments of safety conditions across the key provisions.^{Footnote 15}

Figure 1 notes the sample sizes for each treatment group at baseline and endline. Despite our efforts to frame our return visit as a free business service from VCCI, we experienced significant attrition between rounds. For each of the three treatment groups, the decline was about 30%. Some of it was due to normal churn, with firms going out of business (4% of baseline sample), moving to an unknown location (3%), or changing into a business line that no longer related to hazardous chemicals (2%). Another 249 firms (21%) refused to participate despite still operating in the same line of business.

Importantly, this attrition between rounds was not systematically correlated with features of the treatment groups and thereby is not a threat to our random assignment. Online Appendix D shows that refusal rates were identical and a variety of reasonable and observable covariates were balanced across the three groups.^{Footnote 16}

Outcome 1: Firm Perceptions of Government Legitimacy

Our first analysis examines the legitimacy mechanism (M1) by studying changes in firm perceptions of government regulatory competence. We begin here, rather than with our general hypothesis, for both theoretical and empirical reasons. Theoretically, the legitimacy mechanism is a critical pathway between participation and compliance for both the deliberative democracy and procedural justice literatures. Empirically, we apply a rigorous difference-in-difference analysis using a dependent variable based on measures of legitimacy that were collected in both the baseline and endline surveys. Such an evaluation of change was not possible in the tests of compliance that follow, because we were only able to audit the factories at endline.

Asking about legitimacy in Vietnam required additional caution, because the question needed to be worded in a subtle enough way that respondents did not think we were asking them to question the authority of Vietnam’s Communist Party rule, which would have generated preference falsification and bias in favor of high legitimacy responses (Kuran Reference Kuran1997). For this reason, we chose to restrict our measure of competence and fairness to the specific application of regulations, by asking firms for their level of agreement with the following statement that adheres to Tyler’s (Reference Tyler2006, 357) characterization of process legitimacy as, “appropriate, proper, and just.” Specifically, we asked whether firms agreed with the statement: “Government officials have sufficient understanding of business like this one to effectively carry out their regulatory duties.”^{Footnote 17} At baseline, only 48% of firms in the Control agreed. By the endline survey, views of government had improved significantly, with 64% of Control firms agreeing with the statement.

Table 2 shows the results of a difference-in-difference analysis testing the effect of our randomized participation intervention on firm perceptions about government across the two rounds. Using an ordered probit specification,^{Footnote 18} we regress Legitimacy on our Participation treatment variable. Participation is coded as 1 if the firm had an opportunity to provide comments on the draft regulation (see formulas (1) and (2) below). This applies only to firms in T2.

(1)$$\eqalignb{ & {\rm{Participation}}\;{\rm{Treatment}} = 1\;{\rm{if}}\;{\rm{T}}2 = 1 \cr & {\rm{Information}}\;{\rm{Treatment}}\,\, = 1\;{\rm{if}}\;{\rm{T}}1 = 1\;{\rm{or}}\;{\rm{T}}2 = 1 \cr & {\rm{Reference}}\;{\rm{Category}} = {\rm{Control}}\;{\rm{Group}} = 1\cr\;{\rm{if}}\;{\rm{T}}1 = 0 \,\,\&\,\, {\rm{T}}2 = 0, \cr}$$

(2)$$\eqalignb{ & Legitimac{y_{it}} = {\beta _0} + {\beta _1}Endlin{e_t} + {\beta _2}Participatio{n_i} \cr &+ {\beta _3}Endlin{e_t} \times Participatio{n_i} + {\beta _5}Hanoi + {\beta _6}Femal{e_i} \cr &+ \lambda + \alpha + {u_{it}}. \cr}$$

TABLE 2.

Difference-in-Difference Analysis of Legitimacy Growth Between Rounds

Ordered probit with standard errors, clustered by province-sector, in parentheses (*** p < 0.01, ** p < 0.05, * p < 0.1). Equation (1) is unadjusted, equation (2) controls only for blocking variables, equation (3) introduces ISIC two-digit sector fixed effects, and equation (4) removes all firms that did not grant access to factory floor. Equations (5) and (6) control for firms receiving Treatment 1. Models 4 and 6 have smaller sample sizes, because we are restricting analyses to firms that permitted endline auditing of compliance.

As formula (2) shows, Participation was interacted with a dummy variable, Endline. Endline was coded 0 for Legitimacy scores recorded in the baseline survey and 1 for those that came from the endline survey. The interaction is displayed in Column 1 of Table 2. Following standard experimental methodology, we included fixed effects for blocking variables used in the randomization process in Columns 2 and 3.^{Footnote 19} Column 4 restricts analysis to firms that allowed auditors to assess compliance (See the discussion of Outcome 2 below).

Although our theory views information as a separate mechanism and does not offer a prediction between information and increased legitimacy, our experimental design allowed for the possibility that regulatory information may influence post-treatment survey responses. We therefore add this treatment and its interaction as controls in Columns 5 and 6. As noted in formula (1), Information is coded as 1 if the firm received the presentation on the forthcoming hazardous chemical law and 0 otherwise. As shown in Figure 1, this applies to firms in both the T1 and T2 groups.^{Footnote 20}

Results are robust across specifications. Importantly, the component term for Participation (β₂) is not statistically significant throughout Table 2, indicating that the treatment groups were statistically balanced in their views of government legitimacy at baseline.^{Footnote 21} The coefficient β₃, the change in Legitimacy over time within the Participation group, is statistically significant at the 0.05 level and robust across specifications. Ordered probit coefficients can be difficult to interpret, so we calculate the predicted probabilities from Table 2’s fully specified Column 3 in Table 3 below.

TABLE 3.

Predicted Probabilities From Legitimacy Analysis

Results calculated from Column 3 of Table 2 using Stata’s margins command.

In sum, Tables 2 and 3 provide strong support for the legitimacy mechanism. The predicted probability of agreement with the Legitimacy statement among firms assigned the opportunity to comment was 49.8%, growing to 73.4% in the endline survey, an increase of 23.6 percentage points. For nonparticipants (Control and T1 groups), Legitimacy was 52.4% (49.1% Agree and 3.3% Strongly Agree) at baseline and 69.2% (61.1% + 8.1%) at Endline. Thus, growth in Legitimacy within the Participation group was about 6.8 percentage points greater than nonparticipants (a 40.5% change).

Note that the information mechanism does not entail any change to firms’ views of government legitimacy. As a result, it is not surprising that neither of the coefficients on Information or its interaction is significant in Column 5 or 6. Much more interesting, however, is that the predicted probability on Endline (β₁) in the fully specified Column 3 indicates that Legitimacy in the Control significantly increased between rounds by 16.8 percentage points. This notable increase in Legitimacy likely results from that fact that all sample firms were exposed to an unusually positive picture of the government’s regulatory design process. For firms in the Control, this included viewing a video on VCCI’s efforts to help government better understand firms’ perspectives on draft regulations, receiving mailed copies of the final regulation, and being offered a voluntary audit and advice on compliance from VCCI.

We caution that our Outcome 1 results are based on a single, self-reported measurement that is subject to perception bias and alternative interpretation. Although we did replicate our tests with an alternative measure of legitimacy, the assessment of regulatory quality in Figure 2 above (See Online Appendix I3), this measure does not have the same benefit of over-time measurement. As a result, the next analyses highlight changes in behavioral outcomes that we deem more reliable.

Outcome 2: Access to the Factory Floor

Having introduced supporting evidence for the legitimacy mechanism, we now move to a series of tests of H1, the general relationship between participation and compliance. For our first direct test of H1, we treat the ability of auditors to enter the factory as a measure of compliance. Because this measure represents a direct behavioral measure that is not subject to post-treatment selection bias or social desirability bias, we consider this analysis to offer the most reliable test of our general theory.

Of the 830 firms that participated in the endline round, 38% did not allow access to their warehouses or factories. Importantly, in each case, representatives of sample firms first met face-to-face with the auditor and answered the endline survey. Only after clearly understanding the auditor’s technical expertise, and ability to recognize regulatory noncompliance, did they then choose to refuse access.^{Footnote 22} This apparent fear about giving access to a true expert appears to indicate that many firms were concerned about VCCI’s ties to the state and skeptical about the claim that no information from the factory floor visit would be disclosed to government regulators. Although providing access to the factory is certainly costly and time consuming, use of random assignment should mean that these costs did not vary across our treatment groups, which we are able to show do, on average, constitute firms of similar sizes, sectors, and business performance.

Thus, there are two interrelated interpretations of the outcome that firms given an opportunity to participate in government’s regulatory design process were more likely to grant government-affiliated auditors access to their factory floors. First, one can view provision of factory access as an indicator of a firm’s opinion of government legitimacy, capturing a firm’s general interest in better understanding the regulation and what tangibly comprises regulatory compliance. When doors are kept shut, the firm chooses not to engage in this constructive back and forth. Second, access to the factory floor can be viewed as a direct measure of compliance with the regulation. A firm that blocks access for a business-friendly audit is more likely to have something to hide and less likely to be compliant with the underlying regulation than a firm that does provide access.

Although we cannot observe compliance for firms that did not provide access, Figure 3 provides some justification for these interpretations. Although firms were randomly assigned to the different treatment groups and therefore are similar on average in terms of both observable and unobservable characteristics, those who allowed access answered survey questions very differently than those who did not. Firms that allowed access were more likely to say the regulation was of higher quality (18.2% vs. 9.36%), more likely to agree with our Legitimacy question in Table 2 (73.0% to 68.0%), and less likely to believe that officials use regulations to extract bribes (42.9% vs. 50.2%). These results indicate that firms providing access were more accepting of regulation than their peers.

FIGURE 3.

Justification of Assumption that Access Proxies Compliance

Note: “Regulation is High Quality” measured using question: How do you rate the quality of this draft regulation relative to the other regulations that you have opportunities to read or give comments on? (5. Much higher; 4. Higher; 3. Similar; 2. Lower; 1. Much lower) We recoded, so that Agree = Much higher and Higher and Disagree = Similar, Lower, or Much lower. Regulators Understand Business measured using “Government officials have sufficient understanding of business like this one to effectively carry out their regulatory duties.” We recoded, so that Agree = Strongly Agree and Agree and Disagree = Strongly Disagree and Disagree. Regulators to Extract Bribes measured using “The government officials may take advantage of the regulation to extract bribes” We recoded, so that Agree = Strongly Agree and Agree and Disagree = Strongly Disagree and Disagree.

Table 4 presents the results of tests of the relationship between access and our experimental treatments. We use a probit specification with standard errors clustered at the firm-industry level in every specification to address the fact that clusters of industries in provinces may share certain features that affect the ability to treat them as independent draws.^{Footnote 23} We regress Access, defined dichotomously (Access = 1, No Access = 0), on our two treatment variables, Information and Participation, which are coded the same as in the previous legitimacy analysis. As before, we begin with an unadjusted model in Column 1, then add design-based controls for blocking variables (λ) in Column 2 and sector (α) fixed effects in Column 3. Column 4 further adds dummies for individual auditors in the endline round, to account for variation in levels of experience and personalities that may have affected their ability to convince firms to permit inspection and their subjective evaluations of compliance.

(3)$$\eqalignb{ &#x0026; \Pr (Acces{s_i} = 1) = {\beta _0} + {\beta _1}Informatio{n_i} \cr &#x0026;+ {\beta _2}Participatio{n_i} + {\beta _3}Hanoi + {\beta _4}Femal{e_i} \cr&#x0026; + \lambda + \alpha + {u_i}. \cr}$$

TABLE 4.

Effects of Experiment on Access of Auditors to Factory Floor

Probit model with standard errors, clustered by province-sector, in parentheses (*** p < 0.01, ** p < 0.05, * p < 0.1). Marginal probabilities instead of coefficients presented. Panel 1 studies whether auditors were able to visit the factory after conducting the endline interview. Panel 2 studies normal attrition in the panel. Equations (1) and (5) are unadjusted, equations (2) and (6) control only for blocking variables, equations (3) and (7) introduce ISIC two-digit sector fixed effects, and equation (4) introduces auditor fixed effects. Adding auditor fixed effects leads to a reduction in sample size to 700, because of difficulties two auditors had in accessing firms in Hanoi. In panel 2, adding blocking variables reduces sample size because some information is missing from firms that declined interviews.

The results of the fully specified estimating equation, displayed in Column 3, are striking. First, despite the earlier finding that Information was associated with firms being over twice as likely to be aware of the regulation, increased knowledge did not make firms more likely to allow access to their factories. In fact, firms receiving the information treatment actually provided marginally lower access than firms from the Control (although the coefficient is insignificant). By contrast, auditors visiting firms that received the participation treatment were 9.3% more likely to gain factory access than when visiting Control firms, and nearly 11% more likely than with firms receiving the information treatment. These results are robust to specification, including linear probability models, and strongly significant (p < 0.01).^{Footnote 24}

Refusal to allow access to the factory is clearly different than normal attrition. Columns 5 through 7 replace factory access with agreement to participate in the endline round at all and show no differences across treatment groups in whether or not firms agreed to the return visit.

We see these results as strong evidence for our theory. Participating groups were far more likely to allow factory audits, which we view as a critical first step toward acceptance of the government’s regulatory regime and compliance with the outcomes that regime produces.

Outcome 3: Factory Compliance with the Hazardous Chemical Regulation

Our second analysis of H1 relates to auditor judgments of actual compliance with worker protections on the factory floor, another measure of actual firm behavior. Auditors who gained access to sample firms’ factory floors created scores of overall compliance for each of the core 10 clauses.^{Footnote 25} Based on these subjective scores, we created dichotomous measures for each clause, scoring a firm as compliant if it received a score of three or above (Compliance = 0 if Assessment <3; Compliance = 1 if Assessment >3).^{Footnote 26} Importantly, when we deemed a particular clause to not be relevant for a firm, based on its industry classification, the measure for that clause received a null score. If, for example, a firm operated in fabricated metal manufacturing (ISIC C25), the clause relating to welding equipment was clearly relevant. By contrast, this clause was clearly not relevant for firms operating in food processing (ISIC C10); therefore, these firms did not receive a score for this clause.^{Footnote 27}

We used these firm- and clause-specific measures to construct our primary dependent variable for factory compliance by calculating the share of relevant clauses with which each firm was judged to be compliant. Formula (4) shows the simple index of average compliance across the audited clauses we created to explore these patterns more systematically. For each firm (i), each relevant clause (k) is coded as 1 if the firm was compliant and 0 if it was not. We sum up the number of instances of compliance and divide by the number of relevant clauses (t) for each firm.^{Footnote 28}

(4)$$Complianc{e_i} = {{\mathop \sum\limits_1^t {Claus{e_{k,i}}} } \over {{t_i}}}.$$

We regress Compliance on our treatment variables following the same specification as the Access regressions above, controlling for blocking variables, and clustering standard errors at the province-sector level.

(5)$$\eqalign{ &#x0026; Complianc{e_i} = {\beta _0} + {\beta _1}Informatio{n_i} + {\beta _2}Participatio{n_i} \cr &#x0026; + {\beta _3}Hanoi + {\beta _4}Femal{e_i} + \lambda + \alpha + {u_i}. \cr}$$

A tricky feature of this analysis is how to address the selection bias problems posed by the refusal of some firms to allow access to their factories. Our auditors were, of course, unable to construct measures of compliance with the target regulation for these firms. As a result of these missing data, there is reason to believe that any compliance variables based solely on factory floor compliance audits suffer from a selection bias that makes a positive relationship between participation and compliance more difficult to identify. We base this assertion on the combination of the evidence of a positive relationship between participation and access presented in the previous section and our exploration of the assumption that a firm that grants factory access is also more likely to be in compliance in Figure 3. In other words, firms that did not allow access probably had something to hide.^{Footnote 29} Based on this reasoning, we simply coded nonaccess as full noncompliance (0%) in our main models (Columns 1 through 4 in Table 5).

TABLE 5.

Effects of Experiment on Aggregate Score of Regulatory Compliance Judgments by Auditors

Ordinary Least Squares (OLS) models with standard errors, clustered by province-sector, in parentheses (*** p < 0.01, ** p < 0.05, * p < 0.1). The first panel analyzes all firms where auditors were given access. The second panel drops firms in the participation treatment that provided comments. Equation (1) is unadjusted, equation (2) controls only for blocking variables, equation (3) introduces ISIC two-digit sector fixed effects, and equation (4) introduces auditor fixed effects. Estimating equations (5) and (6) restrict the analysis to districts where auditors were able to access more than 80% of factories in the jurisdiction, leading to a smaller sample size of 207. Because these models use selection strategy at the district level, standard errors are now clustered at district level. Equations (7) and (8) restrict the analysis to a matched sample of firms that allowed access across treatment groups. The matches are created using coarsened exact matching (CEM) based on observable characteristics of the firms. Again, this procedure is restrictive and limits the sample size to 162 firms.

Some will understandably view equating nonaccess with full noncompliance as too strong an assumption. We take three approaches to this address challenge. First, we perform a bounds analysis, following the approach described in Angrist, Bettinger, and Kremer (Reference Angrist, Bettinger and Kremer2006). Second, we limit our analysis to administrative districts where auditors received nearly perfect access by firms (see Columns 5 and 6). This limits our statistical power by cutting our sample size by 75%, but it also ensures that access to the factory floor is not associated with our experimental treatments. Third, we use coarsened exact matching (CEM) to identify audited firms from the Participation group that were similar to audited firms from the Information and Control groups based on observable characteristics (Iacus, King, and Porro Reference Iacus, King and Porro2012).^{Footnote 30} We then drop all unaudited firms and perform the analysis on the matched set of 162 operations (Columns 7 and 8).

A first review of our factory floor dependent variable Compliance reveals an average of just 36% across all clauses and groups.^{Footnote 31} We do, however, also find initial evidence that this variable was influenced by the chance to participate. Firms in T2 had average overall compliance scores of 40%, compared with 35% in T1 and 36% in the Control. Individual clauses, such as “washing facilities” and “lighting systems,” also exhibited meaningful differences across groups.^{Footnote 32}

Moving to our regression analyses, Table 5 follows the same progression as Table 4. The fully specified Column 3 shows that participation treatment firms demonstrated 5.5 percentage points greater compliance than the 35.4% compliance rate recorded in the Control—a 15.1% improvement. The results are robust across specifications, including the addition of auditor fixed effects, and statistically significant at the (p < 0.05) level. Again, firms in the information treatment demonstrated marginally worse compliance, although the effects are not statistically significant.^{Footnote 33}

Thus far, our analyses of compliance have assumed that not permitting access to the factory floor is equivalent to noncompliance. Figure 4 relaxes this assumption with a bounds analysis, where we randomly assigned a compliance score to firms that did not provide access to auditors. The goal is to test how much compliance would be required from unaudited firms to diminish the positive relationship between participation and regulatory compliance.

FIGURE 4.

Bounds Analysis of Average Treatment Effect

Note: Range bars depict upper and lower bounds at different levels of simulated compliance of the unaudited group. Dashed horizontal lines represents an average treatment effect (ATE) of zero. The thick dashed vertical line represents average compliance among firms that allowed audits (61%). The thin dashed lines represent one standard deviation shifts from mean compliance. Estimated derived from Table 5 (Column 3).

We begin by assigning each unaudited firm a compliance score of zero. We then regress compliance on the treatment conditions, replicating Column 3 (Table 5) and plot the resulting ATE (diamond) of participation and confidence intervals (range bars). Next, we randomly assign a compliance score, but restrict the mean, so that the average unaudited firm received a compliance score of 1%. We repeated this exercise 1,000 times, incrementally increasing the average score until we reached average compliance in the unaudited group of 1 (or 100%), where every firm receives perfect compliance. Figure 4 reports the sensitivity of our analysis to assumptions about the unaudited groups. The dashed vertical lines depict average compliance (61%) and standard deviations observed in the set of firms that allowed auditing. Using this approach, we can observe the sensitivity of our estimates to changes in the assumed level of compliance in the unaudited group. We find that the participation treatment would generate a statistically significant ATE up until 32% average compliance in the unaudited group (40% if we accept a 90% confidence interval). These scores are within one standard deviation of average compliance in the audited group. Although not statistically significant, the ATE remains positive until it reaches 60% average compliance in the unaudited group. We never observe statistically significant negative effects (at the 95% level) for the participation treatment, regardless of our assumptions about the unaudited group. As a result, the bounds analyses give us confidence that our regulatory compliance results are not the result of selection bias in our ability to audit.^{Footnote 34} The bottom line is under the weaker assumption that compliance in the unaudited firms was less than half the compliance in the audited firms; we would observe results significant at the 0.05 level. If we were a bit more generous and assumed compliance in the unaudited group was 60% of that in an audited firm, we would observe a significant treatment effect at the 0.10 level.

The rest of Table 5 presents robustness tests with less technical solutions to the problems raised by nonaccess. Columns 5 and 6 present results from limiting the analysis to subprovincial jurisdictions, called districts, where auditors received near perfect factory access (>80%). Our assumption is that, for political or sociocultural reasons, firms in these areas felt greater trust that the audit would not lead to negative consequences. As a result, there was less likely to be selection bias associated with our experimental conditions in these groups, allowing us to more accurately measure the effects of participation on compliance. The coefficients on Participation Treatment in Column 5 and 6 are remarkably similar to the previous estimates in Table 4, corresponding to a 5.7 percentage point increase in average compliance in Column 6. Because of the dramatic reduction in sample size, however, the standard errors are larger and the results are not statistically significant. Results in Columns 7 and 8 using CEM also have similarly sized coefficients but are underpowered because of the data trimming necessary for matching. Online Appendix N shows the experiment was most effective among SMEs, which tend to be excluded from Vietnam’s highly captured policy-making process.

Our identification of a significant relationship between the opportunity to comment on a draft of the target regulation and subsequent factory floor compliance is particularly impressive in light of the array and degree of real-world challenges that threatened to obscure it.^{Footnote 35} In particular, noise was generated by the opacity of the draft document, which obscured firms’ understanding of their obligations and our efforts to measure compliance.

Was the Participation Effect Still Driven More by Information than Legitimacy?

One potential threat to our findings above is that participation may have generated a higher dosage of information, because the participation group simply had more opportunities to learn about the hazardous chemical regulation than the other groups. More opportunities could generate either more regulatory knowledge or greater worries about potential government enforcement. If this was correct, participation could simply be seen as a stronger treatment for the M2 mechanism. Our information treatment was designed to address these concerns by providing firms with the text of the clauses and the video outlining responsibilities. Nevertheless, all firms in the participation group also received a response report from the Labor Safety Bureau in April 2015.

To test whether this additional round of contact with government influenced compliance, we randomly assigned 97 T1 firms to receive the response report. This allows us to test the effect of Participation among firms with extremely high levels of knowledge about the regulation. All of these firms saw the 11 clauses, watched the video, and received a response report. In Table 6, we calculate the ATE of participation only among highly informed respondents. Focusing on the fully specified Columns 3 and 5, firms in the participation group were about 11.8% more likely to provide access and had average compliance scores that were 6.8 percentage points higher than firms in the information group. These sizable substantive effects give us further confidence that legitimacy is more important than repeated learning in generating regulatory compliance. Furthermore, the randomly assigned response report has no discernable effect on factory access or compliance when analysis is restricted to only the T1 information group (See Online Appendix M).

TABLE 6.

Differentiating Legitimacy From Learning (Limited to Firms Assigned to Receive Government Response Report)

Standard errors, clustered by province-sector, in parentheses (*** p < 0.01, ** p < 0.05, * p < 0.1). Analysis restricted to firms that received the government response report. The first panel analyzes whether firms were given access using a probit specification. The second panel studies compliance with the regulation. Equation (1) is unadjusted, equation (2) controls only for blocking variables, equation (3) introduces ISIC two-digit sector fixed effects, and equation (4) introduces auditor fixed effects. Estimating Equations (5) and (6) test compliance with the regulation, treating non-access as zero compliance. Estimating equations (7) and (8) restrict the analysis to districts where auditors were able to access more than 80% of factories in the jurisdiction.

Was the Participation Effect Driven by Substantive Change?

In Table 7, we examine whether our findings are an artifact of the substantive change mechanism (M3 above), which holds that participation may have altered the regulation, making it easier to comply. Perniciously for our findings, these changes might be idiosyncratic to particular firms and hard to detect, implying that participation may have generated regulatory changes that increased compliance for that specific group of commenters. To address this potential threat, we drop the 28% of firms in the participation treatment that provided comments seen as truly substantive. This left only firms that had the opportunity to comment, but did not exercise it fully enough to affect the regulation. It is therefore impossible for the remaining firms to have contributed to changes in the regulation. Because providing substantive comments is not randomly assigned, we again use CEM to identify firms from the Control and Information conditions that were similar to responders in observable characteristics and dropped them as well (Iacus, King, and Porro Reference Iacus, King and Porro2012). This allows as unbiased a comparison as possible between noncommenters and those likely to be noncommenters in the control conditions.^{Footnote 36}

TABLE 7.

Differentiating Legitimacy From Substantive Change (by Dropping Commenting Firms)

Standard errors, clustered by province-sector, in parentheses (*** p < 0.01, ** p < 0.05, * p < 0.1). Analysis uses coarsened exact matching (CEM) to identify noncommenters in Control and T1 groups. All commenters and potential noncommenters are dropped from this analysis. The first panel analyzes all interviewed firms to see whether or not they were given access using a probit specification. The second panel studies compliance using an OLS specification. Equation (1) is unadjusted, equation (2) controls only for blocking variables, equations (3) and (5) introduce ISIC two-digit sector fixed effects, and equations (4) and (6) introduce auditor fixed effects.

Table 7 shows that dropping commenters (and potential commenters in C and T1) actually increases the ATE of participation. The participation treatment is now associated with between 17.3% higher access to the factory in the pre-registered model and 9.9% when auditor fixed effects are included, and between 6.4 and 13.1 percentage points greater overall compliance, although the result with auditor fixed effects is only significant at the 0.1 level. The fact that the coefficient increases substantively when commenting firms are dropped is consistent with the legitimacy mechanism and not the positive or the negative form of the substantive change mechanism. The significance of results weakens somewhat when we use auditor fixed effects.