Introduction
Do self-efficacy beliefs affect whether citizens engage in development-enhancing behaviors? We consider this question in the context of a popular development paradigm known as “short-route accountability” (World Bank 2003). This model posits that citizens ought to be well-positioned to actively monitor public service inputs; and if they took individual and collective actions to apply pressure on government service providers, such as schools and health clinics, this would help to ensure acceptable levels of delivery. It offers a stark alternative to “long-route” or “electoral accountability,” in which citizen wants and needs are filtered through elections and political office holders. While important, elections occur only every few years, and specific, localized needs can be obscured within the electoral logic, especially if elections are not competitive.
One might assume that as the beneficiaries of public services, citizens would be incentivized to take these short-route pro-development actions (Fung Reference Fung2006; Fung and Wright Reference Fung and Wright2001). Nevertheless, even in democratic countries, where citizens enjoy the political freedoms to engage in such a manner, front-line providers frequently report very low levels of citizen engagement (Khemani et al. Reference Khemani, Dal Bo, Ferraz, Finan, Stephenson Johnson, Odugbemi, Thapa and Abrahams2016). Social scientists have pointed to “information asymmetries” as the main bottleneck preventing this type of citizen engagement, particularly in low socioeconomic settings (Andrabi, Das, and Khwaja Reference Andrabi, Das and Khwaja2017; Banerjee et al. Reference Banerjee, Banerji, Duflo, Glennerster and Khemani2010; Berlinski et al. Reference Berlinski, Busso, Dinkelman and Martinez2016; Bjorkman Reference Bjorkman2007; Bruns, Filmer, and Patrinos Reference Bruns, Filmer and Patrinos2011; Joshi Reference Joshi2013; Keefer and Khemani Reference Keefer and Khemani2011; Olken Reference Olken2007; Reinikka and Svensson Reference Reinikka and Svensson2004; Reference Reinikka and Svensson2005). That is, citizens are either unaware that problems exist, or do not know what actions to take to hold officials accountable.
Many empirical studies have investigated whether greater transparency through disseminating information to citizens would address such concerns, leading to better development outcomes (Arkedis et al. Reference Arkedis, Creighton, Dixit, Fung, Kosack and Levy2019). With a few exceptions, most have found largely null or even negative effects relative to expectations. Studies carried out in India (Banerjee et al. Reference Banerjee, Banerji, Duflo, Glennerster and Khemani2010), Chile (Mizala and Urquiola Reference Mizala and Urquiola2013), Kenya (Lieberman, Posner, and Tsai Reference Lieberman, Posner and Tsai2014), Mexico (Chong et al. Reference Chong, De La O, Karlan and Wantchekon2014), and Uganda (Buntaine, Daniels, and Devlin Reference Buntaine, Daniels and Devlin2017) all tested, in one way or another, the effects of providing information to citizens, and ultimately found limited or no impact on behaviors and development outcomes.
We contribute to this literature by theorizing the possibility of an efficacy trap in citizen engagement, and we experimentally test an intervention that aims to bring individuals out of this trap. We first highlight the potential role of self-efficacy beliefs in mediating the link between information and action, identifying the importance of mastery experiences that may be in short supply among the population of interest. We report on a series of experimental studies in different regions of rural Tanzania, designed to test the effects of a simple, meeting-based intervention called Validated Participation (VP), which was developed in partnership with the East African civil society organization, Twaweza. Footnote 1 Broadly, the studies test whether self-efficacy beliefs can be augmented and sustained for increased citizen action.
Overall, we do not find evidence that the addition of VP led to increased self-efficacy or change in parent-reported behaviors relative to subjects who only received standard information treatments. Such findings contribute to a larger body of evidence highlighting the challenges of increasing accountability through such “short-route” channels. Nonetheless, we also do not believe that the results from these experiments are fully dispositive of the theory outlined. We discuss some more encouraging findings from a small-scale, downstream study of teachers from study schools as well as more general observations that should inform future research.
Theory of the efficacy trap
We use the term self-efficacy to describe a person’s sense of being capable of affecting change. Footnote 2 More specifically, internal efficacy is the perception that the self has the abilities and competence to participate; while external efficacy is the belief that institutions and authorities are responsive to the person’s attempts at influence. Moreover, it is useful to distinguish between individual efficacy, the belief that the person alone can produce change; and group efficacy, the belief that the person, when joined by a collective, can produce change (Abramson and Aldrich Reference Abramson and Aldrich1982; Bandura Reference Bandura1977; Craig and Maggiotto Reference Craig and Maggiotto1982).
It stands to reason that those who lack a sense of either internal or external self-efficacy will be less likely to engage authorities and to participate in activities that they perceive are beyond their influence (Caprara et al. Reference Caprara, Vecchione, Capanna and Mebane2009; Smith Reference Smith1989). But what shapes self-efficacy beliefs in the first place? Prior research has demonstrated that in addition to personality traits, individuals from high socioeconomic status (SES) families or those living in high SES neighborhoods are more likely to have stronger self-efficacy beliefs (Boardman and Robert Reference Boardman and Robert2000; Brady, Verba, and Schlozman Reference Brady, Verba and Schlozman1995; Gecas and Seff Reference Gecas and Seff1989; Hughes and Demo Reference Hughes and Demo1989). Previous studies have also linked self-efficacy to political participation in both democratic (Gottlieb Reference Gottlieb2015; Rudolph, Gangl, and Stevens Reference Rudolph, Gangl and Stevens2000; Solhaug Reference Solhaug2006; Valentino, Gregorowicz, and Groenendyk Reference Valentino, Gregorowicz and Groenendyk2009) and non-democratic contexts (Grossman, Michelitch, and Santamaria Reference Grossman, Michelitch and Santamaria2017; Young Reference Young2019). Low self-efficacy citizens may not fully take advantage of service provision opportunities that demand active engagement. And yet, if such beliefs are important for citizen action, this could suggest a particularly heavy constraint on the poor, where socioeconomic conditions may be difficult to change without their engagement. Indeed, this is the “efficacy trap” depicted in Figure 1. Because poor people with low self-efficacy may not pressure and monitor the government to provide the services they need, they are ultimately less likely to capture the benefits of the very services that offer the potential to escape their lower SES status – to escape poverty.
Theory of the efficacy trap and the role of Validated Participation (VP). This study examines relationships depicted by shaded arrows.
Is there a way out of this efficacy trap? Also as shown in Figure 1, we propose that the central mechanism linking high SES to self-efficacy is what psychologists call mastery experiences (Bandura and Ramachaudran Reference Bandura and Ramachaudran1994). People with higher education, incomes, and status enjoy more opportunities to practice participation, to be acknowledged for their efforts, and to observe some concrete changes associated with their actions. As such, they are much more likely to develop self-efficacy beliefs throughout their lives, as compared with those who lack such experiences. We consider the prospect of deliberately creating opportunities for mastery experiences that build on more universal observational and social skills that do not require high levels of education, but are also not routinely acknowledged in everyday life. This is critical, because the act of mastering a skill may not be sufficient for an individual to recognize the achievement or its value, and external acknowledgment or validation helps to cement that belief.
We propose a specific intervention, called Validated Participation, described in Section 4, which features discussion-based activities led by a facilitator trained to encourage and to acknowledge the contributions of participants and to provide positive feedback about their value in a collective setting. We hypothesize that this intervention can help nudge individuals out of an efficacy trap, and into a more “virtuous circle,” in which citizens who perceive greater self-efficacy engage more. This itself can become a mastery experience, and may ultimately lead to better services and human development outcomes. As we note in Figure 1, however, in this study, we focus our examination on just a few steps of the theory, from the exogenous introduction of the intervention as a mastery experience to beliefs, engagement, and school-related outcomes.
Citizen engagement in primary education in Tanzania
We investigate self-efficacy beliefs and citizen participation in the context of a series of experimental studies in Tanzania, the sixth largest African country by population at over 56 million. With a GDP per capita under $1,000 USD, about half of the country’s population lives below $1.90 USD per day (World Bank 2017). We focus on the primary education sector, a key area where citizens (generally, parents) routinely play important roles in the delivery of this public service, but in many developing country contexts, including Tanzania, such involvement is highly limited. In 2001, Tanzania introduced universal free primary education, which increased enrollment from 52% in 2000 to over 94% of eligible children in 2008 (Valente Reference Valente2015). However, the quality of public schools, teaching effort, and learning outcomes remain poor (Palmer et al. Reference Palmer, Wedgwood and Hayman2007; Wedgwood Reference Wedgwood2007). In 2015, Twaweza found that only one-third of grade 3 students could pass a grade 2 learning assessment on literacy and numeracy (Twaweza 2017).
Additionally, both government and nongovernmental actors have expressed that citizens generally do not exercise their rights to monitor teaching and school inputs or to apply pressure on officials to improve the quality of education (Mbiti et al. Reference Mbiti, Muralidharan, Romero, Schipper, Manda and Rajani2019; Tanzania Ministry of Education and Vocational Training 2013, p. 8). And this sentiment was echoed in a 2015 nationally representative survey conducted by our local civil society partner, Twaweza, which found that 20% of parents had never met with their child’s teacher, 43% had never visited the school, and 88% had never helped out at the school within the past 12 months (Twaweza 2016). Additionally, while primary schools have school management committees (SMCs), consisting of teachers and parents, to help oversee school governance, active parental participation and collective efficacy are generally low (Masue Reference Masue2014; Masue and Askvik Reference Masue and Askvik2017). Thus, a central aim of many nongovernment actors has been to improve the quality of education in East Africa by increasing citizen participation and engagement. While there are many unique features that distinguish Tanzania from other countries, the core concerns are widely applicable to other contexts with underperforming social services, particularly in the education sector.
Validated participation intervention
Developed in partnership with Twaweza, VP is a novel meeting-based intervention designed to give participants opportunities for successful engagement or mastery experiences. These were conducted in a quasi-public setting where participants were validated by peers and particularly by an authority figure, namely the facilitator. In this context, the meeting group comprised about 25 adult citizens, who were parents or caretakers of children (for ease, “parents”). The content consisted of a selection of different exercises, each of which was designed to be conceptually consistent with the theorized sources of efficacy: citizens were given equal opportunities to participate in critical discussions and decision-making, and their ideas and actions were validated by an authority figure and each other. These exercises included deliberating and voting on how to spend small grants for the school (e.g. science posters vs. maps), which were almost immediately implemented, a discussion of school needs, and collectively generating a list of actions that parents could undertake to help improve their children’s learning. Another unique feature of the intervention was that the facilitator displayed visual images with narrative intent and encouraged discussion about what participants could infer from those images. The intention was to encourage participation even among those with low literacy or initial uneasiness talking about the subject at hand. Taken together, these exercises were designed to give citizens the opportunities to practice participation through collective engagement, gaining crucial mastery experiences with positive reinforcement. See Figure 2 showing the VP intervention activities in progress. The facilitator was instructed to encourage participation and to echo what individual parents said in order to signal being heard and recognized throughout the exercises. The intervention involved no new skill development – simply eliciting and validating skills people already possessed to provide a boost of efficacy, and in so doing, to potentially facilitate more active engagement.Footnote 3
VP intervention in progress. The facilitator is conducting a discussion about artwork with about 25 parent participants at their school (left), and later generating a list of pro-education actions that parents are volunteering and committing to take (right).
We view the challenge of boosting efficacy in a relatively short period of time to be a substantial hurdle. Perceptions of efficacy are generally built up over a lifetime of experiences. Thus, one could easily imagine that a much longer intervention may have more enduring effects. But as a research study, it would also be more expensive to administer, and would likely suffer from greater problems of attrition. The central goal of our studies, described in the following section, was to investigate whether a short course of intensive validation could substantively affect those perceptions and to evaluate associated behavior change.
Staged experimental studies
We report on the findings of two closely related studies, both conducted in Tanzania with our partner Twaweza: first, a school-level pilot experiment carried out in Bukoba rural district located in Kagera region in the northwest corner of the country (Study 1, Bukoba), for which we also conducted two qualitative follow-up studies 6 months and 2 years later; and second, an individual-level experimental study conducted in Kilosa district, located in Morogoro region in the center of the country (Study 2, Kilosa).Footnote 4,Footnote 5,Footnote 6
Study 1: Piloting and follow-up interviews in Bukoba
First, we conducted a pilot school-level study from August 2016 to January 2017 in Bukoba. We experimentally tested our intervention as a series of four weekly 90-minute meetings with a sample of 598 adult caregiversFootnote 7 across 24 public primary schools. Schools were randomly assigned to one of the three treatment groups: (1) a pure control group in which parents were only surveyed; (2) an Information Workshop (IW) group in which parents were lectured on the importance of education and how parents could contribute; and (3) the VP group in which parents received the same informational lectures but also participated in the aforementioned exercises that were led by the facilitator.Footnote 8 Within each school, 25 eligible parents were selected by lottery balanced on gender, totaling 200 parents per treatment arm. Such a clustered design was both more expensive than an individual-level study to implement and limited statistical power given our funding constraints, and it was also more realistic in terms of how such an intervention might be delivered in practice. This design also allowed us to study outcomes over a longer period of time, because we could isolate treatment effects by the school community without concern over spillovers from treated to control participants.Footnote 9
Trained enumerators conducted face-to-face baseline and endline surveys in Kiswahili. We first developed an education-specific self-efficacy scale as a battery of nine questions, with responses ranging on a scale from 1 to 5. We averaged over the questions to create an overall Efficacy Score as well as component Internal, External, Individual, and Group Efficacy Scores. See SI Section 3 for the full set of questions and how we constructed these indices.Footnote 10 We also measured parents’ public efforts (speaking with teachers and monitoring their absence); private efforts (checking homework and feeding breakfast); and goals for their children (certainty of graduating secondary school). We included a behavioral component in the survey, asking parents whether they would like to leave a constructive comment for the head teacher. Lastly, to examine student test scores, we conducted child learning assessments at baseline and approximately 6 months post-endline.Footnote 11 To analyze these data, we used OLS, regressing endline outcomes (Yi2 ) on Validated Participation (VP) or Information Workshop (IW) treatment indicators, controlling for baseline outcome measures (Yi1 ) as well as demographic covariates (x i ) with cluster-robust standard errors at the school level:
$${Y_{i2}} = \alpha + {\beta _1}V{P_i} + {\beta _2}I{W_i} + {\beta _3}{Y_{i1}} + \lambda {{\bf{x}}_i} + {\epsilon _i}$$
In order to measure longer term school-level outcomes, research assistants followed up with two waves of structured interviews with head teachers and classroom teachers of the study schools. The first was approximately 6 months after the conclusion of the intervention in a sample of 12 schools.Footnote 12 The second set of interviews occurred 2 years later in 2018 in order to study longer term effects in all 24 study schools. We asked the teachers whether they observed changes in terms of educational outcomes (student and teacher attendance, grades, and pass rates), the performance of the specific students whose parents participated in the study, and the responsiveness of local government officials to the school’s needs. We also asked them to tell us about levels of parental engagement at the school (taking any notable actions at school, interacting with teachers regularly, participating at meetings and whether those who speak at the meetings are mostly male elders or more diverse, making material contributions to the school, and helping with schoolwork). Although such actors might ordinarily be induced to overreport positive outcomes associated with the study, these tendencies would bias against detecting any effect. Furthermore, all of the schools in the Bukoba study were considered “study” sites, thus the teachers were unaware of treatment assignment status. Thus, we would expect to see any social desirability or related biases to be similar in responses across all three treatment arms. We then employed multiple research assistants to independently code these interview transcripts. To analyze these interview data, we use OLS regressing outcomes (Yi ) on VP or IW treatment indicators, with respondent-type fixed effects (head or classroom teacher, indexed by t) and cluster-robust standard errors at the school level:
$${Y_i} = {\alpha _t} + {\beta _1}V{P_i} + {\beta _2}I{W_i} + {\epsilon _i}$$
Study 2: Individual-level random assignment in Kilosa
From May to June 2018, we carried out Study 2, which examined the effects of a single 90-minute meeting with a sample of 1,633 parents across 16 public primary schools.Footnote 13,Footnote 14 Parents were recruited to schools and upon arrival, assigned to one of the three treatment arms: (1) a pure control group in which parents were only surveyed; (2) an IW group in which parents watched an informational video on education; and (3) the VP group in which parents watched the same video but also participated in the aforementioned exercises that were led by the facilitator.
Approximately 540 parents were randomly assigned to each treatment arm, and each group’s activities were held in separate classrooms to guard against spillover effects – specifically, the possibility that those who received different treatments would discuss their experiences and influence one another’s attitudes and behaviors, contaminating our ability to disentangle the effects of the different treatment conditions. Enumerators conducted a face-to-face survey with parents in Kiswahili immediately after the meeting. In this context, any experimental intervention that involves a potentially foreign-funded NGO may elicit responses on surveys and in practice that correspond to what citizens believe the NGO wants to “see” or “hear,” with the hopes that such compliance will be rewarded with financial or other forms of support. Recognizing this, both of our studies hold such pressures constant across treatment and control groups, which ought to weigh against being able to detect treatment effects in an experimental setting.Footnote 15 Additionally, we asked about willingness to take certain actions related to government accountability (supporting a pro-education political candidate, raising a public goods issue at a community meeting, raising that issue with a government official) and feelings of being respected by others (peers, authorities, the SMC).
As the design of Study 2 was cross-sectional, we use OLS regressing outcomes (Yi ) on VP or IW treatment indicators, controlling for demographic covariates for covariate adjustment (x i), with school fixed effects:
$${Y_i} = {\alpha _s} + {\beta _1}V{P_i} + {\beta _2}I{W_i} + \lambda {{\bf x}_i} + {\epsilon_i}$$
Association between efficacy and active citizenship
Before turning to the experimental results, we first explore the theorized link between efficacy and pro-development behaviors through observational analyses of naturally occurring variation in efficacy beliefs in both study settings. Figure 3 shows the positive associations between self-efficacy and self-reported parental behaviors as well as child assessment scores at baseline, controlling for all individual and household sociodemographic covariates.Footnote 16
Positive linear relationships between our educational self-efficacy score (x-axis) and citizen participation/pro-development outcomes (y-axis) from both studies. From Study 1, Bukoba (rows 1–2), these plots show the predicted probabilities for student test scores and parental outcomes at baseline. From Study 2, Kilosa (row 3), these plots show the predicted probabilities for pro-education political behaviors for the SO (control) parents (with 95% CIs). OLS models, controlling for demographic covariates, school cluster-robust standard errors.
While we cannot infer any causal relations from such associations, these initial findings are at least consistent with the proposition that self-efficacy positively affects active citizen behaviors and development outcomes. Moreover, from a measurement perspective, it is reassuring to find these patterns, especially with the child assessment scores which were measured independently from the parent surveys and thus, not subject to social desirability bias.
Experimental results
Study 1: Effects on parent and student outcomes in Bukoba pilot
Turning to our experimental findings, Figure 4 summarizes the results from Study 1 in Bukoba.Footnote 17 Since we preregistered the positive direction of predicted treatment effects, we report 90% confidence intervals based on a two-sided test, the lower bound of which is equivalent to that of a 95% confidence interval for a one-sided test. With respect to efficacy, while the estimates for both VP and IW are positive compared to control, all estimated effects are modest, and none are statistically different from zero at α = 0.05. For parental behaviors, VP increased the probability of asking about teacher absence by 11 percentage points (90% CI = [1.7, 20]); and looking at homework by 16 percentage points (90% CI = [8.1, 24]). We find similar estimates for IW such that the difference between VP and IW (shown in black) is essentially zero. In our pre-analysis plan, even with more optimistic predictions about self-efficacy, we indicated that we did not expect to see treatment effects with respect to student test scores, and indeed, that is what we find.
Study 1 (N = 598 parents; 24 school clusters): Average treatment effects of VP (blue) and IW (red) compared to control on endline efficacy and active citizenship outcomes (with 95% and 90% CIs). OLS models controlling for baseline measures and demographic covariates, with school cluster-robust standard errors. VP–IW difference (black).
Study 1 was expressly designed as a “pilot,” and at its conclusion, we reflected on potential shortcomings in design and implementation that we could address in follow-up research. First, we surmised that the active involvement of live facilitators in the IW meetings may have resulted in IW respondents receiving some “dose” of VP, which would have impeded our ability to estimate differences between these treatment arms, but would not have explained the generally weak effects overall. Second, because of the similarity of baseline and endline questionnaires, endline responses may have been a function of subjects’ attempts to recall how they answered at baseline. Third, because of the relatively small sample in the pilot, there was some important imbalance in the pre-treatment distribution of covariates, including higher baseline efficacy in the VP and control groups relative to IW.Footnote 18 All of these factors may have weighed against detecting the full extent of expected treatment effects, and we sought to make adjustments in our second study in Kilosa.
Despite the estimated null findings, we note that we found important differences in facilitator notes concerning the content of what parents discussed in IW as compared with VP meetings. During the IW workshops, parents were curious about what the sponsoring NGO, Twaweza, would do to help solve school problems after the meetings ended.Footnote 19 By contrast, in the VP meetings, parents expressed interest in volunteering and initiating peer networks with other parents who had not been involved in the study; and ideas and actions reflected greater inclinations toward active citizenship. SI Section 9 discusses this qualitative evidence in more detail.
Study 1: Follow-up with teachers on school-level outcomes
Beyond beliefs and attitudes as self-reported on surveys, we sought to learn whether the treatment had any impact on Study 1 schools in the medium- and long term. In the absence of detailed administrative records concerning parental engagement and grades of the children of our study parents, we took advantage of the Study 1 research design by returning to schools to interview multiple teachers and head teachers with respect to a number of dimensions of parental involvement and student outcomes 6 months, and then 2 years posttreatment. While such data are clearly inferior to directly observed parent behaviors and school interactions, the interviews offered a relatively low-cost alternative, and provided insight into the perspectives of key actors in this sector.
First, for the 6-month post-study assessment, we directed research assistants to conduct interviews with 24 teachers from a randomly selected number of Study 1 schools using a structured guide, asking them about their impressions of parents’ behaviors at their schools. We then transcribed these teacher interviews and independently coded them blind to treatment status by whether there was a positive change (i.e. more pro-education actions by parents observed in the past few months), no change, or negative change in parents’ behaviors relative to the desired outcome of more active engagement. We found that 82% of teacher comments indicated a positive change in the VP schools, but for IW and control schools, the proportion of positive teacher comments were much less at 54% and 11%, respectively.Footnote 20 We provide examples of those comments and our coding scheme in SI Section 9.
Approximately 2 years later, we again completed structured assessments with 117 teachers and head teachers in all 24 schools and independently coded their comments blind to treatment status. Compared to control schools, Figure 5 shows that teacher assessments of parent behaviors and of school-related outcomes are consistently more positive for the VP schools relative to IW schools. Several of these differences (shown in black) are statistically and substantively significant. First, the frequency of teacher–parent interactions in VP compared to control schools improved by 0.64 (95% CI = [0.36, 0.92]) on a 3-point scale from never to multiple times. Compared to IW schools, this difference is 0.73 (95% CI = [0.46, 0.99]).
Downstream follow-up in Study 1 schools, 2 years later (N = 117 teachers; 24 school clusters): Average treatment effects of VP (blue) and IW (red) compared to control on parental active citizenship outcomes 2 years later as reported on structured assessments with teachers (with 95% and 90% CIs). OLS models with respondent type FE and school cluster-robust standard errors. VP–IW difference (black).
With respect to private behaviors within the household, the likelihood of parents being helpful with children’s schoolwork was greater in VP communities by a substantially large 52 percentage points (95% CI = [23, 81]).Footnote 21 Compared to IW schools, this difference is 38 percentage points (95% CI = [16, 60]). Additionally, parents were more likely to make contributions to VP schools compared to IW schools by 26 percentage points (95% CI = [0.12, 51]). Finally, to assess educational outcomes, we asked, “Have education outcomes like student and teacher attendance, marks, and pass rates improved/stayed the same/worsened in the past 2 years? Please explain.” We find that assessments of educational outcomes for VP schools were better than control schools by 0.38 (95% CI = [0.068, 0.7]) on a 3-point scale from worse to better. Compared to IW schools, this difference is 0.26 (95% CI = [0.015, 0.51]).Footnote 22
These findings, while at odds with the null effects detected on our parent endline surveys, provide some suggestive evidence that VP may have longer term, positive consequences for the “co-production” of better educational outcomes (Bebbington Reference Bebbington1999; Ostrom Reference Ostrom1996; Woolcock and Narayan Reference Woolcock and Narayan2000). But how might we reconcile these differential findings? It may be that there were small differences at the endline, yet our data were simply too sparse to precisely estimate them. Consistent with our theory shown in Figure 1, these small differences in efficacy and engagement may have magnified over time through the “virtuous cycle,” generating more mastery experiences, greater efficacy beliefs, engagement, and so on. Particularly if VP parents mobilized other non-study parents in their schools, as qualitatively reported, these changes in behavior would multiply and be visible at the school level. As we discuss below, the consistency of the findings in the teacher study with our theoretical predictions leads us ultimately to conclude that the findings from the endline parent study are not fully dispositive with respect to our theory.
Study 2: Effects on parent outcomes in Kilosa
We implemented Study 2 – a same-day individual-level study of parents – to address the aforementioned concerns raised in Study 1. Figure 6 shows, as predicted, that VP leads to a modest increase in parents’ educational self-efficacy. Interestingly, the effects of VP were larger in the case of the overall efficacy score by 0.071 (90% CI = [0.0098, 0.13]), external efficacy by 0.076 (90% CI = [0.0035, 0.15]), and group efficacy by 0.088 (90% CI = [0.011, 0.17]); smaller for individual and internal efficacy, and in the latter two cases, the effects of the VP treatment fall below statistical significance.Footnote 23
Study 2 (N = 1,633 parents; individual-level treatment assignment across 16 schools): Average treatment effects of VP (blue) and IW (red) compared to control on same-day, post-meeting efficacy, pro-education political behaviors, and feeling respected (with 95% and 90% CIs). OLS models controlling for demographic covariates with school fixed effects and cluster-robust standard errors. VP–IW difference (black).
In the case of group efficacy, the single intervention caused a 0.11 standard deviation increase in self-reported beliefs. As a highly social intervention, the effects appear most promising for socially- and externally oriented outcomes. While the individuals who received the IW also report higher levels of self-efficacy than the control, the estimated difference is smaller, and not statistically different from zero. In turn, the differences between IW and VP groups are close to zero and not statistically significant.
We also examined the effects of treatment on questions about likely pro-education political behaviors and feelings of respect from others. In all cases, the effects from VP were at least as large and in the predicted direction as compared with IW. VP parents reported statistically significant positive effects for feeling respected by other parents (0.1, 95% CI = [0.031, 0.17]) and by authorities (0.15, 95% CI = [0.064, 0.24]).Footnote 24 However, in no cases were the effects of VP substantively or statistically different from IW.
Discussion
This project contributes to long-standing questions about what might encourage or discourage citizen engagement in civic affairs, particularly in developing country settings. Self-efficacy beliefs are a logical gateway for active citizenship, and in our observational (nonexperimental) analyses, we find that such beliefs strongly predict such behaviors, while recognizing that in practice, there is likely a feedback loop between the two. Such correlations suggest that there may indeed be an “efficacy trap,” but the bigger question around which our studies were framed concerned whether people could be nudged into feeling a heightened sense of self-efficacy when treated with external validation of their ideas and actions, especially compared to citizens who were treated only with relevant information.
In most of our analyses, we find that our experimental intervention did not substantially increase parent-reported feelings of self-efficacy or active citizenship behaviors, especially relative to those parents who received a more conventional informational intervention. And as such, our study contributes more evidence to a large body of scholarship investigating the role that information and modified informational interventions may play in democratic accountability toward human development. Nonetheless, we do note some important exceptions detected in our qualitative research, including from enumerator notes and semi-structured interviews with teachers, both of which provide some evidence of greater parent engagement in the long term among those who received VP, even relative to those in the IW treatment arm. To be clear, these data were collected in a less structured fashion and in much smaller samples as compared with the parent-level data at the heart of our study. Moreover, the data are based on the responses of key informants who were asked to provide aggregated accounts of parental behavior. And yet, our systematic coding of those responses yielded findings that were consistent with our main hypotheses.
Where does that leave us? Ultimately, we conclude that the bulk of our evidence does point to a null finding. But the discrepancies between our open- and closed-ended data leave open several possibilities that need to be addressed in future research. It is possible that treatment effects take longer to become evident than we (and other scholars) routinely consider; and/or that one set of measurement approaches yields less valid responses than we typically assume. Nonetheless, we believe that the notion of “efficacy traps,” and the possible role of VP to enhance efficacy and active citizenship cannot yet be dismissed. Particularly given the relatively low-cost of administering the treatment, and the fact that we detect only positive, and no negative effects, other scholars could incorporate this treatment into related studies of citizen behaviors. We offer ideas and training guides as a supplement to this article should others decide to do so.
Future research can also help to identify the exact types and dosages of validation that can be delivered most effectively and efficiently. This would require larger scale studies in terms of numbers of households and locations; the unbundling of treatment arms to better understand the effects of constituent parts; and longer term studies that would allow for observation of the full vicious/virtuous cycle, including through the causes and consequences of socioeconomic status. For those organizations – including schools, civil groups, and other government and nongovernment entities – that seek to increase such active citizen engagement, these findings serve as a reminder that citizens need to believe they are capable of taking those actions and that their efforts will be well received by authorities. This study identifies specific strategies for organizational leaders to create opportunities for targeted citizens to successfully engage and to explicitly recognize those efforts. Our intervention protocol offers several clear, inexpensive, and easily adapted steps that could be implemented by a wide range of actors involved in social development.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/XPS.2020.47.
Acknowledgments
At Twaweza, we thank the many colleagues and staff members who provided feedback and contributed in the planning and implementation of the study, including Risha Chande, Glory Blasio Emmanuel, Aidan Eyakuze, Kees de Graaf, Sophia Komba, Varja Lipovsek, Benjamin Masebo, Zaida Mgalla, Kitila Mkumbo, Happiness Nkwera, Youdi Schipper, Godfrey Telli, and Richard Temu. We are grateful to Kondrada Haule, Scarion Rupia, Prosper Tarimo, and Evidence Yohana for being excellent facilitators and for their help in the refinement of the intervention and the various measurement tools; to Jasanna Britton, Amy Lieberman, and Bethany Park for their important contributions in the development of the intervention; and Mahiri Mwita for translation advice. Survey research and project implementation were carried out by Economic Development Initiatives (EDI) Limited under the supervision of Mwenge Godlaid, Marie Mallet, and Respichius Deogratias Mitti. We also thank Meghan Perdue for project administration, and Anna Weissman for qualitative coding assistance. Graeme Blair, Donghyun Danny Choi, Alexander Coppock, Benjamin Fifield, Daniel de Kadt, Kristin Michelitch, Yuki Shiraito, Lauren Young, Imai Research Group, Paluck Lab, and participants from WGAPE Abu Dhabi, MIT Political Science, Penn State Political Science, UC Berkeley Political Science, ISPS Experiments Workshop at Yale University, the International Development Research Seminar Series at LSE, APSA 2017, and ASA 2019 provided helpful comments. We thank the Editor, our Associate Editor, and anonymous reviewers.
