1. Introduction
Growing links to social media are a notable feature of contemporary civic activism. Dramatic events in the Arab world, Ukraine and Turkey have taken place against the backdrop of intense internet-sharing activity by participants (Anderson Reference Anderson2011; Tufekci and Wilson Reference Tufekci and Wilson2012; Jost et al. Reference Jost, Barberá, Bonneau, Langer, Metzger, Nagler, Sterling and Tucker2018). There are many other, less dramatic examples, of relying on social media to broadcast a message, build awareness or organize for a cause (Jost et al. Reference Jost, Barberá, Bonneau, Langer, Metzger, Nagler, Sterling and Tucker2018). They include social movement campaigns, extensions of traditional non-governmental organizations into the blogosphere and smaller-scale civic campaigns. How do online social networks, understood as people communicating with people, help civic forms of political engagement emerge and how do they help to sustain them?
In principle, digitally-linked social networks can help overcome two main difficulties confronting civic activism: identifying supporters in a large and diffuse group of people on an issue of common concern, and helping supporters move closer to achieving the goals of the cause (Olson Reference Olson1971). Whether used in isolation or as a complement to traditional means of conducting campaigns, one may expect social media to increase grass-root engagement with social and political issues in local, national and global communities.
Existing arguments posit that online social networks help bottom-up civic activism via three broad mechanisms: information diffusion, social influence and critical mass dynamics (González-Bailón Reference González-Bailón, Graham and Dutton2014). Reliable information on an issue is critical for progress on a cause. Social discussions, interpersonal trust and the various selective benefits and incentives that accrue in social networks may prove motivational and even transform an individual's view on an issue. Critical mass dynamics refer to strength in numbers, the idea that networked technology can build awareness of the strength of a cause and change motivations for acting based on the idea that success is imminent.
We build on existing arguments about the role of social networks for political and civic activism (Bode et al. Reference Bode, Vraga, Borah and Shah2014) to develop a series of testable hypotheses, and we then embed a field experiment in a civic campaign to test them. Our case is drawn from Bulgaria. The case involves a civic campaign to protect the remaining Black Sea beaches from, often illegal, development.
Uncovering the existence of attitudinal or behavioral patterns in an online social network would be of limited value, unless we can contrast these patterns with the counterfactual patterns that would be observed were the networked or online element absent (Fowler et al. Reference Fowler, Heaney, Nickerson, Padgett and Sinclair2011). Thus, in order to identify the added value of the networked element, we need to compare citizens who would be exposed to political information via social media to citizens who would not be exposed to such information, and citizens who would be exposed to that information via different means of communication. The major issue is that citizens self-select into social networks, and it is hence difficult to disentangle selection from influence.
By construction our study can relate members of activist Facebook pages to some meaningful population of interest. We recruit two different samples. One sample is drawn from the pro-environment activist population in Bulgaria, and the other is a representative sample of Bulgarian Facebook users recruited via a traditional in-person survey. In both samples, we encourage a randomly assigned share of respondents to enroll in a campaign to protect natural resources. Thus, we observe who, among those encouraged to join, selects to join. In this way, we can re-create a part of the process of selection that occurs on social media. In combination with the background information collected on respondents, this allows us to profile the type of people who respond to invitations to contribute to civic causes on online social media. Because we randomly assign citizens to three experimental conditions, an encouragement to like a Facebook page, to sign-up to an email newsletter, or to control, the groups receiving an encouragement are ex-ante comparable to the control group. Thus, we can ultimately identify the causal effects brought about by the use of social media as a civic instrument. The few previous studies that randomly assigned citizens to social media influence via Facebook provide mixed evidence on whether Facebook positively affects political engagement (Bond et al. Reference Bond, Fariss, Jones, Kramer, Marlow, Settle and Fowler2012; Theocharis and Lowe Reference Theocharis and Lowe2016).
We find that social media are effective at selecting potential proponents of a cause. The people who select themselves into civic campaigns via the network tend to have different personalities (more adventurous), care more about the issue (nature) and are more likely to have engaged in activism prior to the campaign than respondents who decide not to follow the campaign. Thus, selection effects produce a potentially potent brew of like-minded, active people. In that sense, our results parallel findings showing that like-minded people gravitate toward each other on Facebook (Bakshy et al. Reference Bakshy, Messing and Adamic2015), and that they select into the same news channels (Arceneaux and Johnson Reference Arceneaux and Johnson2013). However, a campaign conducted via Facebook does not appear more selective than a campaign conducted via a more traditional email newsletter. This is probably less of a surprise than we expected given the widespread use of Facebook in Bulgaria, and elsewhere.
Beyond that, our results suggest that such a campaign does at best have minimal effects on attitudes, and does not substantially transform what followers believe about the cause or think of their civic activism. Overall, we find little evidence that the campaign mobilized people to become active or to discuss the cause more with friends online. We even uncover some darker overtones: people who followed the Facebook campaign were less likely to think that environmental activism was appreciated by others. Email newsletters appear to be generally ineffective for most of the outcomes we measure. In sum, we find that the media we study produce self-selected communities. While these may play an oversized role at key junctures, there is little evidence that they change how members think on an everyday basis.
These findings speak to the broader debate on the influence of the Internet on civic engagement. Boulianne (Reference Boulianne2015)'s meta-analysis reports that social media use is positively correlated with participation, but that questions remain on whether the relationship is causal and transformative. We find that the relationship is not transformative, but that the civically active select themselves into social media campaigns.
2. Social media and civic campaigns
We define social media as a subset of new digital communication technology with peer-to-peer access. A more precise definition in the literature has been elusive. As Gohdes (Reference Gohdes2014, 17) writes: “… digital communication technology and new digital media are seldom clearly defined, but generally they include the usage of peer-to-peer online platforms, such as Facebook, Instagram, Youtube, and Twitter, and voice over internet provider services (VoIP, such as Skype).” We focus on one of the most popular social media services, Facebook. The ubiquity of the service in many parts of the world, together with the interactive, peer-to-peer nature of the medium is key for our purposes.
Studies on voter turnout in democracies have used experimental methods to identify a causal effect between seeing socially-shared content and electoral participation (Bond et al. Reference Bond, Fariss, Jones, Kramer, Marlow, Settle and Fowler2012; Messing et al. Reference Messing, Bakshy and Fiore2014). However, beyond anecdotal and journalistic accounts, there is little evidence to date that social media messages affect political attitudes or behaviors beyond turnout (Broockman and Green Reference Broockman and Green2014). This mixed picture is well illustrated by Coppock et al. (Reference Coppock, Guess and Ternovski2016)'s results who find that publically posted messages on Twitter did not affect online sharing behavior related to a political cause, while private messages did.
Other studies have looked more explicitly at self-selection into social networks such as Facebook (Bakshy et al. Reference Bakshy, Messing and Adamic2015) and Twitter (Barberá et al. Reference Barberá, Jost, Nagler, Tucker and Bonneau2015). Bakshy et al. (Reference Bakshy, Messing and Adamic2015) find that Facebook users befriend others who are more likely to share their political outlook, and Barberá et al. (Reference Barberá, Jost, Nagler, Tucker and Bonneau2015) show that discussions of political events on Twitter lead to relatively homogeneous retweet networks, especially if the nature of the discussion is more partisan. There is therefore evidence that selection effects are important when we try to understand political activism on social media. At the same time, both studies emphasize that political homogeneity is far from complete, and that there is room for social influence. There is disagreement on whether social networks aggravate polarization (Settle Reference Settle2014), or help reduce it (Barberá Reference Barberá2014). Civic movements in authoritarian regimes have benefited from digital media, even though the more skillful authoritarian regimes such as Russia and China have striven to co-opt peer-to-peer online networks for regime survival (Gunitsky Reference Gunitsky2015).
Existing evidence on the effectiveness of email campaigns in changing behaviors is equally mixed. Nickerson (Reference Nickerson2007) shows that email is ineffective at boosting voter turnout, while Han (Reference Han2016) provides some evidence for the effectiveness of email in mobilizing members of an organization, the National Association of Doctors, to sign a petition. However, recent evidence from the US also suggests that emails that urge email subscribers to pay attention to a specific local political issue might backfire and make citizens less, not more, likely to pay attention to that issue (Butler and Hassell Reference Butler and Hassell2018).
2.1. The networked benefits for civic activists
Identifying other people with similar interests is an enduring problem confronting civil society groups. By allowing members to choose, or self-select membership by interest, social media such as Facebook allow people to more easily build communities. In the case of specific campaigns, centered around a single issue or an array of related issues, the advantages of social media stand out even more clearly. Facebook pages promote a specific cause of interest to a specific set of people. Seeing that others, especially friends, have joined is an additional factor of attraction. Expectations about the possibilities of networked technology may also exert a pull.
Online networked communities dedicated to a cause are likely to attract certain types of people. Research in psychology shows that some people are more likely to take the initiative and lead (Verplanken and Holland Reference Verplanken and Holland2002). Margetts et al. (Reference Margetts, John, Hale and Reissfelder2015, Reference Margetts, John, Hale and Yasseri2016) demonstrate that personality type is an important predictor of social media activism. They argue that citizens who are more open to experience should be more likely to cross the threshold of social media activism (Margetts et al. Reference Margetts, John, Hale and Yasseri2016). Moreover, research on personal values and decisions shows that action on congruent values is more likely when people perceive values central to their well-being. People who care deeply for a cause are more likely to be attracted to a community fighting for it (Margetts et al. Reference Margetts, John, Hale and Reissfelder2015; Vaccari et al. Reference Vaccari, Valeriani, Barberá, Bonneau, Jost, Nagler and Tucker2015).
Information is a necessary condition for success on a common cause (Lieberman et al. Reference Lieberman, Posner and Tsai2014). By design, social networks facilitate the flow of information. In countries where traditional media and other outlets of information are known to be poor sources of politically-relevant and credible information, social media can assume outsized importance. The low costs of entering the media market via social media means anyone enthusiastic enough can start providing information. Whereas other information sources may need to address a wider set of issues, a network community of interest can choose to zero in on a common cause. This may increase knowledge about the object of the campaign.
Being embedded in a social network can create various side-effects that may be beneficial for a cause. The relative visibility of acting within a network can create a variety of possible benefits to personal involvement (Klandermans Reference Klandermans2002). Social esteem and peer-pressure, are more likely to be operative in a network setting, where others know about one's actions (Gerber et al. Reference Gerber, Green and Larimer2008).
Less tangible benefits of community include a feeling of belonging and of dedication. Individuals participating in campaigns desire social recognition, a sense of belonging to a group (Kitts Reference Kitts2000), or a common cause (Goodwin Reference Goodwin1997) to motivate them to pay the costs of participating. Sharing one's opinion online can work differently depending on whether one is involved in a community or not. People can thus become more engaged because their opinion matters to others, increasing the importance of the issue to the participant.
Our study builds on Bennett and Segerberg (Reference Bennett and Segerberg2012)'s seminal distinction between collective and connective action. In some ways, our research focus is primarily on how adding a social media aspect to a campaign could help overcome collective action problems, by, for example, incentivizing members to join a protest. Critical mass dynamics, or coordinated effort is central for a group's success (Klandermans and Oegema Reference Klandermans and Oegema1987). Social media are well-suited for that purpose: people receive information about the intentions of others, and can set their actions in response to the anticipated behavior of others. Collective action such as formulating and sharing a common position e.g. via petitions, and holding rallies and protests, is the holy grail of civic activism. Protest can lead to change (Bratton and van de Walle Reference Bratton and van de Walle1992), and contentious politics helps social interests achieve representation (Tarrow Reference Tarrow1998). An online community comprised of many people is more likely to receive support from other groups and make an impression on official authorities.
2.2. The civic role of social media in façade democracies
The importance of social media for civic causes may vary according to the opportunities provided by the system for grievance redress. In established democracies, civic associations, well-functioning party networks and working state institutions better ensure that political choices broadly reflect grassroots social preferences (Putnam Reference Putnam2000). However, countries such as Bulgaria, Turkey and Ukraine that have transitioned to democracy more recently, feature a veneer of political freedoms, while governments also cater to special interests. Many Eastern European countries have weak civil society associations (Howard Reference Howard2003). Political parties are non-programmatic, and there are no neighborhood clubs and organizations, which make social media all the more relevant for civic activism.
In façade democracies, or hybrid regimes (Diamond Reference Diamond2002), special interests hollow out institutions. Traditional media are easily silenced, and existing institutions lose credibility with the public. Overtime, the politics of corruption multiplies popular grievances. Thus, a market is created for new ways of doing things. Informal institutions arise, assuming counter-culture overtones of resistance, centered around communities of creative defiance. In façade democracies, politics often happens online. Citizens can act as journalists to quickly reach like-minded supporters via point-to-point networks. Others can contribute information. When little (or fake) debate is allowed in news outlets, social media become a focal point.
Table 1 summarizes data from European democracies on views of social media as a platform to have one's voice heard on political issues.Footnote 1 The data indicate that in younger European democracies, people believe that their voice is not really heard in the political system—and, at the same time, they are more likely to recognize online social media as a potential tool to redress grievances the system does not meet. Similar gaps do not exist in Western Europe. Our findings speak most directly to the set of cases that have these characteristics.
Voice heard: traditional politics and social media
As governments in façade democracies are at least nominally committed to individual rights and freedoms, network disruptions to stop or harass civic activists are difficult or impossible. A recent quantitative study found that Sweden, an advanced democracy, shuts down the internet more often than Bulgaria, a façade democracy (Howard et al. Reference Howard, Agarwal and Hussain2011). In façade democracies, therefore, we expect social media to play an outsize role for civic activism. Unlike the case of authoritarian regimes, such as Egypt, where the main objective of citizen activists is to bring down the government, in façade democracies the issues are classic collective-goods problems: keeping a patch of the city green such as in Gezi Park, Istanbul (Tufekci Reference Tufekci2017).
2.3. Hypotheses
We next outline our hypotheses. Our main interest lies in showing the effects of a networked digital campaign on attitudes and perceptions related to civic activism and the cause being championed. We can contrast a campaign conducted on social media to the no campaign case and show that social media add some value. However, it would be more instructive to also include a non-networked digital technology, such as email, among the experimental groups. We do so by conducting a campaign via email newsletters.
If our arguments about the network are correct, non-networked digital communication technologies should be unlikely to have similarly strong effects. Deprived of the dynamic, interactive environment, digital communications should not exert a comparable pull, nor produce the same effects. Since email and Facebook will provide the same information, if what is at work is information, we expect to see the same patterns among subjects enrolled in either. If Facebook differs, it must be that some feature of the social network is at play—the ability to share, or to have others share what you have shared.
In addition to its theoretical value, our comparison has important practical applications. Many campaigns are currently conducted via email newsletter, and organizers may want to probe the efficacy of a medium, that only guarantees that the message is conveyed. Social media convey information but also link the recipients.
Hypothesis 1 An online social network civic campaign will attract disproportionately people with predispositions for the cause, risk-acceptant personality traits and activist profiles (i), a selectivity effect that will be stronger for networked than for non-networked online communication technologies (ii).
Hypothesis 2 Campaigns conducted via online social networks will result in information-diffusion, will increase social engagement of participants around the issues, and will increase perceptions of success in terms of achieving the objective of the campaign (i), effects that will be stronger for networked campaigns than for campaigns involving non-networked online communication technologies (ii).
3. Embedding an experiment in a campaign-rollout
Studying the impact of social media on bottom-up politics has traditionally been plagued by a methodological problem: “Even the most trenchant empirical contributions have been saddled thus far with limitations, such as the unrealistic assumption that ‘users joined the movement the moment they started sending Tweets about it’ ” (Jost et al. Reference Jost, Barberá, Bonneau, Langer, Metzger, Nagler, Sterling and Tucker2018, 86). It is difficult to distinguish and separate selection from causality (Fowler et al. Reference Fowler, Heaney, Nickerson, Padgett and Sinclair2011). Campaigns, online or not, are neither created, nor joined randomly. In a typical example, a researcher would observe a Facebook page and offline activity by people linked to the site. One may be tempted to infer that social media were an effective means of mobilizing people. The problem is that the researcher does not observe a world where organizing via social media was unavailable or where the campaign was conducted via another medium. We address this problem with the use of a field experiment. In our experiment, we use the rollout of a campaign to randomize the eligibility of people to enroll in a Facebook-coordinated campaign or in an email-coordinated campaign for a cause. The control group receives no invitation.
Our case is drawn from Bulgaria. The case involves a civic campaign to protect Black Sea beaches from (often illegal) development. Environmental causes are popular, yet ruling parties have done little to ensure that private development is consistent with the law or the public interest. The threats to nature in Bulgaria are many. Perhaps foremost among them is the ability of special interests to take over land for commercial development while circumventing the country's legislative requirements for local and national review of development plans. The country's legal system is weak and seldom indicts suspected malfeasance. Public opinion holds many cases of rezoning in natural parks to be deliberate, possibly illegal, and in the service of the select few over the public interest.
Ours is a campaign of civic vigilance. Nature—underdeveloped beaches, dune-associated flora and fauna—are a resource, to be enjoyed and passed on to future generations. Bulgaria's post-communist elite has proved adept at finding smaller or bigger doors in the country's legislative and administrative institutions to privatize public land and to develop it, for substantial personal gain. One mechanism, for example, is to say that the law protects dunes against development, but also to spend 20 years to produce a map of the dunes. This leads to paradoxical developments such as heavy machinery ploughing away dunes, while the controlling agencies give interviews in front of the melting dunes to explain that the dune is not on a map as the law required and so “this was not a dune.” In this setting, social media help civic society in the following way. Campaigns form around resources at risk. If and when an encroachment is noticed, people begin an aggressive campaign of sharing pictures and notes about the violation. If the uproar is sufficient, the country's populist leaders step in and “personally” stop the development. This mechanism can stop encroachments before they have advanced enough to reach a critical point of no return.Footnote 2 Key to this mechanism is user connectivity and personal initiative. For example, pages coordinating such activities need to pick up a photo and spread the message quickly. Our experimental setup allows for peer effects: even though the encouragement to join the Facebook page does not come from a friend, if and when one shares a picture or comments, friends are engaged and messages spread.
3.1. Experimental setup
The campaign in which we embedded our experiment was called ZaMoreto,“For the sea”. We randomly assigned subjects to receive an encouragement to take one of the two treatments (to like the Facebook page or to sign-up to an email newsletter), or not to receive any encouragement. People received the same or very similar information via email as they did via the Facebook page, but they did not have access to the interactive, networked features social media offer. Hence, what differs between them is that one is a non-networked digital campaign, and the other, a campaign involving an online social network. The Facebook page we relied on for the Facebook campaign, ZaMoreto was newly-launched and had a small following (about 200 likes) by the time people got the encouragement. Moreover, while the page was set-up in cooperation with members of the activist community, their involvement was informal and low-key. We would define the campaign as a leaderless campaign, centered on a common cause. As such, this is a very typical example of the contemporary social media landscape.
Because assignment to the social media and newsletter groups is random, any systematic post-treatment differences between the groups will likely be due to the campaign and the means of conducting it. Our campaign is setup in ways that mirror some of the basic choices many or most people would typically have to make when contemplating the launch of a civic campaign. In terms of the sheer popularity of the two means of promoting civic causes via different electronic media, email newsletters and Facebook are by far the two most popular choices. Thus, this comparison speaks directly to a very practical choice involved in the lives of a large number of civic campaigns.
3.2. Recruitment, assignment and outcome measurement
During the first stage, we chose to recruit subjects via two channels. First, a professional polling agency approached 3332 subjects between the ages of 18–60 that were sampled randomly within all Bulgarian towns with a population of 50,000 or more. A filtering question made sure that only people with Facebook accounts received invitations to participate in the study. Subjects were dispersed around the country, and represented a tiny proportion of all Facebook users. As a result we do not believe they were connected in any way among themselves at the start of the experiment. Since Facebook use is wide-spread in Bulgaria (between 1/2 and 2/3 of all Bulgarians are on the social network), our sample is representative of a large and growing part of the Bulgarian population. At this stage, the polling agency recorded basic demographic information and views about the environment, afterward we used Bernoulli (simple) random assignment to allocate subjects to one of three experimental conditions, with resulting probabilities of 0.34 (Facebook condition), 0.34 (newsletter condition) and 0.32 (control).Footnote 3
The research team then emailed all subjects in the representative sample if they would participate in our study, register in the online panel and answer questions online. The first online survey wave included questions about views on the environment and some information on social attitudes and political behavior. A total of 527 out of 3332 subjects confirmed participation in the study and answered the first wave of the questionnaire. We refer to this group as the representative sample, reflecting the manner of its recruitment, and distinguishing it from our second sample, a sample of environmental activists. For that, we posted a widget on the Facebook page of the NGO-umbrella organization ZaDaOstanePriroda (SoThatNatureRemains), an existing page with 20,000 “likes.”Footnote 4 Users were invited to take part in a study, and 648 environmental activists filled in the baseline questionnaire with demographic information, and views of the environment. Afterward we used again Bernoulli (simple) random assignment to allocate subjects to one of three experimental conditions, with resulting probabilities of 0.31 (Facebook condition), 0.35 (newsletter condition) and 0.34 (control), and emailed all registered activists the first wave of the online questionnaire, which was answered by 357 respondents. This constitutes our activist sample. Comparing the demographic and attitudinal profile of the activist sample to that of randomly selected Facebook users will give us a sense of how online communities differ from the general population.
Over the next eight weeks, the newly created page was filled with content about specific surviving wild areas around the sea coast, including threats to their survival, and possible strategies for keeping them wild. About five posts per week were posted, always including a picture and brief text. The same content was assembled and mailed to subjects in the email newsletter group once a week. We logged activity on the Facebook page, including shares, likes and comments. A snapshot of the page, which has continued to be active, is provided in Figure A6 of the Online Appendix. We also provide a snapshot of a typical email newsletter (see Appendix Figure A7). Moreover, we display a typical post that gained a larger number of likes and shares, including comments by subjects, in Figure A8.
After eight weeks, we deployed the second wave of the online survey, in which we asked subjects about their views on the state of the environment, the likelihood of success of civic initiatives and other outcome variables. Respondents always received a customized, unobtrusive link that made sure answers were matched to the correct interviewee.
To reduce attrition between the two waves of the online survey, we entered everyone filling out the survey in a raffle with a few small prizes, such as environmentally themed T-shirts and a GPS walking-band. Sixty-four percent of first wave respondents filled in the second wave of the survey.
Participation in the study did not pose risks for participants. All participants were informed about the general area of the study, but were not informed about the research design because that would have undermined our ability to carry out the study's objectives. Participants could select to be updated, upon completion of the study, when a summary of the results was passed on to them. The identity of the respondents was never known to the principal investigators, as a standard confidentiality measure. Upon completion of the study, both the newsletter group and the control group also received invitations to like the Facebook pages of ZaMoreto and ZaDaOstane. The hypothesized benefits of the treatment for civic action could thus accrue to these groups as well.
4. Results
One unique feature of our study is that we can say more about the types of respondents selecting into social media campaigns. First, we systematically compare respondents recruited via the representative Facebook sample and those we recruited from the activist Facebook page of the NGO community ZaDaOstanePriroda. This comparison is observational in nature. It is nonetheless of interest since few researchers have been able to compare the profiles of a dedicated online community to a more representative sample of the online population.
Table 2 and Figure 1 show how a sample of Bulgarian Facebook users compares to a sample of Bulgarian environmental activists on several important attributes, including key demographics, social connectedness, personality traits, attitudes and perceptions, such as love for nature and trust in public institutions, and prior self-reported civic activism.Footnote 5 Our questions on personality traits, and civic activism come from standard batteries used by the European Values Study.Footnote 6
Comparing the types of respondents attracted by a green page, ZaDaOstane, to a representative sample of Facebook users on previous civic activism, attitudes and personality traits. Solid line—representative, broken line—activists.
Covariates that predict belonging to the activist sample over belonging to the representative sample—based on linear regression
***p < 0.01, **p < 0.05, *p < 0.1, robust standard errors (HC2).
4.1. How do members of an existing Facebook activist community compare to the general population that uses Facebook?
Column 1 in Table 2 displays demographics and social connectedness only, while column 2 adds attitudes and personality traits, and column 3 additionally accounts for self-reported activism. As one would expect—the respondents recruited via the NGO's page are significantly more likely to be university graduates, and to be more embedded within their social networks. Table 2 also shows that environmental activists are more likely to get their news from social media than the representative sample and more likely to live in larger towns. Importantly, however, they do not markedly differ on self-reported economic status, gender or age. Column 2 shows that respondents recruited via the NGO's page are significantly more likely to love nature, and they are less likely to trust public institutions. Moreover, they are significantly more likely to love danger and adventure than the general population. Finally, as one would expect from the activist sample these differences translate into higher self-reports of civic and environmental activism (column 3).
4.2. Who joins a campaign if encouraged?
We know that Facebook pages promoting causes, whether green causes, regime change, or Humphreysan rights, tend to attract people with relatively intense preferences for these causes. We may conjecture that these people are the ones that are also more active as citizens (Norris Reference Norris2011). What we suspect, but have not been able to establish to date, is that joining a community of other activists, by joining a Facebook page, produces independent causal effects on the joiners.
In our study, we randomly assign people to either a control group or to two experimental groups that receive an encouragement to join (follow) a campaign. People who are given an encouragement select whether to take it or ignore it. For most experimental studies, the stage of people selecting to take the prescribed treatment is of no direct interest. For us it is: it shows the type of person who responds to a recruitment effort. In our case, it also shows whether the type of respondent who selects to follow a social media coordinated campaign is systematically different from the type of respondent who would sign up to follow an email newsletter campaign.
Contrary to what we expected, the Facebook and newsletter campaigns attracted the same share of respondents to sign up (58 percent in the Facebook group compared to and 57 percent in the newsletter group).
Figure 2 shows that some of the differences observed in Figure 1 between the representative and the activist samples also predict whether a respondent joins a Facebook or newsletter campaign on environmental issues if invited to do so. Appendix Table A4 models selection on observable pre-treatment covariates into liking either the Facebook campaign or signing up to the newsletter, for the representative sample, systematically based on the covariates recorded in wave 1. The most important variables that positively predict whether a respondent chooses to follow the Facebook campaign are educational attainment, an adventurous personality and prior civic activism. Using randomization inference we find that, taken together, the differences on observable attributes between subjects who take the encouragement to like the Facebook page and subjects who take the encouragement to sign up to a newsletter are not significant (randomization inference-based p-value of 0.62; see Figure A1 in the Online Appendix).
Comparing the profiles of respondents who join our Facebook campaign when invited to do so to those who decide not to join, to respondents who decide to follow the newsletter campaign and those who decide not to sign up; solid line: compliers; dashed line: non-compliers.
Overall, the findings are hence consistent with the first part of Hypothesis 1. Members with predilections for the cause are especially likely to respond to recruitment appeals. However, there is little evidence for the second part of the hypothesis, that a networked campaign exerts a stronger pull than the non-networked campaign as both campaigns recruited a similar number of participants. While the Facebook campaign was selective on education, and prior activism, and the email campaign was not, taken together the observable differences between email newsletter sign-ups and Facebook followers were not significant. This is the first benchmark finding of this nature that we are aware of.
What we cannot see from these observational data is whether the distinct preferences and self-reported actions of those respondents who are engaged in environmental activist groups are purely a product of selection (Facebook groups attracting respondents who are more active), or whether there is also an independent effect of exposure to the campaign on environmental attitudes and activism. Thus, we do not know whether the differences between respondents who select to be active and those who are inactive are driven by selection only, or selection compounded by changes to the joining population. Via the randomly assigned encouragement to join the Facebook page or sign up to the email newsletter, based around a new initiative to protect the remaining undeveloped Black Sea beaches, we intend to identify the causal effects of following such a campaign. The next section reports our experimental results.
5. Experimental evidence
5.1. Do social media campaigns have a causal impact: attitudinal measures
The Facebook page ZaMoreto featured some of the behavior we think is theoretically relevant for influence to occur. In Appendix Figure A8 we include snapshots from the page, or from users’ shares of the page. They include comments such as “we are awake,” “this is a resource we must protect,” “only citizens can protect what is left,” and an example of a user's Facebook friend responding to a plea to use wild nature as a tourism resource with an example of another campaign that is already doing this for the mountains. We next check if this type of online behavior produced any transformative effect on users’ attitudes.
Because we randomly encourage subjects to follow the campaign either via Facebook or email newsletter, or assign them to control, we can identify the causal effects of the encouragement on their attitudes. Issues of panel attrition, and covariate balance are addressed in detail in the Appendix (see Figures A3 and A4, as well as Table A3). As shown in Table A3, two out of 23 covariates show some imbalances between the three experimental conditions in the post-treatment survey wave. However, a randomization inference-based test shows that all covariates, taken together, are no more predictive of treatment assignment than were we to re-assign all subjects to the three experimental groups under the assumption that there is no effect of the pre-treatment covariates on treatment assignment for any subject (p = 0.32). Moreover, attrition rates between the pre- and post-treatment waves do not vary significantly between experimental groups (randomization inference-based p-value of 0.51).Footnote 7 While we cannot test if the groups, post-treatment and post-attrition, differ on unobservables, we purposefully deployed encouragements that we thought would not deter or encourage survey participation in the future.
Figure 3 displays our manipulation check: Are subjects who were encouraged to like the Facebook page and subjects who were encouraged to subscribe to the newsletter more likely to recall the campaign logo of the environmental NGO? Figure 3 shows the Intent-to-Treat (ITT) effects of the encouragement to like the Facebook page, and of being encouraged to sign up to the newsletter on logo recall, once without adjustment for pre-treatment covariates and once covariate-adjusted.
Manipulation check: ITT of encouragement to like to Facebook page and sign-up to the newsletter on campaign logo recall, unadjusted and covariate-adjusted, 95 percent confidence intervals (CIs).
We find that being encouraged to like the Facebook page made subjects 13 percentage-points more likely to recall the campaign logo, a large and statistically significant effect. There is also a substantial, statistically significant effect of the newsletter encouragement on logo recall, albeit with 9 percentage-points, not as large. This shows that subjects in both treatment conditions are significantly more likely to recall the campaign logo than subjects in the control group, and that the manipulation succeeded, a pre-condition for influencing subjects’ attitudes and perceptions.
We now exploit the exogenous nature of experimental assignment to the treatment groups to draw conclusions about the effects of the two campaign methods on attitudes and perceptions related to the environment and to civic activism. In the post-treatment wave, we measure attitudes and perceptions on scales ranging from 0 to 2, 0 to 3 and 0 to 4. The English translations of the questions are available in Appendix Figure A2. For ease of interpretation, we rescale each measure to have a mean of 0, and a standard deviation of 1 in the control group. That means the treatment effects of the campaign conducted via Facebook or email newsletter can then be interpreted as the effects in standard deviations the campaigns would have among an untreated population. Power simulations displayed in Appendix Table A1 and Figure A2 show that our field experiment is powered at 68 percent to identify small, but meaningful treatment effects of 0.25 standard deviations, and at 100 percent to identify moderate to large effects of 0.50 standard deviations.
We first report evidence on the effects we expected to see based on our theoretical considerations on social interactions, perceived outcomes of activism, knowledge and perceived importance of the environment.Footnote 8 Figures 4, 5 and 6 plot the covariate-adjusted ITT effects of treatment assignment, the encouragement to either like the Facebook campaign page or sign-up to the campaign email newsletter, on self-reported outcomes, estimated using linear regression with robust standard errors (HC2). To correct for differential probabilities of assignment to experimental conditions between the representative and activist samples, we always use inverse probability weights to weight to the inverse of the probability of assignment to the experimental condition the subject is in. While Figure 4 shows whether being encouraged to follow a campaign affected interactions with others on social media and offline, Figure 5 displays the effects of treatment assignment on attitudinal outcomes related to civic engagement. Figure 6 plots the effects on perceived issue importance and environmental knowledge.
Effect of treatment assignment (ITTs) on social interaction—covariate-adjusted, 95 percent CIs.
Effect of treatment assignment (ITTs) on activism-related attitudes—covariate-adjusted, 95 percent CIs.
Effect of treatment assignment (ITTs) on environmental attitudes and knowledge—covariate-adjusted, 95 percent CIs.
The corresponding covariate-adjusted Complier Average Causal Effects (CACE), based on IV regression with robust standard errors (HC2), where the randomly assigned encouragements are used as instruments for the endogenous decisions to like the Facebook page or to sign-up to the newsletter, are displayed in Figure 7. In order to identify the CACE, we need to assume that the randomly assigned encouragements only affect outcomes via liking the page or signing-up to the newsletter, not via other, alternative channels.Footnote 9
CACEs of liking the Facebook page and subscribing to the newsletter on attitudinal outcomes—covariate-adjusted, 95 percent CIs.
The main result from this study is that both the Facebook and the email campaign conducted over eight weeks had only minimal effects on attitudes and perceptions related to the environment and to civic activism. Most estimated effect sizes are very small (in the region of − 0.12 to 0.12 standard deviations), and out of 16 estimates, 6 have a positive sign, and 10 have a negative sign. Hence, overall, the campaign was ineffective at changing perceptions and attitudes, no matter whether conducted via Facebook or email.
There is some weak evidence, not statistically significant by any conventional standard, that the newsletter (p = 0.06) and the Facebook campaign (p = 0.18) might have positively affected discussion about the environment with friends in real life. But both campaigns made little difference to subjects’ discussion behavior online. The ITT estimates, − 0.07 and 0.02 standard deviations, are very small, and not statistically significant.
There is no evidence that liking the Facebook page positively affected other behavioral, attitudinal or knowledge outcomes in Figures 5 and 6: subjects are neither more likely to perceive that they are able to make a difference, nor do they report a higher probability of participating in environmental protests in the future. Surprisingly, subjects who followed the Facebook campaign think that others are less likely to appreciate those who campaign (see Figure 4) and we also report a negative estimate of the email campaign on issue importance, which is consistent with Butler and Hassell (Reference Butler and Hassell2018)'s finding from an email experiment in the United States.
Figure 8 shows that effects do not differ between the representative and the activist samples. Due to ceiling effects one might expect that following a campaign should have larger effects on the attitudes, perceptions and environmental knowledge of the non-activist population. However, we do not find that the effects differ. Overall, there is hence weak or non-existent support for Hypothesis 2 in its different parts. The environmental campaign conducted via Facebook did not change attitudes and perceptions related to civic engagement, nor did it positively affect perceptions of internal or external efficacy.
ITTs on representative sample (column 1) ITTs on activist sample (column 2)—covariate-adjusted, 95 percent CIs.
6. Conclusion
In line with expectations, we show that activists on social media sites are unrepresentative compared to the general Facebook population, specifically in relation to education, social connections, personality traits and previous civic activism. We also show that those who take up the encouragement and follow the campaign via Facebook are more educated, and more likely to have been active before. Thus, the network draws in the more active types.
What is less well understood is whether these differences are due to selection effects (activists selecting into online communities) or if social media independently affect attitudes and preferences of citizens who use them. Our results allow for some, albeit preliminary, conclusions to be drawn. Against expectations, the networked campaign did not attract a larger number of participants than the campaign run via an online newsletter, and overall, there were few observable differences between subjects who decided to follow a networked compared to a non-networked campaign. Finally, and most importantly, we find little evidence that the social media campaign shaped how citizens and activists saw a cause or how they related to the campaign.
While our results are hardly generalizable to mass mobilization in the context of high salience events such as revolutions, or mobilization in response to specific government actions, they speak to the difficulties a large number of non-governmental organizations face when attempting to inform, organize and mobilize citizens around causes such as the environment or mid-level corruption. While social media are effective means of organizing, our experiment suggests that messages that are shared on Facebook groups in order to inform and mobilize citizens had little lasting impact beyond “likes” and “shares.” Thus, in the language of Bennett and Segerberg (Reference Bennett and Segerberg2012), our intervention may have induced some connective online action, but that hardly translated into measurable gains from a collective action perspective, as measured by the desire to take part in protest. Perhaps this is due to the fact that our campaign was relatively modest, and did not concern some of the big issues agitating digitally-connected mass audiences (Tufekci Reference Tufekci2017).
The null results from this field experiment do not stand alone. They are in accordance with other work that speaks to the limits of social media in influencing attitudes and behaviors. Earlier work shows that public tweets are ineffective at affecting online activism related to environmental causes (Coppock et al. Reference Coppock, Guess and Ternovski2016), that signing-up to Facebook had—if at all—negative effects on self-reported electoral participation (Theocharis and Lowe Reference Theocharis and Lowe2016), and that Facebook ads were ineffective at persuading voters (Broockman and Green Reference Broockman and Green2014). We also have confidence in our experimental results because the data show the observational patterns and associations between activism and background attributes that we would expect based on theory and earlier work. While the intervention did not take place in a context of mass mobilization, the salience of the environmental issue in Bulgaria is high, and the environmental activists we collaborated with expected the campaign to be effective. We also show that the intervention was memorable to respondents. Facebook users who took the encouragement to follow the campaign were 21 percentage-points more likely to recall it than users allocated to the control group. Overall, this study adds to a growing body of work showing that it is more difficult than commonly assumed to change political attitudes and behaviors via social media interventions.
Our results are both good and bad news with regard to democratic practice. They are good news because they suggest that growing concerns about campaigns exerting disproportionate influence on citizens’ attitudes and opinions on social media may be overstated. In contrast to the expectation that Facebook or other social media can easily make citizens believe in causes that they would not otherwise adhere to, the results of this study show that following a Facebook page on a political issue for multiple weeks did not change citizens’ opinions. At the same time, they provide some bad news for civic organizations who would like to engage the general public on important issues of public policy. It appears difficult to turn citizens into activists by providing them with accurate information about policy problems.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/psrm.2020.13.
