On 3 June 2017, a British man drove a truck into a crowd of pedestrians on the London Bridge before going on a stabbing rampage in a nearby market. According to friends and neighbors, the man had been inspired to carry out the violence by watching extremist videos on YouTube.Footnote ¹ On 27 October 2018, another man stormed into a synagogue in Pittsburgh, Pennsylvania, shooting and killing eleven people during weekend prayers. Shortly before the attack, he had written on social media of his fight against the Jews for facilitating an influx of migrants into the United States. Although he acted alone, investigators believe that he was radicalized by engaging with extremist content online.Footnote ²

These stories illustrate a growing phenomenon that has gripped the attention of many governments in recent years. The digital revolution and the rapid expansion of information and communication technologies have made it easier for terrorist groups to recruit supporters and inspire violence around the world. Groups espousing extremist ideologies are now actively using social media platforms to disseminate and promote their ideas.Footnote ³ The Islamic State (IS), for example, recruited thousands of individuals around the world by disseminating militant propaganda through elaborate campaigns on Facebook, Twitter, YouTube, and Telegram. Similar trends were seen with far-right extremist websites which inspired terrorism by promoting hate against minorities.

To combat radicalization, governments began enacting policies to regulate social media content and prevent hate speech, misinformation, and violent propaganda from influencing their citizens. Some focused on challenging extremist ideologies by flooding social networking sites with counter-speech campaigns. The US State Department, for example, led a campaign to encourage individuals sympathetic to IS to “think again” and “turn away” from the group.Footnote ⁴ Another approach, led by technology companies, focused on taking down content promoting violence—an initiative that resulted in the suspension of millions of accounts from various social media platforms in just a few years.Footnote ⁵

But efforts to counter violent extremism extend beyond online content. Many governments looked for ways to prevent extremists from inspiring violence in the “offline” world too. At the forefront of these initiatives were community engagement programs, which sought to encourage citizens to serve as “early warning systems” by sharing information with the government on individuals who might pose security risks. The idea was that citizens with personal connections to radicalizing individuals, like friends or family members, are best positioned to detect changes in behavior that might convey early signs of radicalization.Footnote ⁶ By raising citizen awareness of violent extremism, governments hoped to access crucial information on potentially radicalizing individuals to monitor their behavior and, if needed, stop them from engaging in violence.Footnote ⁷

Despite these increased efforts to combat extremism, we know very little about what happens in the online world when counter-extremism efforts take place on the ground. There is a large literature on how counter-radicalization programs affect targeted communities in areas other than extremism.Footnote ⁸ There is also a growing body of research on the effects of counter-speech campaigns on online engagement with violent groups.Footnote ⁹ However, there is little empirical work on how “offline” efforts to counter extremism shape the online behavior of those sympathetic to violent extremist groups.

This study provides the first large-scale, systematic analysis of the effects of on-the-ground counter-extremism initiatives on engagement with terrorist groups in the online world. I focus on a large program to prevent radicalization in the United States, which was initiated by the Obama administration to counter jihadi-inspired terrorism, and examine how it shaped the behavior of IS sympathizers on Twitter.

I build on a large body of research on terrorism, insurgency, and online mobilization to test two mechanisms by which counter-extremism efforts might shape online engagement with violent groups: deradicalization and strategic behavior. Deradicalization refers to individuals with extreme worldviews adopting more moderate positions after exposure to counter-extremism interventions. In the context of support for IS, this might be observed as a decline in rhetoric supporting the group, its ideology, and its actions around the world, as well as disengagement from online networks affiliated with it. Strategic behavior refers to extremist supporters becoming aware of government surveillance and altering their behavior to avoid detection. This might be observed as attempts to alter online identities or migration to private social media platforms that are less observable by the public.

Drawing on rich Twitter data on IS sympathizers in the United States and granular information on community engagement events taking place between 2014 and 2016, I examine how counter-extremism interventions affected online engagement with IS on social media. I identified the timing and location of dozens of community engagement activities taking place across the United States and matched them with geolocated data generated by IS sympathizers on Twitter: several million tweets with rich user-profile metadata. I study changes in online pro-IS rhetoric, as well as other behaviors that are indicative of awareness of surveillance, such as profile-picture or screen-name changes and the promotion of encrypted online platforms. Using difference-in-differences models, I examine how IS sympathizers behaved online after these events took place in their vicinity.

I find that community engagement activities, which encouraged citizens to “say something” if they “see something,” led IS sympathizers on Twitter to engage in strategic behavior. In the weeks following counter-extremism events, the group's supporters in event areas significantly reduced the number of tweets endorsing IS, including posts expressing sympathy with the group, describing life in IS-controlled territories, and mentioning foreign fighters. While this decrease could be interpreted as evidence of deradicalization, I find that proximity to counter-extremism activities led IS sympathizers to take additional actions that indicate their continued support for the group and their awareness of surveillance. After counter-extremism events, these users altered their public identity on Twitter by changing their profile pictures and screen names, and increased the number of propaganda-disseminating accounts they followed on the platform. When the group began migrating to Telegram in the end of 2015, these users advertised the new platform to their followers, suggesting that they may have switched to Telegram to avoid detection.Footnote ¹⁰

These results challenge the assumptions underlying many counter-extremism interventions, which presume that citizen vigilance can help sway at-risk individuals from the path of violence. By showing how government-sponsored community engagement activities propelled IS sympathizers to behave strategically on social media, the study points to an underappreciated consequence of these emerging efforts. As violent groups are increasingly using online platforms for recruitment, it is important to understand how on-the-ground activities to prevent radicalization shape the online behavior of extremist supporters. After all, the next perpetrator of a terrorist act might be inspired by interacting with these groups online.

Data Sources

Counter-Extremism Events in the United States

In August 2011, the Obama administration initiated a counter-radicalization strategy, Empowering Local Partners to Prevent Violent Extremism in the United States, which focused on strengthening the government's engagement with local communities whose members may be targeted by violent groups.Footnote ¹¹ While the plan's official goal was to target violent extremism across all ideologies, in practice, the vast majority of its activities focused on jihadi-inspired extremism—an approach that generated much opposition among civil rights activists.Footnote ¹²

A large part of these efforts consisted of community engagement events, which sought to increase citizen awareness of the threat of extremism and encourage collaboration with the government to prevent violence. These activities included, for example, community roundtables that brought together government officials and members of local communities to strengthen relationships and share information, and community awareness briefings, in which government officials presented details on the process of radicalization and online recruitment by terrorist groups.Footnote ¹³ Figure 1 shows advertisements of community engagement events in New York and Colorado, and Figure 2 presents photos from activities in Georgia and Arizona.

Figure 1.

Advertisements for community engagement events aimed at increasing local awareness of violent extremism (data from CrowdTangle, a public insights tool owned and operated by Facebook)

Figure 2.

Photos from community engagement events in Atlanta, GA, and Phoenix, AZ (photos courtesy of Islamic Speakers Bureau, Atlanta, and Islamic Community Center of Phoenix)

Although these events attracted mostly community leaders and some interested citizens, information on the government's call to “see something, say something” was shared throughout the community. Individuals in areas that were targeted by counter-extremism activities were encouraged to keep an eye out for suspicious behavior, even if they did not attend these meetings. Take the case of Sal Shafi. In response to the government's call for vigilance, he decided to report his son's behavior to the FBI after observing him consume extremist content online, changing his appearance, and trying to travel to Turkey.Footnote ¹⁴ Many family members in a similar situation informed government authorities of their relatives’ behavior out of a desire to prevent violence.Footnote ¹⁵

Local communities were also encouraged to monitor their members’ online behavior, as many extremist sympathizers maintained public profiles on social media.Footnote ¹⁶ Amani Ibrahim, for example, tried to stop her teenage son from going on the Internet after becoming aware of his advocacy for IS on Twitter. When her efforts were not successful, she followed the advice of a community leader and reported her son's behavior to the government, which resulted in him being arrested and sentenced to several years in prison.Footnote ¹⁷ Indeed, many anti-terrorism indictments against American citizens drew heavily on information shared on social media.Footnote ¹⁸ Thus, even though individuals who engaged with IS online did not attend community engagement events, they were likely aware of them; some even shared information on such events and talked about the government's monitoring of social media with their Twitter followers (Table 1).

Table 1.

Twitter posts by IS sympathizers in the United States, sharing information on community engagement events

I collected information on community engagement events taking place between 2014 and 2016 from newsletter reports published by the US government.Footnote ¹⁹ I gathered data on the dates of these events, the cities in which they took place, and the type of engagement activity carried out in each event. Figure 3 shows the number of community engagement events by month. The online supplement provides more information on these activities and detailed summary statistics tables.

Figure 3.

Community engagement to counter extremism: monthly number of community roundtables held by the Department of Homeland Security in the United States from 2014 to 2016

Islamic State Sympathizers on Twitter

To assess whether these activities influenced the behavior of extremist supporters online, I use Twitter data on IS sympathizers in the United States who interacted with the group on the platform between 2014 and 2016. I provide an overview of the data collection process, which includes (1) identifying accounts of IS supporters on Twitter and downloading information on their posting history; (2) coding the extent to which their posts reflect support for IS; (3) measuring changes in their public Twitter profiles; (4) predicting users’ locations; and (5) geographically matching users with community engagement events. More details on each of these steps are in the online supplement.

1. Identifying IS accounts on Twitter

Using Twitter's public APIs,Footnote ²⁰ I collected detailed information on accounts of IS sympathizers on Twitter who followed at least one of about 15,000 accounts that actively disseminated IS propaganda on the platform.Footnote ²¹ These users engaged with the group's Twitter networks in various capacities. Some expressed strong sympathy with the group and its ideology; some actively “retweeted” its propaganda, while others were more passive in their online engagement with the group. I downloaded every available piece of information on these accounts before they were suspended from the platform, including data on user profiles, screen names, locations, historical tweet timelines, and lists of friends and followers. In total, I collected information on 30,358 users in the United States, who posted a total of 15,140,867 tweets between 2014 and 2016.

2. Measuring online expression of extremist ideology

Using the historical tweet timelines of these accounts, I measured the extent to which each post represented pro-IS content. Since the volume of tweets was large, I used supervised machine learning to classify tweets into different content categories: expressions of sympathy with the group, discourse on life in IS-controlled territories, communications about the group's actions in the Syrian civil war, and mentions of foreign fighters. I used a crowdsourcing platform to manually label a training set of about 30,000 randomly selected posts, and trained models to predict the content of unlabeled tweets in each category.Footnote ²² I created a tweet-level index variable capturing pro-IS sentiment across all four categories. The index was constructed by summing the predicted content scores of each tweet along the four categories, and normalizing the sum to range between 0 and 1. To make the tweet-level data easy to interpret at the user level, I generated an aggregated variable that measures this content in each user's Twitter posts in the week before and one to four weeks after community engagement events.Footnote ²³ Table A9 in the online supplement shows summary statistics for this variable.

3. Measuring changes in profile-level metadata

In addition to content, I measured other online actions that individuals might take to evade surveillance. Drawing on user-level metadata provided by Twitter's public APIs, I collected information on IS sympathizers’ profile pictures and screen names, and the number of propaganda-disseminating accounts they followed on the platform. This includes weekly observations of IS sympathizers’ profile information, sampled every seven days, from January to June of 2016. I created a user-week-level data set measuring changes in these variables in the week before and one to four weeks after community engagement events, and used it to examine whether those users who were near counter-extremism activities were more likely to alter their online identities in their aftermath.Footnote ²⁴ Section 4 of the online supplement provides details on the user metadata. Summary statistics show that the vast majority of IS sympathizers did not change their profile pictures or screen names during this period (Table A10 in the online supplement).

4. Predicting IS supporters’ locations

Since few social media users enable geotagging of their posts or provide location information in their accounts,Footnote ²⁵ I estimated the locations of IS sympathizers on Twitter with an algorithm that predicts user locations from geographic information available in their networks.Footnote ²⁶ In several contexts, this algorithm has been able to predict geolocation relatively accurately, outperforming other methods that rely on network data.Footnote ²⁷ I use predicted locations in this study to avoid relying on the small subset of users with reported locations. Section 3 of the online supplement provides details on the method, along with information on prediction accuracy and model stability.

5. Matching IS sympathizers to community engagement events

To determine which users were near community engagement events, I matched users’ predicted coordinates with geographic information on localities across the United States. I used the US Census TIGER database to access spatial data on US localities (cities, towns, etc.), and coded each location for whether it was targeted by counter-extremism programming.Footnote ²⁸ Figure 4 displays the predicted locations of IS sympathizers on Twitter, along with the locations of community engagement events taking place between 2014 and 2016. In my analysis, I compare the online behavior of IS sympathizers who were in event areas to those who were not, before and after each event.

Figure 4.

Estimated US locations of Islamic State sympathizers on Twitter, mapped along with community engagement events of the Office of Civil Rights and Civil Liberties

Empirical Strategy

To analyze the relationship between community engagement activities and the online behavior of IS sympathizers on Twitter, I measured IS sympathizers’ online behaviors before and after each event, which included information on the tweets they posted, whether they changed their profile pictures or screen names, and the number of propaganda-disseminating handles they followed. For each individual in each event, I created binary indicators to distinguish between (1) observations appearing before and after each event and (2) individuals inside or outside the area of the event.Footnote ²⁹ Specifically, for each community engagement activity, I created the variable post, which is coded 1 when an online action took place after the event and 0 otherwise, and the variable in event area, which is coded 1 when the action was taken by an individual in the area of the event and 0 otherwise.

Since I study the behavior of IS sympathizers around dozens of community engagement events, I conduct a pooled difference-in-differences analysis where I examine all community engagement events simultaneously. For each of these outcomes,

I estimate the following ordinary least squares regression:Footnote ³⁰

(1)$$y_{i, j, k} = \beta _1 {\smallcaps \rm POST}_{i, k} + \beta _2 \;{\smallcaps \rm IN \;EVENT \;AREA}_{\,j, k} + \beta _3( {\smallcaps \rm POST}_{i, k} \times {\smallcaps \rm IN \;EVENT \;AREA}_{\,j, k}) + \alpha _k + \varepsilon _j$$

where y _i,j,k is an online action i (expressing pro-IS rhetoric, changing profile pictures or screen names, or following/unfollowing of propaganda-disseminating accounts) by user j surrounding event k; post_i,k is 0 when the action took place before event k, and 1 afterwards; in event area_j,k is 1 when the action was taken by an individual in the area of event k, and 0 otherwise; and α _k is an event fixed effect. In all specifications, β ₃ is the difference-in-differences coefficient of interest, reflecting how the online behavior of IS sympathizers after community engagement events is different between individuals who were in an event area and those who were not.Footnote ³¹ Standard errors are clustered at the user level.

The key identifying assumption is that in the absence of a community engagement event, individuals who are in the event area and individuals who are not follow parallel trends in their online support for IS. While it is certainly likely that the US government targeted specific areas that it deemed more likely to have individuals “at risk” of radicalization,Footnote ³² changes over time in support for IS should not be significantly different between the groups before the occurrence of community engagement events at the high frequency (i.e., a few days).

To empirically test this assumption, I visually examine whether the two groups display parallel trends before community engagement events. Figure 5 plots pre- and post-time trends in pro-IS rhetoric for the individuals who were in event areas (black) and those who were not (gray). The x-axis is the number of days between a community engagement event and when IS followers posted on Twitter. To observe time trends for all events simultaneously, I normalized the difference in days between the events and the timing of Twitter posts.

Figure 5.

Pro-IS rhetoric: parallel trends

The time trends in pro-IS rhetoric are parallel in the pretreatment period. Only after community engagement events do we observe a shift in those trends, where pro-IS content by individuals in event areas decreases, but the rhetoric of those outside of event areas does not change. We also observe that the average pro-IS rhetoric before community engagement events is higher for individuals in event areas. Table A11 in the online supplement presents a statistical test of the parallel-trends assumption; there is no difference in the time trends between the groups, except when expanding the pretreatment data back to thirty days before the events.

Results

I first present results for content produced by IS sympathizers on Twitter, showing that community engagement events led to a decrease in pro-IS rhetoric. I then present evidence suggesting that this decrease is likely driven by strategic behavior, by showing that individuals near community engagement events also changed their profile pictures and screen names, and increased their following of propaganda-disseminating accounts. Finally, I present results suggesting that these users also sought to avoid detection by migrating to Telegram, an encrypted online platform that became popular among IS supporters during the years of this study.

Changing content: a decrease in pro-IS rhetoric

Panel A of Figure 6 reports the findings for IS sympathizers’ pro-IS rhetoric on Twitter. The figure plots coefficients on the interaction term, post in event area, estimated from regressions where the dependent variable is measured at different time intervals in the post-treatment period, ranging from one to thirty days.Footnote ³³ I find that community engagement activities reduced expressions of support for IS among users in event areas. In almost all specifications, the difference-in-differences coefficient is estimated at 0.005. By way of reference, the pretreatment difference between users inside and outside the event areas is 0.006, so the magnitude of the estimated effect is economically meaningful. Since this study examines a period when IS was actively expanding its territorial control and publicizing its cause on the Internet, this decrease is notable.

Figure 6.

The impact of community engagement on Islamic State sympathizers on Twitter

Strategic online behavior

Since a decrease in online expressions of support for IS can be interpreted as evidence of deradicalization but also as strategic behavior, I next examine additional actions that are more likely to be associated with strategic behavior. If these behaviors take place alongside a decline in pro-IS rhetoric, then it would be reasonable to infer that the decrease in online expressions of support for IS is not driven by deradicalization but by awareness of surveillance in the wake of community engagement events.

In Panels B and C of Figure 6, I examine whether community engagement events propelled IS sympathizers to change their profile pictures and screen names on Twitter. The figures present difference-in-differences coefficients estimated from regressions in which these outcomes are measured across different time windows, from one to four weeks before and after the events.Footnote ³⁴ I find that IS sympathizers were significantly more likely to change their profile pictures and screen names after community engagement activities took place in their areas, a trend that was strongest about two to three weeks after the events.Footnote ³⁵ Since altering Twitter profiles was very infrequent (about 3 percent of the sample), the concentration of these changes in areas with counter-extremism activities is revealing.

Panel D shows that IS sympathizers in event areas also increased the number of propaganda-disseminating accounts they followed on Twitter.Footnote ³⁶ When compared to the pretreatment difference between users inside and outside event areas, the change reflects about a 30-to-55-percent increase in the number of Twitter handles disseminating IS propaganda. This cuts against the deradicalization interpretation because even though IS sympathizers in event areas refrained from publicly endorsing the group on the platform, they continued to passively engage with the group's Twitter networks by following more accounts.

To check that these changes are in fact happening simultaneously, I next examine whether reducing pro-IS rhetoric, changing profile pictures, changing screen names, and following propaganda-disseminating accounts are done by the same users, as opposed to a mix of different users, in event areas. In the analysis, each of these actions was examined separately. It is possible that users who reduced their pro-IS rhetoric did not also change their profile picture, and vice versa, even though in the aggregate this happened more often in event areas. Figure 7 presents difference-in-differences coefficients from regressions where the dependent variables (reported in the rows) reflect different combinations of online actions.Footnote ³⁷ I find that many users in event areas took two or more actions after counter-extremism events, and some even engaged in three or more.Footnote ³⁸ This provides further support for the strategic-behavior interpretation.

Figure 7.

Engaging in several online actions simultaneously

Migration to Telegram

Finally, I examine whether IS sympathizers migrated to private communication channels after counter-extremism events. Telegram is a messaging application that became popular among IS supporters when their accounts were suspended from mainstream platforms like Twitter and Facebook.Footnote ³⁹ In its early days, Telegram offered only one-to-one messaging, but in September 2015 it launched a “channels” service that enabled sharing content with an unlimited number of followers. Shortly after the launch, IS channels proliferated on the platform, making Telegram one of the organization's main online hubs in recent years.Footnote ⁴⁰ Several major terrorist attacks were linked to engagement with IS recruiters on Telegram, including the attacks in Paris, Berlin, and Istanbul in 2015, 2016, and 2017.Footnote ⁴¹

To assess whether IS sympathizers on Twitter became interested in Telegram after community engagement events, I ran several additional tests. First, I measured mentions of Telegram in these users’ Twitter communications, by coding each tweet for whether it included the word telegram.Footnote ⁴² I find that about 3 percent of the users mentioned Telegram between 2014 and 2016, and that mentions of the platform became significantly more frequent after 22 September 2015, when Telegram launched its channels. Panel A of Figure 8 shows the frequency of IS sympathizers’ mentions of Telegram on Twitter, and Table 2 shows some examples. It can be seen that much of the discourse on Telegram related to either the opening of new accounts on the platform, or the advertisement of IS Telegram channels.

Figure 8.

Mentions of Telegram in Twitter posts. After Telegram launched its channels service, Islamic State sympathizers in event areas significantly increased their references to this alternative platform.

Table 2.

Examples of English-language tweets mentioning Telegram, posted by Islamic State sympathizers in the United States

Second, I examine whether mentions of Telegram among individuals in event areas were more numerous after community engagement events. Panel B of Figure 8 presents difference-in-differences coefficients, where the dependent variable is an indicator coded 1 if a user mentioned the word telegram in one or more tweets and 0 otherwise. As before, I estimate models for different time windows, ranging from one to four weeks after the events. I find that IS sympathizers close to community engagement activities significantly increased their mentions of Telegram after these events. While the changes were less evident in the first two weeks, they became noticeable after three weeks.

Third, I test whether Twitter usage in event areas went down after Telegram became a viable alternative to Twitter. I counted the number of tweets each user posted in the weeks after community engagement events, and examined whether it differed between users inside and outside event areas after the launch of Telegram's channels on 22 September 2015. After the introduction of IS's Telegram channels, IS sympathizers in event areas posted fewer tweets after community engagement events than those outside the event areas (Figure 9). Coupled with the previous findings, this indicates that they might have migrated to Telegram to avoid detection.

Figure 9.

After the introduction of IS's Telegram's channels, sympathizers close to community engagement activities posted fewer tweets in the aftermath of those events.

Taken together, these results show that on-the-ground activities to counter violent extremism led to changes in the online behavior of IS sympathizers on Twitter. After community engagement events, these users reduced their online expressions of support for IS, took steps to change the appearance of their Twitter profiles, and promoted migration to Telegram, an alternative encrypted platform popular among the group's supporters.