Introduction
Negative positions regarding vaccines pre-date the current coronavirus disease (COVID-19) pandemic. The recent development and availability of COVID-19 vaccines present an opportunity to study how negative dominance can manifest online in the form of vaccine resistance or vaccine hesitancy. As a tenet of crisis risk communication, the concept of negative dominance posits that negative messages “outweigh” positive ones amidst emergent events, and that risk communicators accordingly need to provide a greater number of positive messages than negative ones. Reference Covello, Peters, Wojtecki and Hyde1,Reference Rozin and Royzman2 Examples of COVID-19 vaccine-associated news events that would potentially lend themselves to negative dominance in online discourse include isolated vaccine-related adverse events, such as concerns over Bell’s palsy among vaccine recipients Reference Higgins-Dunn and Lovelace3,Reference Smith-Schoenwalder4 and allergic reactions in recipients in the United States;Reference Fierger and Caldwell5,Reference Wu6 a new mutation of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) emerging in multiple locations around the world and attendant concerns about vaccine resistance Reference Kupferschmidt7; and aftereffects of politicization of vaccine development prior to the 2020 US presidential election. Reference Kelsey8 The literature linking online information to vaccine hesitancy is extensive. Prior research has looked at media coverage to identify vaccine concerns among the public and its impact on vaccine-related beliefs and behaviors, Reference Larson, Smith and Paterson9–Reference Smith, Ellenberg, Bell and Rubin12 the spread of misinformation and fake news on the Internet, Reference Carrieri, Madio and Principe13–Reference Hansen and Schmidtblaicher15 and the role of social media in aiding vaccine hesitancy, Reference Salvador Casara, Suitner and Bettinsoli16–Reference Mo Jang, Mckeevera, Mckeevera and Kyoung Kima20 among others. Surprisingly, however, research to date has yet to explicitly explore negative dominance of vaccine-related information online using more recently developed tools for analyzing big data. In fact, for being such a widely accepted model of risk communication, there are no studies to our knowledge applying the model of negative dominance to analyze vaccine hesitancy. This study aims to address this gap in the research literature.
Per Covello’s formulation of negative dominance in the context of risk communication, “negative” content (eg, failures, what is not being done to address a crisis, adverse events) carries greater valence in audiences’ minds than positive, solution-oriented messages. However, the model does not provide a way to measure this greater valence or weight. Several studies have attempted to measure valence and weight of online content outside the negative dominance model. One study categorized Google search terms by the valence of words used in queries to procure vaccine information (eg, “vaccine benefits” as positive, “vaccine risks” as negative, and “vaccine” as neutral). The researchers found that those who used negative search terms (eg, “vaccine risks”) were led to web content that had 3.6 times more vaccine myths per website than those who used neutral terms and 4.8 times more myths than those who used positive search terms (eg, “vaccine benefits”). Reference Ruiz and Bell21 In another study, the researchers quantified the impact of sentiment on diffusion of information by measuring the speed in which a tweet is retweeted for the first time (in seconds) and popularity (in number of retweets and favorites). They found that negative tweets tended to spread faster than positive ones, but that positive tweets outperformed negative ones in terms of popularity. Reference Ferrara and Yang22 Specifically, they found that positive tweets were retweeted 2.5 times more than negative or neutral ones, and favorited 5 times more than negative or neutral ones. However, their study did not deal with the spread of health information during a pandemic, which may have resulted in a different pattern. Karafillakis and colleagues looked at the methods and analyses used in over 80 studies monitoring vaccine conversations on social media. They found that the majority of studies (70%) employed sentiment analysis, or how people feel about vaccines, most often labeling text as “positive,” “negative,” or “neutral” in the user’s stance toward vaccines. Reference Karafillakis, Martin and Simas23 Toxicity analysis is a newer type of analysis that has yet to be applied to this area of research. Toxicity has been defined as “a rude, disrespectful, or unreasonable comment that is likely to make you leave a discussion.” 24–Reference Salminen, Hopf and Chowdhury29 Toxic language is a form of negative content that has a detrimental effect in the quality of online discussions. We believe that controversial topics, such as the debate over vaccines, tend to attract users with extreme views who may use more forceful language, which also may be more toxic than average. We believe that toxicity can be applied within the negative dominance model as a way to measure the valence of the reaction users have to negative vaccine-related content.
To test the negative dominance model for vaccine-related information online, we start by offering a multi-platform approach to measuring the weight of vaccine-related content online: (1) attention given to the topic of vaccines in the form of Google searches, vaccine-related media stories and social media posts; (2) engagement with vaccine-related information online in the form of social media metrics and shares; and (3) toxicity of social media discourse related to vaccines. We aim to test the following hypotheses regarding negative dominance in the context of online activity surrounding COVID-19 vaccination:
H1
If the negative dominance model is applicable to vaccine-related activity online, then the peaks in attention in Google searches, peaks in volume of published media stories, and social media posts will coincide with the incidence of adverse events related to COVID-19 vaccination.
H2
If the negative dominance model is applicable to vaccine-related activity on social media, then social media posts highlighting adverse events related to COVID-19 vaccination will garner more engagement than all other vaccine-related posts occurring within the same time frame.
H3
If the negative dominance model is applicable to vaccine-related activity on social media, then posts referencing adverse events related to COVID-19 vaccination will have a higher average toxicity score when compared with the average toxicity score for all other vaccine-related posts during the same time frame.
Methods
To test our hypotheses, we started by looking at patterns of attention in Google searches for “vaccine” in the United States from December 29, 2019, to January 2, 2021, using Google Trends. Google Trends measures interest over time for search terms based on the volume of Google searches. Google Trends gives the option of searching volume by topic, which it defines as “a group of terms that share the same concept in any language,” or by natural language, which is tied to the language given. 30 We looked at results for both. We paid special attention to peaks in web searches, news searches, image searches, and YouTube searches. We used CrowdTangle and the search term “vaccine” to look at patterns of attention in Facebook, Instagram, and Reddit during the same period of time. CrowdTangle is a public insights tool owned and operated by Facebook that allows researchers to analyze Facebook, Instagram, and Reddit data across time using Boolean search terms. 31 Finally, we used Media Cloud and the search term “vaccine” to look at patterns of attention in media coverage by top US media sources during the same period of time. Media Cloud is an open-source platform that allows researchers to analyze media coverage of a particular topic over time. 32 We focused our search on the US Top Sources collection, which includes 87 newspapers and digital native sources listed by the Pew Research Center in 2019. Like Google Trends, CrowdTangle and Media Cloud are available to researchers free of charge. In the case of CrowdTangle, in particular, researchers are asked to participate in a training webinar before using their services.
We examined and compared the peaks In Google Trends, Facebook, Instagram, Reddit, and Media Cloud data to see if they were driven by negative or adverse events regarding the COVID-19 vaccine. In order to make comparisons across platforms, we calculated a trend score for “vaccine” for all non-Google data by taking the count of interactions for each day on Facebook, Instagram, and Reddit, and dividing that number by the highest total of interactions during the time period for that specific platform. For the Media Cloud data, we calculated the trend score based on the number of stories per day instead of social media interactions. We checked the resulting trend lines against the trend lines plotted using the values in their original scales; we did this to make sure that they followed the exact pattern before comparing across platforms.
Starting with Google Trends, we looked at the related queries coinciding with each of the peak dates to understand what seemed to be driving those peaks. Google Trends defines “rising” related queries as “queries with the biggest increase in search frequency since the last time period. Results marked “Breakout” had a tremendous increase, probably because these queries are new and had few (if any) prior searches.” 30 We made notes of specific events and themes that emerged from our analysis as potential drivers of attention.
We then examined the posts that received the most engagement in Facebook, Instagram, and Reddit during the peak periods identified and compared their influence using social media metrics. CrowdTangle provides the number of interactions for each post as 1 measure of engagement; what counts as interactions, however, vary by platform. For example, the number of interactions for Facebook represents the sum total of number of likes, comments, shares, and the use of other emoticons to indicate sentiment reactions to the post (which CrowdTangle appropriately refers to as “weights” on its dashboard 31), while the number of interactions for Instagram represents the sum total of likes and comments. We made notes of the specific events and themes referenced in the posts.
Finally, we created a list of Boolean search terms to represent some of the events and themes that seemed to have driven peaks in attention: “vaccine AND nurse faints” in reference to the video of the nurse in Tennessee who fainted after receiving the vaccine that went viral; “vaccine AND allergic reaction” due to reports of health workers who experienced allergic reactions to the vaccine; and “vaccine AND Bell’s palsy” (we also used the alternative “Bells palsy” as many posts left out the apostrophe) due to several reports of people who experienced that side effect after getting the vaccine. We used those Boolean search terms to extract vaccine-related posts from Facebook, Instagram, and Reddit, using CrowdTangle. We created and uploaded separate data sets per platform and topic to Communalytic. Reference Gruzd and Mai33 Communalytic is a web-based tool that uses machine-based learning through Google and Jigsaw’s Perspective API to analyze the toxicity of online conversations. It assigns toxicity scores from 0 to 1.0 to posts for over 7 different attributes, including toxicity, severe toxicity, attack on character, insult, and profanity, among others. In their study of antisocial behavior online, Gruzd and colleagues suggested thresholds of > 0.70, > 0.80, and > 0.90 to indicate the probability of high toxicity. Reference Gruzd, Mai and Vahedi34 For the purpose of this study, we focused on the standard toxicity score (as opposed to severe toxicity) and used the > 0.70 threshold to indicate the probability of high toxicity. We took the average toxicity scores for posts referencing adverse events related to the COVID-19 vaccine and compared them with all other vaccine-related posts for each platform.
Results
Figure 1 shows the Google Trends interest over time for “vaccine” in the United States by week, from December 29, 2019, through January 2, 2021. Values range from 0 to 100, with the latter indicating peak popularity for the term. 30 We observed several peaks in the data. The first significant peak occurred in the week of November 8, with news searches for “vaccine” reaching a popularity score of 47. Searches for “vaccine” increased consistently starting on the Wednesday of the Thanksgiving holiday week (November 22) through the first 2 weeks of December, reaching peak popularity (100 score) in the week of December 6 for news searches and the week of December 13 for YouTube searches. We then looked specifically at the period of November 8 through December 31, 2020. We found that peaks in popularity differed based on the type of search: popularity of the search term “vaccine” peaked on December 30 in web searches, on December 12 in news searches, and on December 19 in both image and YouTube searches.
Google Trends interest over time for “vaccine” in the United States by week, from December 29, 2019, through January 2, 2021. Values range from 0 to 100, with the latter indicating peak popularity for the term.
In terms of social media data, we observed differences by platform. Figure 2 shows the volume of interactions on Facebook, Instagram, and Reddit related to the word “vaccine” for English-only posts. The Facebook data include interactions with public Pages, Groups, and Verified Accounts. We observed the following peaks in the data: Monday, November 9 (10.23M interactions), Tuesday, December 8 (7.59M interactions), Monday, December 14 (8.79M interactions), Friday, December 18 (9.67M interactions), and Monday, December 21 (8.09M interactions). The Instagram data showed some similar peaks, but, unlike the Facebook data, “vaccine” on Instagram peaked on Monday December 21 (8.91M interactions). The Reddit data show peaks on November 9 (highest, with 478.4K interactions), December 8, December 18, and December 21. While we observed other peaks earlier in the year, most notably during August 8–11, 2020 (depending on the platform), we found that the greatest volume of vaccine-related activity started on November 8–9 (depending on the platform) and lasted through the end of December 2020. This is something that is shared across all data sources discussed, thus far.
Volume of interactions on Facebook, Instagram, and Reddit related to the word “vaccine” for English-only posts.
We then compared the peaks from Google Trends and our social media data to those trends in the number of published news media stories. Figure 3 shows a similar pattern to the Google Trends and social media data from November through December 2020, with peaks on November 9 (569 stories), December 8 (582 stories), December 14 (671 stories), and December 18 (569 stories). Outside of the months of November and December, the next highest peak occurred on May 18, 2020 (309 stories), which differs from the Google Trends and social media data.
Google Trends and social media data from November through December 2020.
Next, we were interested in understanding what drove the peaks in vaccine-related activity on these platforms. To test our hypothesis regarding negative dominance, and whether or not negative content was driving these peaks in activity, we dove deeper into the data, looking at the specific dates of peak activity for each platform. Figure 4 compares the peaks in vaccine-related activity for all the data. We observed that, except for Facebook and Reddit, which had the highest number of vaccine-related interactions on November 9, 2020, peak dates were different for all other data sources. For Facebook and Reddit, the posts with most interactions referenced Pfizer and BioNTech’s announcement that their vaccine was over 90% effective and that they would seek FDA approval. Vaccine-related searches on Google News peaked on December 12; we observed that references to news stories reporting adverse or negative reactions to the vaccine dominated the rising related Google News search queries on that date. According to Media Cloud, the number of media stories by US Top Sources was highest on December 14, when the first doses of the COVID-19 vaccine were administered in the United States. Google Image and YouTube vaccine-related searches peaked on December 19. We observed that the top rising related queries for that date referenced the nurse who fainted after receiving the COVID-19 vaccine (YouTube) and facial paralysis (Image search). Vaccine-related activity on Instagram peaked on December 21; however, the top posts did not focus on adverse effects but rather on political commentary and/or conspiracy theories. The peak in vaccine-related Google Web searches on December 30 also was not focused on adverse effects but rather on vaccine distribution at different locations, specifically in Florida, with the top rising related query being, “How long does the COVID vaccine last?”
Comparison of peaks in vaccine-related activity for all the data.
Figure 5 compares the interest over time for “Pfizer vaccine,” “nurse vaccine,” “Bell’s palsy” (topic), “side effects Covid vaccine,” and “Moderna vaccine” from November 8 through December 31, 2020. The Bell’s palsy topic “includes terms that share the same concept in any language,” beyond the natural language of “Bell’s palsy” (for example, facial nerve paralysis). 30 Overall, “Pfizer vaccine” and “Moderna vaccine” together outranked other terms in web searches and news searches, except on December 11, when “Bell’s palsy” outranked both in web search (this is after Pfizer and BioNTech released their FDA briefing documenting 4 cases of Bell’s palsy from the trials Reference Rahhal35 and the media coverage that followed). “Bell’s palsy” dominated the image search on December 12 and again on December 28, after a video of a nurse from Tennessee alleging that she developed Bell’s palsy from the vaccine was posted on YouTube on December 26 (the video has since been flagged as misinformation Reference Funke36). “Nurse vaccine” dominated YouTube searches, specifically on December 18–19 after another Tennessee nurse, Tiffany Dover, fainted on camera after receiving the vaccine. The video from Inside Edition went viral on YouTube 37 and rumors followed that she had died afterward (also not true 38). In sum, H1 was partially supported. While adverse events seemed to have driven the peaks in Google Image and YouTube vaccine-related searches, that was not the case for Google Web and news searches or for social media, where the highest peak for 2 out of the 3 platforms was driven by the announcement of the Pfizer/BioNTech vaccine, a positive development in the fight against the pandemic.
Comparison of interest over time for “Pfizer vaccine,” “nurse vaccine,” “Bell’s palsy” (topic), “side effects COVID vaccine,” and “Moderna vaccine” from November 8 through December 31, 2020.
In terms of engagement, Facebook, Instagram, and Reddit, along with other social media platforms, have been aggressively countering misinformation related to COVID-19 since March 2020. Reference Shu and Shieber39 We looked at the number of interactions for posts including the terms “vaccine” AND “allergic reaction,” “nurse faints,” and “Bell’s/Bells palsy” and compared them vis-à-vis the total number of all “vaccine” posts and interactions. Posts highlighting adverse reactions to the vaccine started peaking on December 8–9, depending on the topic and platform (“vaccine AND Bell’s Palsy” started rising on December 8 on Facebook, while “vaccine allergic reaction” started peaking on December 9 in all platforms, and interactions with “vaccine nurse faints” concentrated on December 17–18). We focused on the period from December 8–31, 2020, to test for negative dominance of posts highlighting adverse reactions. While the posts highlighting adverse reactions to the vaccine were a small portion of all “vaccine” tweets combined during this time period, we observed that on dates when one of the adverse reaction topics peaked, the ratio of number of posts to interactions surpassed that for all “vaccine” posts. Figure 6 shows the volume of interactions for “allergic reaction,” “nurse faints,” and “Bell’s/Bells palsy” on social media as a percentage of total interactions from all vaccine-related content. As evident in the figure, the amount of engagement with posts focused on these adverse effects was almost imperceptible when compared with all vaccine-related engagement, disproving negative dominance on these platforms. Posts about allergic reaction(s) to the vaccine, which had the most interactions among all the adverse effects posts, amounted to less than 1% of all total vaccine-related engagement. In sum, H2 was not supported.
Volume of interactions for “allergic reaction,” “nurse faints,” and “Bell’s/Bells palsy” on social media as a percentage of total interactions from all vaccine-related content.
In terms of media reporting, we observed a shift in media reporting on December 9, when news broke of the Alaskan health workers who suffered allergic reactions to the vaccine. Prior to that, media stories with negative vaccine-related content focused on general “side effects”; on and after December 9, stories began focusing on “allergic reaction” to the vaccine. While media stories focusing on vaccine side effects represented just 1% of all media stories with “vaccine” in the title published from November 8 through December 8 (N = 5766), media stories focused on allergic reactions to the vaccine represented 3.4% of all media stories with “vaccine” in the title published from December 9 through December 29 (N = 4756). Since Media Cloud provides the number of Facebook shares for each story, we compared the volume of shares for stories focusing on side effects to those focusing on allergic reactions to the vaccine. We found that the former represented 11.6% of all Facebook shares of media stories with “vaccine” in the title, while the latter represented 17.6%. While these numbers fail to prove negative dominance, they do show an increased level of engagement with negative or adverse effects content than what the social media data showed.
Finally, to test H3, we analyzed the social media posts for toxicity using Communalytic. Table 1 shows average toxicity scores for the top vaccine-related posts on Facebook, Instagram, and Reddit. Most vaccine-related posts on Facebook, Instagram, and Reddit had low toxicity scores (≤ 0.30). This is not surprising given the efforts by these platforms to curb hateful speech and misinformation through monitoring and moderation. We found similar results for posts highlighting adverse events (ie, allergic reaction, Bell’s palsy, nurse faints after receiving the vaccine), with over 85% of posts scoring 0.30 or less. We observed that many of these posts were informational in tone. These findings fail to prove negative dominance for vaccine-related content in terms of the probability of finding high toxicity in the Facebook, Instagram, and Reddit posts that we analyzed. Therefore, H3 was not supported.
Toxicity scores for vaccine-related posts on social media (December 8–31, 2020)
Overall, we did not find evidence of negative dominance for vaccine-related content in our study. This is not to say that there is no negativity in online content surrounding the COVID-19 vaccine, but rather that we did not find a significant amount of it in our data at the threshold levels that we would expect for negative dominance to operate (> 0.70). Reference Gruzd, Mai and Vahedi34 One possible explanation for this is that the social media platforms’ (ie, Facebook, Instagram, and Reddit) moderation efforts aimed at curbing misinformation and monitoring hateful speech Reference Shu and Shieber39 may be having the added effect of thwarting negative dominance of vaccine-related information. An alternative and related explanation could be that the users who have more intense opinions about this matter are self-monitoring and/or not being as outspoken for fear of being deplatformed, Reference Hatmaker40 unfriended, and unfollowed. Yet another explanation could be that these users have moved to other platforms not analyzed in our study where they may be expressing their views. Reference Parler41 We would need to conduct further research to be able to substantiate any of these claims. Google Trends data may suggest negative dominance for image and YouTube searches (ie, searches for images linking the vaccine to Bell’s palsy and the video of the nurse fainting after receiving the COVID-19 vaccine). The Media Cloud analysis points to the episodic nature of news and information – that is, the nature of news media to address a topic or issue and then move on to the next news item on the editorial agenda. Social media can reflect this temporal nature of news flow or it can attempt to sustain or build on it. One of the ways that users of social media express their particular position is to attract more interactions by using toxic language, but we did not see evidence to back this up on the social media platforms that we studied.
Our study has several limitations. First, our analysis is limited to 3 specific types of adverse events (ie, allergic reactions to the vaccine, Bell’s palsy, and the incident of the nurse fainting on camera after receiving the vaccine); it is not inclusive of all negative events regarding the COVID-19 vaccine that may have occurred, like for example stories about new strains and vaccine effectiveness. Additionally, as noted by the documentation for the Perspective API, toxicity scores should not be interpreted as actual degree of toxicity but rather as the probability or likelihood that a reader would interpret the post as toxic, not of severity. 42 As with many automated tools, there is a degree of measurement error when using the Perspective API. 43 Studies have cited the occurrence of false positives Reference Awal, Cao, Mitrovic and Lee44 and the inability of automation to interpret words in context and detect sarcasm, for example, or other nuances in language. Reference Gruzd, Mai and Vahedi34 To address this limitation, Google and Jigsaw have crowdsourced human raters for samples of large data sets to improve the process of training data 45 and provide users with the ability to suggest comment scores. 46 They also use fairness indicators to help mitigate unintended bias. 47 Future research focused on testing the validity of the measure should consider employing human coders to validate scores assigned by the Perspective API to a sample of the data. When it comes to Google Trends data as well, the results are based on a sample of the full population of search requests, which inevitably also introduces some degree of error. 48
In addition, while our analysis includes mention of the keywords within the body text of news stories, our social media analysis is limited to users’ posts and does not include users’ comments. This is one of the limitations of CrowdTangle, which allows researchers to search and collect posts but not users’ comments. We believe that our results, in particular, toxicity scores, might be different if we were able to analyze the comments left by users to vaccine-related posts.
Finally, our study is a descriptive study using social media data, and, as such, we caution against making generalizations based on our findings. Reference Olteanu, Castillo, Diaz and Kiciman49 For one, platforms tend to attract certain demographics (some are overrepresented/underrepresented or entirely absent from some of the platforms). Reference Karafillakis, Martin and Simas23,Reference Olteanu, Castillo, Diaz and Kiciman49 There is also a problem of user access –whether because of socioeconomic background, age, or other reasons – that can make generalizations unfeasible. Further research can help demonstrate methodologically how to navigate some of the challenges inherent in working with social media data.
Conclusion
What we have described in this research article is a multi-platform approach to monitor the COVID-19 infodemic online. Our approach is based on trends data that look at the appearance and flow of news and information around vaccines through mainstream news media and compare them to online search activity, posting behavior and engagement on social media platforms. What has become clear through this approach is that the temporal nature of news and information corresponds to the appearance of “hot button” issues related to the vaccine. We suggest that governments and health organizations on the local, state, or national level employ a multi-platform approach to monitor both mainstream and alternative news media for trends with a particular focus, in this case, on those related to the COVID-19 vaccines. Through monitoring trends in search, news and social media posts, governments, as well as health organizations can better plan to deal with issues supporting vaccine acceptance and defending against vaccine resistance.
Conflict(s) of Interest
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this paper.
