Introduction
In fall 2017, California was hit by the largest wildfires in the state’s history at that time. Reference Di Liberto1 Due to the rapid spread of fires and poor air quality, officials recommended that schools in the affected counties close on days not on their original calendar. Reference Boisrond2 Across the United States, unplanned school closures (USCs) are common due to natural disasters and inclement weather. Reference Wong, Shi and Gao3 USC monitoring enables public health agencies to respond to and prepare for future emergencies.
Researchers at the Centers for Disease Control and Prevention (CDC) have identified USC announcements using online systematic searches (OSS), which include daily scans of Google Alert, Google News, and LexisNexis. Reference Wong, Shi and Gao3 Schools and school districts often announce USCs on social media, which are not detected by OSS. Prior studies have shown that schools and their districts use Twitter and Facebook to make USC announcements. Reference Jackson, Mullican and Tse4–Reference Jackson and Ahmed6
As summarized elsewhere, Reference Muniz-Rodriguez, Ofori and Bayliss7,Reference Finch, Snook and Duke8 information dissemination is 1 of the 4 major uses of social media applications during natural disasters, which the findings of this study further evaluated. Coupling with Geographic Information Systems, social media may provide useful spatiotemporal information pertinent to the response of natural disasters by federal government agencies, such as the CDC. Reference Wang and Ye9
This cross-sectional analysis aims to demonstrate the complementary value of using Facebook and Twitter alongside OSS to retrieve USC announcements by schools and school districts. The counties in California impacted by the wildfires in October and December 2017 served as the study area. Analysis of public-school districts (Figure 1) and private schools (Figure 2) is presented in the main text. Analysis of individual public schools (Figure 3) is presented in the Online Supplementary Materials.
Unplanned school closure (USC) data pertinent to public-school districts (combined for both northern and southern California in the 2 respective wildfires in October and December 2017) with: (a) USC announcements identified; (b) social media accounts identified; (c) USC announcements identified among districts with social media accounts; (d) USC announcements identified via OSS among districts without social media accounts; and (e) USC announcements among districts with both Facebook and Twitter accounts.
Unplanned school closure (USC) data pertinent to private schools (combined for both northern and southern California in the 2 respective wildfires in October and December 2017) with: (a) USC announcements identified; (b) social media accounts identified; (c) USC announcements identified among private schools with social media accounts; (d) USC announcements identified via OSS among private schools without social media accounts; and (e) USC announcements among private schools with both Facebook and Twitter accounts.
Unplanned school closure (USC) data pertinent to public schools (combined for both northern and southern California in the 2 respective wildfires in October and December 2017): with (a) USC announcements identified; (b) social media accounts identified; (c) USC announcements identified among public schools in districts with social media accounts; (d) USC announcements identified via OSS among public schools in districts without social media accounts; (e) USC announcements among public schools in districts with both Facebook and Twitter accounts.
Methods
Data Collection and Manipulation
Counties in California affected by the wildfires in October and December 2017 were identified on the CAL FIRE website; all fires and counties listed on the website were considered in this study. 10 Data on public schools, public-school districts, and private schools in the affected counties were retrieved from the National Center for Education Statistics (NCES) website. 11 Variables included locality, student population, and student/teacher ratio. In addition, free/reduced lunch proportion is a variable for the individual public schools.
Facebook and Twitter accounts owned by the private schools and public-school districts were identified using Google, Twitter, and Facebook search functions. Private schools and public-school districts that had posted at least once on their respective accounts were classified as having coverage. USCs due to the wildfires and announced on Twitter and Facebook were identified by reading all posts released from October 2, 2017, when the first fire (Timm Fire) started in January 9, 2018, to when the last fire (Holiday Fire) was extinguished.
Previous studies have shown that public-school districts often make announcements on behalf of the individual public schools. Reference Jackson, Mullican and Tse4,Reference Ahweyevu, Chukwudebe and Buchanan5 For this project, Twitter and Facebook coverage information for the public-school districts was used as a proxy for individual public schools. Therefore, all public schools under districts with active coverage were coded as having active Facebook or Twitter accounts. For the purposes of this study, charter schools were coded as public schools. According to the NCES, charter schools are publicly funded schools with a legislative charter exempting them from certain rules and regulations put in place by the district or other entities. We acknowledge that charter schools may operate with their own rules and regulations outside district guidelines. 11 USC data on the wildfires retrieved via OSS by CDC researchers were released to the authors and were appropriately coded into all the data files as a new variable. The Online Supplementary Materials provide details that are not presented in the main text.
Statistical Analysis
Statistical analyses were performed using R version 3.4.2 (R Development Core Team, Vienna, Austria) and RStudio version 1.1.383 (RStudio Team, Boston, MA). A test for normality was performed on all the continuous variables of school characteristics, and non-parametric tests were used for variables that were not normally distributed. To compare USCs identified on Facebook and Twitter to those identified by OSS: school locale, total student population, student-teacher ratio, proportion of students receiving free/reduced-priced lunch (public school only) of schools with at least 1 wildfire-related USC announcement were summarized by frequencies. These descriptive statistics were then compared using a chi-square test for frequencies, the Welch’s t-test (2-sample) for normally distributed sample means, or a Wilcoxon signed-rank test for sample medians that were not normally distributed. Univariate and multivariable logistic regressions were performed to determine whether the characteristics mentioned above were associated with having active Facebook and/or Twitter account(s), and having USC announcements identified via Facebook, Twitter, and/or OSS. With the “sjstat” package in R, the crude and adjusted odds ratios were converted to the crude and adjusted relative risks (aRR) for better interpretation. Statistical significance was determined at α = 0.05 level.
Results
Public-School Districts
Of the 362 public-school districts, 290 were found to have active social media accounts. Among these, 53 (18%) had Facebook accounts only, 60 (21%) had Twitter accounts only, and 177 (61%) had both. We identified the USC announcements of 115 (32%) districts, of which OSS identified 104 (90%), Facebook, 58 (50%), and Twitter, 37 (32%). Forty-four (38%) announcements were identified by OSS alone, 1 (1%) by Facebook only, and 7 (6%) by Twitter only. Thirty-three (29%) announcements were identified on both OSS and Facebook, 5 (4%) on both OSS and Twitter, 3 (3%) on both Facebook and Twitter, and 21 (19%) via all 3 methods (Figure 1a, 1b, 1c). Among the 72 public-school districts without social media accounts, 21 (29%) had an announcement identified by OSS (see Figure 1d).
Among the 362 public school districts, 58 out of the 230 districts with Facebook accounts had wildfire-related Facebook USC announcements (Tables 1, 2). Of the 58 announcements identified on Facebook, 1 (2%) was on Facebook only, 3 (5%) were on Facebook and Twitter, 33 (57%) were on Facebook as well as identified by OSS, and 21 (36%) were identified by all 3 methods (see Figures 1a, 1b, 1c). The percentage of schools with Facebook USC announcements varied by geographic location: 34% (28/58) city, 33% (19/58) suburb, 12% (7/58) town, and 21% (12/58) rural. After controlling for other factors, rural public-school districts were 45% less likely than city districts to have active Facebook accounts (aRR = 0.55; 95% CI: 0.30, 0.86, P < 0.01) (see Table 1). Suburban and town public-school districts were 72% and 61%, respectively, less likely than city districts to have Facebook USC announcements, after adjusting for other factors (suburb: aRR = 0.28, 95% CI: 0.13, 0.57, P < 0.01; town: aRR = 0.39, 95% CI: 0.13, 0.98, P = 0.05) (see Table 2).
Adjusted relative risk of districts with active Facebook accounts (N = 362, n = 230) and Twitter accounts (N = 362, n = 237) among all public-school districts in our sample
Adjusted relative risk of announcements on Facebook among public-school districts with Facebook accounts in our sample (N = 230, n = 58)
Of the 362 public school districts, 237 had active Twitter accounts of which 37 made a USC announcement. Seven (19%) announcements were on Twitter only, 3 (8%) were on Facebook and Twitter, 6 (16%) were on Twitter as well as identified by OSS, and 21 (57%) were on all 3 (see Figure 1a, 1b, 1c). The number of districts per locality with active Twitter USC-announcements—city, 16 (43%); suburb, 17 (45%); town, 2 (6%); rural, 2 (6%)—were significantly lower than those without: city, 42 (21%); suburb, 128 (64%); town, 9 (5%); rural, 21 (11%); P = 0.03. Compared to city districts, town and rural districts were 66% and 38%, respectively, less likely to have Twitter accounts, after adjusting for other factors (town: aRR = 0.34, 95% CI: 0.15, 0.64, P = 0.02; rural: aRR = 0.62, 95% CI: 0.34, 0.93, P < 0.01) (see Table 1). Regression was not performed as there was no statistically significant association observed between having Twitter accounts and student population and student/teacher ratio.
Facebook and Twitter
We identified 177 districts with both Facebook and Twitter accounts, of which 53 (30%) had made a USC announcement (see Figure 1e). Thirty-seven (70%) and 35 (66%) announcements were made on Facebook and Twitter, respectively, and 44 (83%) were identified by OSS (see Figure 1e). Of those announcements, 0% (0/53) were on Facebook only, 11% (6/53) were on Twitter only, and 9% (5/53) were identified by OSS only; 25% (13/53) were found on Facebook and identified by OSS, 9% (5/53) were found on Twitter and identified by OSS. Six percent (3/53) of the announcements were on both Facebook and Twitter; 40% (21/53) were identified by all 3 methods. Districts with Facebook and Twitter accounts were 73% and 65%, respectively, less likely to be located in town and rural areas than in cities, after adjusting for other factors (town: aRR = 0.27, 95% CI: 0.11, 0.57, P < 0.01; rural: aRR = 0.35, 95% CI: 0.16, 0.66, P < 0.01) (Table 3).
Adjusted relative risk of districts with active Facebook and Twitter accounts among all public-school districts in our sample (N = 362, n = 177)
Maps
Figures S1, S2, and S3 show that there are more public-school districts in southern California counties observed than in the north. Figure S1 identifies that in the south there are more public-school districts with Facebook and/or Twitter coverage than in the northern districts; but, according to Figure S2, there is a similarity in the number of districts that made an announcement on Facebook and/or Twitter accounts in the counties observed. Districts on the western coast of California had more USC announcements identified by OSS than those farther inland, as seen in Figure S3.
Results for the individual public schools are in the Online Supplementary Materials (Figure 3).
Private Schools
Of the 1289 private schools, 878 were found to have active social media accounts. Among these, 681 (78%) had Facebook only, 32 (4%) had Twitter only, and 165 (19%) had Facebook and Twitter. We identified the USC announcements of 104 (12%) private schools, of which OSS identified 47 (45%), Facebook, 67 (64%), and Twitter, 29 (28%). Thirty (29%) were identified by OSS alone, 35 (34%) by Facebook only, and 7 (7%) by Twitter only. There were 10 (9%) identified on both OSS and Facebook, 0 (0%) on both OSS and Twitter, 15 (14%) on both Facebook and Twitter, and 7 (7%) via Facebook, Twitter, and OSS (Figure 2a, 2b, 2c). Among the 411 private schools without social media accounts, 21 (5%) had an announcement identified by OSS (see Figure 2d).
Of the 1289 private schools in wildfires-affected counties, 846 had active Facebook accounts, of which 67 had an announcement. Of these 67 schools, 35 (52%) announcements were on Facebook only, 15 (22%) were on Facebook and Twitter, and 10 (15%) were on Facebook as well as identified by OSS (see Figures 2a, 2b, 2c). Geographic location varied for schools with Facebook announcements: 54% (36/67) were in cities, 27% (18/67) were in the suburbs, 10% (7/67) in towns, and 9% (6/67) in rural areas. For every increase of 1000 students in the total student population, there was a 10% increase in risk for private schools with active Facebook accounts, after adjusting for other factors (aRR = 1.1, 95% CI: 1.0, 1.1, P < 0.01) (Table 4).
Adjusted relative risk of schools with active Facebook accounts among all private schools in our sample (N = 1,289, n = 846)
Online systematic search (OSS)
Of the 1289 private schools, 47 (4%) had USC announcements identified by OSS. Among these, 30 (64%) were on OSS alone, 10 (21%) announcements were also found on Facebook and OSS (but not Twitter), 7 (15%) were also on Facebook, Twitter, and identified by OSS. Of the 878 private schools that had social media accounts, 122 had USC announcements, 26 (21%) of which had an announcement identified by OSS. Of these 26 schools, 9 (35%) schools’ announcements were identified on OSS alone, 10 (38%) on Facebook and OSS, and 7 (27%) on Facebook, Twitter, and OSS (see Figure 2a; see Figure 2c). Of 411 private schools without Facebook or Twitter accounts, 21 were identified with USC announcements by OSS (see Figure 2d). Private schools were 9 times and 4 times more likely to have made announcements captured by OSS if they were located in towns and rural areas, respectively, than in cities (town: aRR = 10, 95% CI: 4.9, 17, P < 0.01; rural: aRR = 5.1, 95% CI: 0.2.0, 10, P < 0.01), after adjusting for other factors. For every increase of 1000 students in the total student population, there was a 20% increase in risk for private schools that made an announcement captured by OSS, after adjusting for other factors (aRR = 1.2, 95% CI: 1.1, 1.3, P < 0.01) (Table 5).
Adjusted odds ratio of announcements captured by OSS among all private schools in our sample (N = 1,289, n = 47)
Facebook and Twitter
Of the 165 private schools with Facebook and Twitter accounts, 36 (22%) were identified with USC announcements (see Figure 2e).
Maps
Figures S4, S5, and S6 show that there are more private schools located in the observed southwestern counties of California. Many of the schools had Facebook and/or Twitter accounts (Figure S4), but very few of these schools used their Facebook and/or Twitter accounts to make a USC announcement (Figure S5). Figure S6 demonstrates that most private schools did not have a USC announcement identified by OSS, but those that did have one were mostly in the northern counties observed.
Discussion
In this case study of October–December 2017 fires in California, we have demonstrated that social media methods are complementary to OSS in USC monitoring. We highlighted the additional number of announcements identified via Facebook and Twitter as a proportion of the cases identified using OSS: 10.6% (11/104) additional public-school districts or 23.0% (166/722) additional public schools, and 121.3% (57/47) additional private schools with USC announcements were identified. Equally important is to note that OSS identified USCs of 44 public school districts (with 165 schools) and 30 private schools that were unidentified via Facebook or Twitter.
Since 2011, OSS has been used by CDC researchers to identify USCs. Reference Wong, Shi and Gao3 In recent years, CDC researchers used both OSS and social media to retrieve USC information as seen in prior studies of Michigan, Reference Jackson, Mullican and Tse4 Georgia, Reference Ahweyevu, Chukwudebe and Buchanan5 and the southern states affected by Hurricane Harvey in 2017. Reference Jackson and Ahmed6 This study provides further evidence of the utility of using social media to retrieve USC information by CDC researchers.
Our study demonstrated that schools use social media to communicate USCs with their students during emergency situations. Reviews of the literature found that social media has been used by institutions to create situational awareness and to quickly disseminate information during emergencies. Reference Muniz-Rodriguez, Ofori and Bayliss7,Reference Finch, Snook and Duke8 Meanwhile, public school districts, individual public schools, and private schools had more announcements on Facebook than on Twitter. This provides evidence that, among schools in our sample, schools are using both social media platforms to communicate, and that Facebook has been more commonly used.
Public school districts, public schools, and private schools located in cities most commonly had an announcement on Facebook, and those in the suburbs had the most announcements on Twitter. Individual public school and private school USCs identified by OSS were more likely to be in towns and rural areas, respectively (see Tables 5 and S5). This is consistent with prior studies, in which locality has been found to be a predictor of USCs in Michigan and Georgia being identified by OSS or social media. Reference Jackson, Mullican and Tse4,Reference Ahweyevu, Chukwudebe and Buchanan5 Therefore, further research among schools in different states will help demonstrate the external validity of our study.
Based on Figures S1–S6, a majority of the public-school districts and private schools were consistently located in the southern counties that were observed, and along the Pacific Coast. We can also see that the quantity of announcements for public-school districts and private schools is similar in the northern and southern counties. Figures S2, S3, S5, and S6 show that there are USC announcements in counties that were made by Facebook and/or Twitter that were not captured by OSS.
There are limitations to this study. First, all data for Facebook and Twitter were manually retrieved by the authors. Second, there was also no gold standard to determine whether the schools that made an announcement officially closed. Thus, this study only focuses at observing the phenomenon of USCs as in prior studies using OSS data. Reference Wong, Shi and Gao3–Reference Jackson and Ahmed6 Third, we did not attempt to identify state-level data from the California Department of Education, just as CDC researchers did not contact state governments for data in a recent study. Reference Jackson and Ahmed6 Fourth, in this study we were not able to explore the proximity analysis for how USC decisions could be made; viewing the correction between burn area or air quality and USCs may be beneficial in future studies. Fifth, it is also important to note that in the NCES private school and public-school district data set, there are no data pertinent to the free or reduced-priced lunch variable.
Future studies may evaluate the accuracy of what is found with the collected social media data and identify gaps found in social media. For a radar of school closures, future research may further study the locational coverage of social media and the impact of social media use in different geographic areas. A comparison in performance between direct monitoring of social media platforms and the use of social media aggregators (eg, HootsuiteTM) may provide additional insights into the feasibility and utility of social media monitoring. Additionally, future researchers may consider adding practice recommendations specific to emergency responses relevant to public health in their study.
The findings of this study and other similar studies Reference Jackson, Mullican and Tse4–Reference Muniz-Rodriguez, Ofori and Bayliss7 hint at the potential utility of harvesting social media and other online data from schools or other agencies. The actions resulting from such data harvesting would depend on the nature of the emergency and the responsibilities of the agencies in charge. For instance, as suggested by Jackson and Ahmed, Reference Jackson and Ahmed6 harvesting school closure data can help inform pandemic preparedness planning and response in the CDC and other public health agencies. The CDC Community Interventions for Infection Control Unit used OSS of news sources to monitor USCs and verify them through searching for the announcements on schools’ websites or social media posts. Reference Jackson and Ahmed6
Conclusion
In conclusion, 62% of public-school districts, 80% of individual public schools, and 71% of private schools had Facebook and/or Twitter coverage. We identified 11, 166, and 57 announcements on Facebook and/or Twitter for public school districts, individual public schools, and private schools, respectively, that were not captured by OSS. Our study demonstrates that Facebook and Twitter can be used complementary to OSS to identify USC announcements by CDC researchers. Social media have previously been advocated as a means to create situational awareness and as a means of risk communication. Reference Muniz-Rodriguez, Ofori and Bayliss7,Reference Finch, Snook and Duke8 Social media can serve as a method to quickly disseminate information pertaining to emergency responses. Our data show that with Facebook and Twitter data, we can create a more comprehensive database when used with OSS to identify USCs to facilitate CDC emergency preparedness and responses.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/dmp.2022.32
Acknowledgments
The authors thank Dr Amra Uzicanin, Dr Hongjiang Gao, Ms Nicole Zviedrite, and Ms Ashley Jackson of the Community Interventions for Infection Control Unit of the Division of Global Migration and Quarantine, National Center of Emerging and Zoonotic Infectious Diseases (NCEZID), CDC, for sharing with us the OSS USC data. BMB and NPC thank the Health Economics and Modeling Unit of the Division of Preparedness and Emerging Infections (DPEI), NCEZID, CDC, for hosting their practicum in the summer of 2018, in which part of this project was conducted. The student authors thank Ms Jennifer Ahweyevu for her inputs upon consultation. The authors also want to acknowledge the institutional and moral support provided by the late Dr Toby Merlin, Director of DPEI, whose premature death saddened all of us who knew him.
Funding statement
ICHF acknowledges support from the CDC (18IPA1808820; 19IPA1908208).
Conflict(s) of interest
We have no conflicts of interest to declare.
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
Ethical standards
This project has been approved by the Institutional Review Board (IRB) of the corresponding author’s university (H15083) and was determined to be exempt from a full IRB review.
