Studies have identified college campuses as uniquely vulnerable to both natural and manufactured disasters and emergencies.Reference Fifolt, Burrowes and McPherson1-Reference Zdziarski, Dunkel and Rollo5 With United States (US) enrollment expected to reach 21,410,000 in 2018, maintaining the safety of postsecondary institutions is a principal concern.Reference Snyder, de Brey and Dillow6 The Clery Act of 1990 requires all federally funded US colleges and universities to provide security reports, safety protocols, and communication channels for disseminating required warnings and emergency notifications to mitigate harm.7-9 Both present and past large-scale campus evacuations and public safety threats on school campuses due to natural disasters, accidents, and campus violence, have drawn attention to the need for further improvements in emergency systems, and that emergency planning and crisis communications need to maintain control and reduce panic in these situations.Reference Kapucu, Berman and Wang2,Reference Zdziarski, Dunkel and Rollo5,Reference Coombs10-Reference Catullo, Walker and Floyd13
STATEMENT OF THE PROBLEM
Much of the current institutional emergency preparedness research is focused in general preparedness, management, and resiliencyReference Fulmer, Portelli and Foltin14-Reference Wu, Qu and Preece19; enrollment into communication systemsReference Han, Ada and Sharman15,Reference Johnson16,Reference Wu, Qu and Preece19-Reference Lee, Chung and Kim21 ; and the assessment of system effectiveness.Reference Schneider22-Reference Sherman-Morris24 These studies highlight that important systematic changes are needed; however, in light of the potential severity of campus emergencies, some of the greatest consequences may come from students failing to respond to these protocols and messages.Reference Kopel, Sims and Chin25,Reference Wang and Kapucu26 Simply getting a message to stakeholders does not ensure that they take action after receiving the message, and past college and university emergencies have shown that quick action can be the difference between life and death, with one of the greatest delays to compliance being message verification.Reference Han, Ada and Sharman20,Reference Gulum and Murray27 Overall, scant research has addressed compliance to emergency notifications,Reference Dow and Cutter28-Reference Sorensen32 and even less describe rapid compliance in college and university settings. To date, only 1 study has focused on assessing theoretical factors related to immediate compliance with emergency notification system (ENS) messages on college campuses.Reference Han, Ada and Sharman20
The current study seeks to add to the limited research regarding student perceptions of emergency notifications and identify factors associated with rapid compliance with emergency notifications in a diverse student population on a range of campus hazards. The 3 study aims are: (1) examine student perception of a current notification system and messages, (2) identify factors associated with rapid message compliance, and (3) explore theory as a predictive tool to determine compliance and identify possible intervention considerations.
THEORETICAL MODEL
Assessing rapid compliance during an actual emergency is difficult; however, prior research has identified that behavioral intentions can be a useful predictor.Reference Paton, Smith and Johnston33,Reference Shaw, Shiwaku and Kobayashi34 The Theory of Planned Behavior (TPB) is a commonly used model assessing intention to act and asserts that actual behavior is a function of behavioral intention.Reference Ajzen35-Reference Ajzen, Kuhl and Beckmann37 The TPB model identifies 3 main constructs: behavioral beliefs, normative beliefs, and perceived behavioral control.Reference Ajzen35 The construct of behavioral beliefs/attitudes is developed through the subjective consequences of the behavior, and normative beliefs/subjective norms are developed through beliefs and motivation to meet expectations of others.Reference Ajzen35 The final construct, perceived behavioral control, captures a person’s perception of how difficult the action will be and is a proxy measure of self-efficacy and volition of control.Reference Ajzen35 Research suggests that the TPB is 1 of the most influential theories in disaster and emergency preparedness planningReference Paton, Smith and Johnston33,Reference Ejeta, Ardalan and Paton38 and has been used to assess topics such as disaster preparedness,Reference Najafi, Ardalan and Akbarisari39,Reference Paek, Hilyard and Freimuth40 flooding,Reference Terpstra and Lindell41 and earthquakes.Reference Nakagawa and Yamamoto42 Fear, as defined in the Protection Motivation Theory (PMT), may also play a role in explaining behavior.Reference Maddux and Rogers43-Reference Prentice-Dunn and Rogers46 A study on fear-arousing communications found that, as fear increases, so does the intention to take action.Reference Sutton47 The PMT explains this through 2 main constructs. The threat-appraisal process engages intrinsic and extrinsic response rewards and the perceived severity and vulnerability of threat.Reference Floyd, Prentice-Dunn and Rogers45 The coping-appraisal process is derived from response efficacy (the belief that taking the action/behavior will work), self-efficacy (the belief that someone can do the action/behavior), and response costs.Reference Maddux and Rogers43 The PMT has been used to assess intentions in disaster/emergency research topics such as earthquakeReference Mulilis and Lippa48 and flood preparedness.Reference Grothmann and Reusswig49
METHODS
This cross-sectional study took place between March 2017 and February 2018 at California State University, Northridge (CSUN), a large diverse, Hispanic, Asian American, Native American, and Pacific Islander serving institution with more than 40,000 students and over 2000 faculty.50,51 CSUN provides emergency notifications disseminated through a multimodal notification system that sends messages by email, text, campus phone, personal phone, and campus computers, as well as secondary distribution through campus outdoor public broadcasting system, marque systems, websites, and social media systems.52
To assess student intention to rapidly comply in an emergency, a 28-question electronic survey instrument was designed and approved by the CSUN Institutional Review Board (IRB) for use in this population. The instrument was pilot tested and revised for clarity, consistency, and understandability. Demographic questions including race/ethnicity, age, year in school, and residence, as well as questions related to student perceptions, preferences, intentions, and a theoretical assessment were developed based on previous studies, as well as collaborations with in-field experts, campus police services, and emergency managers. Age and school attendance were continuous measures, with age calculated in years and attendance in semesters. Sex and race/ethnicity items were “female,” “male,” or “other” and “American Indian, Native American or Alaska Native,” “Asian,” “Black/ African /African-American,” “Caucasian/non-Hispanic White,” “Hispanic/Latino/Latina,” “Native Hawaiian or Other Pacific Islander,” and “Other.” Due to low numbers, gender was dichotomized to “female” and “male” and race/ethnicity was reduced to “Caucasian/non-Hispanic White,” “Asian/Pacific Islander,” “Black/African/African-American,” “Hispanic/Latino/Latina,” and “Multiracial/Other,” with “Caucasian/non-Hispanic White” as the reference group.
Student class level was measured as “graduate” or “undergraduate” and campus residence was assessed by asking whether students lived “off-campus” or “on-campus.” Students were asked if they were aware of the ENS and if they had ever received an alert. For those who had received a past emergency notification, they were asked to rate the relevance and understandability of the messages received on a 4-point Likert scale from “not very relevant” or “not very understandable” to “very relevant” or “very understandable.” Perceptions of importance were measured by asking, “How important do you think it is to have a campus ENS?” Responses ranged on a 4-point Likert scale from “not very important” to “very important.” Safety was assessed with, “Do you feel safer knowing that there is a campus ENS?” Responses options were: “It does not make me feel safer,” “I feel somewhat safer,” and “I feel much safer.”
The constructs of threat appraisal, response efficacy, behavioral beliefs/attitude, subjective norm, and perceived behavioral control were used to develop questions to assess student intention to act on specific emergency messages.Reference Ajzen, Kuhl and Beckmann37,Reference Maddux and Rogers43 Because all the scales were individually tailored to the scenarios, a priori reliability cannot readily be established; however, typical Cronbach’s alpha of these items in studies using tailored questions range between 0.750 and 0.940.Reference Han, Ada and Sharman20,Reference Lee, Chung and Kim21,Reference Demuth, Morss and Lazo53-Reference Xiao, Peng and Yan56 Construct scales were assessed through 7-point scale questions that ranged from “completely disagree” to “completely agree.” Attitude toward ENS messages/ behavioral beliefs (Cronbach’s alpha = 0.716), vulnerability (Cronbach’s alpha = 0.828), and intrinsic and extrinsic benefits (Cronbach’s alpha = 0.740) were summative measures, each built from 3-question scales. Subjective norm (Cronbach’s alpha = 0.734), perceived behavioral control (Cronbach’s alpha = 0.670), severity (Cronbach’s alpha = 0.752), and response efficacy (Cronbach’s alpha = 0.821) were summative measures from 4-question scales.Reference Prentice-Dunn and Rogers46,Reference Ajzen57 The complete measure of the threat appraisal was developed by combining the final summed variables of vulnerability and severity, then subtracting intrinsic and extrinsic benefits.
Intention to rapidly comply with emergency notifications, without stopping to verify the information first, was assessed for each of the 7 unique scenarios. After the scenario and notification were presented, rapid compliance was assessed through 4-point Likert scales. The response options included: “I will not follow the directions,” “I will verify first then may or may not follow the direction,” “I will verify first and then follow the directions,” and “I will immediately follow the directions.” Due to limited variability, final outcomes for regression models were dichotomized to: “I will immediately follow the directions” and “I will not immediately follow the directions.”
The scenarios were developed in collaboration with the CSUN Police Services and the CSUN Office of Emergency Preparedness and reflect a variety of major and minor incidents that could prompt a mass campus alert chosen from 28 possible disasters and emergencies listed on the Department of Homeland Security’s Ready website.58 Based on prior research and an interview with the CSUN Chief of Police, 7 scenario topics were considered most relevant: robbery, active shooter, building fire, chemical spill, riot, air quality advisory, and a health advisoryReference Zdziarski, Dunkel and Rollo5,Reference Han, Ada and Sharman20,52,59 (Table 1).
Scenario Design
A priori power analysis was performed using G*power software version 3.1.9.2Reference Faul, Erdfelder and Lang60 suggesting a minimum sample size of 987 participants. The nonprobability sample consisted of undergraduate and graduate students from 58 classes within the 8 colleges at CSUN. Data were collected (n = 1725) beyond the sample size to ensure sufficient variability, while not overpowering the models.Reference Peduzzi, Concato and Kemper61 List-wise deletions were used to remove incomplete responses and the final analytic dataset included 1529 student responses. All participants were informed of risks and consented to participate in the study. IRB approval and exemption of the study was awarded by the CSUN IRB before survey distribution.
ANALYSIS
Data were analyzed using Statistical Package for the Social Science (SPSS) version 24 for data management and analyses with an alpha level of 0.05.62 To address the first aim of the study, descriptive statistics were calculated to assess the demographic profile of the sample and ENS perceptions. Bivariate analyses were conducted to explore patterns in perceptions across demographics through t-tests and chi-square tests. To address the second and third aim of the study, 7 separate multivariate logistic regression models assessed the association between behavioral intention to rapidly comply (immediately follow the direction without verification) on specific emergency notifications. All models included the hypothesized primary predictors: threat appraisal, response efficacy, attitude, subjective norm, and perceived behavioral control, adjusted for covariates (age, gender, race/ethnicity, and on/off campus residence). Final models showed significant improvements in log-likelihood and good model fit. Receiver operating characteristic curve (ROC) for all models was above 0.7, well within the suggested values demonstrating reasonable discrimination and adequate sensitivity.Reference Hanley and McNeil63-Reference Swets65 The potential for multicollinearity was assessed, the highest correlation among the independent predictors was between Subjective norm and perceived behavioral control (r = 0.697), within commonly established thresholds of 0.5 and 0.8.Reference Berry and Feldman66,Reference Vatcheva, Lee and McCormick67
RESULTS
The average age of respondents was 22.6 years (SD = 4.70) and nearly 75% were female (Table 2). The most frequently reported race/ethnicity within the sample was Hispanics/Latinos/Latinas (52.5%), followed by Asians/Pacific Islanders (15.7%), Non-Hispanic Whites (15.5%), Blacks/African Americans (4.8%), and Multiracial/Others (11.5%).
Demographic Characteristics of the Sample
Percentages may not total 100% due to rounding. Institutional Statistics were retrieved from Reference 70.
a Data exclude international students and unknowns.
b Data not currently available.
c Only included undergraduate students.
When asked about prior knowledge of the ENS, 25.0% of students did not know there was a campus ENS, and a majority (84.2%) acknowledged that it was very important to have an ENS on campus (Table 3). When asked about perceptions regarding past alerts received, 20.8% reported never receiving alerts or not remembering the content. Among those who did report receiving and remembering the content of alerts (n = 1228), 58.0% considered alerts very relevant and 78.0% considered alerts very understandable. When asked if students felt safer knowing there was an ENS on campus, 57.6% of students felt much safer, 37.4% felt somewhat safer, and 5.0% did not feel safer (Table 3).
Perceptions of the Emergency Notification System
Percentages may not total 100% due to rounding.
Bivariate analyses were conducted to assess differences in the proportional distribution of student perceptions. Specific perception measures were dichotomized or collapsed due to limited variability. Those reporting prior knowledge attended the university longer than those with no prior knowledge (P < 0.001). Those with no prior knowledge were less likely to report that the system did not make them feel safer, compared with respondents with prior knowledge (adjusted odds ratio [AOR] = 0.58; 95% CI: 0.46-0.73). Males were significantly more likely to report the system as “not very important” than females (AOR = 12.38; 95% CI: 4.11-37.27), and were less likely to report that the system did not make them feel safer compared with females (AOR = 0.66; 95% CI: 0.53-0.84).
Among those who recalled receiving an emergency message, males were more likely to report the messages as “not very relevant” than females (OR = 1.94; 95% CI: 1.49-2.52), and minority respondents were less likely to report the messages as “not very relevant,” than nonminority respondents (OR = 0.53; 95% CI: 0.39-0.73). Additionally, those reporting the messages as “very relevant” had been attending the university longer than those reporting “not very relevant” (P < 0.05).
When considering student responses to scenarios, although there was variability, 66.1% reported an intention to rapidly comply without verification, 24.5% elicited an intention to verify the situation first and then comply, and the remaining 9.4% of responses identified intention to not comply with the emergency notification messages. The active shooter scenario elicited the greatest intention to rapidly comply without verification (84.2% of students) with 11.2% intending to verify first and then comply. The robbery scenario elicited the least intention to rapidly comply without verification (48.3% of students) with 33.3% intending to verify first and then comply.
Results from separate multivariable logistic regression models suggested that the TPB constructs were most predictive of rapid compliance (Table 4). Higher attitude scores were associated with rapid compliance in all 7 emergency notification scenarios (AOR range = 1.06-1.12; 95% CI = 1.01-1.17). Higher subjective norm scores were associated with rapid compliance in nearly all emergency notification scenarios except the active shooter notification scenario (AOR range = 1.06-1.11; 95% CI = 1.01-1.17). Higher perceived behavioral control scores were only associated with rapid compliance in the air quality advisory notification scenario (AOR = 1.07; 95% CI = 1.03-1.12). Constructs related to the PMT were also associated with specific emergency notification scenarios. Higher threat appraisal scores were only associated with rapid compliance in the building fire emergency notification scenario (AOR = 1.02; 95% CI = 1.00-1.05); whereas, higher response efficacy scores were associated with rapid compliance in robbery, active shooter, and building fire scenarios (AOR range = 1.06-1.08; 95% CI = 1.02-1.13). These results show unique patterns of association across theoretical constructs (Figures 1 and 2).
Regression Analysis of Factors Associated With Compliance
All models control for campus residence, age, gender, and ethnicity.
Abbreviations: AOR, adjusted odds ratio; 95% CI, 95% confidence interval.
* P < 0.05.
** P < 0.01.
*** P < 0.001.
AORs Associated With Compliance Displayed by TPB
AORs Associated With Compliance Displayed by PMT
DISCUSSION
When considering the first aim, the study identified unique patterns in student perceptions of both messages and the ENS, suggesting specific populations may benefit from education and intervention. One-quarter of respondents were unaware of a campus ENS even though the median year of enrollment was 2015, suggesting that most students taking the survey attended for at least 2 years. Despite low awareness of the ENS, a large majority of respondents believed that it was very important to have an ENS on campus, which is concordant with other studies.Reference Han, Ada and Sharman20,Reference Kopel, Sims and Chin25 Similar to prior research, male respondents were significantly more likely to report the system as not very important compared with females.Reference Kopel, Sims and Chin25 This finding highlights an important opportunity for emergency managers and campus police services to strategically promote the ENS among males, encouraging them to consider the importance of emergency notification, as well as the seriousness of compliance with notifications.
The perceptions of safety are counterintuitive when compared with the perceptions of importance. Only 57% of students felt much safer because of the ENS, suggesting some students may either believe the system is ineffective, or notifications may not promote student safety during impending dangers, even if received. It is important to note, however, that ENS may not be considered in a person’s perceived general safety assessment of a college or university campus. In comparison to males, females were 1.5 times more likely to report that the system did not make them feel safer, a phenomenon that has been seen when considering other devices to mitigate threat/danger.Reference Yavuz and Welch68
Three-quarters of respondents reported prior knowledge of the system and recollection of the messages received. Males and non-Hispanic White students were more likely to report the messages as “not very relevant,” than females and ethnic minority students. Perceptions may play a role in the overall consideration of message relevance with scenarios that elicit varying perceived risk or safety responses. A prior study identified that females may have significantly higher risk perception regarding emergencies and higher perceptions of the benefits of emergency notifications.Reference Ada, Sharman and Han69 Results of the current study corroborate and extend findings in relation to perceived relevance of the notifications received. In this study, females attributed more importance and relevance to emergency notifications than males. In addition, when considering potential racial/ethnic differences, respondents identifying as a minority race/ethnicity were more likely to report the messages as relevant compared with nonminority respondents, suggesting that cultural differences may also play a role in the variation of relevance perceptions and risk related to emergency notifications and emergency situations. The concern for emergency managers and campus emergency administrators are the ramifications of diminished perceptions of relevance and how these may affect notification compliance. There were also significant differences in the length of institutional attendance among the relevant responses. On average, those who reported the messages as very relevant had been enrolled for a longer period of time, compared with those who did not see the messages as very relevant, suggesting that their importance increases after receiving more of them or longer enrollment at the institution.
Regarding the second and third aims, findings suggest that health behavior theory may be a useful predictive tool in assessing rapid compliance across a range of emergency scenarios, confirming and expanding on previous studies. Research has identified subjective norm as a strong predictor of rapid compliance with emergency notification messages, with our findings extending the body of knowledge to include attitude as an important predictor across a range of scenarios.Reference Han, Ada and Sharman20 In fact, attitude was the only predictor that was significant across all 7 different emergency notification scenarios. The finding that attitude was a universal predictor in scenarios may indicate that a student’s intention to rapidly comply is largely dependent on their beliefs about whether rapid compliance with emergency messages elicits a beneficial outcome. Moreover, a respondent’s beliefs about emergency messages, even before a campus emergency may occur, may influence compliance behaviors. These findings present a compelling case for continued promotion of ENS on college campuses.
Perceived opinions of referent groups, such as families, faculty, and friends, may also play a role in rapid compliance, illustrated by the significance of subjective norms across many of the emergency notification scenarios. Our results suggest that a student’s intention to rapidly comply with emergency messages is linked to their perceived expectations of persons of influence in their lives. This expands upon prior research in the field that subjective norms are a strong motivator of intention to rapidly comply.Reference Han, Ada and Sharman20 The only scenario where subjective norm was not associated with intention to rapidly comply was in the active shooter scenario. This may be due in part to the campus trainings, videos, and media attention that have been dedicated to this topic.Reference Han, Ada and Sharman20 These resources provide information about what to do in an active shooter scenario and, as such, subjective norm may not be a driver of compliance.
Another component to subjective norm is the influence of social pressure or cultural norms on behavior. Changing normative beliefs around emergencies and emergency notifications can be a challenge to campus emergency personnel. Future research needs to consider how to effectively shift both attitude and norms to benefit compliance behaviors. Although an institution cannot control the beliefs of referent groups, creating a culture of compliance with emergency notifications may be developed through normalizing compliance with emergency drills and tests of ENS. To date, campus compliance culture has not been adequately assessed on this and many campuses across the United States. This may be a potential barrier to campus emergency personnel. Moreover, the campus surveyed may not have performed emergency drills and tests of ENS often enough to create a culture of compliance. Further investigating how to identify and measure campus compliance culture and test the effects of emergency drills and ENS on compliance behavior is an important next step. The final construct of the TPB, perceived behavioral control, was much less predictive of intention to rapidly comply. Although speculative, students may feel that they have more control over rapidly complying in slow moving scenarios compared with fast moving scenarios, a finding that warrants further investigation.
Overall, we should consider the benefits of all of the constructs of the TPB in explaining rapid compliance. Although no causal conclusions can be drawn from cross-sectional data, these findings suggest rapid compliance is not heavily dependent on beliefs of whether directions can be followed or how much control they have; instead, it is much more dependent on a person’s attitude and the attitudes of those around them. These findings in conjunction with results of other studies have demonstrated the consistency of the attitude and norms constructs.
Compared with the TPB, the PMT was not as universally predictive. Overall, the PMT explained rapid compliance intention in faster developing scenarios, such as robberies, active shooters, and building fires where the ENS may be the initial alert received. These results are consistent with previous studies that showed perceived threat was associated with compliance in rapidly developing emergencies that have the ability to cause severe damage, with the current study adding to the research by testing both arms of the PMT.Reference Han, Ada and Sharman20 Threat appraisal was the least predictive of rapid compliance intention compared with response efficacy. Specifically, threat appraisal was only a motivator in the building fire scenario. This may be related to the safety concerns of students, along with the frequency of California fires. These results suggest that, in rapidly developing scenarios, the belief that rapid compliance will lead to safety may explain more of the protection motivation than threat appraisal of the emergency event itself.
STUDY LIMITATIONS
This study should be interpreted in light of the following limitations. First, the results of this study cannot be generalized to all universities but may reflect those of students in colleges and universities with a similar setting and demographic profile. Despite the use of nonprobability sampling, the demographic profile of the sample was similar to that of the university, except for gender which was similar to the demographic profile of College of Health and Human Development, the college most represented in the study.70 To this end, all models controlled for gender. Second, the specific scenarios included may not be equally relevant across regions of the country. Third, although the survey completion rate was high, there was no way to determine an accurate response rate and to identify patterns among those who did not respond. Fourth, this cross-sectional study design cannot establish a temporal relationship and does not support causal inference. Finally, the study is based on self-reported data, particularly data focused on intention to act/respond, rather than actual action/response, although there have been ample studies linking intention to act with actual behavior in emergency situations.Reference Han, Ada and Sharman20,Reference Paek, Hilyard and Freimuth40,Reference Floyd, Prentice-Dunn and Rogers45,Reference Ada, Sharman and Han69
CONCLUSIONS
Promotional materials, emails, and system tests can highlight both the importance and the benefit of an ENS, while also identifying potential system gaps and updates in contact information. Future emergency notification messages sent by campus emergency managers and police services should continue to be clear and concise. The demographic differences in the perceived relevance of messages provide an opportunity for emergency managers and campus administration to develop tailored interventions and promotion. Targeting new students and providing them with information about the ENS may increase the belief of relevance, as well as future rapid compliance with messages received. Additionally, emergency managers and campus police services may benefit from leveraging specific health behavior theory constructs to motivate intention to rapidly comply. ENS promotion, as well as regular tests and drills, may normalize the receipt of emergency messages, increasing knowledge and awareness about the system along with attitude and social norm. Tests should be done in conjunction with interventions promoting rapid compliance that may take the form of games, classes, events, and informational materials, and should focus around faster developing scenarios where protection motivation is a strong motivator.Reference Han, Ada and Sharman20,71,72
Future studies should consider testing emergency notifications to assess intention in real-time while capturing student response and identifying differences, patterns, and factors related to specific interventions. Finally, additional research is needed to explore associations with complete noncompliance. This may be a unique subset of the population that needs to be understood so emergency managers can tailor targeted interventions and messages.
Although emergency notifications are viewed as highly important, there is limited research into compliance with these messages, particularly on college and university campuses, which are especially vulnerable to emergency situations. This research contributes to the literature by (a) investigating patterns in student perceptions to identify potential intervention that can improve compliance, and (b) exploring the predictive ability of health behavior theory in relation to rapid compliance with emergency notifications, additional theoretical constructs and emergency scenarios to prior studies. The results of this study highlight the need for continuing promotion and interventions to improve awareness and compliance with ENS messages by leveraging the constructs from the TPB and PMT.
