Rising sea levels and extreme weather events are already impacting people all over the world and are predicted to continue. ¹ The Intergovernmental Panel on Climate Change warns that low-lying coastal areas are increasingly vulnerable to the effects of sea level rise, storm surge, flooding, and extreme storm events. ¹ Such potential for disasters in the future necessitates the need for emergency preparedness at all levels, to include the individual/household level. Despite the lack of evidence-based emergency preparedness recommendations, ^{Reference Heagele2} researchers endeavor to determine whether emergency preparedness education regarding what community members need to obtain to be prepared for emergencies can improve preparedness levels ^{Reference McNeill, Alfred and Mastel-Smith3} or whether such emergency preparedness really matters. ^{Reference Clay, Goetschius and Papas4} Hurricane Katrina decimated New Orleans in August of 2005; the impacts are still felt today. ^{Reference Gibbens5} Superstorm Sandy devastated New York City (NYC) in October 2012, ^{Reference Benimoff, Fritz and Kress6} Hurricane Maria demolished Puerto Rico in 2017, ^{Reference Pasch, Penny and Berg7} and Hurricane Florence destroyed areas of eastern North Carolina in 2018. ⁸ These events point toward an underlying need to create an evidence-based instrument to measure household emergency preparedness (HEP), guide education efforts to improve HEP, and ultimately determine the impact of HEP among community-dwelling residents.

A Delphi study was recently completed for the purpose of generating a consensus on the concept definition of HEP from experts and community stakeholders representing multiple disciplines and countries to develop an all-hazards, valid and reliable Household Emergency Preparedness Instrument (HEPI). ^{Reference Heagele, McNeill and Adams9} Delphi participants came to a consensus that adequate HEP is defined as completion of several preparedness actions in addition to assembling a home disaster supply kit that would enable households to endure the common conditions that disasters present, regardless of location. ^{Reference Heagele, McNeill and Adams9} These common conditions include power outages, deficiencies with drinking water, and being forced to shelter in place for a few days. ^{Reference Heagele, McNeill and Adams9}

The creators of the HEPI defined the construct of HEP and developed the initial instrument questions while establishing evidence to support face, content, and criterion validity of the instrument. ^{Reference Heagele, McNeill and Adams9} The HEPI - Delphi yielded a 51-question instrument built through qualitative content analysis and group consensus achieved with 3 rounds of data collection. Scaling of the original Delphi instrument was based on similarity of items as presented by the expert panel before pilot-testing in a general population. The next step of instrument development is to administer the instrument to a representative sample, perform psychometric testing on the instrument to establish construct validity, and make revisions if necessary. The purpose of this cross-sectional study was to pilot test the HEPI to perform psychometric testing and generate reliability and construct validity data. A second purpose was to measure and compare the HEP levels of a convenience sample of faculty, staff, and students from an urban university that is comprised of a socially vulnerable population in a physically vulnerable geographical location.

New York City is the most populous city in the United States (US) and is characterized by diverse people, neighborhoods, landscapes, and aging infrastructure. The city is vulnerable to coastal storms (erosion and flooding), rising sea levels, high winds, high heat, winter weather, earthquakes, fires, chemical/biological/radiological/nuclear incidents, emerging infectious diseases (epidemics and pandemics), and the approximately 22 million people are heavily dependent on energy, public transportation, water, and wastewater treatment. ¹⁰

Emergency management agencies in NYC began to aggressively implement emergency preparedness educational interventions after the 9/11 attacks in 2001. ^{Reference Martins, Louis-Charles and Nigg11} Martins et al. analyzed data about the emergency preparedness levels of 2,001 NYC residents collected in the months before the 2012 Superstorm Sandy. ^{Reference Martins, Louis-Charles and Nigg11} They found that NYC residents engaged in emergency preparedness activities at a greater rate than community members in most geographical locations in the United States. The NYC residents were considered well prepared for disasters, particularly with gathering supplies and developing a family communication plan. Martins et al. attribute these high preparedness levels to high levels of social capital, trust in the government, and community emergency preparedness educational interventions. ^{Reference Martins, Louis-Charles and Nigg11} Some of these community interventions include: (a) training health-care providers (HCPs) about how to educate their medically vulnerable patients about emergency preparedness resources ¹² ; (b) the Ready New York and Know Your Zone public education campaigns ¹³ ; (c) the Notify NYC emergency alert system for risk communications ¹³ ; (d) the NYC Community Emergency Response Teams and Community Organizations Active in Disasters teams of community volunteers who assist their neighbors before and during disasters ^13,14 ; (e) training community- and faith-based organizations about how to prepare their employees, services, and facilities for emergencies ¹³ and; (f) free HEP classes with distribution of disaster supply kits for participants. ¹⁵

Assessments of college and university campus preparedness levels and plans are abundant. ^{Reference Abedin and Shaw16–25} However, little is known about the emergency preparedness levels among college students. College students are seldom considered a vulnerable population even though they are living in large institutions and are often geographically separated from families that provide important instrumental and social support. ^{Reference Claborn26} Less than half of students receive emergency preparedness information or training during student orientation. ^{Reference Cheung, Basiaga and Olympia17} Although little research has been done to determine the preparedness levels among college students, what studies have been done demonstrate that college students are unprepared for disasters. ^{Reference Claborn26–Reference Tkachuck, Schulenberg and Lair29}

The City University of New York (CUNY) is the third-largest university system in the United States, in terms of enrollment, with more than 274,000-degree-credit students, continuing, and professional education students enrolled at 25 campuses located in all 5 NYC boroughs. The university has 1 of the most diverse student bodies in the United States, with students hailing from 208 countries, but mostly from NYC. ³⁰ As of fall of 2019, the race/ethnicity of all CUNY students was 0.3% American Indian/Alaska Native, 21.2% Asian/Pacific Islander, 25.2% Black, 30.2% Hispanic, and 23.1% White. ³¹ Undergraduate enrollment in fall 2019 consisted of 97.2% under the age of 45 and 2.7% were 45 or older. ³² As of fall 2016, 34.5% of CUNY’s approximately 7600 full-time faculty were from underrepresented groups. ^{Reference Torres33}

Recruiting CUNY faculty, staff, and students as a convenience sample for this pilot study provided a large, diverse, and possibly globally representative group of participants. Culture bias is manifested when an instrument is not conceptually equivalent for cultural subgroups, resulting in different psychometric properties for these groups. ^{Reference Waltz, Strikland and Lenz34} The assumption of universal applicability of standardized instruments normed on particular cultural or racial/ethnic majority population needs to be tested. ^{Reference Ramirez, Ford and Stewart35} The diverse sample obtained from CUNY faculty, staff, and students was ideal to evaluate culture bias of the HEPI.

Methods

This pilot study received institutional review board (IRB) approval from Hunter College, CUNY (protocol #2019-1037). The participants’ consent was implied by completing the questionnaire. The participants were not compensated for their participation.

No specific race, ethnicity, gender, socioeconomic status, employment status, marital status, citizen status, or education level was targeted for recruitment. Inclusion criteria was noninstitutionalized, English-reading, adults who identified as either a CUNY faculty, staff, or student. Exclusion criteria was persons less than 18 y of age and non-CUNY faculty, staff, or students.

Given that this was a pilot study, assuming a simple design of 1 population self-classified into 2 groups based on responses to the survey (prepared or not prepared), the minimum sample size was calculated as N > 50 + 8m (where m is the number of groups), which is the common method for determining sample sizes in this scenario. ^{Reference Green36} In this case, m = 2 (prepared or not prepared) so the minimum sample size required per group was 66 participants. Based on potential missing data of approximately 10%, the minimum sample size sought for each group was 75 participants, or 150 participants total. Nine of the 25 CUNY campuses granted permission for the researchers to recruit participants from their campuses. Deans and department heads chose whether or not to forward the IRB-approved recruitment and consent script by means of e-mail and served as gatekeepers for recruitment. As a result, varying numbers of CUNY faculty, staff, and students on participating campuses received the link to the survey.

Data were collected using Qualtrics online survey software version February 2020 (Qualtrics, Provo, UT) to distribute the HEPI. ^{Reference Heagele, McNeill and Adams9} The HEPI is an all-hazards, comprehensive, 51-question instrument used to ascertain if a respondent is prepared for disasters. There is support for face, content, and criterion validity of this instrument. ^{Reference Heagele, McNeill and Adams9} The HEPI questions are objective and ask about what the respondent presently owns or does in a dichotomous format. As this was a pilot test of the new instrument, virtual focus group-style questions were asked at the conclusion of the HEPI. Participants were asked to rate the perceived clarity of the HEPI instructions and questions, the difficulty of the HEPI, and the reasonableness of the length of the HEPI. In addition, 1 open-ended question was included to inquire about anything the participant would like the researchers to know about the experience of completing the HEPI. Open-ended interview questions provide the participants the opportunity to describe their experience in greater detail. ^{Reference Streubert and Carpenter37} Focus groups representative of the intended respondents are useful for (a) developing insights into lay-person perceptions of HEP, (b) informing authors about the language and terminology that particular groups of people use, and (c) revealing unexpected additional issues related to HEP that were not identified through the conventional means of the literature review or the Delphi study. ^{Reference DeVellis38}

Data Analysis

The open-ended question responses were analyzed with qualitative content analysis by means of NVivo software for Mac version 11.4.1 (QSR International Pty Ltd, Doncaster, Victoria, AU). Participants’ responses were coded, counted, and placed into categories by 2 researchers separately first, then discrepancies were discussed and agreed upon, and then the data was summarized. These data were primarily used to learn about the participants’ experience with completing the HEPI.

The level of statistical significance for Type I error was set at P < 0.05. Evidence-based estimation of effect size is not currently available. On the basis of expert judgment, Type II error was set at beta = 0.20. All data analyses were completed with the Statistical Package for the Social Sciences for Windows version 26.0 software (International Business Machines Corporation, Armonk, NY).

Nine nondemographic items dispersed randomly throughout the scales were left blank. The most commonly occurring subscale response was used to replace those missing items before summing the subscale scores for Preparedness Actions and Planning (PAP) and Disaster Supplies and Resources (DSR). These subscales represent basic preparedness applicable to all individuals/households. Assumptions appropriate for statistical testing were evaluated and corrections were applied as required for the various statistical analyses used to test hypotheses.

It is appropriate to measure reliability of the HEPI by looking at the internal consistency of the questions. Internal consistency indicates how well the questions fit together conceptually, which can be assessed by looking at the inter-item correlations. ^{Reference DeVellis38,Reference Devon, Block and Moyle-Wright39} This looks at the relationship among questions that are all indicators of the latent variable of HEP. Questions that have a strong relationship to HEP will have a strong relationship to one another. ^{Reference DeVellis38} The HEPI would be considered internally consistent to the extent that its questions are highly intercorrelated, as evidenced by coefficient alphas greater than .65. ^{Reference DeVellis38}

The HEPI is intended to measure actual preparedness for individuals/households; resulting in questions restricted to a dichotomous (yes or no) response format. For example: Have you prepared and discussed a family emergency plan? Scales using this type of response format do not perform well when subjected to traditional factor or component analyses. An alternative approach that is based on polychoric correlations was used to examine the factor structure of the new instrument. ^{Reference Lorenzo-Seva and Ferrando40}

The Access and Functional Needs (AFN) subscale is highly individualized and only applicable to a specific subset of the population (participants with a disability, aged greater than 65, on at least 1 prescription medication, or who are pregnant) and was not included in the factor or reliability analysis. The same is true for the Special Actions (SA) subscale that is applicable to only those who have specific lifestyle or family characteristics (Table 1).

Table 1.

HEPI scaling after factor analysis

Note: Items included in Factor Analysis are shaded in gray. Original Scaling: Preparedness Actions (PA), Communication Plans (CP), Evacuation Plans (EP), Disaster Supplies (DS), and Access & Functional Needs (AFN). Scaling after Factor Analysis: Preparedness Actions and Planning (PAP) and Disaster Supplies & Resources (DSR). Scales remaining but not applicable to all: Special Actions (SA) and Access & Functional Needs (AFN).

^a One-point weight added for items included in an emergency supply kit.

Results

The survey was accessed by 348 participants with 295 consenting, meeting the inclusion criteria, and beginning the survey. Data were reviewed for completeness and retained for the 284 participants who completed at least 95% of the survey (Table 2), 26.7% (n = 71) of which identified as having a country of origin outside of the United States. Approximately 50% of the surveys were completed by students (n = 140) compared with faculty/staff (n = 138); likewise, almost 50% reported they were married or partnered (n = 145) compared with not married or partnered (n = 133). Caucasian participants (n = 148) were most frequently represented, followed by Asian (n = 52), Hispanic/Latino (n = 39), Black/African American (n = 23), and Other (n = 14) participants. Faculty/staff were better prepared than students, people 50 y and older were more prepared than those 29 and younger, and members of the Hispanic/Latino and White/Caucasian group were significantly better prepared (P ≤ 0.001) than those in the other race/ethnicity groups. See Table 2 for additional information on the demographic characteristics and general preparedness of the CUNY sample.

Table 2.

Demographic characteristics and GP of CUNY sample

Note: c = comparison group. HCP discussion of HEP = health-care provider discussion of household emergency preparedness).

* P < 0.05.

** P < 0.01.

*** P ≤ 0.001.

Regarding factor analysis, the adequacy of the polychoric correlation matrix was assessed favorably resulting in a highly significant Bartlett’s statistic, a very good KMO (.92) indicating the sample size was sufficient for factor-analysis, and a narrow bootstrap 95% CI of .902 to .904. This indicates the quantity and quality of the data were appropriate for this method of factor analysis. The first factor analysis, based on a 4-factor solution (representing the 4 scales organized from the data generated in the Delphi study of experts), did not yield a reasonable solution. The factor analysis output indicated a 2-factor solution might be optimal and when run, 31 of the 34 general preparedness questions loaded on 1 of 2 dimensions and explained 35% of the variance represented in this sample. The study participants did not discriminate types of preparedness items as precisely as expected. The sample represented in this study is likely more consistent with the general population than the Delphi experts and changes to scaling were made to reflect the 2 dimensions represented in the factor analysis. The first dimension presented as the PAP scale (Cronbach’s alpha = .86) and the second dimension presented as the DSR scale (Cronbach’s alpha = .80). Together, the 2 scales represent the General Preparedness (Cronbach’s alpha = .89) of individuals or families. Yes-responses to the questions were summed across the subscales and for the General Preparedness (GP) scores. Nine of the DSR items were assigned an extra point if the item was included in an emergency supply kit. Refer to Table 1 for more information on the HEPI before and after factor analysis and how survey questions were re-clustered to better fit the 2 dimensions of GP identified in the factor analysis process.

Higher scores on the HEPI indicate higher levels of preparedness. Excluding the AFN and SA subscales that are not applicable to all respondents, the minimum score a participant could receive on the GP scale is 0 and the maximum score is 40 (1 point for each yes-response and 0 points for each no-response). The mean GP score for this sample was 15.28 and the range was 1 to 37.

Of the 127 participants reporting AFN, 122 responded to the question of HCP discussion of HEP. Only 10 of those individuals and 5 individuals who did not report access and/or functional needs (a total of 5.4% of the total sample) responded positively to the question. Those who had a discussion with their HCP were some of the best prepared study participants with a GP M = 19.33 (8.89) compared with M = 15.06 (6.83) for those whose HCPs did not discuss HEP with them (t(282) = 2.32; P = 0.021). See Table 2 for a complete description of sample characteristics and group differences in GP.

To examine the overall impact of the significant demographic variables on GP, the following variables were dichotomized: married or partnered to not married or partnered, faculty/staff to students, Hispanic/Latino and Caucasian/White to any other race/ethnicity, graduate degree to no graduate degree, annual income $75,000 or greater to less than $75,000, HCP discussed HEP to HCP did not discuss HEP, owned home to did not own home, and previous home damage due to disaster to no previous home damage due to disaster. Using standard multiple regression, GP was regressed on the 9 variables, together, they explained 20% of the variance found in the level of preparedness for this diverse group (R ^{Reference Heagele2} = .205; adjusted R ^{Reference Heagele2} = .178; F(9, 268) = 7.67; P < 0.001). However, only 1 of the 9 dichotomized variables, previous home damage due to disaster event, provided a significant independent impact on GP (unstandardized B = 3.17; t = 3.70; P < 0.000) having the greatest unique impact on GP represented by a part correlation of r = .20.

The core theme from the qualitative data was that participation in the HEPI helped participants learn what is needed to be prepared, made them realize that they were unprepared, and inspired them to develop a HEP plan. Several participants recommended that HEP educational resources be included at the completion of the HEPI. Five participants wrote they felt anxious after completing the HEPI. The authors will include in the instructions for researchers to disclose to their IRB and potential participants that completion of the HEPI may cause anxiety. The second most common theme from the qualitative data was that the HEPI was user-friendly, with descriptors of “easy,” “quick,” “understandable,” and “clear” commonly mentioned. This theme was supported by the quantitative data, with 98% (n = 278) of the sample rating the clarity of the instructions for the survey as extremely clear, 98% (n = 277) rating the clarity of the questions as extremely clear, 77% (n = 218) rating the length of the survey as extremely reasonable, and 75% (n = 212) rating the level of difficulty of the survey as extremely easy on a 5-point rating scale.