On March 11, 2020, the World Health Organization (WHO) declared coronavirus disease 2019 (COVID-19) a pandemic. ^{Reference Cucinotta and Vanelli1} Efforts to control community spread of the highly transmissible virus have been variable. ^{Reference Cho2–Reference Yan, Zhang, Wu, Zhu and Chen4} However, 1 issue remains clear: The success of any community-level mitigation strategy to control COVID-19 spread depends upon vast numbers of individuals drastically altering their day-to-day personal behavior. Where adherence to guidelines and policies such as mask wearing and physical distancing have been low, rapid community transmission and rates of illness have consistently been the highest. ^{Reference Lewnard and Lo5}

One challenge in understanding and assessing individual adherence to mitigation strategies has been the evolving guidelines on these behaviors. Initially, public health experts in the United States did not recommend the regular use of nonmedical masks in public. However, by June 2020 guidelines changed ⁶ as evidence had converged on several key strategies still overwhelmingly endorsed as critical: frequent hand washing; wearing masks in public; practicing good cough/sneeze hygiene; avoiding close contact with others outside the home of unknown vaccine status; and cleaning and sanitizing high-touch surfaces. ⁷ Additionally, COVID-19 is primarily spread by means of respiratory droplets, but the CDC and WHO still recommend frequent sanitation of surfaces, leading to some controversy. ^{Reference Ives and Mandavilli8} These personal mitigation behaviors have been shown to reduce rates of COVID-19 community transmission, ^{Reference Cheng, Wong, Chuang, So, Chen and Sridhar9,Reference Clase, Fu, Joseph, Beale, Dolovich and Jardine10} and yet adherence has been inconsistent. Despite vaccine availability, uptake has been low, new variants continue to arise, and breakthrough infections are common, making other mitigation factors critically important. Thus, it is crucial for researchers and public health professionals to understand individual engagement in COVID-19 mitigation behaviors, both for reducing the human and societal costs of the current pandemic, and for preparing for future pandemics. ^{Reference Arden and Chilcot11} Such understanding begins with establishing the ability to measure the construct of individual mitigation behaviors reliably and validly.

The purpose of this research was to begin to establish a valid and reliable scale of viral respiratory illness (VRI) mitigation behaviors, particularly for the vulnerable pregnant population, with a next step to include validating this scale in a non-pregnant population. A Pandemic-Related Pregnancy Stress scale has been developed ^{Reference Preis, Mahaffey and Lobel12} as well as a Postpartum Specific Anxiety Scale for use during global crises, ^{Reference Silverio, Davies, Christiansen, Aparicio-García, Bramante and Chen13} but these do not directly measure COVID-19 or other VRI mitigation behaviors. As of the writing of this paper, we are aware of only 1 study that has examined the psychometric properties of evidence-based COVID-19 mitigation scales outside of measures of reliability. ^{Reference Breakwell, Fino and Jaspal14} This scale—the COVID-19 Preventative Behaviors Index, or CPBI—was developed and evaluated in the United Kingdom, and measures perceived likelihood of following mitigation behaviors in the future as opposed to current practices and also includes items that may not be evidence-based (ie, intending to change their lifestyle, or intention to watch the news ^{Reference Breakwell, Fino and Jaspal14} ). Other studies of COVID-19 mitigation have generally measured 1 specific behavior, like social distancing, ^{Reference Al-Hasan, Yim and Khuntia15,Reference Reinders Folmer, Brownlee, Fine, Kuiper, Olthuis and Kooistra16} shelter-in-place orders, ^{Reference DeFranza, Lindow, Harrison, Mishra and Mishra17} and mask-wearing. ^{Reference Goldberg, Gustafson, Maibach, Ballew, Bergquist and Kotcher18} Some studies have utilized checklists to examine aggregate “compliance” with several different public health measures. ^{Reference Nofal, Cacciotti and Lee19,Reference Van Rooij, de Bruijn, Reinders Folmer, Kooistra, Kuiper and Brownlee20} However, these checklists also contain behaviors that are not evidence-based (eg, participating in virtual events) and may even be actively discouraged by public health authorities (eg, stockpiling medical masks ^{Reference Nofal, Cacciotti and Lee19} ). Incorporating as many of the evidence-based strategies as possible—while eschewing those strategies that are not—is an important facet of construct validity ^{Reference Smith21} that has yet to be robustly investigated.

In developing a mitigation scale for VRIs like COVID-19, we evaluated the need in light of several factors, including: (1) low vaccination rates worldwide and the rise of more infectious variants indicating the COVID-19 pandemic is far from over; (2) the increasing likelihood that another VRI pandemic will emerge within the next 2 decades; and (3) the morbidity and mortality associated with more common respiratory viruses, such as seasonal influenza and respiratory syncytial virus (RSV), can be reduced through the same mitigation behaviors as those examined in the current study. We also considered nomological validity, ^{Reference Cronbach and Meehl22} or the importance of validating the extent to which a given measure matches up to the theory or evidence base that exists, and the extent that the new scale correlates to distinct, but theoretically related, constructs of interest within a nomological net. ^{Reference Smith21,Reference Cronbach and Meehl22} If a new measure can be validated as such, it becomes a more useful tool in evaluating public health interventions that seek to change health behaviors and the course of community spread for COVID-19 or other VRIs. ^{Reference Smith21,Reference Hagger, Gucciardi and Chatzisarantis23}

One way to examine whether a new measure is conceptually robust is to explicitly test a priori hypotheses about how an individual’s score on such a scale would correlate with other constructs within this nomological net. ^{Reference Ziegler, Booth and Bensch24} Engaging in evidence-based mitigation behaviors is likely related to knowing and understanding the health guidelines ^{Reference de Melo Ghisi, Abdallah, Grace, Thomas and Oh25} and the credibility of the source of those guidelines. ^{Reference Hocevar, Metzger and Flanagin26} Likewise, we would expect that a scale measuring VRI mitigation behaviors using the COVID-19 pandemic as its test case would positively correlate with intention to receive a COVID-19 vaccination, ^{Reference Shrank, Patrick and Brookhart27,Reference Achtymichuk, Johnson, Al Sayah and Eurich28} while expressing “adherence fatigue” for continued following of public health guidelines would likely negatively correlate to this scale. ^{Reference Rahmandad, Lim and Sterman29} Last, studies have shown that health behaviors tend to cluster within individuals and those who engage in 1 health-promoting behavior are more likely to engage in others. ^{Reference Poortinga30,Reference Pronk, Anderson, Crain, Martinson, O’Connor and Sherwood31} Thus, we might expect that more attention to viral respiratory illness mitigation is positively correlated with endorsing other healthful activities, like healthy nutrition, abstaining from harmful substances during pregnancy, and receiving a seasonal flu vaccine.

Finally, as of this writing, we are unaware of COVID-19 or other VRI mitigation scales that have been rigorously tested with pregnant populations specifically. Existing infectious disease theory suggests, and emerging evidence has shown, that COVID-19 infection significantly increases the risk of severe maternal, fetal, and neonatal morbidity and mortality for pregnant and postpartum populations. ^{Reference Villar, Ariff, Gunier, Thiruvengadam, Rauch, Kholin, Roggero, Prefumo, Do Vale, Cardona-Perez and Maiz32} Likewise, data have shown that pregnant people are at high risk from complications and death due to other VRIs, like the seasonal flu. ^{Reference Somerville, Basile, Dwyer and Kok33} This vulnerable population deserves increased research attention, including rapidly developed and validating a mitigation scale for use in ongoing studies. Furthermore, pregnancy is considered a “teachable moment” when health education is more likely to “stick” and health behaviors are more likely to be initiated. ^{Reference Olander, Darwin, Atkinson, Smith and Gardner34} Studying COVID-19 mitigation behaviors in this population will contribute to the knowledge base for the general population, as well as provide a foundation for better understanding how pregnant people manage preventable risk from communicable diseases in general.

This study seeks to remedy the gaps in our current understanding by first developing, and then validating, a short VRI mitigation scale with a pregnant sample in the United States using the COVID-19 pandemic as its test case over 2 points in time. We then tested a wide array of behavioral and attitudinal measures that comprise a nomological net to develop a fuller picture of the construct validity for this new measure.

Methods

Participants

Pregnant Californians were offered a Web-based survey in both English and Spanish (the 2 most common languages in the state) during 2 waves of cross-sectional data collection; Wave 1 was collected between June 6 to July 29 of 2020, while Wave 2 was collected from December 24, 2020 to January 27, 2021. Surveys took approximately 20-30 min to complete, and participants were offered $10 gift cards for their participation. Participants were recruited primarily using StudyPages, a service that leverages social media to solicit participation. Currently, pregnant individuals who resided in California and were between the ages of 18 and 45 were eligible. Informed consent was obtained by means of Web-based survey. University of California, Davis’ Institutional Review Board approved the study. In total, 588 participants completed the survey during Wave 1 and 454 completed the survey during Wave 2.

Missing Data

Because this was a Web-based survey, we used several strategies to ensure high data quality. Participants who attempted to take the survey multiple times, who completed the survey in less than 10 min, or who did not reach the end of the survey instrument had their data removed before analysis. Participant age was asked twice, once at the beginning of the survey and once at the end; any participant without matching responses for these 2 items also was removed before analyses. In all, 216 participants were not included in the analytic sample as a result of these quality checks. The data also were screened for impossible values and outliers. Sporadic missing data were limited to under 10% for each variable and handled using list-wise deletion. The resulting analytic samples are comprised of 433 participants for Wave 1 and 387 participants for Wave 2. Demographic features of the analytic samples can be found in Table 1.

Table 1.

Demographic characteristics across data collection wave

Note: Every participant did not respond to every item; therefore Ns do not necessarily sum to total analytic sample size

Measures

Survey items consisted of several categories: demographic features of the sample, 7 mitigation behaviors that make up the scale itself, and 13 items within the nomological net of the scale (4 hypothesized predictors of the mitigation scale, 3 items indexing attitudes or behaviors regarding the COVID-19 pandemic, and 6 health behavior items unrelated to COVID-19). See Appendix 1 for details on the specific items within these categories, and Table 2 for scale item wording.

Table 2.

Summary statistics of specific viral respiratory illness mitigation strategies across waves

Data Analysis

Testing Assumptions

We examined whether each sample (wave 1 and 2) was adequate to conduct exploratory and confirmatory factor analysis. Sample sizes must be over 300³⁵ and there must be more than 20:1 sample to variable ratio. ^{Reference Osborne, Costello and Kellow36} Each individual item in the scale should correlate at levels of .30 or higher with at least 1 other item in the scale, average inter-item correlations should be at least r = 0.15, Bartlett’s test of sphericity should be significant (P < 0.05), and the items should have a Kaiser-Meyer-Olkin (KMO) value of .60 or higher. ^{Reference Clark and Watson37–Reference Tabachnick and Fidell39}

Exploratory Factor Analysis

We used exploratory factor analysis (EFA) using Principal Factors Analysis in Stata, ^{Reference Carpenter40} examining solutions with 1, 2, and 3 factors across the 7-item overall scale. There were several criteria we used to choose the model that fit the data best: (a) eigenvalues for included factors should be at least 1, ^{Reference Kaiser41} but ideally much larger ^{Reference Russell42} ; (b) only retaining items that loaded onto a stable factor with a factor loading of more than 0.4⁴⁰; and (c) only retaining items that showed a communality of at least 0.3. ^{Reference Costello and Osborne43}

Confirmatory Factor Analysis

After conducting an exploratory factor analysis, we validated the chosen factor structure in the second wave of data using Confirmatory Factor Analysis (CFA). The sample size, sphericity, and KMO were tested to ensure that the sampling and data quality were adequate for these purposes. ^{Reference Worthington and Whittaker35,Reference Osborne, Costello and Kellow36,Reference Pett, Lackey and Sullivan38–Reference Carpenter40} That model was then tested for goodness-of-fit using 3 measures: (a) comparative fit index (CFI) and Tucker-Lewis index (TLI) over 0.95 indicating “good” model fit, and CFI and TLI between 0.90 and 0.95 indicating “adequate” model fit ^{Reference Hu and Bentler44} ; and (b) root mean square error of approximation (RMSEA) under 0.05 considered a “good” fit, and RMSEA values between 0.05 and 0.10 considered “acceptable”. ^{Reference Browne and Cudeck45} Error terms among our items were allowed to correlate if there were theoretically sound reasons for the addition of these parameters. Finally, we report the internal consistency (using Cronbach’s α; ^{Reference Cronbach46,Reference Bland and Altman47} ) at each wave and over both waves. Cronbach’s α > 0.8 (but <0.9) are considered “good” and Cronbach’s α between 0.6 and 0.8 are considered “acceptable” for this short, nonredundant scale. ^{Reference Nunnally and Bernstein48,Reference Streiner49}

Exploring Demographic Corollaries of the Scale

After confirming an adequate goodness-of-fit and adequate internal consistency of the scale, we described how the mitigation scores varied across demographic characteristics by using either univariate ordinary least squares regression or 1-way analysis of variance. We also described whether and how scores changed over the 2 waves of data collection.

Testing Nomological Validity

We began testing the nomological validity of the scale by examining the relationships between the scale scores and distinct constructs of interest. These constructs fall into 3 categories: (a) predictors of COVID-19 mitigation, (b) other attitudes and behaviors that relate to the COVID-19 pandemic, and (c) other (nonpandemic) related health-promoting behaviors. A significant relationship between 2 constructs was considered present if the correlation reached a P < 0.05 significance level. We also assessed the strength of the correlations, which were considered “weak” if the absolute value was less than 0.20; “moderate” if the value was between 0.20 and 0.40; and “strong” if the correlation was more than 0.40.

Results

Testing Assumptions

Both wave 1 and wave 2 samples were adequate for conducting EFA and CFA analyses. The Bartlett test of sphericity was significant for both samples: χ² = 680.92, P < 0.001 for wave 1; χ² = 488.79, P < 0.001 for wave 2. For wave 1, KMO = 0.71; and for wave 2, KMO = 0.70. Sample sizes of 433 and 387 provided large enough samples for factor analysis with a ratio of over 50 participants per variable in each wave. For wave 1, inter-item correlations range from 0.11 to 0.56, with all correlations significant at P < 0.05. Inter-item correlations average r = 0.29 with all other items in the scale, and each item correlated with at least 1 other item on the scale at or above r = 0.45 See Table 2 for correlations between individual items in the scale.

Exploratory Factor Analysis

Table 2 shows response rates for each item across both waves. Based on our a priori criteria, we extracted a single factor and retained all of the items. The eignenvalue for the 1-factor solution was 2.1 (2-factor solution = 0.7; 3-factor solution = 0.2). Table 3 shows results from EFA and CFA. The proportion of the scale variance explained by 1 factor is 0.89. In the 1-factor solution, all factor loadings are >0.4, and communality are all >0.3, confirming that each item shares common variance with the other items. By contrast, factor loadings for the 2^nd and 3^rd factors in the 2- and 3-factor solutions were largely below 0.4. Therefore, we created the scale as a unidimensional measure, labeled “Viral Respiratory Illness Mitigation Scale (VRIMS)”.

Table 3.

Results of Factor Analysis

Note: Single-factor scale developed during wave 1 showed an acceptable fit for the data during wave 2: CFI = 0.97, TLI = 0.94, RMSEA = 0.061

Confirmatory Factor Analysis

Using wave 2 data (n = 387), we conducted a CFA using the sem package in Stata. A 1-factor structure was confirmed to be an adequate-to-good fit for the data. The RMSEA = 0.061, the model CFI = 0.97 and model TLI = 0.94. See Table 3 for standardized factor loadings for each item. As expected, error terms between certain items were significantly related: (a) the 2 mask-wearing items, (b) the 2 hand-washing items, and (c) the 2 surface-sanitizing items (all error term correlations significant P < 0.001). These error terms were allowed to correlate in the final CFA model.

Given the EFA and CFA results, scores on individual items were averaged, then the scale was standardized and centered for further analyses for the entire sample (n = 820). Figure 1 shows the distribution of this standardized/centered scale over the 2 waves of data collection. Scale averages shifted slightly over time: (a) wave 1 mean = −0.38, SD = 0.68; and (b) wave 2 mean = 0.39; SD = 0.53. One-way ANOVA showed that the wave 2 scale average is marginally higher than wave 1: F(817) = 3.28; P = 0.071. Over both samples, the measure showed acceptable reliability, with Cronbach’s α = 0.72 (wave 1 α = 0.74 and wave 2 α = 0.70). The measure is negatively skewed and tail-heavy at both waves.

Figure 1.

VRIMS score distribution by data collection wave.

Exploring Demographic Corollaries of the Scale

VRIMS scores varied significantly by several demographic categories (see Figure 2). Older participants scored higher than younger participants (β = 0.01; P = 0.017), and Hispanic participants scored higher than non-Hispanic participants (F(813) = 18.14; P < 0.001). Participants living in more urban contexts scored marginally higher than those living in more rural contexts (β = 0.04; P = 0.066), while Black and Asian/Pacific Islander participants scored marginally higher than white and multiracial participants (F(692) = 2.34; P = 0.05). Primiparous participants scored lower than multiparous participants (F(816) = 5.83; P = 0.016). Essential worker status was not significantly related to VRIMS score.

Figure 2.

Average VRIMS score by participant demographic category.

Testing Nomological Validity

See Table 4 for hypothesized and observed nomological validity correlations. Overall, the hypothesized relationships were observed in expected directions with at least P < 0.05 significance. Receiving or intending to receive both the seasonal flu vaccination and the COVID-19 vaccination were important exceptions to this finding: 29.9% of wave 2 respondents indicated that they were unsure whether they would receive the COVID-19 vaccine if it were made available to them, and although the correlation between this intention and the VRIMS was in the expected direction, it did not reach statistical significance (0.08; P = 0.19). Additionally, drinking sugary beverages was not related to the VRIMS score (r = 0.06; P = 0.11), although the other health-promoting behaviors (that do not index viral respiratory illness mitigation) were significantly related in the expect direction.

Table 4.

Hypothesized and observed relationships between VRIM Scale and other indicators of nomonological validity (n = 820)

Note: Hypothesized relationship between indications and VRIMS score denoted as follows: + weak positive correlation; ++ moderate or strong positive correlation; - weak negative correlation; - - moderate or strong negative correlation; ns non-significant correlation

^a Items available only for wave 2 data collection