Introduction
Disasters are among the most leading causes of untimely deaths, deterioration in health status, and decline in quality of life. In general terms, disasters are major events leading to social devastation. Natural and man-made disasters, the number and severity of which have increased significantly, have occurred and continue to occur, and thousands of people have been affected and lost their lives and economic losses have been experienced due to these disasters. 1 From 2010 through 2020, the number and severity of hydro-meteorological and climatological natural disasters have increased significantly as a result of climate change. More than two thousand climate-related extreme weather disasters have occurred and thousands of people have died. 2,3
Health care services have an important role in minimizing disaster-related mortality and morbidity rates. Health care organizations should strive to provide uninterrupted care, but disasters are inevitable and thus preparedness is essential. Nursing services under this system have an important place in disaster management in terms of the number of their manpower, roles, and responsibilities. Reference Veenema4 Nurses have a continuous and undeniable contribution to strengthening and improving health care systems. Reference Baker Ghazi5
In the international field, importance is attached to determining the disaster preparedness of nurses. Reference Betka, Bergren and Rowen6,Reference Khilji, Raziq and Shoaib7 Increased level of disaster preparedness reduces the death rates from disaster events. 8 Nonetheless, nurses are insufficiently prepared for disasters. Reference Tilahun, Desu, Zeleke, Dagnaw and Andualem9–Reference Songwathana and Timalsina11 Nurses’ lack of knowledge, skills, and education regarding disaster risk management should be addressed and more research into this topic is needed. Reference Molassiotis, Guo, Abu-Odah, West and Yuen Loke12,Reference Loke Yuen, Guo and Molassiotis13 Different measurement tools are used to identify these needs of nurses in the literature; however, there is no standard universal measurement tool for disaster preparedness of nurses. This study aimed to assess the psychometric properties of scales developed or adapted to evaluate the sudden-impact natural disaster preparedness of nurses.
Methods
Inclusion and Exclusion Criteria
The studies that met the following eligibility criteria were included in the review: (a) having a sample consisting of nurses, (b) aimed to develop tools to determine the disaster preparedness of nurses or to evaluate the cross-cultural adaptation of tools, (c) indicating the psychometric properties of these tools, (d) have been published in English, and (e) published from January 2010 through June 2021 in selected databases. Interventional, emergency preparedness tools, qualitative, and non-interventional studies that did not aim to evaluate psychometric scale properties, discussion papers, grey literature, and studies including literature reviews were excluded.
Search Strategy
Web of Science (Thomson Reuters; New York, New York USA); PubMed (National Center for Biotechnology Information, National Institutes of Health; Bethesda, Maryland USA); CINAHL (EBSCO Information Services; Ipswich, Massachusetts USA); Scopus (Elsevier; Amsterdam, Netherlands); and ProQuest (Ann Arbor, Michigan USA) databases were scanned to identify the words in the title, abstract, and keywords of the relevant studies. Keywords were generated by choosing the English words and word groups appropriate for the terminology of Medical Subject Headings (MeSH). Then, these keywords were scanned in detail with various combinations in databases. The following search strategy was developed using Boolean connectors for each database: (nurs) AND (disasters) AND (preparedness) AND (evaluation OR assessment OR psychometric OR measure OR propert OR develop OR reliab OR valid OR method OR tool OR instrument OR scale OR survey OR questionnaire OR instrument OR version OR cross-cultural OR adaptation). This study focuses only on disaster preparedness scales. Emergency scales were not evaluated. Finally, the references sections of the identified studies were searched except for grey literature. After the duplicates were removed via Mendeley software (Mendeley Ltd.; London, UK. Version 1.19.8), titles were searched, and the full texts of the studies that were relevant to the subject and that met the eligibility criteria were downloaded.
The Consensus-Based Standards for the Selection of Health Measurement Instruments (COSMIN) risk of bias checklist was used to evaluate the quality of psychometric properties of the tools. Measurement properties, including content validity, structural validity, criterion validity, reliability, and internal consistency, and cross-cultural validity were evaluated. Reference Mokkink, de Vet and Prinsen14 The result of each study on a measurement property was rated against the updated criteria for good measurement properties on a three-point rating scale: sufficient (+), insufficient (–), and indeterminate (?). The Development of a Disaster Preparedness Tool Kit for Nursing and Midwifery Report of the World Health Organization (WHO; Geneva, Switzerland) and International Council of Nurses (ICN; Geneva, Switzerland) “Core Competencies in Disaster Nursing” (version 2.0) were used to evaluate the scale contents. This report provides content areas and competencies for developing a toolkit for disaster preparedness of nurses, which includes actions required in mitigation/prevention, preparedness, response, and recovery/rehabilitation phases in the context of external (international) and internal (national and local) action and their consequences for individuals, communities, and health systems. 15
Search Process
A total of 3,350 potentially relevant studies were identified from the databases using the described search strategy (Figure 1). After the elimination process, the remaining 41 studies were assessed for eligibility, and as a result of this final assessment, the psychometric properties of six studies were included in the review. All potentially relevant studies were screened by author and a collage of the author independently. The details of the studies are presented in Table 1 under the following headings: study title, country/year, language versions, participants, sample size, number of items, and dimensions measured. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) writing guideline was used for reporting. Reference Page, McKenzie and Bossuyt16
PRISMA Flow Diagram.
Abbreviation: PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.
Characteristics of Included Studies (n = 6)
Ethical Considerations
No formal ethical review was required for this study, as the examined articles were identified from open-access search engines and electronic databases.
Results
This systematic review included six studies. Four of the studies aimed to adapt the Disaster Preparedness Evaluation Tool (DPET) scale to Chinese (DPET-C), Reference Chen, Chou, Liao, Ho and Chung17 Mainland China (DPET-MC), Reference Wang, Lu and Sun18 Korean (DPET-K), Reference Han and Chun19 and Classical Arabic (DPET-Arabic), Reference Al Khalaileh, Bond, Beckstrand and Al-Talafha20 whereas the other two were the Chinese Hospital Nursing Department Disaster Preparedness Scale (Chinese-HNDDPS) Reference Shi, Li, Xu, Wang, Li and Feng21 and the Iranian Nurses’ Disaster Preparedness (Iranian-DPT) scale Reference Tavan, Menati, Azadi, Sayehmiri and Sahebi22 (Table 1).
The Item-Level Content Validity Index (I-CVI) and the Scale-Level Content Validity Index (S-CVI) were used in the Chinese-HNDDPS scale. The I-CVI was calculated to be 0.83–1.0 for each entry and 0.95–1.0 for each dimension, whereas the S-CVI was found to be 0.96. Reference Shi, Li, Xu, Wang, Li and Feng21 In the DPET-C scale, the average of I-CVI was found to be 0.98 for all scale items. Reference Chen, Chou, Liao, Ho and Chung17 In the DPET-MC and the DPET-Arabic scales, I-CVI and S-CVI were not used, but face validity was achieved. Reference Wang, Lu and Sun18,Reference Al Khalaileh, Bond, Beckstrand and Al-Talafha20 In the DPET-K scale, experts reached 80% or higher agreement on all questionnaire items. Reference Han and Chun19 The DPET-Arabic and the Iranian-DPT scales also evaluated the content validity, and positive expert opinions were reported in terms of items and content. Reference Al Khalaileh, Bond, Beckstrand and Al-Talafha20,Reference Tavan, Menati, Azadi, Sayehmiri and Sahebi22
The DPET-K translation study used the disaster nursing tool to ensure criterion validity. The correlation between the items of the scales ranged from r = 0.481 (P <.001) to 0.740 (P <.001). Reference Han and Chun19 External criterion data to ensure criterion validity were missing in the other scales. Reference Chen, Chou, Liao, Ho and Chung17–Reference Tavan, Menati, Azadi, Sayehmiri and Sahebi22
Structural validity analyses of the DPET-C, the DPET-MC, and the DPET-K translations of the DPET scale gave positive results for the five-factor structure, while for the three-factor structure, the same analysis of the DPET-Arabic gave positive results. The five-factor structure of DPET-C explained 65.13% of the variance. Kaiser-Meyer-Olkin (KMO), Bartlett Test of Sphericity v2, degrees of freedom (df), and P value were found to be 0.96, 2439691, 0.703, and <.0001, respectively. Reference Chen, Chou, Liao, Ho and Chung17 Percentage of the total variance explained was 64.06% in the DPET-MC, 66.94% in the DPET-K, Reference Wang, Lu and Sun18,Reference Han and Chun19 64.00% in the DPET-Arabic, Reference Al Khalaileh, Bond, Beckstrand and Al-Talafha20 and 67.57% in the seven-factor structure of the Iranian-DPT. Reference Tavan, Menati, Azadi, Sayehmiri and Sahebi22
With regard to cross-cultural validity, several arrangements were made on all scales on such issues as the number of items, item splitting, and the contents of the dimensions. The DPET-Arabic and the DPET-MC studies used the guidelines for the process of cross-cultural adaptation of self-report. Reference Beaton, Bombardier, Guillemin and Ferraz23 The DPET-K scale used the instrument development guidelines by DeVellis. Reference DeVellis24 The Chinese-HNDDPS scale performed the cross-cultural adaptations in five aspects. Reference Shi, Li, Xu, Wang, Li and Feng21 The DPET-C scale used the adaptation of Brislin’s translation model Reference Jones, Lee, Phillips, Zhang and Jaceldo25 and a second group as a cross-sample for comparison. Cross-validity was performed and the structure of the model was observed to exhibit an adequate fit for both groups (P >.05), indicating an adequate cross-validity. In other scales, however, multiple group factor analysis was not performed to ensure cross validity.
The studies used Cronbach’s alpha coefficient for reliability analyses, in which Cronbach’s alpha values ranged from 0.785 to 0.97. Unlike the other scales, Cronbach’s alpha coefficients of the Iranian-DPT scale’s sub-dimensions (knowledge, attitude, and practice) were <0.70, which were 0.65, 0.61, and 0.58, respectively. Reference Chen, Chou, Liao, Ho and Chung17–Reference Tavan, Menati, Azadi, Sayehmiri and Sahebi22
The scales addressed three disaster phases (prior to, during, and after a disaster) under varied dimensions. In the adaptation studies of the DPET scale, sub-dimensions included the pre-disaster, response, and recovery phases. Reference Chen, Chou, Liao, Ho and Chung17–Reference Al Khalaileh, Bond, Beckstrand and Al-Talafha20 The Iranian-DPT scale measured preparedness at the level of knowledge, attitude, and practice, whereas the Chinese-HNDDPS scale measured preparedness at the level of concerns, obstacles, action, and perception. Reference Shi, Li, Xu, Wang, Li and Feng21,Reference Tavan, Menati, Azadi, Sayehmiri and Sahebi22 However, the scales lacked comprehensive data on the action required at the three phases of a disaster in the context of external and internal levels specified in the WHO Tool Kit for Disaster Preparedness for Nursing and Midwifery. 15
The evaluation of the qualities of the psychometric properties of the scales is presented in Table 2. Based on the evaluation, all scales were rated sufficient in content validity, structural validity, and cross-cultural validity. One scale was rated sufficient in criterion validity while the others were rated indeterminate. One scale was rated insufficient in reliability and internal consistency while the others were rated sufficient. Reference Mokkink, de Vet and Prinsen14,Reference Terwee, Prinsen and de Vet26,Reference Prinsen, Vohra and Rose27
Quality of Psychometric Properties of Disaster Preparedness Tool in Nurses
Note: “+” = sufficient, “–” = insufficient, “?” = indeterminate. Not all information for “+” reported. According to COSMIN risk of bias checklist.
Abbreviations: Chinese-HNDDPS, Chinese Hospital Nursing Department Disaster Preparedness Scale; DPET-Arabic, Classical Arabic Disaster Preparedness Evaluation Tool; DPET-C, Chinese Disaster Preparedness Evaluation Tool; DPET-K, Korean Disaster Preparedness Evaluation Tool; DPET-MC, Mainland China Disaster Preparedness Evaluation Tool; Iranian-DPT, Iranian Nurses’ Disaster Preparedness Scale.
Discussion
Disaster preparedness of nurses is a growing research field in the international context. The results obtained from this study analyzing the tools used to measure disaster preparedness of nurses has shown the need for new and comprehensive tools with sufficient psychometric properties and the capability of revealing the preparedness of nurses under real disaster situations.
A measurement tool to be developed to measure the disaster preparedness of nurses is expected to include mitigation/prevention, preparedness, response, and recovery/rehabilitation phases and cover them in the context of external and internal action levels. 15,28 The scales reviewed within the scope of the present study examined the disaster preparedness under different dimensions but with similar titles and addressed the disaster phases (Table 1). However, the scales lacked comprehensive content regarding the impact of disasters on individuals, communities, and health systems in the context of internal and external action. Researchers should consider these content areas in their future studies of scale development.
The Likert scale, which indicates varying degrees of agreement with and endorsement of the declarative sentence, is the most widely used scoring method in assessment tool study. Reference DeVellis24 A six-point Likert scale was used in the DPET scales while three-point and five-point Likert scales were used in the Iranian-DPT and the Chinese-HNDDPS scales, respectively. The item number of the scales varied (Table 1) in the studies included in this review, showing that the item numbers of the scales are compatible to be a reliable scale. It is suggested that the number of items should be more than 30 to develop a reliable scale. Reference Carmines and Zeller29–Reference Özdamar31
A standard translation process is needed to ensure both linguistic and cultural adaptation in the translation of measurement tools developed in different languages. Reference Beaton, Bombardier, Guillemin and Ferraz23 The translation process should be evaluated accurately and rigorously. Reference Hoben, Mahler and Bar32,Reference Squires, Aiken and van den Heede33 Authors who will carry out cross-cultural scale adaptation studies adhere guidelines to conduct the process thoroughly. The scales complied with the guidelines for the process of cross-cultural adaptation. The adaptation studies of the scales examined in the study used various guidelines and models, including the guidelines for the process of cross-cultural adaptation of self-report, Reference Beaton, Bombardier, Guillemin and Ferraz23 Brislin’s translation model, Reference Jones, Lee, Phillips, Zhang and Jaceldo25 and the instrument development guidelines presented by DeVellis. Reference DeVellis24 These guidelines, which increase the generalization capacity of the collected data, allow authors to research the similarities and differences between the societies where the measurements are carried out. Reference Capık and Gözüm34
The reviewed studies discussed structural validity within the scope of construct validity. Structural validity also has expressed the relationships between the questions in the measurement tool. Each item in the measurement tool should have integrity with all other items. Reference Terwee, Prinsen and de Vet26 Structural validity was sufficient in all the scales. The DPET-Arabic study used factor analysis to test the construct validity of the scale, however, no CFI (Comparative Fit Index) or TLI (Tucker–Lewis Index) or comparable measure >0.95 or RMSEA (Root Mean Square Error of Approximation) <0.06 data were found, which are used for testing structural validity. Reference Mokkink, de Vet and Prinsen14 The results of the other studies have yielded stronger evidence for structural validity than did the DPET-Arabic. More evidence about structural validity should be presented in future studies so that the relationships between the questions in the measurement tool can be expressed more clearly.
The studies used internal consistency to measure reliability. The scales demonstrated appropriate and acceptable internal consistency (Cronbach’s alpha(s) ≥ 0.70). However, the Cronbach’s alpha value for the sub-dimensions of the Iranian-DPT scale was below 0.70 while its total internal consistency was 0.785. A low alpha level in the scale sub-dimensions requires some items to be reviewed and, if necessary, removed. Reference Mokkink, de Vet and Prinsen14 If researchers bear this in mind in future studies, this will help to increase the power of the items of a scale, as interrelated, to measure the same construct and ensure the items on a scale have a certain conceptual structure.
The COSMIN criteria require that for the criterion validity to be sufficient, the correlation with gold standard ≥ 0.70 must be met. Reference Mokkink, de Vet and Prinsen14 The criterion validity was investigated in detail only in the DPET-K study, in which this ratio was 0.74. Reference Han and Chun19 Examining the relationship between the results obtained from the used measurement tool and those from other measurement tools are important data for seeing the validity of a study. The lack of a gold standard to be used to compare nurses’ preparedness for disasters may have prevented the criterion validity from being tested in the rest of the studies.
Content validity is defined as the most important measurement property among the psychometric properties. Reference Mokkink, de Vet and Prinsen14 The information provided for content validity in the DPET-MC, the DPET-K, and the Iranian-DPT studies were sufficient, however, study results should present more comprehensive evidence for the design and analyses of content validity. This is needed to perform an effective evaluation regarding the relevance, comprehensiveness, and comprehensibility of scale items. Reference Terwee, Prinsen and de Vet26
Limitations
The studies included in this systematic review are limited to the tools used to measure the disaster preparedness of nurses. The exclusion of grey literature may have caused publication bias. Not all types of disaster preparedness assessment tools/emergency preparedness tools were analyzed or included in the study. Because the study only included articles in English, it is recommended that future studies look into articles published in other languages.
Conclusions and Implications
Scales assessing the disaster preparedness of nurses have both sufficient and insufficient aspects. Therefore, there is a need to develop and validate a comprehensive assessment tool to evaluate the disaster preparedness of nurses.
This examination of the scales developed or adapted to determine the disaster preparedness of nurses may be guiding for researchers to develop eligible scales. The results of the present study may be considered as a reference that will contribute to the scale development and adaptation studies on disaster preparedness in the international field.
Conflicts of interest/funding
No conflicts of interest have been declared by the author. The author declares that there is no funding associated with this project.
