Disasters that cause great losses, limit the functions and abilities of society in various aspects, and pose a serious threat to society have occupied an important place in human life since ancient times.1 Due to reasons such as climate change, distorted urbanization, poverty, environmental degradation, and global changes in recent years, the frequency and size of disasters have been increasing worldwideReference Kalanlar2, which is a source of concern at both national and international levels.Reference Park and Kim3
While 308 natural disasters were reported worldwide in 2019, resulting in 24,396 deaths and affecting the lives of 97.6 million people4, these figures reached 387 natural disasters, resulting in 30,704 deaths and affecting 185 million people in 2021.5 Turkey is a country with a high risk of humanitarian crisis and disaster, particularly with large-scale earthquakes (Marmara earthquake in 1999, Van earthquake in 2011, Elazığ earthquake in 2020, and Maraş earthquake in 2023 [twin earthquakes]), floods, mining accidents, and terrorist attacks.6, Reference Taşkıran and Baykal7 Following these major disasters in many regions of the world and in Turkey, the importance of disaster preparednessReference Park and Kim3, Reference Taskiran and Baykal8 and the necessity of successful disaster management have started to be emphasized.Reference Barrimah, Adam and Al-Mohaimeed9, Reference Veenema10 At this stage, nurses who are equipped with comprehensive knowledge and skills in disaster management and have developed competency for a rapid and coordinated response are needed in disasters.Reference İytemür and Yeşil11, Reference Zhou, Turale and Stone12
Disaster management is defined as “organization, planning, and implementation of disaster preparedness, disaster response and disaster recovery measures”.13 The objective of disaster management is to reduce or prevent losses caused by hazards and provide rapid assistance and effective recovery to the affected populations and communities.14 In this context, the World Disaster Nursing Institute was established in 2008, and then ICN announced the qualifications for disaster nursing in 2009. The ICN emphasizes that every nurse should have core competencies for planning and implementing disaster care and preparedness, as well as managing the disaster process.15 According to ICN (1997), competency is defined as “a level of performance that demonstrates the effective application of knowledge, skills and judgment”16. In the ICN report of 2009, the competency roles of nurses in disaster prevention and mitigation, preparedness, response, and recovery are described in four areas.Reference Thobaity, Plummer and Williams17 However, it is seen that these roles have been inadequate in disasters experienced over years, and the ICN has defined eight roles for the core competencies of specialist nurses in 2019. These roles include preparedness and planning, communication, incident management systems, safety and security, assessment, intervention, recovery, law, and ethics.18 It can be concluded that these roles also include the disaster management process. In order to develop these roles, nurses’ awareness of disaster risks and hazards should be increased, and their knowledge, skills, and competencies should be improved to adapt and manage during disasters.Reference Stoto, Savoia and Nelson19–Reference Said and Chiang21 Turkey has experienced and continues to experience many types of disasters such as earthquakes, floods, landslides, fires, and terrorist attacks.6 The recent Kahramanmaraş earthquake on February 6 2023 affected approximately 14 million people in 11 provinces.22 Such major disasters reveal the importance of nurses’ competencies in disaster management as well as in basic practices. Although the foundations of the disaster management system in Turkey were established legally and institutionally in the 1940s, recent disasters have highlighted numerous deficiencies in the interventions and outcomes of the system. While the Nursing Law enacted in 1954 and updated in 2007 defines the powers of nurses, it does not address disaster nursing. Only among the duties of nurse managers is the following statement found: “In extraordinary circumstances, in accordance with disaster plans, collaborates with relevant units, develops or ensures the development of emergency plans, develops protocols, and/or ensures their development, and prepares the team for implementation when necessary”. 23 Additionally, all hospitals nationwide are required to develop their own Hospital Disaster Plans.24 In hospital disaster plans, the nurse manager, nurses responsible for intensive care services, emergency room nurses, and operating room nurses are included in the team. Furthermore, the responsibilities of the responsible nurse and nurses are specified in hospital disaster plans. It is crucial for nurses to be prepared for disasters and to be able to perform their duties outlined in these plans in order to mitigate disaster damages effectively.
Although the number of disasters continues to increase all across the world and national and international nursing organizations and the World Health Organization have repeatedly warned nurses for disaster preparedness, it is seen that the disaster nursing training is at inadequate levels in many countries and there are limited number of studies on this subject.Reference Taskiran and Baykal8 In these warnings, there is more emphasis on issues related to the current level of competency, skills, and experience among nurses in disaster preparedness and response15 and it is stated that every nurse should have relevant competencies, update their knowledge, and participate in drills while maintaining at least a basic level of professionalism.18 However, in the ICN’s 2019 report, in addition to the competencies for response/implementation in the core competence roles of nurses, issues related to disaster management such as communication, incident management systems, safety and security, assessment, recovery, law, and ethics are also mentioned. However, in the literature review, no tools have been utilized that measure competencies related to disaster management. The studies generally focus on basic practices (I can assess airway patency and respiration, I can apply urinary catheter etc.) and disaster preparednessReference Kalanlar2, Reference Taşkıran and Baykal7, Reference Taskiran and Baykal8, Reference Glow and Colucci25–Reference Alan, Eskici and Sen29, using tools to measure only these competencies.Reference Celik30–Reference Aliakbari, Aein and Bahrami33 It is of great importance to determine the competencies of nurses related to disaster management and to plan appropriate training programs. However, no study has been found conducted on disaster management. Therefore, in this study, we aimed to introduce the Nurse Competency Assessment Scale in Disaster Management into the literature through evaluating its psychometric properties.
Methods
Study Design
The aim of this study was to develop the “Nurse Competency Assessment Scale in Disaster Management (NCASDM)”, a new tool for measuring nurses’ competencies related to disaster management, and to test its psychometric properties. Three basic scale development stages, which are recommended in the literature and widely used, were followed in order to create the scale.Reference Boateng, Neilands and Frongillo34, Reference DeVellis and Thorpe35 An item pool was created in the first stage, then a preliminary evaluation of the items was performed, and the psychometric properties of the draft scale were evaluated in the third stage (figure 1).
Procedures
Stage 1. Creating the Item Pool
Qualitative research method was used at this stage.Reference DeVellis36, Reference Carpenter37 The scale items were obtained by conducting online, semi-structured, in-depth individual interviews with 27 nurses who experienced disasters or worked in disasters. Among the participants who were interviewed, 21 were female, 16 were undergraduate graduates, and 20 were working in state hospitals. The average number of years of professional experience was calculated as 14 (3-32) years. The participants had worked in different disasters such as earthquakes, floods, fires, and terrorist incidents or had experienced these disasters themselves. Interviews including disaster, disaster management stages (pre-disaster, disaster moment, and post-disaster), and disaster nursing services management were evaluated using content analysis. An item pool was created based on these interviews and the guidelines proposed in the literature.Reference Onwuegbuzie, Bustamante and Nelson38, Reference Zhou39 At this stage, the following steps were followed respectively: the qualitative data obtained from the interviews were transcribed; and the transcripts were read repeatedly by the researchers, and important quotations directly related to the nursing and disaster management stages were independently determined. Each statement obtained from the transcripts was examined in terms of its explicit and hidden meanings. The quotations reflecting the nurses’ opinions and views on disaster management were corrected and converted into general attitude statements. Researchers followed the nurses’ definition of disaster management when determining the expressions to be included in the item pool. The researchers held three different meetings to evaluate the citations and attitude statements and to decide on the items. While making the decision, criteria such as whether the items evaluated disaster management, were comprehensible, focused on nurses, and could be responded in a short time were taken into consideration. As a result of the meetings, an item pool consisting of 67 items was created. Two of these meetings were attended by a person experienced in disaster nursing studies and an expert in the field of measurement and evaluation. The researchers created a 5-point Likert-type scale with the response options “1: Strongly Disagree”, “2: Disagree”, “3: Neutral”, “4: Agree” and “5: Strongly Agree” through revising the expressions in the scale.
The items were clearly and briefly expressed.
Stage 2. Preliminary Evaluation
Content validity
A total of 15 experts from the fields of nursing management, disaster management, public health, and measurement and evaluation were asked to assess the scope validity of the scale according to the classification determined by Davis.Reference Davis40 The experts consisted of academicians with at least a PhD degree in nursing management (12), disaster management (1), public health (1), and measurement and evaluation (1). The Item-Content Validity Index (I-CVI) was calculated for each item, and the Scale-Content Validity Index (S-CVI) was calculated for the entire scale. Each item was evaluated using the quadruple rating form proposed by Davis (1992 ) (1 = the item is not relevant, 2 = the item needs major revision, 3 = the item needs minor revision, and 4 = the item is relevant). The number of experts who assigned 3 and 4 points to determine I-CVI and S-CVI was divided by the total number of experts (Davis, 1992 )40. According to this assessment, the content validity index of a newly developed measurement tool should be at least 0.80.Reference Davis40, Reference Grant and Davis41 According to the expert opinion, seven items with an I-CVI <0.80 were removed. It was decided that 2 items below I-CVI <0.80 were to be revised in accordance with expert opinions and remain on the draft scale. The item “I can cope with more than one situation in disasters” was revised as “I cope with many problems that arise at the time of a disaster” and the item “I participate in the unit-specific planning in health care requirements” was revised as “I act in accordance with the changing health care requirements at the time of a disaster." The content validity index of the remaining items was found to be >0.80. A draft scale of 60 items was created at the end of this scale.
Pilot study
After obtaining an expert opinion, a revised scale should be applied to a group with similar characteristics as the sample.Reference Epstein, Santo and Guillemin42 The aim here is to assess the comprehensibility of the items. In this study, the draft scale, which was created after the necessary corrections were made based on expert opinions, was applied to 25 nurses with similar characteristics to the target group to check the comprehensibility, readability, and response errors. According to the feedback of the participants, it was found that 9 items had poor comprehensibility. These items were revised in accordance with the opinions of the researchers and a measurement and evaluation expert, and the final form of the 60-item draft scale was achieved.
Stage 3. Psychometric Evaluation
Item reduction
The item-total correlation values of each item were evaluated before the factor analysis. It is recommended to remove items with an item-total correlation coefficient below 0.40 and which increase the total Cronbach’s alpha value of the scale when removed.Reference Field43, Reference Kartal and Bardakçi44 In this study, the item reduction process was not performed because there were no items with an item-total correlation coefficient below 0.40.
Construct validity
First, in the scale development process, exploratory factor analysis (n = 697) was applied to determine the construct validity of the scale, and then confirmatory factor analysis (n = 348) was applied to test the verification of this new factor structure.Reference Brahma45, Reference Yaşlioğlu M46 EFA was conducted with principal component analysis and direct Varimax orthogonal rotation test to determine the factor structure of the scale. Principal component analysis, which is one of the best estimation methods of EFA, was chosen to capture the maximum variance and to identify the main factors by simplifying complex data.Reference Alavi, Visentin and Thapa47 It is recommended that factors with eigenvalue coefficients greater than 1 obtained in EFA should be included, items should not have cross-loading, factor loadings should be higher than 0.30, there should be at least three items in each factor, and the explained variance ratio should be at least 50%.Reference Boateng, Neilands and Frongillo34, Reference DeVellis and Thorpe35, Reference Carpenter37 Five stages were followed respectively as required in the EFA for construct validityReference Şencan48: (1) evaluating the suitability of the data set for factor analysis (Barlett’s Test), (2) calculating the sample size value (KMO Test), (3) obtaining factors (Factor Loads; Eigenvalue, Explained Variance), (4) reversing factors, and (5) naming factors.
The factor structure obtained as a result of EFA was tested with CFA. CFA is used in scale development and validity analysis and aims to determine the accuracy of a predetermined structure. In addition to representing variables and factors, CFA uses path analysis diagrams to try to confirm hypotheses.Reference Child49 The following tests were used to evaluate the model: x2/sd (Adjusted Chi-Square Statistics), RMR (Root Mean Square Residual), GFI (Goodness of Fit Index), CFI (Comparative Fit Index), NNFI (Non-Normalized Fit Index), NFI (Normalized Fit Index), IFI (Increasing Fit Index), RFI (Relative Fit Index), and RMSEA (Root Mean Square Error of Approximation). Convergent and discriminant validity of the scale was assessed and the average variance was extracted (AVE); composite reliability (CR) were reported.
Reliability
Cronbach’s alpha coefficient, item-total score correlation analysis, and test-retest analysis were performed to determine the reliability of the scale. The t test was applied in independent groups to determine the distinctiveness of questions in lower and upper groups. In evaluation criteria, Cronbach’s alpha was >.60Reference Sport50, item-total score correlation was >0.40Reference Çokluk, Şekercioğlu and Büyüköztürk51, and In-Class Correlation Coefficient (ICC) was >0.70Reference Weir52.
Participants
It is recommended that the minimum sample size should be 300 for scale development studies.Reference Tabachnick and Fidell53, Reference Worthington and Whittaker54 Studies conducted with large samples indicate fewer data errors and more effective factor analysis.Reference Yong and Pearce55 Therefore, it is recommended to reach the possible largest sample.Reference Park and Kim3 6, Reference Tabachnick and Fidell53 In this study, it was aimed to reach a sample size of 5-20 times the number of scale items, which is the most widely used method in the literature for scale validity and reliability studies.Reference Talbot56
Each stage of the research was carried out on different sample groups. A total of 27 nurses who had experienced or worked in disasters participated in the item pool stage, 25 nurses participated in the pilot study, 697 nurses participated in the item reduction and EFA, 348 nurses who were randomly selected from the EFA stage participated in the CFA and internal consistency, and 40 nurses participated in the test-retest. It is considered best to use different samples for CFA analyses (whenever available); however, using the same sample is also a common practice. It is important to split the data set and ensure adequate sample size in case of requiring the same sample. In this study, the sample size was also split.Reference Brown57, Reference Newsom58 During the creation of the scale items, 27 nurses were included using the snowball sampling method. Nurses working under the “Turkey Disaster and Emergency Management Authority” (AFAD) were identified as resource persons to reach potential participants. The other participants were reached in accordance with the recommendations of nurses who had experienced a disaster or worked in a disaster and agreed to participate in the study. At each step of the second and third stages, where the scale items were pre-evaluated and psychometric properties were tested, the nurses were determined using the convenience sampling method. Having at least 1 year of professional experience was determined as inclusion criteria.
Data Collection
At the first stage of the research (creation of the item pool), preliminary interviews were conducted with nurses known to the researchers working at AFAD to identify nurses for individual in-depth interviews. These participants from all over Turkey were reached using snowball sampling method. Preliminary interviews were conducted over phone, information was given about the purpose and content of the research, and the appropriate day and time for the interview were planned by taking the informed consents of those who volunteered to participate. Individual in-depth interviews were conducted remotely because the participants and the researchers were living in different cities. All of the interviews were recorded. Interviews were conducted between June 2021 and March 2022, and lasted an average of 42.7 minutes (min = 14 max = 108).
The data for the second and third stages of the research (item quality assessment and testing the psychometric properties of the scale) were collected in four hospitals (two private hospitals, a university hospital and a ministry of health hospital) in a province located in the south-west of Turkey after obtaining institutional permissions. The study questionnaire was created in the online program because it was found to be more convenient to collect data online by nursing service managers. The questionnaire created online was shared by the administrative nurses with all nurses through the communication network of the institution (e-mail, WhatsApp, etc.). The “Informed Consent Form” was added to the first page of the questionnaire and the participants were able to proceed to the questions after providing their approval. The response time of the data collection tool was 10-15 minutes for each participant. The data were collected between January and May 2023.
Measures
The questionnaire consisted of three sections. An informed consent form was used in the first section. Here, the nurses were informed about the purpose, scope and ethical aspects of the study. The second section included a personal information form (ten questions about age, gender, marital status, educational status, years of service in the profession, years of service in the institution, unit, institution, disaster training status, and type of training). The third section included the “Draft- Nurse Competency Assessment Scale in Disaster Management.” The scale is rated on a 5-point Likert-type scale.
Data Analysis
The qualitative data obtained from the interviews conducted for the creation of the scale items were evaluated by the researchers using content analysis. After the scale data were transferred to the computer environment by the researchers, they were analyzed with IBM SPSS Statistics 22 and Amos 22 programs with the support of an expert consultant in the field of statistics. The characteristics of the participants and the scale scores were determined by descriptive statistics (number, percentage, average, standard deviation). I-CVI and S-CVI were calculated with the classification proposed by Davis (1992) to determine the content validity of the items. Item-total score correlations were calculated by using Pearson correlation analysis. In order to evaluate the construct validity of the scale, EFA and CFA were performed. The EFA extraction method was principal component analysis, and the rotation method was the Varimax orthogonal rotation test. In the reliability test, the internal consistency of the scale was determined by calculating the item-total/subscales correlation and Cronbach’s alpha coefficient. Item distinctiveness was evaluated with a 27% lower and upper quarters t test. The temporal stability of the scale was assessed using the test-retest method (ICC). In addition, “Average Variance Extracted (AVE)” was calculated for the convergent validity of the items and “Composite Reliability (CR)” was calculated to assess the convergent validity. In addition, “average variance extracted (AVE)” was calculated for convergent validity and “composite reliability (CR)” was calculated to assess convergent validity. Good fit and acceptable fit values of confirmatory factor analysis were given in Table 1.
Validity and reliability procedure.
Confirmatory factor analysis values
Ethical Considerations
Official permission was obtained from the hospitals where the research was carried out. The nursing staff who took part in the study provided informed consent. All procedures included in this study complied with the principles of the Declaration of Helsinki. The study was approved by Mersin University Clinical Research Ethics Committee (date: 28.04.2021; no: 2021/337)
Results
General Characteristics of the Participants
It was determined that the ages of the 697 nurses participating in the study ranged between 17-60 and the average age was 34.3 ± 8.6; the majority of the participants were women (79.6%), held bachelor’s degrees (69.7%), and were married (59.5%). When the occupational characteristics of the nurses were examined, it was found that most of them had 1-5 years of professional experience (43.3%); 43.2% were working in specialized units, 50% were working in state hospitals, and 69.2% received disaster training (60.4% of those who received disaster training received only theoretical background). The characteristics of the participants are given in detail in Table 2.
Demographic characteristics (N = 697)
Content Validity
Items were submitted to 15 experts in order to evaluate content validity and item relevance was assessed. Seven items with an I-CVI value below 0.80 were removed from the scale. As a result, the S-CVI value was 0.95, and the I-CVI values were in the range of 0.80-1.00. These values indicated good content validity.
Pilot Application
The scale was applied to 25 nurses with similar characteristics to those in the main sample group. The implementation was made face-to-face to ensure that it served its purpose. Feedback was received on whether the items were clear and comprehensive. After this stage, several expressions that were considered to challenge comprehensibility in the draft scale were changed. For instance, the question “I take part in the creation of hospital policies and procedures specific to disaster situations” was changed to “I take part in the creation of disaster-related policies and procedures in the hospital;” the question “I take part as a trainer in disaster trainings conducted in the hospital when necessary” was changed to “I take part as a trainer in disaster trainings planned in the hospital;” and the question “I follow the instructions of the team leader in case of disaster” was changed to “I follow the instructions of the incident management team in case of disaster.”
Item Reduction
The item-total correlation values of the items ranged between 0.64 and 0.91 prior to the EFA. Because the item-total correlation coefficient of all items was above 0.40, no items were removed from the scale.
Construct Validity
First of all, the EFA was conducted to evaluate the validity of the structure, followed by the CFA. The compatibility of the data to factor analysis was evaluated before the EFA. The data and sample were found to be compatible for factor analysis (KMO = 0.980, Bartlett Sphericity Test = 64572.171, df = 1770, P < 0.01).
Factor analysis was applied with 60 items after the item analysis of the draft scale. The distribution of the items according to the factors was analyzed using Basic Components Analysis and Varimax Vertical Rotation Test. A total of 17 items with a factor load value below 0.40 and a variation of 0.10 or less between at least two factors were removed from the scale and the analysis was repeated. As a result of the analysis, a two-factor structure was obtained, and it was determined that these factors collectively explained 79.10% of the total variance in the scale scores (Factor 1 = 25.22%; Factor 2 = 53.87%). The researchers evaluated whether the items in the factors were grouped logically. As a result of the evaluations, the factors were grouped as (1) disaster preparedness (9 items) and (2) disaster response (34 items). There were no reverse scored items in the scale (Table 3).
Exploratory factor analysis results NCASDM (N = 697)
After the two-factor structure of 43 items was obtained using the EFA, the CFA was applied to test the construct validity of the scale. Assuming that the scale consisted of two interrelated subscales, the fit indices were evaluated to determine the level of fit between the data. No changes were made to the model and the fit indices showed acceptable or good fit (χ Reference Kalanlar2/df = 3.261; NNFI = 0.92; RMSEA = 0.059; GFI = 0.92; RFI = 0.95; NFI = 0.95; CFI = 0.925; IFI = 0.93; RMR = 0.07) (Table 4). The CFA model road diagram for the scale is given in Figure 2.
Confirmatory factor analysis fit indices (N = 348)
CFI, Comparative fit index; GFI, Goodness-of-fit index; IFI, Incremental fit index; NFI, Normed fit index; NNFI, Non-normed fit index; RFI, Relative fit index; RMSEA, Root mean square error of approximation.
Path diagram for Nurse Competency Assessment Scale in Disaster Management
In this study, the convergent and discriminant validity of the scale was tested. The standardized factor loads of all items were above 0.50 and were statistically significant. AVE values of the subscales were found to be between 0.87 and 0.92, and CR values were found to be between 0.94 and 0.96 (Table 5). CR values of the subscales were found to be between 0.87 and 0.92.
Reliability analysis of NCASDM, AVE, CR values
M: Mean; SD:Standart eviation, AVE: Average varience extracted, CR: Composite reliability
Reliability Analysis
The item-scale correlation coefficients ranged between 0.638 and 0.907. The item-subscale correlation coefficients were 0.638–0.779 for disaster preparedness and 0.822–0.907 for disaster response. Cronbach’s alpha coefficient of the 43-item NCASDM was 0.99. Cronbach’s alpha coefficients of the subscales varied between 0.96 and 0.99 (Table 5).
The test-retest method was used to determine the temporal reliability of the scale. The NCASDM was applied twice to 40 nurses in 2 weeks of intervals. ICC was calculated by comparing the scores obtained from the test-retest. It was found that there was no statistically significant difference between the mean scores obtained from the two measurements (t = –0.492; P = 0.63, P > 0.05). It was found that there was a statistically positive, strong, and significant correlation between the mean scores obtained from both measurements (r:0.95, P < 0.001). The ICC was calculated by comparing the scores obtained from the test-retest. The ICC value of the total average scale was 0.98 (Table 6). It was determined that the difference between the total score average of the general NCASDM and subscales and 27% lower (n = 188) and upper (n = 188) quarter groups was statistically significant (P < 0.001) (Table 7).
Test–retest results of NCASDM (N = 40)
M: Mean; SD:Standart eviation; ICC, İntraclass correlation coefficient
Comparison of the NCASDM mean scores for the lower and upper groups of 27%.
Scale Information
NCASDM consists of 43 items and two subscales: disaster preparedness (9 items) and disaster response (34 items). It is a 5-item Likert-type scale that evaluates nurses’ competencies related to disaster management, including 1 = strongly disagree and 5 = strongly agree responses. The total score and subscale scores of the scale are evaluated based on the average score. The average score of the NCASDM and its subscales varies between 1 and 5. An overall and subscale average score approaching 1 suggests that nurses are inadequate for disaster management, while they are adequate in scores approaching 5. In this study, the average total score of the NCASDM was calculated as 3.40 (±1.199), and the average total score of the subscales was calculated as 2.94 (±1.269) for disaster preparedness and 3.53 (±1.181) for disaster response.
Discussion
In this methodological study, NCASDM was developed to assess the competencies of nurses related to disaster management, and its validity and reliability were tested. The literature review was performed to detect previously developed and similar measurement tools. It was concluded that the current scales focus on the role of nurses in disasters, their disaster preparedness, and disaster competencies.Reference Celik30–Reference Aliakbari, Aein and Bahrami33 However, in the disaster process, in addition to acute care, nurses should also be involved in disaster management in all stages before, during, and after the disaster.Reference Tzeng, Feng and Cheng26, Reference Evans and Baumberger-Henry59 Particularly after COVID-19, which affected the whole world, strained health systems caused great loss of life and demands for nurses’ competencies in disaster management, including knowledge and skills to provide appropriate care during disasters, increased.Reference Jackson, Bradbury-Jones and Baptiste60 Therefore, it was seen that there was a need for a valid and reliable measurement tool developed to assess the competencies of nurses related to disaster management in disaster situations.
Qualitative interviews were conducted with nurses who took part in a disaster or experienced a disaster to create a draft item pool, which is one of the most basic stages of scale development. As a result of the data analysis obtained from the interviews, a draft scale of 67 items was created. The draft scale was submitted to expert opinion and the content validity was evaluated with the classification proposed by Davis.Reference Davis40 It is recommended that I-CVI values should be above 0.78 and S-CVI values should be above 0.80 for item-level content validity in a draft scale.Reference Davis40, Reference Polit, Beck and Owen61 At this stage, in accordance with the recommendation of experts, 7 items with low item-level content validity were removed.Reference Davis40 It can be concluded that the remaining items had good content validity and adequately represented the structure.
In the scale development study, it is recommended to evaluate the construct validity by performing EFA and then CFA.Reference Worthington and Whittaker54, Reference Schmitt62 The first step in EFA is to evaluate the compatibility of the data for factor analysis. In this study, the KMO coefficient (>0.70) and Bartlett sphericity test (P < 0.001) showed that the data showed conformity with normal distribution and that the sample size was perfectly suitable for factor analysis.Reference Boateng, Neilands and Frongillo34, Reference Carpenter37, Reference Yaşlioğlu M46 According to the EFA results, the two-factor NCASDM explained 79.10% of the total variance, and this rate was higher than the 50% recommended in the literature.Reference Carpenter37 In addition, the factor load values were above the minimum acceptable value (>0.40).Reference Boateng, Neilands and Frongillo34 These values indicate that the structures obtained as a result of the EFA can comprehensively evaluate the competencies of nurses related to disaster management.
It is recommended to use the CFA and report compliance indices to test whether this factor structure obtained from the EFA is adequate to explain the model.Reference Kline63 In this study, the compliance indicators were also above the minimum acceptable values (x2 /sd = >2; RMR = ≥0.05; CFI = ≥0.90; RMSEA = ≤0.05; GFI = ≥0.90; RFI = ≥0.90; NFI = ≥0.90; IFI = ≥0.90).Reference Kline63, Reference Wang and Wang64 These CFA results confirmed that nurses’ competencies related to disaster management included a two-dimensional structure as disaster preparedness and disaster response. At this stage, some items were removed from the scale because they compromised the factor structure of the scale or were loaded in two or more factors with a difference of less than 0.10 and had a low distinctive power.Reference Çokluk, Şekercioğlu and Büyüköztürk51 In addition, the convergent and distinctive validity results of the scale were also at an acceptable level.Reference DeVellis36, Reference Bagozzi and Kimmel65 These findings support that NCASDM has a good level of construct validity and is a valid measurement tool.
After the construct validity, our 43-item scale consisted of a two-factor structure evaluating the competencies of nurses regarding disaster management. When the items allocated under Factor I, II were evaluated by the researchers, they were named as “disaster preparadness” and “disaster response.”
The first factor is related to disaster preparedness, which consists of 9 items and explains 25.22% of the total variance. This factor includes statements such as disaster trainings, drills, duties, and responsibilities in disaster plans, and material and drug management in the pre-disaster process. It can be said that the items in this factor are compatible with the items in the ICN 2019 disaster nursing core competency roles.18 In addition, it is seen that other scales frequently used in the literature include items related to the basic professional knowledge and skills of nurses in disaster preparedness and basic level of disaster nursing competency in disaster preparedness.Reference Celik30–Reference Aliakbari, Aein and Bahrami33
The second factor consists of 34 items explaining 53.874% of the total variance. This factor includes items related to communication, incident management systems, safety and security, assessment, intervention, recovery, law, and ethics during and after the disaster. This factor is compatible with the items in the ICN 2019 disaster nursing core competency roles. In the scale developed on this subject, under the heading of disaster response competency, the ICN 2009 disaster nursing core competency roles include care requirements and core nursing practices under the heading of response (safe care, individual care and family care, collaborative care, psychological care, and care of disadvantaged groups).Reference Ahn, Jang and Kim32
The reliability of the scale was evaluated by item – total score correlation analysis, Cronbach’s alpha internal consistency coefficient, test-retest method, and t test in independent groups (lower and upper groups of 27%). It is expected that Cronbach’s alpha value is above 0.60 and item-total score correlation values are above 0.30 in the evaluation of internal consistency.Reference Field43, Reference Tabachnick and Fidell53 As a result of the reliability analysis, the Cronbach’s α value of NCASDM and its subscales was found to be above 0.90, while all item-total score correlations were above 0.40 and the ICC value was above 0.70. The presence of a significant difference between the lower and upper groups of 27% (P < 0.05) also supports the internal consistency of this scaleReference Büyüköztürk66. These results have shown that the scale is quite reliableReference Sport50 and stableReference Weir52, and that the items correctly distinguish the participants.Reference Çokluk, Şekercioğlu and Büyüköztürk51 All these psychometric analyses have shown that NCASDM is a valid and reliable measurement tool.
Limitations
Although this study was prepared following the steps proposed in the literature on the development of a psychometric scale, it has some limitations. First, NCASDM was developed and analyzed in the sample of Turkish nurses. Therefore, the generalizability of the results for other countries is limited. In addition, the same sample group was split, which is a common practice for CFA analyses.
Implications
In this study, the psychometrics of the 60-item draft scale were analyzed among nurses. As a result of the analysis, it was concluded that the Nurse Competency Assessment Scale for Disaster Management, which consists of 43 items and 2 dimensions, is a valid and reliable tool for assessing the competence of nurses in disaster management. The scale, which includes disaster preparedness and disaster response stages, meets the validity and reliability criteria. It has adequate psychometric properties. This scale contains few items and 2 dimensions, providing a practical and comprehensive assessment of disaster management competencies. This may provide feedback to nurse managers about the competencies of nurses, and facilitate the determination of in-service training topics for nurses and the achievement of organizational goals and outcomes by creating disaster plans. In addition, health institutions can use this scale to evaluate disaster drill practices and receive feedback for improving disaster plans.
Acknowledgments
The authors are particularly grateful to all participants who contributed to this study.
Author contribution
All authors agree with the content of the manuscript and meet the authorship criteria.
Conceptualization and study design: Ayşe Yildiz Keskin, Handan Alan.
Data collection: Ayşe Yildiz Keskin.
Data analysis: Ayşe Yildiz Keskin, Handan Alan.
Drafting of the manuscript: Ayşe Yildiz Keskin, Handan Alan.
Manuscript writing: Ayşe Yildiz Keskin, Handan Alan.
Manuscript revision and approval of the submitted version: Ayşe Yildiz Keskin, Handan Alan.
Funding statement
No sources of funding were received for this research.
Competing interest
The authors of this research declare no conflicting interests.
