Disasters are human, natural, or technologically induced events that result in physical, economic, and social losses at both individual and societal levels. They have a profound impact on people’s daily lives and activities, often causing disruptions or complete cessation. The report on the countries with the most earthquakes from 1900 to 2016 shows that Turkey is the 4th country with the most earthquakes. 1 Earthquakes, landslides, floods, and avalanches are the most common and significant economic losses from natural disasters in Turkey. According to a report published by the Presidency of Disaster and Emergency Management (AFAD) in 2019, over the past 25 years, 19 948 people have lost their lives, and 63 103 people have been injured due to natural disasters in Turkey. The report also indicates that the economic cost of these disasters to Turkey amounts to approximately 42 million dollars. 2 Among the various natural disasters experienced, earthquakes alone account for the highest economic burden, comprising 90.47% of the total. It is reported that as a result of earthquakes, around 850 000 people have become homeless, and Turkey has suffered a loss of about 38 million dollars. 2 Turkey also experiences man-made disasters frequently. In the past 25 years, 2822 people died and 2177 were injured in such incidents. 3
In order to reduce preventable damage caused by disasters, it is crucial to increase public awareness, implement disaster preparedness measures, and ensure the correct implementation of response actions when disasters occur. 3 Disaster challenges are often caused by a lack of information about disaster preparedness, failure to anticipate potential crises, and a lack of preventive planning. 4 While it is not possible to completely eliminate the negative impacts of disasters, these impacts can be reduced through the implementation of disaster management strategies, sharing of best practices, learning from past experiences, preparedness planning, and mitigation efforts.Reference Seneviratne, Baldry and Pathirage 5
Media, or mass media, serves as a means of conveying messages to a broader society. It encompasses various information sources, including those related to disasters, and plays a vital role in organizing information for citizens during natural disasters.Reference Cowan, McClure and Wilson 6 The use of the Internet for obtaining information is increasingly widespread today. In Turkey, as of 2021, 92% of households have Internet access. The frequency of internet use in the 6-15 age group is 82.7%. It has been reported that 20.9% of children who use the Internet do so to search for health-related information. 7 , 8 Informative websites contribute to increasing individual awareness and raising public awareness on important issues. The topic of disaster management, which is addressed in a multidimensional manner, is also a significant public health concern. Therefore, websites dedicated to disaster preparedness should be up-to-date, easily readable, and suitable for society’s level of information. So et al.Reference So, Franks and Cree 9 examined the readability of 386 websites on natural disaster preparedness on the Internet, specifically for families, and reported that the content had low readability, and the information provided exceeded the society’s knowledge level.Reference So, Franks and Cree 9 In a study conducted by Friedman et al.,Reference Friedman, Tanwar and Richter 10 the readability and suitability of 50 websites on disaster preparedness were analyzed, revealing a need for easily understandable and accessible disaster preparedness resources.Reference Friedman, Tanwar and Richter 10
People seeking information turn first to the Internet.Reference Hesse, Nelson and Kreps 11 Therefore, it is important to determine the influence of the organizations that create the websites and communicate appropriate messages to the public.
Unfortunately, no study has yet been published evaluating disaster preparedness websites in Turkey. The aim of this study is to evaluate the readability and appropriateness of the content of websites on disaster preparedness prepared for the public.
Methods
Ethics committee approval was not obtained because only publicly available information was used for the study.Reference So, Franks and Cree 9 We conducted a cross-sectional evaluation of web-based disaster preparedness materials using a standard search engine (Google) that Turkish people would likely use to search for resources. 12
Sampling Procedure
We limited our focus to disaster preparedness websites associated with gov.tr (government), edu.tr (education), mil.tr (military), bel.tr (municipality), and org.tr (organization) under consideration. We used 25 keywords for disaster preparedness materials for society (Table 1).
Search strings used for an evaluation of the readability, quality, and content of online disaster preparedness for Turkish society
Inclusion Screening
Terms were searched on the Google search engine from September 10-October 10, 2022. Of the top 100 websites for each keyword, 299 websites associated with gov.tr (government), edu.tr (education), mil.tr (military), bel.tr (municipality), and org.tr (organization) were included in the study. Two different groups of authors reviewed each website separately. Repetitive websites, websites written in foreign languages, those containing fewer than 10 sentences of information, chat or forum websites, commercial blogs, websites containing only pictures, tables, or videos, research articles, news websites, websites accessible only by registration or for a fee, and websites with inappropriate content were excluded from the study.Reference So, Franks and Cree 9 , Reference Bozkurt, Sahin and Bagci 13 The senior author was consulted when the review results were inconsistent. After applying the exclusion criteria, the remaining 95 websites were included in the study. The flowchart of the research is presented in Figure 1.
Flowchart of the research.
Measures
The readability and comprehensibility of information resources on the Internet is as important to users as content, accuracy, and reliability. The readability of the materials was assessed using the Ateşman Readability Index. Quality assessment was performed using the Journal of the American Medical Association (JAMA) criteria and the Quality Criteria for Consumer Health Information (DISCERN) scale. Each website’s content was evaluated using a 26-item form developed by researchers based on existing literature, demonstrating an internal consistency with a Cronbach’s alpha of 0.88. The form included questions on topics such as general information about disasters, the types of disasters more prevalent in Turkey, preparedness measures before disasters, and actions to be taken during and after a disaster (Table 2).
Measurement and scoring protocol for evaluation tools used in assessing the readability, quality, and content of disaster preparedness websites
Readability Assessment of Websites
Whether a text is easy or difficult to understand can be objectively measured with the readability index.Reference Ateşman 14 The readability of the materials was assessed using the Ateşman Readability Index. There are many formulas based on the number of words and syllables used to determine the readability level of texts and to mathematically calculate the difficulty of texts.Reference Dubay 15 , Reference Goldbort 16 Generally, 3 different variables are used to measure readability: number of syllables, number of words, and number of sentences.Reference Temur 17 Ateşman’s formula is an index used to measure the readability of Turkish textsReference Ateşman 14 that can be calculated online.(http://okunabilirlikindeksi.com/)18 and is a Turkish adaptation of the FleschReference Flesch 19readability formula. 18 , Reference Flesch 19
Ateşman’s formula is based on word and sentence length:
Readability score = 198 825 – 40 175 x word length (total syllables/total words) - 2610 x sentence length (total words/total sentences).
The classification of readability level according to the Ateşman Readability Index is shown in Table 3.
The classification of readability level according to the Ateşman Readability Index
Quality Assessment
Journal of the American Medical Association (JAMA) criteria
Basic information on websites includes authorship, references, patent rights, and currency. Scores range from 0 to 4, with 1 point awarded for each criterion. The weakest quality is awarded 0 point, and the highest quality is 4 points.Reference Silberg, Lundberg and Musacchio 20
Quality Criteria for Consumer Health Information (DISCERN)
The DISCERN criteria Part 1 consists of an 8-item questionnaire designed to assess the reliability of information. Each item is rated on a scale from 1-5, with higher scores indicating better reliability. The total score can range from 8-40, with higher scores suggesting that the information is more reliable.Reference Charnock, Shepperd and Needham 21
Content Evaluation of Websites
In this study, a questionnaire consisting of 32 items, each to be answered “yes” or “no,” was prepared by utilizing the existing literature. The aim was to conduct a comprehensive evaluation of the content on websites. In order to assess the reliability of the questionnaire, Cronbach’s alpha coefficient, which indicates internal consistency, was calculated and a value of 0.82 was obtained. During the reliability analysis, items that exhibited a negative correlation value below 0.20 in the statement-total correlation were subject to re-evaluation. Negatively contributing items were methodically re-evaluated, and a refined iteration of the questionnaire was conducted by gradually removing those that consistently showed a negative effect. After these rigorous adjustments, the questionnaire was subjected to a comprehensive re-analysis to assess its internal consistency. The final version of the form consists of 26 items and the Cronbach’s alpha coefficient is 0.88, indicating a very high level of consistency and reliability.
This instrument, called the Content Evaluation Form of Disaster Preparedness Websites (CEFW), was carefully crafted to measure the information disseminated by the websites under study. Scores from the form ranged from 0-26, with higher scores indicative of a more substantial volume of information available on the evaluated websites. In the conclusive version of the CEFW, item factor loadings exhibited a range from 0.507-0.816, collectively explaining 68.17% of the total variance.
Statistical Analysis
The SPSS 15.0 program was used to analyze the data in the study. The values of each website from the scales were entered and a dataset was created. The Kolmogorov-Smirnov test was used to test the normality of the data. Categorical data were presented as frequencies and percentages, while continuous data were presented as mean ± standard deviation, minimum, and maximum values. The t test and ANOVA test were used to compare means between groups. The ordinary least squares method was used to determine different groups in the ANOVA test. Statistical significance was set at P≤0.05.
Results
In the distribution of the websites included in the study according to information sources, government institutions were the most numerous (45.2%). Distribution of websites according to information sources is given in Figure 2.
Categorization of websites according to information sources.
There is a definition of disaster in 20 (21.1%) of the evaluated websites and disaster is the main topic of 44 (46.3%) of them. In the study, 34 (35.8%) websites specified what not to do during and after a disaster, 41 (43.2%) specified what to do during a disaster, and 18 (18.9%) featured plans related to vulnerable groups (disabled, elderly, pregnant, etc.) who need to be rescued as a priority (Table 4).
Distribution of website content evaluation results
The item with the highest percentage of yes responses was “Websites include elements such as pictures, videos, audio recordings, podcasts, etc. to strengthen the narrative” with 74.7%, while the item with the lowest percentage of yes responses was “The website emphasizes that early warnings from reliable institutions should be taken seriously” with 14.7%.
The main theme of most of the websites of non-profit organizations (NPOG) was disaster (72.2%). The only group that did not include contact information for obtaining information on disaster preparedness was the websites of other organizations (OG) (0.0%). The websites that mentioned how to contact the authorities during or after a disaster most frequently were municipal organization (MOG) websites (62.5%).
OG websites both emphasized the importance of volunteer teams in disasters the least (P = 0.005) and contained the least information about what to do during disasters (P = 0.032). The websites containing the most information on how to prevent the damages caused by disasters are government institutions (GIG) websites (P = 0.026). The information content about the disaster plan that should be prepared in places such as homes, workplaces, schools, and hospitals before a disaster was mostly found on MOG websites (P = 0.014). In addition, MOG’s websites contained the most pictures, videos, audio recordings, and podcasts to strengthen the narrative (P = 0.008) and almost all MOG websites used these elements.
It was found that the CEFW scores of the websites belonging to MOGs were higher than the scores of the other websites. 36.8% of the websites whose content was evaluated scored higher than the mean CEFW score. The scores obtained from Ateşman index ranged between 9.70-105 and the mean score was 61.51±17.41, and 53.7% of the websites had a score above the mean.
The first part of the DISCERN scale consists of 8 items. According to these first 8 items, the scores of the websites vary between 17-40 and the mean (SD) is 30.07 (2.31). On the DISCERN scale, 51.6% of the websites scored above mean.
According to the JAMA criteria, the scores ranged from 0-4 and the mean (SD) is 1.88 (1.10). Others group websites scored higher on the scale than other websites. According to the mean JAMA score, 55.8% of the websites scored above the mean (Table 5).
Assessing the readability, quality, and content of online disaster preparedness for Turkish society according to information sources
* ANOVA test
SD: Standard Deviation
a,b Groups with different letters differ significantly (P ≤ 0.05).
Discussion
In the study, the MOG websites were the group most likely to contain information on how to contact authorities during or after a disaster. None of the other group websites included contact information for obtaining disaster preparedness information, and there was no mention of how to reach authorities during or after a disaster. However, the official website of Washington D.C. in the US contains informative text on how to call 911. 22 This is the result of the fact that municipal, state, or federal websites take care to explain whom the public should call in emergencies and share contact numbers. It is necessary to communicate to institutions the importance of including contact information on all websites for obtaining information about disaster preparedness.
The GIG websites contain more intensive information on the prevention of damages caused by disasters compared to other website groups. The importance of using volunteer teams in disaster situations and the necessity of preparing disaster plans are mostly emphasized on NPOG websites. In contrast, the websites of OG did not provide information on these topics. In Turkey, there are many organizations that volunteer in disasters, such as AFAD 23 (Disaster and Emergency Management Presidency), GEA 24 (Mother Earth), and AKUT 25 (Search and Rescue Association), and they have their own websites. 23 – 25 In addition to such NPOG websites, it is desired that public organization and MOG websites also include a call for volunteers.
In the study, three-quarters of the online materials included visual or audio materials that would strengthen the narrative. Friedman et al.,Reference Friedman, Tanwar and Richter 10 in their study analyzing disaster websites, revealed that there is a need for easily accessible, visually appropriate disaster preparedness resources.Reference Friedman, Tanwar and Richter 10 It was assumed that readers would show more interest in the subject with enriched narratives.Reference Lee, De Simone and Ebrahimi 26
A study conducted in Turkey in 2010 found that the average educational level of the population aged 15 years and older was equivalent to 8 years, which is the last year of secondary school. 27 In the British website screening study conducted by So et al,Reference So, Franks and Cree 9 the average required reading level of the materials was at the ninth-grade level, according to the Flesch-Kincaid readability score. Moreover, the educational level in that study is higher than the educational level of adults in the US, which corresponds to the eighth grade.Reference So, Franks and Cree 9 It is a prerequisite for comprehension that the readability of health-related information texts published on the Internet should be parallel to the reading level of the country’s population.Reference Tolu 28 The goal of readability studies is to make language more understandable. Chall et al.Reference Chall, Zakaluk and Readibility 29 reported that the formulas developed to determine readability levels may not give the same results for readability in different languages.Reference Chall, Zakaluk and Readibility 29 The readability level of the online materials examined in our study was at the ninth and tenth grade level. This indicates that the informative texts of the online materials should be redesigned and understandable for people with lower health literacy. The clearer and more understandable the information on emergency preparedness on websites, the more likely individuals will be able to apply it.
According to JAMA’s criteria, OG websites were better than others in terms of author information, bibliography, patent rights, and timeliness. In our study, the group with the lowest average in this regard was the municipal websites. One of the reasons for the difference between the groups may be that JAMA criteria are international, and OG sites are based internationally. Similarly, a study by Gökay et al.Reference Gokay, Gorurgoz and Veneer 30 on dental health reported that the quality of Turkish websites was lower than foreign language websites according to the criteria of JAMA, and other group websites were richer in terms of content compared to other websites.Reference Gokay, Gorurgoz and Veneer 30
In our study, the websites emphasizing that warnings (SMS, news, announcements) from reliable institutions about an imminent disaster should be taken seriously were compared, but no statistically significant difference was found between the 4 website groups. In total, 81 (85.3%) websites do not have this information. In a study compiled by Goniewicz et al.,Reference Goniewicz and Burkle 31 early warning systems against multiple hazards were mentioned in websites informing the public against disasters in Japan.Reference Goniewicz and Burkle 31 It was also found that plans for individuals who need to be rescued first (disabled, elderly, pregnant, etc.) were insufficient (81.1% of the websites) and there is no statistically significant difference between groups in this regard. Brown et al.Reference Brown, Haun and Peterson 32 reported that individuals with physical and mental disabilities, the elderly, and children are the most vulnerable groups in disaster situations.Reference Brown, Haun and Peterson 32
Strengths and Limitations
Concerning our search methodology, we adopted an approach that employed a replicable protocol within the most prevalent search engine to capture data from disaster preparedness websites. Additionally, we implemented adjustments for multiple comparisons to mitigate the risk of false-positive outcomes. Moreover, deliberate design elements, such as the utilization of independent researcher groups and employing four distinct tools, furnish us with a more comprehensive insight into the existing resources. This contributes to the evaluation of the readability, quality, and content of online disaster preparedness materials tailored for societal needs.
The study has several limitations. Notably, the website search was confined to a specific brief timeframe, utilizing a sole search engine and conducted exclusively in one language. Furthermore, the absence of comparable studies in the literature evaluating the content and readability of disaster websites poses a challenge in making meaningful comparisons.
Conclusion and Future Directions
In this study, it was found that the readability level of informational texts about disasters in online websites was higher than the level of education in the society. However, disaster literacy can be increased and damages that may arise from disasters can be reduced by preparing websites in accordance with the readability level of the society. This demonstrates the importance of understanding and effectively using life-saving information during and after disasters; taking early warnings seriously; knowing how to reach authorities and how to approach individuals who need to be rescued first; and being organized during and after disasters. For this reason, it was thought that it would be useful to include more information on these topics in online materials. The importance of digital skills in education, health, and economy, both in the private sector and in public administration, is becoming increasingly clear. In this context, improving disaster literacy will help to spread disaster awareness in society. Websites should provide high-quality and accurate information about disasters and should be constantly updated and organized. Better preparation of online materials in readability and content quality will increase readership and make it easier to create disaster awareness in society.
Author contribution
BK, FNÖM, SS, OHA, and MAA: data collection, statistical analysis, and article writing; SM and MFÖ: statistical analysis and article writing.
Funding
No funding or grant support.
Competing interest
The author(s) declare none.
Ethical standard
Ethics committee approval was not obtained because only publicly available information was used for the study.
