The US federal government has identified public health emergency preparedness as a public health priority. Emergency preparedness education empowers citizens and at-risk populations to increase their readiness for emergencies. This need is particularly pronounced for vulnerable populations, including pregnant and postpartum women and their children. 1 Evidence indicates that a sizeable percentage of Americans remain unprepared or underprepared. A 2007 American Public Health Association national survey found that among mothers of young children, 58% reported not having a 3-day supply of water and only 14% reported being “fairly well prepared” or “very well prepared” for a public health crisis. 2 Hurricane Katrina demonstrated the vulnerability of pregnant women and infants during disasters and underscored the importance of emergency preparedness, especially for underserved, low-income populations.Reference Callaghan, Rasmussen and Jamieson 3 During times of disaster, the risk for pregnancy complications (for example, prematurity and low birth weight) increases; this risk is exacerbated by factors like high stress and insufficient drinking water and nourishment.Reference Callaghan, Rasmussen and Jamieson 3 , 4
Various governmental, nonprofit, and professional organizations produce emergency preparedness web pages for the public. These Internet-based educational resources are particularly important in providing key preparedness recommendations during infectious disease emergencies when a need exists to minimize person-to-person contact. However, many of these web pages require substantial literacy skills for comprehension of information.Reference Friedman, Tanwar and Richter 5 Friedman et alReference Friedman, Tanwar and Richter 5 noted that web pages from sources generally considered to be more credible, ie, nonprofit and government websites, are also written in the most difficult language.Reference Friedman, Tanwar and Richter 5 Although various educational resources are available to improve emergency readiness at the individual level, the appropriateness of these resources for a population of low-income pregnant and postpartum women is undemonstrated.
Methods
Internet-based resources for emergency preparedness generally take the form of self-directed learning. This educational approach relies on the initiative of the audience to learn without the aid of a teacher, and it assumes that the target audience will seek out and understand the information needed to answer their questions or to address their concerns. We examined the suitability of online self-directed learning resources available to pregnant and postpartum women. For this study, only free, easily accessed, Internet-based resources were considered. We chose to evaluate the readability and suitability of a sample of web pages produced by the Centers for Disease Control and Prevention (CDC), the United States’ lead public health agency.
Although the selected web pages included some general emergency preparedness information, all pages were directed toward “pregnant women” or “parents of young infants” or “new mothers.” Some of the pages followed the all-hazards approach to preparedness, whereas other pages offered information about specific types of emergencies and disasters. The web pages reviewed are detailed in Table 1. The web pages were assessed between February 25, 2014, and May 15, 2014.
Sample of Centers for Disease Control and Prevention Preparedness Web Pages Examined
We assessed suitability by using the Suitability Assessment of Materials (SAM) tool.Reference Doak and Root 6 This tool has been used by researchers to evaluate educational materials on a range of health topics, including online emergency preparedness resources.Reference Friedman, Tanwar and Richter 5 Two raters independently scored 2 sample government health-focused web pages on an unrelated topic as a pilot test to practice using the SAM tool and to test the similarity of rating technique. Discrepancies in scoring technique were resolved through discussion. The 2 raters then scored a sample of 7 CDC web pages about the topic of emergency preparedness for pregnant and postpartum women.
The SAM tool requires an assessment of reading grade level. For this assessment, we used the Flesch-Kincaid Grade Level results obtained by using the embedded reading level tool within Microsoft Word 2011 (Microsoft Corp, Redmond, WA). A limitation of using the Flesch-Kincaid Grade Level formula within Microsoft Word is that the tool only displays grade level results up to grade 12. This limitation is acceptable, however, given the purpose of the readability examination, because any grade level exceeding 12 would be deemed equally inappropriate for general audiences. A common recommendation for reading level is the 6th to 7th grade. 7
We used a standard SAM scoring sheet consisting of the following 6 general categories: (1) content, (2) literacy demand, (3) graphics, (4) layout and typography, (5) learning stimulation and motivation, and (6) cultural appropriateness. These categories were then evaluated by using more specific criteria. For example, in the case of literacy demand, the specific criteria included reading grade level, active voice, vocabulary, context, and road signs. Each criterion received a score of 0 to 2 (0=not suitable, 1=adequate, 2=superior, or not applicable) and the final SAM score was typically listed as a percentage of the total possible points. Interpretation of the final SAM scores was as follows: 70–100% was considered superior material, 40–69% was considered adequate material, and 0–39% was considered not suitable material.
This study did not involve human subjects and therefore no institutional review board approval was necessary.
Results
The results of the SAM scoring are outlined in Table 2. Five of the 7 resources examined were considered adequate. The infections web page was considered not suitable; the raters were split between not suitable and adequate for the smallpox page. None of the resources examined were considered superior.
Average SAM Criteria ScoresFootnote a
a SAM indicates Suitability Assessment of Materials.Reference Doak and Root 6
b The score for cultural appropriateness takes only one criterion into account (“Logic, language, experience”), because the other criterion, “Cultural Images,” was considered not applicable for every web page examined. The scores vary widely mostly for this reason.
c The raters were split on the overall SAM score for this web page.
In the case of the infections web page, the literacy demand was particularly poor. The page received a score of zero from both raters for both the reading grade level and vocabulary, because the material was written at a 9th-grade reading level and the vocabulary contained many uncommon words. The majority of these uncommon words were related to infection symptoms and treatments, including the names of the specific antibiotics used to treat certain infections. The description of the infections web page stated that, “Information on this website was created for women who are pregnant and concerned about the effects of certain infections on their unborn children.” Given that the intended audience was pregnant women, the content of the page included details more relevant to health care providers. Content relevant to pregnant women following a disaster, particularly infection prevention information, was not the focus of the page. Although the web page included links to information about infection prevention, the links were not included in the body of the page for ready access by the reader.
The 2009 H1N1 flu web page had the highest scores overall and was the closest to receiving a superior score. The other adequate web page scores were on the lower end of the range, which suggested that more substantial changes would be needed before they could be considered superior materials. The purpose of the information in the 2009 H1N1 flu web page was evident; the content was focused on behaviors relevant to pregnant women. The question-and-answer format stimulates learning and tailors the information to the intended audience. With some improvements in graphics and layout, this web page may have received a superior score.
Graphics was a category with consistently low scores across the sample of web pages. There was an apparent lack of focus on the visual representations of information in Internet-based resources. Layout and typography was another category with mediocre scores, indicating that the visual presentation of the web pages examined did not improve readability and may, in fact, have made the information less accessible to the intended audience.
In addition to the final SAM scores, we analyzed inter-rater reliability for the SAM analysis. The Cohen’s Kappa value of all of the raters’ SAM scores for each web page was 0.738, representing a substantial level of agreement.Reference Viera and Garrett 8 The major scoring differences occurred in categories where raters disagreed about the applicability of a particular criterion and whether it should be assigned a score of zero or “not applicable.”
Discussion
The information about disasters for pregnant women and mothers of infants contained in the sample of emergency preparedness web pages was varied. Although most of the web pages were considered adequate according to the SAM analysis, none of the web pages received a score of superior overall. Many of the web page scores were on the lower range of adequate. For an at-risk, low-literacy population unlikely to seek preparedness information, these “adequate” web pages are almost certainly insufficient. Perhaps the clearest deficiency in the reviewed web pages is the lack of sufficient audience segmentation. Some of the preparedness information seemed to be aimed at health care providers and public health professionals, whereas other information was more appropriate for the general public.
The SAM scores highlighted particular communication weaknesses that can be targeted for improvement. One particularly weak area for the sample CDC web pages was the graphics category. The sample web pages rarely provided sufficient graphics, and the graphics they contained were uncaptioned. Strong graphics are important for reaching many audiences, but for a low-literacy population they are all the more essential.
These self-directed learning resources take for granted the fact that members of this population will be interested in the topic of preparedness and have the resources necessary to access the information (eg, Internet access). The assumption is that women will either seek out this preparedness information on their own or will be directed toward these resources by someone like a health professional. In fact, some evidence suggests that low-income pregnant women rely heavily on interpersonal sources for pregnancy-related information. A study conducted by Song et alReference Song, Cramer and McRoy 9 of the information needs and information-seeking behavior of women enrolled in a subsidized prenatal care program found that interpersonal sources of information were most frequently used for acquiring health and pregnancy-related information. Song et al also found that the Internet was the least-used source of information among the low-income pregnant women enrolled in their study. Likewise, Boyd et alReference Boyd, Gazmararian and Thompson 10 found that low-income women considering H1N1 vaccination for themselves and their children preferred the presence of a trusted professional (eg, a doctor) to answer questions and explain relevant information to help with their decision-making process. The format of an informational web page does not lend itself to interactivity or interpersonal communication; it is unclear how these preferences would change in the event of a pandemic or disaster when personal contact with the usual, preferred sources of information may be limited. The crafting of effective preparedness materials would benefit from stakeholder participation in the message creation process and appropriate audience segmentation. Suitability of intervention materials must be considered, and materials should be audience tested. The SAM analysis revealed a recurring weakness in the graphical representations of preparedness information. An increased focus on visual communication may help to reach communities with low literacy skills.
Creating educational resources, like preparedness-themed web pages, without first raising awareness of the importance of emergency preparedness in a way that is suitable for pregnant and postpartum women is counterproductive. Even if resources are both readable and suitable for the intended audience, the success of such an intervention requires that the target audience engage in information-seeking behavior, and this is not an assured outcome. Using an audience-centered approach to emergency preparedness education will help to ensure that emergency preparedness messages reach those pregnant and postpartum women at the greatest risk of adverse outcomes in the case of a public health emergency. An audience-centered approach takes into account the demographic profile, knowledge, attitudes, and values of the intended recipient of information.
This study had several limitations. The SAM tool has notable limitations, including rater subjectivity and a lack of clear connection between the factor being assessed and the rationale for its importance. Despite its limitations, however, the SAM analysis provides an approximation of the suitability of written materials. This study examined a limited number of web pages from one governmental source. Applying the SAM tool to a larger sample of web pages would strengthen our understanding of the content of materials available to pregnant and postpartum women. Additionally, it has been suggested that the Flesch-Kincaid Grade Level scores may give lower grade level scores than some other readability assessment tools. Thus, our scores may overestimate the readability of some materials; however, they remain acceptable tools for estimating the readability of materials. 7
Conclusions
Public health emergencies and disasters have a disproportionate impact on vulnerable populations. Improving maternal emergency preparedness is a public health priority. More research is needed to identify the strengths and weaknesses of various approaches to how preparedness information is communicated. The assessment of CDC preparedness web pages identified several key areas for improvement. Strong graphics and layout make web pages more suitable for audiences with low literacy. Audience segmentation is necessary for more focused communication, and stakeholder involvement in the message creation process is crucial. If these CDC web pages are considered some of the best available preparedness resources for pregnant and postpartum women, more work is needed to improve the suitability of Internet-based preparedness educational materials, especially for audiences with low literacy and low incomes.
