A mobile application, or app, is defined as a self-contained program or piece of software that is designed to fulfill a particular purpose and can run on a variety of mobile devices, such as smartphones and tablets. 1 , 2 Mobile apps were first developed in the early 1990s, but did not gain widespread popularity until the early 2000s.Reference Clark 3 It is estimated that roughly one-fifth of US adults have used at least one health app; by 2017, half of mobile phone users are projected to have utilized at least one health app.Reference Boudreaux, Waring and Hayes 4 Mobile health apps are now available for clinical populations (eg, asthma patients) as well as for general health and preparedness (eg, cardiopulmonary resuscitation and first aid).Reference Marcano Belisario, Huckvale and Greenfield 5 - Reference Zanner, Wilhelm and Feussner 7 In 2012, it was estimated that 85% of US adults owned a mobile phone, of which roughly 53% were smartphones.Reference Boudreaux, Waring and Hayes 4 With such a large number of US adults having access to a mobile phone, a mobile disaster health learning app may be a useful platform to assist in US preparedness efforts in health and disaster health settings.
Notably, the fields of disaster health and medicine are still underrepresented in app usage and utilization.Reference Case, Morrison and Vuylsteke 8 In 2012 Case et al identified 5 types of apps used in disaster settings: disaster scene management, remote monitoring of casualties, medical image transmission (teleradiology), decision support, and field hospital information technology (IT) systems.Reference Case, Morrison and Vuylsteke 8 They noted that only teleradiology and field hospital IT mobile systems had been activated in real disasters.Reference Case, Morrison and Vuylsteke 8 Furthermore, equipment limitations and skills deterioration were cited as obstacles to end user effectiveness and compliance.Reference Case, Morrison and Vuylsteke 8 However, despite these issues, there is much potential in using mobile apps in general health preparedness and by health professionals as an extension of their toolkits. Mentler et al noted the development and need for an integrated mobile application system for managing mass casualty incidents for emergency medical services providers.Reference Mentler, Herczeg and Jent 9
While mobile apps have been utilized in a variety of public health settings, there are limited research investigations on the use of mobile apps as an educational or learning platform for health professionals. Previous research has examined the importance of online web-based technologies as a useful enhancement to learning. One example is the use of push technology, which is a means through which information is delivered automatically to all end users from a central location or server.Reference Umbach 10 One study noted the importance of Twitter and Facebook (as push technology) to supplement a curriculum on ultrasound concepts.Reference Bahner, Adkins and Patel 11 Medical students who received curriculum tweets or Facebook posts voiced approval on the concise yield of information and ease of use.Reference Bahner, Adkins and Patel 11 Another study examined the usefulness of delivering prenatal health information via a text messaging app and found that users’ health knowledge improved over time.Reference Remick and Kendrick 12
Despite these studies, there is still a dearth of research on the utility of apps in health learning or training frameworks. Most of the research to date has been focused on the use of web tools (eg, podcasts, video blogs, and blogs) and online programs that use an app interface (eg, Facebook and Twitter). There has been a lack of research examining mobile apps as a self-contained program on a cellphone. As mobile apps continue to evolve, there is an opportunity to explore the use of apps in a disaster health learning context. This is of emerging importance, as mobile apps are becoming increasingly ubiquitous in the United States. With such prevalence comes the opportunity to deliver the latest information and curricula to health professionals who work in disaster settings.
The objectives of this article were to identify the current state of disaster health apps being utilized in an educational, learning, and training perspective; to describe and place apps in a health learning context; and to adapt an app learning framework for the disaster health workforce. In this article, a framework consists of concepts and existing theory that provides an organizing structure for the phenomenon under study.Reference Dahabreh 13 , 14 We seek to enhance this definition by structuring our framework to enhance learning outcomes by using mobile apps in the disaster health field. This work seeks to add to the small but growing literature describing apps for health learning and training settings.
METHODS
Study Design
This systematic literature review was conducted by using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist.Reference Moher, Liberati and Tetzlaff 15 The review examined articles written or translated into English, with no necessary date or geographic restrictions, using the PubMed (US National Library of Medicine, Bethesda, MD) and the Cumulative Index to Nursing and Allied Health Literature (CINAHL; EBSCO, Ipswich, MA) databases. Gray literature searches were not conducted. For the purposes of this study, the term app was defined as “a self-contained program or piece of software designed to fulfill a particular purpose; an application, especially as downloaded by a user to a mobile device.” 2
Data Collection
Combinations of the following key words were used: mobile applications, mobile apps, learning, disaster, and health.
Inclusion Criteria
These were as follows:
-
∙ Articles that included 2 critical elements: “mobile apps” and “learning.”
-
∙ Articles that had a strong focus on the health professional audience.
Exclusion Criteria
These were as follows:
-
∙ Articles that did not include both critical elements.
-
∙ Articles with exclusive focus on web 2.0 tools (eg, podcasts, video blogs, blogs).
-
∙ Articles with exclusive focus on personal digital assistants (PDAs).
-
∙ Articles examining apps that were outside of this article’s definition of app.
-
∙ Articles on learning from a general population or patient population perspective (eg, apps for diabetes self-management).
-
∙ Articles that did not deal with apps or with the use of apps in mobile settings.
-
∙ Articles that did not have a component of learning or training.
-
∙ Articles that were not peer-reviewed.
-
∙ Articles that were not published in English.
Screening and Analysis Procedures
All references generated from article searches underwent an initial screen consisting of a title and abstract review. Articles selected for inclusion at this stage contained a “mobile apps” element and a “learning” component, per the inclusion criteria. The secondary screening consisted of a full article review, wherein each article was examined for how mobile apps were used and what learning models (if any) or elements of learning were applied. Last, a tertiary screening was conducted, which consisted of another full article review, with particular emphasis on what types of mobile technology were used, what elements of learning were utilized, and how mobile apps were used in specific target populations. The final list of articles generated from the tertiary screening was reviewed for useful learning models, elements of learning that could aid in development of an app learning framework, and use of mobile apps within the context of learning. All review phases were conducted by 3 of the authors, with agreement on article selection and exclusion. Results from these searches informed the proposed framework for mobile apps in disaster health learning.
RESULTS
The search strategy generated a total of 159 references; of these citations, only 107 remained after removing duplicate entries. As described above, these 107 references underwent an initial screen, which resulted in 47 additional records being removed under the exclusion criteria. The secondary screen was conducted on the remaining 60 references; this secondary review excluded an additional 29 references. Finally, the remaining 31 articles were included in the qualitative analysis where they were reviewed again using our exclusion criteria. This tertiary screen removed 14 citations, which resulted in a final set of 17 articles for analysis (Figure 1).
Flow Diagram for Selection of Articles for Qualitative Analysis. Based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist, 2009.
All references were peer-reviewed articles obtained through PubMed; all citations obtained from CINAHL were duplicates. Per the 3 main aims of this article, none of the 17 references adequately identified the current state of disaster health apps utilized in a learning or training context. Several learning models were identified, important learning concepts were gleaned, and these elements were deemed critical to a learning framework for use of apps in the disaster health field (Table 1). In analyzing the 17 articles, several prominent concepts contributed to our iterative understanding of considerations for the use of apps in disaster health learning and therefore informed the proposed framework. Eight articles focused on specific apps in a learning context.Reference Albrecht, Folta-Schoofs and Behrends 16 - Reference Yoo and Lee 23 Seven articles described the general attitude of health professionals and students in using mobile technologies and apps in learning.Reference Nuss, Hill and Cervero 18 , Reference Renner, Kimmerle and Cavael 20 , Reference Archibald, Macdonald and Plante 24 - Reference Székely, Talanow and Bágyi 28 Four articles examined the current status of apps for a number of different types of health professionals and how they are applied in a broad learning context.Reference Székely, Talanow and Bágyi 28 - Reference Lewis, Burnett and Tunstall 31 Six references mentioned the lack of quality checks on app content.Reference Bibault, Leroy and Blanchard 25 , Reference O’Connor and Andrews 27 , Reference Haffey, Brady and Maxwell 29 - Reference Murfin 32 Lastly, 3 references had some discussion of learning models, which will be examined in greater detail in the discussion section.Reference Renner, Kimmerle and Cavael 20 , Reference Teri, Acai and Griffith 22 , Reference Bullock 26
Learning Elements and Models Identified During the Review Process
a The first 3 articles discuss learning models.
b Augmented reality is the integration of technology into learning that allows individuals to practice certain techniques in simulated scenarios without breaching ethical boundaries.Reference Albrecht, Noll and von Jan 17
DISCUSSION
In this literature review, 3 of the 17 articles that passed the tertiary screening had a preexisting, defined learning model discussed within the article.Reference Renner, Kimmerle and Cavael 20 , Reference Teri, Acai and Griffith 22 , Reference Bullock 26 Bullock utilized O’Leary’s knowledge management classification as well as Sfard’s acquisition and participation models to help further understand the use of technology for learning and sharing knowledge.Reference Bullock 26 O’Leary’s knowledge management framework is composed of 2 processes: conversion and connection.Reference Bullock 26 , Reference O’Leary 33 Of these 2 processes, the “connection” pathways are the most relevant to technology and learning. Four connection pathways are identified: people to knowledge, knowledge to knowledge, people to people, and knowledge to people.Reference Bullock 26 , Reference O’Leary 33 Technology can be used to support learning through all 4 of these pathways. Examples include search engines (people to knowledge), blogs and wikis (knowledge to knowledge), social media (people to people), and apps or electronic libraries (knowledge to people).Reference Bullock 26 Mobile apps can help to facilitate these pathways through Sfard’s acquisition of knowledge process.Reference Bullock 26 , Reference Sfard 34 In brief, Sfard’s acquisition and participation models examine how individuals acquire knowledge and the bonds between individuals and others that aid in learning.Reference Bullock 26 , Reference Sfard 34 Bullock uses these models as part of a process through which mobile technology can assist in collaborative learning.Reference Bullock 26 However, despite their usefulness, these models are being examined for general health learning. There is little discussion regarding the unique environment that disaster health professionals operate in and whether apps could be a useful supplement for learning in the disaster health field. These 2 learning models provide little insight into the types of learning that are helpful for disaster health.
Renner et al describe a system of reflective learning, wherein participants could learn in their environment by reflecting on work situations that have occurred.Reference Renner, Kimmerle and Cavael 20 Reflection is defined as a process “in which people recapture their experience, think about it, mull it over, and evaluate it”; this can be seen as a type of informal learning.Reference Renner, Kimmerle and Cavael 20 , Reference Boud, Keogh and Walker 35 While reflection is a normal, everyday process, it is not generally used intentionally as a mechanism to improve learning or performance.Reference Renner, Kimmerle and Cavael 20 Mobile apps can be used to assist reflective and collaborative learning by allowing participants to document and reflect on daily experiences, as well as collaboratively reflect on difficult situations for learning purposes.Reference Renner, Kimmerle and Cavael 20 Reflective learning (via mobile app) may be a useful tool in disaster health learning by contributing to experiential learning after exposure to an event. However, using only reflective learning is not sufficient for an app learning framework for disaster health, as it excludes other important components related to use of apps in disaster health settings.
Teri et al defined and examined the pedagogical impact of a mobile app in a health/science learning environment.Reference Teri, Acai and Griffith 22 Mobile apps in a pedagogically sound learning context are defined as “the digital support of adaptive, investigative, communicative, collaborative, and productive learning activities in remote locations which offer a variety” of environments for teachers and learners to function and learn in.Reference Teri, Acai and Griffith 22 Literature searches on mobile learning yield common themes such as a flexible learning environment, performance support, and user-centered environments.Reference Teri, Acai and Griffith 22 These components are crucial to any app learning framework but, similar to reflective learning, represent only a partial framework for examining mobile apps in a disaster health learning context.
It is important to place the previously described learning models within the environment of disaster health learning. In this perspective, individually, none of the 3 models reviewed earlier in this discussion are sufficient to fully describe an app learning framework for disaster health. However, each model has important elements that should be adapted into a new proposed framework that is helpful for disaster health learning, as well as mobile app development. In particular, the O’Leary framework’s “connection pathways” are important mechanisms in considering how technology can facilitate the transfer of knowledge to people.Reference Bullock 26 Additionally, Renner’s reflective learning model can facilitate a positive effect on user learning and user engagement.
Just-in-Time Learning and Training
An important aspect of disasters is their unpredictable nature. Disaster health professionals are often deployed during or shortly after an incident has occurred. As such, a recurring theme in the literature is a concept known as just-in-time (JIT) learning, which should play a key role in any disaster learning framework.Reference Nuss, Hill and Cervero 18 , Reference Archibald, Macdonald and Plante 24 , Reference Bibault, Leroy and Blanchard 25 JIT learning spans the range of providing point-of-use clinical reference information to short, focused lessons that health workers can first use and then apply in the field. In disaster health settings, having timely and accurate information is critical to the success of any disaster response and therefore is an important element in developing a mobile app for disaster health professionals. JIT learning has roots in O’Leary’s knowledge management processes. In particular, the connection paths from O’Leary’s 4 pathways can help facilitate and link knowledge to individuals.Reference Bullock 26 , Reference O’Leary 33 Mobile apps are thought to be most effective in the knowledge to people pathway; this makes intuitive sense, as mobile apps (by their nature) are connecting individuals to some type of information or knowledge. Medical residents in one study by Archibald et al noted that using mobile apps during residency eliminated the need to carry large textbooks around because all the information was available by using the app.Reference Archibald, Macdonald and Plante 24 Especially relevant to medical training were mobile apps containing drug interaction information, which can be critical information to have in an emergency situation.Reference Archibald, Macdonald and Plante 24 The study by Bibault et al of French medical residents noted that residents used their smartphones to access a variety of apps, including drug dosage, radiation dose calculators, and literature searches. These described uses have implications for JIT learning.Reference Bibault, Leroy and Blanchard 25 In fact, there is some concern regarding the use of such apps, with the implication being that medical residents may slowly forget how to conduct proper drug or radiation dose calculations without such apps.Reference Bibault, Leroy and Blanchard 25 However, in the field of disaster health, there is less concern regarding loss of skills due to disuse and more concern for accuracy due to the variety of emergency scenarios a disaster health professional could potentially face. In emergency settings, JIT learning could prove vitally important. A radiation or drug dose calculator in disaster settings could mean the difference between administering a proper dose and a fatal one. JIT learning in disaster health is also not restricted to drug or radiation information. Mobile apps that supply information on a variety of disaster scenarios could be used to provide critical information at the right time during an actual emergency. As discussed by Nuss et al, medical students noted that the use of apps allowed streamlined access to information, which connected knowledge to real-world needs.Reference Nuss, Hill and Cervero 18 Lastly, JIT learning and training is important in allowing health professionals to access information when and where they need it; this ease of access may prove vital during a disaster or emergency incident (assuming intact networks and stable power supply).
Content Quality and Transparency/Reliability of Information
Two important and related components of a mobile app are the content quality and transparency of information sources. Multiple studies have noted the lack of content quality checks and the opaque nature of information sources on mobile health apps.Reference Bibault, Leroy and Blanchard 25 , Reference O’Connor and Andrews 27 , Reference Haffey, Brady and Maxwell 29 , Reference Khatoon, Hill and Walmsley 30 Information in an app should clearly state “current as of.” Haffey et al reviewed over 300 medical apps and found only 6 apps that provided medical references for the app information.Reference Haffey, Brady and Maxwell 29 Other studies noted the lack of peer-reviewed articles or other evidence-based metrics as references in app content; therefore, the accuracy of information is very much on the end user of an app and not on the developer.Reference Khatoon, Hill and Walmsley 30 This has particular salience for medical professionals. If the quality of a medical learning app’s content is not verified and the original sources of information are not available, any information an app provides could be erroneous or outdated, potentially putting lives at risk.Reference Bibault, Leroy and Blanchard 25 Transparency and consistency of content quality are all the more important given the lax regulatory environment and relatively weak oversight of app stores.Reference Bibault, Leroy and Blanchard 25 , Reference O’Connor and Andrews 27 , Reference Haffey, Brady and Maxwell 29 - Reference Murfin 32 These conditions are especially true for disaster health workers, who may be working in high-risk areas where there is little time for content quality checks or tracking down original sources of information. Currently, informal word of mouth, professional recommendations, and institutional development of apps appear to be the more common ways of assessing content quality.Reference Archibald, Macdonald and Plante 24
User Engagement and Motivation
Another component for consideration in an app learning framework should be user engagement and motivation to use a mobile app for learning purposes. Studies have explored the importance of user engagement and comfort in using apps in health learning contexts.Reference Albrecht, Noll and von Jan 17 , Reference Nuss, Hill and Cervero 18 , Reference Teri, Acai and Griffith 22 , Reference Lewis, Burnett and Tunstall 31 One study examining the use of apps in a health science curriculum noted that mobile apps in higher education often feature interactive components such as games and quizzes to engage users, as well as to assist in the learning process.Reference Teri, Acai and Griffith 22 Importantly, a user’s environment can affect how likely they are to use a mobile app. In one study, a medical student noted their comfort level around using medical apps grew with increased exposure to apps while in training; other positive feedback included using apps to meet specific needs on medical student caseloads.Reference Nuss, Hill and Cervero 18 From a pedagogical perspective, it is necessary for mobile apps to be designed to promote active engagement in the learning process, which can lead to increased understanding and confidence of knowledge.Reference Teri, Acai and Griffith 22 Additionally, mobile apps can support reflective learning, which is an informal learning pathway.Reference Renner, Kimmerle and Cavael 20 Apps that cue individuals to reflect on their knowledge, surroundings, and current experiences can compensate for low awareness or motivation.Reference Renner, Kimmerle and Cavael 20 Therefore, apps with a reflective learning component could help facilitate users into building connections between knowledge and environmental context, as well as increase motivation to continue using the app. In addition, reflection can be used to continue and enhance professional development.Reference Renner, Kimmerle and Cavael 20 This may play an important role in motivating an individual to use an app and to continue engaging with it. These implications extend into the disaster health arena; mobile apps are only effective as long as users continue to trust and utilize them in their normal activities. This is all the more relevant for disaster health apps, which can help promote both JIT learning and training, as well as learning on a regular basis through consistent engagement, motivation, and possibly self-reflection by the user.
Proposed Framework Based on Systematic Review
Based on this literature review, the generation of a new learning framework for disaster health professionals is warranted since none of the sources describe a model of app use in disaster health learning. The proposed framework describes the use of mobile apps in learning by incorporating elements of adult learning, as well as components of learning models discussed above. A proposed framework for using mobile apps in a learning context for the disaster health workforce is presented in Figure 2. Based on our qualitative analysis, 3 central elements are suggested for enhancing the learning value of apps in a disaster health environment. These elements are user engagement and motivation, content source and quality, and JIT learning and training. This framework is most useful from the learning and educational perspective; however, it is vitally important that mobile app developers also be aware of these concepts. This may allow the development of a “common language” that both disaster health professionals and mobile app developers can communicate in, which may lead to more effective development of learning apps specifically for the disaster health workforce. In the framework, each of the elements takes equal precedence; no one element is preeminent among the three. Each element is a necessary base component of the other elements, and the summative qualities of the combined elements are used to influence the overall use of mobile apps for learning purposes.
Proposed Framework for Mobile Apps in Disaster Health Learning.
These elements also integrate a number of concepts that have been elaborated on in the literature but that have not been resolved in the field. Multiple studies have examined the issue of user engagement and continued motivation to use an app. In addition, a number of studies have noted the need for transparency in content source and a thorough examination of content quality. Finally, JIT learning and training are of particular importance because disaster health professionals need to call upon particular skills that may not be required absent an emergency. This concept is similar to O’Leary’s connection pathways, one of which is the transfer of knowledge to people.Reference Bullock 26 , Reference O’Leary 33 In such a context, health workers in disaster settings may need to refresh or also synthesize information that is not readily familiar to them, and this supports a JIT approach to learning.
These 3 principles are not unique to disaster health but are of utmost importance given the contextual environment that disaster health professionals operate in. In a disaster incident, there is often little time to start the user engagement process; however, the motivation to learn is high. Additionally, disaster health professionals must have already made the determination that an app is useful well before an incident occurs. This leads to the second element, which is content quality and source. In a disaster scenario, should a disaster health worker find an app useful (user engagement), they are counting on receiving accurate information and do not have the time nor the resources to ensure the information provided is correct and from reputable sources. This is where ensuring the content quality and transparency of content source is critical to achieving a successful outcome. There is also a potential opportunity to utilize reflective learning in user engagement, which may have important learning applications.Reference Renner, Kimmerle and Cavael 20
This proposed framework provides conceptual constructs considered vital to enhancing learning via mobile apps by the disaster health workforce. The main concepts established in the literature as being of particular importance are engaging users, ensuring the content is of high quality and from reputable sources, and ensuring that health care professionals can utilize the apps to achieve timely and critical learning in preparation for, response to, and recovery from disasters or emergency situations in their environment.
Limitations
Limitations include restricting this literature search to peer-reviewed publications written in English. Additionally, given the rapidly evolving field, other references may have come online since March 2015, which was when this literature search was last updated. Furthermore, gray literature searches were not conducted for this study.
Strengths
Some of the strengths of this work include analysis of the literature in a relatively novel area, as well as the proposal of a new app learning framework for use in the disaster health field. Another asset of this work includes using a clear app definition. Restricting the definition of app allowed us to standardize the results of the literature search and specifically hone in on certain elements unique to mobile apps.
Future Implications for Research and Practice
Future research should continue examining the use of mobile apps in disaster settings utilizing a conceptual learning framework like the one proposed. Mobile apps in general continue to generate high interest in the education and medical training fields. However, the onus will be on researchers to demonstrate that mobile apps are valued in the disaster health field, that apps will be an accepted tool for disaster health learning and training, and that the use of apps will contribute to better outcomes in disaster and emergency settings.
Future practice should examine the use of this framework for developing mobile learning apps tailored to the disaster health field and to examine their effectiveness in a learning context. In particular, mobile app developers may find this framework relevant for creating a learning app specific to the disaster health workforce. Additional areas of practice can include piloting an actual app through the 3 main elements of the framework seeking insight from end users to examine the practical implications of using apps in the field. There is much potential to expand and refine this framework as more research is conducted and more experience is gained.
CONCLUSIONS
This article examined the current state of mobile apps in disaster health settings, sought to understand and situate apps in a health learning context, and proposed an app learning framework built from prior learning models but attuned to the disaster context and the perceived needs of the disaster health workforce. Despite the limited number of publications, this article identified several learning models that were utilized in routine medical and health environments. As described, none were sufficient as a model of app use in disaster health learning. Therefore, an app learning framework was proposed that encapsulates important elements from these prior models and adds additional components and concepts. Future research should examine the usefulness of this framework in a simulated exercise or real-world disaster health environment. Additional research should continue examining the health and disaster health literature as other mobile learning concepts are introduced or clarified.
Disclaimer
The findings and conclusions in this report are those of the authors and do not necessarily represent the official positions of the Uniformed Services University of the Health Sciences, the Department of Defense, or the Centers for Disease Control and Prevention.
