The gap between what we know works on the basis of research evidence and what happens in practice, the “know-do gap,” has been identified as one of the greatest challenges facing health systems today.1 The gap results in enormous avoidable morbidity and mortality and is also responsible for many millions of dollars being wasted on ineffective or even harmful interventions. This waste is particularly damaging in low-income settings and health-crisis situations in which there the supply of health care resources is already inadequate.
After the 2004 tsunami in the Indian Ocean, The Cochrane Collaboration established a Web site called Evidence Aid (www.evidenceaid.org). Through Evidence Aid, The Cochrane Collaboration hoped to improve health in crisis settings by supporting the translation of the relevant high-quality health research available in its Cochrane reviews into practice in crisis settings.Reference Tharyan, Clarke and Green2 Reference Clarke3
Four years after the establishment of Evidence Aid, we interviewed humanitarian aid workers to establish what information they needed to address the know-do gap and how best to provide that information. While our primary intent was to evaluate Evidence Aid, the interviews provided an opportunity to explore more generally the types of health research information needed by humanitarian organizations and how that information could be provided most effectively.
METHODS
Identification of participants followed a “snowball” technique, building from contacts of the Evidence Aid team and existing collaborators. Messages were posted in relevant newsletters and e-mail distribution lists. Potential participants received a written explanatory statement, and their participation constituted implied consent.
Semistructured telephone interviews were conducted by 1 interviewer (T.T.). All interviews were conducted in English. Where possible, interviews were audio recorded and transcribed verbatim; otherwise detailed notes were taken and written up immediately after the interview. Data were analyzed in emerging themes using NVivo8 software (www.qsrinternational.com/products_nvivo.aspx). Participants were provided with a draft of the report to review and provide feedback.
Ethics approval was granted by the Monash University Standing Committee on Ethics in Research Involving Humans (Clayton, Victoria, Australia). The project was funded by The Cochrane Collaboration Steering Group.
RESULTS
Interviews were conducted in mid-2008 with 19 participants from a wide range of organizations, including the World Health Organization (4), international and local humanitarian aid organizations (including Médecins Sans Frontières, Oxfam Great Britain and Oxfam Australia, the International Rescue Committee, and others) (8), academic departments (4), government emergency management organizations (2), and health information advocacy groups (2) (numbers do not add to 19 because 1 participant had more than 1 role).
Interviewees were based in Africa (1), North and South America (3), Europe (8), Asia (2), Australia (4), and the Middle East (1) and included people responsible for the provision of care, management of service delivery, and strategic planning and evaluation. Several interviewees were in senior leadership positions, and at least 4 were currently involved in undertaking research in crisis health care. Interviewees had worked in provision or coordination of health care in a range of disaster situations, including Indonesia and Sri Lanka after the 2004 tsunami, the Solomon Islands after the 2007 tsunami, America after Hurricane Katrina, sub-Saharan Africa, conflict situations in Lebanon, and displaced populations in Sudan, the Democratic Republic of Congo, Myanmar, and Thailand.
The thematic analysis identified several key themes. Participants identified the need for research evidence to support their practice and, in particular, provided insight into the following:
-
• The types of questions that should be addressed
-
• The types of users for whom the information should be designed
-
• The type of information that was needed
-
• How this information should be structured
-
• That summaries of information were valuable
-
• That information about the context of the research was vital
Participants also discussed the value of research evidence and proposed approaches for how research evidence resources should be developed to be most useful. Their responses in each of these areas are described in the following text.
The Need
Participants were clear that there was a need for health research to inform health care practice in crisis settings and that there was currently a gap in provision of this kind of information. “[There is] nowhere to go to get a proper overview of these issues.” “Something is needed.”
“Effort needs to be put into gathering together available evidence and making sure people know what works and what doesn't work; this could be very valuable.”
Interviewees highlighted the range of clinical situations in which this evidence was needed, from crisis situations, to unstable but not emergency situations, conflict and postconflict contexts, and stable situations in low and middle income countries.
“There is a spectrum of things that could benefit from evidence, from a well-defined intervention for a condition that is similar between people in a predictable environment (malaria in stable developing countries) to a complex poorly defined solution that needs to be tailor-made in an unpredictable setting.”
Types of Questions
One of the strongest messages from interviewees was that there are primarily 2 types of evidence that are needed in crisis situations: clinical evidence and health systems delivery evidence.
“The Cochrane Library is useful, contains lots of clinical evidence that is fundamental, but health systems delivery issues are even more important—how can we get the aid to people?”
“Response to disasters is much more related to management of the situation, as compared with specific questions like how to treat crush injuries, for example, the value of outside response for search and rescue.”
Types of Users
Interviewees noted that there are several potential audiences for health research information in crises. There was a sense that the primary audience would be in the head offices of humanitarian agencies, particularly people who were responsible for developing guidelines and similar documents, because the policies for the provision of health care are usually established before crisis situations. “In most crisis situations, humanitarian organizations will deliver preprepared packages.”
There was also an acknowledgment that reliable information was needed at the time of disasters, when unexpected issues arise.
“You can't always anticipate everything. Like after the tsunami, we went in not knowing that we were going to face lots of adult tetanus.”
Interviewees noted that aid workers in the field might also benefit from provision of access to relevant health research information. “People on the ground . . . they need to know too.” However interviewees suggested that field workers may be less interested in reading about the best available evidence.
“[You have to] rely on people in the field wanting to be evidence-based; this is not always the case.”
Types of Information
Interviewees thought that a single site that contained summaries of systematic reviews of relevant research and also included other links to, commentaries on, or appraisals of other relevant information, such as World Health Organization guidelines and the Sphere handbook (www.sphereproject.org) would be more useful than any of these resources in isolation.
“If you are a decision maker and you need information, you need to go to a single resource site—probably want broad information, then overview, then ‘click on’ for more depth.”
“Would be good to see the evidence behind standard things. Few of the Sphere guidelines have evidence; it would be good to review these as a first thing.”
Structure
Participants suggested that resources should be classified in relation to systems of care, rather than using a structure based on disorders (as is common in the classification systems used for research more generally). “In crisis situations, [the] WHO [World Health Organization] advocates a system focus, not a disorder focus. . . . Policy making is systems focused, different from clinicians, who are disorder focused because they see patients not populations.”
Other interviewees suggested that the information resource might be structured around questions relevant to different types of emergency or to the country that had been affected.
Summaries of Information
Participants liked the concept of summaries of systematic reviews; however, they suggested that summaries based on the findings of a single review of a single topic were too focused. “Questions are narrow and limited. Research questions rather than clinical questions; only one individual treatment not whole range of treatments.”
This led to suggestions that overviews of reviews might be more useful.
“Summaries should be broader, perhaps one per prevention, diagnosis, therapy, rehabilitation.”
“[There are] many reviews on depression—hard to find answer to ‘how best to treat depression.’”
Interviewees also suggested that the language of the evidence summaries should be simple.
“Language needs to be edited so that non–clinically trained people can understand. Most areas might have only 1 clinical officer.”
Context
Interviewees wanted any summaries of research to be accompanied by discussion of the context in which the research has been undertaken and the applicability of the research to crisis situations.
“To make it more contextually relevant, a little background would be useful, to understand the difference between the epidemiology after the earthquake in Pakistan compared with the current crisis in Somalia.”
“[Include] comments from people who are familiar with the context.”
To be useful, participants said that evidence needed to be “Relevant to their situation, the condition of the person to whom they are delivering care, country, level of resources, available drugs and equipment, education level in terms of profession, language, reading ability, [and] way of conveying information diagrammatically.”
Practical suggestions were made about how to deal with relevance. “A lot of the studies have been done in very Western settings; maybe flag which studies have been done in developing contexts and enable searching by this?”
Relevance was also raised in terms of framing the information provided from a “What action should I take?” perspective “[I need information that is] well structured to make operational decisions, concrete answers . . . things that are very operational, like ‘shoot for the target’ rather than ‘this may be useful.’”
Interviewees also suggested that what constitutes the best, evidence-based treatment in nonemergency situations might not always be possible in emergency situations, and information was needed on what the best possible treatment would be in these circumstances.
“Often best practice is not possible because resources are not available so need second best options. People know what best practice is, but need to know what to do if best practice is not possible.”
Value of Evidence
Several interviewees suggested that people do not understand the methods or realize the value of systematic reviews of research evidence. “There is a lack of understanding of what systematic reviews are; the concept of accumulation of scientific evidence is not widely understood among health workers, policy makers, or funding agencies.”
Interviewees suggested that the value of systematic review methods and the results of this approach in providing the best available evidence should be explained. “Need . . . to explain . . . the process of the reviews so people can know to trust them as the best evidence available.”
Development
Interviewees were clear that development of evidence resources for use in crisis settings should be undertaken on the basis of a strong, active collaboration with humanitarian aid organizations. It was thought that resources should be “Developed in consultation with organizations working in these situations to better meet their needs in terms of scope, et cetera, but also to get it used.”
There was a sense that the current environment presents a promising opportunity to contribute to meeting the information needs of humanitarian organizations. “Good opportunity, particularly now; the time is right. ”
Several interviewees (from within and outside the World Health Organization) suggested that development and dissemination of evidence-based resources might be mediated by the World Health Organization.
Participants thought that collaboration between research providers and humanitarian aid organizations could do the following:
-
• Contribute to refining the purpose, content, and format of evidence summaries and the Web site
-
• Identify topics for summaries and highlight gaps and priorities
-
• Provide expert commentary on context and generalizability
-
• Commission new systematic reviews in priority areas
-
• Maintain a communication mechanism on the Web site to encourage visitors to suggest new content or comment on existing content
-
• Provide updates on work under way
-
• Disseminate summaries
-
• Suggest links to relevant materials
-
• Facilitate the use of evidence in planning by humanitarian aid organizations
One interviewee noted that there may be some barriers to the collaborative process “It's the same guys on every technical advisory group worldwide on this. . . . I’d encourage you to consult more widely but [I’m] at a loss to know how to do this. It's not going to be a priority, unfortunately.”
COMMENT
This small study identified a clear need for information from research to support health care in crisis settings. Participants highlighted that there is a gap in provision of this kind of information. It is important to note that the participants also suggested that there had been a shift in the thinking of the organizations for which they worked toward a more evidence-based approach. These factors encourage attempts to provide the findings of relevant health research to people wanting to plan and provide evidence-based health aid in crisis settings.Reference Mills, Robinson and Attaran4
While the participants welcomed information from health research, they noted that the relevance of health research is central to its usefulness. Participants wanted contextual information to be provided with the research to enable them to decide whether the results of the research were relevant. This finding parallels a recent call for researchers to “do all they can to describe the context of their trial”Reference Treweek and Zwarenstein5 to enable research users to establish whether the results are likely to apply in their setting.
Participants also emphasized that while research into clinical interventions is important, research into health systems and how to provide interventions most effectively may be more important to improving health care practice in crises. They noted that, as has been suggested in public health,Reference Victora, Habicht and Bryce6 research designs other than randomized controlled trials will be needed to provide evidence to answer these types of questions.
A range of potential actions resulting from a proposed collaboration between researchers and humanitarian aid agencies is a useful output from this study and will form a platform for future work.
There are several limitations to our study. We included only a limited sample of participants who might not be representative of the humanitarian aid community. We did, however, advertise widely and interviewed all who volunteered to participate. The interviews were publicized and conducted in English and via telephone or the Internet, restricting participation to people who felt sufficiently comfortable in speaking English and had access to reliable telecommunication. We had difficulty identifying and interviewing humanitarian aid workers who provide frontline health services, and their views may be underrepresented; however, several of the interviewees had previously been in roles in which they had provided frontline health services and could provide insight from their experience.
CONCLUSIONS
Humanitarian aid workers identify a clear and currently unmet need for reliable, relevant information from health research to inform practice in crises. To meet this need and support the translation of health research into practice, strong collaborations must be established between humanitarian organizations and producers of health research information to ensure that the resources developed are relevant, disseminated, and used.
Evidence Aid Evaluation Working Group Members:
Kathie Clark, Cochrane Collaboration Ombudsman; Mike Clarke, UK Cochrane Centre; Paul Garner, Cochrane Infectious Diseases Review Group; Sally Green, Australasian Cochrane Centre; Claire Harris, Centre for Clinical Effectiveness, Southern Health; Pisake Lumbiganon, Thai Branch of Australasian Cochrane Centre; Helen Smith, Cochrane Infectious Diseases Review Group; Prathap Tharyan, South Asian Cochrane Network; and Tari Turner, Monash Institute of Health Services Research, Monash University.
Author Disclosures: The author reports no conflicts of interest.
Funding/Support: The project was funded by The Cochrane Collaboration Steering Group. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript, however they were provided with a copy of the manuscript before publication.
Our thanks to the interviewees for generously sharing their time, experience, and insights and to the Evidence Aid Evaluation Working Group for their support and good advice in planning and undertaking this evaluation. We would also like to thank Rebecca Ryan from the Cochrane Consumers and Communication Review Group for insightful feedback on the draft report and details of the current content of Evidence Aid and Sally Ross from the Centre for Clinical Effectiveness for editorial support.
