No CrossRef data available.
Article contents
Outcomes of evidence-based clinical practice guidelines: A systematic review
Published online by Cambridge University Press: 01 November 2004
Abstract
Objectives: This study aimed to investigate whether evidence-based clinical practice in health care improves outcomes for patients, personnel, and/or organizations.
Methods: A systematic review of studies was conducted with various quantitative and qualitative methods up to the Spring of 2002. Protocols were used in quality assessment. Data synthesis is descriptive in a narrative form.
Results: Of 305 assessed articles, eight studies were included. The outcomes in the included studies were related to the experiences of patients and personnel and to organization concerning changed patient care and resource utilization. Because the included studies are heterogeneous in design, focus of research area, and scientific quality, the scientific foundation for the findings is weak. There is some support that evidence-based clinical practice guidelines, when put to use, improve outcomes (i) for patients—less likelihood of showing worsening of skin condition and disruption of skin condition improves more rapidly for infants; (ii) for personnel—support in daily work situation; and (iii) for organizations—decreased admission rates and length of stay, less resource utilization and reduced costs.
Conclusions: There is a need for further research as the findings are based on a rather limited number of studies. There is a tendency toward support for the idea that outcomes improve for patients, personnel, or organizations if clinical practice in health care is evidence-based, that is, if evidence-based clinical practice guidelines are used, although these findings could be specific to the settings and context of the studies reported in this systematic review.
Information
- Type
- GENERAL ESSAYS
- Information
- International Journal of Technology Assessment in Health Care , Volume 20 , Issue 4 , November 2004 , pp. 427 - 433
- Copyright
- © 2004 Cambridge University Press
A growing interest in evidence-based health care (EBHC) has been demonstrated by several kinds of health-care professionals since Sackett et al. (23) stated “the practice of evidence-based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research” (p. 2). There are two main ways in which research evidence can be used: (i) to improve clinical practice and (ii) to improve health service management (9). EBHC and clinical guidelines aim to promote effectiveness and, thereby, improve quality (6). One way to provide EBHC is to use clinical practice guidelines that aim at reducing inappropriate variation in practice (26). In a systematic review including eighteen studies and involving more than 467 healthcare professionals, the authors identified and assessed studies of the introduction of clinical practice guidelines in nursing, midwifery, and other professions allied to medicine. They conclude that there is some evidence that guideline-driven care is effective in changing the process and outcome of care. Grimshaw and Russel (10) investigated the effect of clinical guidelines on medical practice. They conclude that explicit guidelines do improve clinical practice when introduced in the context of rigorous evaluations. However, the size of the improvements varied greatly. Neither of these two reviews (10;26) considers whether the clinical guideline used is evidence-based.
There are many approaches to change clinical care and to implement guidelines. Depending on what the changes aim at, the target group, the clinical setting and the barriers and facilitators found, many of these approaches could have some value and may be effective (11). In a study aimed at determining which attributes of clinical practice guidelines influence the use of guidelines in decision making in clinical practice, it was demonstrated that evidence-based recommendations were used more than recommendations for practice that were not based on research evidence (12). The interventions that are shown to be effective to promote a tendency toward change in clinical practice are educational outreach visits, reminders, multifaceted interventions, and interactive educational meetings (1). The aim of this systematic literature review is to investigate whether evidence-based clinical practice in health care improve outcomes for patients, personnel, and/or organizations.
METHODS
Study Selection
In the process of specifying the assessment problem and determining inclusion criteria, a structure presented by Flemming (7) was used and the question at issue was “To what extent are outcomes improved for patients, personnel and/or organizations if clinical practice in health care is evidence-based?” This review explored the outcomes of evidence-based clinical practice in studies conducted with quantitative as well as qualitative methods. All categories of health-care professionals are included as well as any kind of health-care or clinical practice, hospital- or community-based. The interventions described in the retrieved studies should state in what way guidelines, practice, or decision making is evidence-based, that is, that the research are critically appraised and that considerations are made upon the strength of the evidence. Studies were included in this review if the outcome was measured in relation to (i) patients concerning health-related quality of life, mortality, morbidity, symptom score, functional status, self-care, experience of care; (ii) health-care professionals concerning experiences of work situation; and/or (iii) health-care organizations concerning cost-effectiveness, changed patients care, and resource utilization.
Literature Search
The literature search was carried out using standard methods and covered databases such as Medline (1985–March 2002), Cinahl (from start to Feburary 2002,), and the Cochrane Library (from start to April 2002). The search strategy in each database included terms covering the supposed content of the studies and the different types of study designs of interest combined with AND or OR. Search terms in Medline were, for example, “Evidence-Based-Medicine,” “Health care category,” “Organization and administration.” Search terms in Cinahl were, for example, “Professional-Practice-Evidence-Based,” “Program-Development,” “Evaluation,” “Professional-Practice-Research-Based,” “Practice-Guidelines.” Search terms in Cochrane Library were “Evidence-based medicine,” “Practice guidelines,” “Implementing” and “research,” “Evidence-based” near “practice”. The literature searches gave 2,824 references at the abstract level. These were reviewed by two independent reviewers. Considering the inclusion criteria, 298 references were regarded as possibly relevant for inclusion. The manual search in the articles' reference lists added fifteen references. Of the 313 references of which a full-text version was ordered, eight were not received for logistic reasons. Finally, 305 articles remained for assessment.
Quality Assessment
A protocol was used with the purpose of identifying relevant studies regarding the inclusion criteria. Each of the 305 articles was assessed for inclusion in the review by one of the authors (C.B.). Two independent reviewers (C.B., A.W.) made the final decision for inclusion. Discrepancies were discussed and resolved by data extraction from the study. Of the 305 articles, 10 were eligible for inclusion and 295 articles were excluded due to insufficient agreement with criteria of inclusion. Each of the ten included articles was reviewed by two independent reviewers (C.B., A.W.) with the purpose of establishing each study's level of evidence. The reviewers used protocols that focused on description and validation of study-design, diagnostics (if relevant) and intervention. Data were extracted regarding the size, precision, and tendency of the effects, also considering clarity, reasonableness, and soundness of results of studies using qualitative methods (8;13;14;19;28) . The criteria for study quality concerning level of evidence were based on Britton, 2000 (2); SBU, 1999 (24); and Willman and Stoltz, 2002 (28); and were formulated as high (I), moderate (II), and low (III).
Data Synthesis
Because the included studies were heterogeneous regarding the overall designs and kind of measurements used, the data synthesis is descriptive in a narrative form. The level of evidence for each study was the foundation for establishing the strength of evidence concerning the grades of recommendation (see Table 1), following SBU–The Swedish Council on Technology Assessment in Health Care (2); modified after Centre for Evidence-Based Medicine (4); Scottish Intercollegiate Guidelines Network (25).
RESULTS
The ten articles included eight studies (Table 2). They were published between 1995 and 2002 and conducted during the period 1994–1999. The overall designs of the studies differ; only Perlstein et al. (20;21) and Perlstein et al. (22) are of a similar design. It is not possible to draw any conclusions about improvement of outcomes from four of the studies due to small populations, short follow-up, large number of dropouts, unclear reporting of sites and time frames, and unclear relation to implementation of guidelines (3;5;17;27) . The remaining four studies show improved outcomes in relation to (i) patients' experiences concerning less likelihood of showing worsening of skin condition and disruption of skin condition improving more rapidly for infants (grade 3, see Table 1) (15;16), (ii) personnel's experiences concerning support in daily work situation (grade 4, see Table 1) (18) Table 3, (iii) organizations concerning decreased admission rates, length of stay, less resource utilization, and reduced costs (grade 4, see Table 1) (20–22), Table 4. In our analysis, the strengths of evidence concerning grades of recommendations are weak.
Outcomes in Relation to Patients' Experiences
In a well-conducted prospective study that evaluated the effectiveness of a clinical practice guideline for neonatal skin care, the results showed that disrupted skin conditions improved more rapidly after implementation of the guidelines and that infants were less likely to show worsening of skin condition scores (15;16). This study was a large multicenter study from 51 sites located all over the United States, with 4,755 registered nurses assigned to all participated units. There were 65 nurses attended as site coordinators and a total of 2,820 infants, and 11,468 observations were performed. In addition to measure of symptom score in infants, skin care knowledge in site coordinators as well as skin care practice in relation to guidelines, care, and treatment given were also measured, which demonstrated increased knowledge and adherence with guidelines.
Outcomes in Relation to Personnel's Experiences
In a qualitative study, Marshall et al. (18) focused on how practitioners experienced the process of implementation concerning guidelines for the management of venous leg ulcer. The result illuminated the obstacles for changing practice in relation to lack of skills, expertise and resources, patients' ability and willingness to comply, as well as the gaining of new skills and knowledge and ways of identifying areas that were in need of improvement. The result also pointed out a positive impact in that the guideline ensured consistency and promoted a team approach. The practitioners' attitudes to and perceptions of guidelines were positive, regarding the guidelines as a support in the daily work situation.
Outcomes in Relation to Organization
In a prospective study that also included a historical control group, the results showed that, after implementation of a guideline for inpatient care of infants with bronchiolitis, there were significant changes in decreased yearly admission rates, length of stay, use of β2-agonist inhalation, nasopharyngeal washes, x-rays, blood gas, antibiotics, and costs (20;21). In a similar study, Perlstein et al. (22), after implementation of guidelines for care of children with acute gastroenteritis, showed a nonsignificant decrease in emergency department encounters, admission rates, and costs and a significant change in decreased mean length of stay and increased admission of children with moderate/major illness severity. Although there was a change in documentation of hydration status and discharge diet, this study failed to show guideline adherence (22). Even though both these studies (20–22) were well conducted with a fairly sufficient follow-up, one has to bear in mind the limitations that follow with comparisons of populations that are not concurrent in time and uncontrollable confounding factors. Factors such as other changes made at the hospital or community that could have had an impact on, for example, costs and admission rates. Such matters are most likely influencing the possibility to generalize from these studies. In conclusion, even though the scientific foundation is limited, it seems the use of evidence-based guidelines can improve outcomes in relation to organizations in the sense of decreased admission rates, length of stay, and less resource utilization, which reduces costs.
DISCUSSION
The findings show a tendency toward support for the idea that an evidence-based clinical practice, constituted by the use of evidence-based clinical practice guidelines, improves outcomes for patients, personnel, and organizations. Caution is needed to interpret and generalize the findings to other settings than those studied. All but one of the studies that are the foundation for the conclusions are from the United States and they describe settings in neonatal and infant care (15;16) and at one children's hospital (20–22). Concerning improved outcomes for personnel, the included study is from the United Kingdom and the setting is primary health care (18). It is possible that the findings are specific to the settings and contexts of each of the studies. There is a need for further research to strengthen the findings and make the scientific foundation for improvements in outcomes for patients, personnel, and organizations sufficient. Presumably, the relevant outcomes should be of importance for patients, such as health-related quality of life, mortality, morbidity, symptom score, self-care, experiences of care, and life situation.
To examine professional behavioral change and the effectiveness of guidelines, the “golden standard” in designing studies is randomized control trials (RCTs) in which the units of randomization and allocation are the provider (26). One example of this type of design has been conducted by Network for Evidence-Based Practice In Northern and Yorkshire (NEBPINY; unpublished data, 2001). This trial is randomized by practice, and the aim was to estimate effects of a multifaceted program to implement evidence-based practice on the recorded process of primary-care and patient-assessed outcomes related to three study conditions—stable angina, asthma, and leg ulcers. The trial was not included in this systematic review because it was not yet published and the draft still lacked information in some of the study sections. It is an interesting trial, and it has the potential to be of inspiration for further research. The qualitative study by Marshall et al. (18) was conducted as a parallel study to the RCT by NEBPINY, to investigate the practitioners' perception of the guideline program. This knowledge is of importance to facilitate the implementation of evidence-based clinical guidelines in the best way.
In this systematic review, one of the inclusion criteria was that the included study should explain in what way the clinical practice could be assumed to be evidence-based, that is, critical appraisal of research results and consideration made upon the strength of evidence. Among the excluded studies there are examples that fail to demonstrate this. If the excluded studies had been further analyzed it is not likely that any of them would have made a profound change to the result of this review due to the fact they did not fulfilled the inclusion criteria. The inclusion criterion about evidence-based clinical practice proved to be one of the most important criteria for inclusion or exclusion in this systematic review. Because the answer to the question at issue should tell something about outcomes in a clinical practice that was evidence-based, there had to be a way to explore this. The choice to limit the criteria to a description of critical appraisal of research and considerations made upon the strength of evidence was taken under the assumption that EBHC will improve critical thinking. This might seem like a rather strict way of defining EBHC, but it is important to separate EBHC from research utilization, which do not always include critical appraisal.
Policy Implications
Much of health-care provision is done under uncertainty. The whole movement toward EBHC tries to limit and reduce this uncertainty by assessments of the outcomes of different options for health care. It is important then that the policy and practice of EBHC is monitored and continuously assessed. It is common among researchers to conclude that more research is needed; however, this is precisely the implication of the findings in this study.
CONCLUSION
As mentioned, there is a need for further research as the findings are based on a rather limited number of studies. We found a tendency toward support for the idea that outcomes improve for patients, personnel, or organizations if clinical practice in health care is evidence-based, that is, if evidence-based clinical practice guidelines are used, although these findings could be specific to the settings and context of the studies reported in this systematic review.
The authors gratefully acknowledge The Swedish Society of Nursing who supported Christel Bahtsevani with a grant. We also thank the School of Health and Society, Malmö University, Sweden, for support in daily work situations and Chatarina Björdal, MNSc, who contributed with great knowledge and enthusiasm in the assessing of references at abstract level.
References
Bero LA,
Grilli R,
Grimshaw JM, et al.
1998
Closing the gap between research and practice. An overview of systematic reviews of interventions to promote the implementation of research findings.
BMJ.
317:
465–
468.Google Scholar
Britton M.
2000
Så graderas en studies vetenskapliga bevisvärde och slutsatsernas styrka [Grading of level of evidence of studies and strength of recommendations] [in Swedish].
Lakartidningen.
97
(40):
4414–
4415.Google Scholar
Button D,
Roe B,
Webb C, et al.
1998
Consensus guidelines for the promotion and management of continence by primary health care teams: Development, implementation and evaluation.
J Adv Nurs.
27:
91–
99.Google Scholar
Centre for Evidence-Based Medicine.
1998.
Levels of evidence and grades of recommendation.
Oxford:
Available at: http://www.cebm.net/levels_of_evidence.asp. Accessed: September 10, 2003].
Dufault MA,
Willey-Lessne C.
1999
Using a collaborative research utilization model to develop and test the effects of clinical pathways for pain management.
J Nurs Care Qual.
13:
19–
33.Google Scholar
Effective Health Care.
1999
Getting evidence into practice.
NHS Centre for Reviews and Dissemination.
The University of York, UK.
5
(1):
1–
16.
Giacomini M,
Cook D,
Guyatt G.
2002
Therapy and applying the results qualitative research.
In: Guyatt G,
Rennie D, eds.
Users' guides to the medical literature a manual for evidence-based clinical practice, vol. 1.
Chicago:
AMA Press:
433–
448.
Gray JAM.
1997.
Evidence-based healthcare: How to make health policy and management decisions.
Edinburgh:
Churchill Livingstone;
Grimshaw JM,
Russel IT.
1993
Effect of clinical guidelines on medical practice: A systematic review or rigorous evaluations.
Lancet.
342:
1317–
1322.Google Scholar
Grol R,
Dalhuijsen J,
Thomas S, et al.
1998
Attributes of clinical guidelines that influence use of guidelines in general practice: Observational study.
BMJ.
317:
858–
861.Google Scholar
Guyatt GH,
Sackett DL,
Cook DJ.
1993
Users' guides to the medical literature II. How to use an article about therapy or prevention. A. Are the results of the study valid?
JAMA.
270:
2598–
2601.Google Scholar
Guyatt GH,
Sackett DL,
Cook DJ.
1994
Users' guides to the medical literature II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients?
JAMA.
271:
59–
63.Google Scholar
Lund CH,
Kuller J,
Lane AT, et al.
2001
Neonatal skin care: Evaluation of the AWHONN/NANN research-based practice project on knowledge and skin care practices.
J Obstet Gynecol Neonatal Nurs.
30:
30–
40.Google Scholar
Lund CH,
Osborne JW,
Kuller J, et al.
2001
Neonatal skin care: Clinical outcomes of the AWHONN/NANN evidence-based clinical practice guideline.
J Obstet Gynecol Neonatal Nurs.
30:
41–
51.Google Scholar
Manangan LP,
Banerjee SN,
Jarvis WR.
2000
Association between implementation of CDC recommendations and ventilator-associated pneumonia at selected US hospitals.
Am J Infect Control.
28:
222–
227.Google Scholar
Marshall JL,
Mead P,
Jones K,
Kaba E,
Roberts AP.
2001
The implementation of venous leg ulcer guidelines: Process analysis of the intervention used in a multi-centre, pragmatic, randomized, controlled trial.
J Clin Nurs.
10:
758–
766.Google Scholar
Oxman AD,
Cook DJ,
Guyatt GH.
1994
Users' guides to the medical literature VI. How to use an overview.
JAMA.
272:
1367–
1371.Google Scholar
Perlstein PH,
Kotagal UR,
Bolling C, et al.
1999
Evaluation of an evidence-based guideline for bronchiolitis.
Pediatrics.
104:
1334–
1341.Google Scholar
Perlstein PH,
Kotagal UR,
Schoettker PJ, et al.
2000
Sustaining the implementation of an evidence-based guideline for bronchiolitis.
Arch Pediatr Adolesc Med.
154:
1001–
1007.Google Scholar
Perlstein PH,
Lichtenstein P,
Cohen MB, et al.
2002
Implementing an evidence-based acute gastroenteritis guideline at a children's hospital.
Jt Comm J Qual Improv.
28:
20–
30.Google Scholar
Sackett DL,
Richardson WS,
Rosenberg W,
Haynes RB.
1997.
Evidence-based medicine: How to practice & teach EBM.
New York:
Churchill Livingstone;
SBU The Swedish Council on Technology Assessment in Health Care.
1999.
Evidence based nursing caring for persons with schizophrenia.
SBU/SSF-report No 4:
Stockholm;
Scottish Intercollegiate Guidelines Network.
2003.
Levels of evidence and grades of recommendation.
Edinburgh:
SIGN,
2000. Available at: http://www.sign.ac.uk/pdf/sign50section6.pdf. Accessed: September 10,
Thomas L,
Cullum N,
McColl E, et al.
2002.
Guidelines in professions allied to medicine (Cochrane Review).
In: The Cochrane Library, Issue 2.
Oxford:
Update Software;
Tranmer JE,
Kisilevsky BS,
Muir DW.
1995
A nursing research utilization strategy for staff nurses in the acute care setting.
J Nurs Adm.
25:
21–
29.Google Scholar
Willman A,
Stoltz P.
2002.
Evidensbaserad omvårdnad—en bro mellan forskning och klinisk verksamhet. [Evidence-based Nursing—a bridge between research and clinical practice] [in Swedish].
Lund:
Studentlitteratur;
Strength of Evidence Concerning Grades of Recommendationsa
Included Studies
Outcomes in Relation to Patients' and Personnel's Experiences
Outcomes in Relation to Organizations
You have
Access
- 111
- Cited by