Over the past two decades, endoscopic ultrasound (EUS) has become standard practice for staging gastrointestinal cancers, based on the diagnostic accuracy of the test (11;12;16;21). EUS staging (in particular celiac lymph node staging) has been demonstrated to be predictive of survival and the ability to achieve complete surgical resection (2;5;14;15). However, studies of patient prognosis following the use of EUS are not designed to compare patient survival or disease progression for patients staged with versus without EUS and, therefore, conclusions about the impact of adopting EUS cannot be made based on this type of evidence. Clinical guidelines recommend the use of EUS for presurgical staging of esophageal cancer where it may potentially improve treatment selection, in particular, selection for surgical resection (Reference Allum, Griffin, Watson and Colin-Jones1). The widespread clinical acceptance of the use of EUS, however, has preceded the current era of more rigorous evidence-based assessment of diagnostic tests.
The clinical value of a diagnostic test depends on how much it improves patient outcomes compared with existing tests (Reference Fryback and Thornbury6). In general, this depends on the accuracy of the test to detect or exclude disease, the impact of this information on treatment decisions, and the effectiveness of treatment. This can be assessed by randomized controlled trials of the new test versus standard practice. Frequently, trial evidence of test effectiveness is not available. Sometimes, studies of test accuracy may suffice when the effect of treatment is already known (Reference Guyatt, Tugwell, Feeny, Drummond and Haynes7;Reference Lord, Irwig and Simes13). In other situations, for example with EUS where the test results are interpreted in a cascade of clinical and diagnostic information (Reference Davies, Deans and Penman4), this may not be the case. In a hierarchical framework of evidence for diagnostic tests, accuracy studies provide level 2 evidence, whereas studies of the impact of the test on choice of therapy or management (“therapeutic impact studies,” level 4), or patient outcomes (level 5), provide evidence of greater clinical relevance (Supplementary Table 1, available online at http://www.journals.cambridge.org/jid_thc) (Reference Fryback and Thornbury6).
An important potential benefit of EUS for staging is in avoiding unnecessary surgery in patients with advanced disease. In these patients, EUS may lead to avoidance of surgical morbidity and mortality and improvements in quality of life. In addition, increased accuracy of staging with EUS may increase appropriate selection of patients for neoadjuvant therapies. This potentially may have a positive impact on chances of cure in those individuals diagnosed at an appropriate stage. A UK trial of the impact of EUS for staging gastroesophageal cancer on patient outcomes is expected to report in 2009 (17). This trial will capture all the downstream effects of EUS staging on patient outcomes. In the interim, studies investigating whether EUS changes patient management may provide preliminary evidence about the clinical impact of the test (Reference Guyatt, Tugwell, Feeny, Drummond and Haynes7). This systematic review was conducted as part of a broader Australian Medical Services Advisory Committee review of EUS (16) to determine the impact of EUS for esophageal cancer staging on patient management and survival.
METHODS
Literature Search
The electronic databases MEDLINE (1966 to May, week 1, 2005), PreMEDLINE (13 May 2005), EMBASE (1980 to 2005, week 20), The Cochrane Library (Issue 3, 2005), and health technology assessment Web sites were searched to identify relevant studies. Systematic search strategies were designed by a consultant health sciences librarian using a combination of indexing and text words covering many terms, including (but not limited to) endoscopic ultrasound, endosonography, gastrointestinal neoplasms (exploded), esophagus tumor, management, decision making, and survival. Reference lists of included articles were screened for relevant papers, and clinical experts were consulted. A single reviewer assessed studies for eligibility. Included studies were checked for eligibility by a second reviewer.
Study Selection
Included studies were controlled studies of EUS for staging esophageal neoplasms reporting therapeutic impact or patient survival. Therapeutic impact studies included were randomized controlled trials or interventional studies recording a pretest management plan and reporting the changes in the management plans. These study designs provide stronger evidence for demonstrating test impact on patient management than retrospective studies (Reference Guyatt, Tugwell, Feeny, Drummond and Haynes7). Exclusion criteria were less than 10 patients, catheter-probe or intraoperative EUS, or studies reporting patient prognosis following the use of EUS without a control group. No language restrictions were applied.
A total of 2,426 nonduplicate citations were screened and 8 eligible studies were identified (Figure 1). These studies included one systematic review, two studies reporting patient outcomes, and five studies reporting therapeutic impact.
QUORUM flowchart of study selection.
Data Extraction
The study characteristics and outcomes of eligible studies were extracted by one reviewer and checked by a second reviewer. Systematic reviews and studies reporting survival outcomes were assessed using quality assessment criteria for systematic reviews and case series described by the Centre for Reviews and Dissemination (Reference Khan, Riet, Glanville, Sowden and Kleijnen18). No formal guidelines for quality assessment of management studies were identified. These studies were assessed by criteria based on the elements described by Guyatt et al. (Reference Guyatt, Tugwell, Feeny, Drummond and Haynes7).
Due to dissimilarities in the characteristics and outcomes of the identified studies, no meta-analysis was performed. Too few included studies were identified to undertake an assessment of publication bias.
RESULTS
Systematic Review
One systematic review of EUS in gastroesophageal cancer published in 1998 was identified (Reference Harris, Kelly and Berry9). This high-quality review concluded that endoscopic ultrasound was highly accurate in staging tumors of the esophagus and stomach. The review did not identify any evidence of the impact of EUS on health outcomes. The two before–after studies of therapeutic impact identified are included in the current review (Reference Jafri, Saltzman, Colby and Krims10;Reference Nickl, Bhutani and Catalano19).
Therapeutic Impact
Study Characteristics
Five studies reported on the impact of EUS on pretest management plans. The design and quality of these studies varied (Table 1). No studies enrolled a consecutive series of patients based on a specific clinical presentation, rather than on referral for testing. No studies incorporated independent review of adequacy of pretest work-up, diagnosis, or contribution of the test to the management decision. In one study, three separate clinicians determined the management plans before and after EUS, and change in the number of concordant management plans, rather than the proportion of patients in whom management changed, was reported (22). The generalizability of the change in management data was limited in different studies by endosonographers determining the management plan (Reference Nickl, Bhutani and Catalano19), by determining the management plan out of a standard clinical setting (22), or by blinding the EUS operator to the pretest management plan (23). Only two of five studies provided concomitantly determined accuracy data (Reference Chong, Caddy, Desmond and Chen3;22), and no studies discussed evidence of the effectiveness of the treatments provided.
Characteristics of Included Studies Reporting the Effect of EUS on Patient Management Plans
a Consecutive refers to a consecutive series of patients presenting with a defined clinical indication.
Ac, accuracy; EUS, endoscopic ultrasound; FNA, fine-needle aspiration; N, no, NR, not reported; Se, sensitivity; Sp, specificity, Y, yes.
The type of outcomes reported and the adequacy of the breakdown of results also varied greatly between studies (Table 1). Three studies investigated patient management plans following esophageal cancer staging by EUS (Reference Chong, Caddy, Desmond and Chen3;Reference Nickl, Bhutani and Catalano19;22). One other study reported outcomes for mixed esophageal indications, including both staging and diagnosis (23). The fifth study only reported change in patient management data for a total population, which included patients with nonesophageal indications (Reference Jafri, Saltzman, Colby and Krims10).
Data Summary
Preston et al. (22) reported that the number of concordant management plans for radical surgery alone in esophageal cancer patients did not change with the addition of EUS information (20 percent with and without EUS). EUS staging information increased the number of patients for whom there were concordant plans for nonsurgical palliation (from 18.5 percent to 24.0 percent, p = .34, t-test). The level of agreement between the three surgeons was low, with a mean level of agreement of 56 percent without EUS data, and 62 percent with EUS data (average of two assessments, p = .39, t-test).
In two studies, EUS for staging esophageal cancer changed patient management in 24 percent and 29 percent of patients (Table 2) (Reference Chong, Caddy, Desmond and Chen3;Reference Nickl, Bhutani and Catalano19). EUS (without fine-needle aspiration [EUS-FNA]) changed patient management in a greater percentage of patients when all indications were considered (40 percent and 74 percent). Management also changed in a greater proportion of patients when data on the use of EUS for esophageal staging and/or diagnosis is combined (32 percent to 55 percent in two studies) (Reference Chong, Caddy, Desmond and Chen3;23). Further investigations were avoided in 14 percent to 33 percent of these patients (Reference Chong, Caddy, Desmond and Chen3;23), and surgery was avoided in 18 percent in a single study (23). Data on the proportion of patients in whom surgery was avoided was not reported separately for patients receiving EUS for staging esophageal cancer in any of the identified studies. Surgery was avoided in 10 percent to 16 percent of patients in populations with mixed indications (Table 2).
Effect of EUS on Patient Management Plans in Patient Populations Including Those Undergoing Staging for Esophageal Cancer
a n/N estimated from figure and reported % change.
EUS, endoscopic ultrasound; FNA, fine-needle aspiration.
Patient Survival
Study Characteristics
Two retrospective cohort studies, reporting survival in patients with esophageal cancer staged with versus without EUS using a historical control group, were identified (Reference Harewood and Kumar8;Reference van Westreenen, Heeren and van Dullemen24). These studies provide low-quality evidence of comparative survival as the potential for selection bias and differences in concomitant therapies is high (Table 3). Neither study reported the inclusion of all consecutive patients eligible for EUS staging.
Studies Reporting on Survival of Patients with Esophageal Cancer Staged with Versus without EUS
a Pearson's χ2 determined post-hoc by reviewers
EUS, endoscopic ultrasound; CI, confidence interval; FNA, fine-needle aspiration; CT, computed tomography; HR, hazard ratio; NR, not reported.
Data Summary
Harewood and Kumar (Reference Harewood and Kumar8) stated that there was no significant change in stage-dependent treatment practices during the time period of the study. EUS staging increased the selection of patients for preoperative neoadjuvant chemotherapy, from 15 percent in the historical control group to 33 percent (p = .01; Table 3). EUS also increased survival (adjusted hazard ratio 0.66; 95 percent confidence interval, 0.47 to 0.90; p < .01) and decreased the tumor recurrence rate (adjusted hazard ratio 0.63, 95 percent confidence interval, 0.43 to 0.87; p < .01; Table 3).
The survival outcomes reported in a different study by van Westreenen et al. (Reference van Westreenen, Heeren and van Dullemen24) were only for patients undergoing surgical resection, rather than for all patients tested or presurgically staged as resectable. The median survival time of patients staged with and without EUS was similar (25.6 versus 28.0 months, respectively; Table 3). The proportion of patients who underwent unnecessary laparotomy was not significantly different for staging with versus without EUS (50 percent versus 44 percent, p = .66, χ2 test).
DISCUSSION
This systematic review identified five studies reporting the therapeutic impact of EUS, and two studies reporting the effect of EUS for staging esophageal cancer on patient survival. In one of the latter studies, EUS increased selection of patients for preoperative neoadjuvant chemotherapy, increased survival, and reduced the recurrence rate (Reference Harewood and Kumar8). The other study found no survival benefit for patients staged with EUS, but only reported data for patients undergoing resection rather than for all patients staged (Reference van Westreenen, Heeren and van Dullemen24). Neither study was designed to assess potential improvements in quality of life following EUS staging to avoid unnecessary surgery in patients with advanced disease. Both studies were retrospective cohort studies with historical control groups and thus do not provide reliable evidence of the effect of EUS on patient survival due to the high potential for bias. Nevertheless, EUS has become accepted in clinical practice as a standard of care.
There is currently an ongoing randomized controlled trial investigating patient outcomes (including survival, treatment selection, complete resection rate, and quality of life, plus health resource utilization) following the addition of EUS to standard testing in the staging of patients with gastric and esophageal cancer (UK COGNATE) (17). Results of this trial are not expected until January 2009. In the absence of trial evidence, studies reporting on the impact of EUS on patient management may provide preliminary evidence of clinical impact (Reference Guyatt, Tugwell, Feeny, Drummond and Haynes7).
Five such studies of therapeutic impact reporting the use of a pretest management plan were identified. These studies indicated that EUS changed management in 40 percent to 74 percent of patient populations with mixed indications. Surgery was avoided in 10 percent to 16 percent of all patients. Two studies reported data specifically for patients undergoing EUS for staging of esophageal cancer. In these studies, EUS changed patient management in 24 percent to 29 percent of patients. However, the impact of EUS in avoiding surgery, an important potential benefit, was not reported separately for patients undergoing EUS for esophageal cancer staging.
Accuracy studies of EUS in staging esophageal carcinoma have estimated the impact of EUS on patient management (Reference Parmar, Zwischenberger, Reeves and Waxman20;Reference Vazquez-Sequeiros, Wiersema and Clain25). In one of these studies, surgery was avoided in 78 percent of patients undergoing EUS-FNA, or 45 percent of the total receiving EUS (Reference Parmar, Zwischenberger, Reeves and Waxman20). In the other study, contraindication to surgical resection based on detection of advanced or metastatic disease in 77 percent of patients undergoing preoperative nodal staging of esophageal carcinoma was reported (Reference Vazquez-Sequeiros, Wiersema and Clain25). These estimates are greater than suggested by the identified therapeutic impact studies. The absence of a pretest recorded management plan in the accuracy studies is likely to result in the inclusion of patients in whom surgery was not planned before EUS. The study by Vazquez-Sequeiros et al. (Reference Vazquez-Sequeiros, Wiersema and Clain25) includes all patients with esophageal carcinoma, including those in whom unresectable disease was detected with other prior imaging (computed tomography). Patients in whom surgery was not planned due to other factors, for example, health status or patient age, are also likely to be included in these studies. Similarly, Parmar et al. (Reference Parmar, Zwischenberger, Reeves and Waxman20) reported that EUS-FNA directed management in all patients biopsied. However, the pretest management plan for these patients was not reported and there may have been no change in the management plan. These discrepancies in the estimates of the effect of EUS on patient management emphasize the role of therapeutic impact studies in providing evidence for the value of EUS.
The EUS management studies varied widely in both study design and the quality of reporting. Twenty years ago, Guyatt et al. suggested factors that should be incorporated into the study design of diagnostic before–after studies to optimize the validity of the results (Table 4) (Reference Guyatt, Tugwell, Feeny, Drummond and Haynes7). Many of these factors have not been addressed in the studies identified in this review.
Before–after Studies of Therapeutic Impact: Optimizing Study Design
Note. Reproduced from Guyatt et al. (1986) (Reference Guyatt, Tugwell, Feeny, Drummond and Haynes7).
One study attempted to ensure the validity of the patient management plans by requiring three separate clinicians to determine the plans (22). The authors then reported the change in the number of concordant plans with the addition of EUS data. However, this approach did not provide information on the proportion of patients who had a change in management. The study did demonstrate that management plans varied between different clinicians (agreement of 56 percent and 62 percent before and after EUS, respectively).
The generalizability of the findings of the studies was also limited for a variety of other reasons. One of the most common problems encountered was authors reporting study results with an inadequate degree of separation by clinical indication, or by nature of management change. This greatly limited the usefulness of the available data for assessing the benefit of EUS in avoiding unnecessary surgery in patients with esophageal cancer, despite identification of five relevant studies. The apparent difference in outcomes when the data are considered for esophageal staging only, rather than for all indications, also highlights the importance of collecting and reporting data by specific clinical indication. Thus, high-quality studies of the impact of EUS on avoiding surgery in patients with esophageal cancer are still required.
A multitude of studies assessing the diagnostic accuracy of EUS have been published (Reference Harris, Kelly and Berry9;Reference Lightdale and Kulkarni12;16). However, evidence of accuracy only provides level 2 evidence in a six-level hierarchical framework of evidence for the efficacy of diagnostic tests (Supplementary Table 1) (Reference Fryback and Thornbury6). The current review, therefore, focused on level 4 and 5 evidence of efficacy, studies of therapeutic impact and patient survival, respectively. Although the studies identified have methodological limitations, they provide a higher level of support for a clinical benefit of EUS than accuracy studies alone (Reference Fryback and Thornbury6). A broader Australian Medical Services Advisory Committee review incorporating these studies determined that the body of evidence was sufficient for a recommendation for public funding in Australia (16).
Therapeutic impact studies can provide important evidence-based information on the clinical utility of diagnostic tests. They can be conducted in routine practice settings and may be simpler and less costly to conduct than a randomized controlled trial. Ideally such studies should be designed to replicate clinical practice, include a pretest management plan and report outcomes separated by indication and the intended purpose of the diagnostic test (e.g., diagnosis or staging) in a consecutive series of presenting patients (Reference Guyatt, Tugwell, Feeny, Drummond and Haynes7).
Recommendations for when these studies are required or where they may replace the need for randomized trials do not currently exist. In some situations, accuracy studies alone may be sufficient for test assessment, and neither trials nor patient management studies are required (Reference Lord, Irwig and Simes13). It has also been stated that where evidence of change in management from an accurate test leads to institution of a treatment proven to be effective, or avoidance of a procedure associated with considerable risk, the benefit of the diagnostic test is established (Reference Guyatt, Tugwell, Feeny, Drummond and Haynes7). Other scenarios where these studies are likely to have an important role include where therapeutic decisions are strongly influenced by factors other than test accuracy, such as individual patient characteristics, prior test results, or patient preference, and also where the new test is used to differentiate between several differential diagnoses.
The lack of comprehensive practical guidelines for the design, reporting, and appraisal of therapeutic impact studies may be the reason for the inconsistent and limited quality of existing data identified in the current review. The development of clear guidelines including recommendations for the role of these studies in providing evidence for diagnostic test effectiveness is essential.
In conclusion, although EUS is established in clinical practice, high-quality evidence about its impact on patient outcomes is not available at this time. In particular, further studies investigating the impact of EUS on patient survival are required. Two therapeutic impact studies have reported that EUS changes management in approximately one quarter of patients undergoing esophageal cancer staging. These studies provide a higher level of evidence of a clinical benefit of EUS than can be obtained from accuracy studies alone and contributed to a recommendation to provide public funding for EUS in staging of esophageal cancer in Australia.
POLICY IMPLICATIONS
Inclusion of therapeutic impact studies in health technology assessments can provide important additional evidence for test effectiveness. This review established that EUS has an impact on the management of patients with esophageal cancer, a necessary condition for the test to lead to improved patient outcomes. This finding provided higher evidence for test effectiveness than existed from accuracy studies alone. A recommendation for public funding of EUS staging in esophageal cancer patients was approved by the Australian Minister for Health and Ageing on February 5, 2007.
CONTACT INFORMATION
Suzanne M. Dyer, BSc(Hons), GradCertPH, PhD (sdyer@ctc.usyd.edu.au), Manager, Health Care Assessment Team, Systematic Reviews and Health Care Assessment, NHMRC Clinical Trials Centre, The University of Sydney, 88 Mallett Street, Camperdown, NSW, 2050, Australia
Dane B. Levison, BSc(Hons) (dlevison@hotmail.com), Health Outcomes Analyst, IMS Health, Level 5 Charter Grove, 29-57 Christie Street, St. Leonards, NSW, 2065, Australia
Robert Y. M. Chen, MBBS, MD (chenr@svhm.org.au), Gastroenterologist, Department of Gastroenterology, St. Vincent's Hospital (Melbourne), 41 Victoria Parade, Melbourne, Victoria 3065, Australia
Sarah J. Lord, MBBS, MS(Epi) (slord@ctc.usyd.edu.au), Epidemiologist, Systematic Reviews and Health Care Assessment, NHMRC Clinical Trials Centre, The University of Sydney, 88 Mallett Street, Camperdown, NSW, 2050, Australia
Stephen Blamey, MBBS, BSc (sblamey@netspace.net.au), Head, Department of Surgery, Monash Medical Centre, 348 Clayton Road, Clayton, Victoria, 3168, Australia
We thank the members of the advisory panel (Gerry FitzGerald, Kwun Fong, Barbara Joss, Trevor Leong, Ian Norton, and Mark Smithers) for contributions to the scope, clinical questions, and interpretation of studies in the broader endoscopic ultrasound Medical Services Advisory Committee (MSAC) review. We also thank the staff of IMS Health (formerly M-Tag Pty Ltd.), including Marc Bevan, John Gillespie, Amanda Ruth, and Richard Kenderes for general support in study retrieval, data checking, and editing during the conduct of the broader review. We thank Silke Walleser and Luke Marinovich for commenting on drafts of this paper and contributions to discussions on the interpretation and role of patient management studies. Frances Bluhdorn designed the literature search strategy. Silke Walleser contributed to data checking. Competing interests are as follows: S.B. is Chair of MSAC, which has provided advice to the Department of Health and Ageing in relation to public funding of endoscopic ultrasound. S.M.D., D.B.L., and S.J.L. are employed by organizations under contracts with the Department of Health and Ageing. S.B. and R.Y.C. were members of the expert advisory panel convened to provide guidance on the review of endoscopic ultrasound conducted by MSAC. R.Y.C. performs endoscopic ultrasound as a clinical consultant. This work was developed from a health technology assessment report prepared by IMS Health under a contract with Commonwealth of Australia, Department of Health and Ageing to conduct assessments for the MSAC to inform recommendations for public funding. The authors’ work was conducted independent of funding arrangements.
