Each year, approximately 80,000 knee arthroscopies are performed in the English National Health Service (NHS) (3). Arthroscopies are costly, and expose patients to small operative and postoperative risk, result in time off work, and interrupt usual activities. Clinical assessment has traditionally guided the decision whether to proceed with arthroscopy, yet clinical assessment does not consistently predict findings at arthroscopy (Reference Laupattarakasem, Laopaiboon, Laupattarakasem and Sumananont14;Reference Oberlander, Shalvoy and Hughston18;Reference Selesnick, Noble, Bachman and Steinberg20;Reference Solomon, Simel, Bates, Katz and Schaffer22).
Others have found magnetic resonance imaging (MRI) may be cost-effective before knee arthroscopy (Reference Weinstabl, Mueliner, Vecsei, Kainberger and Kramer25), and improves the prediction of findings at arthroscopy for mechanical knee disorders (Reference Lawson and Nutton15;Reference Warwick, Cavanagh, Bell and Marsh24). Given the long waiting lists, the NHS in our region decided to undertake a randomized controlled trial (RCT; 252 participants) to test whether MRI scanning in patients waiting for knee arthroscopy reduced the need for arthroscopy (Reference Bridgman, Richards and Walley1). In the intervention arm, surgeons were provided with the patients’ orthopedic clinical notes, which included the original management plan and a completed MRI report card. For each patient in the nonintervention arm, surgeons were provided with the patient's orthopedic clinical notes and a blank MRI report card (Reference Bridgman, Richards and Walley1). The RCT showed that providing an MRI report to a group of experienced surgeons did not result in any statistically significant difference in the proportions of patients proceeding to arthroscopy. This lack of difference in progression to arthroscopy contrasts with findings from observational studies and smaller RCTs, whereby patients having an MRI were less likely to receive an arthroscopy than patients who were not referred for MRI (Reference Bridgman, Richards and Walley1;Reference Marx7). A decision about whether or not to proceed with arthroscopy on the basis of clinical notes and an MRI scan is a decision “. . .made in the face of incomplete knowledge of the patient's true condition and the therapeutic effect of a given management strategy” (Reference Kuipers and Kassirer11). Although MRI is relatively accurate when compared with diagnosis at arthroscopy, little research has examined the influence of MRI on the decision-making process (Reference Igbokwe9;Reference MacKenzie, Dixon and Keene16).
We sought to increase the understanding of reasons for the lack of effect of an MRI in this RCT by identifying factors influencing surgical decision making. We asked surgeons to Think Aloud (TA) as they made their decisions about proceeding with arthroscopy for a subset of patients from the original RCT. The TA method has been used to identify the processes contributing to general and therapeutic decision making (Reference Elstein, Shulman and Sprafka4;Reference Newell and Simon17;Reference Offredy19). More recently, researchers have used the TA method to focus on decision making in specific contexts. TA is particularly useful in trying to understand decision making where there is uncertainty about the process, as is the case in decision making about arthroscopy (Reference Kuipers, Moskowitz and Kassirer12).
This study aimed to: (i) identify decisions made by orthopedic surgeons about whether or not patients on a waiting list for knee arthroscopy should proceed to arthroscopy, and (ii) describe similarities and differences between surgeons’ decisions for the same patients.
METHODS
The North Staffordshire Local Research Ethics approved the study. Participants in the study were five orthopedic surgeons employed at a tertiary teaching hospital. All were male with a mean age of 45 years, and they comprised all the surgeons previously involved in the RCT of MRI and knee arthroscopy (Reference Bridgman, Richards and Walley1). All were experienced in undertaking arthroscopy and had been practicing orthopedic surgeons for more than 10 years. Each orthopedic surgeon was given twelve anonymized sets of clinical notes, and twelve MRI scan reports, just as they were in the original RCT (Reference Bridgman, Richards and Walley1). Surgeons were blinded as to whether or not an arthroscopy actually had been undertaken during the RCT, and therefore also blinded to the actual diagnosis at the time of arthroscopy. They were asked to “think aloud” (TA) while reviewing the twelve sets of clinical notes and MRI scan reports – “. . .verbalizing overtly all thoughts that (in adult participants, at least) would normally be silent” during a specific task (Reference Gilhooly, Green, Ericsson and Simon5). The surgeons were asked to “. . .tell us everything you are thinking about from the time you first look at each patient's clinical record until you have reached a decision about whether or not to proceed with arthroscopy” and “. . .to talk constantly from the time you begin until the time you reach a decision.” The surgeons were not asked to justify or explain their rationale for each decision as this would detract from thinking aloud. The procedure was piloted with a senior orthopedic trainee who had not been involved in the RCT.
The twelve sets of clinical notes used for the TA were a subsample of patients in the original RCT (Reference Bridgman, Richards and Walley1). Notes were selected to include patients from both arms of the original RCT, include nine men and three women with an average age of 49 years (range, 21 to 70 years), and a range of clinical diagnoses and planned procedures. Patient case notes and MRI scan reports recorded age, sex, the original diagnosis formulated by their orthopedic surgeon following clinical assessment, that original assessing surgeon's confidence in their clinical diagnosis (reported on a visual analogue scale 0–100, where 0 = no confidence and 100 = total confidence in their clinical diagnosis), the procedure originally planned, pathology at MRI, and the radiologist's confidence in MRI findings (reported on a visual analogue scale 0–100, where 0 = no confidence and 100 = total confidence in their imaging diagnosis).
Before participants commenced the main task, they completed two small TA exercises (solving an arithmetic problem and a driving task) with the first two authors (S.D. and G.W.) to help them become familiar with thinking aloud (Reference Green, Gilhooly, Ericsson and Simon6). Participants were then asked to TA as they decided whether they would offer arthroscopy to each of the twelve patients. Although the researchers remained in the room to ensure the surgeons did not revert to “thinking quietly,” the surgeons were asked to act as if they were alone during the exercise. Apart from very occasional prompts from the researchers to “Please keep talking,” the researchers avoided eye contact and other verbal communication with the surgeons during the task as the surgeons focused on the sets of case notes and MRI reports. Audiotapes were made of the TA process.
Audiotapes were transcribed and independently verified by both researchers (SD and GW) attending the TA sessions. Transcripts were then analyzed using several steps. Key words and phrases indicating the surgeon's focus of attention were marked (Reference Khan, Tress, Hare and Wark10). Relationships between the factors being focused on and the final TA decision to proceed with arthroscopy or not, were identified. Aspects of the patient case notes and MRI report focused on by each of the five surgeons while thinking aloud, types of key phrases/words, and final decisions about whether or not to proceed with arthroscopy were then summarized into separate tables for each patient, with a mark indicating attention was given to that factor by the surgeon when thinking aloud. Factors were grouped into broad “factor group” (FG) headings to aid interpretation of the tables (FG1 = demographics, FG2 = original clinical diagnosis, FG3 = original treatment plan, FG4 = MRI report, FG5 = TA decision making, and FG6 = final TA decision). When summary tables were agreed by consensus to require more detail, verbatim quotes from the relevant surgeon are presented in the Results section.
RESULTS
Patient age, sex, original assessing surgeon's clinical diagnoses, original confidence in clinical diagnoses, original planned procedure, pathology at MRI, and original radiologist's confidence in the MRI findings are presented in Table 1 for each of the twelve patients (P). The radiologist's confidence in the pathology identified at MRI was not recorded in three patients (P8, P9, and P12). Five patients were from the intervention arm of the original RCT where the MRI scan report was seen by the surgeon (P4, P8, P9, P11, and P12), and seven were from the nonintervention arm.
Patient Demographics, Original Surgeon's Diagnosis at Clinical Assessment, Original Surgeons’ Clinical Confidence in Clinical Diagnosis, Pathology at MRI, Radiologists’ Clinical Confidence in MRI Findings, Surgeons’ TA Decisions, Diagnosis, and Procedure Undertaken at Arthroscopy for Each of the 12 Patients
a Patient was in intervention arm (MRI scan seen) of original randomised controlled trial.
AbMM, abnormal medial meniscus; LB, loose body; AbLM, abnormal lateral meniscus; LMT, lateral meniscal tear;
AbACL, abnormal anterior cruciate ligament; MMT, medial meniscal tear; OA, osteoarthritis; RACL, ruptured anterior cruciate ligament;
Y, yes, proceed with arthroscopy; N, No, do not proceed with arthroscopy.
Table 1 also includes surgeons’ TA decisions about whether to proceed with arthroscopy for each patient, diagnosis at the time of arthroscopy, and the procedure undertaken during the original RCT. In the TA study, there was agreement between all five surgeons about proceeding to arthroscopy for only five of twelve patients (P3, P4, P5, P6, and P9). Four of these patients had an original surgeon's clinical diagnosis of meniscal abnormality. In the fifth patient (P5), the clinical diagnosis was of a loose body. In all five patients, a meniscal tear was among the pathologies originally identified at MRI, and the diagnosis of meniscal tear was ultimately confirmed at arthroscopy during the RCT (Table 1).
In no instance was there agreement between surgeons about not proceeding to arthroscopy. However, more surgeons decided not to proceed to arthroscopy than to proceed for P1, P8, and P12. Only one patient (P2) of the twelve in this TA study did not actually have arthroscopy in the original RCT, and the majority of surgeons in our TA study decided that arthroscopy ought to have been offered to P2 (Table 1).
Summary tables (Supplementary Tables 1–3, which can be viewed online at www.journals.cambridge.org/thc) are presented for three of the twelve patients (P7, P8, and P10), to illustrate the key issues focused on by each of the surgeons in the decision-making process.
Supplementary Table 2 summarizes key issues focused on for P7. Three surgeons (S2, S3, and S4) decided that they would offer arthroscopy for P7 (see Supplementary Table 1, Factor Group 6 – FG6), with their final decision being influenced by the symptoms found at clinical assessment (including fixed flexion deformity and pain) and/or the MRI finding of a medial meniscal tear (FG5). Two surgeons decided not to offer arthroscopy, thinking aloud that a more invasive intervention was required; one surgeon (S5) was skeptical of the merits of MRI scans in people aged over 70 years, and wondered whether P7 had received previous arthroscopies and was fit for anesthesia (FG5). Aspects of the clinical notes and MRI scan report were focused on by all surgeons during the TA process. For example, P7's demographics (FG1), reason for referral (locking and giving way), presence of comorbidities, clinical diagnosis (FG2), original treatment plan (FG3), and MRI findings (FG4). S2 also focused on pain and TA the symptoms indicating a classic bucket-handle tear (FG2). He also desired additional diagnostic tests such as a plain radiograph to rule out bony deformity, and an arthrogram to ascertain the status of the articular cartilage (FG4). This surgeon was also concerned that loose bodies can be missed on MRI, and yet are easily identified and removed at arthroscopy. A different emphasis on comorbidities was also apparent for P7. S2 reporting that comorbidities just confuse things (FG2), and S3 thought they could affect decisions. S4 thought that the problem was chronic: “. . .so the main problem is the osteoarthritis then . . ., I'm not sure whether the MRI scan has helped in this case as I would rather rely on my clinical symptoms and not the MRI scan findings” (FG4).
Only one surgeon (S5) decided to proceed with arthroscopy for P8 (Supplementary Table 2). That decision was influenced by agreement between the MRI scan report and the clinical assessment: “. . .[P8] would benefit from surgical debridement for the osteoarthritis. MRI scan suggests the meniscus is normal, so the likely symptoms in the medial joint would be degenerative and I think MRI scan [has] more or less shown what the clinical findings [have] shown” (FG5). Four surgeons decided not to offer arthroscopy for P8. The reason behind the TA decision was not clearly stated by S1. S2 and S4 decided to “wait and see”, with S2 stating – “. . .[I] would probably consider surgery later on” (FG5). S3 decided against arthroscopy as “. . .no abnormality found in the knee other than a bit of wear and tear which will be normal for her age so she shouldn't have an arthroscopy” (FG5).
Two surgeons (S1 and S2) decided not to proceed with arthroscopy for P10, while three decided for arthroscopy (Supplementary Table 3). All paid attention to demographic factors (FG1) and progressive worsening, but attention was not consistent with regard to other factors such as pain, injury (FG2), original surgeon's and radiologist's confidence (FG3 and FG4). S5, who recommended arthroscopy, stated that seeing the MRI scan would have increased the confidence of the original surgeon in his clinical assessment (FG4). S1 noted that: “. . .I do understand the decision to scope this lady, but the scientific evidence would probably back me up if I said that she will not get any benefits from it. All in all, I would probably explain the situation . . . and if it were left to me, I would not arthroscope her, although, if she insisted that she wanted something done, I would undertake arthroscopy” (FG5).
DISCUSSION
There were no cases where all surgeons were agreed about not proceeding to arthroscopy. Although the TA reasons for proceeding sometimes differed, all surgeons agreed about proceeding with arthroscopy for five of the twelve patients. Table 1 illustrates that agreement occurred when the original clinical diagnosis was validated by the MRI report, and when OA was not the primary diagnosis.
For some cases, the decision about whether or not to proceed with arthroscopy was clear from the clinical assessment alone. This was particularly true for patients with meniscal symptoms. When surgeons are confident in their clinical diagnosis of meniscal damage, an MRI may not confer any advantage over clinical examination alone. However, it is also important to rely on the clinical examination in symptomatic patients when an MRI may be negative (Reference Crawford, Walley, Bridgman and Maffulli2;Reference Igbokwe9).
Internationally, the rate of arthroscopy for the treatment of osteoarthritis (OA) of the knee has increased dramatically over the past decade, despite a lack of trials or consensus guidelines on the management of this condition (Reference Siparsky, Ryzewicz, Peterson and Bartz21). Importantly, there is no “gold” level evidence demonstrating beneficial effects for arthroscopic debridement for the treatment of OA (Reference Kuusela and Pallab13). Among patients with OA of the knee, MRI may be overly sensitive and inadequately specific (Reference Stuart and Lubowitz23). We found that traditional views about arthroscopy, including the value of debridement for OA, are still present in a group of relatively young surgeons in a major teaching hospital. Six patients had a clinical diagnosis including OA. In one person with a clinical diagnosis of OA (P3), in whom an abnormal lateral meniscus had also been diagnosed clinically (and reported in the MRI), all five surgeons decided to proceed with arthroscopy. Of the remaining five patients with a clinical diagnosis of OA, between one and four of the surgeons made a TA decision to proceed with arthroscopy (Table 1).
We found that surgeons were more willing to recommend not proceeding with arthroscopy in the TA analysis than in the original RCT. The TA method found that surgeons’ decisions can be influenced by patient wishes. For example, one patient with OA in our study (P10) was believed unlikely to derive benefit from arthroscopy by two assessing surgeons. Nevertheless, it was stated that an arthroscopy would be offered if the patient insisted. One explanation for the continued use of arthroscopy for knee OA, despite limited evidence, may be the perception by the surgeon of minimal morbidity associated with the procedure, and the hope held by patients that arthroscopy may provide some relief from the symptoms (Reference Siparsky, Ryzewicz, Peterson and Bartz21). Involving patients in decision making about proceeding with arthroscopy is desirable. However, patients and doctors need to be able to base such shared decision making on best available evidence. Many surgical interventions are performed on evidence about clinical outcomes derived only from uncontrolled cohort studies (Reference Siparsky, Ryzewicz, Peterson and Bartz21). The underlying principles of decision making in the orthopedic setting are not formalized, and clinicians often characterize it as an informal intuitive process (Reference Hollingworth, Todd and Bell8). Our study supports this characterization; decision making about arthroscopy by surgeons is not necessarily uniform or evidence-based. To our knowledge, no best practice guidelines have been developed to help guide decision making for patients with mechanical knee disorders.
This study has highlighted different opinions between surgeons about the benefits of MRI in diagnosing knee disorders. For example, when the clinical diagnosis is not straightforward, MRI is believed useful. S5 stated about P3: “. . .this is where the MRI scan [has] helped significantly the clinician in this case, because the clinician was quite reluctant to say I'm very confident this man has all these [diagnoses]. He's a young man, he's had an injury before, I'm not sure if an arthroscopy might help here. An MRI scan would confirm all the findings [and] then that would make the clinical decision much easier.” However, reporting pathologies in the MRI scan report that were unable to be treated surgically, for example a medial collateral ligament strain, was frustrating to some surgeons as they were not relevant to their surgical decision making. Also, the reporting of OA changes to the patella in almost every patient's MRI report was of concern. In future trials, it would be useful to outline only clinically relevant knee pathologies in the MRI report.
For six of the eleven patients who proceeded to arthroscopy in the original RCT, at least two surgeons per patient decided not to proceed to arthroscopy in the TA analysis. In most instances, this decision was consistent with the findings of the MRI report. For the one patient (P2) who did not proceed to arthroscopy in the original RCT, three surgeons decided that the patient should actually have had an arthroscopy. In the original RCT, treatment plans were amended for 47% of participants in the intervention arm, although in that trial the decision was still made to proceed with arthroscopy (Reference Bridgman, Richards and Walley1). In this descriptive study, it was not possible to adopt the procedure followed in some health professional situations where TA decision making is tested against “gold standard” outcomes. Furthermore, we cannot be certain that the TA process used here perfectly mirrors the usual clinical situation, when TA researchers are not present (Reference MacKenzie, Dixon and Keene16). However, it did represent the decision-making process followed in the original RCT where the MRI and case notes were reviewed together away from the patient encounter. Furthermore, although it is accepted that the case and MRI scan notes given to each surgeon cannot replicate the “natural patient encounter,” others have noted that the skill of problem-solving based on predigested clinical data is a familiar activity for surgeons (Reference Khan, Tress, Hare and Wark10). A limitation of this study was the small number of surgeons able to be included. Additional research on arthroscopy decision making at other NHS Trusts, and indeed internationally, would add to our understanding of the decision-making process and the relationships between MRI and arthroscopy.
A strength of this study is that the decision-making task was explored concurrently rather than retrospectively, as concurrent verbal reports are more “consistent with behavior” than retrospective reports (Reference Gilhooly, Green, Ericsson and Simon5). However, such a concurrent design did not allow for a full exploration of each decision made. A subsequent project, analyzing qualitative interviews with each surgeon and also with the twelve patients whose case notes and MRI scan reports were used for the TA task, is currently being completed to explore surgeons’ views about the value of MRI scans for mechanical knee disorders more generally, and patients’ views about the original RCT, their decisions, and outcomes.
POLICY IMPLICATIONS
This study has implications for the current NHS focus on “patient choice.” Patients are now able to choose a treatment provider from a list of available providers at time of original clinical assessment and diagnosis. The treating provider does not necessarily re-examine the patient until the day of surgery. Therefore, the treating provider only has available the clinical notes from the original assessing clinician and MRI scan (if available) on which to base their treatment plan. Given the variation we have demonstrated between surgeons about the merits of proceeding with arthroscopy, it is possible that treating surgeons could be in the ethically and professionally difficult position of providing surgery to patients whom they do not believe will benefit from arthroscopy. The additional complication is that, once a patient has been offered arthroscopy and wishes to proceed, it may be difficult for surgeons to reverse that original treatment plan.
CONCLUSION
In conclusion, there tended to be disagreement between surgeons about proceeding with arthroscopy, particularly when OA was clinically diagnosed. Clinicians agreed about proceeding with arthroscopy when patients had clinically diagnosed meniscal abnormalities and tended to be younger. At a time when litigation is relatively common, surgeons need to be able to justify their decisions based on evidence. Further research should be undertaken to provide such an evidence base.
SUPPLEMENTARY MATERIAL
Supplementary Table 1: http://www.journals.cambridge.org/thc
Supplementary Table 2: http://www.journals.cambridge.org/thc
Supplementary Table 3: http://www.journals.cambridge.org/thc
CONTACT INFORMATION
Sarah Derrett, DipCpN, MPH, PhD (sarah.derrett@ipru.otago.ac.nz), Senior Research Fellow, Injury Prevention Research Unit, Dunedin School of Medicine, University of Otago, P.O. Box 913, Dunedin, New Zealand 9054
Gayle D. Walley, RGN Dipl., BA (g.walley@keele.ac.uk), Research Nurse, Orthopaedic Research Unit, The Guy Hilton Research Centre, Department of Trauma and Orthopaedics, University Hospital of North Staffordshire, Thornburrow Drive, Stoke on Trent ST4 7QB, UK
Stephen A. Bridgman, MD, FRCS, FFPH (sbridgman@hssd.gov.gg), Director of Public Health/Chief Medical Officer, Directorate of Public Health, Princess Elizabeth Hospital, Rue Mignot, St. Martins, Guernsey GY4 6UU, The Channel Islands
Paula Richards, BSc, MB, BS, MRCP, FRCR, (paula.richards@uhns.nhs.uk), Consultant Radiologist, Radiology Department, University Hospital of North Staffordshire NHS Trust, Princes Road, Hartshill Road, Stoke-on-Trent ST4 7LN, UK
Nicola Maffulli, MD, MS, PhD, FRCS, (n.maffulli@qmul.ac.uk), Centre Lead and Professor of Sports and Exercise Medicine, Centre for Sports and Exercise Medicine, Queen Mary University of London, Barts and The London School of Medicine and Dentistry; Consultant Trauma and Orthopaedic Surgeon, Trauma and Orthopedic Department, Mile End Hospital, Bancroft Road, London E1 4DG, UK
