In recent years, diagnostic errors have been gaining notoriety as important contributors to patient morbidity, mortality, and healthcare expenditure.Reference Newman-Toker and Pronovost 1 – Reference Balogh, Miller and Ball 9 Previous studies of cases of diagnostic errors have demonstrated that major errors – those with the potential for significant impact on both therapy and outcome – occur commonly and are most often the result of a combination of system-based and cognitive factors.Reference Carroll and Buddenbaum 8 – Reference Custer, Winters and Goode 12 Autopsy-based studies and reviews of malpractice claims have demonstrated that diagnostic errors are evident in 20 to 28% of cases and across a wide variety of clinical conditions, with up to 8% of these errors considered lethal.Reference Winters, Custer and Galvagno 3 , Reference Custer, Winters and Goode 12 , Reference Shojania, Burton and McDonald 13
Data characterising diagnostic errors in the paediatric population are limited. In a multisite survey of ambulatory paediatricians, Singh et al determined that diagnostic errors occurred commonly, and that nearly half of respondents reported patient harm as a result of these errors.Reference Singh, Thomas and Wilson 14 In paediatric cardiology, diagnostic errors have been reported in a wide range of clinical settings and domains, including in-hospital and ambulatory settings, echocardiography, and paediatric and neonatal ICUs.Reference Custer, Winters and Goode 12 , Reference Amaral and Granzotti 15 – Reference Goldstein, Metlay and Cox 24 Diagnostic errors may contribute significantly to poor outcomes in patients with critical CHD. Goldstein et al reported that diagnostic errors occur more frequently in children who die in the ICU compared with those who die in the emergency department or general ward.Reference Davalos, Samuels and Meyer 25 Recent studies have evaluated the paediatric and neonatal ICU patient population and noted a diagnostic error rate up to nearly 20%, mostly as a result of misdiagnoses of infectious, neurologic, and gastrointestinal conditions.Reference Custer, Winters and Goode 12 , Reference Cifra, Jones and Ascenzi 23 , Reference Goldstein, Metlay and Cox 24 However, no reports have specifically described the prevalence or impact of diagnostic errors in the paediatric cardiac ICU setting.
Many factors may uniquely contribute to the vulnerability of the critically ill cardiac population to errors in diagnosis: the wide variety of cardiac conditions, need for emergent surgery and cardiopulmonary bypass, unpredictable response to reparative and palliative surgery, and limited knowledge of care providers.Reference Thiagarajan, Bird and Harrington 26 Despite growing knowledge of their pervasiveness and societal burden, diagnostic errors are a relatively unexplored field within quality in general and paediatric cardiac critical care in particular.Reference Balogh, Miller and Ball 9 As such, a survey is a reasonable start to inform more systematic investigations in the paediatric cardiac ICU. The objective of this study was to improve the understanding of the perceived frequency and types of diagnostic errors in paediatric cardiac ICUs, as well as contributing factors and the perceived impact of such errors.
Materials and methods
Setting and participants
This is a cross-sectional, multicentre, survey-based study. We sampled a diverse variety of paediatric cardiac ICU care providers representing the most common practitioners in paediatric cardiac critical care. Specifically, we surveyed cardiac critical care registered nurses and medical provider groups. Registered nurses consisted exclusively of bedside nurses and provider groups included academic attending and trainee physicians – cardiology, critical care medicine, surgery, and neonatology – advanced practice providers – nurse practitioners and physician assistants – and hospitalists from three tertiary paediatric cardiac centres in North America. Each participating centre is a part of an academic children’s hospital with a dedicated paediatric cardiac ICU as defined by a specifically delineated cardiac ICU and primary paediatric cardiac ICU care providers. Surgical volumes at the participating centres for the year that surveys were administered were 957 cases for Site A, 783 cases for Site B, and 350 cases for Site C. Site A had 30.9% of cases classified as category 4 or 5 according to the Society of Thoracic Surgeons–European Association for Cardio-Thoracic Surgery, Site B had 28.1%, and Site C had 23.1%. Each paediatric cardiac ICU is staffed by attending physicians, fellows, and advanced practice providers with 24/7 in-house attending physician coverage. Institutional review board approval or waivers were obtained at all three study sites.
With permission, we adapted a 23-item survey developed by Singh et alReference Singh, Thomas and Wilson 14 The survey was administered using an internet-based website (www.SurveyMonkey.com). The original survey was modified so that the language in certain questions and response choices were more pertinent to the paediatric cardiac ICU environment and potential respondents, particularly in relation to specific missed diagnoses or procedures. Additional questions specific to the paediatric cardiac ICU setting were also incorporated. Our survey, after modification, consisted of 21 questions (Supplementary appendix 1). Similar to the original survey, we defined a diagnostic error as an event that occurs when diagnosis is unintentionally delayed, incorrect, or missed, as evidenced by subsequent acquisition of more accurate diagnostic information.Reference Graber, Franklin and Gordon 10 Our modified survey assessed demographic information, self-reported frequency of diagnostic errors, and previous training about diagnostic errors. As with the original survey, we sought to assess respondents’ perceptions of the most common process breakdowns and contributory factors associated with diagnostic errors, the most commonly misdiagnosed conditions, and the most effective strategies for error prevention. The most significant modifications to the original survey were made in those questions that sought to identify: the most commonly misdiagnosed conditions, and the frequency and severity of harmful sequelae resulting from diagnostic errors. The answer choices for misdiagnosed conditions were expanded so that they included conditions commonly encountered in paediatric cardiac ICU patients such as pulmonary overcirculation, circulatory shock, and arrhythmia. Respondents were also queried about their perception of the frequency of diagnostic errors occurrences that resulted in varying degrees of harm ranging from no harm to death. For most survey questions, response scales with forced ranking of the top three response choices were used. The modified survey was independently pilot tested by four attending cardiac intensivists for clarity and ease of completion. On the basis of pilot testing results, the estimated completion time for the survey was 15–20 minutes.
Members of the study team – J.M.C., M.L.M., R.A., and A.Y.S. – identified eligible study participants from each participating institution’s paediatric cardiac ICU. Registered nurses, advanced practice providers, hospitalists, and attending intensivists who considered the paediatric cardiac ICU as their primary location of employment were included. Trainee physicians who spent a minimum of 3 months in the paediatric cardiac ICU over the course of 3 years of fellowship training were also considered eligible. All eligible participants had a working e-mail account affiliated with their respective institution; e-mail addresses were obtained with the assistance of supervisory staff including Nurse Managers and Fellowship Program Directors, and study team members themselves. Each eligible participant was sent an initial e-mail invitation by a member of the study team to complete the survey followed by additional reminders 2 and 6 weeks following the initial request. Participants accessed and completed the survey anonymously.
Data analyses
Data were analysed using STATA software (version 12; StataCorp., College Station, Texas, United States of America). Respondents were divided into two groups according to role. Group 1 consisted of bedside registered nurses and Group 2 consisted of attending physicians, fellows, hospitalists, advanced practice providers, and other providers. The number of respondents was inadequate for sub-group analysis. Respondent demographics, as well as variables associated with the frequency, contributing factors, associated conditions, and strategies for prevention of diagnostic errors, were evaluated with standard descriptive statistics. Continuous variables are reported as means ± standard deviation for normally distributed variables and median and interquartile range for not normally distributed continuous variables. Categorical variables are reported as counts with percentages and analysed using the Chi-square test. Continuous variables were analysed using the Student’s t-test for normally distributed variables and Wilcoxon’s signed-rank test for skewed variables. For survey questions that asked respondents to rank choices from a list, weighted averages were calculated by first assigning 3 points for the first choice, 2 points for the second choice, 1 point for the third choice, and 0 for all other choices, and then computing the averages.Reference Singh, Thomas and Wilson 14 To evaluate for non-randomness of ranking, Freidman’s χ2 was used, examining items in pairs and adjusting the significance level for multiple comparisons. Analysis of variance was used to determine whether statistically significant differences were present between responses to each question by each participating centre.
Results
Between October 2014 and January 2015, a total of 435 survey invitations were sent to paediatric cardiac ICU care providers at the three participating study sites, including 144 individuals at Site A, 156 at Site B, and 132 at Site C. The overall response rate was 46%; 62% of respondents self-identified as registered nurses. Among respondents, registered nurses had more years of clinical practice in comparison with provider groups (9.8 versus 6.6 years, p=0.04). No differences were observed between the two groups with allocation for teaching and administrative responsibilities; however, compared with provider groups, registered nurses had more allocated clinical time (78 versus 68%, p=0.02) and less research time (6 versus 18%, p<0.01). Less than half of the respondents received formal or informal training about diagnostic errors in medical or nursing school or during clinical training (Table 1). Participant responses were analysed by centre, and no clustering effect was noted.
Baseline characteristics of survey respondentsFootnote *.
DE=diagnostic errors; IQR=interquartile range
* Data presented as n (%).
** Includes medical doctors and advanced practice providers. Statistical analysis was performed with STATA software (version 12; StataCorp., College Station, Texas, United States of America)
Reported rates and types of diagnostic errors
Overall, 79% of paediatric cardiac ICU practitioners – 92% of providers and 68% of registered nurses – reported five or more diagnostic errors yearly, whereas 46% – 32% of providers and 63% of registered nurses – reported 20 or more diagnostic errors yearly. Most (59%) reported ten or more harmful diagnostic errors, whereas 65% reported five or more permanently harmful errors per year. A minority of respondents (16%) reported five or more errors per year that resulted in severe, permanent harm, and nearly 10% reported that diagnostic errors were associated with five or more deaths per year at their centres.
Non-cardiac conditions including medication side effects and psychiatric conditions were ranked as the most commonly misdiagnosed conditions in paediatric cardiac critical care. In addition, registered nurses identified sepsis as commonly misdiagnosed, whereas provider groups highly ranked pulmonary overcirculation and viral illness being misdiagnosed as bacterial illness (Table 2).
Respondents’ ranking of conditions most commonly misdiagnosed in paediatric cardiac critical care.
Friedman test showed significant differences among the ranked options between nurses and provider groups (p=≤0.001)
* Includes medical doctors and advanced practice providers
Process breakdown
Provider groups identified delays in obtaining diagnostic studies related to the severity of the patient’s illness as the breakdown in diagnostic processes that was most strongly associated with error, whereas registered nurses reported delays in interventions related to the severity of the patient’s condition (Table 3). Among registered nurse and provider groups, system-related errors – specifically, suboptimal care coordination – were reported to be the most common contributory factors for diagnostic errors. Inadequate data assessment was the cognitive factor most strongly associated with diagnostic errors; anchoring on one diagnostic or treatment plan was also ranked highly as a contributor to diagnostic errors. Among miscellaneous factors leading to errors in diagnosis, provider groups most highly ranked time/workload, frequent interruptions, and provider inexperience (Table 4).
Respondents’ ranking of breakdowns in the diagnostic processes most commonly associated with diagnostic errors.
Friedman test showed significant differences among the ranked options between nurses and provider groups (p≤0.001)
* Includes medical doctors and advanced practice providers
Respondents’ ranking of most common contributory factors for diagnostic errors.
Friedman test showed significant differences among the ranked options between nurses and provider groups (p=≤0.001)
* Includes medical doctors and advanced practice providers
Methods to reduce diagnostic errors
Improved teamwork between all members of the healthcare team was the most highly ranked clinician-based method to minimise diagnostic errors. Provider groups also endorsed close follow-up of patients and solicitation of informal second opinions. Respondents ranked establishing feedback pathways to communicate changes in diagnoses to previous providers as the systems-based solution to minimise diagnostic errors (Table 5).
Respondents’ ranking of likely effective provider-based and system-based solutions.
EMR=electronic medical record
* Includes medical doctors and advanced practice providers
Friedman test showed significant differences among the ranked options (p=≤0.001)
Discussion
This study is the first to explore diagnostic errors in the dedicated paediatric cardiac ICU setting. Our findings build on recent data evaluating diagnostic errors in a variety of paediatric clinical settings including ambulatory care and paediatric and neonatal intensive care, in both academic and community-based centres. Our study determined that cardiac ICU providers perceive that diagnostic errors in the paediatric cardiac ICU occur frequently; nearly half of them reported diagnostic errors occurring more than 20 times per year. Diagnostic errors leading to harm were also common. Delays in diagnostic studies and interventions were the most commonly reported process breakdowns, and inadequate care coordination, data assessment, and high clinician workload were identified as important contributory factors. All clinicians ranked improving teamwork and feedback pathways as strategies most likely to be effective in preventing or mitigating the impact of diagnostic errors.
Our finding that paediatric cardiac ICU practitioners perceive harmful diagnostic errors occur frequently is aligned with recent findings suggesting the ubiquity of harmful and costly diagnostic errors in medicine. Diagnostic errors, compared with other safety concerns, are found to account for the most severe patient harm, the largest fraction of malpractice claims, and the highest total of penalty payouts; they are also thought by physicians to be more likely to cause death.Reference Tehrani, Lee and Mathews 6 , Reference Balogh, Miller and Ball 9 , Reference Cifra, Jones and Ascenzi 23 , Reference Graber 27 In children, Singh et al found that most general paediatricians report diagnostic errors to occur one to two times per month, and harmful diagnostic errors one to two times per year.Reference Singh, Thomas and Wilson 14 In comparison, we found that practitioners report diagnostic errors and harmful diagnostic errors occurring at a higher perceived frequency and severity in the paediatric cardiac ICU setting. More recently, Cifra et alReference Cifra, Jones and Ascenzi 23 noted that diagnostic errors occurred in 21% of cases discussed at the Morbidity and Mortality Conferences in the paediatric ICU. A 12.1% incidence of errors and high inter-rater reliability of error detection was found using a structured medical review tool to evaluate for diagnostic errors.Reference Singh, Thomas and Wilson 14 , Reference Goldstein, Metlay and Cox 24 Furthermore, our findings are consistent with the experience that inpatient diagnostic errors were more likely to be lethal.Reference Tehrani, Lee and Mathews 6 , Reference Cifra, Jones and Ascenzi 23
Interestingly, psychiatric conditions and medication side effects were among the most common conditions thought by care providers to be associated with diagnostic errors in the paediatric cardiac ICU. Our findings were similar to those of Singh et al who similarly found that medication side effects and psychiatric conditions were mostly commonly misdiagnosed among paediatricians.Reference Singh, Thomas and Wilson 14 Physician providers also highly ranked congestive heart failure resulting from excessive pulmonary circulation and misdiagnosis of viral illness as bacterial as conditions or occurrences associated with diagnostic errors.Reference Singh and Sittig 28 , Reference Schiff, Kim and Abrams 29 This finding suggests that the application of newer technologies within the paediatric cardiac ICU population has not yet achieved reliable prediction of cardiac output and the components of relative pulmonary and systemic blood flow. Similarly, current clinical biomarkers intended to screen for bacterial infections in critically ill children following congenital heart surgery are limited by inadequate sensitivity and specificity.Reference Shin, Jin and Hao 30 Consequently, there is widespread and variable antibiotic use among children’s hospitals in the United States, indicating an inability to target therapy accurately in situations of a suspected clinical infection.Reference Gerber, Newland and Coffin 31 By association, diagnostic errors in these settings may have important deleterious implications in the management of susceptible patients with shunted pulmonary circulation or in targeting appropriate therapies for patients with suspected infection.
We found that delays in investigation and intervention of abnormal medical data were ranked as the process breakdowns that were most highly associated with diagnostic errors in the paediatric cardiac ICU. Contrary to the ambulatory setting,Reference Singh, Thomas and Wilson 14 the significance of data management in the ICU has important implications in timely diagnosis and patient management.Reference Singh and Sittig 28 , Reference Schiff, Kim and Abrams 29 With considerable advances in technology, intensive care medicine is progressively becoming a quantitative domain.Reference Maher, Chang and Shin 32 , Reference Bronicki and Chang 33 Intensive care practitioners practice under conditions of uncertainty and are required to process voluminous amounts of patient data and think heuristically to arrive at binary decisions within short time frames. Paediatric cardiac surgery has been commonly associated with high patient complexity, cognitive overload, need for multiple specialists, rapidly changing plans, and uncertain prognosis.Reference Thiagarajan, Bird and Harrington 26 As a result, paediatric cardiac ICU practitioners face considerable diagnostic uncertainty, which can contribute to unplanned re-interventions and increased mortality in this vulnerable population.Reference Custer, Winters and Goode 12 , Reference Thiagarajan, Bird and Harrington 26 , Reference Mazwi, Brown and Marshall 34 Not surprisingly, participants in our study also identified inadequate care coordination, data assessment, and physician workload as key factors that lead to errors in diagnosis. Our findings suggest that cognitive and data management solutions to aid practitioners in better understanding when and how patients require further investigational studies and/or therapies may mitigate the occurrence and impact of diagnostic errors.Reference Singh and Sittig 28 , Reference Schiff, Kim and Abrams 29 , Reference Thammasitboon and Cutrer 35
Our findings suggest that targeting interpersonal communication and interactions with the healthcare delivery system by improving teamwork and feedback pathways may be effective strategies to mitigate diagnostic errors in the paediatric cardiac ICU. The frequent handover of patient information during provider shift changes intrinsic to the inpatient healthcare delivery highlights that multiple clinicians share diagnostic and management decisions. Our findings are consistent with high reliability organisation theory, which suggests that cross training, standardisation, and shared situational awareness are system-based solutions that lead to decreased variability and improved efficiency, thereby promoting safety in complex environments such as the paediatric cardiac ICU.Reference Wilson, Burke and Priest 36 , Reference Weick and Sutcliffe 37
Participants in our survey noted breakdowns in several dimensions of the Diagnostic Error Evaluation and Research taxonomy of the diagnostic framework.Reference Singh and Sittig 28 , Reference Schiff, Kim and Abrams 29 We found that diagnostic errors in the paediatric cardiac ICU do not result from one specific area but are a result from breakdowns in various limbs of the diagnostic process. As such, it is sensible to approach the problem of diagnostic errors from a multifaceted perspective.Reference Cifra, Jones and Ascenzi 23 , Reference Singh and Sittig 28 , Reference Schiff, Kim and Abrams 29 Recent reports focus on a cohesion of solutions with workflow in addition to grouping errors based on clinical context rather than specific cause.Reference Newman-Toker and Pronovost 1 , Reference Graber 27 A multifaceted approach addresses the breadth of contributory factors, which call for different but similarly integrated mechanisms to reduce diagnostic errors.Reference Balogh, Miller and Ball 9 , Reference Cifra, Jones and Ascenzi 23 Interestingly, our findings demonstrated that respondents did not rank highly the electronic health record as a stratagem to address diagnostic errors. This is contrary to findings in the outpatient setting and may represent differences related to a need for acute, time-critical decision-making in the paediatric cardiac ICU.Reference Winters, Custer and Galvagno 3 , Reference Hartzband and Groopman 38
Our study had some important limitations. First, this study was administered to paediatric cardiac ICU care providers in large, academic centres with dedicated paediatric cardiac ICUs. Some studies have noted a complex and inverse relationship between paediatric cardiac surgical case volumes and poor outcomes, particularly with higher levels of surgical complexity; this may be extrapolated to suggest a lower frequency of diagnostic errors in centres such as the three that are participating in this study.Reference Welke, O’Brien and Peterson 39 – Reference Pasquali, Jacobs and He 41 As such, the results may not be generalisable to smaller paediatric cardiac ICUs, ICUs with mixed cardiac and non-cardiac populations, and non-academic programmes. Participant responses were analysed by centre, and no important differences were found to suggest clustering. Second, the survey-based nature of this study with self-reported responses is subject to recall bias of undetermined significance and can result in both over- and under-estimation of error frequency. Although their ability to evaluate the incidence and aetiology of diagnostic errors is limited, survey-based studies have identified diagnostic errors as a major safety concern among both providers and patients.Reference Graber 27 Seeking the perceptions of medical care providers in various settings is widely accepted as a means for gaining insightful and useful information to understand the nature of diagnostic errors and improve patient safety.Reference Singh, Thomas and Wilson 14 , Reference Källberg, Ehrenberg, Florin, Östergren and Göransson 42 – Reference Thomas and Newman-Toker 46 Additionally, we included non-physician providers as part of the study participant where traditionally the diagnostic process is associated with physicians. The decision to include non-physician providers in this study speaks to the close relationship between physician and non-physician team members in the paediatric CICU and the key role of bedside nurses in the gathering of medical information necessary for the diagnostic process. Interestingly, the responses of the two participant groups were remarkably concordant. Finally, owing to the self-reported nature of our data, we were unable to identify which errors actually caused harm, and our findings require additional validation.
Conclusions
A growing body of literature has identified diagnostic error as a relatively under-reported yet significant contributor to patient morbidity and mortality. This study provides insight into the severity of the problem as perceived by frontline providers in the cardiac ICU. Inadequate care coordination, data assessment, and high clinician workload were identified as important contributory factors. Strategies targeting improvements in teamwork, feedback pathways, and provider workload were endorsed as effective approaches to mitigate the incidence and impact of diagnostic errors. These findings may provide insights for understanding the contributory factors, as well as the complex interplay of the systemic and cognitive factors, underlying diagnostic errors. As such, the data from our study could be used to inform future efforts to identify and mitigate diagnostic errors in the paediatric cardiac ICU.
Acknowledgements
None.
Financial Support
Cardiovascular Intensive Care Research Fund, Stanford Children’s Hospital, Palo Alto, California.
Conflicts of Interest
None.
Ethical Standards
Institutional Review Board approval or waiver was obtained from Stanford University School of Medicine, Northwestern University Feinberg School of Medicine, and University of Michigan School of Medicine.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1047951117002906
