In 2014, to reduce overutilisation of unnecessary imaging and build upon pre-existing adult guidelines, appropriate use criteria for initial paediatric outpatient transthoracic echocardiograms were endorsed by multiple national cardiovascular societies and the American Academy of Pediatrics. Reference Douglas, Khandheria, Stainback and etal1–Reference Campbell, Douglas, Eidem and etal4 These criteria classify common indications for echocardiograms as appropriate (A), may be appropriate (M), and rarely appropriate (R) based on the anticipated diagnostic yield of the study. This document provides guidance for cardiologists and non-cardiologists alike in evaluating whether an echocardiogram is indicated for the diagnostic work-up of various signs or symptoms in a patient without known CHD. In 2020, a subsequent appropriate use criteria consensus guideline provided further direction regarding the best utilisation of transthoracic echocardiograms in the longitudinal follow-up of patients with known CHD. Reference Sachdeva, Valente, Armstrong and etal5
On 18 March, 2020, Kentucky Governor Andy Beshear signed an executive order to cease all elective medical procedures in response to the COVID-19 pandemic effective through 27 April, 2020. Reference Friedlander and Stack6 Shortly thereafter, the American Society of Echocardiography affirmed the importance of limiting rarely appropriate studies, particularly with the new potential for transmission of the novel coronavirus during exams. Reference Kirkpatrick, Mitchell, Taub, Kort, Hung and Swaminathan7,Reference Barker, Lewin and Donofrio8 No specific additional guidance was given to paediatricians and paediatric sub-specialists to help them determine whether or not outpatient echocardiograms should be considered elective procedures. Prior studies suggest about 11–14% of transthoracic echocardiograms ordered by paediatric cardiologists and up to 19% of transthoracic echocardiograms ordered by primary care providers were performed for rarely appropriate indications. Reference Anderson, McCracken and Sachdeva9–Reference Kourtidou, Evers, Jorgensen, Kronmal, Lewin and Schultz11 Following publication of the initial paediatric appropriate use criteria guideline and an educational intervention, rarely appropriate indications accounted for ∼ 4–6% of total studies performed in paediatric cardiology clinics. Reference Sachdeva, Douglas and Kelleman12
The echocardiogram lab at Norton Children’s Hospital (Louisville, KY) is an open-access echocardiography lab where both cardiologists and non-cardiologists may request transthoracic echocardiograms. Additionally, paediatric cardiologists from Norton Children’s Hospital interpret transthoracic echocardiograms ordered by predominantly non-cardiologists from 27 regional sites throughout Kentucky. The appropriateness of outpatient transthoracic echocardiograms interpreted by paediatric cardiologists at Norton Children’s Hospital as classified using the appropriate use criteria guidelines for initial outpatient transthoracic echocardiogram and follow-up of known CHD have not previously been described. We sought to determine the appropriate use criteria classification and diagnostic yield of all outpatient transthoracic echocardiograms interpreted at Norton Children’s Hospital from Kentucky facilities both prior to and during the time period when elective medical procedures were prohibited in Kentucky due to the COVID-19 pandemic. We furthermore sought to compare the proportion of studies ordered for rarely appropriate indications between these two time periods and factors which might be associated with any observed differences. We are not aware of other studies describing the impact of the COVID-19 elective procedure restrictions on the appropriateness of paediatric outpatient transthoracic echocardiograms performed.
Materials and methods
Study design
The time period during which an executive order in Kentucky limited elective medical procedures was 3/19/2020-4/27/2020. A baseline time period for comparison from 3/21/2019-4/28/2019 was selected one year prior to initiation of the elective procedure restrictions (and using the same weekdays) to limit confounding from any seasonal variability in echocardiogram volume or indications as well as potential unrecognised early impact from COVID-19. All outpatient transthoracic echocardiograms interpreted at Norton Children’s Hospital on patients < 18 yrs of age by paediatric cardiologists performed at Kentucky facilities during these time periods were reviewed. The study excluded outpatient transthoracic echocardiograms interpreted at Norton Children’s Hospital on patients > 18 yrs of age, not interpreted by paediatric cardiologists, performed at facilities outside Kentucky, for follow-up of non-congenital pathology (i.e. chemotherapy screening) or performed outside of these time periods.
A retrospective chart review was completed to collect deidentified data for each subject having a transthoracic echocardiogram meeting eligibility criteria which was stored in a secure spreadsheet. Demographics collected included timing of exam (pre-COVID or during COVID restrictions), performing location, type of exam (initial or congenital follow-up), and ordering provider type (i.e. paediatrician, paediatric cardiologist, or paediatric sub-specialist). Transthoracic echocardiogram indications were determined using any available clinical information in the echocardiogram order, echocardiogram report, or electronic medical record (Epic, HeartLab Encompass, and McKesson Cardiology CPACS). Diagnostic yield was determined using final diagnosis from the echocardiogram report.
For initial outpatient paediatric transthoracic echocardiograms, indications were classified as appropriate, may be appropriate, or rarely appropriate using the 2014 appropriate use criteria. Reference Campbell, Douglas, Eidem and etal4 For congenital follow-up transthoracic echocardiograms, the 2020 appropriate use criteria Reference Sachdeva, Valente, Armstrong and etal5 were used to determine if the indication was appropriate, may be appropriate, rarely appropriate, or unclassifiable, including evaluation of length of time from prior exam. Unclassifiable studies were categorised into two sub-categories: unclassifiable indications and insufficient information for classification. Unclassifiable was defined as an indication not included in the appropriate use criteria guideline, and insufficient information was defined as a study with insufficient information to determine appropriate use criteria classification. Study indications were initially classified by paediatric residents (M.S.) or (C.M.) with indication classification reviewed and reclassified, when appropriate, by a board-certified paediatric cardiologist (A.N.). Diagnostic yield was determined as previously described, classifying findings as normal, incidental, or abnormal and further classifying abnormal findings into minor, moderate, and severe findings. Reference Sachdeva, Allen and Benavidez10 For findings of unclear severity, classification was determined by a board-certified paediatric cardiologist (A.N.) Determination of diagnostic yield included transthoracic echocardiograms with indications which were unclassifiable or for which insufficient information was available to classify indication.
Statistical analysis
Descriptive summaries were used to understand the sample characteristics overall and for each period under investigation. A bivariate analysis was conducted comparing the proportions of appropriate, may be appropriate, and rarely appropriate outpatient paediatric transthoracic echocardiograms between the pre-COVID time period and during COVID restrictions. Similarly, the diagnostic yield proportions were compared between the two time periods, categorised as normal, abnormal (with subtypes minor, moderate, and severe), and incidental findings. A logistic regression was used to assess whether a rarely appropriate indication was associated with an abnormal transthoracic echocardiogram finding, while controlling for the timing of the ECHO (pre-COVID vs during COVID restrictions). Odds ratios and 95% confidence intervals were calculated, and p-values < 0.05 were considered statistically significant.
Finally, comparisons of appropriateness proportions and diagnostic yield between the two time periods were stratified by provider type, categorised as (1) paediatric cardiologist, (2) paediatrician, or (3) paediatric sub-specialist. Differences between the time periods were tested using chi-square tests, and two-tailed p-values less than 0.05 were considered statistically significant. Statistical analysis performed using R: A language and environment for statistical computing. 13 This study was deemed exempt by the University of Louisville and Norton Children’s Hospital IRB.
Results
A total of 767 transthoracic echocardiograms were reviewed, and 486 met inclusion criteria: 364 pre-COVID versus 122 during COVID. Transthoracic echocardiogram indication was classifiable in 354 (72.8%) of studies: 135 (38.1%) were initial outpatient transthoracic echocardiogram and 219 (61.9%) were congenital follow-up transthoracic echocardiogram. Regarding timing of study, 265 (74.9%) were performed pre-COVID and 89 (25.1%) during COVID restrictions (Fig 1). Of the classifiable studies, 221 (62.4%) were appropriate (A), 15 (4.2%) may be appropriate (M), 118 (33.3%) were rarely appropriate (R). Stratified by study type, of initial outpatient transthoracic echocardiograms, 72/135 (53.5%) were appropriate, 12/135 (8.9%) may be appropriate, and 51/135 (37.8%) were rarely appropriate. In contrast, appropriate studies were more frequent among congenital follow-up studies (149/219, 68%) versus may be appropriate (3/219, 1.4%) or rarely appropriate (67/219, 30.6%) with p = 0.001. The unclassifiable studies were categorised as unclassifiable indications (U) and insufficient (I). For initial outpatient transthoracic echocardiograms, 7 met unclassifiable criteria, and 96 had insufficient information indications. Of the congenital follow-up transthoracic echocardiograms, 27 were unclassifiable, and 2 were insufficient information.
Distribution of outpatient transthoracic echocardiograms (TTEs) meeting inclusion criteria in pre-COVID and during COVID time periods and whether TTE indication was classifiable using 2014 or 2020 appropriate use criteria (AUC) as applicable.
Rarely appropriate transthoracic echocardiograms: pre-COVID restrictions and during COVID
There was a significant difference in the proportion of rarely appropriate transthoracic echocardiograms pre-COVID versus during COVID (p = 0.002). The proportion of may be appropriate (M) transthoracic echocardiograms was relatively low during both time periods (4.9% pre-COVID versus 2.2% during COVID; Table 1).
Appropriate use criteria (AUC) classification of all outpatient paediatric transthoracic echocardiograms performed pre-COVID and during COVID elective procedure restrictions at Norton Children’s Hospital
p = 0.002.
Diagnostic yield of transthoracic echocardiograms
Of initial outpatient transthoracic echocardiograms, the majority were normal: 135/193 (69.9%) pre-COVID versus 29/45 (64.4%) during COVID. There was no significant difference in diagnostic yield of initial outpatient transthoracic echocardiograms with 25/193 (13.0%) abnormal studies pre-COVID versus 7/45 (15.5%) during COVID. There were 33/193 (17.1%) incidental findings on pre-COVID transthoracic echocardiograms and 9/45 (20.0%) of transthoracic echocardiograms done during COVID with isolated patent foramen ovale being noted most frequently. The majority of abnormal findings were minor in both the pre-COVID, 20/193 (10.4%), and during COVID time period, 6/45 (13.3%). Common minor findings consisted of mild valve abnormalities and small atrial septal defects. Moderate abnormalities identified included larger atrial septal defects expected to require intervention and more significant valve abnormalities: 5/193 (2.6%) pre-COVID and 1/45 (2.2%) during COVID (Table 2). There were no severe abnormalities identified during either study time period. Overall, both the types and frequency of congenital heart lesions identified before and during COVID elective procedure restrictions were similar (Fig 2). Logistic regression results imply that rarely appropriate indication (OR 0.52, CI 0.20–1.20, p = 0.147) and timing of study during COVID restrictions (OR 0.91, CI 0.37–2.05, p = 0.827) were both associated with lower odds of abnormal transthoracic echocardiogram findings, although these results were not statistically significant.
Diagnostic yield of outpatient transthoracic echocardiograms pre-COVID versus during COVID.
Classification of initial outpatient paediatric transthoracic echocardiograms (TTEs) performed pre-COVID and during COVID elective procedure restrictions at Norton Children’s Hospital
Atrial septal defect (ASD); Bicuspid aortic valve (BAV); Left ventricle (LV); Mitral valve prolapse (MVP); Patent ductus arteriosus (PDA); Patent foramen ovale (PFO); Persistent pulmonary stenosis (PPS); Pulmonary hypertension (pHTN); Pulmonary stenosis (PS); Right ventricle (RV); Ventricular septal defect (VSD).
* This patient with tiny PDA met abnormal minor criteria because they were outside of the neonatal period at time of echocardiogram finding.
Provider type impact on appropriateness of transthoracic echocardiogram indication and diagnostic yield
The data were stratified by provider type (paediatric cardiologist, paediatrician, and non-cardiology paediatric sub-specialist). Cardiologists ordered the majority of outpatient transthoracic echocardiograms before (195/265, 73.5%) and during COVID (77/89, 86.5%), limiting the ability to compare transthoracic echocardiogram indications by provider type.
Discussion
To our knowledge, this is the first study to evaluate the impact of COVID elective procedure restrictions on physician ordering of outpatient paediatric transthoracic echocardiograms. We found that following the Kentucky executive order to cease all elective medical procedures in response to the COVID-19 pandemic, outpatient paediatric transthoracic echocardiograms ordered for rarely appropriate indications at a tertiary Kentucky children’s hospital decreased significantly. Importantly, despite a reduction in overall number of transthoracic echocardiograms ordered and, specifically, transthoracic echocardiograms ordered for rarely appropriate indications, diagnostic yield between the two time periods was similar. There was a non-significant trend toward more appropriate utilisation of transthoracic echocardiogram by paediatric cardiologists and the majority of transthoracic echocardiograms were ordered by cardiologists during COVID elective procedure restrictions.
Our findings of an overall reduction in volume of paediatric outpatient transthoracic echocardiograms and, disproportionately, rarely appropriate studies suggest paediatric providers may have regarded these tests as elective medical procedures. To our knowledge, no additional guidance or educational intervention was provided to Kentucky paediatricians during the COVID-19 pandemic regarding appropriate use criteria guidelines. However, as the majority of transthoracic echocardiograms during COVID were ordered by cardiologists, these providers may have reviewed the American Society of Echocardiography or appropriate use criteria guidelines, affecting practice behaviour. Reference Kirkpatrick, Mitchell, Taub, Kort, Hung and Swaminathan7–Reference Barker, Lewin and Donofrio8 These findings may also be suggestive of changes in healthcare-seeking behaviours during the COVID-19 pandemic, as fewer patients may have presented for non-urgent and non-emergent medical complaints during this period. As a result, this may have led to a higher proportion of patients with pathologic findings and comorbidities seeking care and prompting appropriate transthoracic echocardiogram orders. Conversely, fewer well children may have sought routine care or sports physicals prompting rarely appropriate diagnostic testing including transthoracic echocardiograms. Supporting this possibility, national data demonstrate stay-at-home orders during COVID significantly reduced routine childhood and adolescent vaccination rates in March–May of 2020. Reference Patel Murthy, Zell and Kirtland14
Cardiologists ordered the majority of echocardiograms both pre-COVID and during COVID. During COVID, the proportion of cardiologist-ordered echocardiograms increased slightly, which may have contributed to the reduction in rarely appropriate studies. Additionally, 68.2% of studies during COVID were follow-up studies versus 59.6% pre-COVID, which may have also resulted in more appropriate utilisation of echocardiograms. However, given that cardiologists ordered the overwhelming majority of outpatient transthoracic echocardiograms in both time periods, we hypothesise there was an intentional change in ordering behaviour between time periods.
The overall diagnostic yield of transthoracic echocardiogram in our study was similar to previously reported findings and not significantly different between time periods: 13% pre-COVID versus 15.5% during COVID Reference Sachdeva, Allen and Benavidez10 While rarely appropriate studies tended to be normal, appropriateness was not predictive of diagnostic yield in our study. However, this observation should be interpreted with caution in the context of a relatively low diagnostic yield and small sample size. Recent studies suggest that the full implications of stay-at-home orders and delayed paediatric medical care may not yet be fully realised. However, beyond postponed vaccinations, paediatricians have already noted lapses in developmental therapies, missed cancer treatments, missed medications, and even delays in diagnoses such as appendicitis and diabetes. Reference Saab, Obeid and Gachi15 Reference Madhusoodhan, Pierro and Musante,16 Thus, the similar diagnostic yield between time periods in this study was critically important. This finding suggests that clinically significant echocardiogram findings were still detected despite more prudent utilisation of echocardiography. This finding also suggests that the appropriate use criteria guidelines provide valuable guidance for paediatric providers in deciding when to order an initial outpatient transthoracic echocardiogram. A study evaluating educational interventions on appropriate use criteria showed a modest decrease in rarely appropriate indications and an unremarkable difference in diagnostic yield. Reference Sachdeva, Douglas and Kelleman12 These results are similar to the findings of our study suggesting that clinicians respond positively to interventions designed to improve usage of appropriate use criteria. Clinically significant cardiovascular diagnoses were detected at the same rate when adherence to these guidelines improved. Further research is needed to determine if treatments were delayed or affected by the COVID-19 pandemic in paediatric cardiology.
When our data are compared to prior studies, it is notable that our study includes a higher percentage of transthoracic echocardiograms ordered for rarely appropriate indications. Prior studies show rarely appropriate indications ranging from 4–19%, and the range in our study was 20.2–37.7%. However, the sample sizes of other studies are larger, data were collected over longer time periods, some had educational interventions associated, and these studies did not include a time period during a pandemic, which likely explains the discrepancy between ordering behaviours noted in our study and previous literature. Reference Anderson, McCracken and Sachdeva9,Reference Kourtidou, Evers, Jorgensen, Kronmal, Lewin and Schultz11,Reference Sachdeva, Douglas and Kelleman12
Currently, we are unaware of other studies evaluating the impact of COVID elective procedure restrictions on physician ordering of outpatient transthoracic echocardiograms. There were several limitations to this study. The sample size was relatively small due to the finite number of transthoracic echocardiograms available to review during the respective six-week study time periods. The sample size of the transthoracic echocardiograms reviewed during COVID was smaller compared to the pre-COVID time period. Additionally, this was a single-centre retrospective study. Because transthoracic echocardiograms are interpreted from several outlying facilities, the clinical information available for classifying study indication was variable and often limited, resulting in more unclassifiable indications from outlying facilities (∼25%.) The echocardiograms that were excluded because of insufficient information to classify indication may have skewed results by excluding predominantly paediatrician-ordered studies from regional sites.
During COVID elective procedure restrictions, outpatient clinic visits were initially reduced to urgent consultations and follow-ups. This likely further impacted diagnostic testing, including physician ordering behaviour related to transthoracic echocardiograms. It is unclear what resources physicians may have sought to make appropriate choices regarding appropriate utilisation. While the absence of severe findings on outpatient transthoracic echocardiograms during COVID and stable diagnostic yield between time periods are reassuring, an interesting question raised by this study is whether urgent and emergent hospital admissions for paediatric cardiac indications changed during the elective procedure restrictions. Further research in this area is needed.
Conclusion
While elective procedures were prohibited during the COVID-19 pandemic in Kentucky, rarely appropriate outpatient paediatric transthoracic echocardiograms decreased significantly. There was no change in diagnostic yield of initial outpatient paediatric transthoracic echocardiograms during elective procedure restrictions, suggesting that clinically significant echocardiogram findings were still detected despite more prudent utilisation of echocardiography. Our findings support adherence to existing appropriate use criteria guidelines in determining which children are most likely to benefit from evaluation by outpatient transthoracic echocardiogram. Future studies are needed to continue to determine what tests, treatments, and services may have been adversely impacted by the COVID-19 pandemic for children and what long-term implications may exist.
Acknowledgements
We are grateful to Mona Amer at the University of Louisville School of Medicine for assistance with data collection.
Financial support
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Conflicts of interest
None
