Paediatric transthoracic echocardiography volume has increased as paediatric echocardiography has become more available and image acquisition standardised.Reference Srivastava, Allada and Younoszai 1 , Reference Lopez, Colan and Frommelt 2 The improved availability, perceived negligible risk, and practicality that make echocardiography a popular diagnostic tool also promote misuse and represent an opportunity for reducing unnecessary and expensive diagnostic studies.Reference Matulevicius, Rohatgi, Das, Price, DeLuna and Reimold 3 – Reference Campbell, Douglas and Eidem 5 There was no official evidence-based practice guideline for the ordering of paediatric transthoracic echocardiograms until the recent publication of the Appropriate Use Criteria for initial outpatient paediatric transthoracic echocardiography.Reference Sachdeva, Allen and Benavidez 4 , Reference Campbell, Douglas and Eidem 5 This research assesses the impact of applying Appropriate Use Criteria to the ordering of echocardiograms in an outpatient practice.
Appropriate Use Criteria for adult transthoracic echocardiography first appeared 10 years ago and have improved test ordering behaviour.Reference Matulevicius, Rohatgi, Das, Price, DeLuna and Reimold 3 , Reference Douglas, Garcia and Haines 6 , Reference Patil, Coggins, Kusnetzky and Main 7 Validation of the adult criteria has provided critical insight into how Appropriate Use Criteria affect ordering behaviour, influence detection of clinically relevant findings, change clinical care, and reduce cost.Reference Matulevicius, Rohatgi, Das, Price, DeLuna and Reimold 3 , Reference Douglas, Garcia and Haines 6 – Reference Boggan, Schulteis, Donahue and Simel 9 The recently published paediatric Appropriate Use Criteria are undergoing the first steps of validation as implementation projects and educational interventions have already occurred.Reference Safa, Aggarwal, Misra and Kobayashi 10 – Reference Lang, Bolin, Hardy, Tang and Collins 14 As paediatric providers increasingly look to the Appropriate Use Criteria to guide their ordering behaviour, our validation study evaluates how adherence to the criteria could be used to reduce inappropriate transthoracic echocardiography volume, detect clinically relevant heart disease, and alter clinical practice.
Materials and methods
We performed an Institutional Review Board-approved retrospective chart review of the institutional electronic medical record to identify all patients less than or equal to 18 years having an initial transthoracic echocardiogram during the calendar year 2014. We chose this period to allow 2 years of follow-up from the time of the initial transthoracic echocardiogram to ascertain whether clinically significant cardiac disease resulting in treatment was diagnosed or missed. Extracted data included date of birth, age at time of study, sex, study indication as specified by the provider in the electronic order and office visit note, outpatient study location, and date of study. International classification of disease coding was not used to determine study indication. Patients were not included in the study if another institution had already performed a transthoracic echocardiogram.
We used ordering provider identity to classify providers as either general paediatrician or specialist and, if specialist, whether trained in paediatric cardiology. Using the Appropriate Use Criteria document, we matched each echocardiographic indication to the corresponding Appropriate Use Criteria rating. When multiple indications were present for the same study, we used the indication with the highest Appropriate Use Criteria rating. We added the physical finding of systolic click as an appropriate indication as the authors of the original Appropriate Use Criteria document have subsequently suggested this modification.Reference Sachdeva, Allen and Benavidez 4
We then reviewed the echocardiographic report findings and classified them as normal or abnormal based on a previously published objective grading system.Reference Sachdeva, Allen and Benavidez 4 Normal results required no intervention, cardiology follow-up, or change in management. Normal results included incidental findings such as patent foramen ovale and aortic arch branching variants not associated with vascular ring. Abnormal transthoracic echocardiographic findings were further categorised as abnormal motivating treatment if they were followed by interventional cardiac catheterisation, cardiac surgery, ablation, device implantation, or treatment with a cardiac medication for ventricular dysfunction or pulmonary over-circulation within 2 years after the initial echocardiogram.
The primary outcome assessed was the proportion of abnormal and abnormal motivating treatment studies with regard to Appropriate Use Criteria rating and ordering provider training. Additional outcomes were the diagnostic yield based on ordering provider type and Appropriate Use Criteria rating consistent with prior literature.Reference Matulevicius, Rohatgi, Das, Price, DeLuna and Reimold 3 , Reference Safa, Aggarwal, Misra and Kobayashi 10 – Reference Lang, Bolin, Hardy, Tang and Collins 14 Results were stratified based on patient age: less than 3 months, from 3 months to 11 months, 1–4 years, 5–9 years, and greater than 9 years.
Descriptive statistics were calculated for all variables of interest and included count, median, standard deviations, and interquartile range when appropriate. We calculated the odds ratios and their 95% confidence intervals to compare the proportion of abnormal transthoracic echocardiograms at varying appropriateness level and used the χ2 test to compare proportions when analysing categorical values; considered statistically significant when p value ⩽0.05.
Results
There were 1383 patients with a median age of 11 years and interquartile range of 7–17 years, of whom 44.6% were female. Test results were normal in 1161 patients (83.9%), abnormal in 222 patients (16.1%), and abnormal motivating treatment in 27 patients (1.9%) (Fig 1). Appropriateness rating was appropriate in 927 studies (67.0%), may be appropriate in 229 (16.6%), and rarely appropriate in 227 exams (16.4%). We could not assign an Appropriate Use Criteria indication to only 24 patients owing to insufficient clinical information and did not include them. All these studies were ordered by general paediatricians and had normal results.
Proportion of both abnormal and normal echocardiograms in each appropriateness category out of 1383 total studies.
The most common Appropriate Use Criteria categories were systemic disorders, murmur, chest pain, and family history (Table 1). Overall, the most common indications were suspected connective tissue disorder (n=100), pathologic murmur (n=97), exertional chest pain (n=93), innocent murmur (n=86), and systemic hypertension (n=75). For studies deemed appropriate, suspected connective tissue disease (n=100), pathologic murmur (n=97), and exertional chest pain (n=93) were the most common indications. For may be appropriate indications, chest wall deformities and scoliosis preoperatively (n=57), chromosomal abnormality with undefined risk of cardiovascular disease (n=37), and family history of congenital left-sided lesion (n=27) were most common. Among rarely appropriate indications, innocent murmur (n=86), palpitations with benign history, electrocardiogram, and no other signs or symptoms (n=24), and syncope with benign family history, normal electrocardiogram, and no other signs or symptoms (n=20) were most common. We found that 38 of the Appropriate Use Criteria were not used once in our population.
Echocardiographic results in each Appropriate Use Criteria category.
Upon application of the Appropriate Use Criteria to 1 year of outpatient initial transthoracic echocardiograms, we found that a large majority of echocardiographic results were normal (83.9%) and that a large majority of examinations with abnormal echocardiographic results were in studies classified as appropriate (81.1%). The odds ratio for a transthoracic echocardiogram with an appropriate indication to have an abnormal result compared with less appropriate indications was 2.73 (95% confidence interval 1.92–3.89, p<.001). Among the echocardiograms with appropriate indications, 80.6% had normal findings (Fig 1). The proportion of abnormal echocardiographic results was not statistically different between rarely appropriate or may be appropriate indications (p=0.971); however, the proportion of abnormal studies was lower for may be appropriate and rarely appropriate indications as compared with appropriate indications (p<0.001 for both). All 27 abnormal motivating treatment results were in appropriate studies. In this group, 11 patients had surgery, eight had interventional catheterisations, five were started on afterload reducing agents for ventricular dysfunction, and three patients died from non-cardiac causes before anticipated interventions. There were an additional six patients under the age of 2 years with at least moderate atrial septal defects seen on their initial study. All had echocardiograms with appropriate indications, but none had an intervention as it is routine practice to delay device or surgical closure of such defects to an age that would be missed by our follow-up period.
The most common abnormal echocardiographic findings were left ventricular inflow/outflow anomalies, left ventricular functional or structural anomalies, and aortic dilation (Table 2). When further stratified into abnormal motiving treatment findings, a different pattern emerged, with patent ductus arteriosus, left ventricular functional or structural anomalies, and coronary artery anomalies being the most common. Only 42 abnormal echocardiograms, 18.9% of our abnormal results, and none of our abnormal motivating treatment results were in indications less than appropriate (Table 3). After 2 years of follow-up within this group, 29 had unchanged findings on repeat transthoracic echocardiography and the remainder were discharged from cardiac care. The most common findings in this group of 42 patients were mild aortic root dilation (n=10), bicuspid aortic valve without stenosis or regurgitation (n=6), mild pulmonary stenosis (n=6), mild mitral valve prolapse (n=6), tiny patent ductus arteriosus without left-sided chamber dilation (n=3), and moderate secundum atrial septal defect (n=3). In eight of the 29 patients, the echocardiographic indication was a chest wall deformity or scoliosis preoperatively. Their echocardiogram showed mild aortic dilation, mitral valve prolapse, or both in all eight. Furthermore, there were only 21 abnormal echocardiograms, 9.5% of our abnormal results, and no abnormal motivating treatment results in studies with rarely appropriate indications. Ten of these patients have been subsequently discharged from cardiac care, and the remainder have findings relatively common in the general population such as bicommissural aortic valve.
Abnormal echocardiographic findings.
Echocardiographic result and appropriateness rating according to patient age.
Most patients were greater than 9 years of age (51.6%) with a similar age distribution seen in normal studies (Table 3). In abnormal studies, there was a higher proportion of patients less than 3 months of age. Because of these divergent trends, the odds ratio of having an abnormal transthoracic echocardiogram result in a patient younger than 1 year relative to over 1 year was 2.08 (p<0.001, confidence interval 1.56–2.77). Studies deemed appropriate or may be appropriate closely followed the age distribution in our overall study population (Table 3). Rarely appropriate studies were more common in patients less than 3 months of age (p<0.001) and less common in patients greater than 10 years (p=0.026).
We also stratified our data based on provider type (Fig 2). There were no differences in patient age or sex between cardiologists and non-cardiologists. Cardiologists ordered proportionally less may be appropriate studies (p<0.001) but more rarely appropriate studies (p<0.001) than their non-cardiologist counterparts. In terms of echocardiographic results, there was a significantly higher proportion of abnormal studies resulting in treatment that were ordered by cardiologists (p=0.006), but the proportion of abnormal studies was nearly identical. Similar trends persist upon further stratification of the non-cardiologist group into general paediatrician and paediatric specialist. Paediatric specialists ordered significantly more may be appropriate studies (p=0.013) and less rarely appropriate studies (p=0.013) compared with their general paediatrician colleagues.
Appropriate Use Criteria rating and echocardiographic result according to ordering provider type.
Discussion
On modelling application of Appropriate Use Criteria to a population of outpatients undergoing initial transthoracic echocardiography, we found that 83.9% of echocardiograms were normal and that 81.1% of abnormal findings were in studies classified as appropriate. Along with the odds ratio of 2.73, this pattern suggests potential for a targeted reduction in imaging volume of as much as 33% by using the Appropriate Use Criteria document. To better understand this potential, we further defined which categories were of high yield, as well as what abnormal findings were seen in less appropriate indications.
The highest-yield Appropriate Use Criteria categories for abnormal echocardiographic results were syncope, murmur, systemic disorders, and outpatient neonates (Table 1). Our overall yield of abnormal echocardiograms was very similar to that seen in several recent studies.Reference Safa, Aggarwal, Misra and Kobayashi 10 , Reference Chamberlain, Pelletier and Blanchard 11 , Reference Lang, Bolin, Hardy, Tang and Collins 14 The indications of exertional syncope, exertional chest pain, and pathologic murmur alone accounted for 55.6% of our abnormal results motivating treatment. When combined with the indication of muscular dystrophy, which accounted for another 18.5% of abnormal motivating treatment findings, these four criteria comprised nearly three-quarters of our patients with an echocardiographic finding that resulted in treatment in our 2-year follow-up window.
For indications related to chest pain and syncope, the presence of exertional symptoms accounted for nearly all patients who had treatment because of an abnormal finding on echocardiography. This pattern and diagnostic yield for abnormal results motiving treatment correlates with that seen in recent literature discussing echocardiograms for syncope and chest pain.Reference Chamberlain, Pelletier and Blanchard 11 , Reference Lang, Bolin, Daily, Tang and Thomas Collins 15 – Reference Nguyen, Fundora and Welch 17 Our yield for the indication of murmur was also similar to other studies; however, our yield for any abnormal result for the indications of chest pain and syncope was higher.Reference Chamberlain, Pelletier and Blanchard 11 – Reference Phelps, Kelleman and McCracken 13 Because the assigned severity of an abnormal echocardiographic result in our study was related to whether an intervention occurred within 2 years, our reported yields are not entirely comparable to other studies that used different or no follow-up periods.Reference Chamberlain, Pelletier and Blanchard 11 , Reference Lang, Bolin, Hardy, Tang and Collins 14 , Reference Johnson, Etheridge, Minich, Bardsley, Heywood and Menon 16 – Reference Angoff, Kane and Giddins 18 Nevertheless, our yield for indications related to chest pain or syncope is similar to other reported yields when considering that only those findings thought to be causing the presenting symptoms resulted in an intervention in our study (Table 1). The 4% yield for syncope and 1.1% for chest pain that resulted in treatment were related to findings such as anomalous origin of the right coronary artery and severe non-compaction. The remainder of the abnormal findings found on studies with indications related to chest pain or syncope were incidental in nature and not related to symptoms.
Results suggest minimal risk of missing clinically significant disease with application of the Appropriate Use Criteria. Only 42 patients had an abnormal transthoracic echocardiogram with an indication less than appropriate, and 13 were discharged from ongoing cardiology care after a subsequent normal transthoracic echocardiogram. Further, only 21 patients with a rarely appropriate indication had an abnormal result, with 10 being discharged on subsequent follow-up. In none of these patients did treatment follow diagnosis, suggesting that the findings were clinically less significant. Moreover, none of the echocardiographic findings in these patients progressed in severity over 2 years. In addition, the most common less than appropriate indication in patients with abnormal echocardiographic results still requiring cardiology care was a chest wall deformity or scoliosis preoperatively. This highlights the importance of considering connective tissue disease evaluation in patients with pectus excavatum and scoliosis, but does not necessarily suggest that these indications should be ranked as appropriate instead of may be appropriate.
All 27 patients with abnormal results motivating treatment had appropriate indications, with 22 having indications with the highest rating of nine. Therefore, with strict application of the criteria, no abnormal findings resulting in treatment over a reasonable 2-year follow-up period would be missed by only performing echocardiograms on patients with appropriate indications. In this scenario, transthoracic echocardiography volume would be reduced by 33.0% or 456 studies. Using the payment for a congenital transthoracic echocardiogram of $594 from the 2014 Medicare hospital ambulatory payment classification system, such a reduction would decrease payments by $270,864 over the study period.Reference Rosenbloom 19 This conservative total does not include professional physician reimbursement. Even with a suggested reduction of echocardiography volume by one-third, the remaining patients with appropriate transthoracic echocardiograms would have 80.6% normal results. Our results suggest that within this group a priority should be placed on those patients with exertional symptoms, pathologic findings on physical exam, significant cardiac family history, and systemic disorders with a high risk of cardiac involvement.
About 90% of our transthoracic echocardiography volume was in patients aged over 1 year old; however, those patients under 1 year of age were statistically more likely to have an abnormal result. This may be explained by symptomatic complaints driving echocardiography volume in older patients, rather than concerning physical exam findings as in neonates and infants.Reference Yi, Kimball, Tsevat, Mrus and Kotagal 20 – Reference Khairandish, Jamali and Haghbin 23 Other than symptoms associated with exertion, symptom-driven echocardiograms within all age groups in our population had low diagnostic yield. With regard to findings on physical exam, on the other hand, they were of high yield in all age groups. Taken together, our results show the importance of pathologic findings on physical exam at any age and that most of the reduction in unnecessary transthoracic echocardiography volume would occur in patients greater than one year.
As in other studies, cardiologists in our study were more likely to be the ordering provider for those patients with an echocardiographic finding resulting in treatment.Reference Rose-Felker, Kelleman, Campbell and Sachdeva 24 Cardiologists in our study though were also more likely to order echocardiograms for rarely appropriate indications. This trend may be related to referral pressure for obtaining a transthoracic echocardiogram or simply the relative ease of obtaining an echocardiogram as a cardiologist.Reference Khairandish, Jamali and Haghbin 25 – Reference Sachdeva, Douglas, Kelleman and McCracken 27 Paediatric subspecialists other than cardiologists also tend to order transthoracic echocardiograms for diagnoses with a known risk of cardiac involvement, all indications rated as appropriate by the Appropriate Use Criteria. This may explain why ordering behaviour was so similar. As nearly 70% of our study population had transthoracic echocardiograms ordered by a cardiologist with many of the remaining studies ordered by a paediatric subspecialist other than a cardiologist, we are limited in our ability to make strong conclusions about transthoracic echocardiograms ordered by general paediatricians. Our results support the great need for continuing educational interventions both for cardiologists and general paediatricians.Reference Sachdeva, Kelleman and McCracken 28 In addition, our study supports the conclusion that when a non-cardiologist orders a transthoracic echocardiogram for an appropriate indication that concurrent referral to a cardiologist is reasonable owing to the higher likelihood of having an abnormal result so that care may be streamlined.
Our study is limited to a single centre but probably reflects the experience of other paediatric cardiac centres with the caveat that our institution attracts patients with connective tissue disease. Nevertheless, transthoracic echocardiograms performed for any connective tissue-related indication comprised less than 10% of our study volume. Our study period also overlaps by 1 month with the publication of the Appropriate Use Criteria; however, a change in ordering provider practice within that brief time was unlikely. Because not every outpatient has a transthoracic echocardiogram performed, we also do not know the true number of children with cardiac pathology who would have an abnormal echocardiogram. As such, any retrospective study such as ours is unable to determine the sensitivity and specificity of the Appropriate Use Criteria or rates of abnormal findings within the general population. Our study is also unable to assess the benefit of a normal echocardiographic result regardless of appropriateness.
This study shows that the Appropriate Use Criteria, with only a single modification for click heard on exam, has practical application to a typical outpatient population with an open-access format paediatric echocardiography lab. This study further shows that by following the rubric created by the Appropriate Use Criteria, unnecessary transthoracic echocardiographic volume can indeed be substantially reduced without affecting the detection of clinically relevant cardiac disease. Further refinements of the Appropriate Use Criteria to potentially remove low-yielding criteria and increase the scoring of higher-yielding criteria will be necessary in the future. Importantly, our findings highlight the critical nature of a thorough cardiovascular exam and history when evaluating patients with potential cardiac involvement. In an era of downward pressure on both medical expenditure and reimbursement with paradoxical increases in imaging volume and complexity, clinicians must rely not only on the Appropriate Use Criteria but also on the foundations of clinical medicine to best serve their patients and responsibly use constrained medical resources.
Acknowledgements
None.
Financial Support
This research received no specific grant from any funding agency or from commercial or not-for-profit sectors.
Conflicts of Interest
None.
