Evaluation of heart murmurs in children is a common clinical problem for paediatricians and family physicians and the most common reason for referral to paediatric cardiologists.Reference Wong, Barker and Warren 1 A heart murmur in a child may be the only clue to the presence of congenital heart disease (CHD), therefore, accurate and timely evaluation of heart murmurs is important. No screening tools exist to assist primary-care physicians in deciding which children with a murmur should be referred to a cardiologist. Thus, clinicians rely on experience and clinical judgement to make this decision, despite evidence for weak auscultation skills among family physicians and paediatricians.Reference Haney, Ipp, Feldman and McCrindle 2 – Reference Germanakis, Petridou, Varlamis, Matsoukis, Papadopoulou-Legbelou and Kalmanti 4
Echocardiography is time consuming and expensiveReference Yi, Kimball, Tsevat, Mrus and Kotagal 5 , Reference Danford, Nasir and Gumbiner 6 and not practical as a screening tool for all asymptomatic children with murmurs. Recently, computer-assisted auscultation has become available to assist clinicians with physical examination of the heart. This technology provides a spectral display – a visual image – of a murmur to complement the auditory information, and allows murmur playback at slower speeds, making the art of auscultation less challenging. In addition, this technology may allow cardiologists to evaluate murmurs transmitted electronically from remote locations, obviating the need for some patients to travel to referral centres; however, to date, this technology has not been rigorously evaluated. Therefore, the objectives of this study were (1) to determine the sensitivity and specificity of commercially – available software at identifying structural cardiac abnormalities in children, and (2) to compare cardiologists’ clinical assessment skills against this software.
Materials and methods
Study design and population
This study was conducted in the Division of Cardiology at the Stollery Children’s Hospital, a tertiary-care referral centre, with 14 fully trained paediatric cardiologists. We consecutively enrolled children aged between 0 and 17 years who had a murmur – either innocent or pathological – as confirmed by an attending cardiologist. Children being evaluated for the first time with an echocardiogram and follow-up patients with known CHD were both eligible. We excluded children with previously operated CHD and children who were unable to sit still for CardioscanTM auscultation (approximately a 90 second procedure).
Audio recordings
CardioscanTM auscultation was performed in a quiet room without extraneous noise with the study subject in the supine position. The patient’s pulse rate was measured manually by a study investigator immediately before CardioscanTM auscultation. A Littmann 4100 stethoscope was then connected to a laptop computer and a trained student (C.L., K.J.Z.) placed the stethoscope in each of the four auscultatory areas – 2nd right intercostal space, 2nd left intercostal space, 4th left intercostal space, and apex – for 20 seconds in each location. CardioscanTM software provided immediate feedback if an insufficient quality recording was obtained in one or more listening areas, in which case the procedure was repeated until a satisfactory recording was obtained. CardioscanTM software then interpreted the presence and timing of a murmur in each auscultatory location. For systolic murmurs, the software determined whether or not the murmur was “potentially pathologic” – that is, warranting referral to a cardiologist.
Echocardiography and electrocardiography
Echocardiography served as the reference standard. Complete two-dimensional transthoracic echocardiograms with colour and spectral Doppler were performed in a dedicated paediatric echocardiography laboratory by a trained sonographer and interpreted by a paediatric cardiologist, in keeping with standards of the American Society of Echocardiography.Reference Lai, Geva and Shirali 7 Sonographers and cardiologists were unaware of the findings of CardioscanTM auscultation. For the purpose of this study, an “abnormal echocardiogram” was defined as one demonstrating “a gross structural abnormality of the heart or intrathoracic great vessels that is actually or potentially of functional significance”, as defined by Mitchell et al.Reference Mitchell, Korones and Berendes 8 The following findings were considered normal: (a) patent foramen ovale and (b) trivial–mild regurgitation of the tricuspid or pulmonary valve. A 12-lead electrocardiogram was obtained the same day of all the subjects.
Cardiologists’ clinical evaluation
Among subjects who were being evaluated in the cardiology clinic for the first time and without a previous history of echocardiography, the attending cardiologist was asked to complete a data form documenting their clinical findings and opinion as to whether the child had a “likely innocent” or “likely pathologic” murmur. This evaluation was carried out with the cardiologist using his/her own stethoscope, in advance of CardioscanTM auscultation, and without knowledge of the echocardiographic findings.
Data analysis
The sensitivity, specificity, and positive and negative predictive values of CardioscanTM software were determined with respect to identification of subjects having an abnormal echocardiogram. The likelihood ratio for a positive test was determined from these results, defined as the magnitude by which a positive test result – that is, potentially pathologic murmur according to CardioscanTM – occurs in a subject with, as opposed to without, the disease of interest – that is, congenital or acquired heart disease.Reference Jaeschke, Guyatt and Lijmer 9 A pre-determined sub-analysis was carried out that included only those subjects having a heart rate between 50 and 120 beats per minute, as the manufacturer indicated that this was the optimum heart rate range for CardioscanTM performance. Analyses were performed using a commercially available statistical package (Stata Version 9, College Station, Texas).
Ethical considerations
The study was approved by the Health Research Ethics Board at the University of Alberta. Written informed consent was obtained from a parent or guardian for all cases.
Results
We approached 150 eligible subjects, of which 20 declined participation and 4 were excluded as they were unable to sit still for auscultation. Therefore, 126 subjects participated (55 female, 44%), having a median age of 1.7 years (range 1 day to 17.6 years). Only 33 subjects (26%) had innocent murmurs (structurally normal hearts); the remaining 93 subjects had structural abnormalities as confirmed by echocardiography (Table 1). The timing of murmurs, as determined by a cardiologist, was systolic in 117 subjects, diastolic in one subject, both systolic and diastolic in four subjects, and continuous in four subjects. The sensitivity of CardioscanTM for the detection of a structural abnormality is summarised in Table 2, as is the specificity, positive predictive value, and negative predictive value.
Cardiac diagnoses as determined by echocardiography.
The number of diagnoses (n=102) exceeds the number of patients having CHD (n=93), because some patients had more than one diagnosis
* Four subjects with a moderate-sized patent ductus arteriosus (PDA) had a continuous murmur; six subjects with a small PDA had a systolic murmur
** This subject had an innocent murmur in the opinion of the attending cardiologist
CardioscanTM test characteristics.
bpm=beats per minute; CI=confidence interval; HR=heart rate; LR+=likelihood ratio of a “potentially pathologic murmur” assessment; NPV=negative predictive value; PPV=positive predictive value
Among subjects with a diastolic murmur (n=5), only one was detected by CardioscanTM as having a diastolic murmur. The 4 patients with a diastolic murmur audible to the human ear but not detected by CardioscanTM had rheumatic heart disease with aortic insufficiency (n=1), an atrial septal defect that required surgical closure associated with both a systolic murmur and a diastolic murmur (n=1), a bicuspid aortic valve with aortic insufficiency (n=1), and Morquio syndrome with aortic insufficiency (n=1). However, the latter patient also had a systolic murmur that was detected by CardioscanTM as a potentially pathologic murmur. Among subjects having a continuous murmur (n=4), all had a moderate-sized patent ductus arteriosus and all were detected by CardioscanTM as having both a potentially pathologic systolic murmur and a diastolic murmur.
A total of 15 subjects had a false negative interpretation by CardioscanTM. Of these, the murmur timing was systolic in 12, both systolic and diastolic in two, and diastolic alone in one. Structural abnormalities in this group are summarised in Table 3. False negative assessment of systolic murmurs included both systolic ejection murmurs – atrial septal defect and aortic stenosis – and holosystolic murmurs – tricuspid and mitral regurgitation. The median age of this group was 7.1 years (range 1.1–17.6). All but one patient had a heart rate between 50 and 120 beats per minute. A 12-lead electrocardiogram carried out the same day as the CardioscanTM recording was abnormal in 9/15. Therefore, addition of an abnormal electrocardiogram to the CardioscanTM interpretation improved the sensitivity (both tests combined) to 87/93 (93.5%, 95% CI 88.6, 98.5). Data on subjects with a false negative electrocardiogram and CardioscanTM interpretation (n=6) are summarised in Table 4.
Echocardiographic findings among subjects with a false negative CardioscanTM assessment.
MR=mitral regurgitation; TR=tricuspid regurgitation
* Indicates patients requiring medical or surgical intervention
Echocardiographic findings among subjects with a false negative CardioscanTM assessment and a normal electrocardiogram.
MR=mitral regurgitation; TR=tricuspid regurgitation.
In total, 24 subjects were being seen in the cardiology clinic for the first time, and the etiology of their murmur had not been previously confirmed by echocardiography, allowing comparison of CardioscanTM’s performance with that of one of the 14 staff cardiologists. Among all, 11 subjects had an abnormal echocardiogram and 13 had an innocent murmur (normal echocardiogram). Among the 11 with an abnormal echocardiogram, the cardiologist assessed the murmur as “likely pathologic” in 10 cases, and CardioscanTM assessed the murmur as potentially pathologic in nine cases. The one case missed by a cardiologist had a right coronary artery from the left sinus and no other lesions. Among the 13 cases with a structurally normal heart, the cardiologist correctly assessed the murmur as “likely innocent” in 12 cases, but CardioscanTM deemed the murmur not “potentially pathologic” in only three cases. The likelihood ratio for a “likely pathologic” assessment by the cardiologist among these 24 subjects was 11.8 compared with the CardioscanTM likelihood ratio of 1.1 among the same subjects.
The sensitivity of the system’s ability to distinguish innocent from pathologic murmurs was examined as a function of intensity among systolic murmurs. There were only six participants having a grade 1/6 murmur and only three participants having a grade 4+/6 murmur based on cardiologists’ clinical assessments, precluding meaningful analysis. Among participants having a grade 2/6 murmur, the sensitivity of the software was 68%, compared with participants having a grade 3/6 murmur where the sensitivity of the software was 97% (p=0.009).
Discussion
We demonstrated that this computer-based auscultation algorithm had modest sensitivity (84%) but poor specificity (30%) when applied to a group of infants, children, and adolescents having a broad range of innocent murmurs and structural cardiac lesions. Specificity was improved when the analysis was limited to children with a heart rate between 50 and 120 beats per minute, but the sensitivity was lower in this group, owing to the poor detection of atrial septal defects, which were relatively more prevalent in this subset of older children compared with the entire study population. In other words, limiting the use of CardioscanTM to the manufacturer-recommended heart rate range of 50–120 beats per minute did not significantly improve the performance of this product.
Addition of a 12-lead electrocardiogram to the CardioscanTM analysis did improve the sensitivity of screening for CHD. All cases with atrial septal defects that had a false negative assessment by CardioscanTM had an abnormal electrocardiogram, and were therefore detected using this combined CardioscanTM– electrocardiogram algorithm. None of the 6 subjects having both a false negative CardioscanTM interpretation and a normal electrocardiogram had haemodynamically significant CHD and none of them required surgical or catheter-based intervention or pharmacological therapy, although four of the six patients required cardiology follow-up.
The likelihood ratio of a CardioscanTM potentially pathologic murmur assessment was 1.2, meaning that this assessment was only 1.2 times more likely in children with – as opposed to without – a structural cardiac problem. This ratio increased to 2.6 when the CardioscanTM interpretation was limited to children with a heart rate between 50 and 120 beats per minute. By comparison, among the 24 subjects without a previous cardiology evaluation, the likelihood ratio of the cardiologists’ clinical assessment was 11.8. Likelihood ratios >10 reflect a diagnostic test, such as clinical assessment by a cardiologist, which generates a large and conclusive change from pre-test to post-test probability.Reference Jaeschke, Guyatt and Lijmer 9 Likelihood ratios between 5 and 10 are considered moderately conclusive. Our findings are consistent with previous investigators who have demonstrated the high sensitivity and specificity of murmur evaluation by paediatric cardiologists.Reference Newburger, Rosenthal, Williams, Fellows and Miettinen 10 – Reference McCrindle, Shaffer, Kan, Zahka, Rowe and Kidd 13
There are a few reports of computer-based auscultation assessment. Tavel and Katz conducted a study of predominantly adult subjects and demonstrated that those with an innocent murmur had a peak frequency that was usually <300 Hz and of shorter duration at peak frequencies compared with subjects with aortic valve stenosis.Reference Tavel and Katz 14 No other cardiac lesions were included in that study. Watrous et al had seven board-certified primary care physicians listen to 100 heart sound recordings and then re-listen to the same recordings with the aid of CardioscanTM interpretation.Reference Watrous, Thompson and Ackerman 15 Sensitivity for simply detecting a murmur increased from 76.6 to 89.1% using the CardioscanTM interface. Correct identification of pathological murmurs improved from 82.4 to 90.0%, and specificity for correctly identifying benign cases – innocent or no murmur – improved from 74.9 to 88.8%. Accordingly, decisions about whether or not to refer improved with the aid of CardioscanTM. However, this study was an evaluation of physicians listening to recorded heart sounds, rather than an evaluation of CardioscanTM versus echocardiography. DeGroff et al recorded heart sounds from 67 children with murmurs, 37 having cardiac pathology.Reference DeGroff, Bhatikar, Hertzberg, Shandas, Valdes-Cruz and Mahajan 16 They developed an artificial neural network that was able to distinguish innocent from pathologic sounds with up to 100% sensitivity and specificity. Likewise, Sepehri et alReference Sepehri, Hancq, Dutoit, Gharehbaghi, Kocharian and Kiani 17 and NoponenReference Noponen, Lukkarinen, Angerla and Sepponen 18 et al have illustrated the potential for computerised screening of CHD in children. To our knowledge, however, no device that interprets the clinical significance of heart sounds other than CardioscanTM has become commercially available.
We did not compare the interpretation of CardioscanTM with an interpretation of cardiologists listening to the CardioscanTM-recorded sounds, but rather with the interpretation of cardiologists listening to a subset (n=24) of previously unevaluated (unknown) cases. The rationale for this was twofold. First, doing so would not reflect the reality of clinical practice; cardiologists perform auscultation using a stethoscope that they are comfortable and familiar with rather than listen to recorded heart sounds on a computer using headphones, and their clinical evaluation of the cardiovascular system involves more than just auscultation. Second, what is important is the correct identification of patients having a cardiac lesion, either congenital or acquired, and in order for CardioscanTM or similar devices to be used as screening tools by primary-care physicians it must be evaluated against the reference standard of echocardiography.
Limitations
We had a relatively low number of subjects having innocent murmurs (n=33), because our centre does not routinely perform echocardiography among children with innocent murmurs, and having an echocardiogram was an inclusion requirement. We also had a low number of subjects evaluated by a cardiologist before echocardiography (n=24), because many were follow-up patients with previously identified cardiac lesions. Nevertheless, we were able to demonstrate that clinical assessment by a cardiologist has a high positive likelihood ratio. The only child who was deemed by their cardiologist to have an innocent murmur but who had an abnormal echocardiogram had a lesion that does not result in a murmur – that is, right coronary artery from the left sinus. This patient’s murmur was indeed “innocent” but with an incidental finding of asymptomatic pathology. Electrocardiograms were interpreted by a fully-trained paediatric cardiologist, and therefore the generalisability of the combined CardioscanTM-electrocardiogram analysis may not apply to situations where the electrocardiogram is interpreted by physicians having less experience with paediatric electrocardiograms. The combined CardioscanTM–electrocardiogram analysis was a secondary analysis, rather than pre-determined. We are unable to comment on the mechanisms by which CardioscanTM software identifies potentially pathologic murmurs, as this is proprietary information that was not shared with the authors. CardioscanTM is not designed to detect clicks, and therefore we evaluated only murmurs and not other additional heart sounds. CardioscanTM does not analyse the 2nd heart sound, limiting the potential to detect atrial septal defects. The proportion of participants having cardiac pathology (74%) was higher than that expected in a primary care practice, and therefore the positive and negative predictive values will very likely differ from what would be observed when used by general practitioners. A few patients had a systolic murmur of intensity 1/6 or 4/6 (or louder); therefore, it was not feasible to evaluate the software across the full spectrum of intensity. However, in order to be useful to primary care physicians, auscultation software needs to be sensitive across all murmur intensities. Therefore, we believe that it is more meaningful to focus on the overall sensitivity, across all murmur intensities, rather than the sensitivity stratified by grade 2/6 versus 3/6 intensities.
Conclusions
In summary, we found that the sensitivity of a commercially-available computer-aided auscultation algorithm was modest and insufficient to recognise a variety of cardiac lesions associated with murmurs, including four subjects with a moderate–large atrial septal defect and one subject with an atrioventricular septal defect, all of which required intervention. However, when combined with an electrocardiogram interpreted by a paediatric cardiologist, sensitivity improved to 93.5% with only minor lesions being missed. Specificity was low, meaning that CardioscanTM analysis would still result in multiple unnecessary referrals of children having innocent murmurs to paediatric cardiologists. CardioscanTM cannot be expected to identify lesions that may not be associated with a murmur, including congenital coronary anomalies, left-to-right shunts with pulmonary hypertension, and hypertrophic cardiomyopathy without left ventricular outflow tract obstruction. CardioscanTM may be a valuable tool for teaching auscultation to trainees, although we did not evaluate that in the present study. Re-evaluation of computer-aided auscultation will remain necessary as advances in this technology become available. Clinical evaluation of the cardiovascular system when performed by a paediatric cardiologist remains a sensitive and specific diagnostic test.
Acknowledgement
None.
Financial Support
Dr Cecilia Lee was supported by a Canadian Institutes of Health Research Health Professional Student Research Award. Mr Kevin Zuo and Dr Andrew Mackie were supported by the Women and Children’s Health Research Institute at the University of Alberta.
Conflicts of Interest
The authors have no conflicts of interest relevant to this article to disclose. CardioscanTM was provided free of charge to the study investigators by Zargis Medical Inc. The authors have not used CardioscanTM in non-study patients. None of the authors have had a financial relationship with Zargis Medical Inc. or any other disclosure. Data analysis and manuscript preparation were performed without involvement of Zargis Medical Inc. or its employees.
Ethical standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national guidelines on human experimentation (Ethical Conduct for Research Involving Humans, Canadian Institutes of Health Research, 1998) and with the Helsinki Declaration of 1975, as revised in 2008, and has been approved by the institutional committees – namely, the Health Research Ethics Board at the University of Alberta.
