The implementation of echocardiography has been evolved with regard to diagnostic accuracy and feasibility within the past decades so that, nowadays, it has become the first-line imaging modality for the evaluation of patients with heart diseases.Reference Gianstefani, Catibog and Whittaker1 However, the difficulty of transferring heavy standard echocardiography machines for bedside examination has led to some limitations in emergent and community-based situations, which is of great importance for the screening of cardiovascular diseases. The early 2000s were accompanied by major advancements in the technical aspects of ultrasound devices, leading to the miniaturization of echocardiography machines and the availability of easily portable devices named the “pocket-sized ultrasound device.”Reference Remenyi, Wilson and Steer2, Reference Sicari, Galderisi and Voigt3
The pocket-sized device has been found to be efficient, feasible and cost-effective in some emergent situations based on a focused diagnostic approach,Reference Kennedy Hall, Coffey and Herbst4, Reference Labovitz, Noble and Bierig5 the examination of hospitalised patients at bedside instead of transferring them to echocardiographic laboratory,Reference Gianstefani, Catibog and Whittaker1, Reference Badano, Nucifora and Stacul6–Reference Reant, Dijos and Arsac8 the evaluation of out-of-hospital individuals with cardiovascular diseases,Reference Trambaiolo, Papetti and Posteraro9 and the education of medical students and physician residents during training courses.Reference Galusko, Khanji, Bodger, Weston, Chambers and Ionescu10, Reference Kobal, Lior, Ben-Sasson, Liel-Cohen, Galante and Fuchs11 Moreover, it can also be used for the screening of heart diseases in population-based studies,Reference Weiner, Wang and Hutter12 particularly in limited-resources area and distant regions with a relatively fair access to physicians and medical equipment.Reference Sicari, Galderisi and Voigt3 Both the European Association of Echocardiography and the American Society of Echocardiography have recommended the use of pocket-sized ultrasound device as a feasible echocardiographic examination in the hands of experienced or fully educated non-experts.Reference Sicari, Galderisi and Voigt3, Reference Spencer, Kimura, Korcarz, Pellikka, Rahko and Siegel13
In this observational study, we sought to determine the diagnostic accuracy and cost-effectiveness of pocket-sized ultrasound device for the screening of structural heart diseases – including CHDs, valvular heart diseases, and cardiomyopathy – in children and adolescents by comparison with the standard transthoracic echocardiography. In the first phase, we examined hospital-based patients with a high burden of structural heart diseases to test our hypothesis with regard to the diagnostic accuracy of pocket-sized echocardiography. In the second phase, we assessed the accuracy and cost-effectiveness of the pocket-sized device in a group of school-aged individuals who were examined for the screening of structural heart diseases in an out-of-hospital setting.
Methods
Study setting and design
This prospective study was performed at two different settings, including school- and hospital-based population. The first phase of study was performed in a hospital-based clinic of paediatric cardiology due to high burden of structural heart diseases. In this prospective investigation, children and adolescents who referred to echocardiographic laboratory in the CHD clinic were consecutively enrolled into our study after being evaluated for inclusion and exclusion criteria. The inclusion criteria consisted of patients with suspected structural heart diseases referred to our clinic. The exclusion criteria included patients aged less than 5 years. When the image quality was low or data were missing or not accessible by the visual approach alone, the echocardiographic examination was considered inconclusive and the patient was excluded as well (one obese adolescent whose pocket-sized echocardiography was inconclusive).
The second phase of the study included school-aged individuals screened for structural heart diseases via echocardiographic examinations using the standard transthoracic echocardiography and the pocket-sized ultrasound device in the urban areas of Tehran, Iran. The second phase of the present study was aimed at evaluating the diagnostic accuracy and cost-effectiveness of the pocket-sized ultrasound device in a population-based investigation. Inclusion/exclusion criteria for second phase were the same as the first phase. The study protocol was approved by the local ethics committee of our institution.
Echocardiographic examinations and interpretations
In a hospital paediatric CHD clinic, in the first phase of study, all patients who were referred for the evaluation of suspected structural heart diseases underwent the standard transthoracic echocardiography with gold-standard fully equipped echocardiography machines (S60, GE Healthcare, Horten, Norway or iE33, Philips Ultrasound, Bothell, WA, United States of America). Experienced university faculty members with at least 10 years of experience in echocardiography carried out the cardiac examinations (AT and NS). All echocardiographic evaluations by the standard transthoracic echocardiography during the study were performed using the standards mentioned in the guidelines.Reference Lai, Geva and Shirali14, Reference Lang, Bierig and Devereux15 The findings of these echo machines were used as gold standard for the evaluation of diagnostic accuracy. Then, the same patient underwent another echocardiographic examination using the pocket-sized ultrasound device (Vscan, GE Healthcare, Milwaukee, WI, United States of America). The Vscan weighs 391 g and measures 13.5 × 7.4 × 2.8 cm3 in size. It provides 2-dimensional greyscale and colour Doppler images with a 1.7- to 3.8-MHz phased-array dual-probe transducer. The Vscan does not possess zoom, spectral Doppler, or the ability to perform velocity or time measurements. The second operator, a sonographer with more than 10 years of experience in echocardiography, performed examinations using the pocket-sized ultrasound device. The sonographer, blinded to the diagnosis achieved by the standard echocardiography, used an unlimited protocol upon examination by the pocket-sized ultrasound device to implement all echocardiographic views along with the adjustment of the gain and depth of the Vscan to reach a final diagnosis. The World Heart Federation’s criteria for the diagnosis of rheumatic heart disease were modified for the pocket-sized ultrasound device, in the absence of spectral Doppler.Reference Remenyi, Wilson and Steer2, Reference Lu, Sable and Ensing16 The echocardiographic parameters were interpreted online at bedside. All echocardiographic measurements evaluated by the pocket-sized ultrasound device were interpreted qualitatively. The interpretation of stenotic, prolaptic, and regurgitant lesion severity was based on a qualitative visual assessment since the Vscan lacks quantitative Doppler capabilities. Valvular regurgitation was categorised as normal, trivial, mild, mild-to-moderate, moderate, moderate-to-severe, and severe based on colour-flow mapping. Valvular prolapse and stenosis were interpreted as either absent or present. Valvular leaflet motion was defined as normal or any degrees of restriction or malcoaptation. The measurement of the ventricular systolic function was performed visually (normal function versus mild, moderate, or severe dysfunction). Heart chamber sizes were measured visually as normal, dilated, or hypoplastic/small. Large vessel abnormalities were reported as normal, dilated, or stenotic based on 2-dimensional images and colour Doppler study. Any other congenital defects or anatomic variations were also reported, when observed, as present or absent.
In the second phase of the study, echocardiographic examinations were performed in selected schools in the urban areas of Tehran. All examinations were performed by the same physicians in the first phase (AT and NS). Through a two-stage cluster sampling method, six schools were randomly selected from districts of Tehran. One class from each school was randomly selected, and all students whose parents signed the consent form were examined. Before the index examination day, the school management department informed the parents regarding the study protocol and the details of the examinations. All examinations were performed by two echocardiographers, experienced university faculty members (AT and NS). All these school-aged children were first evaluated using a fully equipped laptop-based portable echo machine (LOGIQ, GE Healthcare, Milwaukee, WI, United States of America). This finding was considered to be the gold standard of diagnosis (i.e., the standard transthoracic echocardiography). Thereafter, the same student was re-examined using the pocket-sized ultrasound device on the same day by the echocardiographer. All protocols for the measurement of the echo variables to reach a final diagnosis were the same as those in the first phase. Moreover, the fully experienced sonographer in this study had examined some patients and healthy individuals with the Vscan before this study and compared them with the standard transthoracic echocardiography to reach a consensus concerning the shortcomings and strengths of the Vscan for the detection of structural heart diseases.
Statistical analysis
The continuous variables are reported as the mean ± the standard deviation, and the categorical variables are reported as numbers (%). The agreement between the echocardiographic findings provided by the two protocols (standard transthoracic echocardiography versus pocket-sized ultrasound device) was assessed using the κ statistics, and a 95% confidence interval was also reported. Kappa values of less than 0.2 were interpreted as slight, 0.21–0.4 as fair, 0.41–0.6 as moderate, 0.61–0.8 as good, and 0.81–1.00 as excellent agreements.Reference Landis and Koch17 To calculate the accuracy of examination by the pocket-sized ultrasound device in comparison with that of the standard transthoracic echocardiography, we considered any valvular heart diseases measured as trivial or greater severity, any grades of changes in the heart chambers and wall thickness, any changes in the vessel sizes, and the presence of any CHDs or anatomic variations to be abnormal findings and compared them with the normal findings. Moreover, we calculated the diagnostic accuracy of the pocket-sized ultrasound device for valvular regurgitation after considering trivial and mild severity as non-pathologic involvements, not requiring further follow-up upon screening. In addition, discordant findings for valvular regurgitation were defined as more than one level of difference in the grading of the valvular regurgitant jet between the standard transthoracic echocardiography and pocket-sized ultrasound device examinations. Moreover, a subgroup analysis was conducted among individuals with major CHDs. We calculated the accuracy of identifying individuals with major CHDs in both cohorts.
Results
Baseline characteristics
A total of 73 patients (42 males, 57.5%) with a mean age of 9.9 ± 3.2 years were evaluated in a hospital-based setting. The most common indications for echocardiography included murmur (39.7%), palpitation (13.7%), and chest pain (6.8%). Other baseline data for the hospital-based individuals are summarised in Supplementary Table 1. Of the143 individuals examined in the school-based population, 90 (62.9%) individuals were male, and the mean of age was 12.8 ± 2.9 years.
The durations of echocardiographic examinations via the standard transthoracic echocardiography and the pocket-sized ultrasound device were 6.7 ± 2.9 min and 3.9 ± 1.3 min, respectively (p < 0.05). The examination time for the standard echocardiography was also longer than the time for the pocket-sized examination in the school-based individuals (p < 0.05). Based on the findings of standard transthoracic echocardiography, the abnormal findings were observed in 89 and 53% of the individuals among the hospital- and school-based individuals, correspondingly. All echocardiographic findings based on the standard transthoracic echocardiography are summarised in Table 1.
Structural heart disease findings based on standard transthoracic echocardiographic examination in hospital- and school-based individuals.
AV = aortic valve; LV = left ventricle; MV = mitral valve; NL = normal; PV = pulmonary valve; RV = right ventricle; TV = tricuspid valve; RHD = rheumatic heart disease
* Abnormal was defined as greater than mild regurgitation
Agreement between pocket-sized and standard echocardiography
Among the hospital-based individuals, the agreement between the two measurements was good or excellent in the majority of the diagnoses, particularly major CHDs; however, the pulmonary valve and tricuspid valve regurgitation showed moderate agreement. Among major CHDs in hospital-based cohort, the agreements for subvalvular stenosis, atrial septal defect, bicuspid aortic valve and patent foramen ovale were good (κ statistics 0.61–0.8), and for other pathologies were excellent (κ statistics 0.81–1). Agreement for bicuspid aortic valve was excellent in school-based population. Among valvular pathologies, agreements for tricuspid valve regurgitation were excellent among hospital-based and moderate among school-based cohorts (0.91 [95% confidence interval 0.80–1] and 0.56 [95% confidence interval 0.12–1, respectively). The agreements for pulmonary valve regurgitation were 0.6 [95% confidence interval 0.28–0.91]) among school-based population. Other values for valvular findings were good or excellent. As can be estimated, there were a few cases of CHDs among the school-based individuals; nonetheless, all findings had an excellent or good agreement, except for the tricuspid valve regurgitation with a moderate agreement. All agreements by kappa statistics are tabulated in Table 2.
Agreement between standard and pocket-sized echocardiography among hospital- and school-based individuals.
AV = aortic valve; LV = left ventricle; NA = not applicable; NL = normal; PV = pulmonary valve; RV = right ventricle; other abbreviations are similar to Table 1
* Data are presented as κ statistics (95% confidence interval)
** Abnormal was defined as greater than mild regurgitation
*** One patient had right-sided aortic arch among school-based individuals
Accuracy of pocket-sized versus standard echocardiography
For the detection of any type of structural heart diseases, the overall sensitivity was 87.5% (95% confidence interval 47.3–99.7%) and the overall positive predictive value was 63.6% (95% confidence interval 39.5–82.4%) among the hospital individuals. These values were 88% (95% confidence interval 77.8–94.7%) and 71.1% (95% confidence interval 63.6–77.6%), respectively, among the school children. When we considered any degrees of abnormalities – even trivial valvular findings – among the hospital-based individuals, the sensitivity of pocket-sized ultrasound device examination for detecting structural heart diseases was acceptable in the majority of the diagnoses, and some measurements showed low sensitivities (ranged from 50 to 60%). Other values are summarised in Table 3. Among the school-based individuals, the sensitivity was low in the measurement of the left ventricular size and dysfunction; other values had high sensitivities (Table 4).
Diagnostic accuracy of measured parameters by pocket-sized compared to standard echocardiography among hospital-based individuals.
CI = confidence interval; LV = left ventricle; MV = mitral valve; NA = not applicable; NL = normal; PV = pulmonary valve; RV = right ventricle; TV = tricuspid valve; other abbreviations are similar to Table 1
* Abnormal was defined as greater than mild regurgitation
Diagnostic accuracy of measured parameters by pocket-sized compared to standard echocardiography among school-based individuals.
CI = confidence interval; LV = left ventricle; MV = mitral valve; NA = not applicable; NL = normal; other abbreviations are similar to Table 1
* Abnormal was defined as greater than mild regurgitation
** One patient had right-sided aortic arch
When we calculated the diagnostic accuracy of findings as normal or mild versus more than mild regurgitation, the amounts of sensitivity and specificity were improved (Supplementary Table 2). In addition, when we examined to discriminate individuals with major CHDs from those without it, the sensitivity and specificity of pocket-sized echocardiography improved in both cohorts. The overall sensitivity and specificity were 86.4% (95% confidence interval 75.7–93.6) and 100% (95% confidence interval 59–100) for hospital-based cohort and 100% (95% confidence interval 2.5–100) and 100% (95% confidence interval 97.4–100) for school-based cohort, respectively. The rates of false-positive and false-negative values for single diagnoses are summarised in Table 5.
The detection rate of major CHDs in both hospital- and school-based population.
All abbreviations are similar to Table 1.
The overestimation of aortic and mitral valves’ regurgitation, defined as a difference of more than two grades, was observed in fewer than 5% of both hospital- and school-based individuals, whereas the overestimation of tricuspid and pulmonary valves’ regurgitation was about 10%. The amounts of underestimated discordant findings, a difference of more than two grades in regurgitation, were 6.8, 16.4, and 9.6% for the regurgitation of mitral valve, tricuspid valve, and pulmonary valve, respectively, among the hospital-based individuals. Underestimated regurgitation of more than two grades was less than 5% for all valves among the school-based individuals (Fig 1).
Differences between the pocket-sized ultrasound device and the standard transthoracic echocardiography in the grading of valvular regurgitation among (a) hospital-based and (b) school-based populations. Notes: “−2” means two grades of underestimation; “−1” means one grade of underestimation; “Equal” means a similar finding; “1” means one grade of overestimation; “2” means two grades of overestimation. AV = aortic valve; MV = mitral valve; PV = pulmonary valve; TV = tricuspid valve.
Cost-effectiveness analysis
The costs of pocket-sized ultrasound device and laptop standard transthoracic echocardiography machines were about 5000 United States’ dollars and 30,000 United States’ dollars, calculating about an 85% decrease in the cost of machine. In addition, the cost of echocardiographic examinations by echocardiographer using the pocket-sized ultrasound device compared with the laptop-based standard transthoracic echocardiography was decreased by 65% for the screening of structural heart diseases, based on the cost of an echocardiographic examination accredited by the Iranian Society of Echocardiography. In total, the cost of an examination reduced by approximately 70% for an individual.
Discussion
In this prospective study, we found that the pocket-sized ultrasound device is a feasible and relatively accurate tool in experienced hands for the screening of structural heart disease among school-aged population. The agreement between the pocket-sized and the standard echocardiography was good to excellent for major CHDs in both cohorts, but it was moderate for tricuspid and pulmonary valves’ regurgitation. We also demonstrated that the pocket-sized echocardiography had relatively high sensitivities and specificities for the majority of structural heart diseases, even non-pathologic valvular heart diseases such as mild regurgitations. In addition, this accuracy improved for the detection of CHDs and the high grades of valvular heart diseases when they were calculated separately. Our investigation extends the findings of prior studies on the pocket-sized ultrasound device to a healthy school-aged population.
Some previous studies have tried to explore the feasibility and accuracy of the pocket-sized ultrasound device in the setting of hospital clinics or out-of-hospital clinics. Cullen et alReference Cullen, Geske, Anavekar, Askew, Lewis and Oh18 found that the standard transthoracic echocardiography and the pocket-sized ultrasound device had good correlations for the left ventricular function and wall motion in patients with myocardial infarction. In experienced hands, the pocket-sized ultrasound device can be used for the assessment of the left ventricular function; however, it cannot be used for comprehensive assessments. In contrast, some other reports have shown that in experienced hands, the pocket-sized ultrasound device is associated with a good diagnostic accuracy for the assessment of valvular heart diseases, not including the pulmonary valveReference Prinz and Voigt19, Reference Andersen, Haugen, Graven, Salvesen, Mjolstad and Dalen20 and mitral and aortic valves’ diseases.Reference Lafitte, Alimazighi and Reant21 In the light of our findings, it appears that the pocket-sized ultrasound device can be employed for the screening of pathologic valvular defects; nevertheless, it may overestimate or underestimate the severity of valvular regurgitation, particularly in lower grades.
The pocket-sized ultrasound device lacks Doppler modalities and has a small window view, precluding echocardiographers from evaluating the exact details or grades of valvular heart disease. Anderson et alReference Andersen, Haugen, Graven, Salvesen, Mjolstad and Dalen20 showed that the pocket-sized echocardiography underestimated the tricuspid valve regurgitation and aortic valve stenosis. Kono et alReference Kono, Fukuda and Shimada22 demonstrated that the sensitivity and specificity of the pocket-sized device were about 95% for tricuspid and mitral valves’ regurgitation greater than moderate severity. Prinz et alReference Prinz and Voigt19 showed an excellent agreement for the detection of the valvular regurgitation, not evaluating pulmonary valve. These findings are relatively similar to our findings and underscore the reliability of the pocket-sized device for the detection of valvular heart disease. Previous reports have not evaluated the accuracy of pocket-sized ultrasound device for the pulmonary valve pathologies, with the majority of them having demonstrated an underestimation of the pocket-sized echocardiography for the pathologies of tricuspid valve.
Nascimento et alReference Nascimento, Sable and Nunes23 demonstrated the feasibility of pocket-sized ultrasound device for the detection of rheumatic heart diseases in school-aged individuals in Brazil. They diagnosed 4 and 0.5% of school children with borderline and definite rheumatic heart diseases, respectively. We diagnosed 4.1 and 8.2% of our patients with borderline and definite rheumatic heart diseases, respectively, in hospital-based cohort. In addition, among the school-aged population, there was only one (0.7%) case of borderline rheumatic heart disease. The agreement between the pocket-sized ultrasound device and the standard transthoracic echocardiography in both settings was excellent (κ = 1). The relatively high rates of borderline and definite rheumatic heart diseases in the hospital-based individuals can be explained by the fact that our hospital is a large tertiary centre dedicated to cardiovascular diseases.
Based on estimations, the prevalence of CHDs is 2.5–20.1% for different diagnoses in children in the United States of America.Reference Benjamin, Virani and Callaway24 In addition, the increasing prevalence of CHDs will add to the growing valvular heart disease population.Reference Le Gloan, Mercier and Dore25 Riley et alReference Riley, Sable and Prasad26 reported that the pocket-sized ultrasound device had a sensitivity of 75% (95% confidence interval 43–94%) for CHDs. The sensitivity of pocket-sized ultrasound device was slightly higher in our report, and there was good to excellent agreement between devices for the detection of major CHDs.
The diagnostic accuracy of pocket-sized device has been found to have good or excellent agreement for the majority of cardiac pathologies. Given its diagnostic values and the low cost of the device, it can also be associated with significant cost benefits in large-scale population-based studies to screen cardiac diseases in healthy individuals and to select appropriate patients for laboratory-based echocardiographic examinations.
Study limitations
This report suffers from some limitations. First, the sample size of the school-based population was too small to allow an evaluation of more cases of CHDs. Second, the pocket-sized ultrasound device examination was performed by an experienced sonographer among the hospital-based patients, whereas it was done by an echocardiographer in the school-based population. The agreement between the standard transthoracic echocardiography and the pocket-sized ultrasound device was good to excellent, and the diagnostic accuracies were good in the majority of the pathologies in both cohorts, except for some valvular heart diseases. Third, the absence of quantitative measurements, Doppler modality, proper views, and high-frequency transducers precluded more accurate evaluations, even in the hands of our fully experienced echocardiographers. What should, however, be taken into consideration is that the European Association of Echocardiography recommends that the pocket-sized ultrasound device examination not be used as a complete diagnostic echocardiographic test.Reference Sicari, Galderisi and Voigt3
Conclusions
This study supports the prior findings with regard to the use of pocket-sized echocardiography for the detection of structural heart diseases. In addition, in the hands of experienced operators, the pocket-sized ultrasound device can be implemented as a cheap and easily portable screening tool in community-based investigations.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1047951119003111
Acknowledgements
We greatly thank the regional representative of GE Healthcare in our country (Tajhizat Pezeshki Pishrafteh Co., Tehran, Iran) for having kindly provided us with the Vscan machine and the Laptop echocardiography machine and the members of the company, who helped us during the study mainly including Dr. Arash Shafii and Mrs. Bahareh Aghababaei Samani. In addition, we would like to warmly thank Mrs. Shahin Kakavand for her collaboration in the conduct of some parts of this study in the Pediatric Echocardiography Laboratory and Dr. Hooman Bakhshandeh for his consultation in the statistical aspects of study, both from Rajaie Cardiovascular Medical and Research Center, Tehran, Iran.
Financial Support
This study was mainly supported by a grant from the National Institute for Medical Research Development under the identification number of 962126. Another portion of the funding was provided by Rajaie Cardiovascular Medical and Research Center, Tehran, Iran. Moreover, the regional representative of GE Healthcare in our country (Tajhizat Pezeshki Pishrafteh Co., Tehran, Iran) provided us with the Vscan machine and the Laptop echocardiography machine.
Conflicts of Interest
None.
Author Contributions
Conception and design of the research: A.T., N.S., M.M.P., A.M., G.O., A.A.G., F.N., M.M., Y.R., and S.H.; acquisition of data: A.T., N.S., B.K., F.R.G., and M.M.; analysis and interpretation of the data: N.S., Y.R., and S.H.; statistical analysis: Y.R.; writing of the manuscript: A.T., N.S., Y.R., and S.H.; critical revision of the manuscript for intellectual content: A.T., N.S., M.M.P., F.R.G., M.M., A.M., G.O., A.A.G., B.K., F.N., M.M., Y.R., and S.H.
