Ensuring competent well-trained paediatric cardiologists requires learning strategies, training, technology, and developmental efforts. Reference Grossman and Salas1,Reference Salas, Stagl and Locke2 Transthoracic echocardiography is a core skill, which trainees must learn in paediatric cardiology training as it is fundamental to the assessment of congenital heart disease (CHD). Although standards required in training and performance of echocardiography and guidelines for the development of paediatric echocardiographic skills have been published, there is limited awareness of how best to train cardiology trainees in order to transfer to live complex cardiac cases or the real-life situation of imaging a cardiac defect not previously encountered. Reference Henry, DeMaria and Gramiak3–Reference Lai, Geva and Shirali7
Transfer is the ability to perform an acquired complex cognitive skill in new, unfamiliar surroundings such as imaging cardiac cases not previously encountered. Near transfer is the ability to apply skills trainees have practiced or previously performed and replicate them in a situation previously encountered, for example, an echocardiogram in a child with tetralogy of Fallot which the trainee previously scanned. Reference van Merriënboer8 Far transfer involves a transfer of skills from a taught similar context to a very different context. Positive transfer is when a previous experience facilitates a positive experience in managing a new previously unseen patient. Negative transfer is when a previous experience impacts negatively on a new previously unseen patient. Distinctive training inputs, training design, training outputs, and conditions of transfer are recognised to impact on successful transfer (Baldwin and Ford model, Fig 1). Reference Baldwin and Ford9 One benefit of this model is that it allows an assessment of the impact of each individual input factor such as training design on training outputs such as learning and retention. Reference Baldwin and Ford9 Trainee readiness and capacity to demonstrate transfer of training skills in echocardiography to real-life complex cardiac conditions remains challenging because of anatomical and physiological complexity and variation between patients, technical skills needed to acquire and interpret high-quality images, and the capacity to use the echocardiogram machine in an active patient. Reference Ceresnak, Axelrod and Sacks10
Baldwin and Ford’s transfer of training model (1988).
Although simulation and boot camp training may assist in the imaging of simple cardiac defects, there is a knowledge gap in how to train echocardiography skills transferable to real-life complex cases given the marked variation in cardiac defects encountered. Reference Skinner, Freeman and Sheehan11,Reference Maskatia, Altman, Morris and Cabrera12 This is important because erroneous diagnoses may result in patient morbidity or mortality. Reference Benavides, Gauvreau, Jenkins and Geva13,Reference Levine, Geva and Brown14 Therefore, the variation theoretical framework proposed by Marton and Booth (1997), originally derived from the phenomenographical school of thought, was employed in this study. Reference Marton and Booth15,Reference Marton and Tsui16 Marton and Tsui specify four patterns of variation: contrast, generalisation, separation, and fusion. Reference Marton and Tsui16 An example of contrast would be the location of ventricular septal defects (Fig 2). An example of generalisation would be that left heart obstructive lesions often occur together, for example, mitral stenosis, bicuspid aortic stenosis, and coarctation. An example of separation would be the distinction between double outlet right ventricle with >50% aortic override and mitral-aortic fibrous discontinuity from tetralogy of Fallot with subaortic ventricular septal defect, aortic override, and antero-cephalad deviation of the infundibular septum. A potential example of fusion would combine two or more cardiac defects to surmise the presence of additional pathology. This could include a child with a double inlet left ventricle, hypoplastic right ventricle, restrictive ventricular septal defect, with the aorta transposed off the hypoplastic right ventricle causing coarctation because of limited flow through the aortic arch.
Comparing and contrasting ventricular septal defect locations (Morrison, M.M. and McMahon, C.J.) (2018). Congenital Heart Disease in Down syndrome. In Tech Open.
Our hypothesis is that trainees exposed to high-variation training develop better transfer skills. The research questions in this study are (1) do trainees exposed to a high-variation training protocol develop greater transfer skills in echocardiography? and (2) if so, do trainees develop a greater capacity to employ the different types of variation in new complex cases?
Materials and methods
The study was conducted in the National Paediatric Cardiology Centre at Children’s Health Ireland, Crumlin, Dublin.
Patients
All trainees (n = 16) were pre-consultant level and post-basic residency training (>year 3) with no or very limited prior echocardiography exposure. They were invited by e-mail to participate in the study. Trainees were randomly allocated from three different strata (fellows, residents, and specialists) to one of the two groups: low-variation or high-variation training.
Materials
Group one: low-variation group
All trainees were exposed to an 8-week course of intensive echocardiography training using real patients, mannequins (Resusci Annie, Cardiac Services Dublin, Ireland), and three-dimensional printed hearts (Fig 3). The cases of low-variation trainees were exposed to are presented in Table 1. These were situations, involving trainer, trainee, and patient in a controlled environment in the echocardiography laboratory during working hours where trainees image cases appropriate to their stage of training. They were supervised by the cardiac physiologist or consultant paediatric cardiologist. They were provided with feedback on the correct probe position, correct cardiac views (subcostal, four-chamber, parasternal long and short-axis, and suprasternal), use of two-dimensional, Doppler, and M-mode echocardiography based upon their technical skills, their echocardiogram report, and their interpretation of the cardiac anatomy.
Training use of mannequins, three-dimensional printed hearts, and live patients. Top left panel: imaging mannequins to obtain standard views. Top right panel: mannequins with superimposed three-dimensional printed hearts. Bottom left panel: selection of different complexity three-dimensional printed hearts. Bottom right panel: live scanning of cardiac patient during boot camp live sessions.
Low and high-variation cardiac defects.
*Low-variation and high-variation training groups received the same duration of overall training.
High-variation group
Trainees in the high-variation group were exposed to increasing variation (increased combinations of cardiac lesions) and greater complexity (Table 1). The sequence and combination of complex problems were decided by two consultant cardiologists within the echocardiography department in accordance with the curriculum. Trainees in the high-variation group were trained in the above lesions, but also in imaging patients with several combinations of lesions, for example, ventricular septal defect with the bicuspid aortic valve, coarctation, and hypoplastic ascending aorta. Specifically, this high-variation group of trainees was asked during their training period to compare and contrast some of these studies and employ fusion, generalization, and separation principles to deduct associated cardiac lesions in specific anatomic and physiologic settings. Reference Marton and Booth15 .
All trainees also attended formal training sessions during a boot camp day (Fig 3) detailing cardiac anatomy, imaging techniques, the sequence of cardiac imaging, and techniques in two-dimensional, Doppler, and colour imaging. Reference Ceresnak, Axelrod and Sacks10,Reference Maskatia, Altman, Morris and Cabrera12 Real patients with a mixture of cardiac lesions from normal to minor defects, moderate defects, and major cardiac complex defects were imaged by the trainees depending on their variation group (Table 1).
Instrument
The unseen patient echocardiograms (hypoplastic left heart syndrome) were used as an instrument to test transfer in paediatric echocardiography. The score for the overall echocardiogram performance was consistent with the expected standard for trainees having completed the boot camp and 2 months of training, not the performance of a perfect echocardiogram by an experienced echocardiographer.
All trainees were interviewed face to face by one researcher. Open and closed questions were asked of trainees focusing on how variation influenced their transfer of training to the echocardiographic performance. Trainees were specifically asked about which variation factors they believed were most helpful to transfer echocardiogram skills in the unseen case. The interview lasted 30 minutes and was audio-taped. This was transcribed by a secretary.
Procedure
The trainees were graded by a consultant imaging cardiologist and senior cardiac physiologist on the blind echocardiogram case based upon:
Echocardiogram performance
Both groups recorded the views they obtained using the image store function on the echocardiography machines.
Echocardiogram report
Both groups reported their findings in a handwritten report detailing the cardiac situs, segmental anatomy, relationship of outflow tracts, atrioventricular valve regurgitation, and cardiac systolic, and diastolic function.
Echocardiogram technical proficiency
Trainees were also recorded as they performed the echocardiograms to determine their capacity to set up the machine, acquire the appropriate sequence of images, and perform two-dimensional, colour, and Doppler echocardiography. This third score was awarded by consensus after discussion between the two blinded reviewers. This was based upon interaction with the patient during the scan, manipulation of the echocardiogram console (optimising focus, depth, and gain), and fluency in acquiring the images required.
Each of these scores was assigned as 0 = fails to meet expectations, 1 = meets expectations, and 2 = exceeds expectations, based upon a previously validated scoring system. Reference Levine, Geva and Brown14
Cumulative score of all three scores
The three individual scores (performance, report, and technical proficiency) were cumulatively added to compare the overall performance of the high-variation to the low-variation group.
Face-to-face interviews using open and closed questions were recorded and transcribed by two dedicated transcriptionists. Coding was performed using concrete words and descriptive narratives. We assessed whether saturation was achieved. Member checking was also performed to ensure reliable recording of trainee statements. Themes were sought as to whether and how the trainees employed variation to further their comprehension of complex cardiac defects and whether this aided transfer during echocardiography.
Ethical approval
Ethical approval was granted from the Institutional Review Board (IRB) Ethics Committee at Children’s Health Ireland, Crumlin, Dublin. Informed written consent was obtained from the study participants.
Results
Demographics were comparable for both groups (Table 2).
Demographics of trainees amongst both groups.
Inter-observer and intra-observer agreement for grading
The inter-observer agreement for the scoring of the echocardiogram performance was 0.87 (only 2 scores out of the 16 differing between the 2 graders) with the 2 patients scoring differently by 1 grade. The intra-observer agreement was 0.93 (only 1 score differing for the 1 reviewer when he tested on the total 16 scores the second time) with 1 grade difference being “meets expectations” compared to “exceeds expectations”.
High-variation compared to low-variation group in aiding transfer of echocardiography skills
Cumulative score for performance, report, and technical proficiency
The high-variation group showed a higher cumulative score than the low-variation group, χ2 = 8.223, p = .016 (Table 3).
Results for low and high-variation groups.
Scores between 0 (does not meet expectations) and 2 (exceeds expectations)
Echocardiogram performance
The high-variation group scored higher on echocardiogram performance than the low-variation group, but this difference did not reach statistical significance, χ2 = 2, p = .368 (Table 3).
Echocardiogram report
The high-variation group scored higher than the low-variation group and this difference almost reached statistical significance, χ2 = 5.94, p = .052 (Table 3).
Technical proficiency
Differences were less noticeable for the technical/patient interaction task, χ2 = 1.44, p = .485 (Table 3).
Employment of variation theory in aiding transfer
Two general themes emerged from the qualitative analysis of the interview data: (1) the use of strategies described in variation theory to aid transfer to describe abnormal hearts and (2) the use of live formative feedback from trainers during hands-on echocardiography training.
Use of strategies described in variation theory in aiding transfer
Trainees repeatedly highlighted the importance of understanding the normal cardiac anatomy before they could compare it to abnormal cardiac structures. They also reported the need to image multiple normal hearts prior to scanning abnormal hearts.
Comparing and contrasting was frequently used by the high-variation group on the test case to differentiate the left ventricle (smooth-walled, septo-phobic atrioventricular valve, more basal location of the atrioventricular valve) from the right ventricle (triangular-shaped, moderator band, septo-philic atrioventricular valve, and trabeculated ventricle) and the aorta (origin of the coronary arteries and head and neck vessels) compared to the pulmonary arterial anatomy (early bifurcation into left and right pulmonary arteries).
“I found the variation theory most helpful for helping to define which ventricle was left ventricle and which was right ventricle morphology based upon moderator band and the septal attachments of the tricuspid valve….” (H1)
Some trainees fused additional cues from the child’s clinical picture, sternotomy scars, and the colour of the child.
“Yes I used all the information I could… the sternotomy scar. cyanosis and the fact that he had a Fontan on echo helped provide me with cues to what was going on… it was helpful to look at all the child not just the echo…….” (H3)
The trainees in the high-variation cohort also described the benefit of high variation translating to them writing richer narrative reports for the interpretative part of the study.
“I think the variation training helped us, allowed us write out a more descriptive description of what we saw and not just make a guess of what we saw.” (H6)
Live formative feedback from trainers during hands-on training
All of the trainees highlighted the benefit of the boot camp. Although they appreciated the lecture series, hands-on training with mannequins, and the three-dimensional printed cardiac models, 10 out of the 16 trainees emphasised the most useful training was received while imaging live patients on the boot camp day with live hands-on formative feedback in the echocardiogram setting. Trainees in both the low-variation and high-variation groups voiced this opinion.
“It was good feedback on the day. There were a lot of instructors there… I think what I do with the feedback is really important because echo is the most complicated type of sonography.” (L1)
Trainees emphasised that the tutor providing descriptive feedback on how the trainee should manoeuvre the probe to optimise image acquisition was far more instructive than the tutor manoeuvring the probe for them.
“For me at least I learnt best when the trainers told me how to move the probe…. I don’t really get it if they took the probe of me and just showed me…that was really important that I could get the images myself….” (H2)
Discussion
The findings of this study confirm our first hypothesis that trainees exposed to a high-variation training protocol develop greater transfer skills in performing transthoracic echocardiography and interpreting complex cardiac cases than trainees exposed to a low-variation training protocol. The cumulative score was significantly higher for the high-variation group compared to the low-variation group. Furthermore, the high-variation group performed somewhat better on the global echocardiography performance, and most of all in the quality of the echocardiogram interpretation vis-à-vis the report they generated. The second hypothesis, that trainees exposed to a high-variation training protocol develop a greater capacity to employ different types of variation than trainees exposed to a low-variation training protocol, was also confirmed by the study findings. The trainees reported the variation training aided them in delineating cardiac anatomy in terms of ventricular morphology, atrioventricular valve attachments, outflow tracts, and combinations of cardiac defects in the unseen cases. The use of the echocardiography boot camp was well received, in line with previous reports. Reference Ceresnak, Axelrod and Sacks10,Reference Maskatia, Altman, Morris and Cabrera12
The findings of this mixed-methods study have reinforced the value of variation training in facilitating transfer of training in a complex skill such as transthoracic echocardiography. Reference Creswell17 This represents one of the first studies demonstrating the use of specific patterns of variation in allowing trainees to show transfer to echocardiography. Up until now, there is no empirical evidence of the relative importance or “interactive” importance of each of these in facilitating transfer skills in echocardiography.
The importance of achieving confidence in delineating normal anatomical features is critical in so far as providing the foundation against which compare and contrast, separation, generalizability, and fusion can be framed. Trainees highlighted they could not conceptualise abnormalities without a firm grasp of what constituted normal first. Several trainees in the low-variation group expressed how challenging it was for them to simply describe what they saw. This was reflected in the reports they generated where they attempted to provide a diagnosis rather than simply describing what they saw. This occasionally resulted in erroneous reporting of the cardiac defect. The challenges which trainees in the low-variation group experienced in managing complex cases, including risks of misdiagnosis, highlights the importance of point-of-care ultrasound (POCUS) trainees not attempting complex diagnostic echocardiography.
Several trainees in the high-variation group specifically used “comparing and contrasting” for different aspects of the unseen case. The ability to compare and contrast may be a more overt way for us to use variation theory in learning to differentiate and define structures. Other forms of variation theory appear to be more nuanced or subconscious, for example, fusion of different cues the patient provided. Kok has previously reported on the beneficial use of case comparisons in building visual expertise in radiological training, highlighting the role of variation theory in other non-cardiology disciplines. Reference Kok18
Clinical variation under simulation conditions allows medical trainees exposure to conditions, which they may not otherwise encounter. Paas and van Merriënboer have reported that high-variability over worked examples was beneficial, but over problem-solving it was not, as a consequence of the combination of problem-solving and high-variation producing cognitive overload. Interestingly, only one of the trainees involved in this study reported cognitive overload in the echocardiography transfer process. Reference Paas and van Merriënboer19 Trainees reported the extensive preparation they received as possibly one of the reasons for preventing overload. They were also reassured at the start of the study that this was an educational exercise and did not impact on actual medical care.
The echocardiography boot camp, use of mannequins, and three-dimensional printed models were well received by both groups of trainees in accordance with previous studies. Reference Ceresnak, Axelrod and Sacks10,Reference Maskatia, Altman, Morris and Cabrera12 However, trainees rated imaging of live cases on the boot camp day and during training as superior to the use of mannequins or 3D models in learning echocardiography. The access to cases of low and high-variation facilitated trainees using transfer in the training situation. Hands-on training with live formative feedback from trainers in optimising images and interpreting the images they were obtaining appear fundamental in preparing for transfer. Allowing the trainees to retain control of the probe while imaging with the trainers continuously tweaking their probe positioning received the most positive feedback from the trainees. This highlights the need to build autonomy for the trainee in acquiring images in newly encountered cases. Reference Kossaify and Grollier20,Reference Nair, Siu and Sloggett21 A continuous back-and-forth dialogue between trainer and trainee during this part of the training process builds rapport between both parties. Reference Swanepoel and DeBeer22 Jaarsma et al. have recently echoed our findings in their study amongst clinical problem-solving in pathology, apprentice interaction and adaptation increased when the apprentice operated the microscope, thereby demonstrating it is not only important to externalise the master performance, but also that of the trainee or apprentice. Reference Jaarsma, Boshuizen, Jarodska and van Merriënboer23
Implications for echocardiography training
These study findings can be incorporated into echocardiography training programmes including hosting an initial boot camp (pre and post-COVID-19) as an introduction to echocardiography. Trainees can image normal cases and low complexity cases until they are competent and confident and then be introduced to high-variation complexity over the following months. The timing of introduction of high complexity cases may vary depending on trainee characteristics Reference Ericsson24 (Fig 1). Imaging complex cases with formative feedback from consultant cardiologists or physiologists is a critical part of training. The importance of the trainees having hands on the probe with immediate feedback from their supervisor while learning is reinforced rather than feedback being provided several days later. Additionally, use of the variation theory is highlighted during this training process. Reference Cheng25
Broader implications for training
Other areas of expertise such as improvisational jazz (e.g., Miles Davis) have shown benefits from variation training of playing multiple instruments, Reference Wopereis, Stoyanov, Kirchener and Van Merrienboer26 and elite sports stars (e.g., Roger Federer) who benefited from a wide sampling of varied sports in aiding transfer rather than just deliberate practice. Reference Epstein27 .
There were several limitations to this single-centre study. There was a limited sample size of 16 trainees given the extensive time and effort in training the patients. The time course of the training for students was limited to 2 months. Ideally, the study would have been conducted over a longer period more consistent with training in paediatric echocardiography. This study solely examined the influence of training design (variation theory) on transfer of training in echocardiography. Further studies evaluating the impact of trainee characteristics and the work environment on facilitating transfer may prove equally productive (Fig 1).
In conclusion, trainees exposed to a high-variation training programme demonstrated a greater capacity to transfer in unseen transthoracic echocardiography conditions. Trainees emphasised the use of variation theory strategies in aiding transfer to describe abnormal hearts and the vital role of formative live feedback during training. These findings may help design training systems to facilitate transfer of training in echocardiography.
Acknowledgements
We are grateful to Ms. Ruth Brennan and Ms. Louise Finnegan who assisted with the interview transcriptions. We are grateful to the children and their parents who attended the boot camp live scanning sessions and also the test sessions for echocardiography.
Financial support
None.
Conflicts of interest
None.
Ethical standards
All procedures were in accordance with the Declaration of Helsinki. Ethical approval for this study was received from the Institutional Review Board at Children’s Health Ireland, Crumlin, Dublin, Ireland.
