Introduction
Capillary refill time (CRT) is a simple clinical finding that allows clinicians to assess patient perfusion.Reference Steiner, DeWalt and Byerley 1 , Reference Fleming, Gill and Jones 2 In mass-casualty disasters, several of the triage algorithms in either adult or pediatric patients rely on CRT: Simple Triage and Rapid Treatment (START), Smart Incident Command System, and Pediatric Triage Tape.Reference Cross and Cicero 3 - Reference Dieckmann, Brownstein and Gausche-Hill 6 Although the evidence for the validity of using CRT in this situation is inconclusive, a delayed CRT detected when using any of these algorithms ultimately can alter the triage category of a mass-casualty victim.
Most commercial simulators do not typically simulate CRT. Unlike in a typical mannequin-based simulation, a mass-casualty drill with many victims requires far more low-fidelity mannequins or human volunteers to be present.Reference Chan, Griswold and Buono 7 - Reference Alexander, Bandiera and Mazurik 9 Using all high-fidelity simulators to simulate CRT for a mass-casualty scenario would be cost-prohibitive, and forcing abnormal vital signs on human volunteers would be difficult and unethical.
Previous tabletop studies outside of the mass-casualty disaster simulation have used videos of CRT as a simulation proxy.Reference Brabrand, Hosbond and Folkestad 10 While this would certainly have visual realism, it provides no functional realism. Functional realism is defined as the ability of the simulation to approximate the inputs and outputs of an action.Reference Curtis, DiazGranados and Feldman 11 For CRT, this means the feel and texture of skin, as well as the expected blanching and return to color that is proportionate to the pressure applied by the health care provider. This is a tactile feedback that is not provided by video-based CRT.
The first aim with this technological innovation was to develop a first-generation prototype of a low-cost CRT simulator, usable in the disaster simulation context. The second aim was to evaluate visual and functional realism of the CRT simulator compared to a video-based CRTReference Brabrand, Hosbond and Folkestad 10 using two separate populations of health care providers.
Methods
Description
A light-emitting diode (LED)-based CRT simulator was developed using a single LED light that transitioned between red and white. The LED was connected to a flat, pressure-sensitive piezoelectric crystal-based receptor, which were both connected to a controller. The LED and the receptor were both encased in a spongy resin molded into a puck-like shape whose texture simulated thick skin, similar to an adult sternum. When pressure was applied through the resin, the piezo receptor detected the pressure and the LED light changed from red to white, proportionate to the amount of pressure applied. When pressure was relieved, the LED gradually transitioned back to its normal red color. The timespan within which the color returned was determined by a simple control dial that could be set from one to 10 seconds. Figure 1, Figure 2, and Figure 3 show the CRT simulator (Supplementary File – Video; available online only). Although clinical CRT is assessed most often on fingertips or earlobes, the shape of the flat, disc-like piezo receptor precluded molding the resin into a finger-shaped cylinder or a helix. For the purpose of prototype development, the simulator was being validated in the context of a mass-casualty drill and not necessarily for only pediatric or only adult victims.
Capillary Refill Time Simulator (Entirety).
Capillary Refill Time Simulator (Close-up).
Capillary Refill Time Simulator (Pressed).
In parallel to the CRT simulator development, seven videos were recorded on an iPod Touch (2nd generation; Apple; Cupertino, California USA) of various CRTs being performed on an index finger with the palmar surface facing the camera. Several sets of videos were taken with three CRT settings: normal (1.5 seconds), mildly delayed (4.0 seconds), and delayed (8.0 seconds). The mildly delayed CRT was simulated using a phlebotomy rubber tourniquet around the base of the finger. The tourniquet was not visible on the screen. The delayed CRT was simulated by performing a double Allen test - forceful compression of both the radial and ulnar arteries of the hand model. The two videos with poorest agreement were discarded. This resulted in two normal CRT videos less than two seconds, one mildly delayed CRT video, and two delayed CRT videos. For the study, these videos were selected by the investigator and the study participant was allowed to tap the screen, which played the video. Each video portrayed two separate compressions of the finger, followed by return of perfusion, and the video ran up to eight seconds before stopping. The video-based CRT was matched with CRT simulator times set at 1.5 seconds, 4.0 seconds, and 8.0 seconds on the dial for videos portraying normal, mildly delayed, and delayed CRTs, respectively.
Populations and Settings
The pilot testing was divided into two separate phases. Data were collected from two separate cohorts of health care providers and Institutional Review Board approval from both participating institutions was obtained.
The first phase established face validity through a tabletop exercise using board-certified or board-eligible physicians in emergency medicine or pediatric emergency medicine. Each participant was block randomized to encounter either the CRT simulator first then the iPod videos, or vice versa. After orienting to the device, the participants were asked to judge whether a simulated CRT was normal or abnormal; there was no particular time limit to these exercises. The order of the CRT portrayals was fixed (normal - delayed - mildly delayed - normal - delayed) for all devices and participants. Immediately following the five CRT portrayals, participants rated the physical and functional fidelity of the device using a 10 cm Visual Analog Scale (VAS), ranging from “completely artificial” to “as if on a real patient.” Participants repeated the five CRT portrayals and VAS on the next device without access to previous answers. All parts of this phase were done in a private room without any assistance or distractions, in ambient office lighting with no windows.
In the second phase, the CRT simulator was embedded within the larger study that examined the use of moulaged adult actors and computerized scenarios portraying pediatric victims.Reference Claudius, Kaji and Santillanes 12 Second year medical students at a single medical school were recruited for this phase. They received a 15-minute didactic overview of the JumpSTART algorithm and were block randomized to practice their triage skills on 11 disaster cases: four adult actors portraying disaster victims followed by a series of seven computerized scenarios, or vice versa. The CRT simulator was placed on the chest of a moulaged actor for a triage level of Immediate (Red). Participants experienced additional laptop computer scenarios with pediatric victims, and were required to interact with the virtual victim using written command buttons (eg, “perform jaw thrust”). One of these buttons was to check CRT, which played a video depicting a CRT reflective of the victim’s physical state. These were identical videos from the first phase. All participants were asked to provide feedback on the two CRT devices on a VAS. All parts of this study were done in private rooms without any assistance. Participants in both phases received a US $50 gift card as compensation.
Outcome Variables
The main outcome variable for the first phase was CRT accuracy. This was determined by whether or not the physician correctly identified the CRT as normal or abnormal. The “mildly delayed” CRT was categorized as abnormal for this study. Accuracy was reported as a percentage correct per device. In addition, perceived fidelity was quantified using an anchored VAS under two constructs: visual realism and functional realism; VAS had previously been used to quantify the sense of “realism” in other medical simulation studies using virtual simulation.Reference Devitt, Kurrek, Cohen and Cleave-Hogg 13 , Reference Hung, Zehnder and Patil 14 Visual realism is defined purely as the realism of the appearance of either the simulator or the video as compared to real life CRT. Functional realism is defined as the realism of the outputs in response to inputs.Reference Curtis, DiazGranados and Feldman 11 , Reference Dieckmann, Gaba and Rall 15 In the context of the CRT simulation, functional realism refers specifically to the tactile “give and feel” response that is felt by the provider, as well as the smoothness of the “blanching” and “return to color” as is expected by the provider (eg, Is the response from the simulator or video what is expected when you interact with it?). Functional realism excluded the notion of the provider’s ability to interpret the CRT itself, because medical students would not be expected to judge or provide inferences on CRT in a clinical context.
Sample Size Calculation and Data Analyses
Sample size calculation was performed using G*Power 3.1.Reference Faul, Erdfelder, Lang and Buchner 16 A power of 0.80, an alpha at 0.05, and a predicted 20 mm (SD=30 mm) difference in VAS yielded a sample size per phase of 27. Descriptive statistics were used to characterize accuracy and VAS for physical and functional realism of the two simulators. Light’s method to determine a mean Cohen’s Kappa values for every pair of phase one physicians with the five simulated CRTs and five video-based CRTs was used.Reference Hallgren 17 , Reference Light 18 The mean Kappa statistics were compared using a paired t-test. A Hills-Armitage approach was used to examine for carryover effects caused by the sequence of devices presented to the participants.Reference Diaz-Uriarte 19 All data were analyzed using SPSS version 19 (IBM Corporation; Armonk, New York USA).
Results
Twenty-seven physicians participated in the first phase, and the results are shown in Table 1. The simulator had significantly higher accuracy scores but poorer visual realism than the video version. The functional realism among this cohort showed no significant difference (P=.06; 95% CI difference, -1.5 to 36). The mean Kappa for the CRT simulator among the 27 physicians was 0.78 (SD=0.25) and for the CRT videos was 0.27 (SD=0.59; 95% CI difference, 0.43 to 0.57; P<.001). The greatest disagreements were found with the mildly delayed CRT in both the simulator (70% agreement) and the video CRT (37% agreement) compared to normal (simulator 100% agreement, video 83% agreement) and delayed CRT (simulator 96% agreement, video 76% agreement).
Physician Responses to CRT Simulator and Video-Based CRT
Abbreviation: CRT, capillary refill time.
a A CRT reading was defined as accurate if a normal reading was made on a normal CRT depicted in either modality, or if a delayed reading was made on a delayed CRT depicted in either modality.
During the second phase, second year medical students were recruited (n=36; Table 2). Twenty-eight (77.8%) used the CRT videos in the computerized scenarios and 31 (86.1%) used the CRT simulator on the live actors. Data were unimodal and parametric for this cohort (mean P>.236), but again, the Hills-Armitage approach did not reveal any carryover effects (P>.17). There was no significant difference appreciated for visual realism but a significantly higher functional realism for the CRT simulator (95% CI difference, 2.5 to 23; P=.01).
Medical Student Responses to In-Drill CRT Simulator and Video-Based CRT
Abbreviation: CRT, capillary refill time.
Discussion
The development and feasibility of a novel, portable model of a CRT simulator using a single LED light was demonstrated. The development of a portable, low-cost CRT simulator prototype was designed to maximize functional realism over visual realism, as the former is noted to be more important in the concept of fidelity as a simulation-based learning environment.Reference Curtis, DiazGranados and Feldman 11 , Reference Dieckmann, Gaba and Rall 15 , Reference Norman, Dore and Grierson 20 The data suggest a marginal advantage in functional realism over the video-based CRT simulation, which also has potential in a mass-casualty simulation.
Development of the CRT simulator required consulting with materials engineers and seed funding. Clinical experts decided a priori to develop a simulator with three key features: portability (and therefore lower cost than a full mannequin), realistic touch and feel, and the ability to alter the length of a smooth transition from a blanched white to a pink color. The engineers (JC and CB) were experienced with LED technologies and materials engineering, and used niche expertise for the simulator development. Other methods of CRT simulation, including hydraulic or magnetic principles, were considered, but the development and pilot testing team did not have sufficient experience to enter those engineering realms. Because the context was mass-casualty disaster simulation with live volunteer victims, the simulator needed to be “placed” on a real person.
Additional methods to simulate CRT prolongation in the disaster context include tourniquets and videos. Like Brabrand et al., the protocol used a video-based CRT using the author’s finger (TPC) under various strangulating circumstances.Reference Brabrand, Hosbond and Folkestad 10 The use of a tourniquet to simulate abnormal CRT has not been described in the health care simulation literature, and may be a simple strategy when using volunteers to simulate a mass-casualty victim. The tourniquet is extremely cheap and economical, and the prolonged CRT would have the highest functional realism – because it is the real thing. Tourniquets have their own disadvantages – prolonged use in a search-and-rescue scenario may be uncomfortable or cause ischemic injury, the tourniqueted body part may not be where the provider checks for the CRT (eg, wrong hand), and the tourniquet method is not possible on a mannequin-based simulator. Tourniquets are not necessarily precise in their CRT prolongations than either the video or the CRT simulator prototype, which could be standardized.
The other simulation that was tested was the iPod video version. This took only a few minutes to develop, and the videos were replicable to multiple devices. Using a touchscreen to start the video replicated the movement needed to elicit a CRT on a real patient, and this may have led to no significant difference in functional realism in the tabletop exercise. In the second cohort part of the pilot testing, the participants interacted with the video with a keyboard and mouse, which was a significant impediment to functional realism as shown with these data. Placing mobile touchscreen devices with videos is a potential solution to demonstrate CRTs in a mass-casualty simulation. The major advantage is standardization of the actual CRT across any number of scenarios; additionally, smartphones and other video-enabled devices using touchscreens are readily available and would be an economical choice. Several disadvantages are also present with video on touchscreen devices - one was that the backlighting of the device was yet another variable with ambient lighting that may have affected clinician reading. Second, touchscreens on smartphones and tablets are sensitized for a light fingertap, not necessarily a forceful and deep push that is expected for a human CRT. The lengthy nature of these simulations can send touchscreen devices into the lock screen or sleep mode in the middle of the scenario. In addition, should the patient’s state improve, changing the initial CRT video to a second one to simulate improved CRT is more difficult than dialing the CRT simulator or removing the tourniquet.
Even though the appearance of CRT was more realistic with video, experienced physicians had significant disagreement on whether a video-based CRT was normal or abnormal. This is consistent with even live CRT assessments in the literature, in which there is variability in inter-rater Cohen’s Kappas from 0.01 to 0.65.Reference Brabrand, Hosbond and Folkestad 10 , Reference Anderson, Kelly, Kerr, Clooney and Jolley 21 - Reference Gorelick, Shaw and Baker 23 One explanation of a higher inter-rater reliability using the CRT simulator is that it oversimplifies the difficulty in seeing CRT in a real patient. There is disagreement in how simulations like mass-casualty scenarios should be depicted – to be overly simplified to allow for better recognition, or to provide maximum confusion as in real life to psychologically stress the participant.Reference Dror, Schmidt and O’Connor 24 Based on these inter-rater reliability data, the CRT simulator appears to oversimplify the CRT recognition process. This may be helpful for novice learners who are learning the triage process and need experience to go through the process; this may not be as helpful for experienced providers who need practice on more nuanced exam findings.
In addition, physicians who encountered the simulator immediately after the iPod tended to score the functional realism of the simulator higher. This was a carryover effect; once the physicians experienced the functional limitations of the iPod, they then rated the functional realism of the simulator much higher. This is in comparison to the cohort who had to complete a VAS on the simulator without having experienced a comparison. None of the participants had encountered a CRT simulator prior to enrollment.
Limitations
There are several limitations to an unbiased testing of the CRT simulator. The decision to use pre-clinical medical students instead of experienced physicians in the second phase was a logistic decision. The concept of CRT is not a difficult one, and even experienced providers can disagree with CRTs;Reference Brabrand, Hosbond and Folkestad 10 , Reference Gorelick, Shaw and Baker 23 however, using pre-clinical students requires caution in generalizing the findings to experienced providers reacting to the CRT simulator in a disaster drill context. For purposes of this research, the influence of CRT on the students’ triage accuracies was not measured, as their inexperience in interpreting CRTs would have biased the data. The number of attempts on the CRT simulator and videos were not recorded; it is possible that increased attempts signal a level of uncertainty; however, many participants simply enjoyed the response of the CRT and continued pressing the simulator.
During the design of the testing process and the videos, the authors who developed the pilot protocol and the videos were not blinded to the development of the CRT simulator; although there were steps to standardize the video development as described above, it is possible that there is bias in the video development that could skew some scores in favor of the CRT simulator.
The luminescence of the LED makes night-time disaster drills problematic, though most disaster drills currently are during daylight hours for logistical purposes.
Conclusion
The production of a LED-based CRT simulator is described for the use in mass-casualty disaster scenarios, which is found to be visually less realistic than a video version, but with sufficient functional realism. The advantages and disadvantages of the CRT simulator, as well as video-based CRT, and the use of tourniquets is presented. Further development should improve the visual realism of the simulator and focus on its validity in disaster triage algorithms that require its use, as well as in clinical scenarios relevant to CRT, such as shock.
Acknowledgments
The authors would like to acknowledge the assistance and participation of Ms. Leighanne Johnson, who assisted in the study protocol.
Author Contributions
TPC, GS, and IC conceived the study idea and secured funding. CB and JC developed the LED-based capillary refill time simulator. TPC, GS, IC, PKP, JK, MGH, AHK, SS, and JJD developed and executed the study protocol and acquired all data for this study. TPC and PKP planned a priori data analysis and performed data analysis. TPC, GS, IC, and PKP drafted the manuscript and all authors provided revisions to the final product and paper.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1049023X17006343
