Introduction
A mass-casualty incident (MCI) typically occurs when the demand for assistance overwhelms the available resources. The goal of health professionals is to do the greatest good for the greatest number of people. Reference Cicero, Overly and Brown1,Reference Jones, White and Tofil2 For this, there are triage protocols that provide guidance on the priorities for attendance. So, patients are triaged based on the seriousness of their condition or injury and the probability of survival. Reference Cicero, Overly and Brown1 These protocols not only improve the priority of care but also minimize the emotional and stressful potential dilemmas of the scenario. Reference Cicero, Overly and Brown1,Reference Wallis and Carley3
In fact, such emotional interferences manifest themselves mainly in incidents involving the pediatric population. Besides the impulse to classify children with higher priority than would actually be appropriate to their situation due to a greater emotional engagement of rescuers with children, especially the younger ones, the child population also differs physiologically from adults. Reference Wallis and Carley3 These include physiological vulnerability to a variety of pathogens, toxins, dehydration, radioactive isotopes, and harsh conditions. Reference Dziuban, Peacock and Frogel4,Reference Lyle, Thompson and Graham5
Also, there are some anatomical differences as thinner skin, floppy and hard-to-maintain airway, higher baseline pulse and respiratory rates, more pliable skeletons, internal organs in closer proximity and not as fixed, and interventions are weight-based. Reference Dziuban, Peacock and Frogel4,Reference Lyle, Thompson and Graham5 Furthermore, children used to developmentally rely on adults, have poor hygiene habits, and their baseline mental status may not include following commands. Reference Lyle, Thompson and Graham5
Disaster medicine is for this reason still a developing field and even less is established in pediatric disaster medicine. Reference Cicero, Overly and Brown1 Therefore, it is necessary to have specific triage protocols for the pediatric population. The goal of triage is to quickly identify the most critically injured patients at the scene and to provide rapid treatment and transfer to definitive care, as needed. Reference Shea, Wei and Donovan6 Some of the most known protocols are Simple Triage and Rapid Treatment (START); JumpSTART (a START adapted version for children up to eight years old); Sort, Assess, Life-Saving Interventions, Treat/Transport (SALT); Pediatric Triage Tape (PTT); and CareFlight. Reference Wallis and Carley3 Among them, one of the most popular is JumpSTART, Reference Cicero, Overly and Brown1,Reference Nadeau and Cicero7 the pediatric version of START, which was developed in 1995 and modified in 2001. Reference Cone and MacMillan8 JumpSTART takes into account the unique aspects of pediatric physiology that theoretically should result in more accurate triage assignments in children. Reference Jones, White and Tofil2 Based on a color-coded triage like START, the algorithm uses clinical aspects (ambulatory status, airway, breathing, circulation, and neurologic status) to categorize the victims as “Minor,” “Expectant,” “Immediate,” and “Delayed.”
Regarding the scarcity of studies attesting to the validity of this tool, as the lack of standardization about which one of the mentioned algorithms is the most reliable, this review sought to compare the performance aspects of applicability - accuracy, speed of use, preferences, and satisfaction - of the JumpSTART triage protocol with different triage systems.
Methods
This systematic review was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) recommendations and registered with the PROSPERO database of systematic reviews with the number CRD42021258415. Reference Page, McKenzie and Bossuyt9 The systematic review question was: “Does JumpSTART have the best performance in a disaster setting with pediatric patients compared to other triage protocols?”
Eligibility Criteria
Studies that compared the utilization (accuracy, efficacy, satisfaction, preferences, speed, and ease of use) of JumpSTART triage protocol with different triage protocols used in the pediatric population (considering the maximum age <18 years) were considered as inclusion criteria. Moreover, only randomized clinical trials (RCTs), quasi-experimental (non-randomized trials), cohort, and analytical cross-sectional studies were considered.
The inclusion criteria also included the peer-reviewed papers in English, Portuguese, Spanish, and Italian languages that were utterly available (full-text) in the databases used for this research. Books, non-academic research, reviews, guidelines, case reports, and content in languages other than the presented ones were represented as exclusion criteria for the retrieved articles from the databases. Studies that did not address JumpSTART, did not focus on pediatric range, and/or did not compare triage protocols were also excluded.
Sources of Information
Five authors (VS, GM, MM, HZ, and MC) conducted the first search on April 29, 2021. The last update was performed on August 12, 2021 by the same authors. The studies were identified in five databases: PubMed (National Center for Biotechnology Information, National Institutes of Health; Bethesda, Maryland USA); Scopus (Elsevier; Amsterdam, Netherlands); MEDLINE Complete (US National Library of Medicine, National Institutes of Health; Bethesda, Maryland USA); Web of Science (all databases included; Thomson Reuters; New York, New York USA); and CINAHL (EBSCO Information Services; Ipswich, Massachusetts USA).
Literature Search Strategy
It was established a research plan embracing the research questions of interest, the keywords, and inclusion and exclusion criteria. For the documents search, the descriptors selected were based on the Medical Subject Headings (MeSH). The keywords used by the authors in all of the databases mentioned above are presented in Table 1. The electronic search was complemented with a manual search through the tracking of citations.
Literature Search Strategy
Abbreviation: START, Simple Triage and Rapid Treatment.
Study Selection and Data Extraction
The authors checked for duplicate documents using the tool “find duplicates” from the software EndNote X9 for Windows (Clarivate Analytics; Philadelphia, Pennsylvania USA), eliminating 24 documents. Besides, another 12 documents not excluded by the tool were manually deleted, leaving 46 articles. The researchers excluded one article that could not be found as PDF files, remaining 45 articles to be analyzed. Then, the five authors (VS, GM, MM, HZ, and MC) independently read the titles and abstracts to ensure that they covered the eligibility criteria, reducing the number of documents to five. Finally, the resulting primary studies that fulfilled the inclusion criteria were analyzed.
All the five authors independently read the six retrieved articles and extracted their data. The relevant data regarding the research aim were entered into a spreadsheet responsible for organizing and systematizing relevant information in columns using the Microsoft Excel spreadsheet Version 7.0.25 (Microsoft Corporation; Redmond, Washington USA) for Windows.
Quality Appraisal
Five review authors (VS, GM, MM, HZ, and MC) independently assessed all included studies for risk of bias with an inter-rater agreement of 81.61%; any disagreement was resolved by discussion. The authors used the Joanna Briggs Institute (JBI) Critical Appraisal Checklists for RCTs, Cohort Study, or Analytical Cross-Sectional Study to independently assess the quality of each retrieved article. 10 The checklists mentioned above have four possible answers: Yes (Y), No (N), Unclear (U), and Not Applicable (NA); the assessment of the bias risk is calculated by the amount of “Ys” that have been answered by all the authors on the checklist. According to the JBI guidelines, when “NA” was selected, this respective question was not considered in the calculation. 10 Based on the risk of bias assessment, each study may be classified as high risk of bias (<49%), moderate risk of bias (from 50% to 70%), and low risk of bias (>70%). Reference Azevedo, Ledesma, Pereira, Oliveira and Bahmad Junior11
Synthesis of Results
The authors performed a narrative synthesis for presenting the studies’ results. The data tabulated on the spreadsheet were used to write up the findings and to discuss the main information. A discussion on the impact of the methodological quality of the included studies was carried out.
Then, the authors established the understanding of the main topics regarding the applicability of the JumpSTART triage protocol compared with different triage systems, grouping them into three topics, which are detailed in Table 2.
Definitions of Applicability Topics
The grouping of the articles in these topics was based on the explicit mention of these terms as one of the main focuses of the analysis of each study. In addition, the characteristics of “sensitivity,” “specificity,” “over-triage,” and “under-triage” were considered as components of the accuracy, and therefore were grouped in the same topic. Only significant results - that is, in line with the systematic review objectives - for each topic were used in the narrative synthesis. In case of uncertainty about the relevancy of the results, a discussion between the authors was conducted. Secondary variables analyzed in this study are presented in Table 3.
Definitions of the Secondary Variables
Abbreviations: START, Simple Triage and Rapid Treatment; SALT, Sort, Assess, Life-Saving Interventions, Treatment/Transport.
For descriptive analysis, the data were also tabulated in spreadsheets using Microsoft Excel Version 7.0.25 (Microsoft Corporation; Redmond, Washington USA) for Windows and the relative frequency of responses were analyzed. For study citation, the Mendeley Reference Manager version 1.19.5 for Windows (Elsevier; London, England) was used.
Results
Overall, the search identified 82 references with 46 unique papers. Of these references, 39 were excluded as they did not meet the inclusion criteria or were not available in full-text. A total of six primary studies were retrieved from the database search. No study was included by the tracking of citations. Therefore, six studies were identified as meeting all the criteria, as shown in Figure 1. Table 4 presents the characteristics of the included studies. The analysis of the results is shown below, presenting insights into the study of this review’s subject.
PRISMA 2020 Flow Diagram for New Systematic Reviews Which Included Searches of Databases, Registers, and Other Sources.
Abbreviation: PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.
Characteristics of the Included Studies
Abbreviations: START, Simple Triage and Rapid Treatment; SALT, Sort, Assess, Life-Saving Interventions, Treatment/Transport.
Study Selection
All six articles (five from the databases + one from the tracking of citations) were analyzed using the JBI Critical Appraisal Checklists according to each type of study. Three articles showed a low risk of bias, Reference Cicero, Overly and Brown1,Reference Nadeau and Cicero7,Reference Price, Brace-Mcdonnell, Stallard, Bleetman, Maconochie and Perkins12 two articles showed a moderate risk of bias, Reference Jones, White and Tofil2,Reference Heffernan, Lerner and McKee13 and one article showed a high risk of bias. Reference Wallis and Carley3 Further information regarding the critical appraisal tools used in this systematic review can be consulted in the supplementary material (available online only).
The authors identified a limited number of publications that attempted to compare specific triage systems. There were few studies, but they are an important initial step in evaluating the existing systems.
Accuracy
A retrospective cohort study Reference Price, Brace-Mcdonnell, Stallard, Bleetman, Maconochie and Perkins12 with a bias risk of 88.89% stated that the accuracy performance of JumpSTART against the patient survival (dead or alive) outcome revealed a sensitivity of 86.3% (83.1%-89.5%) and a specificity of 84.8% (84.2%-85.5%). Concerning to the sensitivity, JumpSTART was just worse than CareFlight - sensitivity of 95.0% (93.0%-97.0%); in this regard, Triage Sort and PTT demonstrate a sensitivity of, respectively, 60.2% (59.3%-61.1%) and 25.0% (20.7%-29.3%). Reference Price, Brace-Mcdonnell, Stallard, Bleetman, Maconochie and Perkins12 Furthermore, the specificity of JumpSTART was just worse than PTT - specificity of 88.4% (84.2%-89.0%); right behind JumpSTART is CareFlight, 78.9% (78.2%-79.6%), and Triage Sort, 60.2% (59.3%-61.1%).
At the same time, the triage protocols’ accuracies were compared using the Injury Severity Score (ISS; a tool that assesses trauma severity ranging from one to 75) ≤15 or >15 outcomes (delayed and immediate, respectively), taking into account that a patient with a score higher than 15 is considered at least serious. Reference Price, Brace-Mcdonnell, Stallard, Bleetman, Maconochie and Perkins12 With this parameter, JumpSTART showed a sensitivity of 43.9% (42.1%-45.8%) and a specificity of 89.8% (89.3%-90.4%). Thus, the sensitivity of this triage algorithm was only better than the PTT, 17.6% (16.2%-19.0%) and worse than all the others: Triage Sort, 76.6 (75.1%-78.2%), and CareFlight, 65.4 (63.6%-67.2%).
Also, the specificity of JumpSTART was the best in this scenario, superior to PTT, 89.5% (88.9%-90.1%); CareFlight, 88.1% (87.5%-88.7%); and Triage Sort, 68.0% (67.1%-69.0%). None of the analyzed triage tools (CareFlight, JumpSTART, PTT, and Triage Sort) performed well when the parameters were the sensitivity and the specificity against the ISS parameter. Reference Price, Brace-Mcdonnell, Stallard, Bleetman, Maconochie and Perkins12 On the other hand, considering their accuracy against the “Survival” parameter, it was observed that the acting of the triage algorithms was better, suggesting a tendency to correctly screen patients who are deliberately injured, falling short of assessing victims with serious injuries that are a little more veiled. Reference Price, Brace-Mcdonnell, Stallard, Bleetman, Maconochie and Perkins12
Parsing JumpSTART, Smart, and clinical decision-making (CDM) with no algorithm, one analytical cross-sectional study with an 87.50% bias risk found that there was no significant difference between Smart triage and CDM, Reference Cicero, Overly and Brown1 but when JumpSTART triage was used, there was greater accuracy than with either Smart (P <.001; OR = 2.03; IQR: 1.30, 3.17) or CDM (P = .02; OR = 1.76; IQR: 1.10, 2.82). JumpSTART outperformed Smart for RED patients (P = .05; OR = 1.48; IQR: 1.01,2.17), whereas no difference was observed between Smart and CDM or JumpSTART and CDM. JumpSTART outperformed both Smart (P <.001; OR = 3.22; IQR: 1.78,5.88) and CDM (P <.001; OR = 2.86; IQR: 1.53,5.26) for YELLOW patients. Reference Cicero, Overly and Brown1 Furthermore, JumpSTART outperformed CDM for BLACK patients (P = .01; OR = 5.55; IQR: 1.47, 20.0). For ambulatory or GREEN patients, there were no significant differences among the Pediatric Disaster Triage (PDT) strategies. Reference Cicero, Overly and Brown1
In a prospective observational study Reference Heffernan, Lerner and McKee13 with a 50.00% bias risk, four different triage systems (JumpSTART, SALT, Triage Sieve, and CareFlight) were compared in terms of accuracy when used in patients less than 18 years old presented to a pediatric specialty hospital/Level I trauma center. This recent American study with 115 patients demonstrated that none of the four systems were extremely accurate and all of them showed a high rate of under-triage. Reference Heffernan, Lerner and McKee13 In this study, the SALT system presented the highest accuracy rates (59%; 95% CI, 50-68) and the lowest under-triage rate (33%; 95% CI, 24- 42), but the difference of three percent in the accuracy showed a minimal clinical significance, and there was overlap in all of the confidence intervals. Reference Heffernan, Lerner and McKee13 JumpSTART had an accuracy of 57% (95% CI, 48-66) and the lowest over-triage rate (4%; 95% CI, 1-8) compared with the other triage systems. Besides that, the most frequent error was triaging a patient as “minimal” when it should have been classified as “delayed.” Reference Heffernan, Lerner and McKee13
A RCT with a bias risk of 66.67% reported no statistical difference between JumpSTART and SALT accuracy. Reference Jones, White and Tofil2 Both protocols had equivalent over-triage and under-triage rates. The reasons provided for inaccurate triage were mostly due to cognitive errors in both protocols. For JumpSTART, affective errors were the second leading type, while for SALT, they were technical errors. Reference Jones, White and Tofil2
In a prospective cohort study Reference Wallis and Carley3 with a 45.45% bias risk, some triage algorithms (PTT, CareFlight, JumpSTART, and START) were compared with the gold standard of ISS, New ISS (NISS), and modified Garner criteria (MGC). Regarding sensitivity, JumpSTART and START had very low rates (3.2% [95% CI, 1.3%-7.5%] and 31.3% [95% CI, 21.5%-42.8%] for the ISS; 2.4% [95% CI, 1.0%-5.0%] and 22.3% [95% CI, 15.6%-30.7%] for the NISS; and 0.8% [95% CI, 0.1%-4.1%] and 39.2% [95% CI, 29.3%-50.0%] for the MGC, respectively), meaning they could not identify patients with serious injuries as well as victims seriously injured in events and major incidents. Reference Wallis and Carley3
In relation to the specificity, JumpSTART performed better than START (97.8% [95% CI, 97.7%-98%] versus 77.9% [95% CI, 77.3%-78.7%] for the ISS; 97.8% [95% CI, 97.6%-98%] versus 77.3% [95% CI, 77.6%-78.3%] for the NISS; 97.7% [95% CI, 97.6%-97.8%] versus 78.7% [95% CI, 77.9%-79.5%] for the MGC, respectively). Reference Wallis and Carley3 In this primary study, CareFlight performed better in terms of sensitivity and specificity (48.4% [95% CI, 43.4%-52.8%] and 98.8% [95% CI, 98.6%-99.1%] for the ISS; 31.5% [95% CI, 28.5%-34.1%] and 99.0% [95% CI, 98.7%-99.3%] for the NISS; and 46.0% [95% CI, 41.2%-50.2%] and 98.9% [95% CI, 98.6%-99.1%] for the MGC, respectively), and PTT performed very similarly (37.8% [95% CI, 32.7%-42.5%] and 98.6% [95% CI, 98.3%-98.8%] for the ISS; 26.1% [95% CI, 23.0%-28.8%] and 98.9% [95% CI, 98.5%-99.1%] for the NISS; and 41.5% [95% CI, 36.8%-45.6%] and 98.9% [95% CI, 98.6%-99.2%] for the MGC, respectively). Reference Wallis and Carley3 Although this paper recommends the use of CareFlight or PTT against JumpSTART/START in major pediatric incidents, Reference Wallis and Carley3 it is important to point out that this primary study presented a high risk of bias.
Speed of Use
A randomized trial compared the speed of applicability of JumpSTART against SALT protocols in a simulated pediatric MCI. Reference Jones, White and Tofil2 By measuring the time taken by paramedics to perform in an apartment building collapse scenario, JumpSTART demonstrated a faster performance rate than SALT. Reference Jones, White and Tofil2 In the study, JumpSTART averaged 26 seconds of application speed, whereas SALT took 34 seconds (P = .02). Reference Jones, White and Tofil2
Preferences and Satisfaction
A cross-sectional study with a low bias risk determined which PDT systems are used in US states/territories and whether there is standardization to their use - JumpSTART, SALT, SMART, Sacco triage method, CareFlight, Triage Sieve, and Clinical judgment. Reference Nadeau and Cicero7 This study showed that JumpSTART is the most commonly used and frequently preferred PDT strategy, but most states use more than one system. Reference Nadeau and Cicero7 JumpSTART is predominantly used and is preferred by most respondents. Fifty-six percent (27/48) of respondents reported having a preference concerning which PDT system they would prefer to use. Seventy-one percent of these respondents with a preference (19/27) identified JumpSTART as their preferred PDT strategy. Reference Nadeau and Cicero7
The same article also sought to understand the relative level of satisfaction with each PDT system, the system(s) preferably among those who use them, the nature of the incidents for which PDT systems are being activated, and the number of patients requesting PDT activation. Reference Nadeau and Cicero7 As a found result, the JumpSTART was preferred by 71% of those stating a preference; it tied with Smart for median satisfaction level. Reference Nadeau and Cicero7
Discussion
Since there is a lack of standardization of triage algorithms for pediatric disaster victims, this review sought to analyze some performance parameters of these tools in order to compare the applicability of JumpSTART with other triage systems. In fact, there is a greater preference among the articles analyzed for the use of this protocol, demonstrating its popularity. However, the fact that it is widely used does not necessarily imply greater validity and reliability. Thus, it was observed that, although it has shown a greater speed of use when compared to SALT, Reference Jones, White and Tofil2 its accuracy in comparison with other protocols still leaves something to be desired. Reference Price, Brace-Mcdonnell, Stallard, Bleetman, Maconochie and Perkins12,Reference Heffernan, Lerner and McKee13
There was a plurality of results in the analyzed studies, demonstrating that the protocol in question can be superior or inferior depending on the evaluation criteria. In a single study, Reference Cicero, Overly and Brown1 the JumpSTART algorithm was shown to have the best accuracy when compared to CareFlight, PTT, and Triage Sort. However, other reviewed articles exhibited JumpSTART’s lower accuracy concerning SALT, Triage Sieve, CareFlight, PTT, and Triage Sort, Reference Wallis and Carley3,Reference Price, Brace-Mcdonnell, Stallard, Bleetman, Maconochie and Perkins12,Reference Heffernan, Lerner and McKee13 or at best expressed some equivalence when compared to SALT. Reference Jones, White and Tofil2
It should be pointed out that the real validity of a triage protocol must be established beyond the comparison among the existing ones. In one study, Reference Heffernan, Lerner and McKee13 all the triage algorithms (JumpSTART, SALT, Triage Sieve, and CareFlight) exceeded the suggested goals for field triage in terms of under-triage described by the American College of Surgeons (Chicago, Illinois USA) and the Centers for Disease Control and Prevention (Atlanta, Georgia USA). 14,15 This last finding agrees on what was exposed in the only RCT included in the review, Reference Jones, White and Tofil2 which demonstrated under-triage equivalence between JumpSTART and SALT. Likewise, concerning over-triage, JumpSTART was minimally superior to SALT, CareFlight, and Triage Sieve with a derisory margin of difference, while the RCT showed no statistical difference between JumpSTART and SALT.
Some studies showed potential confounding factors when gathering accuracy data, such as the applicator of the screening protocol’s characteristics (eg, years working as a paramedic, previous experiences with disasters, training with the tool used, and emotional bias), Reference Cicero, Overly and Brown1,Reference Jones, White and Tofil2 the scenarios where the algorithm is applied, Reference Cicero, Overly and Brown1 and unpredictable technical limitations of the training simulation. Reference Jones, White and Tofil2 In addition, it was pointed out that accidents with mass casualties can cause instincts to override the rationality of using screening protocols, Reference Jones, White and Tofil2 which is a factor to be recognized and corrected in training. In fact, it is known that there is a tendency to over-triage when victims are children due to greater distress from the rescuer. Reference Wallis and Carley3
Concerning the preferences and satisfaction results of screening algorithms, despite JumpSTART being the preferred screening tool in most US states, Reference Nadeau and Cicero7 this understanding cannot be extrapolated to the rest of the world as it is a very particular reality arising from a single survey among the reviewed articles. Even so, the same results indicate the lack of standardization regarding this matter, as well as the importance of looking for more evidence to support this normalization. Reference Lyle, Thompson and Graham5,Reference Nadeau and Cicero7,Reference Jenkins, McCarthy and Sauer16
With this scarcity of studies in mind, further research about the best tool is needed, as well to improve an already used device or perhaps to develop an entirely new one, Reference Lerner, Schwartz and Coule17 in order to reach a global gold standard protocol, such as it is recommended by multiple noteworthy multidisciplinary expert consensus committees. Reference Nadeau and Cicero7,Reference Jenkins, McCarthy and Sauer16
Limitations
The authors are aware that this study has some limitations. Only a few papers were retrieved for this review because it is still an unexplored subject in the current literature and the heterogeneity of the primary studies’ measures to present the outcomes made compiling the results more difficult. In addition, some retrieved papers (50%) showed a high or moderate risk of bias in the JBI checklist, which may interfere with the validity of the results presented in these primary studies. Lastly, because this paper is a systematic review, the analysis is restricted by each of the component studies’ limitations.
Conclusion
This review showed that there is insufficient evidence to support the validation of JumpSTART as a universal triage protocol given the heterogeneity of the results regarding accuracy and applicability in the analyzed studies. However, no other protocol has proven to be substantially superior to JumpSTART. Considering that, it is urgent to expand research in the area to optimize one of the protocols already in use or to develop a new triage method. It is expected that the standardization of triage procedures in the setting of a pediatric disaster will enable better management of resources as well as the reduction of emotional biases involved in this situation.
Conflicts of interest/funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. The authors report no conflict of interest.
Supplementary Materials
To view supplementary material for this article, please visit https://doi.org/10.1017/S1049023X22000127
