Introduction
Mass-casualty incidents (MCIs) are sudden surges of casualties exceeding immediately available clinical resources, requiring rapid allocation of scarce resources through a standardized sorting process—the operational task that MCI triage algorithms are designed to perform.
Numerous MCI triage algorithms have been developed across operational tiers, including Simple Triage and Rapid Treatment (START); JumpSTART; Sort, Assess, Life-saving interventions, Treatment/transport (SALT); CareFlight Triage; the Triage Sieve; and the Modified Physiological Triage Tool (MPTT), among others.
In diagnostic accuracy research, the reference standard classifies the true status of each case against which the index test is assessed. In MCI triage research, the same function applies, but the outcome is contested: there is no single agreed criterion for which patients should be classified “critical.” Across the literature, this criterion has been operationalized through life-saving intervention (LSI) lists, expert-panel consensus categories, anatomic severity thresholds (Injury Severity Score [ISS] >15), composite physiological scores, and outcome-based proxies such as mortality or intensive care unit (ICU) admission—constructs that do not measure the same thing.
The choice of reference standard is not merely terminological. Vassallo, et al demonstrated only a low-to-moderate correlation between ISS and LSI need (r = 0.334; P < 0.001), with 7.6% of patients receiving an LSI having an ISS below the major-trauma threshold of 15.Reference Vassallo, Fuller and Smith1 An anatomic-severity standard therefore codes as non-critical some patients who required urgent intervention. Garner, et al introduced resource-based criteria (operative procedures, fluid resuscitation, airway management, tension pneumothorax decompression, and invasive central nervous system monitoring) as a direct measure of patient need rather than anatomic burden.Reference Garner, Lee and Harrison2
The clinical significance of reference-standard choice is illustrated by Wallis and Carley, who applied four triage tools to 3,461 children against three parallel reference standards; the Pediatric Triage Tape’s sensitivity ranged from 26.1% to 41.5% depending solely on which reference standard was used.Reference Wallis and Carley3 Kilner and colleagues similarly observed that reference-standard variation across studies is large enough to determine whether a triage algorithm is considered clinically acceptable.Reference Kilner, Brace and Cooke4
The problem is compounded by incomplete reference-standard reporting. Franc and colleagues found that the majority of studies (n = 22 of 32) did not specify the reference standard used, and Marcussen and colleagues reported moderate-to-high risk-of-bias and called for a standardized protocol for future live simulations.Reference Franc, Kirkland and Wisnesky5,Reference Marcussen, Bräuner and Alstrøm6
No prior review has treated reference-standard constructs as the primary object of systematic extraction, classification, and reporting-completeness assessment. Prior systematic reviews—including Kilner, Marcussen, and Franc—documented reference-standard heterogeneity as a limitation, without taking it as the primary subject of inquiry.Reference Kilner, Brace and Cooke4–Reference Marcussen, Bräuner and Alstrøm6
This methodology scoping review followed the scoping review guidance of Munn, et al and Peters, et alReference Peters, Marnie and Tricco7,Reference Munn, Peters and Stern8 and addressed three objectives: (1) map and classify reference-standard constructs across the MCI triage validation literature using a pre-registered taxonomy; (2) assess reporting completeness using the Reference Standard Reporting Index for Mass-Casualty Incident triage (RSRI-MCI), an eight-item instrument developed a priori; and (3) identify reporting patterns that may inform methodological recommendations for future MCI triage validation research.
Methods
Design and Reporting Framework
The review followed the scoping review framework of Arksey and O’Malley as refined by Levac and colleaguesReference Levac, Colquhoun and O’Brien9 and is reported in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR).Reference Levac, Colquhoun and O’Brien9–Reference Arksey and O’Malley11 The review protocol was prospectively registered on the Open Science Framework (OSF) before screening commenced; registration DOI is https://doi.org/10.17605/OSF.IO/UNB74. Protocol deviations were documented in a dated deviation log.
Research Question
Three linked questions were addressed: (1) what reference-standard constructs are used to define the “critical casualty” across the MCI triage literature; (2) how completely are those constructs reported with respect to eight pre-specified items; and (3) what methodological recommendations follow for the next generation of MCI triage validation studies. The unit of analysis was the source (publication).
Eligibility Criteria
Sources were eligible if they described, derived, validated, or applied a reference standard for distinguishing critical from non-critical casualties in MCI triage at any operational tier, reported original research or methodological development, and were available as English full-text. Review articles not proposing an original reference-standard construct, records lacking an extractable definition, and non-English full-texts were excluded. No date restriction was applied. Verbatim eligibility criteria are in the Supplementary File.
Information Sources and Search Strategy
Four bibliographic databases were searched on May 5, 2026: MEDLINE via PubMed (National Center for Biotechnology Information, National Institutes of Health; Bethesda, Maryland USA); Scopus (Elsevier; Amsterdam, Netherlands); Web of Science (Core Collection; Clarivate Analytics; London, United Kingdom); and CINAHL via EBSCOhost (EBSCO Information Services; Ipswich, Massachusetts USA), each using two complementary Boolean strategies. Full search strings are in Supplementary File 1. Citation chasing, hand-searching, and gray-literature searching were not performed. Database yields are shown in the PRISMA-ScR flow diagram (Figure 1).
PRISMA-ScR Flow of Records through Scoping Review.
Note: Four bibliographic databases (MEDLINE via PubMed, Scopus, Web of Science, and CINAHL via EBSCOhost) were searched on May 5, 2026. Record-by-record exclusion reasons at the full-text stage are provided in Supplementary Table S2.
Selection Process
Records were managed in Rayyan (Rayyan Systems, Inc.; Cambridge, Massachusetts USA) for de-duplication and title-and-abstract screening. Full-text screening was performed in duplicate; disagreements were resolved by discussion. Exclusion details are in Supplementary Table S2.
Data Charting (Data Extraction)
A structured extraction form covered bibliographic metadata, study design, operational tier, sample size, index tests, verbatim reference-standard text, family classification, and the eight RSRI-MCI items. The form was piloted on five sources and frozen before full extraction. The primary reviewer extracted all fields for all included sources; the second reviewer independently re-extracted reference-standard family classification and RSRI-MCI coding for all sources. Funding sources of included studies were not charted because they were not considered directly relevant to the methodological mapping objectives of this review; this is acknowledged as a limitation.
Reference Standard Taxonomy
Reference standards were classified into one of five pre-specified families: LSI-Based, Composite/Hybrid, Expert Panel, Outcome/Resource, and Anatomic Severity. A pre-registered residual category was empty after full-text review (documented as a protocol deviation). Operational definitions and representative sources are presented in the Results; family assignments are in Supplementary Table S3 (Section A).
Reporting Completeness Assessment with RSRI-MCI
Formal risk-of-bias appraisal was not conducted, consistent with scoping review methodology. Reporting completeness was assessed using the RSRI-MCI, an eight-item instrument developed a priori for this review (no validated alternative exists). The eight items were: R1, explicit definition; R2, components listed; R3, time window specified; R4, rationale provided; R5, dead/expectant handling; R6, binary mapping described; R7, missing-data handling; and R8, limitations acknowledged. For R1, a citation to a published reference standard was accepted as explicit reporting. Full coding rules are in Supplementary Methods A.
Synthesis and Analysis
Synthesis was descriptive and narrative. Reference-standard family distribution, RSRI-MCI reporting frequencies, and inter-rater agreement were summarized using percentage agreement supplemented by Gwet’s Chance-Corrected Agreement Coefficient (AC1), with 95% bootstrap confidence intervals (1,000 resamples; seed 20260417); Krippendorff’s α was reported as a cross-check.Reference Gwet12,Reference Krippendorff13 Computational details are in Supplementary Methods A.
Ethical Approval
This methodology scoping review used only published literature and did not involve human participants, identifiable patient data, or biological samples. No ethics committee approval was required.
Results
Search and Selection
The search yielded 1,478 records; after de-duplication, 529 were screened at title and abstract, of which 76 advanced to full-text review. Sixteen were excluded at full-text stage (non-English, n = 4; no extractable reference standard, n = 2; and review without original reference-standard construct, n = 10), yielding 60 included sources (Figure 1).
Characteristics of Included Sources
Summary characteristics of the 60 included sources are reported in Table 1; per-source details (country, study design, population, operational triage tier, sample size with cohort identifier, index test(s), brief reference standard, and reference standard family) are provided in Supplementary Table S1; publication-period trends in reference-standard family distribution are summarized in Supplementary Table S5.
Summary Characteristics of the 60 Included Sources
Table 1. Long description
A table summarizing characteristics of 60 included sources, categorized by year of publication, geographic origin, study design, population, operational triage tier, and reference standard family. The table has 6 main categories and 27 rows. Row 1: Year of Publication, 2001-2010, 10, 17 percent. Row 2: Year of Publication, 2011-2015, 14, 23 percent. Row 3: Year of Publication, 2016-2020, 20, 33 percent. Row 4: Year of Publication, 2021-2025, 16, 27 percent. Row 5: Geographic Origin, United States, 27, 45 percent. Row 6: Geographic Origin, United Kingdom/South Africa-Linked, 19, 32 percent. Row 7: Geographic Origin, Continental Europe, 7, 12 percent. Row 8: Geographic Origin, Asia, 4, 7 percent. Row 9: Geographic Origin, Other (Middle East and South Asia), 3, 5 percent. Row 10: Study Design, Registry Simulation, 28, 47 percent. Row 11: Study Design, Drill or Live Simulation, 10, 17 percent. Row 12: Study Design, Consensus or Methods-Development, 9, 15 percent. Row 13: Study Design, Prospective Cohort, 8, 13 percent. Row 14: Study Design, Real MCI, 5, 8 percent. Row 15: Population, Adult Only, 34, 57 percent. Row 16: Population, Mixed Adult and Pediatric, 11, 18 percent. Row 17: Population, Pediatric Only, 15, 25 percent. Row 18: Operational Triage Tier, Prehospital Primary Triage, 46, 77 percent. Row 19: Operational Triage Tier, Secondary Triage, 7, 12 percent. Row 20: Operational Triage Tier, Primary Triage (Consensus/Methods-Development), 5, 8 percent. Row 21: Operational Triage Tier, Mixed Tier, 2, 3 percent. Row 22: Reference Standard Family, LSI-Based, 23, 38 percent. Row 23: Reference Standard Family, Composite/Hybrid, 18, 30 percent. Row 24: Reference Standard Family, Expert Panel, 9, 15 percent. Row 25: Reference Standard Family, Outcome/Resource, 7, 12 percent. Row 26: Reference Standard Family, Anatomic Severity, 3, 5 percent.
Note: Percentages may not sum to 100 due to rounding. Per-source characteristics are listed in Supplementary Table S1. Multi-country sources were assigned to the dominant methodological origin or author-network category used for geographic-origin summarization.
Abbreviations: MCI, mass-casualty incident; LSI, life-saving intervention.
Included sources spanned from 2001 through 2025 (median 2017), with 45% from the United States and 32% from the United Kingdom and South Africa-linked research network.
Registry simulation dominated (n = 28; 47%), followed by drill or live simulation (n = 10; 17%), consensus or methods-development papers (n = 9; 15%), prospective cohort (n = 8; 13%), and real MCI retrospective analyses (n = 5; 8%). The field rests on a small number of actual MCI datasets: five retrospective real-MCI cohortsReference Kahn, Schultz and Miller14–Reference Castro Delgado, Gan and Cabrera García18 and the Camp Bastion 2011 prospective military cohort, represented across three linked Vassallo publications.Reference Vassallo, Horne and Ball19–Reference Vassallo, Horne and Smith21
Linked publication clusters sharing the same patient cohort are flagged in Supplementary Table S1.
Reference-Standard Family Distribution
The most frequent reference-standard family was LSI-Based (n = 23; 38%), clustering around four methodological anchors: Garner 2001, Wallis 2006, Lerner 2015, and Vassallo 2016.Reference Garner, Lee and Harrison2,Reference Wallis, Carley and Hodgetts22–Reference Vassallo, Smith and Bruijns24 The second most common family was Composite/Hybrid (n = 18; 30%), in which two or more orthogonal components – typically an LSI list combined with mortality, anatomic severity, or a physiologic score – were merged into a single reference standard. Expert-Panel Standards (n = 9; 15%) were largely confined to simulation-based and drill-based studies. Outcome/Resource standards (n = 7; 12%) and Anatomic Severity Standards (n = 3; 5%) accounted for the remainder. Family-level operational definitions and representative sources are given in Table 2; the distribution is summarized graphically in Figure 2.
Distribution of Reference-Standard Families across 60 Included Sources.
Note: The LSI-based family accounted for 38% of included sources. Together with composite/hybrid frameworks, many of which incorporated intervention-based components, LSI-based and multi-component reference standards accounted for approximately two-thirds of the field.
Sensitivity Reclassification of Borderline Family Assignments
A sensitivity analysis reclassifying five borderline sources (Kahn 2009, Tiyawat 2024, Vernon 2024, Hupert 2007, and Castro Delgado 2022) changed no family proportion by more than five percentage points; the conclusion that LSI-Based and Composite/Hybrid standards account for approximately two-thirds of included sources was robust (Supplementary Table S3).
Reporting Completeness Assessed with RSRI-MCI
Table 3 reports RSRI-MCI item-level reporting frequencies and inter-rater agreement.Reference Landis and Koch25 The mean number of items coded Y was 5.6 of 8.0 (70%). Reporting was substantially more complete for basic descriptive elements (R1, definition, 98%; R2, components, 97%; and R4, rationale, 88%) than for operational implementation details. The lowest-frequency item was R3 (time window for outcome capture, 40%), followed by R5, R7, and R8 (57%-58% each) and R6 (binary mapping, 65%). Under a stricter convention that required in-text restatement of the reference standard rather than accepting a citation to a published standard, R1 reporting completeness was 83% rather than 98%.
Reference Standard Family Taxonomy with Operational Definitions and Representative Sources
Table 2. Long description
A table categorizing reference standards by family, operational definitions, and sources. The table has 6 rows and 3 columns. Column headers are Family, n (%), Operational Definition, and Representative Sources. Row 1: Family: LSI-Based, n (%): 23 (38%), Operational Definition: The reference standard is operationalized as the receipt of one or more LSIs from a defined list within a specified time window, Representative Sources: Garner 2001 (modified Baxt criteria); Wallis 2006 (pediatric Delphi); Vassallo 2016 (32-item adult civilian Delphi LSI list); Lerner 2015 (consensus-based gold standard); Malik 2021; Xu 2024. Row 2: Family: Composite/Hybrid, n (%): 18 (30%), Operational Definition: The reference standard combines two or more orthogonal components (typically LSI receipt plus mortality, transfusion, or anatomic severity) into a single critical/non-critical mapping, Representative Sources: Kahn 2009 (modified Baxt + delayed criteria); Wallis 2006 PTT (ISS >15 + NISS >15 + Garner-modified); Tiyawat 2024; Vernon 2024; Price 2016 (death + ISS >15). Row 3: Family: Expert Panel, n (%): 9 (15%), Operational Definition: The reference standard is the scenario-correct triage category assigned a priori by an expert panel or scenario designer, typically in simulation- or drill-based studies, Representative Sources: Cone 2009 (SALT pilot); Jones 2014; Cicero 2016; Wolf 2014 (ASAV); Heller 2023. Row 4: Family: Outcome/Resource, n (%): 7 (12%), Operational Definition: The reference standard is a clinical outcome or resource-use endpoint (in-hospital mortality, ICU admission, surgical intervention, length-of-stay, or composite outcome), Representative Sources: Sacco 2005; Bala 2012; Cross 2015; Muguruma 2019 (PPATS); Martin-Rodriguez 2019. Row 5: Family: Anatomic Severity, n (%): 3 (5%), Operational Definition: The reference standard is an anatomic injury severity index (ISS, NISS, or related score) above a defined cut-point, Representative Sources: Hupert 2007; Cheung 2013; Zhu 2022 (Braden + NISS >15).
Note: Family assignment was performed independently by two reviewers; disagreements were resolved by discussion. Per-source family assignment is given in Supplementary Tables S1 and S3.
Abbreviations: LSI, life-saving intervention; ISS, Injury Severity Score; NISS, New Injury Severity Score; PTT, Pediatric Triage Tape; PPATS, Pediatric Physiological and Anatomical Triage Score; SALT, Sort, Assess, Life-saving Interventions, Treatment/Transport; ASAV, Amberg-Schwandorf Algorithm for primary triage; ICU, intensive care unit.
Inter-Rater Agreement between Primary and Secondary Reviewer for Family Classification and the Eight RSRI-MCI Items Across 60 Included Sources
Table 3. Long description
A table with 9 rows and 6 columns. The columns are labeled Item, Reporting (Y/N), Reporting (%), Agreement (n/N), Agreement (%), and Interpretation. The rows are labeled with different items related to reference standards. Each row provides data on reporting and agreement percentages, along with an interpretation section that includes AC1 values and alpha values. The table captures the reporting and agreement details for each item, highlighting the completeness and inter-rater reliability for various aspects of reference standards.
Note: AC1 = Gwet’s AC1 (chance-corrected coefficient robust to prevalence imbalance) with 95% percentile bootstrap confidence interval; α = Krippendorff’s alpha (reported as a cross-check; sensitive to prevalence imbalance for highly skewed items such as R1). Landis-Koch (1977) interpretation: AC1 ≥ 0.81 = Almost Perfect; 0.61–0.80 = Substantial; 0.41–0.60 = Moderate. Overall agreement aggregates all nine constructs across all 60 sources (540 cells total).Reference Landis and Koch25
Reporting columns show primary-reviewer coding only; the Family row contains no Reporting column entries because family is a categorical assignment rather than a Y/N coded item. Agreement columns show concordance between primary and secondary reviewer coding. Source-level dual-reviewer coding is provided in Supplementary Table S3 (Section A); item-by-item disagreement patterns are described in Supplementary Table S3 (Section B).
Abbreviations: RSRI-MCI, Reference Standard Reporting Index for Mass-Casualty Incident triage research; RS, reference standard; Y, explicitly reported; N, not reported; U, unclear or not extractable (combined as not-Y for reporting column).
Inter-rater agreement was substantial (Table 3): family-level agreement 93% (56/60); and overall agreement 83% (450/540 cells).
Gwet’s AC1 was used as the primary chance-corrected agreement index; Krippendorff’s α was reported as a cross-check. Both can collapse under high prevalence asymmetry (eg, R1: α = 0.016 despite 83% raw agreement); AC1 was stable under this condition.
Family-level AC1 was 0.919 (95% CI, 0.837–0.980) and item-level AC1 ranged from 0.657 (R3) to 0.833 (R2). Confidence intervals were computed by non-parametric percentile bootstrap (1,000 resamples; seed 20260417). Source-level dual-reviewer coding are in Supplementary Table S3 (Section A).
Foundational Methodological Sources
Four methodological sources were repeatedly used as reference-standard anchors: Garner, et alReference Garner, Lee and Harrison2 (five-item resource-based criteria, six-hour surgical window); Wallis and CarleyReference Wallis, Carley and Hodgetts22 (pediatric Delphi-derived standard); Lerner, et alReference Lerner, McKee and Cady23 (five-category functional definitions); and Vassallo, et alReference Vassallo, Smith and Bruijns24 (32-item adult civilian Delphi LSI consensus list).
Discussion
Summary of Evidence
Although these four sources are frequently cited together, they do not form a single, evolving standard. Garner, Wallis, Lerner, and Vassallo each defined the critical casualty differently – through different component lists, time windows, category definitions, and rules for reducing those categories to a binary critical/non-critical outcome. A triage tool evaluated against one of these standards is therefore not necessarily measured against the same construct as a tool evaluated against another. Where any of these standards is applied outside the population, health system, or operational context in which it was developed, that extrapolation should be reported explicitly.
Interpretation: What Findings Mean for MCI Triage Research
The LSI-Based family dominated, but LSI lists differed in composition, counting rules, and time horizons. Garner, et alReference Garner, Lee and Harrison2 used five resource-based criteria with a six-hour surgical window; Lerner, et alReference Lerner, McKee and Cady23 provided functional definitions for five triage categories; Vassallo; et alReference Vassallo, Smith and Bruijns24 defined Priority 1 by receipt of one of 32 LSIs within one hour. These frameworks are conceptually related but operationally distinct, and the dominance of the LSI family masks this internal heterogeneity.
Anatomic Severity thresholds have documented limitations as MCI triage reference standards. The finding that only three of 60 sources used a purely anatomic reference standard suggests the field has largely converged away from purely anatomic constructs, with ISS-based comparisons retained only as secondary analyses.
The 40% overall reporting frequency for R3 highlights an important source of ambiguity, particularly among studies using Intervention-based or Outcome-based reference standards. An algorithm evaluated against LSI within one hour of definitive care arrival is not measuring the same construct as one evaluated within 24 hours of injury, and pooled estimates that ignore this distinction conflate materially different reference standards.
Time-window reporting was lowest in Expert Panel (0%) and Anatomic Severity (0%) families and highest in Outcome/Resource (71%); lowest in drill/live-simulation studies (10%) and highest in prospective cohorts (62%). This pattern is partly expected: a defined time window is intrinsic to Intervention-based and Outcome-based standards – the interval within which an LSI must be delivered or an outcome ascertained – but is conceptually less-applicable to Anatomic-Severity standards, which are static injury scores, and to Expert-Panel standards, whose critical/non-critical categories are assigned a priori without reference to an outcome-capture interval. For the LSI-Based and Outcome/Resource families, where the chosen window materially changes which casualties are classified critical, an unreported time window is a genuine reporting gap rather than an inapplicable item. Full stratified frequencies are in Supplementary Table S4. Two sources, coded N (citing Lerner category definitions only), yield 43% as an upper bound for R3.Reference Tiyawat, Liu and Huabbangyang26,Reference Xu, Malik and Chernbumroong27
Reference-standard non-reporting is a field-wide pattern. Prior systematic reviews documented incomplete reference-standard reporting as a field-wide limitation; the present review complements those findings by identifying which specific components are most frequently absent – most notably the time window for outcome capture.Reference Franc, Kirkland and Wisnesky5,Reference Marcussen, Bräuner and Alstrøm6,Reference Marcussen, Bendix Bräuner and Alstrøm28
Only five sources (8%) analyzed retrospective data from actual mass-casualty events: Kahn 2009,Reference Kahn, Schultz and Miller14 Peev 2013,Reference Peev, Naraghi and Chang15 Peng 2021,Reference Peng and Hu16 Zhu 2022,Reference Zhu, Xu and Shao17 and Castro Delgado 2022;Reference Castro Delgado, Gan and Cabrera García18 the Camp Bastion 2011 military cohort contributes analogous operational data. The remaining sources rest on registry simulation (47%) and drill/live simulation (17%).
Kahn, et al found that START failed pre-specified accuracy thresholds at an actual civilian MCI and produced substantial over-triage.Reference Kahn, Schultz and Miller14 The remaining real-event analyses used heterogeneous reference standards that preclude direct comparison.
Is Reference-Standard Heterogeneity Itself a Deficit?
Heterogeneity is not a methodological deficit: distinct operational contexts warrant distinct reference-standard formulations. However, the constructs grouped here are not epistemically equivalent – Outcome-based standards measure what happened to the patient, Intervention-based standards what was done, and Expert-Panel standards what a panel judged should have been done, with the last carrying additional limitations when used to validate expert-derived algorithms. The reporting-completeness findings (R3-R8 below 65%) identify a separate problem: even where context-specific adaptation is warranted, its operational specifics are inconsistently documented.
Relationship to Prior Systematic and Scoping Reviews
Prior reviews (Franc 2022, Marcussen 2022-2023, and Kilner 2011) documented reference-standard heterogeneity as a field limitation; the present review reverses that framing by treating reference-standard constructs as the primary object of inquiry.
Geographic Asymmetry between Methodological Literature and MCI Burden
Of 60 included sources, 77% originated in the United States or the United Kingdom/South Africa-linked network; foundational reference standards were therefore developed within Anglophone civilian and military trauma systems. This geographic locus does not overlap with the locus of global MCI burden, dominated by events in lower- and middle-income settings in Asia, the Middle East, and sub-Saharan Africa. The transferability of LSI lists and time windows to resource-constrained environments is an open empirical question, and original reference-standard work from the geographies bearing most MCI burden remains the most consequential gap in the field. Population-stratified reference-standard family distributions and RSRI-MCI reporting frequencies are provided in Supplementary Table S6.
Methodological Recommendations for Future MCI Triage Validation Research
The item-level pattern maps onto five reporting elements that would substantially improve comparability and replicability of MCI triage validation studies (Box 1). Items already reported by nearly all studies (R1, R2, R4) are not included.
Five Reporting Elements for Reference-Standard Descriptions in MCI Triage Validation Studies
A worked exemplar Methods passage addressing all eight RSRI-MCI items is in Supplementary Box S1. Future MCI triage validation studies should treat the five reporting elements identified by the RSRI-MCI (Box 1) as Methods-section requirements on par with the description of the index test itself. Beyond reporting completeness, quantifying how much measured triage-tool accuracy shifts as a direct function of reference-standard choice – rather than which standards are used – is a logical next step that the present mapping is intended to support.
Limitations
Key limitations are: (1) English-language restriction (registered as a deviation; four excluded records predominantly German-language); (2) single-reviewer title-and-abstract screening; (3) Embase (Elsevier; Amsterdam, Netherlands), grey literature, and citation chasing not performed, leaving conference proceedings and operational reports under-represented; (4) RSRI-MCI is a descriptive a priori framework, not an externally validated instrument – item-level frequencies are completeness assessments, not quality scores (R1 range: 83%-98% depending on coding convention; R3 lowest AC1: 0.657, lower bound 0.505); and (5) funding sources of included studies were not charted.
Conclusions
This methodology scoping review treats reference-standard constructs in MCI triage research as the primary object of systematic inquiry. Across 60 sources spanning from 2001-2025, operationalized reference standards trace to four foundational methodological papers (Garner 2001, Wallis 2006, Vassallo 2016, and Lerner 2015) and map onto five families, with LSI-Based and Composite/Hybrid constructs accounting for approximately two-thirds of included sources. The field reports what the reference standard is (definition, components, and rationale) almost universally, but reports how it is operationalized (time window, dead/expectant handling, binary mapping, missing data, and limitations) in only one-half of sources; the time window for outcome capture (R3) is reported in 40% overall, although its relevance varies across reference-standard families.
Supplementary Materials
To view supplementary material for this article, please visit https://doi.org/10.1017/S1049023X2610908X
Author Contributions
ÖA conceived the study, designed the protocol, performed the systematic searches, conducted title-and-abstract screening, performed full-text screening and data extraction, conducted all analyses, and drafted the manuscript. RKE served as the second reviewer, independently re-extracting family classification and RSRI-MCI item coding for all included sources and adjudicating disagreements with the primary reviewer by discussion. OG supervised the project as senior author, guided the conceptual framing and methodological direction, advised on the design of the reference-standard taxonomy and the RSRI-MCI, and provided final scientific oversight. DK and HI contributed to study conception and design and critically reviewed and revised the manuscript for important intellectual content. All authors meet all four ICMJE authorship criteria and approved the final manuscript.
Funding
No specific funding was received for this study.
Competing Interests
ÖA declares none; RKE declares none; DK declares none; HI declares none; OG declares none.
Use of Artificial Intelligence Tools
Large language model-based AI tools (Claude - Anthropic [San Francisco, California USA]; version Sonnet 4; used May 2026) were used for assistance with grammar checking and language editing of the manuscript. No AI-generated text, figures, or data appear in this manuscript. All scientific content, study design, data extraction, analysis, and interpretation were performed by the authors. The authors take full responsibility for the integrity of the work.
