Introduction
Triage – the sorting of patients according to urgency of need for clinical care1 – is an essential part of delivering effective and efficient emergency care.Reference Hinson, Martinez and Schmitz2 Over-triage, when a patient is assigned a higher level of clinical acuity than is reality, can serve as a protective measure. It is often deliberately built into Emergency Medical Services (EMS) systems so that the most urgent conditions are always identified.Reference Hinson, Martinez and Schmitz2 But over-triage can come with substantial costs: when non-urgent patients are consistently being tended to in a more urgent manner, finite time and resources are diverted away from those in greater need of care.Reference Hinson, Martinez and Schmitz2–Reference Dami, Golay and Pasquier4 Under-triage, the assignment of a lower level of clinical acuity than is reality, is also problematic as it can lead to delays in care for patients that are in need of time-sensitive interventions.Reference Hinson, Martinez and Schmitz2 In either case, excessive rates of inaccurate triage can lead to increased morbidity and mortality.Reference Hinson, Martinez and Schmitz2 Whenever EMS resources are directed away from patients in need of urgent care, the entire health care system is stained and there is unnecessary expenditure of time, resources, and personnel.Reference Michailidou, Goldstein and Salazar5 In resource-constrained settings where most EMS systems are in the early stages of development and resources are limited, poor triage in EMS only serves to compound other challenges.
The Western Cape province of South Africa is home to 6.6 million people, two-thirds of whom live in metropolitan Cape Town.6 South Africa is one of few nations in Africa with a fully-developed EMS.Reference Naidoo7 Though private EMS also exists, the large majority of South Africans living in the Western Cape rely on government EMS.Reference Hardcastle, Finlayson and van Heerden8 The Western Cape Government (WCG) EMS system is large, utilizing 261 ambulances and 1,823 staff at 49 bases to provide emergency care across the province.9,10
Communications, including dispatch, are a critical part of any EMS system’s efforts to provide optimal care. The WCG EMS dispatchers take calls from those requesting EMS services, determine the priority of the call, and then strategize vehicle placement and response. The emergency medical dispatch method used by WCG EMS dispatchers is home grown, but shares many similarities with a commonly-used system called criteria-based dispatch (CBD);Reference Clawson, Olola and Heward11,Reference Newton12 CBD allows for flexible call taking to identify key information such as a patient’s complaints, location, and need for care, instead of strict protocols for questioning callers.Reference Clawson, Martin and Hauert13 Guidance in the WCG EMS dispatch system is similar: rather than having strict protocols for gathering information, WCG EMS uses more flexible guidelines to direct dispatchers in decision making. This guidance is only available in a printed manual (Appendix 1; available online only); no computer-based methods are used in the dispatch process. While arguably more efficient than alternative methods, CBD often lends itself to dispatcher over-triage; this is especially the case if dispatchers are not adequately trained.Reference Clawson, Martin and Hauert13 This may also be the case for WCG EMS. Most WCG EMS dispatchers lack prior health care experience and receive brief training upon being hired, leaving them particularly vulnerable to over-triage. This, in combination with evidence that most EMS dispatching systems are poor at triage,Reference Bohm and Kurland14 suggests the need for an evaluation of dispatch triage accuracy in the Western Cape.
Identifying condition categories most prone to incorrect priority one (P1) dispatching will provide an opportunity to strategically improve triage guidelines for categories with the highest over-triage. Refining and re-introducing only these category guidelines is a cost- and time-effective approach to the problem. It will lead to improved resource utilization, which is particularly important in resource-constrained settings.
A recent study identified the ten most common dispatch complaints presenting in WCG EMS, which accounted for approximate two-thirds of all callsReference Alshehri, Pigoga and Wallis15 (Table 1).
Most Common Dispatch Complaints Presenting in Western Cape Government EMS from December 2016 to November 2017
Abbreviation: EMS, Emergency Medical Services.
These ten most common dispatch complaints, for which improved dispatch triage accuracy may have substantial impacts on the whole system, were reviewed to evaluate potential mismatches between priority levels assigned by dispatchers and triage levels found by EMS personnel on-scenes. The purpose of this study was to examine the accuracy of dispatcher triage over a one-year period in the WCG EMS system.
Methods
A retrospective analysis of existing dispatch and EMS data to assess the accuracy of dispatch-assigned priorities in the Western Cape was conducted. The mismatch between call priority – the level of urgency determined by dispatcher call center agents – and triage – the level of clinical need determined by EMS personnel – was analyzed.
Western Cape Government EMS tracks response and patient care data using two systems. Upon receipt of a call requesting an EMS response, dispatchers situated in emergency communications centers utilize computer-aided dispatch (CAD) technology to process the call, record relevant information, and initiate an EMS response. If a response is deemed necessary, there are two dispatch-assigned priority levels: “P1” for high-priority emergencies requiring a rapid response and priority two/“P2” for lower-priority calls. The level of response is chosen by the dispatcher based on information gathered during the call, including the patient’s chief complaint.
Then EMS personnel record response data, including demographics, triage level, and care rendered, on a tablet via electronic patient care reporting (ePCR) software. The WCG EMS utilizes the South African Triage Scale (SATS) to evaluate patients; SATS was initially developed to triage acute, undifferentiated patients presenting to health facilities in low-resource African settings.Reference Gottschalk, Wood and DeVries16 It was later incorporated for use by WCG EMS. Triage levels are defined by five colors: red is most emergent and blue is least. Color assignments are derived via combination of a Triage Early Warning Score (TEWS) based largely on vital signs and identification of any key clinical conditions that may represent the need for urgent care.
Patient records for a one-year period – December 1, 2016 through November 30, 2017 – were obtained from WCG EMS in the form of Microsoft Excel (Microsoft; Richmond, Washington USA) exports. Dispatch data were extracted from the CAD system and contained information from the dispatch perspective, including basic caller demographics, location, dispatch complaints, and assigned dispatch priority levels. The EMS data were also provided in the form of exports from the ePCR database; these data included basic demographics along with an EMS triage assignment and standardized patient care datapoints. In both CAD and ePCR data, each EMS response was denoted by a unique incident reference number. All data were deidentified prior to analysis and stored in encrypted Microsoft Excel files on a university-based computer, accessible only by study personnel. Patients were not involved in design, conduct, reporting, or dissemination plans for this research.
Duplicate incident reference numbers were removed from each dataset. All calls received by dispatch that did not generate an EMS response and did not have an associated ePCR record were excluded, as were CAD records with dispatch complaints not listed as being one of the ten most-common (identified in a previous study using the same datasetReference Alshehri, Pigoga and Wallis15). All ePCR and CAD data were merged to join records sharing incident reference numbers. Additional records were removed if missing dispatch-assigned priority levels or EMS triage levels.
The initial dispatch-assigned priority level of each patient encounter was compared to the actual severity of the call, as approximated by EMS triage level. Triage levels assigned by EMS personnel on-scene were considered the gold standard, defining over- and under-triage by dispatchers. Orange and red EMS triage levels were considered appropriate for P1 responses; all other levels (blue, green, and yellow) were accurate if assigned P2.
Patients assigned a higher level of acuity by dispatchers (based on assigned priority) than by EMS personnel upon arrival to a scene (based on triage level) were considered to be over-triaged; this was calculated as the proportion of false positives.Reference Lentz, Jenson and Hinson17 Patients assigned a level of lower acuity by dispatchers than by EMS personnel upon arrival to a scene were defined as under-triaged, calculated as the proportion of false negatives. Any patient given matching priority and triage levels was described as accurately triaged.
Sensitivity and specificity were also used to determine accuracy. Sensitivity was defined as the true positive rate (ie, percentage of those that suffered from a high-priority condition based on EMS provider documentation that were correctly identified as high-priority cases by dispatchers), and specificity the true negative rate (ie, percentage of those did not suffer from a high-priority condition based on EMS provider documentation that were correctly identified as low-priority cases by dispatchers).Reference Lentz, Jenson and Hinson17 Additionally, negative and positive predictive values (NPV and PPV, respectively) were calculated, with NPV representing the proportion of low priority (P2) cases marked as low priority by dispatch out of all patients marked low priority by dispatch, and PPV representing the proportion of high priority (P1) cases marked as high priority by dispatch out of all patients denoted high priority by dispatch.
These metrics, along with associated 95% confidence intervals, were calculated for each of the ten dispatch complaints included in the final merged dataset, and collectively for all records. Simple descriptive statistics were also calculated. All statistics were generated using SAS statistical software (SAS; Cary, North Carolina USA).
Ethical approval for this study was obtained from the University of Cape Town Human Research Ethics Committee (Cape Town, South Africa; HREC REF: 722/2017) and WCG Department of Health (Cape Town, South Africa).
Results
After removing duplicates, merging CAD and ePCR records, and removing records missing key datapoints, 185,166 records were suitable for inclusion (Figure 1). All ePCR records were matched with CAD records; however, approximately one-half (51.6%; n = 277,415) of CAD records did not have ePCR records to pair with and were thus excluded. Approximately one-quarter (28.8%; n = 74,974) of merged records were excluded due to missing variables.
Flow of Included Data.
Non-cardiac pain (19.7%) and respiratory (18.1%) and obstetric (12.1%) complaints accounted for the majority of EMS responses (Table 2).
Distribution of Dispatch Complaints Resulting in EMS Responses
Abbreviation: EMS, Emergency Medical Services.
It was more common for dispatchers to assign a lower priority to calls (P2; n = 124,843; 67.4%) than higher priority (P1; n = 60,323; 32.6%). Yellow was the most commonly assigned triage level, accounting for nearly one-half (46.0%) of triage levels assigned by EMS personnel (Figure 2). The lowest triage score, blue, was rarely assigned (0.8%). Approximately one-quarter (27.5%) of calls were identified as high priority by EMS based on triage designation (red or orange).
Distribution of Triage Levels Assigned by EMS Personnel for the Ten Most Common Dispatch Complaints in WCG EMS.
Across all dispatch complaints, the over-triage rate was 67.6% (95% CI, 66.34-68.76) and the under-triage rate was 16.2% (95% CI, 15.44-16.90; Table 3). Over-triage was highest for obstetric complaints (79.3% [95% CI, 78.21-80.41]) and lowest for musculoskeletal (60.8% [95% CI, 59.35-62.17]) and respiratory complaints (60.6% [95% CI, 60.00-61.18]). Under-triage was highest for respiratory complaints (29.9% [95% CI, 29.23-30.69]) and lowest for obstetric complaints (10.1% [95% CI, 9.25-10.89]).
Over- and Under-Triage Rates, Sensitivity, Specificity, and Positive and Negative Predictive Values of the Ten Most Common Dispatch Complaints in WCG EMS
Abbreviations: EMS, Emergency Medical Services; NPV, negative predictive value; PPV, positive predictive value; WCG, Western Cape Government.
Dispatch triage sensitivity for all included records was 49.2% (95% CI, 48.10-50.38), specificity 71.9% (95% CI, 71.00-72.92), PPV 32.5% (95% CI, 30.02-34.88), and NPV 83.8% (95% CI, 81.93-85.73). The conditions most sensitive for detecting an urgent need for care were motor vehicle accidents (MVAs; 99.5% [95% CI, 98.88-100.18]) and convulsions (96.6% [95% CI, 93.37-99.81]). Abdominal complaints (93.9% [95% CI, 92.76-94.96]) and non-cardiac pain (95.0% [95% CI, 94.64-95.44]) were the conditions with the most specificity in dispatch triage.
Discussion
This study provides the first evaluation of dispatch triage accuracy in the WCG EMS system, identifying that the system is suffering from both under- and over-triage. Despite variance across dispatch complaints, both under- and over-triage remained higher than widely accepted norms, and all rates were significantly above acceptable target metrics described in similar studies. Under-triage rates, which ranged from 10.1% to 29.9%, were substantially above published target rates for under-triage, which generally suggest less than five percent.Reference Sasser, Hunt and Faul18,Reference Peng and Xiang19 Over-triage rates, which ranged from 60.8% to 79.3%, were also high in comparison to recommendations that over-triage rates be, at maximum, 50.0%.Reference Sasser, Hunt and Faul18,Reference Peng and Xiang19
As expected, where over-triage was high, under-triage tended to be lower. For example, both obstetric and respiratory complaints saw high over-triage and low under-triage rates. Especially in systems such as WCG EMS, where training is minimal for dispatchers, it is not uncommon to see conservative over-triage rates.Reference Hosseini, Maleki and Gorji20 Excessive over-triage generally represents conservative dispatching, with dispatchers placing less risk in the priority decision. Patient outcomes are likely to be worse if a critical patient does not receive care in a timely matter. Frequently assigning higher priority levels reduces the likelihood of this happening. But, on a systems level, there are also many costs to both EMS and facilities, including decreased response efficiency and crowding of emergency departments.Reference Clawson, Olola and Heward11
These under- and over-triage rates found in this study support previously-established perceptions held by WCG EMS dispatchers and EMS personnel suggest that dispatch triage accuracy is poor.Reference Alshehri, Pigoga and Wallis15 They also align with previous data suggesting that triage accuracy in medical dispatch systems tends to be unreliable.Reference Bohm and Kurland14 Other studies have found that CBD systems are sensitive to critical conditions with minimal under-triage.Reference Bohm and Kurland14 This was not entirely the case in the WCG EMS system, where both under-triage and sensitivity were far from target ranges.
Additional measures of accuracy were utilized to further assess dispatch triage. Though there are no well-defined targets for sensitivity and specificity of dispatch triage, the overall rates found in WCG EMS were somewhat lower than those found in other studies evaluating medical dispatch triage.Reference Feldman, Verbeek and Lyons21,Reference Flynn, Archer and Morgans22 Some conditions suffered from very poor sensitivity (musculoskeletal complaints, non-cardiac pain, and vomiting/diarrhea). For these conditions, many patients in need of high-priority labelling are not being identified, which may lead to delays in receiving life-saving care and poorer outcomes. Convulsions and MVAs had better sensitivities but specificities well below all other conditions. This is unsurprising given that improving sensitivity in detecting high-acuity calls often results in decreased specificity. Maintaining specificity remains important, though, because it reduces the strain on the system by preventing the direction of unnecessary resources to calls that are incorrectly marked high priority.
At present, challenges with over- and under-triage place numerous strains on the system at-large, with misdirection of resources costing leading to economic and time costs. The lack of timely care for the patients in greatest need is likely reducing the impact of quality care provided by WCG EMS and generating poorer patient outcomes than necessary.
These findings lend to multiple opportunities for improving WCG EMS dispatching. At present, most dispatchers have no prior health care training and receive minimal on-the-job training. More intensive training and robust protocols, which were also suggested in previously conducted focus groups, may effectively improve the situation.Reference Alshehri, Pigoga and Wallis15 Increasing training, including more information on the concept of triage and strategies to implement it within call taking, could have many benefits. In addition to improving accuracy of priority assignments, it could increase general efficiency of calls and ensure that patients are being assigned appropriate dispatch conditions. A substantial amount of over-triage is likely stemming from conservative high-priority assignments given to what are generally perceived as high-risk conditions (eg, obstetric complaints). Protocols to further aid dispatchers in assigning appropriate priorities could increase dispatcher confidence, allowing them to more comfortably assign low-priority status to patients that meet pre-defined criteria. Additionally, automation of some priority assignments may prove useful in conditions were patients are currently experiencing consistent over- or under-triage.Reference Voskens, van Rein and van der Sluijs23 In improving these metrics, both patients and the WCG EMS system itself will benefit.
Limitations
This study has several limitations. Based on existing WCG EMS data, it appears that the number of calls assigned P1 has decreased over time: 42.5% of calls were P1 in 2015 compared to 32.6% during this study’s timeframe.10 Over-triage rates were likely decreasing prior to this study. It would be useful to know if this decrease continued throughout the study period; however, this study did not assess more granular trends.
The gold standard against which dispatch priority assignments were based were triage assignments made by EMS personnel; this may have limited accuracy given that they were not validated by in-depth record reviews and that results are not triangulated with any other indicators (eg, Glasgow Coma Scale or other severity scores). However, the triage scale used – SATS – is considered robust, factoring in both a numerical scoring system and discriminators (clinically-significant conditions).Reference Gottschalk, Wood and DeVries16 The combination of factors considered by WCG EMS personnel when evaluating triage on-scene is complex and may mitigate risks stemming from lack of data triangulation in this study.
Additionally, lack of consensus on what most appropriately constitutes over- and under-triage and lack of validated target metrics for these measures are inherent to all triage studies.Reference Lentz, Jenson and Hinson17,Reference Tanabe, Gimbel and Yarnold24,Reference Wuerz, Travers and Gilboy25 Multiple established target metrics have been reported on here to mitigate this. However, the fact these target metrics have been developed in high-income countries, further compounds this challenge. Comparing outcomes in resource-constrained settings to high-income country standards, simply because there are no well-defined targets for many developing EMS systems, may be problematic given variability in development and resource levels.
Conclusion
Although WCG EMS is more developed that most other African EMS systems, data are likely generalizable because of similar resource constraints. Outcomes are likely useful for comparisons between regions and provide insight into the challenges other systems may face as they mature.
Results of this study will be used to motivate the development of more rigorous training programs and resources for WCG EMS dispatchers, including improved dispatch protocols for conditions suffering from high over- and under-triage. These improved protocols will allow for dispatchers to more easily identify high- and low-priority conditions, and dispatch EMS resources as appropriate based on those determinations. In combination with routine assessment of target metrics related to triage, these changes are likely to improve cost- and time-effectiveness and improve the system’s overall efficacy.
Conflicts of interest/funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. No conflicts of interest to report.
Author Contributions
MFA and LAW designed this study. MFA and JLP collected and analyzed data. All authors contributed to the writing of the manuscript and have approved of the final version.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1049023X20001041
