Introduction
The Glasgow Coma Scale (GCS) is a widely used and validated tool for assessing a patient's level of consciousness and for detecting acute changes in a patient's neurologic status. Developed in 1974, the GCS has been universally accepted for assessment and documentation of a patient's neurologic status.Reference Teasdale and Jennett 1 , Reference Jennett and Teasdale 2 The GCS score is the sum of three independent measures: eye opening (range: one through four), verbal response (range: one through five), and motor response (range: one through six). These three components are combined for an overall high score of 15 and a low score of three. The GCS is also used as a prognostic indicator of functional patient outcome.Reference Jennett, Teasdale, Braakman, Minderhoud and Knill-Jones 3 Particularly, the GCS motor sub score has shown robust prognostic value in both the CRASH and IMPACT models.Reference Perel, Arango and Clayton 4 , Reference Murray, Butcher and McHugh 5 However, the GCS also has several limitations. Specifically, it does not assess directly brainstem responses. In addition, the value of the verbal and eye components are diminished in intubated, aphasic, or aphonic patients, and in patients who have facial or ocular injuries that impede eye evaluation.Reference Stevens and Sutter 6 Several studies have raised concerns about the inter-rater reliability of GCS scoring.Reference Gill, Reiley and Green 7 - Reference Bazarian, Eirich and Salhanick 9 Inter-rater reliability is the degree of agreement among raters. Overall, the goal of standardized scoring systems is to maximize inter-rater reliability and to minimize the error of measurement. In health care, the reliability of a scoring system such as the GCS is significant because the patient's treatment is often based on this scoring systems.Reference Tujin, Janssens, Robben and Van Den Bergh 10
The GCS is applied widely in the emergency setting; it is used to guide trauma triage and for the application of essential interventions such as endotracheal intubation and other management strategies. However, inter-rater reliability of GCS scoring has been shown to be low for inexperienced users, especially for the motor component.Reference Juarez and Lyons 11 - Reference Segatore and Way 13 As a result, there are concerns regarding the accuracy and validity of GCS scoring among various types of emergency care providers. The objective of this study was to determine the degree of accuracy of GCS scoring between various levels of emergency care providers within the current Emergency Medical Services (EMS) system.
Methods
This was a prospective observational study of the accuracy of GCS scoring by various types of emergency medical providers using standardized video vignettes. The study was reviewed and approved by the Institutional Review Board of the University Medical Center of Southern Nevada (Nevada USA) on January 8, 2013. The study commenced on January 28, 2013 and closed on March 30, 2013. The cohort came from several parts of the United States (Nevada, Texas, Florida, and Minnesota). Most of the subjects were attending scheduled, educational settings and asked to participate in this study. Verbal assent of subjects was required. Twelve high-definition video vignettes of simulated adult patients exhibiting 12 different GCS scores were recorded using a Canon XL H1 high-definition video camera (Canon USA, Inc.; Melville, New York USA). These vignettes were independently reviewed by two board-certified neurologists and scored. Inter-rater agreement between the two neurologists was high (Cohen's κ = 1) and the GCS scores were deemed accurate.
The video was edited using Final Cut Pro 7 (Apple, Incorporated; Cupertino, California USA) and compressed with Apple Compressor (Apple, Incorporated). Two of the vignettes were discarded because of technical issues and time constraints leaving 10 video vignettes in the protocol. A digital recording of the vignettes was placed on iPad tablet computers (Apple, Incorporated) for viewing. Research assistants obtained verbal assent and then showed the video to a convenience sample of subjects. Glasgow Coma Scale prompts and similar memory devices (eg, smart phones) were not allowed. Following each vignette, a 10-second interval allowed the subjects to score the vignette just watched. The video could not be stopped once started. Subjects were asked to record each of the three components of the GCS and the total GCS score. The scores for each vignette were recorded on a standardized score sheet and later entered into a Microsoft Excel for Mac 2011 database (Microsoft Corporation; Redmond, Washington USA). Statistical analysis was completed using statistical functions within the database.
Results
A total of 217 emergency health care professionals took part in the study. Subjects were asked to provide basic demographic data, including age, gender, and years of experience. They were asked to self describe their role in the health care system (Tables 1 and 2). There were 2,084 total GCS observations with complete data sets for analysis (Table 3). Overall GCS scoring accuracy for all levels of providers was 33.1% (95% CI, 30.2-36.0; Table 4). The highest accuracy was observed on the verbal component of the GCS (69.2%; 95% CI, 67.8-70.4). The eye-opening component was the second most accurate (61.2%; 95% CI, 59.5-62.9). The least accurate component was the motor component (59.8%; 95% CI, 58.1-61.5). Resident physicians were the most accurate in using the GCS (51.0%; 95% CI, 44.5-57.5) while nurses were least accurate (29.0%; 95% CI, 10.3-47.7; Table 5). Accuracy was the least for patients with moderate head injuries (GCS nine though 12; Figure 1). A total of 9.2% of subjects assigned GCS scores that do not exist in the GCS scoring system (eg, zero).
Demographics of Cohort Group
Abbreviations: AEMT, advanced emergency medical technician; CCP, critical care paramedic; EMT, emergency medical technician.
aSeven subjects (3.2%) did not disclose gender.
Provider Descriptions
GCS Scoring Accuracy by Component and Provider
Abbreviations: AEMT, advanced emergency medical technician; CCP, critical care paramedic; EMT, emergency medical technician; GCS, Glasgow Coma Scale.
Summary of Accuracy of GCS Scoring by Vignette
Abbreviation: GCS, Glasgow Coma Scale.
Summary of GCS Scoring Accuracy by Provider Level
Abbreviations: AEMT, advanced emergency medical technician; CCP, critical care paramedic; EMT, emergency medical technician; GCS, Glasgow Coma Scale.
GCS Scoring Accuracy by Total GCS Score.
Abbreviation: GCS, Glasgow Coma Scale.
Discussion
In this study, only one-third of total GCS scores assigned to standardized patients in the video vignettes were accurate. This discrepancy is worthy of discussion as the GCS is used to detect changes in a patient's neurologic status during care. It is also used to guide care strategies and procedural interventions (eg, endotracheal intubation and trauma center destination). While there were some differences in scoring accuracy between providers of various levels, overall accuracy was poor. Several studies have examined intra-rater and inter-rater reliability of the GCS scoring. Few have looked at accuracy. Menegazzi and colleagues compared intra-rater reliability in emergency physicians and paramedics and found a significant level of agreement in terms of intra-rater reliability.Reference Menegazzi, Davis, Sucov and Paris 14 An Australian study found a high level of inter-rater reliability amongst critical care nurses, regardless of experience, but found reduced reliability in the motor component of the GCS.Reference Heron, Davie, Gillies and Courtney 15 Gill and colleagues compared GCS scores between two residency-trained emergency physicians and found only a moderate degree of inter-rater reliability (exact κ = 0.4). The agreement percentage for the total GCS was 32.0%.Reference Gill, Reiley and Green 7 Holdgate and colleagues found that the level of agreement in GCS scores between doctors and nurses was high, although a significant number of total GCS scores differed by two or more points.Reference Holdgate, Ching and Angonese 8 Bazarian and colleagues found that prehospital GCS scores were usually two points lower than scores assigned in the emergency department by attending physicians.Reference Bazarian, Eirich and Salhanick 9 Kerby and colleagues found that agreement was high (97.9%) between prehospital and emergency department GCS scores for patients with mild head injuries (GCS: 13-15). However, agreement in scores for patients with moderate head injuries (GCS: nine though 12) was only 9.3%. Scoring was somewhat better for patients with severe head injuries (GCS: three through eight) at 63.3%.Reference Kerby, MacLennan, Burton, McGwin and Rue 16 These results are similar to what was observed in this study. Lane and colleagues developed an instructional video to help improve GCS accuracy amongst EMS providers. The before-video accuracy was 14.7% and improved to 64.0% post video.Reference Lane, Báez, Brabson, Burmeister and Kelly 17
The complexity of the GCS may be the major reason for accuracy and inter-rater reliability issues. In this study, subjects were more accurate in each of the three components of the GCS than in the total GCS score. It seems intuitive that adding three separate scores increases the chances of an error in the total GCS score. Ultimately, the total GCS score is the most commonly utilized score in emergency care. Because of this, there has been a push to simplify neurologic scoring. Gill and colleagues evaluated five simplified scoring systems (GCS eye component, GCS verbal component, GCS motor component, Simplified Verbal Scale [SVS], and Simplified Motor Scale [SMS]) and found that these simplified systems applied in the out-of-hospital setting approached the same test performance as the total GCS for the prediction of four clinically relevant traumatic brain injury (TBI) outcomes (emergency department intubation, TBI, neurosurgical intervention, and hospital mortality).Reference Gill, Steele, Windemuth and Green 18 In a further study, Gill and colleagues found that the SMS (composed of three components) appeared to possess the best inter-rater reliability when compared prospectively with: the GCS total score; the three components of the GCS; the Alert, Responds to Verbal Stimuli, Responds to Painful Stimuli, or Unresponsive scale; and the Alert, Confused, Drowsy, or Unrepsonsive scale in the evaluation of a cohort of emergency department patients with altered mental status (Table 6).Reference Gill, Martens, Lynch, Salih and Green 19
Comparison of Different Neurologic Scoring Systems
It seems that the popularity of the GCS is disproportionate to the evidence that supports its role as a tool in the assessment of TBI. The Brain Trauma Foundation (New York, New York USA) found that the level of evidence supporting the prehospital use of the GCS was low (primarily, Level III evidence and indirect studies) resulting in a weak strength of evidence recommendation. 20
Limitations
There are several limitations to this study. First, the use of a convenience sample allowed for the possible introduction of bias into the cohort. For example, potential subjects unsure of their GCS scoring capabilities may have declined the opportunity to participate in the study resulting in an increase in proportion of subjects who were comfortable with GCS scoring. Second, the possibility that some of the GCS scores were inaccurate must be considered. Although there was complete agreement between the two expert neurologists, there was the possibility that both may have been wrong on any given measure. Third, video vignettes using relatively young actors may not accurately represent the patient population encountered by the subjects. However, video vignettes can actually improve control over confounding variables.Reference Braakman, Avezaat and Maas 21
Conclusion
This study demonstrated that GCS scores are highly inaccurate and vary between health care providers. The GCS was developed to allow different providers to measure a patient's neurological status objectively, thus allowing equivalent measurements among different providers. Although commonly used with good intentions, the complexity of the GCS makes overall application and usage unreliable. Based on this study, and others, simpler, alternate scoring systems should be developed and validated for application in emergency care situations.
Author Contribution
Authors BB, JF, and MC conceived and designed the study. Authors LJ, SD, CG, and JF prepared the video. Authors BB, JF, and WF supervised data gathering. Authors BB and WF completed data and statistical analysis. Authors BB and MC prepared the manuscript. The video vignettes used in this study are available online at: https://www.youtube.com/watch?v=-qrLe7AKBjk.
