Introduction
Performance feedback is fundamental to medical education. Reference Ende1,Reference Kelly and Richards2 In the hospital environment, feedback comes in many forms, including morbidity and mortality conferences, simulation sessions, workplace-based assessments, and real-time feedback sessions. Reference Kelly and Richards2–Reference Gonazalo, Yang and Huang5 In Emergency Medical Services (EMS), however, the delivery of performance feedback and the provision of patient outcome information between hospital and prehospital staff are infrequent or lacking. Reference Cash, Crowe, Rodriguez and Panchal6–Reference Li, Munder, Chaudhry, Madan, Gribko and Arora9
Much of the existing literature regarding feedback to prehospital providers relates to measurable outcomes with protocolized interventions. Literature regarding general feedback or patient follow-up is sparse. Reference Oostema, Chassee, Baer, Edberg and Reeves10–Reference Weston, Jasti, Lerner, Szabo, Aufderheide and Colella13 No previous investigations have described how feedback or patient follow-up relates to EMS job satisfaction.
The objective of this study was to describe the frequency of feedback received by ground-based EMS professionals within the Southeastern Minnesota (USA) region. The secondary objectives were to identify factors associated with receiving feedback and how feedback and follow-up on patient outcomes related to EMS provider job satisfaction.
Methods
Study Design and Setting
An anonymous, online survey was administered to 456 EMS providers from twenty-three Basic Life Support (BLS), fifteen Advanced Life Support (ALS), and six combined BLS/ALS ground-based EMS agencies as part of a quality improvement project. An initial survey email was sent to agency point-of-contacts with request for dissemination to their agency staff on January 28, 2020, with one reminder email sent prior to survey conclusion on March 1, 2020. The study was reviewed by the Mayo Clinic Institutional Review Board (Rochester, Minnesota USA) and deemed exempt.
Survey
The survey included ten multiple-choice and five Likert-type questions and was modeled, in part, on a prior work by Cash, et al Reference Cash, Crowe, Rodriguez and Panchal6 (Figure 1). An initial logic error in the survey required early respondents to provide a response to questions regarding types of feedback received, even if they indicated not receiving feedback in the prior 30 days. Surveys received prior to survey correction that indicated no feedback received in the prior 30 days but then selected type and/or form of feedback received in the later questions were removed from subsequent analyses of these questions. Standard demographic measures were collected.
Survey Questions
Abbreviations: ALS, Advanced Life Support; BLS, Basic Life Support; EMS, Emergency Medical Services.
*Questions affected by initial logic error in survey.
Data Analysis
Survey responses are summarized with frequency counts and percentages. Comparisons of survey responses between demographic groups are performed using global Fisher’s exact tests. All tests are two-sided and P values less than .05 are considered significant. Follow-up pairwise Fisher’s exact tests were implemented when the global test was significant.
Results
Characteristics of Study Subjects
Ninety-five survey responses were received, for an approximate response rate of 20.8%. One respondent indicated working for both ground and flight EMS agencies and was excluded from subsequent analysis. The remaining ninety-four respondents represented nineteen different EMS agencies, including a hospital-affiliated EMS agency, with nine respondents indicating employment with at least two agencies within the region. Thirteen surveys were submitted before survey correction. Table 1 provides a summary of respondent demographics.
Summary of Survey Respondent Demographics (N = 94)
Abbreviations: EMS, Emergency Medical Services; EMT, emergency medical technician.
Summary of Feedback Among Entire Cohort
The majority of respondents (65.9%) indicated rarely or never receiving positive/reinforcing feedback (Table 2). Similarly, 60.6% indicated rarely or never receiving constructive criticism or feedback regarding something that did not go well. One-half reported that they had not received any type of feedback in the past 30 days. The majority of respondents were dissatisfied with the quantity (86.1%) and quality (73.4%) of feedback received.
Summary of Survey Responses, Overall (N = 94)
Abbreviation: EMS, Emergency Medical Services.
a Participants could provide more than one response
Impact of Feedback on Job Satisfaction
The majority (93.6%) of respondents indicated that feedback on patient care was important to their job satisfaction (Table 2). This ranking was maintained across all demographic groups with no significant difference occurring between gender (P >.99), certification level (P = .662), hospital affiliation (P = .685), or years of experience (P = .684). Respondents who indicated often receiving positive or reinforcing feedback (6.4%) were more likely to indicate being satisfied with the quantity (P = .033) and quality (P <.001) of feedback received compared to respondents who received less frequent positive feedback (Table 3).
Comparison of Feedback Frequency with Satisfaction
Feedback Trends by Provider Certification
Paramedics were more likely to receive feedback from hospital staff at the receiving facility compared with emergency medical technicians (EMTs; 44.3% versus 18.2%; P = .013; Table 4), but EMTs were more satisfied with the quantity of feedback they were given (P = .020). Despite these differences, the majority of paramedics and EMTs were dissatisfied with the quantity (93.4% paramedics versus 72.8% EMTs; P = .020) and quality (80.3% paramedics versus 60.6% EMTs; P = .202) of feedback they received. Only four (6.6%) paramedics and nine (27.3%) EMTs indicated that they were satisfied with the feedback quantity (P = .010) they received; however, there was no significant difference in satisfaction with the quality of feedback (P = .202).
Summary of Survey Responses by Respondent Certification and Agency Affiliation
Abbreviation: EMS, Emergency Medical Services.
a Participants could provide more than one response.
Feedback Trends by Gender
Males and females were equally likely to receive feedback in the past 30 days (P = .094). No differences were observed based on gender.
Feedback Trends by Hospital Affiliation
Although no difference in the frequency of positive feedback was observed, hospital-affiliated agency respondents (HARs) were more likely to receive negative feedback compared to non-HARs (P = .001; Table 4). Twenty-seven (52%) HARs and eight (23%) non-HARs indicated that they sometimes, often, or always received negative feedback (P = .016). Also, HARs reported more feedback in the past 30 days compared to non-HARs, with thirty-one (60%) HARs and eleven (31%) non-HARs indicating any type of feedback (P = .021).
Largely, HARs were more likely to receive feedback from staff at the receiving facility (46% versus 11%; P <.001) and less likely to not receive feedback altogether (27% versus 56%; P = .008) compared with non-HARs. Similarly, HARs received more verbal feedback relative to their non-hospital-affiliated counterparts (65% versus 29%; P = .017).
Feedback Trends by Years of Experience in EMS
Respondents with less experience were more likely to report any type of feedback (≤2 Years: 89%; 3-10 Years: 55%; 11-15 Years: 47%; ≥16 Years: 26%; P = .009; Table 5). Those with fewer years of experience indicated being satisfied with the quantity of feedback more often than respondents with more years of experience (≤2 Years: 44%; 3-10 Years: 9%; 11-15 Years: 20%; ≥16 Years: 9%; P = .035) with those respondents with less than two years of experience having higher satisfaction than all other respondents (≤2 Years: 44%; >2 Years: 11%; P = .019).
Summary of Survey Responses by Years of Experience
Abbreviation: EMS, Emergency Medical Services.
a Participants could provide more than one response.
Those with fewer years of experience in EMS were more likely to report being satisfied with the quality of feedback compared to those with more years of experience (≤2 Years: 78%; 3-10 Years: 36%; 11-15 Years: 27%; ≥16 Years: 9%; P = .002). Respondents with less than two years of experience were more likely to report being satisfied compared to all other respondents (≤2 Years: 78%; >2 Years: 21%; P = .001) and respondents with over fifteen years of experience were more likely to report being unsatisfied compared to all other respondents (≥16 Years: 91%; <16 Years: 68%; P = .030).
Discussion
In almost all areas of health care, there are opportunities for evaluation of performance and follow-up of patient outcomes with the goal of continued quality improvement in the care delivered to patients. Reference Kelly and Richards2–Reference Gonazalo, Yang and Huang5 Although EMS providers are often the point of first medical contact with patients, they experience missed opportunities for feedback or follow-up available to hospital-based teams. Prior studies indicate that prehospital providers desire more feedback and that they desire follow-up on their patients in terms of outcomes. Reference Morrison, Cassidy, Welsford and Chan7–Reference Li, Munder, Chaudhry, Madan, Gribko and Arora9,Reference Harrahill and Gunnels14
Feedback can be divided into both negative (constructive) and positive (reinforcing) types. Reference El Boghdady and Alijani3 Concerningly, this study found the majority of respondents indicated never or rarely receiving either related to their patient care. Both types of feedback are essential, and their absence likely impacts patient care—mistakes go uncorrected and good performance is not reinforced. Reference Ende1
This study found that one-half of respondents had not received any type of feedback in the prior 30 days. This was significantly less feedback compared with the study by Cash, et al which found 69.4% of EMS providers nation-wide reported receiving some type of feedback in the same time period. Reference Cash, Crowe, Rodriguez and Panchal6 Similar to prior literature, the current study found that those more likely to receive feedback included paramedics and those working for the hospital-affiliated agency. These two findings may be interrelated, as this hospital-affiliated agency primarily employs paramedics with surrounding rural, volunteer-based non-affiliated agencies more likely to be staffed by EMTs.
This survey found respondents with fewer years of experience were more likely to receive feedback and more likely to be satisfied with the quality and quantity of feedback received. The increased amount of feedback received may be explained by on-the-job training with close supervision or more frequent evaluations during a probationary period. Alternatively, newer personnel may remember any feedback received compared with more seasoned EMS providers, leading to recall bias. This finding that more seasoned EMS providers are more dissatisfied with the limited amount of feedback they are receiving serves as an important reminder that feedback is essential in health care throughout all levels of one’s career.
The reason EMS providers may not be receiving adequate feedback or patient follow-up is likely multifactorial. In addition to historically being overlooked as part of the health care team, EMS agencies often do not have access to the same electronic medical record utilized by the hospital, they often provide independent care without real-time physician oversight, and small rural or volunteer agencies may have less interactions with hospital staff, leading to less opportunities to solicit feedback. Reference Tsai, Choi, Sullivan and Willias8 Additionally, the fast-paced, high-acuity environment of the emergency department (ED) leads to its own barriers in soliciting or responding to feedback requests by both EMS providers and ED staff, respectively.
One of the most longstanding barriers is concern for patient confidentiality and uncertainty about whether sharing of patient information for quality assurance and education with EMS is permissible under federal or state law or institutional policy. Reference Harrahill and Gunnels14 Although dissemination of patient information to hospital-affiliated EMS personnel without explicit patient consent, for the purpose of quality improvement and education, is permissible, Minnesota state law requires patient consent prior to sharing information to non-hospital-affiliated EMS agencies. 15 This is exacerbated by a lack of ED staff familiarity with the relevant regulations for the sharing of information with EMS crews. Additionally, there remain time and logistical barriers to delivering this feedback, both real-time and after shift.
The majority of respondents indicated that feedback on patient care was important to their job satisfaction. This level of high importance was maintained across all demographic groups, indicating how significant the lack of consistent feedback may be to overall job satisfaction among prehospital providers. Future research is needed to better evaluate what factors contribute towards feedback satisfaction in the prehospital environment.
Limitations
This study has several important limitations. Although based upon a survey developed by Cash, et al Reference Cash, Crowe, Rodriguez and Panchal6 , the survey was developed de novo for this study. As a consequence, the survey has not been validated, nor has the accuracy and cultural appropriateness of the survey. This survey was sent to EMS agencies within a specific region of Minnesota and may not be generalizable broadly to agencies in other regions. Further, because the survey was distributed through agency points-of-contact rather than directly to individual EMS providers, the response rate can only be estimated. As a convenience sampling was performed, results only apply to those completing the survey rather than the region population as a whole. The initial survey response was compromised by a survey logic error, although this was rapidly corrected. The terms “feedback” or “patient follow-up” were not defined, instead relying on respondent interpretation of both. While one individual may recognize a simple word of encouragement as feedback, another may expect a more detailed review of the care they provided to a patient. This lack of consensus may lead to an under-estimation of the amount of feedback EMS professions are receiving. Lastly, the study is subject to recall and reporting bias in terms of feedback or patient follow-up received over a 30-day time period.
Conclusions
Within the cohort of survey respondents, the paucity of feedback received by EMS personnel is a source of dissatisfaction. Feedback on patient care strongly relates to overall job satisfaction. These findings suggest system-wide opportunities for structured feedback processes, focusing upon both quality and quantity of delivered feedback, to improve both patient care and staff satisfaction.
Conflicts of interest/funding
The authors have indicated they have no financial relationships relevant to this article to disclose. No funding was received for this work. An abstract with limited data from this study is being presented at the NAEMSP 2021 virtual meeting.
Author Contributions
SM, AL, AK, LM, and MS conceived the study concept. SM collected the data and AM analyzed the data. SM wrote the first draft and all authors read and approved the final manuscript.
