Hostname: page-component-7b7567c8c-7pwfn Total loading time: 0 Render date: 2026-11-15T05:51:07.245Z Has data issue: false hasContentIssue false
  • English
  • Français

Impact and learner perspectives of a spaced-education platform to assess hand hygiene auditor competency

Published online by Cambridge University Press:  18 September 2024

Ana M. Vaughan-Malloy,
Renée Lehane,
Mary M. Wells and
Thomas J. SandoraOpen the ORCID record for Thomas J. Sandora[Opens in a new window]
Ana M. Vaughan-Malloy*
Affiliation:
Infection Prevention and Control, Boston Children’s Hospital, Boston, MA, USADivision of Infectious Diseases, Department of Pediatrics, Boston Children’s Hospital, Boston, MA, USADepartment of Pediatrics, Harvard Medical School, Boston, MA, USA
Renée Lehane
Affiliation:
Infection Prevention and Control, Boston Children’s Hospital, Boston, MA, USA
Mary M. Wells
Affiliation:
Infection Prevention and Control, Boston Children’s Hospital, Boston, MA, USA
Thomas J. Sandora
Affiliation:
Infection Prevention and Control, Boston Children’s Hospital, Boston, MA, USADivision of Infectious Diseases, Department of Pediatrics, Boston Children’s Hospital, Boston, MA, USADepartment of Pediatrics, Harvard Medical School, Boston, MA, USA
*
Corresponding author: Ana M. Vaughan-Malloy; Email: ana.vaughan-malloy@childrens.harvard.edu
Rights & Permissions [Opens in a new window]

Abstract

Objective:

To ensure whether spaced education, which increases long-term knowledge retention, could be integrated into auditor competency assessment.

Design:

Quality improvement project.

Setting:

Academic, freestanding children’s hospital.

Participants:

Hand hygiene (HH) auditors.

Intervention:

We enrolled trained HH auditors in an online spaced-education platform to assess mastery of knowledge, delivering 46 unique questions at spaced intervals followed by rationale; we retired questions after 3 correct answers. An e-mailed 10-item survey gauged participant satisfaction with the program. The Wilcoxon signed-rank test was used to compare change in median knowledge score from first to final attempt.

Results:

A total of 12,120 questions were attempted by 126 auditors, and 49 (39%) completed the entire course. Median knowledge score increased significantly by 10.5 percentage points (IQR 4–15) between first and final attempts (P < 0.001). Thirty auditors (27%) responded to the survey. The majority agreed the number and complexity of questions were appropriate (57% and 67%, respectively). Eighty-seven percent reported the platform easy to navigate, and 77% agreed adequate time was provided for completion. Free-text suggestions included delivering fewer questions at a narrower spacing interval over a shorter time frame because of competing work demands.

Conclusions:

Auditor knowledge of HH indications and technique is critical to ensuring data validity. A spaced-education competency program improved HH auditor knowledge in the short term. Completion rate was low, and some participants expressed a desire for fewer questions over a shorter time frame. This study offers insight into ways to optimize spaced education as a potential tool for HH competency assessment.

Information

Type
Original Article
Information
Infection Control & Hospital Epidemiology , Volume 45 , Issue 10 , October 2024, pp. 1208 - 1212
Check for updates
Copyright
© The Author(s), 2024. Published by Cambridge University Press on behalf of The Society for Healthcare Epidemiology of America

Introduction

Hand hygiene (HH) is fundamental to preventing healthcare-associated infections. Reference Boyce and Pittet1–Reference Tartari, Fankhauser and Masson-Roy6 Limited staffing and stretched resources may result in Infection Prevention and Control (IPC) programs relying on clinical or administrative personnel for auditing HH performance. Consistent knowledge of HH indications and techniques among auditors is paramount to ensuring data validity. 7

Historically at our institution, HH auditors were trained directly by the IPC manager, but over time onboarding of new auditors was de-centralized, with previously trained individuals recruiting and training new auditors. Beginning in 2019, a series of live, interactive HH auditor refresher sessions were held to review HH basics and the role and responsibilities of auditors and to ensure all auditors remained competent. Although many auditors participated, scheduling time away from primary responsibilities to attend these sessions was challenging. We therefore sought to simplify the process by which auditors are deemed competent annually.

Spaced education can optimize long-term knowledge retention by capitalizing on two concepts: the spacing effect and the testing effect. The spacing effect involves repeated exposure to learned material at spaced intervals over time to reinforce and consolidate retention. Longer intervals are associated with improved retention, but the ideal spacing depends on how long the knowledge must be retained. Widely spaced intervals are preferred for material with which the learner is more familiar, and shorter intervals may be better for less familiar material. The testing effect improves encoding of knowledge through active retrieval of learned information during testing. Reference Flett, Bousvaros, Carpenter, Millrinen, Martin and Sandora8–Reference Timmer, Steendijk, Arend and Versteeg16 In this quality improvement (QI) initiative, we evaluated whether spaced education could be integrated into an annual HH auditor competency assessment and whether it was associated with increased knowledge in the short term.

Materials and methods

Setting and participants

HH auditors at our 485-bed quaternary care freestanding children’s hospital participated in the competency program. These auditors represent individuals from both clinical (eg, registered nurse, clinical assistant, and technologist) and nonclinical backgrounds (eg, patient experience representatives and administrative support). An infection preventionist (IP) acting as the institutional HH coordinator, a physician Associate Hospital Epidemiologist, and a program manager comprise the HH program leadership. They provide initial training and oversee the annual knowledge and skills-based competencies for all HH auditors. Auditing is performed across all locations using a secret shopper method, with auditors encouraged to provide real-time feedback when lapses in performance are observed.

Spaced-education activity for HH auditors

We developed 46 unique questions based on the initial training all HH auditors received when onboarding into the program (Appendix 1). Questions spanned several themes: HH moments and behaviors, HH tools, personal protective equipment (PPE), and precautions. We used multiple choice and true/false formats and pilot-tested questions with a subset of IPs and auditors. Questions were subsequently refined for clarity based on feedback. An informational e-mail was sent to introduce the new competency activity to all HH auditors before the rollout. A reminder e-mail was sent four months after initial rollout, and monthly announcements of the competency expectation and mobile app capabilities were made at auditor-attended meetings. Utilizing a spaced-education platform (Qstream®, Burlington, Massachusetts), starting in November 2021 three questions per week were delivered by e-mail to auditors, with two weeks between repeat delivery of a given question. New auditors were added on a rolling basis as they completed their training, with the expectation of competency completion within 12 months of enrollment for all auditors. Participants could opt to receive more questions weekly but were unable to change the spacing interval. Questions were answered directly from the e-mail or through the platform’s mobile application (app). Upon submitting a response, immediate feedback was provided with an explanation for the correct answer (Figure 1), with knowledge scores posted in real time on a rolling basis within the platform. Once answered correctly three times, the information was deemed mastered and the question retired; otherwise, five total attempts were allotted prior to question removal. To allow newly trained auditors sufficient time to complete the competency, the platform remained available through October 2023. Each auditor worked through one cycle of the platform, with some participating to completion and others not completing the competency. Median initial and final attempt scores for each question were calculated. The proportion of questions answered correctly by a participant at first, and last attempts were provided to each individual. To add an element of competition, the platform displayed a running leaderboard of participant scores.

Figure 1.

Sample question and explanation. “X” indicates an incorrect response, and a checkmark indicates the correct answer.

After completion of the spaced-education activity window, a 10-item anonymous survey consisting of 9 Likert-scale questions and one free-text comment was e-mailed to gauge participant satisfaction with the competency program (Appendix 2). Reference Harris, Taylor, Thielke, Payne, Gonzalez and Conde17

Data analysis

Data analysis was performed upon closure of the platform and availability of survey responses. The Wilcoxon signed-rank test was used to assess for a significant change in median knowledge score from first to final attempt. A univariate linear regression model was created to evaluate whether change in score was associated with score at first attempt, to decide whether adjustment for baseline score was indicated. Frequencies and percentages were used to summarize Likert scale survey responses, and free-text comments were reviewed for additional suggestions.

Ethical considerations

As this project was a QI initiative to optimize our HH auditing program, Institutional Review Board review was not required at our institution.

Results

We enrolled 126 HH auditors in the online spaced-education course. By role, auditors were registered nurses (74%), administrative support personnel (7%), technicians (6%), occupational or physical therapists (5%), clinical assistants (4%), patient experience representatives and educators (3%), and a perfusionist (1%). A total of 12,120 questions were attempted. All 126 auditors attempted at least one question; the median number of unique questions attempted was 46 (IQR, 25–46), and 49 auditors (39%) completed the entire course (ie, they retired all 46 questions by correctly answering each one 3 times). Among those who completed the course, 39 (80%) were registered nurses, 3 (6%) were clinical assistants, 2 (6%) were administrative support personnel, 2 (6%) were physical therapists, 2 (6%) were technicians, and 1 (2%) was a patient experience representative. A minority (10%) of auditors utilized the platform’s mobile app; the remainder logged into the online platform.

The proportion of questions answered correctly by auditors within each theme improved from initial to final attempt (Figure 2), with the greatest difference in scores for questions related to HH tools (68% vs. 86%, respectively). The median knowledge score among all participants increased significantly by 10.5 percentage points between initial and final attempts (P < 0.001). Scores of four participants (3%) decreased from first to last attempt. Thirty auditors (27%) responded to the survey. Most respondents agreed the number and complexity of questions were appropriate (57% and 67%, respectively). Twenty-six (87%) reported the platform was easy to navigate, and 23 (77%) agreed adequate time was allotted for completion. Nineteen of 30 participants (63%) reported the course content was relevant to the auditor role, and the spaced-education model worked well. Seventy-three percent (22/30) agreed immediate feedback that included the rationale for the correct answer was helpful. A minority (30%) found the individual competition to be enjoyable.

Figure 2.

Knowledge scores. HH, hand hygiene; PPE, personal protective equipment.

Eleven respondents submitted free-text comments addressing the time and effort required to complete the competency; 9 (82%) reported challenges associated with competing priorities in busy clinical settings. One auditor wrote: “I found it a challenge often to get to see [the online platform] while working in a busy clinic…often would try to set aside…uninterrupted time to read and focus without phone calls or other staff interruptions.” Several respondents recommended fewer questions at a narrower spacing interval over a shorter time frame, noting it was “[t]oo long before more questions were given,” and “I also missed many weeks as these emails frequently dropped off my radar amongst the many other emails I’d sort through each shift.”

Five individuals expressed a preference for an annual computer-based module instead of a spaced-education course.

Discussion

Providing a standardized approach to HH auditing serves to minimize over- or underestimating reliability to optimal practice. Variability in the training IPC professionals receive for HH has been described, and the issue is likely compounded when roles outside the discipline are tasked with auditing HH reliability. Reference Tartari, Fankhauser and Masson-Roy6 Although spaced education has been used in the healthcare setting across a variety of topics, to our knowledge, there are no published studies applying this approach to evaluating HH knowledge as a component of auditor competency. Reference Flett, Bousvaros, Carpenter, Millrinen, Martin and Sandora8,Reference Janssen, Shaw and Bradbury9,Reference Phillips, Heneka, Bhattarai, Fraser and Shaw11–Reference Timmer, Steendijk, Arend and Versteeg16,Reference Kalynych, Edwards, West, Snodgrass and Zenni18 Our spaced-education course was associated with a significant increase in auditor knowledge over a 24-month period. Feedback received from auditors via survey was favorable overall; most participants endorsed the platform worked well, was easy to navigate, and provided an appropriate number and complexity of relevant questions, with helpful, real-time feedback on responses. Insightful, constructive feedback was also provided by a small number of individuals to improve or change the experience, notably with requests to make the competency less burdensome and more optimal for their workflow. Suggestions included delivering fewer total questions and reducing the spacing interval to allow for an overall shorter window of time for competency completion, or abandoning the spaced-education model in favor of an annual computer-based module.

Although most auditors endorsed ease of use of the platform, completion rate was low. Though the program was designed to allow sufficient time for auditors to complete it annually, the large number of questions, a noted dissatisfier, and number of correct responses needed to retire each question were potential challenges to completion. Picking a subset of questions on common themes important for HH auditor knowledge could decrease overall time and effort without sacrificing content needed to maintain competency. Additionally, defining two correct responses as mastery of a question and reducing the interval between delivered questions would allow completion of the program more expediently.

Our participants relied heavily on answering questions via e-mail, requiring a computer if auditors did not have access to their inbox on a mobile device. Given low usage of the platform’s app by our participants, we recommend groups interested in this educational approach highlight the time saved through using the app. Developing a short “how to” for auditors may elicit a higher participation and completion rate.

Benefits of this approach compared with our prior auditor competency assessment strategy include not needing to coordinate schedules or repeatedly secure space for in-person sessions. Participants receive standardized content and immediate feedback on responses. Additionally, with successes demonstrated in simulation-based training for IPC, we embedded recorded scenarios from the perspective of an auditor within the competency, testing participant ability to identify HH moments correctly. Reference Tartari, Fankhauser and Masson-Roy6 The online platform reduces the time our HH program leadership must commit annually for auditor competency assessment. The platform provides exportable documentation of participation to efficiently review and check off individuals as competent and to identify opportunities for improvement. One potential downside to this approach is the resources and infrastructure to access and maintain the technology, which may limit feasibility for some HH programs.

Though feedback was overall favorable, notable limitations to this intervention include low completion rates for both the entire competency and the satisfaction survey. Although the course was introduced after the peak of the COVID-19 pandemic, staff shortages, high turnover, and healthcare worker burnout may have affected both the bandwidth and motivation of auditors to participate in the active engagement required for this type of learning activity. We did not have data from a control group of auditors to compare the increase in knowledge observed after our intervention with other potential modes of auditor training. Additionally, as the survey was anonymous, we could not evaluate how much of the education program the respondents completed. Generalizability of our experience may also be limited outside of pediatric academic settings, in institutions without access to a spaced-education platform, and where auditing is performed by disciplines other than registered nurses, the predominant role among our HH auditors.

Despite survey results supporting use of the platform, subsequent meetings and informal feedback demonstrated most auditors preferred to demonstrate their knowledge competency once annually. Although we have not ruled out re-introducing a modified spaced-education in the future that addresses the survey responses, our HH program is now embarking on a third iteration of auditor competency assessment consisting of an annual computer-based module with 20 multiple choice and true/false questions. As recurring enterprise mandatory education is delivered on this platform, the auditors will be able to demonstrate their competency in a familiar manner and can complete this competency in a single session. As this approach may result in less long-term retention of knowledge compared with spaced education, we will continue to consider how to assess and optimize HH education going forward.

A spaced-education competency program improved HH auditor knowledge in the short term. Despite low completion rates, participants provided an overall favorable review of the format, though some expressed a desire for fewer questions over a shorter time frame. Further research should focus on how best to balance minimizing effort and disruption for participants while maximizing long-term retention of knowledge, as well as directly measuring the impact of this approach on auditor accuracy. Spaced education is an evidence-based teaching method that is currently underutilized in infection prevention and control, and our study offers insight into ways to optimize this learning approach as a potential tool for HH competency assessment.

Supplementary material

The supplementary material for this article can be found at https://doi.org/10.1017/ice.2024.115.

Acknowledgments

None.

Financial support

None reported.

Competing interests

A.M.V.M. reports stock ownership in the following health-associated companies: Asana, Aurora, Ayr Wellness, Bionano Genomics, Butterfly Network, Canopy Growth, Cresco Labs, CRISPR Therapeutics, Cronos Group, Curaleaf Holdings, Editas Medicine, Green Thumb Industries, High Tide, Iovance Biotherapeutics, Jushi Holdings, Moderna, Organigram Holdings, Pacific Biosciences of California, Personalis, Pfizer, SNDL, Terrascend, Tilray Brands, Trulieve. All remaining authors report no conflicts of interest relevant to this article.

References

Boyce, JM, Pittet, D, Healthcare Infection Control Practices Advisory Committee, HICPAC/SHEA/APIC/IDSA Hand Hygiene Task Force. Guideline for hand hygiene in health-care settings. Recommendations of the Healthcare Infection Control Practices Advisory Committee and the HICPAC/SHEA/APIC/IDSA Hand Hygiene Task Force. Society for Healthcare Epidemiology of America/Association for Professionals in Infection Control/Infectious Diseases Society of America. MMWR Recomm Rep Morb Mortal Wkly Rep Recomm Rep 2002;51:1–45.Google ScholarPubMed
Ellingson, K, Haas, JP, Aiello, AE, et al. Strategies to prevent healthcare-associated infections through hand hygiene. Infect Control Hosp Epidemiol 2014;35:937–960. doi: 10.1086/677145 CrossRefGoogle ScholarPubMed
Glowicz, JB, Landon, E, Sickbert-Bennett, EE, et al. SHEA/IDSA/APIC practice recommendation: Strategies to prevent healthcare-associated infections through hand hygiene: 2022 update. Infect Control Hosp Epidemiol 2023;44:355–376. doi: 10.1017/ice.2022.304 CrossRefGoogle ScholarPubMed
Linam, WM, Margolis, PA, Atherton, H, Connelly, BL. Quality-improvement initiative sustains improvement in pediatric health care worker hand hygiene. Pediatrics 2011;128:e689–698. doi: 10.1542/peds.2010-3587 CrossRefGoogle ScholarPubMed
Pittet, D, Hugonnet, S, Harbarth, S, et al. Effectiveness of a hospital-wide programme to improve compliance with hand hygiene. Infection Control Programme. Lancet Lond Engl 2000;356:1307–1312. doi: 10.1016/s0140-6736(00)02814-2 CrossRefGoogle ScholarPubMed
Tartari, E, Fankhauser, C, Masson-Roy, S, et al. Train-the-Trainers in hand hygiene: a standardized approach to guide education in infection prevention and control. Antimicrob Resist Infect Control 2019;8:206. doi: 10.1186/s13756-019-0666-4 CrossRefGoogle ScholarPubMed
World Health Organization. WHO guidelines on hand hygiene in health care. https://www.who.int/publications-detail-redirect/9789241597906. Published 2009. Accessed November 30, 2023.Google Scholar
Flett, KB, Bousvaros, A, Carpenter, J, Millrinen, CE, Martin, P, Sandora, TJ. Reducing redundant anaerobic therapy through spaced education and antimicrobial stewardship interventions. J Pediatr Infect Dis Soc 2018;7:317–322. doi: 10.1093/jpids/pix090 Google ScholarPubMed
Janssen, A, Shaw, T, Bradbury, L, et al. A mixed methods approach to developing and evaluating oncology trainee education around minimization of adverse events and improved patient quality and safety. BMC Med Educ 2016;16:91. doi: 10.1186/s12909-016-0609-1 CrossRefGoogle ScholarPubMed
Menon, V, Chubaty, A, Clezy, K, et al. Infectious diseases and antimicrobial prescribing: online spaced education for junior doctors. J Glob Antimicrob Resist 2020;22:226–230. doi: 10.1016/j.jgar.2020.02.008 CrossRefGoogle ScholarPubMed
Phillips, JL, Heneka, N, Bhattarai, P, Fraser, C, Shaw, T. Effectiveness of the spaced education pedagogy for clinicians’ continuing professional development: a systematic review. Med Educ 2019;53:886–902. doi: 10.1111/medu.13895 CrossRefGoogle ScholarPubMed
Barsoumian, AE, Yun, HC. Augmenting fellow education through spaced multiple-choice questions. Mil Med 2018;183:e122–e126. doi: 10.1093/milmed/usx020 CrossRefGoogle ScholarPubMed
Choe, AI, Woodard, S, Thompson, BM, Walter, V, Fotos, JS, Kasales, CJ. Spaced education: Randomized trial comparing learning efficiency of the adaptive versus nonadaptive spaced education systems among radiology residents. J Am Coll Radiol 2022;19:706–710. doi: 10.1016/j.jacr.2022.03.010 CrossRefGoogle ScholarPubMed
Kerfoot, BP. Adaptive spaced education improves learning efficiency: a randomized controlled trial. J Urol 2010;183:678–681. doi: 10.1016/j.juro.2009.10.005 CrossRefGoogle ScholarPubMed
Matos, J, Petri, CR, Mukamal, KJ, Vanka, A. Spaced education in medical residents: An electronic intervention to improve competency and retention of medical knowledge. PLoS One 2017;12:e0181418. doi: 10.1371/journal.pone.0181418 CrossRefGoogle ScholarPubMed
Timmer, MCJ, Steendijk, P, Arend, SM, Versteeg, M. Making a lecture stick: the effect of spaced instruction on knowledge retention in medical education. Med Sci Educ 2020;30:1211–1219. doi: 10.1007/s40670-020-00995-0 CrossRefGoogle ScholarPubMed
Harris, PA, Taylor, R, Thielke, R, Payne, J, Gonzalez, N, Conde, JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42:377–381. doi: 10.1016/j.jbi.2008.08.010 CrossRefGoogle ScholarPubMed
Kalynych, C, Edwards, L, West, D, Snodgrass, C, Zenni, E. Tuesdays are great for teaching tips: a spaced education strategy for faculty development. J Grad Med Educ 2021;13:434–435. doi: 10.4300/JGME-D-20-01249.1 CrossRefGoogle Scholar
Figure 0

Figure 1.Sample question and explanation. “X” indicates an incorrect response, and a checkmark indicates the correct answer.

Figure 1

Figure 2.Knowledge scores. HH, hand hygiene; PPE, personal protective equipment.

Supplementary material: File

Vaughan-Malloy et al. supplementary material 1

Vaughan-Malloy et al. supplementary material
Download Vaughan-Malloy et al. supplementary material 1(File)
File 30.8 KB
Supplementary material: File

Vaughan-Malloy et al. supplementary material 2

Vaughan-Malloy et al. supplementary material
Download Vaughan-Malloy et al. supplementary material 2(File)
File 39.3 KB