Setting

The University of Chicago Medicine is a major teaching hospital located in Chicago, Illinois. It serves as the primary nexus of clinical care for the south side of Chicago, as the principal teaching hospital for the University of Chicago’s Pritzker School of Medicine, and as a regional referral center for cancer, inflammatory bowel disease, specialty surgery, and many other complex clinical conditions. With an inpatient capacity of 680 beds, the health system facilitates >20,000 hospital admissions and almost 500,000 outpatient visits annually. The medical center provides a full spectrum of care from primary care through tertiary and quaternary care, including transplant and a free-standing pediatric hospital. This study was deemed a quality improvement initiative and not human subjects research and was therefore not reviewed by the institutional review board.

Hand Hygiene Monitoring Technology

The GOJO SMARTLINK electronic monitoring system is comprised of 5 main components: activity counters, dispenser actuation counters, data receivers, a secure server, and a digital monitor. Activity counters are mounted near the doorway in each patient room. These devices are comprised of 2 “detection zones,” invisible cones that monitor thermal infrared energy (ie, heat). A room entry or exit is captured when a human body walks through both detection zones, displacing heat in the zone. Infrared energy has been applied in other settings and is highly reliable at detecting human presence. Incorporation of 2 zones within each activity counter allows for a level of internal validation intended to prevent a room entry or exit from being captured when (1) a person walking in the hallway passes by the doorway or (2) a person in the patient room walks near the doorway without exiting. No wearable devices are needed for the operation of this system.

Dispenser actuation counters are inserted into all ABHR and soap dispensers. These counters are unique housing systems that sit inside the dispensers, resulting in no visible indications of monitoring. Counting mechanisms have been deployed in hospitals throughout the United States and have proven to be accurate in their ability to detect the number of times a dispenser is actuated.

Data receivers are installed throughout the hospital to capture both “pings” confirming the device is online and data counts from each activity counter and dispenser. All data captured (ie, each room entry, room exit, soap dispenser actuation, ABHR dispenser actuation) are time-stamped based on the minute and second the data are received by a receiver. A secure cloud-based server captures all data from the receivers and stores the information at the device level. An online user interface allows for secure login during which all monitored data can be reviewed in tabular or graphical format. This system allows for assessment of data by hospital building, hospital floor, hospital unit, and time.

Finally, a digital monitor is installed at the nurses’ station of each hospital unit, displaying real-time compliance data. Lag time is minimal from the time of activity to the time of display on the digital monitor because the data are sent from each device through a receiver to the cloud-based server. This lag time is often only a few minutes. Compliance data can be displayed in a variety of formats, including tables and trends. Compliance is calculated using a defined period of time as follows:

$${{{\rm No}{\rm .}\,{\rm soap}\,{\rm dispenser}\,{\rm actuations{\plus}No}{\rm .}\,{\rm ABHR}\,{\rm dispenser}\,{\rm actuations}} \over {{\rm No}{\rm .}\,{\rm room}\,{\rm entries{\plus}No}{\rm .}\,{\rm room}\,{\rm exits}}}$$

Data can be evaluated at the hospital floor or building level, but they cannot be accurately assessed at a level of granularity more detailed than the hospital unit. To obtain room-level compliance data, the system would need to assign each soap or ABHR dispenser to a unique room. Then, only HH events performed at dispensers assigned to the room where activity occurred (entry or exit) would be included in compliance percentages. In other words, if an HCP washed their hands in the hallway and then walked into a patient room 2 doors down, the system would not be able to associate the HH event with the room entry. As such, this system is limited to calculating performance by simply adding all HH activities (ie, dispenser activations) and all room activities (ie, entries and exits) during a defined period to provide unit-level compliance. Furthermore, this system cannot distinguish between HCP and visitor activity.

Validation Approach

Our team followed a rigorous validation approach to assess the sensitivity and positive predictive value (PPV) of the HHMT after calibration by the vendor. Each device encounter was treated as an independent event. First, a planned path (ie, a route created throughout a unit or hospital area that allows for systematic validation of HHMT through purposeful activation of each activity counter and HH dispenser) was followed for each hospital unit where the HHMT was installed.Reference Limper, Garcia-Houchins, Slawsky, Hershow and Landon ¹⁴ Thus, system efficacy was assessed by quantifying its ability to capture purposeful behavior of system investigators. The planned path was documented in detail and was compared to the raw data collected by the HHMT. Devices that were unable to accurately capture planned path behavior were adjusted and reassessed, as recommended for the validation of these systems.Reference Limper, Garcia-Houchins, Slawsky, Hershow and Landon ¹⁴

Next, trained observers used direct observation to document room activity on a hospital unit. These observations, collected using the gold standard for measuring HH behavior, were also compared to raw data collected by the HHMT. This behavioral validation phase quantified system efficacy by quantifying its ability to accurately detect real-world behaviors in the hospital or clinic environment.Reference Limper, Garcia-Houchins, Slawsky, Hershow and Landon ¹⁴ Because this particular HHMT cannot distinguish HCP activity from visitor and patient activity, direct observation was used to estimate the proportion of room activity contributed by patients and their families on each unit in an attempt to measure any effect on HHMT accuracy.

A sample size of 123 was deemed necessary to accurately calculate the sensitivity of the HHMT. An α of 0.05 equated to Z_X/2=1.96, and sensitivity was conservatively set at 80% based on preliminary observations. The precision or maximum marginal error was set at 10% and prevalence was set at 50%. While prevalence of HH performance is known to range between 20% and 40%, the prevalence for the sample size calculation was an estimate of the proportion of behavior related to HH (eg, entering into a room, exiting a patient room, or actuating a dispenser) of all human movement on the hospital unit. In other words, it was estimated that 50% of movement on the unit was either in the hallway or patient room. This is a conservative estimate based on routine observation of provider behavior, specifically focused on walking through the unit.

Statistical Analysis

Sensitivities, the probabilities that true activity will be captured by the system, were calculated separately for the planned path phase and behavioral validation phase. “True activity” was defined as activity performed by the investigator team during the planned path phase and as activity observed by direct observation during the behavioral validation phase. Similarly, PPVs, the probabilities that activity captured by the system really occurred, were calculated for both planned path and behavioral validation. These analyses facilitated the assessment of system efficacy (ie, accuracy during purposeful activity conducted during the planned path phase) and effectiveness (ie, accuracy during real-world activity documented during behavioral validation). These fundamental epidemiologic measures were then stratified by estimated proportion of visitors and physical location. Analyses were conducted using Excel 2011 (Microsoft, Redmond, WA) and Stata/SE 13.1 for Mac (StataCorp, College Station, TX).