California Health and Safety Code section 1288.55 requires hospitals to report data on central-line–associated bloodstream infections (CLABSIs), methicillin-resistant Staphylococcus aureus bloodstream infections (MRSA BSIs), vancomycin-resistant Enterococcus infections (VRE BSIs), Clostridioides difficile infections (CDIs), and surgical-site infections (SSIs) to the California Department of Public Health (CDPH) via the National Healthcare Safety Network (NHSN). The CDPH analyzes and disseminates these data in annual reports 1 after implementing a data quality review plan including creation and dissemination of quality assurance and quality control (QA/QC) reports. These reports include data-quality alerts generated by the NHSN. Due to the coronavirus disease 2019 (COVID-19) pandemic, the CDPH suspended regulatory enforcement requirements for hospital healthcare-associated infection reporting from January 1 through June 30, 2020. 2 Additionally, the Centers for Medicare and Medicaid Services (CMS) issued an HAI reporting exception for 2020-Q1 and 2020-Q2. However, hospitals reporting data via automated electronic mechanisms were encouraged to maintain those systems for reporting data to the NHSN. Although reporting requirements were reinstated as of July 1, 2020, there are concerns about data quality due to the possible impacts of the COVID-19 pandemic on hospital HAI surveillance and reporting activities. We examined several markers of data completeness and quality for both halves of 2020 compared with the corresponding halves of 2019.
Methods
All comparisons assessed data downloaded for the 2019 and 2020 annual reports. 1,3 The download dates for these data were April 2, 2020, and April 1, 2021, respectively. We compared the number of data-quality alerts issued by the NHSN that were outstanding (as of the download date) for CLABSI, laboratory-identified (LabID) infections (ie, MRSA BSI, VRE BSI, and CDI), and SSI. The NHSN issues alerts when a facility reports the intention to report HAI data for a particular month in their monthly reporting plan, then fails to report adequate data. 4 Examples of omissions that would lead to incomplete records for summary or procedure (denominator) data include failure to record patient days, central-line days, or data necessary for NHSN risk adjustment models. Omissions that would lead to incomplete records for event (infection) data include failure to enter any events or state that none were observed for the month, omission of admission date, or failure to report whether the patient died of CLABSI or SSI. We compared alerts by HAI type for the first and second halves of 2020 to the corresponding halves of 2019. We also performed tests for significant differences between the number of alerts issued per facility for 2019 versus 2020, using the Wilcoxon signed-rank test.
Data-quality alerts do not capture situations in which a facility did not include HAI type(s) in their monthly reporting plans. For this reason, we also compared the number of facility-months of data reported from acute-care hospitals (excluding rehabilitation units), critical-access hospitals, and long-term acute-care hospitals for CDI, MRSA BSI, VRE BSI, and CLABSI. We tested the difference for each of the 398 facilities operating in 2020 using the Wilcoxon signed-rank test. We did not compare facility months of data for SSIs because many hospitals in 2020 canceled or postponed some less-urgent surgical procedures, to increase bed availability, to ensure proper staffing, and to ensure sufficient personal protective equipment to care for COVID-19 patients. Reference Diaz, Sarac, Schoenbrunner, Janis and Pawlik5 Therefore, a change in the number of months reported for SSI may indicate that certain facilities discontinued some surgical procedures rather than a data-quality issue.
Our third data-quality check compared of the number of facilities with >10% mismatch between the total number of patient days reported for CLABSI versus the total patient days reported for MRSA BSI and VRE BSI. We used the χ2 test to test for significant differences. Analyses were performed using SAS version 9.4 software (SAS Institute, Cary, NC).
Results
Table 1 shows the results of our analysis of missing data alerts, facility months of data, and mismatches between CLABSI and LabID patient days. The numbers of alerts issued in both halves of 2020 and for all HAI types were significantly higher (at α = 0.05) than the number of alerts in the corresponding half of 2019. The numbers of facility months of data reported were higher for 2019 for both halves and all infection types. Although the numbers of patient-day mismatches were higher for each half of 2020 compared with the corresponding halves of 2019, the differences were not significant at α = 0.05.
HAI Data Quality Indicators in California Hospitals by Half-Year, 2019 Versus 2020
Note. HAI, healthcare-associated infections; NHSN, National Healthcare Safety Network; CLABSI, central-line–associated bloodstream infections; SSI, surgical site infection; CDI, Clostridioides difficile infection; MRSA BSI, methicillin-resistant Staphylococcus aureus bloodstream infection; VRE BSI, vancomycin-resistant enterococci bloodstream infection; LabID, laboratory identified.
a LabID includes CDI, MRSA BSI, and VRE BSI.
b Procedures included in SSI surveillance are abdominal aortic aneurysm repair; appendix surgery; bile duct, liver, or pancreatic surgery; cardiac surgery; coronary bypass with chest and donor incisions; coronary bypass graft with chest incision; gallbladder surgery; colon surgery; cesarean section; spinal fusion; open reduction of fracture; gastric surgery; hip prosthesis; heart transplant; abdominal hysterectomy; knee prosthesis; kidney transplant; laminectomy; liver transplant; kidney surgery; ovarian surgery; pacemaker surgery; rectal surgery; small bowel surgery; spleen surgery; thoracic surgery; vaginal hysterectomy; and exploratory laparotomy.
c Facility months of data do not include rehabilitation hospitals and units.
Table 2 shows the breakdown of data alert types by half year for 2019 and 2020. Alerts for denominator data (summary and procedure data) were far more common than alerts for events (infections).
Alert Types by Half-Year, 2019 Versus 2020
Note.CLABSI, central-line–associated bloodstream infection; SSI, surgical-site infection.
a Laboratory-identified (LabID) events include Clostridioides difficile infection, methicillin-resistant Staphylococcus aureus bloodstream infection, and vancomycin-resistant Enterococcus bloodstream infection.
Discussion
These comparisons suggest possible data-quality issues with data for all tracked HAI types and from both halves of 2020. Possible limitations in data quality should be considered when interpreting California HAI data from the NHSN for 2020. Data-quality issues in the second half of 2020, after reporting requirements were reinstated, suggest a possible direct effect of the COVID-19 pandemic on reporting. This effect may also apply to NHSN data in other jurisdictions. The strengths of this analysis include the universal enrollment of California hospitals in the NHSN and the standardization of reporting through the NHSN system, which makes it easier to detect reporting anomalies.
How these data-quality issues may bias standardized infection ratios, incidence rates, and other estimates derived from California 2020 NHSN data remains unclear. Infection and procedure counts are most likely underestimates of true values, but the magnitude is unclear. If missing data are nondifferential with respect to the facility HAI incidence (ie, high-incidence facilities were no more or less likely to report data than low incidence facilities), then point estimates for SIRs and incidence rates derived from these data should be unbiased.
Unfortunately, nondifferential reporting is not guaranteed. Also, we did not account for all possible misclassifications in 2020 data. We paused our regular HAI surveillance and NHSN reporting validation activities with hospitals during 2020, so it is unclear how accurately infection, denominator, and risk-adjustment variables were reported. Also, facilities may have added or changed patient-care locations without changing the corresponding mapping in the NHSN system. Because data cannot be entered into the NHSN database for unmapped areas, infections may have been underreported. Also, type of patient-care location is used in some risk-adjustment models; therefore, incorrect mapping could have introduced bias to the SIRs.
Our findings underscore the challenges of maintaining adequate surveillance during healthcare system crises and the need to make reporting systems more resilient to these crises. Ways to increase resilience could include automating systems to reduce the need for manual data entry and designing electronic medical record forms and processing tools with surveillance needs in mind. Reference Massoudi and Sobolevskaia6,Reference Sudat, Robinson, Mudiganti, Mani and Pressman7
Acknowledgments
The findings and conclusions in this article are those of the author(s) and do not necessarily represent the views or opinions of the California Department of Public Health or the California Health and Human Services Agency.
Financial support
No financial support was provided relevant to this article.
Conflicts of interest
All authors report no conflicts of interest relevant to this article.
