Discussion

Our study suggests that LLM augmented with NHSN criteria can support accurate and time-saving review of suspected CLABSIs, including the ability to identify secondary BSIs.

Traditional CLABSI surveillance is time-consuming, requiring rigorous EMR review, integration of clinical, laboratory, and radiographic data, and expert judgment to determine whether a BSI meets NHSN CLABSI definitions or is secondary to infection from another site. Surveillance activities occupy 25–50% of IP work hours. ^{Reference Wiemken and Carrico4,Reference Knighton, Engle, Berkson and Bartles6} Accurate surveillance is critical, both for guiding infection prevention interventions and for ensuring the validity of publicly reported outcomes.

In our implementation, we fine-tuned the LLM to extract data from the EMR and apply NHSN definitions to suspected CLABSI cases, effectively functioning as a secondary reviewer. This is a retrieval-augmented generation strategy that directs the LLM to draw from reliable sources to reduce risk of inaccuracies. ^{Reference Abosi, Kobayashi and Ross7,Reference Gao, Xiong and Gao8} Prompt engineering allowed for building guardrails into the LLM context and for targeted data extraction, minimizing missed elements and synthesizing information across the medical chart. The tool resembled “consulting another IP” with built-in knowledge of NHSN criteria, but without bias.

The use of LLMs in infection prevention is not yet widespread. Rodriguez-Nava et al. demonstrated the ability of a secure version of ChatGPT to identify CLABSI using real life notes. Their method had higher sensitivity (80%) than specificity (35%), taking only 5 minutes for data input and review. ^{Reference Rodriguez-Nava, Egoryan, Goodman, Morgan and Salinas1,Reference Wu, Langford, Shenoy, Carey and Branch-Elliman9} Abosi et al used educational scenarios to evaluate the accuracy of four LLMs, concluding that LLMs show promise as supplemental tools for infection prevention programs. ^{Reference Abosi, Kobayashi and Ross7} Our study expands on this early work by demonstrating performance in live operational settings, accuracy in identifying CLABSI and attribution of secondary BSIs, an area requiring nuanced human judgment. While more complex cases still necessitate IP adjudication, our findings suggest that LLMs can reliably streamline the initial stages of review.

Potential benefits to LLM tools include reallocation of IP time to prevention activities. ^{Reference Wiemken and Carrico4,Reference Marra, Nori, Langford, Kobayashi and Bearman5} LLMs offer consistency in applying surveillance definitions reducing inter-rater variability. By continuous exposure to new cases and data, LLMs are able to learn and improve. ^{Reference Abosi, Kobayashi and Ross7} However, the use of LLMs with protected health information requires safeguards to assure data security, ^{Reference Marra, Nori, Langford, Kobayashi and Bearman5} model inaccuracies or hallucinations remain a risk, and reliance on AI without human oversight could compromise surveillance integrity. ^{Reference Marra, Nori, Langford, Kobayashi and Bearman5}

Our study was limited by its small sample size and single-center design, and larger, multicenter evaluations are needed for generalizability. This study is an early-stage evaluation of the LLM tool. Continued refinement of IPs’ prompting and LLM data extraction will improve performance.

Future directions include refinement of prompts to enhance accuracy, LLM integration for other HAIs, and assessment of broader impact on IP workload and institutional outcomes. Our LLM tool does not automatically screen the EMR and determines CLABSI cases, however continued learning from applications such as this may advance future use of proactive AI surveillance.

In conclusion, an LLM augmented with NHSN criteria can accurately and efficiently support CLABSI surveillance. Such tools can help improve IP efficiency and maintain surveillance accuracy in increasingly resource-constrained healthcare environments.