Several factors suggest that ambulatory care sites may be an important yet underrecognized source of viral healthcare-associated infections (HAIs) among children. The majority of pediatric encounters take place in ambulatory settings. More clinic visits occur during periods of high prevalence for viral infections, and clinics have many features that increase the likelihood of exposure to both contaminated surfaces and fomites.Reference Hall 1 – Reference Harris 3 However, little is known about the frequency of and risk factors for viral HAI attributable to outpatient encounters.Reference Hall 1 Our objective was to develop and validate a surveillance definition for healthcare-associated influenza-like illness (HA-ILI) based upon electronic health record (EHR) data that could be used in future studies to characterize HA-ILI risk among children visiting primary care clinics and to evaluate the effectiveness of different prevention strategies.
METHODS
We conducted a prospective cohort study of children <6 years old who presented to any of 31 clinics within a pediatric primary care network over 3 respiratory seasons (October–April 2012–2013 through 2014–2015) for a well or nonrespiratory acute visit. Study procedures were approved by The Children’s Hospital of Philadelphia Institutional Review Board.
We identified ILI encounters using a set of International Classification of Disease, Ninth Revision, Clinical Modification (ICD-9-CM) codes and patient encounter data that had been validated for ILI syndromic surveillance in community and acute care settings from the network’s shared EHR (Fig. 1).Reference Marsden-Haug, Foster and Gould 4 HA-ILI cases were defined as ILI in children who returned for an ILI visit within 24 hours to 8 days after a non-ILI related clinic visit. This 8-day interval was selected based upon incubation periods for the 3 respiratory viruses most commonly associated with nosocomial respiratory infections among young children.Reference Hall 1 Non-ILI cases were defined as no documented ILI within 8 days after a non-ILI clinic visit. Data were extracted daily by a biomedical informatics analyst. We sampled all patients (n=2,743) that fulfilled ICD-9 criteria for HA-ILI and randomly sampled 1 in 100 patients with non-ILI cases for weekday visits and 1 in 10 for weekend visits (n=9,313) to achieve adequate numbers of observations from all days of the week. We then conducted structured telephone interviews with parents of 464 children with ILIs (16.9%) and 1,084 children without an ILI (11.6%) to measure sociodemographic and household characteristics and presence of respiratory symptoms or fever at any time since the index visit (Fig. 1).
Sampling strategy for the identification of healthcare-associated influenza-like illness (ILI) cases and non-ILI cases among children <6 years presenting for nonrespiratory visits.
Interviews were conducted within 1–5 days after the 8-day risk window. Parents of 286 children with ILIs (10.4%) and 805 children without an ILI (8.6%) declined participation. Patients who presented with any ICD-9 coded illness included in the ILI syndromic surveillance definition, or recorded fever at the initial visit, were excluded.
Case Confirmation
A systematic chart review of both index and follow-up visits among all enrolled children was performed by trained research staff to validate HA-ILI cases identified using the EHR case definition (Online Supplementary Appendix). As an alternate reference standard, the telephone interviews with parents were also used to validate the EHR surveillance strategy. Interview items for case confirmation measured whether their child had respiratory symptoms or fever at the index visit, whether the child became ill with respiratory symptoms or fever after the initial clinic visit, and recorded temperature for reported fever (Online Supplementary Appendix).
Analysis
To represent the population from our sample and to estimate sensitivity and specificity, we developed weights equaling the inverse of the product of (1) the probability of selection (1, 1 of 10, or 1 of 100) multiplied by (2) the probability of parental response. We compared patients with cases and non-ILI cases identified through EHR criteria to both chart review and parental interview as 2 different reference standards. We calculated positive predictive value (PPV) and negative predictive value (NPV) with associated confidence intervals in the enrolled sample, then we applied sampling weights to derive population estimates for sensitivity, specificity, PPV and NPV. Confidence intervals accounted for the weighting. Analyses were conducted using STATA 13 (StataCorp LP, College Station, TX).
RESULTS
Among the 1,548 cases for which the patients’ parents were interviewed, 1,352 cases fulfilled our inclusion criteria, 372 cases met the EHR case definition for HA-ILI, and 980 were deemed non-ILI cases. The majority of patients (67.6%) were <2 years old and 44.7% attended childcare or school outside of the home (Online Supplementary Appendix).
We confirmed 351 HA-ILI cases (94.3%) and 978 non-ILI cases (99.8%) based on chart review. The PPV and NPV, and after applying sampling weights, the sensitivity and specificity of the EHR criteria compared to chart review, are presented in Table 1A. Among the 21 HA-ILI cases that were not confirmed by chart review, 5 patients had no documented fever and 7 patients had no discharge codes consistent with ILI at the follow-up encounter.
Validation Measures for HA-ILI International Classification of Disease, Ninth Revision, Clinical Modification (ICD-9-CM) Diagnosis Codes Compared to Chart Review and Parental Interview
NOTE. PPV, positive predictive value; NPV, negative predictive value; Se, sensitivity; Sp, specificity; N/A, not applicable.
a Case +, cases for which ICD-9 HA-ILI definitions were met.
b Case –, cases for which the ICD-9 HA-ILI definition was not met, ie, non-ILI cases.
We confirmed 186 HA-ILI cases (53%) and 932 non-ILI cases (95%) by parental interview. The PPV and NPV, and after applying sampling weights, the sensitivity and specificity of EHR criteria compared to parent interview, are presented in Table 1B. Among the 186 HA-ILI cases not confirmed by parental interview, the most common reason for the discrepancy was parental report of ILI symptoms at the initial visit. A total of 48 patients who were non-ILI cases (5.1%) developed respiratory symptoms and fever within 8 days of the initial visit based upon parental reports.
DISCUSSION
Although existing literature supports the efficacy of syndromic surveillance for ILI in pediatric and adult emergency departments, no studies have explored its use in identifying outpatient HA-ILI.Reference Hiller, Stoneking and Min 5 , Reference Fowlkes, Dasgupta and Chao 6 An EHR-based definition of HA-ILI was highly sensitive for the identification of children who developed a medically attended ILI within 8 days of a primary care clinic visit. This finding has important implications for HAI surveillance because recent studies suggest an increased risk of acquiring an ILI after a pediatric office visit.Reference Simmering, Polgreen and Cavanaugh 7 , Reference Feemster, Localio and Middleton 8
Our HA-ILI case definition detected approximately 99% of cases confirmed by chart review and had a PPV of 94%. In contrast, when comparing the ICD-9 HA-ILI definition to data from parental interviews, the PPV was significantly lower (Table 1B). This result may be related to possible misclassification; parents reported ILI symptoms at the time of the initial visit that were not recorded in the EHR for many HA-ILI cases. However, we suspect that these results may reflect recall bias due to the 9–13-day interval between the initial visit and the follow-up telephone interview. Even more notable is the low sensitivity when our case definitions failed to identify an HA-ILI but the parent reported that their child developed ILI symptoms after the initial clinic visit. This finding suggests that an EHR-based case definition may underestimate the true rate of HA-ILI because a significant proportion of ILI episodes may not result in a healthcare encounter.Reference Byington, Ampofo and Stockmann 9 , Reference Stockwell, Reed and Vargas 10
This study has limitations that may impact the use of EHR data for HA-ILI surveillance. First, we only captured patients who returned to or contacted their primary care practice after contracting an HA-ILI. Second, this study took place prior to the conversion to ICD-10 codes. Third, our HA-ILI definition was syndromic and we were unable to confirm new acquisition of a respiratory pathogen between the initial visit and subsequent visit. Nevertheless, our results indicate that using carefully selected diagnostic codes and temperature measurements recorded in an EHR to detect medically attended cases of HA-ILI could provide a reliable, time-sensitive approach to HAI surveillance in pediatric outpatient settings. Developing methods to identify patients who do not return to clinic is necessary to accurately quantify the risk of ambulatory HA-ILI.
ACKNOWLEDGMENTS
The authors thank Mark Ramos for his help with data manipulation. We also thank the network of primary care physicians and their patients and families for their contributions to clinical research through the Pediatric Research Consortium at The Children’s Hospital of Philadelphia.
Financial support: K.A.F. receives grant support from the Agency of Healthcare Research and Quality Mentored Clinical Scientist Research Career Development Award program (grant no. 1K08HS020939).
Potential conflicts of interest: All authors report no conflicts of interest relevant to this article.
SUPPLEMENTARY MATERIAL
To view supplementary material for this article, please visit http://dx.doi.org/10.1017/ice.2016.147.
