In 2009, the National Pediatric Cardiology Quality Improvement Collaborative (NPC-QIC) began with a focus on improving survival for infants with hypoplastic left heart syndrome (HLHS) during the interstage period between the stage 1 and stage 2 procedures. Reference Kugler, Beekman and Rosenthal1 Since that time, NPC-QIC has reported successful reduction of interstage mortality from 9.5 to 5.3% Reference Anderson, Beekman and Kugler2 and expanded its inclusion criteria to encompass a broader group of patients from diagnosis to 12 months of age in its “Phase 2” registry, which began in 2016. Concurrently, several other collaborative research and improvement networks focused on improving care for children with heart disease have been established. Many of these networks require rigorous enrollment practices to conduct high quality research and improvement work. Reference Ferreira-González, Marsal and Mitjavila3–Reference Bufalino, Masoudi and Stranne5 Furthermore, ensuring complete and accurate data has become particularly important given the increasing calls for public transparency in congenital heart surgery. Reference Lihn, Kugler, Peterson, Lannon, Pickles and Beekman6
In contrast to many other collaborative networks, NPC-QIC enrollment is dependent on individual centers identifying eligible infants with almost all centers requiring consent to participate. Although self-reported enrollment audits suggest that > 90% of eligible infants were enrolled in NPC-QIC Phase 1, no method exists to verify the effectiveness of the enrollment process designed to capture all potential patients. Furthermore, there have been anecdotal reports of confusion about enrollment eligibility in Phase 2, and it remains unclear whether centers are consistently applying the same criteria to enroll infants in NPC-QIC Phase 2. As such there is potential for sampling biases within and across centers. In contrast to NPC-QIC, the Pediatric Cardiac Critical Care Consortium (PC4) registry does not require consent for enrollment and includes all patients admitted to a PC4 intensive care unit. Reference Gaies, Cooper and Tabbutt7 Furthermore, PC4 has a well-established, rigorous auditing process to monitor for missing cases and incentives for participating hospitals to submit all eligible cardiac intensive care unit admissions to the registry. Reference Krumholz4,Reference Bufalino, Masoudi and Stranne5 Because most infants with single ventricle disease are admitted to cardiac intensive care units in the perioperative period, PC4 is a potential source for identifying the denominator of infants who are eligible for NPC-QIC enrollment.
The purpose of this study was to determine the percentage of eligible infants with HLHS enrolled in NPC-QIC after admission to a PC4 intensive care unit by linking the NPC-QIC registry with PC4 data for those centers who participate in both registries. We also examined how enrollment varied within patient subgroups and across NPC-QIC hospitals.
Materials and method
The Institutional Review Board at the University of Michigan, which provides oversight for the PC4 data coordinating center, determined that this study was not regulated research.
The NPC-QIC includes a voluntary registry that receives data from 69 pediatric cardiac programs which have joined on a rolling basis since 2008. There is a standard dataset with data definitions, online web-based data entry, and data quality checks. The data are housed in a secure server at Cincinnati Children’s Hospital Medical Center. Institutional review boards at each institution reviewed and approved participation in the registry. Phase 2 of the NPC-QIC registry began in August 2016. Infants followed at participating centers are eligible for enrollment into the Phase 2 registry if they are diagnosed with HLHS or other single ventricle disease requiring single ventricle palliation through one year of age, with the intention to pursue surgical intervention (e.g. Norwood stage 1 procedure or variant). Most individual center Institutional Review Boards require that families provide consent prior to enrollment for each individual patient as NPC-QIC’s work does not clearly meet all the stipulations for a broad waiver of consent, although a few centers have pursued a universal waiver of consent. Across all NPC-QIC centers, most have a waiver of consent for enrollment of patients who die prior to being approached for consent as obtaining consent after death is considered to make conducting the research not practicable.
PC4 is a quality improvement collaborative that collects data on all patients with primary cardiac disease admitted to the cardiac intensive care unit (CICU) service of participating hospitals. Reference Gaies, Cooper and Tabbutt7 PC4 maintains a clinical registry to support research and quality improvement initiatives. Each participating PC4 center has a trained data manager who has completed a certification examination. Local data managers collect and enter data in accordance with the standardized PC4 Data Definitions Manual. The PC4 registry shares common terminology and definitions with applicable data points from the International Pediatric and Congenital Cardiac Code, Society of Thoracic Surgeons (STS) Congenital Heart Surgery Database and American College of Cardiology Improving Pediatric and Adult Congenital Treatment Registry, as previously described. Reference Gaies, Cooper and Tabbutt7 Participating centers are audited on a regular schedule. Audit results suggest complete, accurate, and timely submission of data across centers, with the most recent published results demonstrating 99.4% overall data accuracy rate and a major discrepancy rate of 0.52% across 2219 encounters. Reference Schuette, Zaccagni and Donohue8 All patients who are admitted to a participating intensive care unit are enrolled in the PC4 registry. Reference Gaies, Jeffries, Jacobs and Laussen9
Study population
All centers who participated in both NPC-QIC Phase 2 and PC4 were eligible for inclusion. Within the included centers, all patients enrolled in the NPC-QIC Phase 2 registry with dates of birth between 1/1/2018 and 12/31/2020 who were admitted 30 days of life were eligible. In the PC4 registry all infants admitted 30 days of life with dates of birth between 1/1/2018-12/31/2020 were included if they met any of the following criteria: 1) fundamental diagnosis of HLHS or HLHS variant who underwent a surgical Norwood or hybrid stage 1 variant or no index operation, or 2) hospitalization included surgical Norwood regardless of fundamental diagnosis, or 3) fundamental diagnosis of definite or possible single ventricle (e.g. “single ventricle, double inlet left ventricle” or “Shone’s syndrome”) and hospitalization included surgery identified as hybrid stage 1 or variant (e.g. pulmonary bands only, patent ductus arteriosus stent only, or Damus-Kaye-Stansel procedure). On a center-by-center basis, we excluded any infants admitted prior to the date when the center was enrolling in both NPC-QIC and PC4. NPC-QIC data included indirect identifiers required for matching and early exit disposition for those patients missing information from the stage 1 hospitalization. PC4 data included baseline demographic information, diagnosis and operation type, perioperative information, and postoperative length of stay only for the neonatal admission. The presence of any STS higher risk factor was noted, including need for cardiopulmonary resuscitation, persistent shock at the time of surgery, hepatic dysfunction, neurologic injury (stroke, cerebrovascular accident, or intracranial hemorrhage > grade II) within 48 hours prior to surgery, and renal failure requiring dialysis. We chose to use the PC4 registry as our primary data source due to its rigorous auditing process. Reference Gaies, Donohue and Willis10
Data linkage
Data linkage between NPC-QIC and PC4 was facilitated by Cardiac Networks United, Reference Gaies, Anderson and Kipps11 an organization which supports integration of pediatric cardiovascular data and collaboration across networks to facilitate research and improvement. Data were linked using indirect identifiers as described by Pasquali et al. Reference Pasquali, Jacobs and Shook12 Patients meeting PC4 cohort criteria outlined above were utilized sequentially for matching: first all patients with a fundamental diagnosis of HLHS, next all patients who had a Norwood, and finally all patients with an eligible diagnosis and procedure combination. Once a PC4 record matched to an NPC-QIC record, the PC4 record was removed from the cohort available for sequential matching attempts. Patients were matched using the following indirect identifiers: date of birth, date of admission, gender, and center. The initial rounds required exact matches on (a) all fields or (b) center, gender, and one date field, allowing a ± 1 day difference for the other date. In recognition of the possibility that data entry errors could be negatively impacting matching results, a series of modified matching attempts were performed as well, allowing for mismatched gender or dates off by > 1 day. Finally, eligible NPC-QIC patients who did not have stage 1 hospitalization information (i.e., hospital admission date was not available for matching) were the final group matched to the PC4 cohort, using center, gender, and date of birth. Additional information (e.g., surgery date/type, diagnosis, etc.) was used to confirm cases matched through these modified matching attempts as well as any NPC-QIC patients matched to multiple PC Reference Krumholz4 records.
Outcome
The primary outcome variable was the percentage of eligible PC4 patients who matched to an NPC-QIC patient record, indicating that the patient was enrolled in NPC-QIC.
Statistical analysis
Standard descriptive statistics were used to calculate percentage of eligible PC4 patients who matched to a NPC-QIC patient record. We used Wilcoxon rank-sum test, chi-square or Fisher’s exact test as appropriate to compare characteristics of eligible patients who were matched in NPC-QIC to those who were not matched to determine if there were any significant differences. NPC-QIC center match rates were compared using a funnel chart with 99% confidence intervals to adjust for the number of eligible patients identified in PC4.
We completed a secondary analysis looking only at PC4 patients who had a primary anatomic diagnosis of HLHS and who had an index operation during their neonatal hospitalization. We chose this population as 100% of them are eligible for enrollment in NPC-QIC. We also performed a specific subanalysis of patients who died during the neonatal hospitalization to determine if other patient characteristics were more common in this group. Finally, we divided centers evenly into two groups based on their enrollment rates and compared the mortality rate in the high enrollment group to the mortality rate in the low enrollment group.
Results
Data linkage
We identified 1,114 infants in 32 PC4 centers who met criteria and 898 eligible patients in the NPC-QIC Phase 2 registry. The overall match rate was 75.5% (841/1114 patients). In total, 70.8% (789/1114) of patients matched exactly on all 4 indirect identifiers allowing for ± 1 day. Match rates varied by PC4 fundamental diagnosis and procedure type (Table 1). As expected, match rates were higher for infants with a fundamental diagnosis of HLHS and those who underwent a Norwood procedure. The additional 4.7% (52/1114) of patients were matched using the modified matching approach described above, indicating at least one data error occurred.
Results of matching based on PC4 fundamental diagnosis and procedures.
Hybrid stage 1 or variant includes any combination of ductus arteriosus stent and pulmonary artery bands. HLHS = hypoplastic left heart syndrome.
Center variation in matching rates
Match rates for each individual center are shown in the funnel plot (Fig. 1). The mean center match rate was 75.5%. There were several centers outside the control limits, indicating that their performance was significantly different than the rest of the group, with 3 having higher match rates and 3 having lower match rates.
Upper and lower control limits represent 99% confidence intervals. Mean match rate represents the population match rate.
Characteristics of matched vs. unmatched population
There were no significant differences in gender, race, insurance type, prematurity, birth weight, birth location, prenatal diagnosis, or presence of an STS higher risk factor between those PC4 patients who matched to the NPC-QIC registry and those who did not match (Table 2). Match rates were lower in PC4 patients who were Hispanic/Latino (66.1%, p = 0.005) and those who had any specified chromosomal abnormality (57.4%, p = 0.002), noncardiac congenital anatomic abnormality (67.8%, p = 0.005), or any specified syndrome (66.5%, p = 0.001). Match rates were higher in patients with HLHS as their fundamental diagnosis (77.9%, p < 0.001), those who underwent an index operation during the neonatal hospitalization (77.7%, p < 0.001) and those who underwent a surgical Norwood during the neonatal hospitalization (81.9%, p < 0.001). Patients with fundamental diagnoses of possible single ventricle anatomy or another anatomic variant had lower match rates (39.5% and 63.6%). There was no association with match rates across PC4 critical care therapies and complications, with the exception of mechanical ventilation (76.2%, p = 0.005), sternum left open (80.0%, p = 0.023) and intraventricular hemorrhage grade II or higher (95.0%, p = 0.037). There were differences seen based on CICU disposition from the neonatal hospitalization with higher match rates in those who were discharged home from the CICU or transferred within the same hospital versus those who were transferred to an outside hospital, discharged to hospice, or discharged as deceased. Similarly, match rates were lower in those patients who had withdrawal of life sustaining therapies (63.6%, p < 0.001) and those who died prior to being discharged home from the neonatal admission (62.6%, p < 0.001).
Comparison of patient characteristics between unmatched and matched patients.
CICU = cardiac intensive care unit, HLHS = hypoplastic left heart syndrome, ICU = intensive care unit.
Secondary analyses
When the PC4 cohort was limited to those patients with a fundamental diagnosis of HLHS who underwent an index operation during the neonatal hospitalization, the match rate increased to 81.1% (634/782). Match rates remained lower in those patients of Hispanic/Latino ethnicity (73.2%, p = 0.046), those with any specified syndrome (72.2%, p = 0.011), and patients who died before discharge home (74.3%, p = 0.048). Match rates were also lower in those who had postoperative or postprocedural bleeding requiring re-exploration (71.6%, p = 0.029) and the duration of mechanical ventilation was shorter in the group that matched (8.3 [4.6-18.2] days vs. 10.6 [5.9-33.5] days, p = 0.002). There was a significant difference in match rates based on whether patients were born at the PC4-affiliated center (77.4%), another institution (85.0%), or had missing data on birth location (76.8%, p = 0.016). There were no other significant differences noted between groups.
When we specifically examined the 190 PC4 patients who died during the neonatal hospitalization to determine whether significant risk factors identified in the larger cohort were more prevalent, we found no differences in ethnicity or fundamental diagnosis category. The deceased group had higher rates of any specified chromosomal abnormality (10.5% versus 3.7%, p > 0.001), any specified noncardiac congenital anatomic abnormalities (25.3% versus 16.7%, p > 0.001), and any specified syndrome (24.7% versus 15.9%, p = 0.003). The deceased group had lower rates of undergoing an index operation (77.9% vs. 97.9%, p < 0.001) and undergoing a surgical Norwood during the neonatal hospitalization (47.4% vs. 85.7%, p < 0.001). The deceased group also had a higher rate of intraventricular hemorrhage grade 2 or higher (5.3% vs. 1.1%, p < 0.001) but there was no difference in rates of delayed sternal closure or need for mechanical ventilation.
In terms of variation in center mortality rates and match rates, the 16 centers with relatively high matching rates had an overall mortality rate of 18% (107/597). In the 16 centers with relatively low matching rates, overall mortality was 16% (83/517, p = 0.408).
Discussion
We demonstrated that linking patients between the NPC-QIC and PC4 registries using indirect patient identifiers is feasible. Rates of matching patients were lower than anticipated based on previous self-reported enrollment audits performed through NPC-QIC, which indicated > 90% of eligible patients were enrolled (internal NPC-QIC data). Rates of matching varied across centers, with some centers significantly higher and other centers significantly lower than average. Patients who were Hispanic/Latino, who died, or who had any genetic syndrome had lower rates of matching in both the primary and secondary analyses. These findings are concerning as they indicate the presence of potential biases in the NPC-QIC registry cohort which could impact both research and quality improvement results and generalizability. Reference Ferreira-González, Marsal and Mitjavila3–Reference Bufalino, Masoudi and Stranne5
Although matching patients between the NPC-QIC and PC4 registries using indirect identifiers was feasible, the rates of matching are lower than other previously reported uses of this method. Reference Pasquali, Jacobs and Shook12 Patients in NPC-QIC who did not match to a PC4 record may not have been included in the PC4 cohort if they were admitted to a neonatal intensive care unit and did not undergo surgical intervention or if their fundamental PC4 diagnosis was not included in our cohort definition (e.g. fundamental diagnosis recorded as total anomalous pulmonary venous connection for a child who also had a complete atrioventricular canal with small left sided structures but did not undergo a Norwood or variant). Because we chose to use a relatively broad cohort in PC4, some of the patients who did not match were likely appropriately identified as being ineligible for NPC-QIC enrollment. For example, an infant with an interrupted aortic arch who underwent a hybrid procedure as initial palliation would have been included in the PC4 cohort but would not be eligible for NPC-QIC enrollment. Similarly, an infant with HLHS delivered with a plan for comfort care only would not have been considered eligible for NPC-QIC enrollment. Our secondary analysis addressed this limitation of identifying a true denominator by including only those PC4 patients who met NPC-QIC eligibility criteria, however the match rate was only slightly higher than for the full cohort. In this group, lower match rates could be due to the family declining to participate, although there is currently no mechanism to track this. Additionally, the PC4 registry does not contain sufficient information to determine if an infant was originally intended to undergo Norwood but transitioned to comfort care based on adverse events after CICU admission. Of note, we found that using a more flexible algorithm with as few as 2 indirect identifiers enabled a higher match rate. This suggests that the lower match rate we found may reflect data entry errors, Reference Bufalino, Masoudi and Stranne5 most likely in the NPC-QIC registry given PC4’s previously described rigorous auditing process. Reference Gaies, Jeffries, Jacobs and Laussen9,Reference Gaies, Anderson and Kipps11
We noted important differences between the matched and unmatched populations that offer additional possible explanations for why the overall match rate is lower than expected. It is possible that lower match rates are due to failure to recognize that patients are eligible for inclusion in NPC-QIC, particularly those who die in hospital, or bias in which families are approached to enroll. Because many centers wait until an infant is preparing to be discharged home from the neonatal hospitalization to enroll patients in NPC-QIC, patients who die in hospital or are transferred to another institution may not be enrolled, particularly since approaching a family for consent following their child’s death is difficult. In response to this concern, NPC-QIC obtained an IRB waiver of consent for deceased patients in March 2018, however it is possible that lower rates of enrollment for patients who die are due to lack of reliable enrollment processes for this population despite the waiver. Also concerning is the lower match rate in those infants with a chromosomal anomaly, extracardiac congenital abnormalities, or any type of syndrome. It is possible that a larger proportion of these patients are not offered surgical palliation or have a fundamental diagnosis other than HLHS so they may not have been truly eligible for NPC-QIC enrollment. However, the lower match rate for infants with any syndrome persisted in the secondary analysis of patients with HLHS who underwent an index operation, suggesting other issues may impact enrollment rates. For example, these patients may have higher rates of in hospital death and therefore be subject to enrollment challenges outlined above. Finally, lower match rates in patients identified as being of Hispanic/Latino ethnicity may represent lower rates of consent in this population, language barriers precluding consent and/or provision of interstage home monitoring as strongly recommended by NPC-QIC, or unconscious biases against enrolling these patients in NPC-QIC.
In addition to variation in match rates across patient characteristics, we also found significant variation in match rates across centers. Importantly, we did not find significance differences in mortality rates between the high matching rate group of centers and the low matching rate group. One explanation for the variation across centers might be that some centers have developed best practices around enrolling eligible infants in NPC-QIC. Unfortunately, due to limitations of data use agreements, the study team was unable to identify which centers had significantly high or low matching rates, so we are limited to conjectures. One center had a 0% match rate which may represent a systematic issue with either enrolling patients in NPC-QIC or delayed data entry. Given reported biases in registries which require consent, Reference Tu, Willison and Silver13–Reference Woolf, Rothemich, Johnson and Marsland15 one best practice may be obtaining a waiver of consent for all patients, a strategy which a few centers have pursued locally but which was not granted universally for NPC-QIC’s Phase 2 protocol. Because local waivers of consent could explain some of the variation in match rates, we verified whether the included centers had used this strategy: one reported a waiver of consent for all patients granted by its local IRB and another center reported that its team obtains family consent for all patients, even those who die. Interestingly, neither of these centers had a match rate outside the control limits of the funnel chart. Without a universal waiver of consent, targeted efforts to increase enrollment of specific groups of patients may represent the best strategy. For example, given the waiver of consent for deceased patients and the importance of understanding this population, enrolling deceased patients retroactively could be an important improvement effort for NPC-QIC centers. To improve enrollment for patients identifying as Hispanic/Latino ethnicity, efforts might focus on ensuring that all consent and home monitoring information is available in Spanish as well as English. NPC-QIC recently began a multicenter quality improvement project focused on improving health equity in the provision of prenatal care to families with an HLHS diagnosis, and this work might expand to include efforts to reduce disparities in enrollment rates for this population, both at a local center level as well as across the network. Another possible approach to monitor these issues in the future might be adding a specific question about NPC-QIC enrollment to other CNU registries’ data forms to provide a prospective denominator. Ultimately, our findings imply that there are biases in NPC-QIC enrollment which may be driven or exacerbated by requirements for consent. This potential for biased enrollment represents an important issue for NPC-QIC and other registries to consider.
Limitations
Limitations of this study include those inherent to retrospective analyses of secondary data. Given the large number of statistical comparisons we report, it is likely that some of them are significant due to chance. Our reports of match rates by race and ethnicity are dependent upon data entered into PC4 which may be inaccurate depending on individual center practices. Because most NPC-QIC centers require parent consent for enrollment, it is impossible to distinguish between patients who are not enrolled in NPC-QIC due to lack of consent to participate and failure to enroll or approach for consent.
Conclusion
In conclusion, we found that it was feasible to link patient records between the NPC-QIC and PC4 registries. Match rates were lower than anticipated based on internal NPC-QIC enrollment audits, with lower rates for some specific patient populations and significant variation across centers. Future work includes local center and network-wide improvement efforts to improve enrollment rates for these populations.
Acknowledgements
We would like to thank the many NPC-QIC and PC4 team members across participating sites for their participation and commitment to improvement.
Financial support
NPC-QIC received funding from participation fees from enrolled centers and a grant from the Children’s Heart Association of Cincinnati. PC4 is supported by funding from the University of Michigan Congenital Heart Center, Champs for Mott, and the Michigan Institute for Clinical & Health Research (NIH/NCATS UL1TR002240). Cardiac Networks United is supported by the Children’s Heart Foundation, in addition to receiving support from the University of Michigan Congenital Heart Center and the Heart Institute at Cincinnati Children’s Hospital.
Competing interests
None.
