The Pediatric Heart Network (PHN), funded by the National Heart, Lung, and Blood Institute (NHLBI) since 2001, conducts multi-centre clinical trials and other studies in the paediatric and congenital cardiac population.Reference Mahony, Sleeper and Anderson 1 Similar to trial networks across other fields, the goal is to support an infrastructure to facilitate important clinical investigations.
Since the inception of the PHN, several notable trends have begun to impact the overall landscape in biomedical research. These have included an exponential increase in the variety of healthcare data captured across numerous sources such as clinical registries, the electronic health record, billing data, various monitors and wearable devices, and genetic and biomarker data, among others.Reference Vener, Gaies, Jacobs and Pasquali 2 – Reference Merelli, Perez-Sanchez, Gesing and D’Agostino 7 At the same time, there has been a concurrent trend characterised by a relative decline federal funding to support medical research and other financial pressures faced by hospitals and health systems.Reference Alberts, Kirschner, Tilghman and Varmus 8 These two trends have prompted greater interest in understanding whether the increasing number and variety of data sets hold the potential to support novel investigation that would otherwise not be possible, particularly through the linkage of disparate sources of information, and allow us to conduct research more efficiently.Reference Vener, Gaies, Jacobs and Pasquali 2 – Reference Pasquali, Jacobs and Farber 6 Despite this, mechanisms to take advantage of these potential benefits within the PHN and other research networks have remained limited.
In this context, the PHN developed the Integrated CARdiac Data and Outcomes (iCARD) Collaborative in 2013 with the goals of leveraging available data sources to aid in efficiently planning, implementing, and conducting PHN studies; supporting integration of PHN data with other sources to foster novel science; and mentoring young investigators in these areas. This article describes lessons learned through the development of iCARD, along with initial efforts and scientific output. This information can aid in further use and dissemination of data integration and optimisation methodologies across other research networks and organisations.
iCARD structure
The PHN consists of nine core clinical sites and a central data coordinating centre.Reference Mahony, Sleeper and Anderson 1 Auxiliary sites also participate by invitation in many PHN studies. iCARD was initiated as a pilot programme beginning in 2013 and became a formal PHN committee in 2015. The committee is comprised of representatives from each of the core sites, interested members from certain ancillary sites, and representatives from NHLBI. Expertise of the committee members is diverse and inclusive of multiple areas related to iCARD’s mission such as data integration and other aspects of data science, clinical trials, outcomes research, and health economics. Depending on the needs of certain projects, iCARD also collaborates broadly with experts across other domains. The committee meets in person twice a year at PHN steering committee meetings and holds regular conference calls. Funding is provided through the PHN.
Several additional related initiatives in the field have been key in supporting the formation and ongoing work of iCARD. These have included the increasing experience with use and linkage of large data sets for a variety of research projects, the work of the Multi-societal Database Committee for Pediatric and Congenital Heart Disease in fostering collaboration across different registries and research groups, the development of uniform terminology and coding now used across many data sets, and a recent working group hosted by NHLBI focused on furthering congenital heart disease data integration in the field, among others.Reference Vener, Gaies, Jacobs and Pasquali 2 – Reference Pasquali, Jacobs and Farber 6 , Reference Franklin, Jacobs and Krogmann 9 , Reference Jacobs 10
Scientific output and focus
Since its inception, iCARD has supported 15 completed or ongoing projects. The projects have focused on two major areas: (1) promoting research efficiency and optimizing PHN study design, and (2) supporting research not possible with PHN study datasets alone.Reference Gaies, Pasquali and Nicolson 11 – Reference McHugh, Pasquali, Hall and Scheurer 16 Two major areas focused in the projects are promoting research efficiency and optimising PHN study design, and supporting research not possible with PHN study data sets alone. These investigations, which are detailed further in the sections below, have spanned the areas of health economics, outcomes research, quality improvement, and implementation science, augmenting the traditional clinical trials focus of the PHN.
In addition, iCARD has also completed an inventory of registries, electronic health record systems, and administrative data sets utilised across PHN sites. This information is posted for use on the PHN website and allows PHN investigators and leadership to better understand the different types and sources of data available when considering planning or conducting a study.
Mentorship
A total of 10 out of the 15 projects to date have been conducted by junior investigators with senior PHN faculty mentorship. In addition, two projects have been selected for PHN Scholar Funding – this programme within the PHN awards grant funding to junior scientists through a competitive application process to support a mentored project and career development.Reference Minich, Pemberton and Shekerdemian 17 iCARD committee leadership and members have also participated in the annual PHN Career Day, which provides academic career development guidance and an opportunity for networking with PHN scientists to young investigators across the PHN. These projects and initiatives have allowed for mentorship and career development in areas such as data science, health economics, outcomes and quality research, and other areas that traditionally have received less attention in paediatric cardiovascular research training.
Initial projects and lessons learned
Projects to date highlight several lessons learned regarding the potential benefits of the data integration model and collaboration supported by iCARD to optimise research efficiency and expand scientific scope and impact. Several example projects are discussed in the following sections.
Promoting research efficiency and optimising study design
Planning trials
It has previously been shown that certain trials conducted in the paediatric cardiovascular population have met a variety of challenges related to aspects of planning and design. For example, in the PHN Angiotensin Converting Enzyme Inhibition in Mitral Regurgitation (ACE in MR) Study, investigators sought to evaluate the efficacy of enalapril versus placebo in reducing left ventricular volume overload in children with moderate or greater mitral valve regurgitation following surgical repair of atrioventricular septal defect.Reference Li, Colan and Sleeper 18 After 17 months, 349 patients had been screened for the trial, only 8 were eligible, and 5 were enrolled. The trial was subsequently terminated due to low enrolment. Several factors were felt to be responsible for the low accrual, including limited accurate information regarding available sample size in the paediatric population given the specific inclusion and exclusion definitions that were to be used in the trial, and limited information regarding baseline treatment rates.Reference Li, Colan and Sleeper 18
With the formation of iCARD, it was recognised that there are now multiple existing data sources in the field available to aid in better informing study design to mitigate some of these previous challenges identified in the ACE in MR Study and others.Reference Vener, Gaies, Jacobs and Pasquali 2 – Reference Pasquali, Jacobs and Farber 6 For example, in the design of a recent PHN trial, iCARD collaborated with the Pediatric Cardiac Critical Care Consortium (PC4), a large multi-centre research and quality improvement collaborative collecting data on patients in the paediatric cardiac intensive care unit.Reference Gaies, Cooper and Tabbutt 19 All PHN core sites participate in PC4. Through this collaboration, we were able to estimate available sample sizes in the specific population of interest, and simulate the impact of various inclusions and exclusions using precise clinical criteria which matched those of the trial (Table 1). This exercise demonstrated that nearly two thirds of eligible patients would be excluded based on the proposed criteria. It allowed the study team to refine their accrual timeline and sample size estimates, and also led to modification of certain criteria felt to be too stringent and not reflective of the original scientific intent.
Using registry data to simulate PHN trial inclusion/exclusion criteria and sample size.
AV = atrioventricular; CPR = cardiopulmonary resuscitation; MCS = mechanical circulatory support; op = operation; PC4 = Pediatric Cardiac Critical Care Consortium; PHN = Pediatric Heart Network.
The table demonstrates the use of the PC4 registry for simulation of the impact of various inclusion and exclusion criteria on proposed PHN trial sample size, using detailed clinical variables matching those to be used in the trial. Application of these criteria resulted in almost two thirds of the initial patient population being excluded (see bolded values). This allowed the study team to refine their accrual timeline and sample size estimates, let to modification of certain inclusion criteria felt to be too stringent and not reflective of the original scientific intent.
Platform to conduct trials
The use of existing data sources as platforms to not only plan but conduct a clinical trial has also been proposed.Reference Lauer and D’Agostino 20 This method could capitalise on existing data, available patient population, infrastructure, and collaboration between sites to streamline the start-up and conduct of a study, optimising timelines and minimising resource needs. These methods have been used in a few select adult cardiovascular trials and other areas of medicine with reported success in facilitating efficient enrolment, leveraging existing data to avoid duplicate collection, and lowering trial costs. For example, the Thrombus Aspiration during ST-segment Elevation Myocardial Infarction trial used a Swedish catheterisation registry as the trial platform and reported enrolling over 7000 patients in ~2 years with a total budget of only $300,000.Reference Frobert, Lagerqvist and Olivecrona 21 However, these methods have yet to be widely adopted and several questions about implementation strategies and utility remain.
In order to extend this “trial within a registry” method to the paediatric cardiovascular population and study its impact in greater detail, iCARD collaborated with investigators from the PHN Residual Lesion Score Study.Reference Nathan, Jacobs and Gaynor 15 The team designed the study to utilise site’s local surgical registry data collected for submission to the Society of Thoracic Surgeons Congenital Heart Surgery Database to support a portion of the Residual Lesion Score study data collection, including patient demographics, pre-operative variables, diagnosis and procedural data, and in-hospital outcomes. Additional Residual Lesion Score study variables such as echocardiographic data and longitudinal outcomes, which are not captured by the Society of Thoracic Surgeons Database, were also collected and integrated with the existing registry data for analysis. This ongoing study will represent the first successful use of existing registry data to support a prospective study in this population. In addition to the study itself, several additional analyses were undertaken to better understand the impact of the methodology. First, to address questions regarding the quality of registry data, a formal audit was conducted prior to Residual Lesion Score study initiation. It demonstrated that across more than 56,000 data elements, the registry data were 98% accurate and 97% complete, and supported the use of the registry data for the study.Reference Nathan, Jacobs and Gaynor 15 In addition, an analysis of potential savings with regard to study costs and resources is underway. Finally, an evaluation of stakeholder perspectives and lessons learned with regard to implementation of this new method was recently completed. Subsequently an additional group of investigators has undertaken a second trial that uses existing data from Society of Thoracic Surgeons Congenital Heart Surgery Database – the ongoing Steroids to Reduce Systemic Inflammation after Neonatal Heart Surgery Trial. 22
Data integration to support novel science
In addition to optimising research efficiency, integration of other data sources with PHN clinical trial and research data sets can support novel investigation otherwise not possible and expand our knowledge beyond that generated from the original study.Reference Gaies, Anderson and Kipps 3 – Reference Pasquali, Jacobs and Farber 6 Several examples are described in the following sections.
Integrated clinical/cost analyses
In the current healthcare environment, there is emphasis both on improving clinical outcomes and reducing costs of care, or optimising healthcare value.Reference Porter 23 However, cost data are not typically captured in clinical trials, including those conducted by the PHN, which makes these types of integrated analyses challenging. In order to overcome this, iCARD supported the linkage of clinical data from two PHN studies with resource utilisation data from the Children’s Hospital Association.
Linkages between the PHN and Children’s Hospital Association data were performed using the method of probabilistic matching of indirect identifiers.Reference Pasquali, Jacobs and Shook 24 This method was developed in order to overcome the limitation that many data sets in the United States do not contain direct identifiers, and has been previously described and validated in the paediatric cardiovascular population, as well as others. Using these methods, three integrated cost-outcome studies have been performed to date. First, data from the PHN Single Ventricle Reconstruction Trial were linked to the Children’s Hospital Association’s data set. This PHN trial enrolled 555 infants undergoing the Norwood operation for hypoplastic left heart syndrome or related anomalies across 15 North American sites, and evaluated outcomes associated with the type of shunt used to support pulmonary blood flow.Reference Ohye, Sleeper and Mahony 25 A total of 10 out of the 15 trial sites submitted data to Children’s Hospital Association during the study period, and from these sites, records were successfully linked using the method of indirect identifiers on 344/353 (97.5%) of patients.Reference McHugh, Pasquali, Hall and Scheurer 16 With the merged data set, two analyses were performed. The fist evaluated important clinical drivers of high costs at the patient level including an assessment of specific complications and length of stay.Reference McHugh, Pasquali, Hall and Scheurer 16 The second analysis delineated the magnitude of variation in cost across hospitals participating in the trial, and underlying factors.Reference McHugh, Pasquali, Hall and Scheurer 14
Another analysis expanded these efforts beyond observational investigation to an analysis of an intervention. The PHN Collaborative Learning Study involved the development and implementation of a clinical practice guideline related to early extubation after infant cardiac surgery (tetralogy of Fallot and coarctation repair).Reference Mahle, Nicolson and Hollenbeck-Pringle 26 Outcomes were evaluated in a group of active sites versus control sites who continued usual practice. The study found a significant increase in early (within 6 hours of surgery) extubation at active sites (12 versus 67%) and a reduction in critical care length of stay in the tetralogy of Fallot cohort, with no significant differences in the control sites.Reference Mahle, Nicolson and Hollenbeck-Pringle 26 However, the impact of the intervention on hospital costs was not studied in the main analysis. Using the same methods described in the preceding sections, the investigators successfully linked study data to the Children’s Hospital Association data set on 96% of eligible patients.Reference McHugh, Mahle and Hall 13 They found a 27% reduction in costs in active sites versus controls for patients undergoing tetralogy of Fallot repair, while there were no significant cost differences in the coarctation cohort.Reference McHugh, Mahle and Hall 13 The investigators were also able to analyse the specific categories of cost that were reduced in the tetralogy of Fallot cohort. Taken together, these data suggested that for certain types of infant heart surgery, an early extubation clinical practice guideline and collaborative learning strategy can not only improve clinical outcomes but also reduce costs of care.
Understanding “real-world” impact beyond the study period and study population
While carefully designed clinical studies are able to give us insights into the effectiveness of a treatment or therapeutic strategy within the setting of the research study, it is often challenging to extrapolate these findings to patients who did not meet inclusion criteria for the study or to follow patient outcomes and adverse events beyond the period of the study. Further, in the setting of quality improvement efforts, it is often unclear whether any effects associated with the implementation of best practices seen early on in the improvement cycle are sustained over time, or wane as attentions turn towards other initiatives. These represent additional areas where integration of the study data with other data sets can broaden our understanding.
A recent analysis that involved linkage of the PHN Collaborative Learning Study data set to the PC4 Registry highlights the value and importance of this approach. As described in the preceding section, the PHN Collaborative Learning Study demonstrated that a collaborative learning approach and the development and implementation of a clinical practice guideline were effective in increasing early extubation after infant cardiac surgery in active versus control sites.Reference Mahle, Nicolson and Hollenbeck-Pringle 26 However, it was unclear if these results were sustained over time after the study period ended. In order to better understand this, investigators linked the study data with the PC4 Registry, and were able to apply the same inclusion and exclusion criteria used in the study to the registry.Reference Gaies, Pasquali and Nicolson 11 This allowed them to compare early extubation rates across eras at centres who participated in the study. They found that while early extubation rates improved significantly during the study, there was a decline in the follow-up period (from 67 to 30%, p < 0.0001), with rates reverting back to baseline levels at all but one participating centre (Fig 1).Reference Gaies, Pasquali and Nicolson 11 In an additional analysis, investigators integrated qualitative data from a semi-structured survey with the main study data set to better understand variability in how sites approached implementation of the clinical practice guideline and their outcomes.Reference Bates, Mahle and Bush 12 They found the site with sustained outcomes employed several unique strategies regarding implementation of the intervention during the study period, including involvement of additional key local staff, regular in-person data reviews to review progress and overcome any challenges in a timely manner, additional data collection when needed to address staff concerns, and creation of supplemental care protocols to support local implementation of the core clinical practice guideline. These data are important in informing better design of future initiatives to promote sustainability.Reference Bates, Mahle and Bush 12 Finally, a third study which also integrated data from the main study with the PC4 Registry is in progress and studying spillover to other patient populations aside from the population included in the initial study. This series of investigations highlights the value of integration of PHN study data with these other data sources in offering an efficient and more comprehensive understanding of the impact of the intervention studied beyond the knowledge gained from the main study.
Linking PHN study data with registry data to understand intervention sustainability beyond the initial study period. The PC4 Registry was linked with data from a PHN study to understand the sustainability of an intervention after the study period had ended. The graph shows data for each site across different eras. The main study showed that early extubation rates after infant heart surgery increased during the study period following the development and implementation of a CPG across participating sites (sites A–D in the figure – see pre-CPG to post-CPG periods). Linkage to the registry allowed evaluation of early extubation rates after the study had ended (follow-up period) in the same cohort of patients, and showed that the gains during the study period were only sustained at one site (A), while other sites reverted back to rates not significantly different from the baseline pre-CPG period (sites B–D). CPG = clinical practice guideline; PC4 = Pediatric Cardiac Critical Care Consortium; NS = not significant; PHN = Pediatric Heart Network.
Challenges
The challenges facing iCARD and similar activities have been detailed previously and are primarily related to contracting/regulatory issues, funding challenges, and overall the lack of a global congenital heart disease data network.Reference Pasquali, Jacobs and Farber 6 Many groups are interested in more broad data integration in the field beyond individual data linkage efforts related to certain PHN and other projects, and formation of congenital heart disease data network that could support multiple lines of investigation and longitudinal assessments across the lifespan.Reference Vener, Gaies, Jacobs and Pasquali 2 – Reference Pasquali, Jacobs and Farber 6 However, there are limited funding mechanisms to support such an effort. Federal funds, with a few exceptions, are primarily directed towards supporting hypothesis-driven projects rather than the infrastructure and expertise necessary to form such a data network that could in turn facilitate multiple projects. Recent efforts supported by institutional and philanthropic funds to promote a more comprehensive approach are described in the following section.
iCARD also faces challenges related to contracting and regulatory issues. Due to the many different data sources which are all governed by different organisations, contracting and the process of approvals for data sharing and linkages can be time-consuming and redundant, and must be repeated for each study given no centralised infrastructure or governance. This has proved to be the most time-consuming portion of the study start-up process for iCARD and can take several months. The experience gained by the iCARD team and PHN with these issues over time have helped to streamline the process.
Future directions
Through the work of iCARD and other groups, it has been demonstrated that integrating information and expertise across various silos can enhance research efficiency and support novel science.Reference Gaies, Anderson and Kipps 3 – Reference Pasquali, Jacobs and Farber 6 However, until recently there remained lack of a comprehensive strategy in the field to optimise these efforts beyond the work of individual groups or research networks, and to collaborate to address the remaining challenges described in the preceding section and by others. While other efforts such as those supported by the National Institutes of Health Collaboratory, the Patient-Centered Outcomes Research Institute, and several precision medicine initiatives have aimed to support better data integration and collaboration across various fields, these efforts remain primarily focused on adult disease and have not extended to the needs and challenges impacting the paediatric and congenital cardiovascular population. 27 – 29
Over the course of 2017, a series of cross-disciplinary stakeholder meetings were held to discuss these issues, involving the NHLBI, registry/research network leads, professional societies and foundations, data science experts, congenital heart centre executive leadership from around the country, and patients and families. As a result of these sessions, Cardiac Networks United was formed with a vision to foster greater collaboration and data integration to accelerate discovery and improvements in care in the field.Reference Gaies, Anderson and Kipps 3 The organisation is currently comprised of five large networks in the field that span multiple phases of care and capture data from more than two thirds of the congenital heart centres in the United States. Cardiac Networks United aims to expand to additional partners and is also integrating with other emerging sources of data beyond traditional clinical information, including genetic/biomarker data, maternal-fetal data, real-time physiologic data captured from monitors and devices, and longitudinal patient-reported outcomes data.Reference Gaies, Anderson and Kipps 3 , Reference Pasquali, Ravishankar and Romano 30 Initial efforts supported by Cardiac Networks United include fostering standardised and shared common data elements across networks to reduce redundancies and costs; standardising regulatory and contracting processes to support data sharing and streamline start-up; conducting collaborative research across data sets to demonstrate the value of integrating data across the lifespan; sharing improvement resources and expertise to promote translation of discovery to improvements in patient care; and developing tools for scalability to support expansion to other networks and emerging data sources.
Through these efforts, early progress has been made in several areas, including reducing redundancies and costs (for example, eliminating redundancies in data capture for new registries and reducing associated information technology and personnel costs up to 50% in one recent case), fostering collaboration and shared expertise through joint scientific meetings and shared research and improvement expertise, and early scientific projects which span multiple phases of care across traditional silos and integrate both scientific and improvement efforts.Reference Gaies, Anderson and Kipps 3 These projects include efforts geared towards reducing cardiac arrest and other complications, reducing post-operative chest tube duration and length of stay, preventing ventricular assist device complications, improving single ventricle outcomes (with newest efforts focused on Fontan and beyond), optimising neurodevelopment, and understanding peri-operative clinical and physiologic variables impacting longitudinal survival and quality of life.Reference Gaies, Anderson and Kipps 3 Cardiac Networks United provides a comprehensive platform with which iCARD and other initiatives can engage to further augment our capabilities for efficient, innovative, and integrated research across the lifespan in the paediatric and congenital cardiac population.
Author ORCIDs
Sara K. Pasquali1 0000-0002-3114-2680
Conflicts of Interest
None.
Financial Support
The PHN studies described in this paper are supported by grants (5UG1HL135685, 5UG1HL135682, 5UG1Hl135683, 5UG1HL135665, 5UG1HL135689, 5U24HL135691, 5UG1HL135678, 5UG1HL135680, 5UG1HL135666, and 5UG1HL135646) from the NHLBI. Dr Pasquali receives funding (U10HL068270) related to iCARD projects from the NHLBI, and support from the Janette Ferrantino Professorship. Dr Anderson receives salary support from the NHLBI (K23 HL133454). The contents of this work are solely the responsibility of the authors and do not necessarily represent the official views of the NHLBI or the National Institutes of Health.
